Then use UNIX filters to transform it into the format one prefers.
For example, NYT has a jsonp feed I have posted before.
# replace "world" with whatever NYT section you prefer
curl -o 1.jsonp https://static01.nyt.com/services/json/sectionfronts/world/index.jsonp
exec sed '/\"guid\" :/!d;s/\",//;s/.*\"//' 1.jsonp
Not every news site is the same of course. Using a list from journalism.org I took a small sampling of other news sites to see if they had, at a minimum, rss feeds. Most of them did but I stopped after about seven.As an experiment to enjoy text-only versions of popular news websites, convert the rss to html for a "text-only" version.
The author of radare2 wrote a filter called "rss2html" some years ago and it works well enough for this experiment, so there is no work involved here.[1] rss2html will transform a page of rss xml to html, txt, etc. Build it with libcurl and it can fetch pages too.
curl -o 1.xml http://www.bbc.co.uk/news/rss.xml
rss2html 1.xml > 1.htm
browser file://1.htm
1. http://web.archive.org/web/20140911194544/http://www.nopcode... (use cvs.nopcode.org mirror) http://www.bbc.co.uk/news/rss.xml
http://feeds.reuters.com/Reuters/worldNews
http://www.latimes.com/world/rss2.0.xml
http://www.wsj.com/xml/rss/3_7085.xml
http://rssfeeds.usatoday.com/UsatodaycomWorld-TopStories
http://feeds.washingtonpost.com/rss/world
I routinely write custom filters. Even if a news site has no rss feed it is not difficult to produce csv or whatever format I need. However it is tedious. A one-time cost.