Live data from Hacker News

Show HN: Python package to collect normalized news from almost any website

github.com

91–100 of 203 posts

Re: Show HN: Python package to collect normalized news from almost any website

#91

Earlier quoted context omitted.

I think you mean feedparser. It knows how to parse the feed. So you have to give it the feed’s url

No, I mean feedreader. A Feedreader is a service which collects and parses rss-files, then makes them accessable in an interface for the user. Similar to a mailclient, but for rss. This package seems to do the first part, collecting and parsing, but lacks the interface. So what is the point of it? And if this is maintaining it's own config for sources, can I even add my own sources? Or is this just an elaborated OPML…

Hey. Alright. Thanks for letting us know. I will check it tonight.

Re: Show HN: Python package to collect normalized news from almost any website

#92
post #10

Earlier quoted context omitted.

Hey. Newscatcher (python) is an open-sourced package that we developed for users' side projects. There are no limits. You can even modify it for your own needs. Newscatcher API is a product that allows you not only get the latest articles but also search by keywords, topic, country etc. Basically, the main feature is that you search for articles that contain a specific word or phrase. This can have an added value for…

Do you guys have any plans to create webhooks that let you know when a feed/search is updated with a new article in real time?

Hey. Not yet. Let us think about that.

Re: Show HN: Python package to collect normalized news from almost any website

#94

For any news junkies heres, I've built https://maagnit.com which gathers both Left and Right leaning sources for any story and displays them altogether on one page. My approach has been, if we can't get neutral/objective coverage, getting comprehensive, 360 desgree coverage is a good alternative. These days, news bias is not only in the way a story is covered, but also which stories are covered. So, maagnit automatic…

When I was a kid CNN used to do this: they would invite a single climate scientist and a single climate change skeptic and have them make their points on equal time. CNN would then say “who’s right? You decide.” And the program would end. This was extremely harmful to the overall mission of informing people and created a false balance between sides which are not equally valid. Today, this is almost universally seen a…

The current status quo on the internet is infinitely worse than that though.

The internet and social media allow you to ignore the correct conclusion because it makes a person uncomfortable with the idea that they could be wrong when everything they read reinforces that incorrect conclusion.

Re: Show HN: Python package to collect normalized news from almost any website

#95

This doesn't "collect" anything, it's just a python package wrapped on top of a sqlite database. If you choose an arbitrary website that doesn't have information in said database you just get "website not supported". There's not even any logic to guess a website's RSS feed URL... And in the end it's just an RSS feed reader?

What would the logic to guess the RSS feed URL look like? I suppose it is easy for wordpress sites, might not be so simple for others?

Re: Show HN: Python package to collect normalized news from almost any website

#97
post #67

Earlier quoted context omitted.

Seems like a terrible idea to me. The two sides are at war, and if the way you sample them is by going 50-50, that encourages them to be more extreme because that will skew the average in their direction. We already see this in the media. You have the host and two people. Lets hear what person A says, lets hear what person B says. Who's right? Impossible to tell, otherwise you must be biased. And no mention that pers…

To be honest I don't find "verifiable fact" to be worth much these days. If coronavirus has done anything it has highlighted how easy it is to do wrong. Journalists and politicians alike getting drawing incorrect insights from data. Meanwhile so little of the data surrounding coronavirus is comparable. You have countries only recording cases if the patient was hospitalised (not if they tested positive) and cause of d…

The point of the early models, which were predicated on no behavior change, was to change behavior. And they certainly did. Well before the official stay-at-home orders, people here in SF were starting to act differently. And when those orders came, people took them seriously. So I don't think we can blame early models for doing exactly what they were supposed to do, which was telling us what could happen if we didn't take the disease seriously.

Re: Show HN: Python package to collect normalized news from almost any website

#98

For any news junkies heres, I've built https://maagnit.com which gathers both Left and Right leaning sources for any story and displays them altogether on one page. My approach has been, if we can't get neutral/objective coverage, getting comprehensive, 360 desgree coverage is a good alternative. These days, news bias is not only in the way a story is covered, but also which stories are covered. So, maagnit automatic…

When I was a kid CNN used to do this: they would invite a single climate scientist and a single climate change skeptic and have them make their points on equal time. CNN would then say “who’s right? You decide.” And the program would end. This was extremely harmful to the overall mission of informing people and created a false balance between sides which are not equally valid. Today, this is almost universally seen a…

So they present the facts and let them speak for themselves? Sounds like they had it right. Are you saying if they don't present their opinion its a failure of journalism? I think this new way of defining journalism is a bad standard.

Re: Show HN: Python package to collect normalized news from almost any website

#99

For any news junkies heres, I've built https://maagnit.com which gathers both Left and Right leaning sources for any story and displays them altogether on one page. My approach has been, if we can't get neutral/objective coverage, getting comprehensive, 360 desgree coverage is a good alternative. These days, news bias is not only in the way a story is covered, but also which stories are covered. So, maagnit automatic…

When I was a kid CNN used to do this: they would invite a single climate scientist and a single climate change skeptic and have them make their points on equal time. CNN would then say “who’s right? You decide.” And the program would end. This was extremely harmful to the overall mission of informing people and created a false balance between sides which are not equally valid. Today, this is almost universally seen a…

> This was extremely harmful to the overall mission of informing people and created a false balance between sides which are not equally valid. Today, this is almost universally seen as a failure of journalism.

Is there data that show CNN debates on climate change correlate with a decrease in belief in climate change? This sounds like folk lore rather than reality to me, but I could be wrong.

Re: Show HN: Python package to collect normalized news from almost any website

#100
post #21

Depending on what you're trying to do, there's also newspaper3k: https://github.com/codelucas/newspaper It's quite easy to get "good" extraction for large numbers of outlets/articles without a massive amount of special-casing, as news articles are nearly universally marked up with RDF metadata (partly for Google News's benefit). Article discovery, and perfect parsing, is quite a bit harder. I ended up rolling a new S…

Is your code on github? I'm actually working on something very similar, and would love to get some ideas from what you've done.
Post reply on HN