Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Python package to collect normalized news from almost any website

github.com

Post reply on HN
Print view View original

61–70 of 203 posts

Re: Show HN: Python package to collect normalized news from almost any website

#61

Post by jeegsy »

doublesCs wrote: post #18
oceanbreeze83 wrote: post #5

For any news junkies heres, I've built https://maagnit.com which gathers both Left and Right leaning sources for any story and displays them altogether on one page. My approach has been, if we can't get neutral/objective coverage, getting comprehensive, 360 desgree coverage is a good alternative. These days, news bias is not only in the way a story is covered, but also which stories are covered. So, maagnit automatic…

Seems like a terrible idea to me. The two sides are at war, and if the way you sample them is by going 50-50, that encourages them to be more extreme because that will skew the average in their direction. We already see this in the media. You have the host and two people. Lets hear what person A says, lets hear what person B says. Who's right? Impossible to tell, otherwise you must be biased. And no mention that pers…

This reminds me, how does a fallacy get discovered or made official? I mean, I followed the link above and saw it has a latin name and everything but yet feels very new and contemporary. Not that I disagree except that to avoid this fallacy, the 'verifiable facts' have to be known by all in advance

Re: Show HN: Python package to collect normalized news from almost any website

#62

Post by slightwinder »

I would be more interessted in a ready-to-use server which I can selfhost and to which I can throw any script for collecting data, filter and process them, which also handles errors and storage.

At the moment the only real solution for this seems to be using your own RSS-Reader (Tiny Tiny RSS for me at the moment), which has the disadvantage of being limited to RSS-sources, as also not allowing much filtering and processing. But with more and more sources moving avway from RSS, I want something which can be fetch alternative sources and integrate them into a unified interface.

In best case it would be even work with any language, be it a shellscript or python, ruby or even java.

Re: Show HN: Python package to collect normalized news from almost any website

#63

Post by kingofpandora »

> Programmatically collect normalized news from (almost) any website.

By this you mean ... Fetch RSS feeds from the websites that we've hardcoded into a Python library?

It seems like the work here was in collecting and cataloging a lot of RSS feeds. You don't seem to be able to arbitrarily "collect news from any website".

Re: Show HN: Python package to collect normalized news from almost any website

#64

Post by artembugara »

slightwinder wrote: post #62

I would be more interessted in a ready-to-use server which I can selfhost and to which I can throw any script for collecting data, filter and process them, which also handles errors and storage. At the moment the only real solution for this seems to be using your own RSS-Reader (Tiny Tiny RSS for me at the moment), which has the disadvantage of being limited to RSS-sources, as also not allowing much filtering and pro…

Well, we hat is quite similar to what we are doing at newscatcherapi.com.

We collect tons of news data and let you query it with an API

Re: Show HN: Python package to collect normalized news from almost any website

#66

Post by marban »

rawoke083600 wrote: post #24
marban wrote: post #8

Self-Plug: https://www.hvper.com (Official Successor of popurls which more or less started the single page aggregator craze.)

Looks good ! How do I get the BIG pink bar away ? I would rather see ads (to support the free version) than that BIG PINK BAR ?

You can either cover my monthly 5 digit server bill or upgrade to a paid version. Sorry to be the bearer of bad news ;)

Re: Show HN: Python package to collect normalized news from almost any website

#67

Post by yomly »

doublesCs wrote: post #18
oceanbreeze83 wrote: post #5

For any news junkies heres, I've built https://maagnit.com which gathers both Left and Right leaning sources for any story and displays them altogether on one page. My approach has been, if we can't get neutral/objective coverage, getting comprehensive, 360 desgree coverage is a good alternative. These days, news bias is not only in the way a story is covered, but also which stories are covered. So, maagnit automatic…

Seems like a terrible idea to me. The two sides are at war, and if the way you sample them is by going 50-50, that encourages them to be more extreme because that will skew the average in their direction. We already see this in the media. You have the host and two people. Lets hear what person A says, lets hear what person B says. Who's right? Impossible to tell, otherwise you must be biased. And no mention that pers…

To be honest I don't find "verifiable fact" to be worth much these days. If coronavirus has done anything it has highlighted how easy it is to do wrong.

Journalists and politicians alike getting drawing incorrect insights from data.

Meanwhile so little of the data surrounding coronavirus is comparable. You have countries only recording cases if the patient was hospitalised (not if they tested positive) and cause of death reporting is a minefield.

So "verifiable fact" is just as easily weaponised / politicised as bullshit. At least bullshit is easier to debunk...

We had this almighty Imperial Model projecting 500k dead meanwhile Sweden just did their own thing.

I don't really have a leaning to the policy setting but it feels that once you wrap some information up with numbers, maths and scientific authority you can parade subjectivity as fact anyway too.

I guess I'm a little jaded after seeing politics and business use spurious pedestalise data which really amounts to numerology when you start peeling back the layers and thinking critically.

Re: Show HN: Python package to collect normalized news from almost any website

#68

Post by slightwinder »

artembugara wrote: post #64
slightwinder wrote: post #62

I would be more interessted in a ready-to-use server which I can selfhost and to which I can throw any script for collecting data, filter and process them, which also handles errors and storage. At the moment the only real solution for this seems to be using your own RSS-Reader (Tiny Tiny RSS for me at the moment), which has the disadvantage of being limited to RSS-sources, as also not allowing much filtering and pro…

Well, we hat is quite similar to what we are doing at newscatcherapi.com. We collect tons of news data and let you query it with an API

Not selfhosted, isn't it? By which I mean it costs money.

Re: Show HN: Python package to collect normalized news from almost any website

#69

Post by marban »

zwaps wrote: post #33
marban wrote: post #8

Self-Plug: https://www.hvper.com (Official Successor of popurls which more or less started the single page aggregator craze.)

That pink bar takes up almost half of my screen on my iphone 11 and makes the website defacto unusable. Not to be unkind, but you should fix that. Instead of being incentivized to pay you, my only instinct was to close that page asap.

That's by design — It's not a charity service.

Re: Show HN: Python package to collect normalized news from almost any website

#70

Post by freddie_mercury »

mahalol wrote: post #36
freddie_mercury wrote: post #27

Earlier quoted context omitted.

So something like the super-dysfunctional political systems of Israel or Belgium or Italy or South Africa? Fixing first past the post voting is not some magic cure-all.

We use the MMP system in NZ and it seems to work quite well, though needs a bit of fine tuning as currently you need 5% to get into a coalition and that is slightly too high.

Since New Zealand passed MMP 30 years ago, every Prime Minister has come from the same two parties that have dominated NZ politics since 1935.

Two party systems also are built on coalitions. It isn't as if "Democrat" or "Labor" is a unified ideology.

I'm all for MMP but I don't think it will fix any problems in American politics.

Just introducing MMP would add another veto point in an American political system that already struggles to actually accomplish anything.

Post reply on HN