Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Winds 2.0: It’s Time to Revive RSS

getstream.io

Post reply on HN
Print view View original

131–140 of 303 posts

Re: Winds 2.0: It’s Time to Revive RSS

#131

Post by Izkata »

abecedarius wrote: post #72
bArray wrote: post #61

Earlier quoted context omitted.

>This dreary state of affairs is due in part to the fact that "checking" an RSS just means downloading the entire file containing all posts the author would like to make public, regardless of how many are new to the reader. This insanely unnecessary bandwidth usage penalizes sites that have long (>10) feeds with complete posts. (The single RSS file on my blog takes up the majority of my bandwidth costs.) Still, excep…

That’d impose a cost on the server, remembering an ID map. Instead make the client send back a value given by the server last time.

I think that's what they meant by "an ID [...] of the last request"

Re: Winds 2.0: It’s Time to Revive RSS

#132

Post by jerf »

abecedarius wrote: post #72
bArray wrote: post #61

Earlier quoted context omitted.

>This dreary state of affairs is due in part to the fact that "checking" an RSS just means downloading the entire file containing all posts the author would like to make public, regardless of how many are new to the reader. This insanely unnecessary bandwidth usage penalizes sites that have long (>10) feeds with complete posts. (The single RSS file on my blog takes up the majority of my bandwidth costs.) Still, excep…

That’d impose a cost on the server, remembering an ID map. Instead make the client send back a value given by the server last time.

That's something both RSS consumers and produces should already support, but don't always, called an E-Tag, a standard part of the HTTP standard. However, the E-Tag is all-or-nothing; either it matches, and the entire request is essentially aborted, or it doesn't match, and the entire file is served up.

Passing "the ID of the last piece of content I saw" would allow the server to return just the updated stuff, or abort early like an E-Tag. However, counterintuitively, as is often the way in computer science, I'm not sure it would be that big a win to be able to return partial content. The vast bulk of the win on most blogs will just be the ability to abort at all, provided just fine by E-Tags.

I would say that if your site is getting hammered by HTTP requests for your RSS, do double-check that you've got E-Tags set up and working correctly. It is in the best interests of the big scrapers to support that properly, as they are paying for that bandwidth too. RSS aggregators don't have to get too large before this becomes a top-priority feature request. Unless the feed is literally changing on roughly the same frequency as it is scanned, it shouldn't be the dominant factor in your bandwidth bill.

Re: Winds 2.0: It’s Time to Revive RSS

#133

Post by mtremsal »

gkya wrote: post #63
jessriedel wrote: post #35

Earlier quoted context omitted.

For reasons I don't understand, it remains incredibly difficult to scrape a well-organized blog and turn it into something I can consume like RSS or a kindle book. Note that many/most WordPress RSS feeds aren't that useful because they only show a snippet rather than the entire post. This dreary state of affairs is due in part to the fact that "checking" an RSS just means downloading the entire file containing all po…

For your RSS issue, you can safely truncate your RSS feed to the latest N articles, where N is a random reasonable number you choose (10 is good, because it's so decimal). RSS reader software will know how to deal with it. WRT scraping, it is getting harder because funny websites w/ no static content where everything is generated via Angular or Perpendicular or sth. are really hard to deal with. Recently my uni switc…

FWIW, I had a similar issue recently: if you cannot find the link targets, you can still simulate a click on the element using phantomjs.

Re: Winds 2.0: It’s Time to Revive RSS

#134

Post by jerf »

jessriedel wrote: post #88
JetSpiegel wrote: post #77

Earlier quoted context omitted.

It might be construed as a feature, since a blog can consist of simply static files, no need for running any code relating to RSS.

It doesn't need code. Just a static text file with a list of publication dates and URLs, each of which points to static HTML file for each publicly available posts. Then each user could check the list and download only what's new, without needing an aggregator. Users would never need to download the same content twice.

You can pretty much do that with RSS, if you want. Slightly decorated with some XML stuff but that'll pretty much gzip right away. Statically, if you use the right web servers and magic invocations the web server may specify.

You'll find your feed readers aren't particularly impressed, though.

Re: Winds 2.0: It’s Time to Revive RSS

#135

Post by allanbreyes »

jake_the_third wrote: post #107
pchristensen wrote: post #15

Earlier quoted context omitted.

I don't think many sites removed RSS, but many new sites and services in the last ~5 years, never implemented or served it in the first place. The typical share icons now are FB, Instagram, and Twitter. Email is less common (although usually there's an email newsletter signup). For a while, there wouldn't be link to RSS on the page, but you could still find the .rss or .atom link in the HTML header. But a lot of time…

Youtube is like this. They don't advertise rss feeds but include them in the page source of youtube channels. In the one or two times I couldn't find the link in the source, I'd just copy the channel id of the channel and use the feed url from another channel. Works like a charm!

A surprising amount of services have this. You can even turn your Gmail inbox into an (authenticated) RSS feed[1]. Small warning: it uses an outdated XML feed namespace, but most readers should handle it fine.

[1]: https://mail.google.com/mail/u/0/feed/atom/

Re: Winds 2.0: It’s Time to Revive RSS

#136

Post by boardwaalk »

harrygeez wrote: post #127
boardwaalk wrote: post #125

I'm surprised rss2email (or something similar) has hardly been mentioned. Getting your RSS in your email makes a surprising amount of sense. You can filter, organize and archive it. You can search it. You can mark it read or put flags on it. You can forward it to friends if that's your thing. And it works everywhere you have an email app. I also have small scripts that run as cron jobs and scrape a few Twitter feeds…

never used it before but I imagine it wouldn't work well with hundreds of updates everyday? Plus centralized feed management is easier with an RSS service like Feedly. I don't want to scour through all my mails for my feeds, as if it wasn't hard enough keeping with mailing lists already.

It would work fine, mail servers and clients are damn comfortable handling those numbers.

I suppose it'd be easier if you never want to have any files to back up, but I already do (emails themselves, git repos, various databases) so storing the rss2email configuration is nothing extra for me.

You can setup filters to put the stuff from RSS in whatever folders you want. By feed itself. By author. By keyword. Just like any other email. I use Sieve with Dovecot/Pigeonhole but any decent hosted mail service should provide something similar.

Re: Winds 2.0: It’s Time to Revive RSS

#137

Post by Izkata »

evrydayhustling wrote: post #80
s73v3r_ wrote: post #56

Earlier quoted context omitted.

Do they? I'm fairly certain they don't. Otherwise everyone would just be torrenting and stripping commercials.

Torrenting is illegal because you (generally) weren't authorized to make a copy. Stripping commercials isn't because you're removing something, or changing the way you read it. There might be some legal case if you have to make another copy of media in order to alter your consumption, but thank God I'm aware of no case attempting to control your personal use of media. (DMCA doesn't count, it's about circumventing thi…

Just to this point, an example: My parents' VCR had a built-in feature that would automatically fast-forward through commercials when playing back something that it recorded.

Re: Winds 2.0: It’s Time to Revive RSS

#138

Post by boardwaalk »

allanbreyes wrote: post #129
boardwaalk wrote: post #125

I'm surprised rss2email (or something similar) has hardly been mentioned. Getting your RSS in your email makes a surprising amount of sense. You can filter, organize and archive it. You can search it. You can mark it read or put flags on it. You can forward it to friends if that's your thing. And it works everywhere you have an email app. I also have small scripts that run as cron jobs and scrape a few Twitter feeds…

Interesting. I'm on the total opposite end... I'd rather have "email2rss" for turning all the spam/newsletters into a feed. I don't want _any_ of that content entering my email. I recommend Kill the Newsletter[1]. It gives you an email address that you can send all your subscriptions into, and an RSS feed to consume it. [1]: https://www.kill-the-newsletter.com/

Couldn't you just filter those newsletters into a folder? I'm not sure how making it RSS is materially different.

Hardly anything I described here goes directly into my inbox, to be clear. That would drive me crazy (I rarely have more than a dozen emails in my inbox -- all things that I need to somewhat immediately need to be act on).

Re: Winds 2.0: It’s Time to Revive RSS

#139

Post by bjt »

unicornporn wrote: post #70

I don't understand why you would ever want this to be a desktop app. Try FreshRSS[1], it's awesome. It works well on shared hosting and it runs on SQLite. TTRSS only gave me trouble. Threw all kinds of strange errors at unexpected times. I don't know how many times it died on me after an upgrade. I eventually gave up and found FreshRSS. Been running (and updating) it over a year, without a single problem. One of the…

> I don't understand why you would ever want this to be a desktop app.

Perhaps because the vast majority of people in the world have no interest in figuring out web hosting so they can read some news articles. That doesn't seem like a viable way to "bring back RSS".

Re: Winds 2.0: It’s Time to Revive RSS

#140

Post by PhasmaFelis »

unicornporn wrote: post #70

I don't understand why you would ever want this to be a desktop app. Try FreshRSS[1], it's awesome. It works well on shared hosting and it runs on SQLite. TTRSS only gave me trouble. Threw all kinds of strange errors at unexpected times. I don't know how many times it died on me after an upgrade. I eventually gave up and found FreshRSS. Been running (and updating) it over a year, without a single problem. One of the…

> I don't understand why you would ever want this to be a desktop app.

I thought you were going to say that it needs to be a website+mobile app for maximum adoption, and I was ready to say that you have a point but full-weight desktop apps still have their place. Then I looked up FreshRSS, and it's an aggregator that you host yourself?

Any answer to the question "how can we get more regular users to adopt a service" that starts "first, they all need to install Apache..." is very doomed.

Post reply on HN