we spent a decade killing structured feeds in favor of algorithmic timelines and now we're rebuilding them because the algorithms need structured feeds. the circle of life, but for protocols.
Now AI agents need what RSS does
41–50 of 71 posts
Re: Now AI agents need what RSS does
#42Earlier quoted context omitted.
Getting 406 browser not supported for ESR Firefox 140. If I set my UA to "FUCKIT" I can use the site perfectly fine. Why is there a User Agent Filter that disables the whole website? This should be maybe a warning, not a complete block.
you know, I had setup some analytics filtering based on geoip because I was getting crazy spam traffic from Chine and Singapore, but that should only be affecting analytics not the whole site. Mind if I ask where you're located? (you can email me privately if preferred: me@dchuk.com)
Europe
IP address has no effect on the User Agent block though...
Re: Now AI agents need what RSS does
#43I have this idea, that instead of browsing completely random things on the internet pushed by what other people are interested in (or want to promote), create an llm that scans through your backlog of projects YOU want to do, and then search the internet for projects/articles about those things, and then create a feed from that.
I'm not sure why I keep reading HN, 99% of the content is uninteresting, probably 99.9% now that every article is about AI. maybe I just like clicking on things.
Re: Now AI agents need what RSS does
#44I need what rss does. Can someone reccomend a way to create an rss feed from a site that has none?
freshrss will do that for you.
It has a built in web scraper.
Re: Now AI agents need what RSS does
#45...And Semantic Web.
Re: Now AI agents need what RSS does
#46I built a site that's similar in concept to Hacker News, but is entirely fed by RSS feed content, that is then bullet-pointed summarized on the article page: https://engineered.at/ But I also extract topics automatically from the content too with LLMs, to allow for dynamic topic pages that users can separately subscribe to to tune their feeds. Haven't promoted it much, but it's pretty amazing what you can do for a co…
This looks great, I've wanted something like this for a while. Finding how to click through to the actual item in the feed was a high point of friction for me. I went to a topic and then clicked on the header of something I was interested in expecting to be brought to the blog post directly. Needing to click on that same title again to be brought to the post was unintuitive to me, I searched around the page, went bac…
Great feedback, should be straightforward to make happen. I’ll try to implement tonight.
Re: Now AI agents need what RSS does
#47[flagged]
Re: Now AI agents need what RSS does
#48Earlier quoted context omitted.
you know, I had setup some analytics filtering based on geoip because I was getting crazy spam traffic from Chine and Singapore, but that should only be affecting analytics not the whole site. Mind if I ask where you're located? (you can email me privately if preferred: me@dchuk.com)
Europe IP address has no effect on the User Agent block though...
Yeah I know and agree, just wondering if something is haywire in that logic somehow. Otherwise it’s a bizarre issue but I’ll get it fixed
Re: Now AI agents need what RSS does
#49Earlier quoted context omitted.
That’s…bizarre. Let me take a look EDIT: just checked in firefox, I don't see an issue. can you email me at me@dchuk.com and maybe I can debug with you?
I just noticed the same thing. UA being blocked for example: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:140.0) Gecko/20100101 Firefox/140.0 Did mess with it some more: Allowed: Opera/9.80 (Windows NT 6.1; U; zh-tw) Presto/2.7.62 Version/11.01 Opera/9.80 (Windows NT 5.1; U; cs) Presto/2.7.62 Version/11.01 406: Mozilla/5.0 (Windows NT 5.1) Gecko/20100101 Firefox/14.0 Opera/12.0 Mozilla/5.0 (Macintosh; Intel Mac OS X 1…
Thanks for this info! Very helpful
Re: Now AI agents need what RSS does
#50I built a site that's similar in concept to Hacker News, but is entirely fed by RSS feed content, that is then bullet-pointed summarized on the article page: https://engineered.at/ But I also extract topics automatically from the content too with LLMs, to allow for dynamic topic pages that users can separately subscribe to to tune their feeds. Haven't promoted it much, but it's pretty amazing what you can do for a co…
As a sysadmin hosting a few blogs, do you mind sharing what IP ranges you crawl from? Or what agent your requests use? Thank you.
I presume you’re politely asking in order to block? Which is fine, I get it. On my phone right now but can update later.
I do want to ask though (and I should make this clear in a FAQ or something): the way I check RSS feeds uses adaptive scheduling, so I intentionally don’t check feeds of sites too rapidly. Then the summarization is based on the full article content but I never render that full content on the site (to avoid traffic hijacking concerns). Given that: what’s the concern?