Live data from Hacker News

The “small web” is bigger than you might think

kevinboone.me

131–140 of 263 posts

Re: The “small web” is bigger than you might think

#131
post #44

I don't expect many people to agree but I think that the "small web" should reject encryption, which is the opposite direction that Gemini is taking. I don't deny the importance of encryption, it is really what shaped the modern web, allowing for secure payment, private transfer of personal information, etc... See where I am getting at? Removing encryption means that you can't reasonably do financial transactions, ac…

TLS client certs are quite a nice approach to identity though.

Re: The “small web” is bigger than you might think

#132

One objection I have to the kagi smallweb approach is the avoidance of infrequently updated sites. Some of my favorite blogs post very rarely; but when they post it's a great read. When I discover a great new blog that hasn't been updated in years I'm excited to add it to my feed reader, because it's a really good signal that when they publish again it will be worth reading.

One of the many things I disagree with Scott Alexander on is that to me, frequent blog updates signal poor quality not excellent writing. Its hard to come up with an independent, evidence-based opinion on something worth sharing every week, but easy to post about what you read lots of angry or scary posts about. People who post a lot also tend to have trouble finding useful things to do in their offline life. It is very unusual that he managed to be both a psychiatrist and a prolific blogger and he quit the psychiatry job before he had children or other care responsibilities.

Re: The “small web” is bigger than you might think

#135
post #44

I don't expect many people to agree but I think that the "small web" should reject encryption, which is the opposite direction that Gemini is taking. I don't deny the importance of encryption, it is really what shaped the modern web, allowing for secure payment, private transfer of personal information, etc... See where I am getting at? Removing encryption means that you can't reasonably do financial transactions, ac…

I think the small web ought to just serve everything over Tor (onion sites). No domain needed, no worries about surveillance. You can put a gateway in front of your minimalist web server or run the gateway on an OpenWRT router. This also helps the network by generating cover traffic and encouraging civilian use.

Re: The “small web” is bigger than you might think

#136
post #68
post #53

Earlier quoted context omitted.

Caveat that Kagi gates that repo such that it doesn't allow self-submissions so you're only going to see a chunk of websites that other people have submitted that also know about the Kagi repo.

But per the instructions, it seems like that if one wants to add your own website, then one needs to add 2 other small websites (that are not on the list already)...so technically it does open things up to those who are not aware of the repo...assuming their site is pulled in when someone wants to add their own website. Obviously this scale is slow...but i think that's kinda the point, eh? Nevertheless, for every 1 p…

Yeah I’ve added my own site along with 3 others and the PR was merged in an hour.

Honestly the hard part was that a lot of the sites I wanted to submit were already there!

Re: The “small web” is bigger than you might think

#137
post #101

Earlier quoted context omitted.

Didn't the creator of the protocol go on a rant when someone made a browser for Gemini that included a favicon? I can't imagine the backlash if someone tried to normalize Markdown. Isn't the entire point of Gemini that it can never be extended or expanded upon? Maybe it would be better to create an entirely different protocol/alt web around Markdown that didn't risk running afoul of Gemini's philosophical restriction…

I think a "markdown-web" that uses some of the Gemini approaches for privacy and auth/identity etc would be pretty nice. Of course, as others have said, we could just use HTML without JavaScript or cookies and we'd be a lot of the way there with 95% less effort but hey in the future we'll probably just query an AI rather than load a web page ourselves.

Given how many people on HN say they like Gemini in principle but wish it weren't so restrictive, some people would use it. All of those people might just be that cross section of HN users, however.

Re: The “small web” is bigger than you might think

#138
post #85

Earlier quoted context omitted.

You can check: https://github.com/kagisearch/smallweb/blob/main/smallweb.tx... >. I can see that your RSS URL is listed there. But it currently does not appear in the search results here: https://kagi.com/smallweb/?search=zahlman >. The reason appears to be this: "If the blog is included in small web feed list (which means it has content in English, it is informational/educational by nature and it is not trying to se…

Why would you only include blogs in your small web index? That must be a minute fraction of what is out there? I can't think of a single blog that I read these days (small or not), yet there are loads of small "old school" sites out there that are still going strong.

> Why would you only include blogs in your small web index?

I am not associated with this project, so this would be a question for the project maintainer. As far as I understand, the project relies on RSS/Atom feeds to fetch new posts and display them in the search results. I believe, this is an easier problem to solve than using a full blown web crawler.

However, as far as I know, Kagi does have its own full blown crawler, so I am not entirely sure why they could not use it to present the Small Web search results. Perhaps they rely on date metadata in RSS feeds to determine whether a post was published within the last seven days? But having worked on an open source web crawler myself, many years ago, I know that this is something a web crawler can determine too if it is crawling frequently enough.

So yes, I think you have got a good point and only the project maintainer can provide a definitive answer.

Re: The “small web” is bigger than you might think

#139
post #7

A little shell function I have in my ~/.zshrc: pages() { for _ in {1..5}; do curl -sSw '%header{location}\n' https://indieblog.page/random | sed 's/.utm.*//'; done } Here is an example output: $ pages https://alanpearce.eu/post/scriptura/ https://jmablog.com/post/numberones/ https://www.closingtags.com/blog/home-networking https://www.unsungnovelty.org/gallery/layers/ https://thoughts.uncountable.uk/now/ On macOS, we…

That's a lovely bit of automation.

What do you mean automation, he’s not even using any AI agent!

Re: The “small web” is bigger than you might think

#140
post #131
post #44

I don't expect many people to agree but I think that the "small web" should reject encryption, which is the opposite direction that Gemini is taking. I don't deny the importance of encryption, it is really what shaped the modern web, allowing for secure payment, private transfer of personal information, etc... See where I am getting at? Removing encryption means that you can't reasonably do financial transactions, ac…

TLS client certs are quite a nice approach to identity though.

[deleted]
Post reply on HN