Live data from Hacker News

The “small web” is bigger than you might think

kevinboone.me

51–60 of 263 posts

Re: The “small web” is bigger than you might think

#51
post #45

Kagi Small Web has about 32K sites and I'd like to think that we have captured most of (english speaking) personal blogs out there (we are adding about 10 per day and a significant effort went into discovering/fidning them). It is kind of sad that the entire size of this small web is only 30k sites these days.

I'm noticing sites that break the rules. I report (flag) them, is that useful or should I just PR to remove them?

PR is better!

Re: The “small web” is bigger than you might think

#52
post #7

A little shell function I have in my ~/.zshrc: pages() { for _ in {1..5}; do curl -sSw '%header{location}\n' https://indieblog.page/random | sed 's/.utm.*//'; done } Here is an example output: $ pages https://alanpearce.eu/post/scriptura/ https://jmablog.com/post/numberones/ https://www.closingtags.com/blog/home-networking https://www.unsungnovelty.org/gallery/layers/ https://thoughts.uncountable.uk/now/ On macOS, we…

That's a lovely bit of automation.

Re: The “small web” is bigger than you might think

#53
post #7

A little shell function I have in my ~/.zshrc: pages() { for _ in {1..5}; do curl -sSw '%header{location}\n' https://indieblog.page/random | sed 's/.utm.*//'; done } Here is an example output: $ pages https://alanpearce.eu/post/scriptura/ https://jmablog.com/post/numberones/ https://www.closingtags.com/blog/home-networking https://www.unsungnovelty.org/gallery/layers/ https://thoughts.uncountable.uk/now/ On macOS, we…

Caveat that Kagi gates that repo such that it doesn't allow self-submissions so you're only going to see a chunk of websites that other people have submitted that also know about the Kagi repo.

Re: The “small web” is bigger than you might think

#54

Earlier quoted context omitted.

Whether the results are less relevant or not depends massively on what you searched and whether the best results even exist in the Marginalia search index or not. If Google is ranking small web results better than Marginalia, that’s actionable. If the best result isn’t in the index and it should be, that’s actionable.

Well to be fair, Marginalia is also developed by 1 guy (me), and Google has like 10K people and infinite compute they can throw at the problem. There has been definite improvements, and will be more improvements still, but Google's still got hands.

Hey Marginalia, cheers. Imo fewer hands can also be an advantage.

There are no PMs breathing down your neck to inject more ads in the search results, you don’t depend on any broken internal bespoke tools that you can’t fix yourself, and you don’t need anybody’s permission to deploy a new ranking strategy if you want to.

Re: The “small web” is bigger than you might think

#55
post #48
post #44

I don't expect many people to agree but I think that the "small web" should reject encryption, which is the opposite direction that Gemini is taking. I don't deny the importance of encryption, it is really what shaped the modern web, allowing for secure payment, private transfer of personal information, etc... See where I am getting at? Removing encryption means that you can't reasonably do financial transactions, ac…

> But what I think is an even better solution is to do it at the content level: sign the content, like a GPG signature How would this work in reality? With the current state of browsers this is not possible because the ISP can still insert their content into the page and the browser will still load it with the modified content that does not match the signature. Nothing forces the GPG signature verification with curre…

Signatures do have similar problems to certificates. But Gemini doesn't avoid them either and often recommends TOFU certificates. I think the comment's point was that digital signatures ensure identity but are unsuitable for e-commerce, a leading source of enshittification.

Re: The “small web” is bigger than you might think

#56
post #48
post #44

I don't expect many people to agree but I think that the "small web" should reject encryption, which is the opposite direction that Gemini is taking. I don't deny the importance of encryption, it is really what shaped the modern web, allowing for secure payment, private transfer of personal information, etc... See where I am getting at? Removing encryption means that you can't reasonably do financial transactions, ac…

> But what I think is an even better solution is to do it at the content level: sign the content, like a GPG signature How would this work in reality? With the current state of browsers this is not possible because the ISP can still insert their content into the page and the browser will still load it with the modified content that does not match the signature. Nothing forces the GPG signature verification with curre…

> you are back to the same problems that certificates have.

Some of the same problems. One nice thing about verifying content rather than using an SSL connection is that plain-old HTTP caching works again.

That aside, another benefit of less-centralized and more-fine-grained trust mechanisms would be that a person can decide, on a case-by-case basis what entities should be trusted/revoked/etc rather than these root CAs that entail huge swaths of the internet. Admittedly, most people would just use "whatever's the default," which would not behave that differently from what we have now. But it would open the door to more ergonomic fine-grained decision-making for those who wish to use it.

Re: The “small web” is bigger than you might think

#57
post #44

I don't expect many people to agree but I think that the "small web" should reject encryption, which is the opposite direction that Gemini is taking. I don't deny the importance of encryption, it is really what shaped the modern web, allowing for secure payment, private transfer of personal information, etc... See where I am getting at? Removing encryption means that you can't reasonably do financial transactions, ac…

Anyone between you and the server can change the content of the page on unencrypted connections. I would love to live in a world where encryption is unnecessary, but unfortunately that world does not exist right now.

Re: The “small web” is bigger than you might think

#58
I’m not sold on gemini. Less utility, weaker, immature tools. Investing on small HTTP based websites is the right direction. One could formalize it as a browser extension or small-web HTTP proxy that limits JS, dom size, cookie access etc using existing Web browsers & user agents.

Re: The “small web” is bigger than you might think

#59

Earlier quoted context omitted.

Feeding llms you mean

Is there a good free-but-subscriber-only solution for blogs? It seems like a contradiction, but in practice it may be manageable.

Loginwall or email newsletter with a summary on the open web.
Post reply on HN