Matadisco – Decentralized Data Discovery • Hacker News Board

Re: Matadisco – Decentralized Data Discovery

#11

Post by gessha » Sat, Mar 28, 2026, 7:42 PM UTC

stingraycharles wrote: post #3

mentalgear wrote: post #2

Decentralised Web Indexes are an important avenue if we truly want to ever break free from the private indexes used and dominated by Google and chatbots that in-transparently return results based on their opaque preferences.

Those indexes cost an incredible amount of money to build, maintain and keep improving. It’s worth it for these orgs as these are considered a competitive edge and the “secret sauce” of these orgs. I find it hard to believe the same amount of investment will be made into a decentralized web index, precisely for the reason that it cannot be monetized.

Are you familiar with any systems that use collaborative filtering to distribute the work and the benefits among users?

Re: Matadisco – Decentralized Data Discovery

#12

Post by [deleted] » Sun, Mar 29, 2026, 7:36 AM UTC

somat wrote: post #9

stingraycharles wrote: post #3

Earlier quoted context omitted.

Those indexes cost an incredible amount of money to build, maintain and keep improving. It’s worth it for these orgs as these are considered a competitive edge and the “secret sauce” of these orgs. I find it hard to believe the same amount of investment will be made into a decentralized web index, precisely for the reason that it cannot be monetized.

No they don't. Sure, if you are trying to capture the entire web with hourly latency for billions of users, that takes significant resources. But if your goal is to catalog the subsection of the web you are interested in with weekly latency for yourself and share that catalog with like minded individuals. any body can do that. Don't make gods out of basic web indexing.

[deleted]

Re: Matadisco – Decentralized Data Discovery

#13

Post by mentalgear » Sun, Mar 29, 2026, 9:37 AM UTC

stingraycharles wrote: post #3

mentalgear wrote: post #2

Decentralised Web Indexes are an important avenue if we truly want to ever break free from the private indexes used and dominated by Google and chatbots that in-transparently return results based on their opaque preferences.

Those indexes cost an incredible amount of money to build, maintain and keep improving. It’s worth it for these orgs as these are considered a competitive edge and the “secret sauce” of these orgs. I find it hard to believe the same amount of investment will be made into a decentralized web index, precisely for the reason that it cannot be monetized.

Doesn't have to be expensive: You can have decentralised cohorts of same interests groups that provide common indexes, with each client verifying the index of every n item they use, reporting bad indexes/cohort users to avoid abuse.

Also there are state-funded open search indexes, like the European Search Index.

So it can be done realisticly, even though the massive private search index lobby push as hard as they can to deny it.