Live data from Hacker News

Dawn of a new era in Search: Balancing innovation, competition, and public good

blog.kagi.com

21–30 of 83 posts

Re: Dawn of a new era in Search: Balancing innovation, competition, and public good

#21
> Google has built a massive index of the internet that covers close to 100% of the accessible web.

While their index (of other peoples stuff) is enormous it far from includes everything. It is easy to disqualify and people would be screaming if content farms would be included. What even is a content farm nowadays? One can return a reasonable article for any query with llms rich in links to other pages that don't exist but could be indexed and are part of the accessible web

If you make a new website with a few thousand pages and a few thousand images it takes quite a while for google to pick up the entire thing, if it even bothers to.

google tries to fill the result page with a small subset of websites. A good thing for users most of the time and the easiest ad money but horrible for new players.

it use to be quite common for bloggers (and others) to follow everything written about them or of interest. google (blog search) and technorati were very useful for that kind of discovery.

The average user might never have noticed that but when it was killed off the www stopped being a community.

We can pretend the index is still there. If you cant get to it it's s much like the llm content.

Re: Dawn of a new era in Search: Balancing innovation, competition, and public good

#22
I think Kagi is correct and that the way we explore information on the internet will look very different in X years with all the changes LLMs will bring. I think the real question will be what will it look like.

I don't think it looks like search today. Google got where they were because they were 10x better than everything else and had an experience focusing on what mattered at the time. I don't think the 10x experience will look like ten blue links. I don't know what that next experience is, but I'll know it when I see it.

Re: Dawn of a new era in Search: Balancing innovation, competition, and public good

#23

Earlier quoted context omitted.

What is "Lenses", is it the Google Lens?

Here's the docs[0]. It's basically a saved partial filter that you can use when searching. So you can e.g. say you only want to search reddit and have that saved as an easily-accessible button. But there's also predefined ones like "Forums" and "Academic", so that's the general idea. The cool thing is you can enable a Lens when using the AI assistant with internet access, so its searches are also constrained to that…

It's essentially search-space limitation, based on what you want to nudge the engine into responding with.

I.e. what power users used to do with Google, before it decided to ignore user intent

Re: Dawn of a new era in Search: Balancing innovation, competition, and public good

#24
post #2

I've been using (and occasionally paying) Kagi on and off for a couple of years now. I truly think they're building something interesting and valuable! While I haven't agreed with every product decision they've made, the founder is very good at both understanding his business and also explaining their decisions. This is a well crafted explainer of the search business and the monopoly case-- much better for sharing wi…

i honestly cant imagine paying for a search engine with whats out there for free

Nothing stopping you from trying for one month to confirm.

Re: Dawn of a new era in Search: Balancing innovation, competition, and public good

#25
post #2

I've been using (and occasionally paying) Kagi on and off for a couple of years now. I truly think they're building something interesting and valuable! While I haven't agreed with every product decision they've made, the founder is very good at both understanding his business and also explaining their decisions. This is a well crafted explainer of the search business and the monopoly case-- much better for sharing wi…

i honestly cant imagine paying for a search engine with whats out there for free

You get what you pay for

Re: Dawn of a new era in Search: Balancing innovation, competition, and public good

#26

> Apple has stated that Bing does not match Google’s search result quality, and they are unwilling to compromise on user experience by offering subpar results. I wouldn't take this statement at face value. This is most likely a BS PR excuse for Apple to maintain their current deal with Google. I wouldn't expect anything less from any large corporation looking to protect $20 billion in annual revenue.

Last I tried at least:

- Current Google was bad compared to original Google (it ignores my keywords, even if I use doublequotes and verbatim)

- DuckDuckGo and Bing managed to be worse

- Kagi is like old Google

Re: Dawn of a new era in Search: Balancing innovation, competition, and public good

#27
I've used Kagi for a year and it was better than Google for most of the searches.

However I think the model will be changed to something more like Perplexity.ai

I've switched to Perplexity and for most of the searches it works better than Kagi.

They'd need to add something like this to survive in the long run, because for exploratory searches tools like Perplexity are really good.

Re: Dawn of a new era in Search: Balancing innovation, competition, and public good

#28
post #8

This is an extremely disappointing post. In the past, I’ve enthusiastically supported and advocated for Kagi. But Kagi advocating for using force to destroy its competitors is completely unacceptable to me and an admission that they do not believe they have a viable product. Antitrust law is arbitrary and evil. If you make more money than your competitors, you have undue market power. If you price below your competit…

> But Kagi advocating for using force to destroy its competitors is completely unacceptable to me Everyone is entitled to their own interpretation, but that is not what the article advocates at all. The article is about what is best for the user given the circumstances, where all other proposed remedies have focused on how to hurt Google, which article argues to be counter-productive. The ruling has already been made…

You have benefited my life immensely, so I will take the time to write the most helpful response I can.

*What is the nature of competition in the free market?*

The premise that Google’s success constitutes a monopoly that must be dismantled to protect consumers is rooted in a flawed interpretation of the market and the moral principles governing it.

In a free market, companies make money by offering the products and services demanded by customers. Google's dominance is not the result of coercion but its ability to meet customer demand effectively. Antitrust law declares that the choices of billions of individuals are wrong — not because those trading with the company have been treated unfairly (if they had, they wouldn’t have traded), but because those who know better through some mystical connection with the good declare it so using arbitrary standards.

There have been many competitors to Google, with at least one of them extremely well funded. There are other massive internet-scale companies that could compete with Google, though they’ve often failed in past attempts. Punishing the owners of Google for its success is to punish the very qualities that drive progress and prosperity.

You suggest that Google's control over the search index and its associated business models warrant government intervention to force “fair” access, because it would promote competition. Whatever else you mean by “fair”, you certainly don’t mean “terms that Google would find agreeable”. If one of the parties to a transaction is an unwilling participant, then it cannot be a fair transaction. We have names for this when the coercing party is anyone but the government: blackmail, protection rackets, theft, mugging, and so on.

*What about property rights?*

Simply put, Google’s indexes are the property of Google’s shareholders. Their property was created through decades of sustained investment and innovation. To force it to share its core technologies with competitors would be a gross violation of property rights—the very foundation of a free and productive society.

Your proposal to treat Google’s search index as an "essential facility" that must be shared “fairly” is a call for expropriation—the forced redistribution of Google’s property for the benefit of those whose sole claim to it is that they have not earned it.

The entire argument against Google is that they must be punished because they are successful. Kagi, even as a PBC, will one day be subject to the exact same arguments if it creates as much value as Google—someone will claim that some aspect of Kagi’s operations constitute a monopoly and Kagi should be forced to provide access to some data or service.

*What does it mean for the government to intervene?*

You claim consumers are harmed by the lack of choice in search experiences. Well, what about me? I love your product and pay for it. So do another 32,633 other subscribers. Apparently, we are free to choose your product, and I’m certain many of us advocate for your products to our friends and family. But your product isn’t for everyone. Maybe not even for most. If people want Google and free search, and no one’s rights are being violated, by what authority does anyone get to tell them they are wrong?

Consumers choose Google because they think it meets their needs, not because they are coerced into using it.

Government’s only tool is violence. It can force people to do something (rare, and usually ineffective), or force people to not do something (very common, and usually equally ineffective but with disastrous side effects). Government can’t create "more diverse competition" by forcing Google’s to share its property; competition is the result of companies striving to innovate and offer better products. Government can’t level the playing field by forcing the competitors to do better, it can only cut off the legs of the best players to bring them down to the level of the competition.

Were your proposal for the Google index actually implement, I speculate it would be a few short years until you would see the government declare all other company’s indices to be illegal and subject to regulation--after all, if its a public good, then it must be supported and defended by the public against threats. I would expect to see forced mergers of other indices into the Google index.

*What about non-moral arguments?*

I’m not going to go into detail, but based on my education and extensive past research, in good faith I assert the following to be true: there are no documented instances of long-lasting monopolies that are not perpetuated by governments; every antitrust intervention has made consumers materially worse off; government interventions nearly always have the effect of significantly reducing competition.

If you identify of a robust counter example to any of those claims, you could probably win a Nobel prize.

*What could you do instead?*

A judge has made an unjust ruling advancing an unjust law toward unjust ends--that doesn’t mean the conversation should shift to deciding on the best unjust punishments. That would be a contradiction: there is no such thing as the most-just-unjust punishment.

You could continue your education and advocacy of the true cost of “free” search. Take out an OpEd in NYT hammering Apple’s duplicitous claims of being privacy respecting while sending every customer’s most intimate thoughts to the Google monstrosity by default. Demonstrate the cost to privacy. Demonstrate how your product exposes resources that are hidden by the search giants. Show people the super powers they can have for a modest fee.

And build a better product. Hell, make a Kagi Phone—I’d preorder one even if it were years away from release—and free us from the Apple/Google hegemony.

Re: Dawn of a new era in Search: Balancing innovation, competition, and public good

#29
post #28

Earlier quoted context omitted.

> But Kagi advocating for using force to destroy its competitors is completely unacceptable to me Everyone is entitled to their own interpretation, but that is not what the article advocates at all. The article is about what is best for the user given the circumstances, where all other proposed remedies have focused on how to hurt Google, which article argues to be counter-productive. The ruling has already been made…

You have benefited my life immensely, so I will take the time to write the most helpful response I can. *What is the nature of competition in the free market?* The premise that Google’s success constitutes a monopoly that must be dismantled to protect consumers is rooted in a flawed interpretation of the market and the moral principles governing it. In a free market, companies make money by offering the products and…

I actually agree with many points you make, but some of them miss nuance that I think the article already addresses (we spent 2 months writing it, started even before the ruling). This is an important matter and I am in delicate position as the public spokesman (lots at risk) for the company and at the same time just a user here like any other HN user (obviously willing to engage with fellow HN users). So for that reason I will resist the urge to address such a complex topic ad hoc and ask you to email me at vlad@kagi.com to continue conversation in a more nuanced, async way where I am happy to engage and address each one of your concerns in good faith.

Re: Dawn of a new era in Search: Balancing innovation, competition, and public good

#30

> The Google Search Index is a unique and irreplaceable resource within the digital ecosystem. Mandating fair access to it or treating it as an essential facility could address the core issues... The article estimates the Google Search Index at 12.5PB. If Kagi thinks that is a big enough moat to be the primary target then, well, I suppose they should know. But I'm also skeptical. You could fit that on about 50 Hetzne…

This puts it in enough perspective for me to ask: why doesn’t a university create a public/open source search index? Seems like a way to get a ton of attention.

Moreover, archive.org has all the data and data storage capabilities many times over. What prevents them from creating an open source search engine?

Post reply on HN