Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Waiting for dawn in search: Search index, Google rulings and impact on Kagi

blog.kagi.com

Post reply on HN
Print view View original

141–150 of 266 posts

Re: Waiting for dawn in search: Search index, Google rulings and impact on Kagi

#141

Post by Nextgrid »

xnx wrote: post #60
CGMthrowaway wrote: post #29

Earlier quoted context omitted.

Google is allowed to be big, be better and win users. But happy customers is not the full test of monopolization. The real question is, "Could a meaningfully better search engine realistically displace Google today?” If the answer is no, then competition is broken

> "Could a meaningfully better search engine realistically displace Google today?” ChatGPT clearly demonstrated that displacing Google is possible. All previous monopoly arguments seemed even more flimsy after that.

ChatGPT did not build a search engine though. They built something else (equally impressive) and then were able to use their weight to enter the web search business where most sites now have to allow them in.

While it's good that building other products is possible, it doesn't detract from the point that search engines are a de-facto monopoly.

Re: Waiting for dawn in search: Search index, Google rulings and impact on Kagi

#143

Post by Nextgrid »

WhereIsTheTruth wrote: post #39

Kagi's "waiting for dawn" is just waiting for Google to legitimize their reseller business Meanwhile, users pay a premium to pretend they're not using Google Fascinating delusion

Users pay a premium to have Google's results cleaned out of spam/trash. It's effectively paying someone to cut out the newspaper ads for you and then give you the resulting ad-free paper.

Re: Waiting for dawn in search: Search index, Google rulings and impact on Kagi

#145

Post by creato »

dragonwriter wrote: post #122
creato wrote: post #95

Earlier quoted context omitted.

robots.txt was being enforced in court before google even existed, let alone before google got so huge: > The robots.txt played a role in the 1999 legal case of eBay v. Bidder's Edge,[12] where eBay attempted to block a bot that did not comply with robots.txt, and in May 2000 a court ordered the company operating the bot to stop crawling eBay's servers using any automatic means, by legal injunction on the basis of tr…

Not only was eBay v. Bidder's Edge technically after Google existed, not before, more critically the slippery-slope interpretation of California trespass to chattels law the District Court relied on in it was considered and rejected by the California Supreme Court in Intel v. Hamidi (2003), and similar logic applied to other states trespass to chattels laws have been rejected by other courts since; eBay v. Bidder's E…

The point is, robots.txt was definitely a thing that people expected to be respected before and during google's early existence. This Kagi claim seems to be at least partially false:

> Google built its index by crawling the open web before robots.txt was a widespread norm, often over publishers’ objections.

Re: Waiting for dawn in search: Search index, Google rulings and impact on Kagi

#146

Post by echelon »

If there are any Kagi folks here, I've come up with a new angle to attack Google's anti-competitive position that could be incredibly effective:

https://news.ycombinator.com/item?id=46681985

https://news.ycombinator.com/item?id=44546519

I'm going to send this idea to my legislators, the EU, Sam Altman, Tim Sweeny, and Elon Musk, et al., I just haven't had time to put this together yet.

Google is a monopolist scourge and needs to be knocked down a peg or two.

This should also apply to the iPhone and Android app stores.

Re: Waiting for dawn in search: Search index, Google rulings and impact on Kagi

#147

Post by cyberrock »

reddalo wrote: post #130
jiehong wrote: post #120

I think one side problem is that part of the web is not even searchable with a search engine. Here are some examples: - Discord - WeChat (is it the web?) - Rednote - TikTok (partially) - X (partially) - JSTOR (it finds daily, but you find more stuff on the website directly) - any stuff with a login, obviously.

> Discord Damn, I can't stand open-source projects that host their "forums" on Discord. It's a nigthmare to use, it's heavy, slow, and it's completely unsearchable from the web. I wonder what went wrong with our society.

First of all not everyone wants spectators and gawkers on all of their conversations. As for open solutions, IRC didn't provide chat history for the common folk (no, most users are not able to host their own Pi Zero bouncer, especially back in 2017), and Matrix development was too slow (Elements implemented message pinning in 2022), so the rest was history. There was just no alternative to Slack or Discord.

Re: Waiting for dawn in search: Search index, Google rulings and impact on Kagi

#148

Post by stacktraceyo »

Is there a crowd indexed style search index? Like instead of relying on the crawling completely you rely on a maybe like an extension in your browser that indexes as people are using their browser. Or maybe indexing your site to this index instead of waiting to be crawled.

Re: Waiting for dawn in search: Search index, Google rulings and impact on Kagi

#149

Post by tyre »

elAhmo wrote: post #59
lolc wrote: post #27

Earlier quoted context omitted.

I guess they'd argue that the people in China don't count, because people in China don't get to choose Google. But yeah, the stats they use from "StatCounter" are clearly not representative for what the world uses.

You can argue that people outside of China don't get to choose something other than Google. Sure, there are recent pushes with default search engine choices and similar initiatives, but there is a reason why Google is paying hundreds of millions of dollars to be the default search engine.

It’s reasonable to see a distinction between the great firewall and the default browser search engine

Re: Waiting for dawn in search: Search index, Google rulings and impact on Kagi

#150

Post by manquer »

lolc wrote: post #27
WhyNotHugo wrote: post #18

The statistics in this article sound like garbage to me. Google used by 90% or the world? ~20% of the human population lives in countries where Google is blocked. OTOH, Baidu is the #1 search engine in China, which has over 15% of the world’s population… but doesn’t reach 1%? These stats are made measuring US-based traffic, rather than “worldwide” as they claim.

I guess they'd argue that the people in China don't count, because people in China don't get to choose Google. But yeah, the stats they use from "StatCounter" are clearly not representative for what the world uses.

Market share is based on factual consumption numbers however subsidized or regulated by a government not free will.

Choice/Free will is an arbitrary line in the sand, one could argue how much choice we have about consuming google search when it is "85-90"% monopolistic business with well documented anti-competitive practices.

Chinese consumers perhaps have more choice than we do, Baidu is only about 60% market share. They do get to choose, it more that Google is not one of the options available to them, it is not like if not Baidu then it is a Phone Book.

Post reply on HN