Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Waiting for dawn in search: Search index, Google rulings and impact on Kagi

blog.kagi.com

Post reply on HN
Print view View original

201–210 of 266 posts

Re: Waiting for dawn in search: Search index, Google rulings and impact on Kagi

#201

Post by walt_grata »

shmeeed wrote: post #195
thisislife2 wrote: post #172

> Layer 3: Paid, subscription-based search Should actually be - Layer 3: Paid, ad-free , subscription-based search. (It's a subtle omission that indicates the direction Kagi search will eventually take).

I don't see how it's conductive to the underlying big stakes discussion if we start condemning Kagi over this omission instead of assuming good faith. It's just distracting from the real issue at hand that is Google's monopoly and the details of the rulings and their enforcement. Call me naive, but I imagine Kagi would be VERY hesitant to force ads on their users, given that such a step would risk alienating a major…

I don't know dude. Every time I've assumed good faith on my paying for something means ad free, I've been screwed by some asshole with an MBA getting into a leadership position high enough to push ads through. I'd rather it be explicit

Re: Waiting for dawn in search: Search index, Google rulings and impact on Kagi

#202

Post by direwolf20 »

Nextgrid wrote: post #139
AlienRobot wrote: post #133

Earlier quoted context omitted.

To me, a lot of problems with "building a search engine" don't seem to be problems with "building a search engine," they seem to be problems with "building a Google." Nobody said a search engine needs to have fresh data, for example. Nor has anybody said a search engine needs to index the entire web. Yet these are two things every search engine tries to do, and then they usually fail to compare with Google. To put it…

I don't think TikTok "succeeded" compared to Youtube? TikTok succeeded in popularizing short-form video, but I'd argue that's a different product. YouTube is still king for longform video. While there might be arguments for building a different product (and LLM-based search like Perplexity is trying it), there appears to be enough demand for a "good Google" that Kagi is trying to address.

If the product is long-form video then sure... but if the product is user attention? Film camera companies still make the best film cameras, but is the product film cameras or is it taking photos?

Re: Waiting for dawn in search: Search index, Google rulings and impact on Kagi

#203

Post by monooso »

direwolf20 wrote: post #196
kavalg wrote: post #178

Earlier quoted context omitted.

Wouldn't it be nice if Microsoft opened the bing index for all.

Don't they? DDG and Kagi use it. I would think you have to pay money but it does seem like they're willing to get partners. edit: this is wrong

This is incorrect. Kagi does not use the Bing index, as detailed in the article:

> Bing: Their terms didn’t work for us from the start. Microsoft’s terms prohibited reordering results or merging them with other sources - restrictions incompatible with Kagi’s approach. In February 2023, they announced price increases of up to 10x on some API tiers. Then in May 2025, they retired the Bing Search APIs entirely, effective August 2025, directing customers toward AI-focused alternatives like Azure AI Agents.

Re: Waiting for dawn in search: Search index, Google rulings and impact on Kagi

#204

Post by direwolf20 »

keeda wrote: post #117

Google's advantage is not just in its index and algorithms, it is that it has built a self-reinforcing flywheel that data mines human attention at massive scale to improve their search results. This comment ( https://news.ycombinator.com/item?id=46709957 ) points out that Google got its start via PageRank, which essentially ranked sites based on links created by humans . As such, its primary heuristic was what humans…

> cookies (for which they built Gmail!)

Can you explain this one?

Re: Waiting for dawn in search: Search index, Google rulings and impact on Kagi

#205

Post by dmje »

decimalenough wrote: post #192
thisislife2 wrote: post #172

> Layer 3: Paid, subscription-based search Should actually be - Layer 3: Paid, ad-free , subscription-based search. (It's a subtle omission that indicates the direction Kagi search will eventually take).

Kagi has been pretty consistent about funding itself with paying users instead of ads. I, for one, am a paid user but would quit if there were ads injected, since not having them (and, more importantly, having result rankings corrupted by them) is the exact thing I'm paying for.

Paid user and early adopter here - same, I think. I'm delighted with Kagi, but the thought of it riddled with ads makes me sad. My understanding is same as yours - this is an attempt at an entirely different business model - moving to ads would be totally contrary to what they're trying (or at least - to date - have been trying) to build.

Really hope they don't go this way...!

Re: Waiting for dawn in search: Search index, Google rulings and impact on Kagi

#206

Post by direwolf20 »

jxmesth wrote: post #157

Honestly, would be very cool if someone could make a search engine of only human-produced content. I know it's going to be hard and compute intensive but I don't think it's impossible. In fact, Google could do it. A paid service for only human made content. Obviously there would be a margin of error as we can never be 100% sure if something really is AI written.

Marginalia Search is a small-web search engine with a curated list of sites and its own index. Sometimes I find it useful to find answers to technical problems because it only searches the kind of site where people write about the technical problems they solved.

Re: Waiting for dawn in search: Search index, Google rulings and impact on Kagi

#207

Post by bayesnet »

thisislife2 wrote: post #172

> Layer 3: Paid, subscription-based search Should actually be - Layer 3: Paid, ad-free , subscription-based search. (It's a subtle omission that indicates the direction Kagi search will eventually take).

It should actually be “Layer 3: Paid, ad-free, asbestos free, subscription-based search”. Come on. I don’t think it’s productive to make decisive and conspiratorial declarations on the future of Kagi search because they didn’t use the magic words you like. [ And since I know freediver is active here I want to state plainly that I would cancel immediately if there is so much as a hint of an ad in the Kagi results ;) ]

edit: h/t to https://xkcd.com/641

Re: Waiting for dawn in search: Search index, Google rulings and impact on Kagi

#208

Post by eloisius »

Nextgrid wrote: post #134
dylan604 wrote: post #112

Earlier quoted context omitted.

Why does Google even need to know about your ladder? Build the bot, scale it up, save all the data, then release. You can now remove the ladder and obey robots.txt just like G. Just like G, once you have the data, you have the data. Why would you tell G that you are doing something? Why tell a competitor your plans at all? Just launch your product when the product is ready. I know that's anathema to SV startup logic,…

Running the bot nowadays is hard, because a lot of sites will now block you - not just by asking nicely via robots.txt, but by checking your actual source IP. Once they see it's not Google, they send you a 403.

Cloudflare’s ubiquity makes bootstrapping a search index via crawler virtually impossible, but what about data sources like Common Crawl?

Re: Waiting for dawn in search: Search index, Google rulings and impact on Kagi

#209

Post by direwolf20 »

ares623 wrote: post #37

Kagi should start building an index of sites that are trying to escape the current slop internet. It’s know they have the Small Web thing. But I’d like to see an index of a “neo internet” that blocks Google et al.

Add your site to Marginalia Search. They accept submissions by email or GitHub PR, and Kagi pulls from Marginalia Search.

Re: Waiting for dawn in search: Search index, Google rulings and impact on Kagi

#210

Post by 8bitsrule »

dylan604 wrote: post #125
Imustaskforhelp wrote: post #124

Earlier quoted context omitted.

It's just so wild how great Duckduckgo is & how under-rated it is. It's available in all major browsers (Here in zen browser, it doesn't even have a default browser but rather on the start page it asks between the three options, google duckduckgo and bing but yes if you press next it starts from google but zen can even start from ddg, its not such a big deal) Duckduckgo is super amazing. I mean they are so amazing an…

I use DDG myself. I just assumed that I'm not a very sophisticated user as I've never had it not serve my needs based on how other people here say it's not very good.

>I've never had it not serve my needs

Same here. It may be 'not very good' for highly specialized or complex technical questions ... but I do research across a broad range of (non-specialized) topics daily. I often need to find 2nd and 3rd points of view on a topic ... or detailed facts about singular events ... and I rarely need to go to the 2nd page. And all ad-free!

It's a remarkable education tool. A curious, explorative kid these days could easily sail WAY beyond their age group using DDG. I can only wish I'd had it.

Their recently added 'Search assistant' consistently provides a couple of CITATIONS to backup its (multi-leveled) responses (Ask for more, get more.) I've seen nothing like it elsewhere. It is even quite good at diggin up useful ... and working ... example code for some languages. Also with citations.

Post reply on HN