Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Dawn of a new era in Search: Balancing innovation, competition, and public good

blog.kagi.com

Post reply on HN
Print view View original

51–60 of 83 posts

Re: Dawn of a new era in Search: Balancing innovation, competition, and public good

#51

Post by pierrefermat1 »

throwaway14356 wrote: post #21

> Google has built a massive index of the internet that covers close to 100% of the accessible web. While their index (of other peoples stuff) is enormous it far from includes everything. It is easy to disqualify and people would be screaming if content farms would be included. What even is a content farm nowadays? One can return a reasonable article for any query with llms rich in links to other pages that don't exi…

It seems like in their mind the snake ate it's tail without realising, so they don't even know what's outside of Google's index.

Re: Dawn of a new era in Search: Balancing innovation, competition, and public good

#52

Post by freediver »

orhmeh09 wrote: post #49
aDyslecticCrow wrote: post #36

Earlier quoted context omitted.

The other three answers to your question were dumb, so I'll try to do better. Kagi has a free trial of 100 searches without requiring a credit card to sign up. Honest to god, try it. Just bookmark it and search once or twice when Google or Bing is giving you meh results. As an engineer... googling for stuff is a good fraction of your job. It's quite reasonable to pay to improve that experience. The only real question…

I don't think it's that amazing for an engineer. Google is much better if you're looking for things related to Intel compilers, for example. Try searching for «intel ipo fuse-ld=lld icx» on both engines. Note: I am a paying Kagi customer

I did - and Kagi results look pretty darn good to me. What were you trying to point out?

Re: Dawn of a new era in Search: Balancing innovation, competition, and public good

#53

Post by [deleted] »

freediver wrote: post #29
abtinf wrote: post #28

Earlier quoted context omitted.

You have benefited my life immensely, so I will take the time to write the most helpful response I can. *What is the nature of competition in the free market?* The premise that Google’s success constitutes a monopoly that must be dismantled to protect consumers is rooted in a flawed interpretation of the market and the moral principles governing it. In a free market, companies make money by offering the products and…

I actually agree with many points you make, but some of them miss nuance that I think the article already addresses (we spent 2 months writing it, started even before the ruling). This is an important matter and I am in delicate position as the public spokesman (lots at risk) for the company and at the same time just a user here like any other HN user (obviously willing to engage with fellow HN users). So for that re…

[deleted]

Re: Dawn of a new era in Search: Balancing innovation, competition, and public good

#54

Post by adamcharnock »

freediver wrote: post #38
adamcharnock wrote: post #7

> The Google Search Index is a unique and irreplaceable resource within the digital ecosystem. Mandating fair access to it or treating it as an essential facility could address the core issues... The article estimates the Google Search Index at 12.5PB. If Kagi thinks that is a big enough moat to be the primary target then, well, I suppose they should know. But I'm also skeptical. You could fit that on about 50 Hetzne…

> The article estimates the Google Search Index at 12.5PB. I realize there was a mistake with the estimated number (thanks for pointing out, should be closer to 180 PB for raw crawl data). Since this is speculative and also does not account for other data needed to actually rank pages, hardware to do it in under 500ms at a scale of billions of queries per day and thus can be misleading in terms of true effort to do i…

Thank you for the reply!

> it is about creating a searchable index of the web scale that has certain quality level that is simply impossible to do anymore for many reasons that would require another article to explain

I am both happy to take your word for it, and also very interested to know more. If you were to write that article then I would love to read it.

Re: Dawn of a new era in Search: Balancing innovation, competition, and public good

#57

Post by bugtodiffer »

icar wrote: post #56

I wish Kagi was cheaper. It's expensive in many countries and it doesn't help that we are bombarded with subscriptions everywhere.

I wish Kagi would use the money to build a search engine.

But instead they use it to build a browser no one wants and an email service no one needs.

Yet their search website is still broken here and there...

Re: Dawn of a new era in Search: Balancing innovation, competition, and public good

#58

Post by squiggy22 »

somethoughts wrote: post #47
ldayley wrote: post #2

I've been using (and occasionally paying) Kagi on and off for a couple of years now. I truly think they're building something interesting and valuable! While I haven't agreed with every product decision they've made, the founder is very good at both understanding his business and also explaining their decisions. This is a well crafted explainer of the search business and the monopoly case-- much better for sharing wi…

I think its instructive to look at the early history of Google and Facebook. In the early years they did not really turn on the ad revenue levers and just focused on increasing users (i.e. Don't Be Evil) - until a decade after offering their respective services. Similarly Netflix is just now starting the ad revenue model after years of only subscription based services. Eventually the temptation for multiple sources o…

I'd argue the minute they turn to an advertising model that subscriptions would churn overnight, but in the same vein as the don't be evil motto being dropped I nod at the scepticism.

Re: Dawn of a new era in Search: Balancing innovation, competition, and public good

#59

Post by ColinHayhurst »

andrewstuart wrote: post #44

Who knows what websites and pages are even on the web? There's no index to the web that I know of apart from Google and DuckDuckGo and maybe this Kagi thing. I want to explore the web - surely search isn't the only way to use the web? I imagine it could be fun to explore the web, lists and graphs of interest where I can hop from here to there via list of links or graphs or nodes or something? Does anyone know of anyt…

Google indexes widely. DuckDuckGo and Kagi have small specialised indexes and as such rely on the larger indexes like Google, Bing and Mojeek. DuckDuckGo used to use Yandex. More information here: https://www.searchenginemap.com/

Re: Dawn of a new era in Search: Balancing innovation, competition, and public good

#60

Post by 1vuio0pswjnm7 »

arnaudsm wrote: post #48
adamcharnock wrote: post #7

> The Google Search Index is a unique and irreplaceable resource within the digital ecosystem. Mandating fair access to it or treating it as an essential facility could address the core issues... The article estimates the Google Search Index at 12.5PB. If Kagi thinks that is a big enough moat to be the primary target then, well, I suppose they should know. But I'm also skeptical. You could fit that on about 50 Hetzne…

I don't buy this number. Text-only common crawl is 20TB. Remove spam and dupes, you're around It's the full Google index history with full HTML that is probably 12PB, but the useful part of the search engine is much smaller.

Does CC publish the methodology for how they determine what to crawl. More particularly, how do they determine what not to crawl.
Post reply on HN