I would love to see Yandex take English search more seriously. They're great for some things, but their spam filters are way too harsh, my email has been marked as spam since the day I made it and I can't respond to support since the email account is flagged. I have frequent issues with their search as well, I constantly have to fill out their captcha's which never let me through anyways
Two upstart search engines are teaming up to take on Google
121–130 of 299 posts
Re: Two upstart search engines are teaming up to take on Google
#122Earlier quoted context omitted.
This is one of those things that I think is interesting about how "normal" people use the Internet. I am guessing they just always start with google. But for me, if I want to look up local restaurants, I go straight to Maps/Yelp/FourSquare(RIP). If I want to look up releases of a band, I go straight to musicbrainz. Info about Metal Band, straight to the Encyclopeadia Metallum. History/Facts, straight to Wikipedia. Re…
"Normal" people don't start with Google, not any more. They start with Facebook, Instagram, X, Reddit, Discord, Substack etc. That's exactly the problem, the world-wide web has devolved back into a collection of walled gardens like things were in the BBS era, except now the boards are all run by a handful of Silicon Valley billionaires instead of random nerds in your hometown.
You have just described how "innovation" is often just a power shift, often one that does not benefit the user. Old is new again, but in different hands; the right hands, of course.
Re: Two upstart search engines are teaming up to take on Google
#123Earlier quoted context omitted.
Mostly because Google bought, developed, acquired or effectively control all the major distribution points with default placement deals: eg Apple, Samsung, Chrome, Android, Firefox. In time remedies are coming though, in the antitrust case lost versus DoJ. Another major factor is that building a search index and algorithms that searches across billions of pages with good enough latency is very hard. Easy enough for 1…
Remember that MS lost the anti-trust too, and after the presidential transfer in 2000, they pretty much dropped it with few consequences for MS.
If I had to guess I'd say history is going to repeat itself and Google will also escape any significant consequences.
Re: Two upstart search engines are teaming up to take on Google
#124Earlier quoted context omitted.
>competing with Google was borderline impossible a decade ago. But in 2024, we have cheap compute, great OSS distributed DBs, powerful new vector search tech. [...] CommonCrawl text-only is ~100TB, Those example 3 bullet points of today's improved 2024 computing power you list isn't even enough to process Google's scale 14 years ago in 2010 when the search index was 100+ petabytes : https://googleblog.blogspot.com/20…
Just serving up content from Reddit and HN and a few other websites would be enough to beat Google for most of us. Sprinkle in the top 100 websites and you have a legitimate contender. There is no open web anymore. Google killed it. There are probably fewer than 100k useful websites in the world now. Which is good for startups, because the problem is entirely tractable.
Search engines only searching the top 100 websites is like, the opposite of the way I want things to go...
Re: Two upstart search engines are teaming up to take on Google
#125Every new search engine I've seen was a Bing wrapper with sometimes light reranking. I understand that competing with Google was borderline impossible a decade ago. But in 2024, we have cheap compute, great OSS distributed DBs, powerful new vector search tech. Amateur search engines like Marginalia even run on consumer hardware. CommonCrawl text-only is ~100TB, and can fit on my home server. Why is no company buildin…
I’m my second year into Kagi and loving it. I actually just upgraded to get Kagi Assistant (basically, cloud access to every LLM out there). But the search alone is worth every penny, and it’s built/operated fully in-house as far as I know. https://kagi.com
[deleted]
Re: Two upstart search engines are teaming up to take on Google
#126Do you guys remember Cuil? Man that was a whacky ass engine. Just gave you all sorts of comical results.
Re: Two upstart search engines are teaming up to take on Google
#127Every new search engine I've seen was a Bing wrapper with sometimes light reranking. I understand that competing with Google was borderline impossible a decade ago. But in 2024, we have cheap compute, great OSS distributed DBs, powerful new vector search tech. Amateur search engines like Marginalia even run on consumer hardware. CommonCrawl text-only is ~100TB, and can fit on my home server. Why is no company buildin…
There is probably room for one or five lifestyle businesses but convincing venture capital to drop the megabux to go big would be a feat and eventually land at some sub-optimal state anyway.
Finding some hack to democratize&decentralize the indexing and expensive processes like JavaScript interpretation, image interpretation, OCR, etc is an open angle and even an avenue for "Web3" to offload the cost. But you will ultimately want the core search and index on a tighter knit cluster (many computers physically close to one another for speed of light reasons, although you can have N of these clusters) for performance so it's a hard nut to crack for making something equitable for both the developers and any prospectors and safe from various takeovers and enshitifies. Let us know if you know a way.
Re: Two upstart search engines are teaming up to take on Google
#128Do you guys remember Cuil? Man that was a whacky ass engine. Just gave you all sorts of comical results.
For anyone not around the hype around that thing was so over the top. It was a for-sure google killer. The first few days it barely loaded due to the load and when it did, yeah, not great results!
Re: Two upstart search engines are teaming up to take on Google
#129Earlier quoted context omitted.
How is it possible that a search for "Hacker News" produces only 65 results? There are thousands of pages out there with that exact phrase on it (including many sub-pages of this site). The first result is almost assuredly the right one, but either they're ruling out a lot of pages as not-what-you-meant, or their index is really small.
It's another feature of Kagi. They know they have thousands of results, but they provide you a single page of most relevant results . If you want to see more because you exhausted the page, there's a "more results" button at the bottom. Kagi reduces mental load by default, and this is a good thing.
That's an interesting positive spin on what is clearly a cost-cutting feature. I'm on the unlimited search tier, so it hasn't really been a big deal, but it's worth noting that clicking the "more results" button charges your account for an additional search too. Or at least it used to.
Re: Two upstart search engines are teaming up to take on Google
#130My hot take on this new era of search engines is that "search is a bug" and even trying to be a search engine is a fool's errand. Search solved a problem of the legacy internet where you wanted information and that information would be on one of a million websites. If someone is going to disrupt Google, it's because they've cut out the middleman that is search results and simply give you what you're asking for . Chat…
Search is still better for getting to specific, existing documents you need. Even the RAG people have been finding that out with hybrid models becoming more popular over time. I also think you can update search indexes more cheaply than further pretraining LLM’s.
Not to mention that the cost per search in terms of compute and energy is so much smaller for web search than for running an LLM. I forget the exact numbers now, but it was orders of magnitude as I recall.
Search engines are just cheaper to run. I don't know that there's a good, long term model for a free LLM-based search replacement because of how much higher the operating costs are, ad supported or not.