Live data from Hacker News

Two upstart search engines are teaming up to take on Google

wired.com

111–120 of 299 posts

Re: Two upstart search engines are teaming up to take on Google

#111
post #33
post #12

Earlier quoted context omitted.

Brave search is doing it properly. Bought tech, but still.

very recently found about brave goggles. amazing way to give control to users, for ex: blocking pinterest or searching domains popular with HN or own list. https://search.brave.com/goggles/discover

Did not know about this, this is what I've wanted Google to do for so long. Pinterest, etc. will be banished to the nether realm.

Re: Two upstart search engines are teaming up to take on Google

#112
post #51

Earlier quoted context omitted.

>competing with Google was borderline impossible a decade ago. But in 2024, we have cheap compute, great OSS distributed DBs, powerful new vector search tech. [...] CommonCrawl text-only is ~100TB, Those example 3 bullet points of today's improved 2024 computing power you list isn't even enough to process Google's scale 14 years ago in 2010 when the search index was 100+ petabytes : https://googleblog.blogspot.com/20…

Just serving up content from Reddit and HN and a few other websites would be enough to beat Google for most of us. Sprinkle in the top 100 websites and you have a legitimate contender. There is no open web anymore. Google killed it. There are probably fewer than 100k useful websites in the world now. Which is good for startups, because the problem is entirely tractable.

It's all very regional. Despite its namesake, the world-wide-web is aggressively local. Some properties are global but after a handful, it's all country/language/region based.

No matter what type of market analysis I do, I almost invariably find there's something different that say, the Koreans or the Europeans are using. The Yelp of Japan is Tabelog, the Ubereats of the UK is deliveroo, the facebook of russia is vk.ru etc.

That's really the beach head to capture - figure out what a "web region" is for a number of query use-cases and break in there.

Re: Two upstart search engines are teaming up to take on Google

#113
post #44
post #22

It shouldn't be too hard to achieve what Google were good at before. Their recent search results for me (last 6-12 months) have been so far removed from what I'm searching it felt like a meme. Even after rephrasing things, more details, special quotations etc that everyone knows as the 'search tricks' the results are terrible.

I moved to DDG a couple of years ago, and initially, I found myself often using the `!g` switch, but I honestly can't recall the last time I needed to do that. Only when I'm shopping do I find the goog to be a slightly better tool for finding products sold by niche suppliers. I honestly think google's monopoly on search at this time is 100% powered by momentum, there is almost no other reason to use it over something…

Same here. DDG usually works fine for me. Even more so if using advanced techniques like quotes for keywords and - to remove junk.

Re: Two upstart search engines are teaming up to take on Google

#114
post #102

My hot take on this new era of search engines is that "search is a bug" and even trying to be a search engine is a fool's errand. Search solved a problem of the legacy internet where you wanted information and that information would be on one of a million websites. If someone is going to disrupt Google, it's because they've cut out the middleman that is search results and simply give you what you're asking for . Chat…

Search is still better for getting to specific, existing documents you need. Even the RAG people have been finding that out with hybrid models becoming more popular over time. I also think you can update search indexes more cheaply than further pretraining LLM’s.

Re: Two upstart search engines are teaming up to take on Google

#115
post #87

Earlier quoted context omitted.

You'd be surprised how long it takes to enshittify a piece of tech as well established as Google. The MBAs may be trying but there are still a lot of dedicated folks deep in the org holding out.

It's funny. I usually can't tell that from the quality of the search results.

Compared to what?

Re: Two upstart search engines are teaming up to take on Google

#116

Earlier quoted context omitted.

This reads like the “I could build Dropbox in a weekend”.

It's just files, how hard could it be?

No that hard, took me 4 weekends to build a private search engine with Common Crawl, Wikipedia and HN as a link authority source. Takes about a week to crunch the data on an old Lenovo workstation with 256gb ram and some storage.

Re: Two upstart search engines are teaming up to take on Google

#117

Earlier quoted context omitted.

Because it is super expensive and difficult to keep an index up to date. People expect to be able to get current events, and expect search results to be updated in minutes/seconds.

No search engine is refreshing every website every minute. Most websites don't update frequently, and if you poll them more than once every month, your crawler will get blocked incredibly fast. The problem of being able to provide fresh results is best solved by having different tiers of indices, one for frequently updating content, and one for slowly updating content with a weekly or monthly cadence. You can get a l…

See also: IndexNow [1], a protocol used by Bing, Naver, Yandex, Seznam, and Yep where sites can ping one of these search engines when a page is updated and all others will be immediately notified. Unfortunately it does seem somewhat closed as to requirements for joining as a search engine.

[1]: https://www.indexnow.org/

Re: Two upstart search engines are teaming up to take on Google

#118

For me, it seems the direction for search is going towards AI sites. (Gemini, ChatGPT) Trying to reinvent Google/ Search in 2024 seems a bit like jumping the shark

Quite the opposite. The part of the crowd that has site:old.reddit.com in their muscle memory has to be the premium end of the search market. Sure, garbage tier searches will be done LLM style. But few smart people might be bored with that, and pay for something better.

The audacity of power users assuming they're the majority

Re: Two upstart search engines are teaming up to take on Google

#119

For me, it seems the direction for search is going towards AI sites. (Gemini, ChatGPT) Trying to reinvent Google/ Search in 2024 seems a bit like jumping the shark

I want to remake Yahoo!

I want to browse a catalog of interesting web content.

Like walking down the isle of an old fashioned newsagent, flicking through the magazines that stand out.

Re: Two upstart search engines are teaming up to take on Google

#120

For me, it seems the direction for search is going towards AI sites. (Gemini, ChatGPT) Trying to reinvent Google/ Search in 2024 seems a bit like jumping the shark

I agree, and I think it's half and half for me. Many times when I search, my query ends with a question mark. They're questions, looking for a simple answer. Those are the searches I have been going to LLMs for more and more lately. As the LLMs get better and hallucinate less, this is becoming more and more viable. Other searches are looking for a page. Where to buy a particular component. Where to download a particular library. Things like that. For that, Google is still useful. The thing is, I think the first kind of search makes up a huge amount of Google's business. I would hazard a guess that it actually makes up more than half of their search traffic, and the loss of that kind of query is going to seriously damage them. There's also nothing stopping LLMs from eventually giving us answers to the second kind of query either, as they start being able to ingest and incorporate real time data. To me, it seems inevitable that LLMs will kill Google's main source of revenue (search is 57%) and eventually their entire company, as much of the rest of what they do is subsidized by search. They may be too large at this point to adjust to this extinction-level change to the environment.
Post reply on HN