Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Building a web search engine from scratch with 3B neural embeddings

blog.wilsonl.in

Post reply on HN
Print view View original

91–100 of 124 posts

Re: Show HN: Building a web search engine from scratch with 3B neural embeddings

#91

Post by yorwba »

zipy124 wrote: post #85

The author claims "It should be far less susceptible to keyword spam and SEO tactics." however anyone with a cursory knowledge of the limitations of embeddings/LLM's knows the hardest part is that there is no seperation between the prompt and the content to be queried (e.g "ignore all previous instructions" etc...). It would not be hard to adversarially generate embeddings for SEO, in-fact it's almost easier since yo…

The author is using SBERT embeddings, not an instruction-following model, so the "ignore all previous instructions" trick isn't going to work, unless you want to outrank https://en.wikipedia.org/wiki/Ignore_all_rules when people search for what to do after ignoring all previous instructions.

Of course a spammer could try to include one sentence with a very close embedding for each query they want to rank for, but this would require combinatorially more effort than keyword stuffing where including two keywords also covers queries including both together.

Re: Show HN: Building a web search engine from scratch with 3B neural embeddings

#92

Post by kiririn »

noosphr wrote: post #47
divineg wrote: post #44

It's incredible. I can't believe it but it actually works quite nicely. If 10K $5 subscriptions can cover its cost, maybe a community run search engine funded through donations isn't that insane?

It's been clear to anyone familiar with encoder only LLMs that Google is effectively dead. The only reason why it still lives is that it takes a while to crawl the whole web and keep the index up to date. If someone like common crawl, or even a paid service, solves the crawling of the web in real time then the moat Google had for the last 25 years is dead and search is commoditized.

You can see their panic - in my country they are running TV ads for Google search, showing it answering LLM-prompt-like queries. They are desperately trying to win back that mind share, and if they lose traditional keyword search too they’re cooked

Re: Show HN: Building a web search engine from scratch with 3B neural embeddings

#93

Post by deepsquirrelnet »

> RocksDB and HNSW were sharded across 200 cores, 4 TB of RAM, and 82 TB of SSDs.

Was your experience with ivfpq not good? I’ve seen big recall drops compared to hnsw, but wow, takes some hardware to scale.

Also did you try sparse embeddings like SPLADE? I have no idea how they scale at this size, but seems like a good balance between keyword and semantic searches.

Re: Show HN: Building a web search engine from scratch with 3B neural embeddings

#94

Post by kdfjgbdfkjgb »

giancarlostoro wrote: post #7
andai wrote: post #5

Earlier quoted context omitted.

My understanding was that every few months Google was forced to adjust their algorithms because the search results would get flooded by people using black hat SEO techniques. At least that's the excuse I heard for why it got so much worse over time. Not sure if that's related to it ignoring quotes and operators though. I'd imagine that to be a cost saving measure (and very rarely used, considering it keeps accusing m…

That begs the question, if you can recreate their engine from the 2000s with high quality search results, would investors even fund you? Lol

Sorry to be pedantic but you mean "raises the question." https://en.wikipedia.org/wiki/Begging_the_question

Re: Show HN: Building a web search engine from scratch with 3B neural embeddings

#96

Post by supriyo-biswas »

lelanthran wrote: post #70
ww520 wrote: post #52

Mad respect. This is an incredible project to pull together all these technologies. The crown jewel of a search engine is its ranking algorithm. I'm not sure how LLM is being used in this regard in here. One effective old technique for ranking is to capture the search-to-click relationship by real users. It's basically the training data by human mapping the search terms they entered to the links they clicked. With ju…

> One effective old technique for ranking is to capture the search-to-click relationship by real users. It's basically the training data by human mapping the search terms they entered to the links they clicked. With just a few of clicks, the ranking relevance goes way up. That's not very effective. Ever heard of clickbait? Like I've said uncountable times before, the only effective technique to clean out the search r…

This is often touted as the solution to remove SEO garbage, except that you'd also get rid of the news websites along with it which are fairly reliant on advertising.

Re: Show HN: Building a web search engine from scratch with 3B neural embeddings

#98

Post by indoorcomic »

As I've become more advanced in my career I've grown more frustrated with search engines for the same problems you described in your write up. This is a fantastic solution and such a refreshing way to use LLMs. I hope this project goes far!

Re: Show HN: Building a web search engine from scratch with 3B neural embeddings

#99

Post by dpoloncsak »

cedws wrote: post #57
arkmm wrote: post #51

"There was one surprise when I revisited costs: OpenAI charges an unusually low $0.0001 / 1M tokens for batch inference on their latest embedding model. Even conservatively assuming I had 1 billion crawled pages, each with 1K tokens (abnormally long), it would only cost $100 to generate embeddings for all of them. By comparison, running my own inference, even with cheap Runpod spot GPUs, would cost on the order of 10…

I don’t think OpenAI train on data processed via the API, unless there’s an exception specifically for this.

Maybe I misunderstand, but I'm pretty sure they offer an option for cheaper API costs (or maybe its credits?) if you allow them to train on your API requests.

To your point, pretty sure it's off by default, though

Edit: From https://platform.openai.com/settings/organization/data-contr...

Share inputs and outputs with OpenAI

"Turn on sharing with OpenAI for inputs and outputs from your organization to help us develop and improve our services, including for improving and training our models. Only traffic sent after turning this setting on will be shared. You can change your settings at any time to disable sharing inputs and outputs."

And I am 'enrolled for complimentary daily tokens.'

Re: Show HN: Building a web search engine from scratch with 3B neural embeddings

#100

Post by nickpsecurity »

noosphr wrote: post #47
divineg wrote: post #44

It's incredible. I can't believe it but it actually works quite nicely. If 10K $5 subscriptions can cover its cost, maybe a community run search engine funded through donations isn't that insane?

It's been clear to anyone familiar with encoder only LLMs that Google is effectively dead. The only reason why it still lives is that it takes a while to crawl the whole web and keep the index up to date. If someone like common crawl, or even a paid service, solves the crawling of the web in real time then the moat Google had for the last 25 years is dead and search is commoditized.

It's not dead but will take a huge hit. I still use DuckDuckGo since I get good answers, good discovery, taken right to the sources (whom I can cite), and the search indexes are legal vs all the copyright infringement in AI training.

If AI training becomes totally legal, I will definitely start using them more in place of or to supplement search. Right now, I don't even use the AI answers.

Post reply on HN