how much did it cost ?
Show HN: Building a web search engine from scratch with 3B neural embeddings
41–50 of 124 posts
Re: Show HN: Building a web search engine from scratch with 3B neural embeddings
#42This is really well written, especially considering the complexity
Re: Show HN: Building a web search engine from scratch with 3B neural embeddings
#43Thank you for sharing! This is one of the coolest articles I have seen in a while on HN. I did some searches and I think the search results looked very useful so far. I particularly loved about your article that most of the questions I had while reading got answered in a most structured way.
I still have questions:
* How long do you plan to keep the live demo up?
* Are you planning to make the source code public?
* How many hours in total did you invest into this "hobby project" in the two months you mentioned in your write-up?
Re: Show HN: Building a web search engine from scratch with 3B neural embeddings
#44It's incredible. I can't believe it but it actually works quite nicely.
If 10K $5 subscriptions can cover its cost, maybe a community run search engine funded through donations isn't that insane?
Re: Show HN: Building a web search engine from scratch with 3B neural embeddings
#45This is really well written, especially considering the complexity
I also love how often the author finds that traditionally glazed databases don't scale as you might think, and turn to stronger storage primitves that were designed exactly to do that
Re: Show HN: Building a web search engine from scratch with 3B neural embeddings
#46Very cool project! Just out of interest, I sent a query I've had difficulties getting good results for with major engines: "what are some good options for high-resolution ultrawide monitors?". The response in this engine for this query at this point seems to have the same fallacy as I've seen in other engines. Meta-pages "specialising" in broad rankings are preferred above specialist data about the specific sought-af…
You could argue that it is not really a search query. There is not a particular page that answers the question “correctly”, it requires collating multiple sources and reasoning. That is not a search problem.
Re: Show HN: Building a web search engine from scratch with 3B neural embeddings
#47It's incredible. I can't believe it but it actually works quite nicely. If 10K $5 subscriptions can cover its cost, maybe a community run search engine funded through donations isn't that insane?
It's been clear to anyone familiar with encoder only LLMs that Google is effectively dead. The only reason why it still lives is that it takes a while to crawl the whole web and keep the index up to date.
If someone like common crawl, or even a paid service, solves the crawling of the web in real time then the moat Google had for the last 25 years is dead and search is commoditized.
Re: Show HN: Building a web search engine from scratch with 3B neural embeddings
#48This is really well written, especially considering the complexity
I also love how often the author finds that traditionally glazed databases don't scale as you might think, and turn to stronger storage primitves that were designed exactly to do that
i knew i've seen this before, they clearly migrated their blog as all posts are uploaded on the 10th, and i already have multiple mentioned repos starred lol
Re: Show HN: Building a web search engine from scratch with 3B neural embeddings
#49A vector-only search engine will fail for a lot of common use cases where the keywords do matter. I tried searching for `garbanzo bean stew` and got totally irrelevant bean recipes.
Re: Show HN: Building a web search engine from scratch with 3B neural embeddings
#50A vector-only search engine will fail for a lot of common use cases where the keywords do matter. I tried searching for `garbanzo bean stew` and got totally irrelevant bean recipes.
What if you build a graph engine then encode those edges into its own embedding space?
Nerdsnipe?