[flagged]
testing? I've been using it for almost 3 years already...
Two upstart search engines are teaming up to take on Google
31–40 of 299 posts
Re: Two upstart search engines are teaming up to take on Google
#32> Ask the search engine Ecosia about “Paris to Prague” and flight booking websites dominate the results. Ecosia’s CEO Christian Kroll would prefer to present more train options, which he considers better for the environment. But because its results are licensed from Google and Microsoft’s Bing, Ecosia has little control over what’s shown. Kroll is ready for that to change.
While I think Google sucks right now and we need something new, this specific reason is so dumb. Unless I add the word "train" or "air" etc. I would much rather be shown either all options or the one that I care about most (if it's flying, then so be it - the search engine can't and shouldn't try to filter out options FOR me without my consent)
Re: Two upstart search engines are teaming up to take on Google
#33Every new search engine I've seen was a Bing wrapper with sometimes light reranking. I understand that competing with Google was borderline impossible a decade ago. But in 2024, we have cheap compute, great OSS distributed DBs, powerful new vector search tech. Amateur search engines like Marginalia even run on consumer hardware. CommonCrawl text-only is ~100TB, and can fit on my home server. Why is no company buildin…
Brave search is doing it properly. Bought tech, but still.
very recently found about brave goggles.
amazing way to give control to users, for ex: blocking pinterest or searching domains popular with HN or own list.
Re: Two upstart search engines are teaming up to take on Google
#34It shouldn't be too hard to achieve what Google were good at before. Their recent search results for me (last 6-12 months) have been so far removed from what I'm searching it felt like a meme. Even after rephrasing things, more details, special quotations etc that everyone knows as the 'search tricks' the results are terrible.
I'm using Kagi for quite some time. It's invisible to me. I search, get 30ish high quality results per search, and I'm a happy camper. No ads, no seo grafting, nothing. Moreover, I can block sites and customize my own search results. This feels good. When I first started using Kagi, it felt like leaving a closed building and stepping out to open air.
Almost same here:
One of the few software tools I care to pay for except Jetbrains.
Only in practice I almost never use filter or block features, because Kagi does out-of-the-box what we always wanted to do our selves in Google: block spam sites.
The ranking also seems to be better for some reason somehow.
The funny thing is it doesn't feel like a step forward, but rather like a step back to Google ca 2009 - 2012 somewhere.
Re: Two upstart search engines are teaming up to take on Google
#35Every new search engine I've seen was a Bing wrapper with sometimes light reranking. I understand that competing with Google was borderline impossible a decade ago. But in 2024, we have cheap compute, great OSS distributed DBs, powerful new vector search tech. Amateur search engines like Marginalia even run on consumer hardware. CommonCrawl text-only is ~100TB, and can fit on my home server. Why is no company buildin…
Because nowdays more than ever content you need is in silos.
Your facebooks/twiters/instagram/stack overflow/reddit ... And they all have limited expensive api's, and have bulk scrapping detection. Sure you can clobber together something that will work for a while, but you can't runn a buissness on that.
Aditionaly most paywalled sites (like news) explicitly whitlist google and bing, and if someone cretes new site, they do the same. As an upstart you would have to reach out to them to get them to whitelist you. and you would need to do it not only in USA but globaly.
Anothe problem is cloudflare and other cdns/web firewalls, so even trying to index mom and pops blog site could be problematic. An d most of the mom and pop blogs are nowdays on som ploging platform that is just another silo.
Now that i think about it, cloudflare might be in a good position to do it.
The AI hype and scraping for content to feed the models have increased dificulty for anyone new to start new index.
Re: Two upstart search engines are teaming up to take on Google
#36Earlier quoted context omitted.
It remains a competitor as long as it continues to capture attention (eye-balls), even if its usability has diminished.
It remains a conmpetitor as long as it continues to be the default search engine in at least two of the most important mainsteam web browsers.
I believe your answer is a subset of what I just said?
Re: Two upstart search engines are teaming up to take on Google
#37Why isn’t there a distributed, decentralized or open index that all of these startups can utilize? I understand that these startups are all are focusing in on different problem areas, but doesn’t it make sense to have something like open street maps so that all of these companies can share their compute resources in order to maintain something competitive with the big guys? Or even if it’s not fully decentralized these startups teaming up to build a bigger index for themselves makes a lot of sense to me.
I have no knowledge of this field but something like that would seem seem to make sense.
Re: Two upstart search engines are teaming up to take on Google
#38Every new search engine I've seen was a Bing wrapper with sometimes light reranking. I understand that competing with Google was borderline impossible a decade ago. But in 2024, we have cheap compute, great OSS distributed DBs, powerful new vector search tech. Amateur search engines like Marginalia even run on consumer hardware. CommonCrawl text-only is ~100TB, and can fit on my home server. Why is no company buildin…
Because it is super expensive and difficult to keep an index up to date. People expect to be able to get current events, and expect search results to be updated in minutes/seconds.
No search engine is refreshing every website every minute. Most websites don't update frequently, and if you poll them more than once every month, your crawler will get blocked incredibly fast.
The problem of being able to provide fresh results is best solved by having different tiers of indices, one for frequently updating content, and one for slowly updating content with a weekly or monthly cadence.
You can get a long way by driving he frequently updating index via RSS feeds and social media firehoses to provide singnals for when to fetch new URLs.
Re: Two upstart search engines are teaming up to take on Google
#39Earlier quoted context omitted.
Some sources update faster than others, you could index news sources hourly and low velocity sites weekly. Google does that. CommonCrawl gets 7TB/month, indexing and vectorizing that is quite manageable.
This reads like the “I could build Dropbox in a weekend”.
Haha yes, but my argument is not about individuals, but about "tech" companies that externalize everything and do not develop tech internally.
Re: Two upstart search engines are teaming up to take on Google
#40Earlier quoted context omitted.
Some sources update faster than others, you could index news sources hourly and low velocity sites weekly. Google does that. CommonCrawl gets 7TB/month, indexing and vectorizing that is quite manageable.
This reads like the “I could build Dropbox in a weekend”.
It's just files, how hard could it be?