Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Compare Google, Bing, Marginalia, Kagi, Mwmbl, and ChatGPT

danluu.com

Post reply on HN
Print view View original

291–300 of 493 posts

Re: Compare Google, Bing, Marginalia, Kagi, Mwmbl, and ChatGPT

#291

Post by beezle »

bambax wrote: post #207

I'm in the camp of those who think Google's results are still very good. I admit I use adblock (uBlock Origin) and won't even try to disable it. I understand the author's point of turning off their ad blocker "to get the non-expert browsing experience" but then they could make a different test with uBlock on for every query and see how it goes. It's also a bit inconsistent to expect results for downloading videos men…

Put me in the camp of google and the rest are horrible for all but very specific/unique technical terms, ie weak neutral currents. Anything that is more "everyday life" is an exercise in futility sorting through trash, often without even the terms you are looking for. And good luck with "verbatim" searches - either ignored or zero results.

Re: Compare Google, Bing, Marginalia, Kagi, Mwmbl, and ChatGPT

#292

Post by the__alchemist »

hannasanarion wrote: post #165

I'm not able to reproduce the author's bad results in Kagi, at all. What I'm seeing when searching the same terms is fantastic in comparison. I don't know what went wrong there. In the Youtube Downloader search, NortonSafeWeb is nowhere to be found. I get a couple of legit downloader websites, and some articles from reputable tech newspapers on how to use them or command line tools. In the Adblock search, ublock Orig…

I use Kagi because I'm trying to remove Google from my life, but their text search is worse than Google in my experience, and the image search is abysmal. I'm wondering how long I can keep this up. I already revert to Google for image search, and am finding myself using either Google or ChatGPT over Kagi more and more for text as well.

Re: Compare Google, Bing, Marginalia, Kagi, Mwmbl, and ChatGPT

#293

Post by Jcowell »

Majromax wrote: post #285
sanderjd wrote: post #280

Earlier quoted context omitted.

I'm curious: what is the rationale for "in an incognito tab" being part of the test harness? It seems pretty arbitrary to me to disable one of the key features - in this case personalization - of the software being evaluated. Or is the evaluation not between "search engines" but rather "search engines without personalization"? If so, then this restriction does make sense. But that is not the evaluation that "normal u…

> I'm curious: what is the rationale for "in an incognito tab" being part of the test harness? It's the closest we can easily get to the 'average user experience'. Someone who has a long account/cookie history with Google has plausibly trained the site to return more relevant results through implicit user-curation of avoiding obvious-to-them SEO-spam on other queries. If we posit that every user eventually trains Goo…

> It's the closest we can easily get to the 'average user experience'

You wouldn’t be really taking the average here though would you? You would be capture the experience someone might have if they were in incognito, using google for the very first time, or using google on another device for the every first time, but not the “average experience”.

Re: Compare Google, Bing, Marginalia, Kagi, Mwmbl, and ChatGPT

#294

Post by ForkMeOnTinder »

anonymoushn wrote: post #258
haizhung wrote: post #256

What always confuses me about the „search has gotten so bad“ mentality is that it is often based on anecdotal evidence at best, and anecdotal recollection at worst. Like, sure, I have the impression that search got worse over the last years, but .. has it really? How could you tell? And, honestly, this should be a verifiable claim; you can just try the top N search terms from Google trends or whatever and see how the…

Probably for the same reason that there are so many more posts about anything that make claims than that explore evidence systematically, especially when the people making the posts stand to gain nothing by spending their time that way. I encounter claims that "protobuf is faster than json" pretty regularly but it seems like nobody has actually benchmarked this. Typical protobuf decoder benchmarks say that protobuf d…

The problem with benchmarking that claim is there's no one true "json decoder" that everyone uses. You choose one based on your language -- JSON.stringify if you're using JS, serde_json if you're using Rust, etc.

So what people are actually saying is, a typical protobuf implementation decodes faster than a typical JSON implementation for a typical serialized object -- and that's true in my experience.

Tying this back into the thread topic of search engine results, I googled "protobuf json benchmark" and the first result is this Golang benchmark which seems relevant. https://shijuvar.medium.com/benchmarking-protocol-buffers-js... Results for specific languages like "rust protobuf json benchmark" also look nice and relevant, but I'm not gonna click on all these links to verify.

In my experience programming searches tend to get much better results than other types of searches, so I think the article's claim still holds.

Re: Compare Google, Bing, Marginalia, Kagi, Mwmbl, and ChatGPT

#295

Post by erybodyknows »

teleforce wrote: post #236
bambax wrote: post #207

I'm in the camp of those who think Google's results are still very good. I admit I use adblock (uBlock Origin) and won't even try to disable it. I understand the author's point of turning off their ad blocker "to get the non-expert browsing experience" but then they could make a different test with uBlock on for every query and see how it goes. It's also a bit inconsistent to expect results for downloading videos men…

I'm also in the same camp who think search results from Google is very good but ChatGPT based search with RAG is better, granted it's a paid version. The latter however is kind of experimental, personally would love to have another column on ChatGPT with RAG (Bing) and the fact the author ignored RAG is rather strange.

For those (like me) wondering what RAG means: “Retrieval Augmented Generation (RAG) represents a groundbreaking approach in information retrieval, where the accuracy of search results directly influences the quality of generated answers. In essence, RAG combines traditional search mechanisms with Large Language Model's ability to understand and generate answers.”

(https://www.linkedin.com/pulse/how-we-increased-search-accur....)

Re: Compare Google, Bing, Marginalia, Kagi, Mwmbl, and ChatGPT

#296

Post by Semaphor »

freediver wrote: post #286
Semaphor wrote: post #189

Earlier quoted context omitted.

With a better prompt, you can get it to list some, but it’s very annoying to do so. Mistral showed that their medium model is far better (yet not good), and the same prompt as in the article gives only one instead of 3 paragraphs of rambling about copyright, and then lists 3 categories of options with examples for each (not good, because ytdl is not one of those listed). Funnily enough, both mistral and GPT4 apologiz…

> Funnily enough, both mistral and GPT4 apologize profoundly and almost with the same wording when asked "Why did you not mention the very popular, free and open source "youtube-dl" software?" Likely because they were optimized for general population, which would not have a use for command line python utility.

I’m clear why they didn’t include it, I wanted them to tell me why, though. And I thought that both of them apologized in almost the same way, was funny.

Re: Compare Google, Bing, Marginalia, Kagi, Mwmbl, and ChatGPT

#297

Post by anonymoushn »

ForkMeOnTinder wrote: post #294
anonymoushn wrote: post #258

Earlier quoted context omitted.

Probably for the same reason that there are so many more posts about anything that make claims than that explore evidence systematically, especially when the people making the posts stand to gain nothing by spending their time that way. I encounter claims that "protobuf is faster than json" pretty regularly but it seems like nobody has actually benchmarked this. Typical protobuf decoder benchmarks say that protobuf d…

The problem with benchmarking that claim is there's no one true "json decoder" that everyone uses. You choose one based on your language -- JSON.stringify if you're using JS, serde_json if you're using Rust, etc. So what people are actually saying is, a typical protobuf implementation decodes faster than a typical JSON implementation for a typical serialized object -- and that's true in my experience. Tying this back…

I agree. You wouldn't use encoding/json or serde-json if you had to deserialize a lot of json and you cared about latency, throughput, or power costs. A typical protobuf decoder would be better.

Re: Compare Google, Bing, Marginalia, Kagi, Mwmbl, and ChatGPT

#298

Post by Semaphor »

mrweasel wrote: post #214
Semaphor wrote: post #204

Earlier quoted context omitted.

Kagi mixes google, bing, some non-profit small-web SE, and their own index.

I don't think they use Bing, but yes, Google, Marginalia, Yandex, Brave and others. I still fail to see how that's different to DuckDuckGo, who also run their own crawler. It's really weird that people are almost hating on DuckDuckGo for how they run their search engine, while applauding Kagi, for doing the same, but with a different business model.

Only if they changed that (which they might have as part of their cost-optimization). They said they mixed bing and google results back then.

Re: Compare Google, Bing, Marginalia, Kagi, Mwmbl, and ChatGPT

#299

Post by whakim »

sanderjd wrote: post #283
marginalia_nu wrote: post #260

Earlier quoted context omitted.

I think the point he's trying to make that the search results page from the mainstream search engines are a minefield of scams that a regular person would have difficulty navigating safely. If he was looking at relevance, yours would be a solid point, but since most of the emphasis is on harm, a smaller sample works. Like "we found used needles in 3 out of 5 playgrounds" doesn't typically garner requests for p-values…

I think this is a good illustration of my frustration with this discussion: I don't think search has gotten bad, I think the web has gotten bad. It's weird to even conceptualize it as a big graph of useful hypertext documents. That's just wikipedia. The broader web is this much noisier and dubious thing now. That's bad for google though! Their model is very much predicated on the web having a lot of signal that they…

But there's still plenty of signal. It isn't as if there are no working YouTube downloaders, or factually correct explanations of how transistors work. It's just that search engines don't know how to (or don't care enough about) disambiguating these good results from the mountains of spam or malware.

Re: Compare Google, Bing, Marginalia, Kagi, Mwmbl, and ChatGPT

#300

Post by treetalker »

marginalia_nu wrote: post #245
golol wrote: post #179

Earlier quoted context omitted.

Well that makes sense, but I wanted to push against the result that the OP seems to take away from their test, which was that Marginalia seems to work well for the common user.

There's also a known bug with Wikipedia in particular, I do index it but the results are never ranked particularly high. I haven't fixed it because I don't want Wikipedia to be the #1 result for every search. Feels like most people are aware of Wikipedia and don't need help finding it.

Thanks for your work!

I have a suggestion for the “About” section at the top of Marginalia’s landing page. I think it would read better like this:

> This is an independent DIY search engine that focuses on non-commercial content, and attempts to show you sites you perhaps weren't aware of [instead] of the sort of sites you probably already knew existed.

Showing one thing “in favor of” another seems contradictory in this case.

Post reply on HN