Live data from Hacker News

Compare Google, Bing, Marginalia, Kagi, Mwmbl, and ChatGPT

danluu.com

161–170 of 493 posts

Re: Compare Google, Bing, Marginalia, Kagi, Mwmbl, and ChatGPT

#161

Earlier quoted context omitted.

Thanks! If you are that "single person" who built Marginalia... hope you are not taking my criticism personally. I am more annoyed by this blog post that uses a few handpicked queries to present generalized long winded conclusions that are completely disproven when using a different set of queries.

Yeah, its me, and to be fair I made a comment to a similar effect myself. Assessing search result quality is very hard, and this is definitely a pretty flattering selection of queries.

On the plus side - in addition to Marginalia's own success, you can take partial credit for how good Kagi search results are (IIRC Marginalia's index is one of the sources for Kagi search results). So... thank you for that!

Re: Compare Google, Bing, Marginalia, Kagi, Mwmbl, and ChatGPT

#162
Pretty biased selection of queries. Article avoids the things that ChatGPT and the others without fresh data can't answer. Look at the trending searches on Google. They are all for fresh info that none of the others can answer. Sports scores. Google probably judges quality weighted by the questions their users actually ask, not this nerd bullshit.

Re: Compare Google, Bing, Marginalia, Kagi, Mwmbl, and ChatGPT

#163

Blah blah blah. Could you lay this article out any worse? What are the queries you used to test? I want to try them too. Buried in here somewhere. Using an adblocker is not expert anything. That you've defined your own opinion for what some of the results should be blows the thing up. Searching youtube downloader, many people would be fine with some of the ad covered but totally functional sites that pop up on Google…

For whatever it's worth, I think your comment would be a whole lot more convincing without its first and last lines, which had the effect of making you sound (at least to me) like you're shallowly dismissing the article.

Re: Compare Google, Bing, Marginalia, Kagi, Mwmbl, and ChatGPT

#164
What's most shocking to me is how much malware there is in all of this. The fact that Google et al aren't constantly in trouble for directly forwarding unwitting users to malware distributors indicates to me just how far our standards have fallen for a "good" search engine. I feel like we'd be happier with search engines that adhered to "first, do no harm" principles.

Re: Compare Google, Bing, Marginalia, Kagi, Mwmbl, and ChatGPT

#165
I'm not able to reproduce the author's bad results in Kagi, at all. What I'm seeing when searching the same terms is fantastic in comparison. I don't know what went wrong there.

In the Youtube Downloader search, NortonSafeWeb is nowhere to be found. I get a couple of legit downloader websites, and some articles from reputable tech newspapers on how to use them or command line tools.

In the Adblock search, ublock Origin is #3, followed by some blogs about ad blocking ethics debates and the bullshit Google has been pulling recently.

In the wider tires grip search, #3 is a physics blog that dives deep into the topic.

In the transistors search, the first reddit link directly answers the question in very similar wording to the hypothetical correct answer spelled out in the rubric. 4/5 of the reddit results are on the correct topic, followed by two SuperUser questinos also on the correct topic, then some linus tech tips and toms hardware articles, also on the correct topic. No Quora questions.

In the vancouver winter snow search, the first several results are from local news papers talking about the anticipated effects of el nino on snowfall, and then a couple of high-quality blogs and weather sites.

Really wondering how Dan got such bad results.

------

Aside from that, the way that the author expects all the results to return the same kind of thing is just... weird? Like, that's not how search engines are supposed to work. A search that gives you 10 links to fundamentally the same thing is a bad search. Search results should cover a breadth of reasonable guesses for what you should be looking for given a query. If you search for "download firefox", and you scroll past the first 5 download links, then you're probably not actually looking for a download link and a blog post about firefox is not "irrelevant" and shouldn't be points against.

This opinion is even borne out in search engine quality metrics that have been industry-standard for decades, like mean reciprocal rank and distributed cumulative gain. What matters is how far you have to scroll to get to a good result, not what proportion of the first N results are good.

Re: Compare Google, Bing, Marginalia, Kagi, Mwmbl, and ChatGPT

#166

I really don't understand why anyone writing articles about ChatGPT uses 3.5. It's pretty misleading as to the results you can get out of (the best available version of) ChatGPT. For comparison, here are all the author's questions posed against GPT4: https://chat.openai.com/share/ed8695cf-132e-45f3-ad27-600da7...

Why does OpenAI continue to offer chatgpt 3.5 if it's so bad?

Chatgpt3.5 is good enought if can give context in query.

Re: Compare Google, Bing, Marginalia, Kagi, Mwmbl, and ChatGPT

#167

I'm not able to reproduce the author's bad results in Kagi, at all. What I'm seeing when searching the same terms is fantastic in comparison. I don't know what went wrong there. In the Youtube Downloader search, NortonSafeWeb is nowhere to be found. I get a couple of legit downloader websites, and some articles from reputable tech newspapers on how to use them or command line tools. In the Adblock search, ublock Orig…

have you customized your results and lowered or raised many domains?

Re: Compare Google, Bing, Marginalia, Kagi, Mwmbl, and ChatGPT

#168
post #133

I think the result grading is too opinionated here. For example, the first query is "download YouTube videos", for which Google is ranked "terrible" for not showing you a command line open source program. But the literal first result is an ad supported site where I can paste in a YouTube link and download it right from the browser. That seems like exactly what most people would want or to the CLI tool the author is s…

Yeah, if one typed “YouTube downloaded cli” you the results the author was thinking.

It seems like the author wants to search to read their kind without specifying what kind of YouTube downloaded they want

Re: Compare Google, Bing, Marginalia, Kagi, Mwmbl, and ChatGPT

#170
post #133

I think the result grading is too opinionated here. For example, the first query is "download YouTube videos", for which Google is ranked "terrible" for not showing you a command line open source program. But the literal first result is an ad supported site where I can paste in a YouTube link and download it right from the browser. That seems like exactly what most people would want or to the CLI tool the author is s…

[deleted]
Post reply on HN