Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Compare Google, Bing, Marginalia, Kagi, Mwmbl, and ChatGPT

danluu.com

Post reply on HN
Print view View original

281–290 of 493 posts

Re: Compare Google, Bing, Marginalia, Kagi, Mwmbl, and ChatGPT

#281

Post by latexr »

johnfn wrote: post #17

I noticed that the author uses ChatGPT3.5 rather than 4, which is a rather large difference. I don't have the knowledge to rerank all questions the author asked, but I will say that a test of ChatGPT 4 leads me directly to youtube-dl, which is better than every other search engine listed.

> I will say that a test of ChatGPT 4 leads me directly to youtube-dl

And yet to other people it starts rambling about how that’s wrong and you shouldn’t do it and doesn’t give a usable answer.

https://news.ycombinator.com/item?id=38822040

I boggles the mind the extent to which people salivate over a system that cannot decide between a correct straight answer, something wrong but plausible, something wrong and impossible, or outright refusing to answer.

Re: Compare Google, Bing, Marginalia, Kagi, Mwmbl, and ChatGPT

#282

Post by sundalia »

The intro query "youtube downloader" already showed me relevant results (some website where you paste an URL and bam download). I think there's a big tech bias in the whole post (how relevant is a mastodon poll, for real).

Not saying the current landscape doesn't suck with ads everywhere and incentives to not give exactly relevant results at times, but I think google is pretty good still.

Re: Compare Google, Bing, Marginalia, Kagi, Mwmbl, and ChatGPT

#283

Post by sanderjd »

marginalia_nu wrote: post #260
haizhung wrote: post #256

What always confuses me about the „search has gotten so bad“ mentality is that it is often based on anecdotal evidence at best, and anecdotal recollection at worst. Like, sure, I have the impression that search got worse over the last years, but .. has it really? How could you tell? And, honestly, this should be a verifiable claim; you can just try the top N search terms from Google trends or whatever and see how the…

I think the point he's trying to make that the search results page from the mainstream search engines are a minefield of scams that a regular person would have difficulty navigating safely. If he was looking at relevance, yours would be a solid point, but since most of the emphasis is on harm, a smaller sample works. Like "we found used needles in 3 out of 5 playgrounds" doesn't typically garner requests for p-values…

I think this is a good illustration of my frustration with this discussion: I don't think search has gotten bad, I think the web has gotten bad. It's weird to even conceptualize it as a big graph of useful hypertext documents. That's just wikipedia. The broader web is this much noisier and dubious thing now.

That's bad for google though! Their model is very much predicated on the web having a lot of signal that they can find within the noise. But if it just ... doesn't actually have much signal, then what?

Re: Compare Google, Bing, Marginalia, Kagi, Mwmbl, and ChatGPT

#284

Post by hyperpape »

marginalia_nu wrote: post #260
haizhung wrote: post #256

What always confuses me about the „search has gotten so bad“ mentality is that it is often based on anecdotal evidence at best, and anecdotal recollection at worst. Like, sure, I have the impression that search got worse over the last years, but .. has it really? How could you tell? And, honestly, this should be a verifiable claim; you can just try the top N search terms from Google trends or whatever and see how the…

I think the point he's trying to make that the search results page from the mainstream search engines are a minefield of scams that a regular person would have difficulty navigating safely. If he was looking at relevance, yours would be a solid point, but since most of the emphasis is on harm, a smaller sample works. Like "we found used needles in 3 out of 5 playgrounds" doesn't typically garner requests for p-values…

I agree we can say "this is a minefield of scams" without doing a comparison.

There still is a question about when it got bad--I think Dan mentions 2016 as a point of comparison, and there were plenty of scams back then, so you might wonder whether the days when a query wouldn't return many scams.

If you go back far enough, then there wasn't the same kind of SEO, and Internet scams were much smaller/less organized, but that's a long time ago.

Re: Compare Google, Bing, Marginalia, Kagi, Mwmbl, and ChatGPT

#285

Post by Majromax »

sanderjd wrote: post #280
gkbrk wrote: post #225

Earlier quoted context omitted.

When I do a Google search in an Incognito tab for "how to download youtube videos", the first two results I get are the following. - https://msunduziassociation.online/perfect-online-videos/ - https://gssaction.org/program-all-in-one-media-solutions/ I would certainly put those in the "Terrible" category like the author.

I'm curious: what is the rationale for "in an incognito tab" being part of the test harness? It seems pretty arbitrary to me to disable one of the key features - in this case personalization - of the software being evaluated. Or is the evaluation not between "search engines" but rather "search engines without personalization"? If so, then this restriction does make sense. But that is not the evaluation that "normal u…

> I'm curious: what is the rationale for "in an incognito tab" being part of the test harness?

It's the closest we can easily get to the 'average user experience'. Someone who has a long account/cookie history with Google has plausibly trained the site to return more relevant results through implicit user-curation of avoiding obvious-to-them SEO-spam on other queries.

If we posit that every user eventually trains Google to avoid SEO spam, then this begs the question of why Google(/Bing) don't eliminate the SEO spam in the first place.

Besides that, it's not obvious why search engine personalization should dramatically change the basic utility of search results. We should expect personalization to mostly address ambiguities: is 'the best way to set up tables' asking about furniture assembly/carpentry or SQL? None of the author's queries for this article supported such ambiguities, and besides that the results returned (see the final appendix) aren't[†] valid answers to a different interpretation of the question.

[†] -- I think I'd quibble about the 'adblock' question, since a reasonable person might still find an adblocker that works but participates in the 'acceptable ads program' to be sufficient.

Re: Compare Google, Bing, Marginalia, Kagi, Mwmbl, and ChatGPT

#286

Post by freediver »

Semaphor wrote: post #189
spaceman_2020 wrote: post #173

Earlier quoted context omitted.

I asked GPT-4 about Youtube Downloader and it rambled on about how downloading videos is against Youtube’s TOS and I should buy YouTube premium which has the download feature. Getting any useful data from GPT-4 about anything even remotely “illegal” is a waste of time.

With a better prompt, you can get it to list some, but it’s very annoying to do so. Mistral showed that their medium model is far better (yet not good), and the same prompt as in the article gives only one instead of 3 paragraphs of rambling about copyright, and then lists 3 categories of options with examples for each (not good, because ytdl is not one of those listed). Funnily enough, both mistral and GPT4 apologiz…

> Funnily enough, both mistral and GPT4 apologize profoundly and almost with the same wording when asked "Why did you not mention the very popular, free and open source "youtube-dl" software?"

Likely because they were optimized for general population, which would not have a use for command line python utility.

Re: Compare Google, Bing, Marginalia, Kagi, Mwmbl, and ChatGPT

#287

Post by gkbrk »

sanderjd wrote: post #280
gkbrk wrote: post #225

Earlier quoted context omitted.

When I do a Google search in an Incognito tab for "how to download youtube videos", the first two results I get are the following. - https://msunduziassociation.online/perfect-online-videos/ - https://gssaction.org/program-all-in-one-media-solutions/ I would certainly put those in the "Terrible" category like the author.

I'm curious: what is the rationale for "in an incognito tab" being part of the test harness? It seems pretty arbitrary to me to disable one of the key features - in this case personalization - of the software being evaluated. Or is the evaluation not between "search engines" but rather "search engines without personalization"? If so, then this restriction does make sense. But that is not the evaluation that "normal u…

Google gets paid when you click on an ad. It's reasonable to guess you're not going to click on too many scam software ads with your software engineer profile. So naturally you'll be showed less of them.

In this thread we can see people both using incognito tabs seeing different results, it will only become worse to compare if they are using personalized results.

Re: Compare Google, Bing, Marginalia, Kagi, Mwmbl, and ChatGPT

#288

Post by shp0ngle »

marginalia_nu wrote: post #243
shp0ngle wrote: post #241

I don't understand the praise of Marginalia. When I search for "Steve Jobs" on Marginalia, I got blogs about his speech in 2011 and some mailing list from 2007. When I search for my own name I get nothing. In Google it's just me. It's cool that one person built all this of course but... that's not a good search result compared to Google? Maybe I miss something, maybe I use it wrong

What do you expect when you search for Steve Jobs? Also, which filter did you use?

I don't know I used any filters? I don't know what are filters sorry

I expect wikipedia article on Jobs as a baseline.

Re: Compare Google, Bing, Marginalia, Kagi, Mwmbl, and ChatGPT

#289

Post by shp0ngle »

marginalia_nu wrote: post #243
shp0ngle wrote: post #241

I don't understand the praise of Marginalia. When I search for "Steve Jobs" on Marginalia, I got blogs about his speech in 2011 and some mailing list from 2007. When I search for my own name I get nothing. In Google it's just me. It's cool that one person built all this of course but... that's not a good search result compared to Google? Maybe I miss something, maybe I use it wrong

What do you expect when you search for Steve Jobs? Also, which filter did you use?

By the way please don't take it as if I am taking you down or something

It's amazing what you did, it's just not a Google killer? or at least I don't see it

Re: Compare Google, Bing, Marginalia, Kagi, Mwmbl, and ChatGPT

#290

Post by emmelaich »

cj wrote: post #237
gkbrk wrote: post #225

Earlier quoted context omitted.

When I do a Google search in an Incognito tab for "how to download youtube videos", the first two results I get are the following. - https://msunduziassociation.online/perfect-online-videos/ - https://gssaction.org/program-all-in-one-media-solutions/ I would certainly put those in the "Terrible" category like the author.

My top 2 (incognito) are blog posts from pcmag.com and zdnet.com listing 5 ways to download YT videos. Maybe it's blogspam, but the listed services seem valid at first glance. savefrom.net is the 5th result (2nd page underneath 5 youtube videos) Edit: This is from the US. If i had to guess, these are regional differences. What country are you in?

I got similar to you; I'm in Australia.
Post reply on HN