Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I won a championship that doesn't exist

ron.stoner.com

Post reply on HN
Print view View original

91–100 of 141 posts

Re: I won a championship that doesn't exist

#91

Post by latexr »

ceroxylon wrote: post #61
simonw wrote: post #3

You don't need to vandalize Wikipedia to get this kind of thing to work. Back in September 2024 I named a whale "Teresa T" with just a blog entry and a YouTube video caption: https://simonwillison.net/2024/Sep/8/teresa-t-whale-pillar-p... (For a few glorious weeks if you asked any search-enabled LLM, including Google search previews, for the name of the whale in the Half Moon Bay harbor it confidently replied Teresa…

When I asked some frontier models, many said that Teresa T is "widely referenced", which is evidence of your popularity and the ripple effects of your posts, so it would be interesting to see the same result from an unknown blog.

> When I asked some frontier models, many said that Teresa T is "widely referenced", which is evidence of your popularity and the ripple effects of your posts

That is some serious Gell-Mann-type amnesia. You’re trusting LLM models to give you accurate information about a subject we’ve already established (and are only talking about because) they can’t be trusted on.

“Widely referenced” is a common term which LLMs obviously pick up. Them outputting those words has no bearing on the truth and says nothing about the “popularity and the ripple effects of [Simon’s] posts”.

Re: I won a championship that doesn't exist

#92

Post by ricardo81 »

Not too dissimilar to googlewhacking where you'd aim to be the only result for a search query on Google.

And in a more indirect way, spamming Google's autosuggest feature to shape what people search for, though that perhaps is more open to factual/real-world information.

Re: I won a championship that doesn't exist

#93

Post by teiferer »

bambax wrote: post #86
pmontra wrote: post #68

Earlier quoted context omitted.

It is not LLM specific. The conclusion of the post states > The web was already being poisoned for search and link ranking long before LLMs existed. But it continues > We are now plugging generative models directly into that poisoned pipeline and asking them to reason confidently about “truth” on our behalf. So it's a shift from trust Google to trust the AI, which might be more insidious or not, depends on the indivi…

It's a shift but it's a little worse. Checking/auditing search results is easier and more ingrained; even if many people don't do it, everyone has been hit by spam at some point, everyone knows it exists. LLMs are the same thing but have an air of authority about them that a web search lacks, at least for now.

To me that's the opposite. Whatever an LLM gives me, I view with skepticism. If I google sth then I quickly get a sense of how much I can trust it and what the BS factor is. I can refine my view in either case, but my a priori trust with an LLM is much lower.

Maybe we just need to work on training the general population to have a similar bias. (It will be harder than it sounds. Unbelievable amounts of capital are being bet on this not happening.)

Re: I won a championship that doesn't exist

#94

Post by seanhunter »

nicole_express wrote: post #33

It's an odd thing here, because I don't really understand why this is LLM-specific at all. If someone came up to me and asked "who's the 6 Nimmt world champion?" I'd google it and probably find the same result, and have no reason not to believe it. I mean, for all I know the game is being made up too, though it has more sources at least.

It's not. He vandalised wikipedia and then talked about LLMs in his writeup to gain attention.

Re: I won a championship that doesn't exist

#95

Post by bambax »

teiferer wrote: post #93
bambax wrote: post #86

Earlier quoted context omitted.

It's a shift but it's a little worse. Checking/auditing search results is easier and more ingrained; even if many people don't do it, everyone has been hit by spam at some point, everyone knows it exists. LLMs are the same thing but have an air of authority about them that a web search lacks, at least for now.

To me that's the opposite. Whatever an LLM gives me, I view with skepticism. If I google sth then I quickly get a sense of how much I can trust it and what the BS factor is. I can refine my view in either case, but my a priori trust with an LLM is much lower. Maybe we just need to work on training the general population to have a similar bias. (It will be harder than it sounds. Unbelievable amounts of capital are bei…

Yes, it's the same for me, but we're not representative of most people I'm afraid.

Re: I won a championship that doesn't exist

#97

Post by latexr »

sb057 wrote: post #51
simonw wrote: post #3

You don't need to vandalize Wikipedia to get this kind of thing to work. Back in September 2024 I named a whale "Teresa T" with just a blog entry and a YouTube video caption: https://simonwillison.net/2024/Sep/8/teresa-t-whale-pillar-p... (For a few glorious weeks if you asked any search-enabled LLM, including Google search previews, for the name of the whale in the Half Moon Bay harbor it confidently replied Teresa…

I mean, the name of that whale is now Teresa T. You gave it that name.

And your name is now Berningular Farshthruster III. I gave you that name.

Which is, of course, silly. It is a name for you, just like Teresa T is a name for the whale, but it’s not your/their name, just like the RRS Sir David Attenborough is not named Boaty McBoatface (to the chagrin of most). Simon does not have the authority to unilaterally¹ name the whale (which is why the exercise makes sense).

¹ Important point. If the name started being recognised and used by consensus of those with the purview to do so (much like the thagomizer²), then Simon would have named the whale, but it would only become its name at that point.

² https://en.wikipedia.org/wiki/Thagomizer

Re: I won a championship that doesn't exist

#98

Post by latexr »

nicole_express wrote: post #33

It's an odd thing here, because I don't really understand why this is LLM-specific at all. If someone came up to me and asked "who's the 6 Nimmt world champion?" I'd google it and probably find the same result, and have no reason not to believe it. I mean, for all I know the game is being made up too, though it has more sources at least.

> I'd google it and probably find the same result, and have no reason not to believe it.

Have you truly looked at the website?

https://6nimmt.com

I’d say there’s obvious reason to not believe it, or at least check another source. The website just seems fishy. Why would a website exist for just that one post? Sure, they could’ve made the website more believable, but that takes more effort and has more chances for something to jump out at you.

And therein lies a major difference between searching the web and asking an LLM. When doing the former, you can pick up on clues regarding what to trust. For example, a website you’ve visited often and has proven reliable will be more trustworthy to you than one you’ve never been to before. When asking an LLM, every piece of information is provided in the same interface, with the same authoritative certainty. You lose a major important signal.

Re: I won a championship that doesn't exist

#99

Post by saghm »

nomdep wrote: post #43
Forgeties79 wrote: post #27

Earlier quoted context omitted.

Which illustrates another problem: unscrupulous actors with big names can spread whatever information they want to millions of people with minimal effort.

Yes, they can. We can be glad that respectable newspapers and TV news channels have never done it and never will. You can even trust than the headlines are accurate summaries of the content of the articles. /s

The existence of a problem in one area doesn't mean that it's not also a problem for it to spread somewhere else

Re: I won a championship that doesn't exist

#100

Post by saghm »

teiferer wrote: post #93
bambax wrote: post #86

Earlier quoted context omitted.

It's a shift but it's a little worse. Checking/auditing search results is easier and more ingrained; even if many people don't do it, everyone has been hit by spam at some point, everyone knows it exists. LLMs are the same thing but have an air of authority about them that a web search lacks, at least for now.

To me that's the opposite. Whatever an LLM gives me, I view with skepticism. If I google sth then I quickly get a sense of how much I can trust it and what the BS factor is. I can refine my view in either case, but my a priori trust with an LLM is much lower. Maybe we just need to work on training the general population to have a similar bias. (It will be harder than it sounds. Unbelievable amounts of capital are bei…

In a discussion with my father-in-law about whether ChatGPT was trained on copyrighted materials, he literally asked ChatGPT and treated its response that it wasn't as useful evidence. He went to MIT, so he's arguably more educated than most people will ever be, so it's hard for me to be optimistic that trying to just explain this to people better will move the needle significantly.
Post reply on HN