Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I won a championship that doesn't exist

ron.stoner.com

Post reply on HN
Print view View original

51–60 of 141 posts

Re: I won a championship that doesn't exist

#51

Post by sb057 »

simonw wrote: post #3

You don't need to vandalize Wikipedia to get this kind of thing to work. Back in September 2024 I named a whale "Teresa T" with just a blog entry and a YouTube video caption: https://simonwillison.net/2024/Sep/8/teresa-t-whale-pillar-p... (For a few glorious weeks if you asked any search-enabled LLM, including Google search previews, for the name of the whale in the Half Moon Bay harbor it confidently replied Teresa…

I mean, the name of that whale is now Teresa T. You gave it that name.

Re: I won a championship that doesn't exist

#52

Post by cityofdelusion »

poglet wrote: post #41

I made a post on Reddit asking for help with a TV, I had made up some (likley incorrect) technical assumptions about the issue. Several years later I asked the LLM about the TV, it used my own post as a citation to tell me what was wrong with it. I am paranoid that this is happening every time I ask a LLM for a product recommendation or a shop recommendation. In the same way as SEO, anyone wanting to sell or convince…

This is becoming a problem real fast. I asked an LLM to find me some reasonable tank-fill inkjet printers with good ratings. It did some research and linked some Reddits as proof. The results looked fishy to me so I cross checked against prosumer review sites for printers and the models suggested were recognized as junk with very poor print quality. Not sure why the LLM rated random redditors higher than say printer SMEs. I feel like I dodged a bullet.

Re: I won a championship that doesn't exist

#53

Post by simonw »

Forgeties79 wrote: post #48
simonw wrote: post #42

Earlier quoted context omitted.

Exactly. I chose to abuse my platform to promote Teresa T as the name of a whale.

Oh god I just realized the implication! I was not directing that at you haha

No I really did abuse my reach for this one! I figured it would be a relatively harmless demo of how easy it is to affect LLM answers if you have a decently trafficked website.

Re: I won a championship that doesn't exist

#54

Post by Forgeties79 »

simonw wrote: post #53
Forgeties79 wrote: post #48

Earlier quoted context omitted.

Oh god I just realized the implication! I was not directing that at you haha

No I really did abuse my reach for this one! I figured it would be a relatively harmless demo of how easy it is to affect LLM answers if you have a decently trafficked website.

Totally agree. I’ve definitely played the same game before, albeit with far less reach

Re: I won a championship that doesn't exist

#55

Post by sdthjbvuiiijbb »

refulgentis wrote: post #44
nicole_express wrote: post #33

It's an odd thing here, because I don't really understand why this is LLM-specific at all. If someone came up to me and asked "who's the 6 Nimmt world champion?" I'd google it and probably find the same result, and have no reason not to believe it. I mean, for all I know the game is being made up too, though it has more sources at least.

Closed it after “This house of cards only needs a $12 domain!”, right under “Sorry, Wikipedia.”, right under their Wikipedia edit.

It's also clearly AI generated writing. That doesn't help its credibility or interest. I'm extremely suspicious of people who use AI to write an ostensibly personal blog, for all the usual obvious reasons.

Re: I won a championship that doesn't exist

#56

Post by SchemaLoad »

nicole_express wrote: post #33

It's an odd thing here, because I don't really understand why this is LLM-specific at all. If someone came up to me and asked "who's the 6 Nimmt world champion?" I'd google it and probably find the same result, and have no reason not to believe it. I mean, for all I know the game is being made up too, though it has more sources at least.

The difference imo is removing the information from the source. Previously you'd use the source of the information to gauge how much you trust it. If it's a reddit post or a no name website you'd likely be skeptical if it doesn't seem backed up by better sources. But now the info is coming from an LLM that you generally trust to be knowledgeable. And the language it uses backs up this feeling.

The OP post is highlighting how incredibly easy it is for a very small amount of information on the web to completely dictate the output of the LLM in to saying whatever you want.

Re: I won a championship that doesn't exist

#57

Post by julianz »

yen223 wrote: post #46

I feel uncomfortable that I can't actually verify that this story is true. Asking Opus 4.7 who the reigning 6nimmt! champion is leads to this article and a warning about a possible hoax

Gemini answers with 3 different champions dating back to 2024 and the list of events that the matches were played at. None of the results mention this guy.

Re: I won a championship that doesn't exist

#58

Post by chrismcb »

simonw wrote: post #53
Forgeties79 wrote: post #48

Earlier quoted context omitted.

Oh god I just realized the implication! I was not directing that at you haha

No I really did abuse my reach for this one! I figured it would be a relatively harmless demo of how easy it is to affect LLM answers if you have a decently trafficked website.

You could have named the whale "Whalie McWhaleFace" so thank you for not doing that at least.

Re: I won a championship that doesn't exist

#59

Post by apublicfrog »

sdthjbvuiiijbb wrote: post #55
refulgentis wrote: post #44

Earlier quoted context omitted.

Closed it after “This house of cards only needs a $12 domain!”, right under “Sorry, Wikipedia.”, right under their Wikipedia edit.

It's also clearly AI generated writing. That doesn't help its credibility or interest. I'm extremely suspicious of people who use AI to write an ostensibly personal blog, for all the usual obvious reasons.

What are you basing that on? I'm usually pretty good at sniffing out AI writing, and it smells human to me.

Re: I won a championship that doesn't exist

#60

Post by wky »

simonw wrote: post #3

You don't need to vandalize Wikipedia to get this kind of thing to work. Back in September 2024 I named a whale "Teresa T" with just a blog entry and a YouTube video caption: https://simonwillison.net/2024/Sep/8/teresa-t-whale-pillar-p... (For a few glorious weeks if you asked any search-enabled LLM, including Google search previews, for the name of the whale in the Half Moon Bay harbor it confidently replied Teresa…

This post has managed to “confuse” Google about the reverse question as well (“who named teresa t whale”):

The humpback whale known as "Teresa T" was named by Simon Willison in September 2024. Background: The juvenile humpback whale was frequently spotted in Pillar Point Harbor near Half Moon Bay, California. Method: Willison gave the whale its name through a blog entry and a YouTube video caption. Significance: The naming was a playful act, which Willison described as a way to create a "championship that doesn't exist" through online documentation.

[…]

Even with no context most humans would see that the quoted significance makes no sense.

Post reply on HN