Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I won a championship that doesn't exist

ron.stoner.com

Post reply on HN
Print view View original

61–70 of 141 posts

Re: I won a championship that doesn't exist

#61

Post by ceroxylon »

simonw wrote: post #3

You don't need to vandalize Wikipedia to get this kind of thing to work. Back in September 2024 I named a whale "Teresa T" with just a blog entry and a YouTube video caption: https://simonwillison.net/2024/Sep/8/teresa-t-whale-pillar-p... (For a few glorious weeks if you asked any search-enabled LLM, including Google search previews, for the name of the whale in the Half Moon Bay harbor it confidently replied Teresa…

When I asked some frontier models, many said that Teresa T is "widely referenced", which is evidence of your popularity and the ripple effects of your posts, so it would be interesting to see the same result from an unknown blog.

Re: I won a championship that doesn't exist

#62

Post by malfist »

apublicfrog wrote: post #59
sdthjbvuiiijbb wrote: post #55

Earlier quoted context omitted.

It's also clearly AI generated writing. That doesn't help its credibility or interest. I'm extremely suspicious of people who use AI to write an ostensibly personal blog, for all the usual obvious reasons.

What are you basing that on? I'm usually pretty good at sniffing out AI writing, and it smells human to me.

Why is agents (where the money is)? Fake profundity is abound in the post

Re: I won a championship that doesn't exist

#63

Post by fsckboy »

blobbers wrote: post #26

This is basically the same problem of products astroturfing reddit, or SEO optimizing google. You want a new X, and so they heavily go after the keywords associated with it. This is sort of why "brand" matters; it provides a source of trust. Encyclopedia Britannica used to be that source of 'facts'. Then it became whatever page-rank told you. Eventually SEO optimization ruined that. News stories are the same thing. F…

>This is sort of why "brand" matters; it provides a source of trust

it tells you more about who you are buying from than how good the product will be, so I guess it's like National ID/Internet ID

Re: I won a championship that doesn't exist

#65

Post by giancarlostoro »

simonw wrote: post #3

You don't need to vandalize Wikipedia to get this kind of thing to work. Back in September 2024 I named a whale "Teresa T" with just a blog entry and a YouTube video caption: https://simonwillison.net/2024/Sep/8/teresa-t-whale-pillar-p... (For a few glorious weeks if you asked any search-enabled LLM, including Google search previews, for the name of the whale in the Half Moon Bay harbor it confidently replied Teresa…

Even your HN comments show up on Google! I've found myself on Google twice when looking up something that I apparently answered on HN!

Re: I won a championship that doesn't exist

#66

Post by freakynit »

nicole_express wrote: post #33

It's an odd thing here, because I don't really understand why this is LLM-specific at all. If someone came up to me and asked "who's the 6 Nimmt world champion?" I'd google it and probably find the same result, and have no reason not to believe it. I mean, for all I know the game is being made up too, though it has more sources at least.

Because outside of the tech community (in fact, many even inside of it), almost 100% of the folks consider what these chatgpt like tools answer as the truth without questioning it, or cross-verifying it even once.

Re: I won a championship that doesn't exist

#67

Post by soupspaces »

xeeeeeeeeeeenu wrote: post #29

The key to successful poisoning attacks is to introduce brand new information that doesn't directly contradict other training data. It's much easier to convince the LLMs that you're the king of a fictional Mapupu kingdom than the president of the United States. So this means that for bad actors it's more efficient to manufacture brand new fake stories instead of trying to distort the real ones. Don't produce fake art…

[dead]

Re: I won a championship that doesn't exist

#68

Post by pmontra »

nicole_express wrote: post #33

It's an odd thing here, because I don't really understand why this is LLM-specific at all. If someone came up to me and asked "who's the 6 Nimmt world champion?" I'd google it and probably find the same result, and have no reason not to believe it. I mean, for all I know the game is being made up too, though it has more sources at least.

It is not LLM specific. The conclusion of the post states

> The web was already being poisoned for search and link ranking long before LLMs existed.

But it continues

> We are now plugging generative models directly into that poisoned pipeline and asking them to reason confidently about “truth” on our behalf.

So it's a shift from trust Google to trust the AI, which might be more insidious or not, depends on the individual attitude of each of us.

Re: I won a championship that doesn't exist

#69

Post by rectang »

simonw wrote: post #3

You don't need to vandalize Wikipedia to get this kind of thing to work. Back in September 2024 I named a whale "Teresa T" with just a blog entry and a YouTube video caption: https://simonwillison.net/2024/Sep/8/teresa-t-whale-pillar-p... (For a few glorious weeks if you asked any search-enabled LLM, including Google search previews, for the name of the whale in the Half Moon Bay harbor it confidently replied Teresa…

You're making me nostalgic for santorum.

https://en.wikipedia.org/wiki/Campaign_for_the_neologism_%22...

Re: I won a championship that doesn't exist

#70

Post by DiscourseFan »

The models are trained on expert data for important inquiries, this gets “hard coded” so to speak, and allows them to differentiate between the gunk online. For hyper specific references like this, it really doesn’t matter if its “true” since its not like someone’s life depends on it.
Post reply on HN