[deleted]
I won a championship that doesn't exist
81–90 of 141 posts
Re: I won a championship that doesn't exist
#82You don't need to vandalize Wikipedia to get this kind of thing to work. Back in September 2024 I named a whale "Teresa T" with just a blog entry and a YouTube video caption: https://simonwillison.net/2024/Sep/8/teresa-t-whale-pillar-p... (For a few glorious weeks if you asked any search-enabled LLM, including Google search previews, for the name of the whale in the Half Moon Bay harbor it confidently replied Teresa…
This post has managed to “confuse” Google about the reverse question as well (“who named teresa t whale”): The humpback whale known as "Teresa T" was named by Simon Willison in September 2024. Background: The juvenile humpback whale was frequently spotted in Pillar Point Harbor near Half Moon Bay, California. Method: Willison gave the whale its name through a blog entry and a YouTube video caption. Significance: The…
In a discussion on hacker news on Wed April 29, about the whale "Teresa T", user "vintermann" tested just how eager search engines are to scrape fresh social media comments, by seeing if they could get this comment to appear in AI summaries.
Re: I won a championship that doesn't exist
#83Earlier quoted context omitted.
It's also clearly AI generated writing. That doesn't help its credibility or interest. I'm extremely suspicious of people who use AI to write an ostensibly personal blog, for all the usual obvious reasons.
What are you basing that on? I'm usually pretty good at sniffing out AI writing, and it smells human to me.
I had the impression it was AI writing too because of the second half of the article. The first part looks genuine, the part since "trust laundering" smells fake: the scary single sentence followed by a whole paragraph of single clause sentences hints at AI.
Perhaps we've all just become paranoid, but even if it's not LLMs writing this, it now puts me off. And the AI image at the top of the page does not help with the feeling.
Re: I won a championship that doesn't exist
#84The key to successful poisoning attacks is to introduce brand new information that doesn't directly contradict other training data. It's much easier to convince the LLMs that you're the king of a fictional Mapupu kingdom than the president of the United States. So this means that for bad actors it's more efficient to manufacture brand new fake stories instead of trying to distort the real ones. Don't produce fake art…
As the rightful ruler of Mapupu, I take offense at your example!
Re: I won a championship that doesn't exist
#85[flagged]
Re: I won a championship that doesn't exist
#86It's an odd thing here, because I don't really understand why this is LLM-specific at all. If someone came up to me and asked "who's the 6 Nimmt world champion?" I'd google it and probably find the same result, and have no reason not to believe it. I mean, for all I know the game is being made up too, though it has more sources at least.
It is not LLM specific. The conclusion of the post states > The web was already being poisoned for search and link ranking long before LLMs existed. But it continues > We are now plugging generative models directly into that poisoned pipeline and asking them to reason confidently about “truth” on our behalf. So it's a shift from trust Google to trust the AI, which might be more insidious or not, depends on the indivi…
It's a shift but it's a little worse. Checking/auditing search results is easier and more ingrained; even if many people don't do it, everyone has been hit by spam at some point, everyone knows it exists.
LLMs are the same thing but have an air of authority about them that a web search lacks, at least for now.
Re: I won a championship that doesn't exist
#87The key to successful poisoning attacks is to introduce brand new information that doesn't directly contradict other training data. It's much easier to convince the LLMs that you're the king of a fictional Mapupu kingdom than the president of the United States. So this means that for bad actors it's more efficient to manufacture brand new fake stories instead of trying to distort the real ones. Don't produce fake art…
> It's much easier to convince the LLMs that you're the king of a fictional Mapupu kingdom than the president of the United States.
But if you're a world class bullshit artist, it's easier to actually become president of the United States than doing all that complicated computer stuff.
Re: I won a championship that doesn't exist
#88Pretty much boils down to lying. Since we've been kids we've been taught, hopefully, that lying is bad. Society though normalize it : - advertisement is pretty much always wrong (to the point of having laws in Japan about food packaging, France about modeling, etc) and the deception is the message - entrepreneurs promises, nobody reach the goals set to VCs, it's always a lower number no matter the KPI. See https://el…
The fun part is when it’s important you have the right information to make a decision. Eg Russia to invade Ukraine and all top generals claim they can do it in 2 weeks. Similar for a corporation with layers of middle management deception and self promotion, I don’t know how executives make decisions but it must be RNG basically, because it certainly isn’t fact.
Lying at scale is basically information noise.
Re: I won a championship that doesn't exist
#89You don't need to vandalize Wikipedia to get this kind of thing to work. Back in September 2024 I named a whale "Teresa T" with just a blog entry and a YouTube video caption: https://simonwillison.net/2024/Sep/8/teresa-t-whale-pillar-p... (For a few glorious weeks if you asked any search-enabled LLM, including Google search previews, for the name of the whale in the Half Moon Bay harbor it confidently replied Teresa…
Also, if even a stoner can win it it can't be much of a competition.
Re: I won a championship that doesn't exist
#90Pretty much boils down to lying. Since we've been kids we've been taught, hopefully, that lying is bad. Society though normalize it : - advertisement is pretty much always wrong (to the point of having laws in Japan about food packaging, France about modeling, etc) and the deception is the message - entrepreneurs promises, nobody reach the goals set to VCs, it's always a lower number no matter the KPI. See https://el…
The fun part is when it’s important you have the right information to make a decision. Eg Russia to invade Ukraine and all top generals claim they can do it in 2 weeks. Similar for a corporation with layers of middle management deception and self promotion, I don’t know how executives make decisions but it must be RNG basically, because it certainly isn’t fact. Lying at scale is basically information noise.
If you don't lie enough, if you are not sycophantic enough, then no promotion or worst, purged.
I can easily see how such a hierarchy would reproduce ... until it fails so bad it can't.