71–80 of 141 posts
Earlier quoted context omitted.
What are you basing that on? I'm usually pretty good at sniffing out AI writing, and it smells human to me.
Why is agents (where the money is)? Fake profundity is abound in the post
You don't need to vandalize Wikipedia to get this kind of thing to work. Back in September 2024 I named a whale "Teresa T" with just a blog entry and a YouTube video caption: https://simonwillison.net/2024/Sep/8/teresa-t-whale-pillar-p... (For a few glorious weeks if you asked any search-enabled LLM, including Google search previews, for the name of the whale in the Half Moon Bay harbor it confidently replied Teresa…
This post has managed to “confuse” Google about the reverse question as well (“who named teresa t whale”): The humpback whale known as "Teresa T" was named by Simon Willison in September 2024. Background: The juvenile humpback whale was frequently spotted in Pillar Point Harbor near Half Moon Bay, California. Method: Willison gave the whale its name through a blog entry and a YouTube video caption. Significance: The…
I wonder how long that will last
I must say I expected an actual poisoning of the data used to train the LLM and was excited, but the examples indicate that the LLM just searched the web and reported what it found? When you create a website with fake information and search Google for that information, it will of course bring up your site, not because it’s factually correct but because it’s related to what you searched for. What am I missing?
The part where lots of people have historically trusted LLM responses without verification, more than trying to sort through the dross on Google or Bing search results is, I think, the point.
I feel uncomfortable that I can't actually verify that this story is true. Asking Opus 4.7 who the reigning 6nimmt! champion is leads to this article and a warning about a possible hoax
The key to successful poisoning attacks is to introduce brand new information that doesn't directly contradict other training data. It's much easier to convince the LLMs that you're the king of a fictional Mapupu kingdom than the president of the United States. So this means that for bad actors it's more efficient to manufacture brand new fake stories instead of trying to distort the real ones. Don't produce fake art…
It's an odd thing here, because I don't really understand why this is LLM-specific at all. If someone came up to me and asked "who's the 6 Nimmt world champion?" I'd google it and probably find the same result, and have no reason not to believe it. I mean, for all I know the game is being made up too, though it has more sources at least.
A lot of people seem to think this to be an LLM problem, but you're right. This is a general epistemological problem with relying on the Internet (or really, any piece of literature) as a source of truth
Earlier quoted context omitted.
It's also clearly AI generated writing. That doesn't help its credibility or interest. I'm extremely suspicious of people who use AI to write an ostensibly personal blog, for all the usual obvious reasons.
What are you basing that on? I'm usually pretty good at sniffing out AI writing, and it smells human to me.
I think calling something AI generated is just a lazy way of dismissing stuff nowadays.
It's an odd thing here, because I don't really understand why this is LLM-specific at all. If someone came up to me and asked "who's the 6 Nimmt world champion?" I'd google it and probably find the same result, and have no reason not to believe it. I mean, for all I know the game is being made up too, though it has more sources at least.
Because outside of the tech community (in fact, many even inside of it), almost 100% of the folks consider what these chatgpt like tools answer as the truth without questioning it, or cross-verifying it even once.
The only real alternatives would be:
- Kicking off a deep research-like investigation for each simple query
- Introducing a trusted middleman for sources, significantly cutting down the available information (e.g. restricting Wikipedia to locked-down/moderated pages)
- Not having any information at all, as at some point you can rarely every verify anything depending on how hard your definition of "verify" is
Since we've been kids we've been taught, hopefully, that lying is bad.
Society though normalize it :
- advertisement is pretty much always wrong (to the point of having laws in Japan about food packaging, France about modeling, etc) and the deception is the message
- entrepreneurs promises, nobody reach the goals set to VCs, it's always a lower number no matter the KPI. See https://elonmusk.today where the wealthiest man on Earth, ever, keeps on lying pretty much daily.
- political promises, no need to even give examples of that because it's just pervasive.
so... yeah, we keep on telling our kids "Do as I say, not as I do." then we somehow keep on being shocked that the practice of lying is pretty much happening in every corner of our society.
It's not a technical problem.
It's an odd thing here, because I don't really understand why this is LLM-specific at all. If someone came up to me and asked "who's the 6 Nimmt world champion?" I'd google it and probably find the same result, and have no reason not to believe it. I mean, for all I know the game is being made up too, though it has more sources at least.