41–50 of 141 posts
I am paranoid that this is happening every time I ask a LLM for a product recommendation or a shop recommendation. In the same way as SEO, anyone wanting to sell or convince needs to do as much as they can to influence the LLM.
Earlier quoted context omitted.
(it probably helps that your name & blog carry some weight, vs. some rando writing something on blogspot or wordpress ;) )
Which illustrates another problem: unscrupulous actors with big names can spread whatever information they want to millions of people with minimal effort.
Earlier quoted context omitted.
(it probably helps that your name & blog carry some weight, vs. some rando writing something on blogspot or wordpress ;) )
Which illustrates another problem: unscrupulous actors with big names can spread whatever information they want to millions of people with minimal effort.
It's an odd thing here, because I don't really understand why this is LLM-specific at all. If someone came up to me and asked "who's the 6 Nimmt world champion?" I'd google it and probably find the same result, and have no reason not to believe it. I mean, for all I know the game is being made up too, though it has more sources at least.
People think that whatever information an "AI" spits out has gone through a round of critical thinking which enhances the trust value of that information.
The early LLM's using groomed data may have had such critical thinking somewhere in the pipeline. So it was already not really trustworthy.
And now? Using agents to search the internet for you?...
Garbage in, garbage out still applies in computing as ever.
Asking Opus 4.7 who the reigning 6nimmt! champion is leads to this article and a warning about a possible hoax
It's an odd thing here, because I don't really understand why this is LLM-specific at all. If someone came up to me and asked "who's the 6 Nimmt world champion?" I'd google it and probably find the same result, and have no reason not to believe it. I mean, for all I know the game is being made up too, though it has more sources at least.
This is a general epistemological problem with relying on the Internet (or really, any piece of literature) as a source of truth
Earlier quoted context omitted.
Which illustrates another problem: unscrupulous actors with big names can spread whatever information they want to millions of people with minimal effort.
Exactly. I chose to abuse my platform to promote Teresa T as the name of a whale.
You don't need to vandalize Wikipedia to get this kind of thing to work. Back in September 2024 I named a whale "Teresa T" with just a blog entry and a YouTube video caption: https://simonwillison.net/2024/Sep/8/teresa-t-whale-pillar-p... (For a few glorious weeks if you asked any search-enabled LLM, including Google search previews, for the name of the whale in the Half Moon Bay harbor it confidently replied Teresa…
Earlier quoted context omitted.
Wikipedia calls this https://en.wikipedia.org/wiki/Citogenesis (after XKCD coined it). You can expect the spicy autocomplete to feed you flattering bullshit. It may cite Wikipedia (it shouldn't), but you should go check out those citations, and validate the claims yourself. It's the least you can do. And if the cited source is Wikipedia... check Wikipedia's sources too. Wikipedians try their best to provide you with…
Perhaps my favorite example of a citogenesis-like process is the legendary arcade game Polybius, which originated as an entry on some German guy's web compendium of arcade games (coinop.org), perhaps as a "paper town", or fake entry that acts as a copyright canary when duplicated elsewhere. Gamer news and special-interest blogs and sites, and even print publications like GamePro picked it up, and I think it was even…