Well yes of course.
In the old days of computing people liked to say “garbage in, garbage out”.
Scientists invented a fake disease. AI told people it was real
11–20 of 97 posts
Re: Scientists invented a fake disease. AI told people it was real
#12Earlier quoted context omitted.
[flagged]
But then mono-opinion- aka certainty - is actually peak uncertainty? Could that number of occurrence be baked into as a sort of detrimental weight?
We need to give the LLMs robot bodies so they can practise medicine and see the illnesses that do and don’t exist first hand
Re: Scientists invented a fake disease. AI told people it was real
#13It sounds like there wasn't really a counter narrative for the models to learn from. This feature of how llms accumulate information is already being gamed by seeding the internet with preferred narratives. I'm not sure how many Medium articles, blog posts and reddit threads I need to put out before grok starts telling everyone my widget is the best one ever made, but it's a lot cheaper than advertising.
I’ve seen an estimate before and it’s in the low 10s.
Re: Scientists invented a fake disease. AI told people it was real
#14Seems to be a failure of the publishing system.
For humans, or Ai, to have any knowledge, we need to have trustworthy sources.
Naturally,when you use publishing systems considered trust worthy, that is going to be trusted.
Re: Scientists invented a fake disease. AI told people it was real
#15It sounds like there wasn't really a counter narrative for the models to learn from. This feature of how llms accumulate information is already being gamed by seeding the internet with preferred narratives. I'm not sure how many Medium articles, blog posts and reddit threads I need to put out before grok starts telling everyone my widget is the best one ever made, but it's a lot cheaper than advertising.
Can a model not just ignor all things that have no counter-argument by default? Like - if there are not flat earthers, widly debunked, drop the idea of a spherical earth? It only exists if it was fought over?
> drop the idea of a spherical earth
I think I see a problem here.
Re: Scientists invented a fake disease. AI told people it was real
#16You’ve seen people game adsense It’s gunna be even wilder when people realise they have an incentive to seed fake information on the internet to game AI product recommendations I’ve already bought stuff based off of an AI suggestion, I didn’t even consider it would be so easy to influence the suggestion. Just two research papers? Mad.
That's already been happening for more than a year now.
Re: Scientists invented a fake disease. AI told people it was real
#17Seems to be a failure of the publishing system. For humans, or Ai, to have any knowledge, we need to have trustworthy sources. Naturally,when you use publishing systems considered trust worthy, that is going to be trusted.
A preprint isn't a published works.
Re: Scientists invented a fake disease. AI told people it was real
#18It sounds like there wasn't really a counter narrative for the models to learn from. This feature of how llms accumulate information is already being gamed by seeding the internet with preferred narratives. I'm not sure how many Medium articles, blog posts and reddit threads I need to put out before grok starts telling everyone my widget is the best one ever made, but it's a lot cheaper than advertising.
People really like using the word "narrative". I guess we're creatures of story.
But this really highlights how much we've been benefiting from living in a high-trust society, where people don't just "go on the internet and tell lies" - filtered by the existing anti-spam and anti-SEO measures intended to cut out the 80% of the internet where people do just make things up to sell products.
LLMs are extremely post-structuralist. They really force the user to decide whether to pick the beautiful eternal fountain of plausible looking text with no ground truth, or a much harder road of distrust, verification, and old-school social proof.
Re: Scientists invented a fake disease. AI told people it was real
#19Well yes of course. In the old days of computing people liked to say “garbage in, garbage out”.
By that logic, LLMs would be essentially useless considering the amount of garbage that exists on the internet. And, honestly, for things like this they are. But they're not marketed as such, and _that_ is the problem.
Re: Scientists invented a fake disease. AI told people it was real
#20It sounds like there wasn't really a counter narrative for the models to learn from. This feature of how llms accumulate information is already being gamed by seeding the internet with preferred narratives. I'm not sure how many Medium articles, blog posts and reddit threads I need to put out before grok starts telling everyone my widget is the best one ever made, but it's a lot cheaper than advertising.
I'm not sure "being gamed" is the lens I would see this particular instance through. People (some at least) have gotten into their heads that they can ask LLMs objective questions and get objectively correct answers. The LLM companies are doing very little to dissuade them of that belief.
Meanwhile, LLMs are essentially internet regurgitation machines, because of course they are, that's what they do. Which makes them useless for getting "hard truth" answers especially in contested or specialized fields.
I'm honestly afraid of the impact of this. The internet has enough herd bullshit on it as it is. (e.g. antivaxxers, flat earthers, electrosensitivity, vitamin/supplement junk, etc.) We don't need that amplified.