Bad Actors Are Grooming LLMs to Produce Falsehoods • Hacker News Board

Re: Bad Actors Are Grooming LLMs to Produce Falsehoods

#251

Post by mjburgess » Sat, Jul 12, 2025, 8:28 PM UTC

mopsi wrote: post #249

mjburgess wrote: post #248

Earlier quoted context omitted.

lol, these are not my original thoughts, nor have i read any russian sources or propaganda. These are extremely the mainstream views in western international relations literature

No, they're not, although Russian propaganda tries to create that impression by amplifying fringe voices like John Mearsheimer and Jeffrey Sachs. If anything, the Western international relations crowd has finally woken up to recognizing Russia as a colonial empire and has found renewed interest in Eastern Europe's perspectives and recent history.

For reference,

John Joseph Mearsheimer (/ˈmɪərʃaɪmər/; born December 14, 1947)[3] is an American political scientist and international relations scholar. He is R. Wendell Harrison Distinguished Service Professor at the University of Chicago.

I'll just leave the Nuland leak: https://www.youtube.com/watch?v=LUCCR4jAS3Y&ab_channel=TheTr...

But yeah, lol. This is russian propaganda. You can find Kissinger giving this analysis in the 90s

Re: Bad Actors Are Grooming LLMs to Produce Falsehoods

#252

Post by LAC-Tech » Sat, Jul 12, 2025, 8:31 PM UTC

Applejinx wrote: post #137

LAC-Tech wrote: post #96

Whatever capabilities Russia has to groom LLMs and and spread disinformation is completely dwarfed by the capabilities of Israel/America. Meaning, yes, you probably do hear Kremlin propaganda, but you have been awash with Israeli/American propaganda since you were born - so much so you probably can't even see it and have internalised much of it.

Leaving aside the Israeli propaganda (certainly the US government shows strong alliance with that), you can't make such a statement without taking into account the nature of what America traditionally is. A liberal multicultural postmodern democracy continually acting as if immigration (both legal and illegal) and diversity are its strengths, particularly when that turns out to be factual (see: large American cities…

I think it's a bit of a childish fantasy to paint one regime is open and the other as having no dissent at all.

The USA absolutely has its overton window, and if you step outside it, bank accounts get shut, you're put on secret no fly lists, private companies who suspiciously act as official public broadcasting channels deplatform you, etc.

And let's not even talk about what Authoritarian western nations like the UK will do to you.

Russian propaganda, at least in English (which I confess is the only way I can consume) it, is also very contradictory. RT oscillates wildly between "global south throwing off the shackles of western imperialism" and "degenerate western nations destroy traditional family values", in effect trying to target both shitlibs and chuds.

Russia is also very multicultural and slavic ethnonationalism is not at all in the mainstream.

Re: Bad Actors Are Grooming LLMs to Produce Falsehoods

#253

Post by johneth » Sat, Jul 12, 2025, 8:49 PM UTC

agentcoops wrote: post #198

throwawayqqq11 wrote: post #172

Earlier quoted context omitted.

IMO the split is between people understanding the heuristic nature of AI and people who dont and thus think of it as an all-knowing, all-solving oracle. Your elder parents having nice conversations with chatgpt is nice aslong it doesnt make big life changing decisions for them, which happens already today. You have to know the tools limits and usecases.

I can’t see that proposed division as anything but a straw-man. You would be hard-pressed to find anyone who genuinely thinks of LLMs as an “all-knowing, all-solving oracle” and yet, even in specialist fields, their utility is certainly more than a mere “heuristic”, which of course isn’t to say they don’t have limits. See only Terrance Tao’s reports on his ongoing experiments. Do you genuinely think it’s worse that s…

> Do you genuinely think it’s worse that someone makes a decision, whether good or bad, after consulting with GPT versus making it in solitude?

Depending on the decision yes. An LLM might confidently hallucinate incorrect information and misinform, which is worse than simply not knowing.

Re: Bad Actors Are Grooming LLMs to Produce Falsehoods

#254

Post by thunky » Sat, Jul 12, 2025, 8:58 PM UTC

imiric wrote: post #235

thunky wrote: post #196

Earlier quoted context omitted.

> Before the bubble does pop, which I think is inevitable Curious what you think a popping bubble looks like? A stock market crash and recession, where innocent bystanders lose their retirements? Or only AI speculators taking the brunt of the losses? Will Google, Meta, etc stop investing in AI because nobody uses it post-crash? Or will it be just as prevalent (or more) than today but with profits concentrated in the…

We've seen this before in 1983 and 2000. Many companies will fold, and those that don't will take a substantial hit. The public sentiment about "AI" will sour, but after that a new breed of more practical tools will emerge under different and more fairly marketed branding. I do think that the industry and this technology will survive, and we'll enjoy many good applications of it, but it will take a few more years of…

> We've seen this before in 1983 and 2000. Many companies will fold, and those that don't will take a substantial hit.

Makes sense, but if the negative effect of the bubble popping is largely limited to AI startups and speculators, while the rest of us keep enjoying the benefits of it, then I don't see why the average person should be too concerned about a bubble.

In 2000, cab drivers were recommending tech stocks. I don't see this kindof thing happening today.

> Yeah, these people are full of shit.

I think it's fair to keep LLMs and AGI seperate when we're talking about "AI". LLMs can make a huge impact even if AGI never happens. We're already seeing now it imo.

AI 2027 says:

  - Early 2026: Coding Automation
  - Late 2026: AI Takes Some Jobs

These things are already happening today without AGI.

Re: Bad Actors Are Grooming LLMs to Produce Falsehoods

#255

Post by 12345hn6789 » Sat, Jul 12, 2025, 9:17 PM UTC

daveguy wrote: post #192

throwawayqqq11 wrote: post #172

Earlier quoted context omitted.

IMO the split is between people understanding the heuristic nature of AI and people who dont and thus think of it as an all-knowing, all-solving oracle. Your elder parents having nice conversations with chatgpt is nice aslong it doesnt make big life changing decisions for them, which happens already today. You have to know the tools limits and usecases.

Yup. Exactly this. As soon as enough people get screwed by the ~80% accuracy rate, the whole facade will crumble. Unless AI companies manage to bring the accuracy up 20% in the next year, by either limiting scope or finding new methods, it will crumble. That kind of accuracy gain isn't happening with LLMs alone (ie foundational models).

My team has measurably gotten our LLM feature to have ~94% accuracy in widespread reliable tests. Seems fairly confident, speaking as an SWE not a DS orML engineer though.

Re: Bad Actors Are Grooming LLMs to Produce Falsehoods

#256

Post by andrepd » Sat, Jul 12, 2025, 9:26 PM UTC

suspended_state wrote: post #233

andrepd wrote: post #206

Earlier quoted context omitted.

As I pointed out in a different comment, the Internet at least was (and is) a promise of many wondrous things: video call your loved ones, talk in message boards, read an encyclopedia, download any book, watch any concert, find any scientific paper, etc etc; even though it has been for the last 15 years cannibalised by the cancerous mix of surveillance capitalism and algorithmic social media. But LLMs are from the ge…

> [...] LLMs are from the get-go a bad idea, a bullshit generating machine. Is that a matter of opinion, or a fact (in which case you should be able to back it up)?

For real? x) Of course it's my opinion, what are your own comments about "silly gifs" and "useless early internet" if not an opinion?Seriously...

Re: Bad Actors Are Grooming LLMs to Produce Falsehoods

#257

Post by mopsi » Sat, Jul 12, 2025, 9:32 PM UTC

mjburgess wrote: post #251

mopsi wrote: post #249

Earlier quoted context omitted.

No, they're not, although Russian propaganda tries to create that impression by amplifying fringe voices like John Mearsheimer and Jeffrey Sachs. If anything, the Western international relations crowd has finally woken up to recognizing Russia as a colonial empire and has found renewed interest in Eastern Europe's perspectives and recent history.

For reference, John Joseph Mearsheimer (/ˈmɪərʃaɪmər/; born December 14, 1947)[3] is an American political scientist and international relations scholar. He is R. Wendell Harrison Distinguished Service Professor at the University of Chicago. I'll just leave the Nuland leak: https://www.youtube.com/watch?v=LUCCR4jAS3Y&ab_channel=TheTr... But yeah, lol. This is russian propaganda. You can find Kissinger giving this ana…

Yes, it indeed is. Mearsheimer's books are financed by the Russian government, and Nuland is an insignificant middle manager in the US State Department whose intercepted phone call about recent events has been hyped into a vast conspiracy theory, as if she were a powerful shadow operative filling Ukrainian government positions.

Re: Bad Actors Are Grooming LLMs to Produce Falsehoods

#258

Post by jahsome » Sat, Jul 12, 2025, 9:32 PM UTC

fc417fc802 wrote: post #45

jahsome wrote: post #25

Earlier quoted context omitted.

People have done this on their own behalf since the dawn of time, so it's not really clear to me why it's so often framed as an AI issue.

The core underlying issue isn't due to LLMs but they greatly exacerbate it. So does the current form of social media. People used to live in bubbles, sure, but when that bubble was the entire local community, required human interaction, and radio had yet to be invented the implications were vastly different. I'm optimistic that carefully crafted algorithms could send things back in the other direction but that isn't…

My point is: the exact same was said of every form of mass communication. Every new form was said to be a herald of the end times, and yet here we are, in many ways stronger than ever.

Im not arguing one way or another, I'm just pointing out a potential fatigue. It's difficult to see how this technology is relatively any more transformative than any of the others.

Re: Bad Actors Are Grooming LLMs to Produce Falsehoods

#259

Post by mjburgess » Sat, Jul 12, 2025, 9:43 PM UTC

mopsi wrote: post #257

mjburgess wrote: post #251

Earlier quoted context omitted.

For reference, John Joseph Mearsheimer (/ˈmɪərʃaɪmər/; born December 14, 1947)[3] is an American political scientist and international relations scholar. He is R. Wendell Harrison Distinguished Service Professor at the University of Chicago. I'll just leave the Nuland leak: https://www.youtube.com/watch?v=LUCCR4jAS3Y&ab_channel=TheTr... But yeah, lol. This is russian propaganda. You can find Kissinger giving this ana…

Yes, it indeed is. Mearsheimer's books are financed by the Russian government, and Nuland is an insignificant middle manager in the US State Department whose intercepted phone call about recent events has been hyped into a vast conspiracy theory, as if she were a powerful shadow operative filling Ukrainian government positions.

OK, so give me some evidence of M' being a russian asset / funded

It doesn't really matter, because the offensive realist tradition is morality neutral

It doesn't really matter who is at fault. Great powers seek to dominate. Russia has a sphere of influence, so does the US. UA is as free as cuba. There is no morality here

if you're radicalised by your own bee hive to think your side isn't a great power seeking power, you're too far gone

There is no debate about the logic of international relations, no one is 'right'. Personally i think peace was possible, it's irrelevant now -- thanks to this jingoism

Nuland isn't even the most relevant player in ukraine in this; you can find senators confessing to deposing the government of UA; and of only funding UA to grind down the russians. The whole machine i. UA is a cynical exercise of US power.

If you're a zealot for your own nations religion, so be it. This is all futile, there is no fight here. RU has won much much more than it ever needed to, well done

So I am 'pro-russia ', but the lines on my map, i assure you, will be less generous than what russia had taken -- all the while your lot will insist how right you are. Well done, they'll take hafl

Re: Bad Actors Are Grooming LLMs to Produce Falsehoods

#260

Post by const_cast » Sat, Jul 12, 2025, 10:25 PM UTC

wiseowise wrote: post #147

tempodox wrote: post #145

Earlier quoted context omitted.

> AI is the new crypto. But it's also way worse than cryptocurrencies, because all the big actors are pushing it relentlessly, with every marketing trick they know. They have to, because they invested insane amounts of money into snake oil and now they have to sell it in order to recover at least a fraction of their investments. And the amounts of energy wasted on this ultimately pointless performance are beyond stag…

In what parallel universe do you live where LLMs are snake oil?

It depends on how they're marketed and what's the prevailing opinion. If we believe LLMs are true, genuine intelligence then yes, I'd say that's snake oil.