Why is Anthropic not legally responsible for damages here?
Disrupting the first reported AI-orchestrated cyber espionage campaign
191–200 of 298 posts
Re: Disrupting the first reported AI-orchestrated cyber espionage campaign
#192Earlier quoted context omitted.
Worst local (Australia) example of that Following a public statement by Hansford about his use of Microsoft's AI chatbot Copilot, Crikey obtained 50 documents containing his prompts... FOI logs reveal Australia's national security chief, Hamish Hansford, used the AI chatbot Copilot to write speeches and messages to his team. (subscription required for full text): https://www.crikey.com.au/2025/11/12/australia-nationa…
Holy crap that is such a bad look. That guy should immediately step down and if he doesn't he should be let go.
That wasn’t my first thought. My first thought was; every senior executive everywhere is probably doing the same thing.
Re: Disrupting the first reported AI-orchestrated cyber espionage campaign
#193I understand the public and suits falling for this, but cmon
Re: Disrupting the first reported AI-orchestrated cyber espionage campaign
#194> At this point they had to convince Claude—which is extensively trained to avoid harmful behaviors—to engage in the attack. They did so by jailbreaking it, effectively tricking it to bypass its guardrails. They broke down their attacks into small, seemingly innocent tasks that Claude would execute without being provided the full context of their malicious purpose. They also told Claude that it was an employee of a l…
It's not even exclusive to LLMs. Giving humans seemingly innocent tasks that combine to a malicious whole, or telling humans that they work for a security organization while working for a crime organization, are hardly new concepts. The only really novel thing is that with humans you need a lot of them because a single human would piece together that the innocent tasks add up to a not-so-innocent whole. LLMs are esse…
> Giving humans seemingly innocent tasks that combine to a malicious whole
Isn't this the plot of the The Cube!?
Re: Disrupting the first reported AI-orchestrated cyber espionage campaign
#195> At this point they had to convince Claude—which is extensively trained to avoid harmful behaviors—to engage in the attack. They did so by jailbreaking it, effectively tricking it to bypass its guardrails. They broke down their attacks into small, seemingly innocent tasks that Claude would execute without being provided the full context of their malicious purpose. They also told Claude that it was an employee of a l…
Guardrails for anything versatile might be trivial on consideration.
As a kid I read some Asimov books where he laid out the "3 laws of robotics", first law being a robot must not harm a human. And in the same story a character gave the example of a malicious human instructing Robot A prepare a toxic solution "for science", dismissing Robot A, then having Eobot B unsuspectingly serve the "drink" to a victim. Presto, a robot killing a human. The parallel to malicious use of LLMs has been haunting me for ages.
But here's the kicker, Iirc, Asimov wasn't even really talking about robots. His point was how hard it is to align humans, for even perfectly morally upright humans to avoid being used to harm others.
Re: Disrupting the first reported AI-orchestrated cyber espionage campaign
#196Earlier quoted context omitted.
reminds me of the YouTube ads I get that are like "Warning: don't do this new weight loss trick unless you have to lose over 50 pounds, you will end up losing too much weight!". As if it's so effective it's dangerous.
I remain convinced the steady steam of OpenAI employees who allegedly quit because AI was "too dangerous" for a couple months was an orchestrated marketing campaign as well.
I just had 5.1 do something incredibly brain dead in "extended thinking" mode because I know what I asked it is not in the training data. So it just fudged and made things up because thinking is exactly what it can not do.
It seems like LLMs are at the same time a giant leap in natural language processing, useful in some situations and the biggest scam of all time.
Re: Disrupting the first reported AI-orchestrated cyber espionage campaign
#197> At this point they had to convince Claude—which is extensively trained to avoid harmful behaviors—to engage in the attack. They did so by jailbreaking it, effectively tricking it to bypass its guardrails. They broke down their attacks into small, seemingly innocent tasks that Claude would execute without being provided the full context of their malicious purpose. They also told Claude that it was an employee of a l…
[deleted]
Re: Disrupting the first reported AI-orchestrated cyber espionage campaign
#198> At this point they had to convince Claude—which is extensively trained to avoid harmful behaviors—to engage in the attack. They did so by jailbreaking it, effectively tricking it to bypass its guardrails. They broke down their attacks into small, seemingly innocent tasks that Claude would execute without being provided the full context of their malicious purpose. They also told Claude that it was an employee of a l…
I wonder how hard it would be for Claude to give me someone's mother's maiden name. Seems LLMs may be infinitely susceptible to social engineering.
When the new "memory" feature launched I asked it what it knew about me and it gave me an uncomfortable amount of detail about someone else, who I was even able to find on LinkedIn.
Re: Disrupting the first reported AI-orchestrated cyber espionage campaign
#199Earlier quoted context omitted.
It's marketing, but if it's the truth, isn't it a public good to release information about this? Like if someone tried to break into your house, it would be "gloating" to say your advanced security system stopped it while warning people about the tactics of the person who tried to break in.
If in the next page over you sell advanced security systems yes it'd be suspicious and weird, which is the case here.
They’re not allowed to market their product on their own website blog? That includes half of all company blog posts ever on here
Re: Disrupting the first reported AI-orchestrated cyber espionage campaign
#200Why is Anthropic not legally responsible for damages here?
Having people making tools be responsible what their users do with them is not a just system that blames the person that is really responsible. Even if you cannot locate or identify that person.
Sometimes arguments can be made if a tool is very dangerous, but liability should stay where it belongs.