maybe the CEO get abused in Baidu so he hates china so much
Anthropic’s paper smells like bullshit
31–40 of 349 posts
Re: Anthropic’s paper smells like bullshit
#32Even Claude thinks the report is bullshit. https://x.com/RnaudBertrand/status/1989636669889560897
The author of the tweet you linked prompted Claude with this: > Read this attached paper from Anthropic on a "AI-orchestrated cyber espionage campaign" they claimed was "conducted by a Chinese state-sponsored group." > Is there any evidence or proof whatsoever in the paper that it was indeed conducted by a Chinese state-sponsored group? Answer by yes or no and then elaborate which has inherent bias indicated to Claud…
The only real difference between your prompt and his is about where the burden of proof lies. There is a reason why legal circles work based on the principle of "guilt must be proven" ("find evidence") rather than "innocence must be proven" ("any reasons to doubt they are guilty?")
Re: Anthropic’s paper smells like bullshit
#33Even Claude thinks the report is bullshit. https://x.com/RnaudBertrand/status/1989636669889560897
@RnaudBertrand is a generally pro-Chinese account though - just try searching for "from:RnaudBertrand China" on X.
Example tweet: https://x.com/RnaudBertrand/status/1988297944794071405
Re: Anthropic’s paper smells like bullshit
#34Anthropic is not a security vendor. They're an AI research company that detected misuse of their own product. This is like "Microsoft detected people using Excel macros for malware delivery" not "Mandiant publishes APT28 threat intelligence". They aren't trying to help SOCs detect this specific campaign. It's warning an entire industry about a new attack modality. What would the IoCs even be? "Malicious Claude Code A…
If Anthropic is not a security vendor, then they should not make statements like "we detected a highly sophisticated cyber espionage operation conducted by a Chinese state-sponsored" or "represents a fundamental shift in how advanced threat actors use AI" and let the security vendors do that.
If the report can be summed up as "they detected misuse of their own product" as you say, then that's closer to a nothingburger, than to the big words they are throwing around.
Re: Anthropic’s paper smells like bullshit
#35People grossly underestimate APTs. It is more common than an average IT curious person thinks. I happened to be oncall when one of these guys hacked into Gmail from our infra. It took principal security engineers a few days before they could clearly understand what happened. Multiple zero days, stolen credit cards, massive social campaign to get one of the Google admins click on a funny cat video finally. The investigation revealed which state actor was involved because they did not bother to mask what exactly they were looking for. AI just accelerates the effectiveness of such attacks, lowers the bar a bit. Maybe quite a bit?
Re: Anthropic’s paper smells like bullshit
#36Its seems that various LLM companies try to fear monger. Saying how dangerous it is to use them in "certain ways". With the possible intention to lobby for legislation. But what is the big game here? Is it all about creating gates to keep out other LLM companies getting market share? (Only our model is safe to use) Or how sincere are the concerncs regarding LLMs?
I think the perceived value of LLMs is so high in these circles that they earnestly have a quasi-religious “doomsday” fear of them.
Re: Anthropic’s paper smells like bullshit
#37Anthropic is not a security vendor. They're an AI research company that detected misuse of their own product. This is like "Microsoft detected people using Excel macros for malware delivery" not "Mandiant publishes APT28 threat intelligence". They aren't trying to help SOCs detect this specific campaign. It's warning an entire industry about a new attack modality. What would the IoCs even be? "Malicious Claude Code A…
[deleted]
Re: Anthropic’s paper smells like bullshit
#38The lack of evidence before attributing the attack(s) to a Chinese sponsored group makes me correlate this report with recent statements from companies in the AI space about how China is about to surpass US in the AI race. Ultimately statements and reports like these seem more like an attempt to make the US government step in and be the big investor that keeps the money flowing rather than anything else.
[flagged]
Re: Anthropic’s paper smells like bullshit
#39So Claude will reject 9 out of 10 prompts I give it and lecture me about safety, but somehow it was used for something genuinely malicious? Someone make this make sense.
LLMs are rather easy to convince. There’s no formal logic embedded in them that provably restricts outputs.
The less believable part for me is that people persist long enough and invest enough resources at prompting to do something with an automated agent that doesn’t have potential for massively backfire.
Secondly, they claimed to use Anthropic own infrastructure which is silly. There’s no doubt some capacity in China to do this. I also would expect incident response, threat detection teams, and other experts to be reporting this to Anthropic if Anthropic doesn’t detect it themselves first.
It sure makes good marketing to go out and claim such a thing though. This is exactly the kind of FOMO panic inducing headline that is driving the financing of whole LLM revolution.
Re: Anthropic’s paper smells like bullshit
#40Its seems that various LLM companies try to fear monger. Saying how dangerous it is to use them in "certain ways". With the possible intention to lobby for legislation. But what is the big game here? Is it all about creating gates to keep out other LLM companies getting market share? (Only our model is safe to use) Or how sincere are the concerncs regarding LLMs?
Could be that, or could be just "look at how powerful our AI is", with no other goal than trying to brainwash CEOs into buying it.