Anthropic’s paper smells like bullshit

Post by saagarjha » Sun, Nov 16, 2025, 9:47 PM UTC

sim7c00 wrote: post #188

jnwatson wrote: post #185

There's a big gap of knowledge between infosec researchers and ML security researchers. Anthropic has a bunch of column B but not enough column A. This was discussed in some detail in the recently published Attacker Moves Second paper*. ML researchers like using Attack Success Rate (ASR) as a metric for model resistance to attack, while for infosec, any successful attack (ASR > 0) is considered significant. ML resear…

ML researchers are not sec researchers. they need to stick to their own game. companies need to use both camps for a good holistic view of the problem. ML is the blue team. sec researchers the red.

Post by saagarjha » Sun, Nov 16, 2025, 9:51 PM UTC

fifhtbtbf wrote: post #201

CuriouslyC wrote: post #96

Earlier quoted context omitted.

The hot mess that is Claude Code (if you multi-orchestrate with it, it'll start to grind even very powerful systems to a halt, 15+ seconds of unresponsiveness, all because CC constantly serializes/deserializes a JSON data file that grows quite large every time you do stuff), their horrible service uptime compared to all their competitors, their month long performance degradation their users had to scream at them to g…

I have the opposite perception: they’re the only company in the space that seems to have a clue what responsible software engineering is. Gemini Code and Cursor both did such a poor job sandboxing their agents that the exploits sound like punchlines, while Microsoft doesn’t even try with Copilot Agentic. Countless Cursor bugs have been fixed with obviously vibe-coded fake solutions (you can see if you poke into code…

Post by scrps » Sun, Nov 16, 2025, 9:53 PM UTC

skybrian wrote: post #205

jmkni wrote: post #58

That whole article felt like "Claude is so good Chinese hackers are using it for espionage" marketing fluff tbh

If we’re sharing vibes, “our product is dangerous” seems like an unusual sales tactic outside the defense industry. I’m doubtful that’s how it works? Meanwhile, another reason to make a press release is that you’ll be criticized for the coverup if you don’t. Also, it puts other companies on notice that maybe they should look for this?

Post by hopelite » Sun, Nov 16, 2025, 9:56 PM UTC

metacritic12 wrote: post #225

prinny_ wrote: post #20

The lack of evidence before attributing the attack(s) to a Chinese sponsored group makes me correlate this report with recent statements from companies in the AI space about how China is about to surpass US in the AI race. Ultimately statements and reports like these seem more like an attempt to make the US government step in and be the big investor that keeps the money flowing rather than anything else.

Anthropic has also been the biggest anti-China LLM in a long while, so it's possible they're using an opportunistic hack (potentially involving actual Chinese IP addresses) as another way to push their agenda.

Post by mrtesthah » Sun, Nov 16, 2025, 10:02 PM UTC

iterateoften wrote: post #117

KaiserPro wrote: post #28

When I worked at a FAANG with a "world leading" AI lab (now run by a teenage data labeller) as an SRE/sysadmin I was asked to use a modified version of a foundation model which was steered towards infosec stuff. We were asked to try and persuade it to help us hack into a mock printer/dodgy linux box. It helped a little, but it wasn't all that helpful. but in terms of coordination, I can't see how it would be useful.…

> you're API is tied to a bankaccount, There are a lot of middlemen like open router who gladly accept crypto.