Live data from Hacker News

Anthropic’s paper smells like bullshit

djnn.sh

91–100 of 349 posts

Re: Anthropic’s paper smells like bullshit

#91
Tldr.

Anthropic made a load of ubsubstantiated accusations about a new problem they dont specify.

Then at the end Anthropic proposed the solution to this unspecified problem is to give anthropic money.

Completely agree that is promotional material masquerading as a threat report of no material value.

Re: Anthropic’s paper smells like bullshit

#92
post #20

The lack of evidence before attributing the attack(s) to a Chinese sponsored group makes me correlate this report with recent statements from companies in the AI space about how China is about to surpass US in the AI race. Ultimately statements and reports like these seem more like an attempt to make the US government step in and be the big investor that keeps the money flowing rather than anything else.

Do public reports like this one often go deep enough into the weeds to name names, list specific tools and techniques, URLs? I don't doubt of course that reports intended for government agencies or security experts would have those details, but I am not surprised that a "blog post" like this one is lacking details. I just don't see how one goes from "this is lacking public evidence" to "this is likely a political stu…

> Do public reports like this one often go deep enough into the weeds to name names

Yes. They often include IoCs, or at the very least, the rationale behind the attribution, like "sharing infrastructure with [name of a known APT effort here]".

For example, here is a proper decade-old report from the most unpopular country right now: https://media.kasperskycontenthub.com/wp-content/uploads/sit...

It established solid technical links between the campaign they are tracking to earlier, already attributed campaigns.

So, even our enemy got this right, ten years ago, there really is no excuse for this slop.

Re: Anthropic’s paper smells like bullshit

#93

Does Anthropic currently have cybersec people able to provide a standard assessment of the kind the community expects? This could be a corporate move as some people claim, but I wonder if the cause is simply that their talents are currently somewhere else and they don’t have the company structure in place to deliver properly in this matter. (If that is the case they are not then free of blame, it’s just a different c…

I throw Anthropic under the bus a lot for their lack of engineering acumen. If they don't have a core competency like engineering fully covered, I'd say there's a near 0% chance they have something like security covered.

What makes you think they lack engineering acumen?

Re: Anthropic’s paper smells like bullshit

#95
post #65

People grossly underestimate APTs. It is more common than an average IT curious person thinks. I happened to be oncall when one of these guys hacked into Gmail from our infra. It took principal security engineers a few days before they could clearly understand what happened. Multiple zero days, stolen credit cards, massive social campaign to get one of the Google admins click on a funny cat video finally. The investi…

A lot of people behind APTs are low-skilled and make silly mistakes. I worked for a company that investigates traces of APTs, they make very silly mistakes all the time. For example, oftentimes (there are tens of cases) they want to download stuff from their servers, and they do it by setting up an HTTP server that serves the root folder of a user without any password protection. Their files end up indexed by crawler…

The security world overemphasizes (fetishizes, even,) the "advanced" part because zero days and security tools to compensate against zero days are cool and fun, and underemphasizes the "persistent" part because that's boring and hard work and no fun.

And, unless you are Rob Joyce, talking about the persistent part doesn't get you on the main stage at a security conference (e.g., https://m.youtube.com/watch?v=bDJb8WOJYdA)

Re: Anthropic’s paper smells like bullshit

#96

Earlier quoted context omitted.

I throw Anthropic under the bus a lot for their lack of engineering acumen. If they don't have a core competency like engineering fully covered, I'd say there's a near 0% chance they have something like security covered.

What makes you think they lack engineering acumen?

The hot mess that is Claude Code (if you multi-orchestrate with it, it'll start to grind even very powerful systems to a halt, 15+ seconds of unresponsiveness, all because CC constantly serializes/deserializes a JSON data file that grows quite large every time you do stuff), their horrible service uptime compared to all their competitors, their month long performance degradation their users had to scream at them to get them to investigate, the fact that they had to outsource their web client and it's still bad, etc.

Re: Anthropic’s paper smells like bullshit

#97
>This involved querying internal services, extracting authentication certificates from configurations, and testing harvested credentials across discovered systems.

How ? Did it run Mimikatz ? Did it access Cloud environments ? We don’t even know what kind of systems were affected.

I really don't see what is so difficult to believe since the entire incident can be reduced to something that would not typically be divulged by any company at all, as it is not common practice for companies to divulge every single time the previously known methodologies have been used against them. Two things are required for this:

1) Jailbreak Claude from guardrails. This is not difficult. Do people believe advancement with guardrails are so hardened through fine tuning it's no longer possible?

2) The hackers having some of their own software tools for exploits that Claude can use. This too is not difficult to credit.

Once an attacker has done this all Claude is doing is using software in the same mundane fashion as it does every time you use Claude code and it utilizes any tools to which you give it access.

I used a local instance of Qwen3 coder (A3B 30B quantized to IQ3_xxs) literally yesterday through ollama & cline locally. With a single zeroshot prompt it wrote the code to use the arxiv API and download papers using its judgement on what was relevant to split the results into a subset that met the criteria I gave for the sort I wanted to review.

Given these sorts of capabilities why is it difficult the believe this can be done using the hacker's own tools and typical deep research style iteration? This is described in in the research paper, and disclosing anything more specific is unnecessary because there is nothing novel to disclose.

As for not releasing the details, they did: Jailbreak Claude. Again, nothing they described is novel such that further details are required. No PoC is needed, Claude isn't doing anything new. It's fully understandable that Anthropic isn't going to give the specific prompts used for the obvious reason that even if Anthropic has hardened Claude against those, even the general details would be extremely useful to iterate and find workarounds.

For detecting this activity and determining how Claude was doing this it's just a matter of monitoring chat sessions in such a way as to detect jail breaks, which again is very much not novel or an unknown practice by AI providers.

Especially in the internet's earlier days of the internet it was amusing (and frustrating) to see some people get very worked up every time someone did something that boiled down to "person did something fairly common, only they did it using the internet." This is similar except its "but they did it with AI,"

Re: Anthropic’s paper smells like bullshit

#98

When I worked at a FAANG with a "world leading" AI lab (now run by a teenage data labeller) as an SRE/sysadmin I was asked to use a modified version of a foundation model which was steered towards infosec stuff. We were asked to try and persuade it to help us hack into a mock printer/dodgy linux box. It helped a little, but it wasn't all that helpful. but in terms of coordination, I can't see how it would be useful.…

If the article is not just marketing fluff, I assume a bad actor would select Claude not because it’s good at writing attacks, instead a bad actor code would choose it because Western orgs chose Claude. Sonnet is usually the go-to on most coding copilot because the model was trained on good range of data distribution reflecting western coding patterns. If you want to find a gap or write a vulnerability, use the same tool that has ingested patterns that wrote code of the systems you’re trying to break. Or use Claude to write a phishing attack because then output is more likely similar to what our eyes would expect.

Re: Anthropic’s paper smells like bullshit

#99

"A report was recently published by an AI-research company called Anthropic. They are the ones who notably created Claude, an AI-assistant for coding. Personally, I don’t use it but that is besides the point." Not sure if the author has tried any other AI-assistants for coding. People who haven't tried coding AI assistant underestimates its capabilities (though unfortunately, those who use them overestimate what they…

The author’s arguments explicitly don’t dispute plausibility. It accurately states that mere plausibility is a misleading basis for this report, but that the report provides nothing but plausibility, and thus is of low quality and dubious motivation.

Anthropic’s lack of any evidence for their claims doesn’t require any position on AI agent capability at all.

Think better.

Re: Anthropic’s paper smells like bullshit

#100

When I worked at a FAANG with a "world leading" AI lab (now run by a teenage data labeller) as an SRE/sysadmin I was asked to use a modified version of a foundation model which was steered towards infosec stuff. We were asked to try and persuade it to help us hack into a mock printer/dodgy linux box. It helped a little, but it wasn't all that helpful. but in terms of coordination, I can't see how it would be useful.…

> now run by a teenage data labeller

sick burn

Post reply on HN