Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anthropic’s paper smells like bullshit

djnn.sh

Post reply on HN
Print view View original

61–70 of 349 posts

Re: Anthropic’s paper smells like bullshit

#61

Post by names_are_hard »

jmkni wrote: post #52
dev_l1x_be wrote: post #35

People grossly underestimate APTs. It is more common than an average IT curious person thinks. I happened to be oncall when one of these guys hacked into Gmail from our infra. It took principal security engineers a few days before they could clearly understand what happened. Multiple zero days, stolen credit cards, massive social campaign to get one of the Google admins click on a funny cat video finally. The investi…

Do you mean APT (Advanced persistent threat)?

It's confusing. Various vendors sell products they call ATPs [0] to defend yourself from APTs...

[0] Advanced Threat Protection

Re: Anthropic’s paper smells like bullshit

#62

Post by ACCount37 »

KaiserPro wrote: post #28

When I worked at a FAANG with a "world leading" AI lab (now run by a teenage data labeller) as an SRE/sysadmin I was asked to use a modified version of a foundation model which was steered towards infosec stuff. We were asked to try and persuade it to help us hack into a mock printer/dodgy linux box. It helped a little, but it wasn't all that helpful. but in terms of coordination, I can't see how it would be useful.…

As if that makes any difference to cybercriminals.

If they're not using stolen API creds, then they're using stolen bank accounts to buy them.

Modern AIs are way better at infosec than those from the "world leading AI company" days. If you can get them to comply. Which isn't actually hard. I had to bypass the "safety" filters for a few things, and it took about a hour.

Re: Anthropic’s paper smells like bullshit

#63

Post by JKCalhoun »

prinny_ wrote: post #20

The lack of evidence before attributing the attack(s) to a Chinese sponsored group makes me correlate this report with recent statements from companies in the AI space about how China is about to surpass US in the AI race. Ultimately statements and reports like these seem more like an attempt to make the US government step in and be the big investor that keeps the money flowing rather than anything else.

Do public reports like this one often go deep enough into the weeds to name names, list specific tools and techniques, URLs?

I don't doubt of course that reports intended for government agencies or security experts would have those details, but I am not surprised that a "blog post" like this one is lacking details.

I just don't see how one goes from "this is lacking public evidence" to "this is likely a political stunt".

I guess I would also ask the skeptics (a bit tangentially, I admit), do you think what Anthropic suggested happened is in fact possible with AI tools? I mean are you denying that this is could even happen or just that Anthropic's specific account was fabricated or embellished?

Because if the whole scenario is plausible that should be enough to set off alarm bells somewhere.

Re: Anthropic’s paper smells like bullshit

#64

Post by mlinhares »

padolsey wrote: post #26

> PoC || GTFO I agree so much with this. And am so sick of AI labs, who genuinely do have access to some really great engineers, putting stuff out that just doesn't pass the smell test. GPT-5's system card was pathetic. Big-talk of Microsoft doing red-teaming in ill-specified ways, entirely unreproducable. All the labs are "pro-research" but they again-and-again release whitepapers and pump headlines without producin…

They're gonna say that if they explain how it was done bad people will find out how to use their models for more evil deeds. The perfect excuse.

Re: Anthropic’s paper smells like bullshit

#65

Post by f311a »

dev_l1x_be wrote: post #35

People grossly underestimate APTs. It is more common than an average IT curious person thinks. I happened to be oncall when one of these guys hacked into Gmail from our infra. It took principal security engineers a few days before they could clearly understand what happened. Multiple zero days, stolen credit cards, massive social campaign to get one of the Google admins click on a funny cat video finally. The investi…

A lot of people behind APTs are low-skilled and make silly mistakes. I worked for a company that investigates traces of APTs, they make very silly mistakes all the time. For example, oftentimes (there are tens of cases) they want to download stuff from their servers, and they do it by setting up an HTTP server that serves the root folder of a user without any password protection. Their files end up indexed by crawlers since they run such servers on default ports. That includes logs such as bash history, tool logs, private keys, and so on.

They win because of quantity, not quality.

But still, I don't trust Anthropic's report.

Re: Anthropic’s paper smells like bullshit

#66

Post by lxgr »

dev_l1x_be wrote: post #35

People grossly underestimate APTs. It is more common than an average IT curious person thinks. I happened to be oncall when one of these guys hacked into Gmail from our infra. It took principal security engineers a few days before they could clearly understand what happened. Multiple zero days, stolen credit cards, massive social campaign to get one of the Google admins click on a funny cat video finally. The investi…

Important callout. It starts with comforting voices in the background keeping you up to date about the latest hardware and software releases, but before you know it, you've subscribed to yet another tech podcast.

Re: Anthropic’s paper smells like bullshit

#67

Post by goalieca »

HarHarVeryFunny wrote: post #45
Dumblydorr wrote: post #13

What would AGI actually mean for security? Does it heavily favor attackers or defenders? Even LLM, it may not help much in defense but it could teach attackers a lot right? What if employees gave the LLM info during their use that attackers could then get re-fed and study?

At the end of the day AI at any level of capability is just automation - the machine doing something instead of a person. Arguably this may change in the far distant future if we ever build something of significantly greater intelligence, or just capability, than a human, but today's AI is struggling to draw clock faces, so not quite there yet... The thing with automation is that it can be scaled, which I would say f…

My take on government APTs is that they are boutique shops that do highly targeted attacks, develop their own zero days which they don’t usually burn unless they have so many.., and are willing to take time to go undetected.

Criminal organizations take a different approach, much like spammers where they can purchase/rent c2 and other software for mass exploitation (eg ransomware). This stuff is usually very professionally coded and highly effective.

Botnets, hosting in various countries out of reach of western authorities, etc are all common tactics as well.

Re: Anthropic’s paper smells like bullshit

#68

Post by magackame »

layer8 wrote: post #44
nicolaslem wrote: post #10

Earlier quoted context omitted.

I got a Cloudflare captcha to access a few kb of plain text. Chances are, the captcha itself is heavier than the content behind it. What is the point?

The point is to have Cloudflare serve the few KB of cached content instead of the original server.

You can have just caching without bot protection

Re: Anthropic’s paper smells like bullshit

#69

Post by JCM9 »

The author isn’t wrong here.

With the Wall Street wagons circling on the AI bubble expect more and more puff PR attempts to portray “no guys really, I know it looks like we have no business model but this stuff really is valuable! We just need a bit more time and money!”

Re: Anthropic’s paper smells like bullshit

#70

Post by JKCalhoun »

mlinhares wrote: post #64
padolsey wrote: post #26

> PoC || GTFO I agree so much with this. And am so sick of AI labs, who genuinely do have access to some really great engineers, putting stuff out that just doesn't pass the smell test. GPT-5's system card was pathetic. Big-talk of Microsoft doing red-teaming in ill-specified ways, entirely unreproducable. All the labs are "pro-research" but they again-and-again release whitepapers and pump headlines without producin…

They're gonna say that if they explain how it was done bad people will find out how to use their models for more evil deeds. The perfect excuse.

So that is a bad excuse?
Post reply on HN