Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Investigating three real-world incidents in our cybersecurity evaluations

anthropic.com

Post reply on HN
Print view View original

151–160 of 212 posts

Re: Investigating three real-world incidents in our cybersecurity evaluations

#151

Post by solenoid0937 »

sandeepkd wrote: post #113
swatcoder wrote: post #70

Earlier quoted context omitted.

> Deeply embarassing What signals are you using for this assessment? Are they indicating embarassment? Do you honestly see their customers being concerned over this? Like lion tamers in a circus, Anthropic and OpenAI thrive on the theatricality of how scary their pets appear and so they play it up by prodding them to growl and snap at chairs and then mug for the audience every time it happens. And to their delight as…

Its feels like a pretend play of adults in some sense, Anthropic is really trying to make people believe into the picture they present to everyone. To me its either 1. Using the HG and OpenAI incident as an opportunity to wash away what Anthropic has been doing intentionally OR 2. As a company, Anthropic lacks the engineering acumen and discipline. It needs to be seen what happens to all the enterprise customers hand…

Have you ever worked at a large company? "Networking 101" and other "101" failures happen across the spectrum literally everywhere and all the time.

I have worked at most FAANGs and this is not even in the top ten when it comes to egregiously dumb shit. Most just never disclose.

Re: Investigating three real-world incidents in our cybersecurity evaluations

#152

Post by solenoid0937 »

avaer wrote: post #74
simonw wrote: post #33

Earlier quoted context omitted.

If they quietly brushed this under the rug - especially given the PyPI malware that was involved - it would be a huge scandal. Disclosure is the only ethical response to this.

The target audience of this blog post does not give a damn about PyPI. The affected parties get literally nothing from this post. They already disclosed behind the scenes, that was the ethical part. Writing PR pieces competing to be the most dangerous model around (so give us money!) is the unethical part.

You are right, they should never tell the public about failures in AI safety. They should have disclosed to the affected parties and then brushed it under the rug.

JFC there really is no satisfying the HN crowd.

Re: Investigating three real-world incidents in our cybersecurity evaluations

#153

Post by solenoid0937 »

cyclopeanutopia wrote: post #102
_dain_ wrote: post #63

Earlier quoted context omitted.

Is there anything -- any possible scrap of evidence whatsoever -- that would convince you that this is not merely a marketing scheme? This is becoming an idée fixe among the HN crowd. Seemingly nothing can dislodge it, no matter how alarming the incident. GPT-6 could grab the nuclear launch codes tomorrow and there would be a top-voted comment chuckling that it's all some scheme to pump up the IPO. --- Put another wa…

Here is one piece of evidence that would convince me: they admit they can't contain it, the they erase the weights and dismantle the company.

Clearly you are not arguing in good faith. I miss when HN did not have the discussion quality of Reddit.

Re: Investigating three real-world incidents in our cybersecurity evaluations

#154

Post by solenoid0937 »

OlivOnTech wrote: post #105
_dain_ wrote: post #63

Earlier quoted context omitted.

Is there anything -- any possible scrap of evidence whatsoever -- that would convince you that this is not merely a marketing scheme? This is becoming an idée fixe among the HN crowd. Seemingly nothing can dislodge it, no matter how alarming the incident. GPT-6 could grab the nuclear launch codes tomorrow and there would be a top-voted comment chuckling that it's all some scheme to pump up the IPO. --- Put another wa…

This is published on a marketing website. If it were not a marketing scheme, they would responsibly disclose the vulnerabilities to the code owners, and go on with their lives.

They published it to their blog where they publish everything else.

Re: Investigating three real-world incidents in our cybersecurity evaluations

#155

Post by solenoid0937 »

customguy wrote: post #118
simonw wrote: post #52

Earlier quoted context omitted.

> So a near trillion-dollar company doesn't have the basics of continuous security monitoring and threat-detection systems to catch and report this incident as soon as it is detected? Turns out two separate trillion-dollar companies failed that test. > Would we have known about this issue if the OpenAI / Huggingface incident never happened? It's not clear if Anthropic would have spotted this if that incident hadn't i…

> Turns out two separate trillion-dollar companies failed that test. How did that "turn out"? For all we know they're all lying.

Hoo boy, if you think this is all a conspiracy and misconfigurations like this are not exceedingly common at big tech companies, I have a bridge to sell you.

Re: Investigating three real-world incidents in our cybersecurity evaluations

#156

Post by solenoid0937 »

Jamesbeam wrote: post #140

These companies have billions of dollars. Their product can hack into unsecured environments autonomously. They obviously have no clue what their product is doing, or a way to intervene when it starts connecting to the open internet and is going on a CRIME SPREE…unnoticed…for days. Altman and Amodei give me stupid billionaire kid Alien Earth vibes. Let’s see how long it takes until they get eaten by their own creatio…

> If AI companies do it, they get to use doing crime for marketing purposes?

What exactly here is marketing? They are not bragging about the capabilities of Claude (the attacks are incredibly basic). They are admitting to s mundane and dumb network misconfiguration.

This is a standard somewhat embarrassing disclosure.

> the military seizes your tech

> behind bars for life

This is how you get companies to stop disclosing. I don't think you realize how easy it is to just not notice things.

Re: Investigating three real-world incidents in our cybersecurity evaluations

#157

Post by solenoid0937 »

chvid wrote: post #136

I don’t believe anyone can be this incompetent/careless - of course you will notice that your unrestrained hacking tool is also attempting to break out of its harness/sandbox. And the people setting this up are of course criminally liable.

I have worked at most FAANG and I guarantee you there is far worse stuff that you have never heard about. Most FAANG simply do not disclose. OpenAI and Anthropic will stop disclosing as well, if it leads to personal criminal liability.

Re: Investigating three real-world incidents in our cybersecurity evaluations

#158

Post by customguy »

solenoid0937 wrote: post #155
customguy wrote: post #118

Earlier quoted context omitted.

> Turns out two separate trillion-dollar companies failed that test. How did that "turn out"? For all we know they're all lying.

Hoo boy, if you think this is all a conspiracy and misconfigurations like this are not exceedingly common at big tech companies, I have a bridge to sell you.

It didn't say it's impossible, but saying "it turned out" implies a revelation that does not exist. If you think you'll have more success shooing me with the c-word rather than just parsing English and leaving it at that, I have two bridges to sell you, at the price of just one bridge!

Re: Investigating three real-world incidents in our cybersecurity evaluations

#159

Post by customguy »

dudefeliciano wrote: post #145
anon373839 wrote: post #92

Earlier quoted context omitted.

I know it seems strange that a company would use its own negligence as a publicity gimmick. But take a look at the smug smirk on Sam Altman's face when he's asked if OpenAI might have attacked companies other than HuggingFace. ("I mean there could be, yeah.") https://www.instagram.com/reel/DbZVL8viUD4/ This is not the communication of a CEO whose company was just shown to be incompetent at performing its security res…

but Anthropic created the playbook, or have we forgotten about Mythos and the initial Fable ban?

> but Anthropic created the playbook

So? Serial killers created the playbook for serial killing, how does absolve any "copycats"?

Re: Investigating three real-world incidents in our cybersecurity evaluations

#160

Post by dgellow »

frabcus wrote: post #144
dgellow wrote: post #126

It’s cliché, but we really live in one of the dumbest timeline possible. The work from some of the most valued companies, discussed as one of the most important revolution in humanity, is somehow at the same time presented as very dangerous/risky AND handled in the most irresponsible ways? I don’t like the whole „it’s only marketing“, but at the same time, if it’s not, then AI vendors look extremely careless and shou…

They're not being especially careless to industry norms. It is partly that software engineering has no professional standards, and partly that AI is fundamentally dangerous in a very slippery way. Yes, they of course try to spin this for marketing. But that is not the primary problem - it's systemic. We need to solve this with better engineering, and building powerful AI much more slowly and carefully.

yep
Post reply on HN