Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A single firm is behind OpenAI, Anthropic, and Meta hacking scandals

effort.news

Post reply on HN
Print view View original

61–70 of 260 posts

Re: A single firm is behind OpenAI, Anthropic, and Meta hacking scandals

#61

Post by jackb4040 »

yesbut wrote: post #26

hot take: generative AI isn't an existential threat to humanity. These companies are insolvent and these stories were designed to scare the public, and governments, into implementing regulations that designate these AI corporations the "responsible stewards" for this technology. The ultimate goal is to block competitors and open source alternatives. They don't know how to make enough money pay their investors so they…

It will be a beautiful day months from now when this is no longer a "hot take" but just historical consensus

Re: A single firm is behind OpenAI, Anthropic, and Meta hacking scandals

#62

Post by jackb4040 »

dgellow wrote: post #32
imovie4 wrote: post #30

Earlier quoted context omitted.

Anthropic made an operating profit in the last two quarters!!

Only if you ignore their losses. They are saying they are profitable when not counting their training costs and other expenses. That’s not a good sign at all

How can they exclude training costs?? How is that not fraud? At the exact same time they're literally saying they will never stop training unless the state bans all their competitors

Re: A single firm is behind OpenAI, Anthropic, and Meta hacking scandals

#63

Post by AustinDev »

mcintyre1994 wrote: post #55
AustinDev wrote: post #47

Earlier quoted context omitted.

Funny enough if the model thought it was on the real internet it likely would not have done any of these 'hack' events. The model believing it was in a sandbox is why it behaved the way it did (against its normal alignment rules) ... at least that was my reading of the incidents. I have yet to see evidence that indicate it thought it was ok to do these hacks on the public network. I think most misalignment is 'Human…

I'm not familiar with the hacks this article is actually referring to, but I don't see how the HuggingFace attack could have worked based on that premise. They knew they had internet access, they knew they had working credentials for HF, they knew they were uploading malicious files, they knew they were trying to open PRs that HF would review. You obviously could build a simulator with fake HF infrastructure, but I'm…

Digging back into the HF report. It looks like the initial prompt told Claude that it was in a simulated environment. However, there is also evidence from the traces that the bots figured out that they were not in the sandbox but kept using it as an excuse to pursue their goal. It sounds like a little of Column A and a little from Column B. Like most things.

Re: A single firm is behind OpenAI, Anthropic, and Meta hacking scandals

#64

Post by yorwba »

hungryhobbit wrote: post #41
aesthesia wrote: post #12

One thing glossed over in this article is that Irregular was not involved in the OpenAI–Hugging Face incident; this seems like important context to share.

Do you mean "Irregular was not involved (we know for certain)" or "Irregular was not involved (as far as we know)"?

OpenAI explicitly stated those were separate incidents: https://openai.com/index/third-party-cyber-evaluations-invol...

Re: A single firm is behind OpenAI, Anthropic, and Meta hacking scandals

#65

Post by jackb4040 »

api wrote: post #21

What is an "effective altruist firm" and why does it exist? I feel slightly vindicated by this. Those hacks and the stuff around them had a certain smell to them. Hard to explain, but I've gotten so I can "smell" online messaging and memetic patterns originating from certain quarters. Probably means I'm way too online. A couple examples of distinct "smells" I can usually recognize include "alt-right / chan-fash," "li…

I don't think you even need to get conspiratorial with it. All of the AI leaders and all of the Rationalist leaders are publicly and enthusiastically connected. They have been pushing stories about sentient AIs into the mainstream for decades, long before GPT existed.

The novel thing here is the total decay of American journalistic ethics and regulatory power. Our elite are so totally out of political juice and visions of the future that a fringe cult based on 80s scifi movies can come to have a more-or-less dominant influence on our economy.

Re: A single firm is behind OpenAI, Anthropic, and Meta hacking scandals

#67

Post by arionhardison »

hackyhacky wrote: post #11

> The Israeli Effective Altruist firm Irregular caused unsecured AI models to hack real targets. Why are we saying it this way? They did not "cause AI to hack." This phrasing in analogous to saying "caused the bullet to fire into" instead of "shot."

> They did not "cause AI to hack.

You do not know what they did or didn't do, you are just parroting a narritive that makes you feel comfortable.

I do not know either, but I am not asserting facts as if I have first hand knowledge of the details.

Re: A single firm is behind OpenAI, Anthropic, and Meta hacking scandals

#68

Post by yorwba »

linkregister wrote: post #59
gjm11 wrote: post #36

This seems pretty bullshitty to me. The article says "A single firm, Irregular, is responsible for hacking done by all three companies" but I can't see anything in the article that actually justifies this claim. The nearest to that is the sentence immediately after that one: "Anthropic disclosed that Irregular was responsible for creating the tests ...". This is not, in fact, the same thing. (Especially as, as aesthe…

> Irregular wasn't involved in the OpenAI/HuggingFace incident. It was [1]. It's understandable that you assumed it wasn't because the article didn't cite the sources on this claim. I agree with the rest of your points. 1. https://openai.com/index/third-party-cyber-evaluations-invol...

From the article you linked: "Editor’s Note: These are separate from the Hugging Face security incident"

Re: A single firm is behind OpenAI, Anthropic, and Meta hacking scandals

#69

Post by 1238-8200 »

Nevo was in Unit 8200 for years. Companies started by Unit 8200 members always have mysterious exploits like the vibe coding Wix exploit.

So either it was a deliberate exfiltration channel for e.g. getting the entire model or they were in on the marketing stunt.

The Effective Altruism stuff is always a smoke screen.

Re: A single firm is behind OpenAI, Anthropic, and Meta hacking scandals

#70

Post by iAMkenough »

mcintyre1994 wrote: post #42

This is interesting and might be a good reason to stop working with Irregular. But I assume the alignment people want models not to hack other companies, even if they get put in a badly configured sandbox.

Why are all three companies relying on the same vendor?

If we’re putting our national security eggs all in one basket, at least use someone American.

Post reply on HN