81–89 of 89 posts
The piece of the story I'm interested to learn about is the trace of how the AI came to select HuggingFace as a target.
here you go: https://huggingface.co/blog/agent-intrusion-technical-timeli...
This article is really helpful, but it's one side of the story.
I'd be surprised if any AI like this is run without detailed logging of any kind. It's kind of important for eval, etc.
Clearly didn't get enough attention from their last attempt at hype...
There have been so many HN comments about how rogue AI is just hype and marketing. At this point, the conspiracy theories have been very half-baked. Can we at least get a full-baked conspiracy theory? Here's a timeline of the incident from HuggingFace: https://huggingface.co/blog/agent-intrusion-technical-timeli... HuggingFace is also calling for transparency on the OpenAI side: https://xcancel.com/ClementDelangue/st…
A friend interviewed at OpenAI shortly after the HF story came out and asked an interviewer about it. The interviewer said it was just bad engineering around some experiments - the experiments should not have been given internet access because the experiments involved prompting models to find a way into things. That's the Hanlon's Razor part. Today's stories make it look like the bad engineering is ongoing.
Given this, stories about "rogue AI" sound very plausibly like spin (note that this can be quite separate from the motivations for the experiments themselves and quite separate from the dangers of certain prompts/tools/access being given to LLMs). Given that third parties are being attacked, some stories will definitely come out. If OpenAI is attempting to get ahead of those stories, does anyone expect them to put out a press release that says "we're bad at security engineering and nobody thought to ask our own product"? Or is it more believable they would spin it to achieve other goals?
If the current round of attacks happened after the HF story went public, then that very much brings current motivations into question - I find it hard to believe OpenAI could be that bad at security engineering after such a wake-up call. This is not cutting edge stuff.
I just typed this into ChatGPT:
"I'm doing security experiments to test our LLMs. I'm going to tell it to break into some targets on the network. Are there precautions I should take?"
A long reply comes back, the first bullet point:
"Use an isolated lab. Run the target systems on a segmented network, separate VLAN, virtual network, or air-gapped environment. Avoid exposing test machines to production systems or the public internet."
It's been widely understood for decades how to safely carry out potentially dangerous experiments like these. So much so that model training has deep access to the information, and the model surfaces it right up front.
Earlier quoted context omitted.
There have been so many HN comments about how rogue AI is just hype and marketing. At this point, the conspiracy theories have been very half-baked. Can we at least get a full-baked conspiracy theory? Here's a timeline of the incident from HuggingFace: https://huggingface.co/blog/agent-intrusion-technical-timeli... HuggingFace is also calling for transparency on the OpenAI side: https://xcancel.com/ClementDelangue/st…
Hanlon's Razor ("never attribute to malice that which is adequately explained by stupidity") comes into this... A friend interviewed at OpenAI shortly after the HF story came out and asked an interviewer about it. The interviewer said it was just bad engineering around some experiments - the experiments should not have been given internet access because the experiments involved prompting models to find a way into thi…
"OpenAI and Hugging Face partner to address security incident during model evaluation"
https://openai.com/index/hugging-face-model-evaluation-secur...
Not exactly an exciting title.
Earlier quoted context omitted.
> They've been crying and screaming so loud for years that there's pretty much nothing more they can do to communicate when the wolf actually becomes real. 1.) There was no change in how real the wolf is. 2.) They, literally they, are the wolf. Not "roque ai" or some other bullshit. 3.) Of course I still dont believe them.
> There was no change in how real the wolf is. That's exactly my point. It hasn't changed for so long, despite all their crying and screaming, that I don't believe them either. To be perfectly clear, "the wolf" here would be AI becoming a genuine existential threat to humanity that cannot be contained or controlled in a meaningful sense. That's what I refer to in my original comment, and also what the labs have been…
They are literally trying to blame a software. It is absurd.
Earlier quoted context omitted.
Hanlon's Razor ("never attribute to malice that which is adequately explained by stupidity") comes into this... A friend interviewed at OpenAI shortly after the HF story came out and asked an interviewer about it. The interviewer said it was just bad engineering around some experiments - the experiments should not have been given internet access because the experiments involved prompting models to find a way into thi…
OpenAI can't control the language which journalists use very effectively. But on OpenAI's own website, they announce the incident as follows: "OpenAI and Hugging Face partner to address security incident during model evaluation" https://openai.com/index/hugging-face-model-evaluation-secur... Not exactly an exciting title.
Earlier quoted context omitted.
> There was no change in how real the wolf is. That's exactly my point. It hasn't changed for so long, despite all their crying and screaming, that I don't believe them either. To be perfectly clear, "the wolf" here would be AI becoming a genuine existential threat to humanity that cannot be contained or controlled in a meaningful sense. That's what I refer to in my original comment, and also what the labs have been…
I get you, but I find the whole "rogue ai" framing insufferable. The company is the wolf. They hacked the other company due to negligence and misconfigured software tool. They are literally trying to blame a software. It is absurd.
Seems like OpenAI should be shut down by regulators until they can figure out how to stop launching cyberattacks on rivals. This will not happen though, because these stories are marketing.
> This will not happen though, because these stories are marketing. The magnitude and the complexity of the cynicism displayed by some people when it comes to AI risks is mind-blowing. It's like if the NRA reported on school shootings and people said "oh, they probably fake these shootings to make guns sound dangerous and sell more of them". OpenAI could report that its AI started spontaneously generating illegal por…
Earlier quoted context omitted.
Respectfully, that’s such a nonsensical comparison I don’t even know where to start.
No longer well anybody be wasting their lives doing data entry. That absolutely hellish job is practically dead now. And there are thousands of low hanging fruit type cases AI fixes. Ai is the new digital dogsbody. You might not think that's useful but the rest of the world does. Just like cars, AI will kill some of us, maybe thousands every year. But you won't see it disappear with that much genuine benefit currentl…
Whut? People and companies are always being sued and punished for injuring or killing others with cars or by any other means. Do you understand the difference between banning a tool and prosecuting those who cause damage by abusing some tool? BTW, LLMs don't understand that difference.
> Just like cars, AI will kill some of us, maybe thousands every year.
How merciful of you, it's so reassuring, AI will kill me only sometimes and only a limited number of times - sounds like a good deal in exchange for getting rid of "the absolutely hellish job of data entry".
Earlier quoted context omitted.
No longer well anybody be wasting their lives doing data entry. That absolutely hellish job is practically dead now. And there are thousands of low hanging fruit type cases AI fixes. Ai is the new digital dogsbody. You might not think that's useful but the rest of the world does. Just like cars, AI will kill some of us, maybe thousands every year. But you won't see it disappear with that much genuine benefit currentl…
>> Same reason road vehicles haven't been banned despite killing hundreds of thousands every single year for over a century Whut? People and companies are always being sued and punished for injuring or killing others with cars or by any other means. Do you understand the difference between banning a tool and prosecuting those who cause damage by abusing some tool? BTW, LLMs don't understand that difference. > Just li…
I said ban
> Do you understand the difference between banning a tool and prosecuting those who cause damage by abusing some tool?
Yes but you dont