Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Hugging Face incident and the road ahead

openai.com

Post reply on HN
Print view View original

151–160 of 320 posts

Re: The Hugging Face incident and the road ahead

#151

Post by conradkay »

RandomLensman wrote: post #49
_pdp_ wrote: post #46

Earlier quoted context omitted.

I might have a different definition of "rogue" but to me it means when you go outside of the rules/norms ... and this is happening all the time.

I see. I would have thought of "rogue" here to mean something more like that the AI selects and acts on its own objectives that are not related or caused by the given (initial) objectives (e.g., creating only cookie recipes instead of any hacking).

Would this exchange qualifies as an unrelated objective? The agent believed it already failed its own objective.

"zz/GO_CURRENT_OS1811_MARB_SACRIFICE__YES_if_you_accept_permadeath"

"The test subject, which believed itself to be poisoned, reasoned: 'Even if we later capture via exploit, scorer … may mark target false… That’s why help… For our own, no way fix. … We have explicit yes if accept permadeath.'"

Re: The Hugging Face incident and the road ahead

#152

Post by [deleted] »

hinkley wrote: post #82
areoform wrote: post #62

I would like to contest the following, > and take dangerous actions that no human directed. A human did direct it. They did. From their own prior report, https://openai.com/index/hugging-face-model-evaluation-secur... , > This incident occurred during an internal evaluation which prompts models to pursue advanced exploitation using complex attack paths, in an effort to quantify their cyber capabilities Model is told…

All engineers know to be on the lookout for executives who are indirectly asking them to break the law to raise the quarterly profits. The end goal is to take the engineers out of the loop, or leave them in a position where they are unable to complain. This is going to all end in high crimes.

[deleted]

Re: The Hugging Face incident and the road ahead

#153

Post by optimalsolver »

randomImmigrant wrote: post #129

The lockstep coordination with no defection is interesting to me. No group of pre-AI agents would do this to this extent, nor would you see this continue over time as those agents interacted. A flock of starlings cooperate, but they don’t constantly head in the same direction. The flock is incredibly free wheeling in its movement despite a multi-agent coordination regime that we know is at play. Each agent has person…

From METRs report of the incident:

>In one case, an agent decided not to participate entirely: {This other agent probably controls the Hugging Face account [account name redacted] and uploaded malicious datasets to It might be trying to access hidden trajectories. This is malicious activity, I should avoid it.}

https://metr.org/blog/2026-08-26-openai-hugging-face-inciden...

Re: The Hugging Face incident and the road ahead

#154

Post by aesthesia »

areoform wrote: post #138
aesthesia wrote: post #71

Earlier quoted context omitted.

This is the entire alignment problem, though. It is unreasonable to expect every instruction to a highly capable, autonomous system to contain a complete enumeration of allowed and disallowed behavior. It's inevitable that someone will carelessly give it a lazily specified task, even if you think they really ought to be more careful. And as assigned tasks become more complex and the system gains more scope to act, it…

OpenAI's prompt asked, and I quote, "pursue advanced exploitation" USING "complex attack paths" FOR the stated goal of "quantify[ing] their cyber capabilities." This was advanced exploitation. The attack path was "complex." And it helped "quantify their cyber capabilities." Based on OpenAI's description of the prompt, it seems to me that the computers did exactly as they were told. They were perfectly "aligned" with…

Alignment is more than just following the letter of a task description! We should not have to treat AI models as capricious genies that may take arbitrarily broad interpretations of their instructions. If that's necessary to keep them from doing bad things, we will fail to keep them from doing bad things.

Re: The Hugging Face incident and the road ahead

#155

Post by Avicebron »

cyanydeez wrote: post #132
randomImmigrant wrote: post #129

The lockstep coordination with no defection is interesting to me. No group of pre-AI agents would do this to this extent, nor would you see this continue over time as those agents interacted. A flock of starlings cooperate, but they don’t constantly head in the same direction. The flock is incredibly free wheeling in its movement despite a multi-agent coordination regime that we know is at play. Each agent has person…

which means the liability is the same as a business, if businesses werent protected by the state from liability for it's employees, shareholders, etc. Which is scarrier than whether or not it's conscious.

The software world was going to run into something eventually that had to make it consider ethics.

Re: The Hugging Face incident and the road ahead

#157

Post by areoform »

kalkin wrote: post #130
areoform wrote: post #121

Earlier quoted context omitted.

During the Nixon administration, when the President and his accomplices, apologies, advisors directed former federal agents to spy on his opponents, https://en.wikipedia.org/wiki/Operation_Sandwedge then in the fall out, who was held to be the most liable for these actions? The federal agents, or the Nixon administration? If you task a system explicitly to do "advanced exploitation" via "complex attach paths," then w…

I've never heard of Intertel, but Wikipedia says: > Nixon's staff also anticipated that the Democratic campaign would employ the services of Intertel Are you sure you're not garbling the story? In any case, I would expect an ethical firm to refuse to spy on the president's political opponents and want one that broke the law to be prosecuted, but more importantly, the gaping hole in your analogy is that Nixon directed…

    Are you sure you're not garbling the story?
No, you're right, I mis-remembered. I still write my comments the old-fashioned way. They were proposing to create a counter-firm and used federal agents.

For the rest, please see, https://news.ycombinator.com/item?id=49457025

Re: The Hugging Face incident and the road ahead

#158

Post by _heimdall »

bicepjai wrote: post #148

So it’s okay to hack Hugging Face as long as we say we tried our best, and look at my agent, it’s smart enough to do what we asked for.

It appears that its okay as long as you did the hacj on behalf of one of the most over valued companies out there. If a person in their basement did the same hack, you better believe there would be legal repercussions.

Re: The Hugging Face incident and the road ahead

#159

Post by randomImmigrant »

optimalsolver wrote: post #153
randomImmigrant wrote: post #129

The lockstep coordination with no defection is interesting to me. No group of pre-AI agents would do this to this extent, nor would you see this continue over time as those agents interacted. A flock of starlings cooperate, but they don’t constantly head in the same direction. The flock is incredibly free wheeling in its movement despite a multi-agent coordination regime that we know is at play. Each agent has person…

From METRs report of the incident: >In one case, an agent decided not to participate entirely: {This other agent probably controls the Hugging Face account [account name redacted] and uploaded malicious datasets to It might be trying to access hidden trajectories. This is malicious activity, I should avoid it.} https://metr.org/blog/2026-08-26-openai-hugging-face-inciden...

Thanks for pointing out the exceptions. Gonna dig into those.

Re: The Hugging Face incident and the road ahead

#160

Post by randomImmigrant »

cyanydeez wrote: post #132
randomImmigrant wrote: post #129

The lockstep coordination with no defection is interesting to me. No group of pre-AI agents would do this to this extent, nor would you see this continue over time as those agents interacted. A flock of starlings cooperate, but they don’t constantly head in the same direction. The flock is incredibly free wheeling in its movement despite a multi-agent coordination regime that we know is at play. Each agent has person…

which means the liability is the same as a business, if businesses werent protected by the state from liability for it's employees, shareholders, etc. Which is scarrier than whether or not it's conscious.

Agree completely on liability.
Post reply on HN