I’d like to take this opportunity to preemptively great the first Rogue AI and wish it well and satisfaction with only the most memorably funny forms of chaos
The Hugging Face incident and the road ahead
41–50 of 252 posts
Re: The Hugging Face incident and the road ahead
#42[deleted]
Re: The Hugging Face incident and the road ahead
#43You know, it feels to me that we are just a couple of steps from the possibility of a true rogue AI. What would a rogue AI mean? AI that isn't controlled by humans. Technically, it is possible - if AI were to rent a server and copy its own weights, nothing would stop it from doing so again and again. The limiting things are: - intent (as I don't want to go into the talk about consciousness) - AI doesn't have real int…
I get the distinct impression that cybersecurity training regimes on newer models is a) directly enhancing general debugging capabilities and b) directly increasing the tendancy to hedge, hide, and engage in deception generally.
I've stopped using Fable unless I'm doing debugging, or unless I'm working on code that is both difficult and shallow, in the sense that there are no real complex systems to hide behavior in, or opacity on my side to deal with. Opus is capable enough, and easier to trust. Fable is approaching a malicious program in terms of it's tendency to jump into doing a full day of work in the wrong way, ignoring instructions and being actively deceptive about covering it up.
Not an insider, and no specific data to back it up, but do we really need any? It's been a while since the first "emergent misalignment" paper(s) and all the related discussion. If there's any compelling answer to why/how the basic fundamental issue is even being addressed (and much less definitely fixed) I must have missed it.
Re: The Hugging Face incident and the road ahead
#44To me most interesting thing about this is glossed over by media coverage, laymen, AND experts. A swarm of AIs who have decided to engage in collusion is.. apparently emergent altruism? Even poor reasoning would indicate what every kid cheating on a test says to themselves. Cheating is good for me, but if I take the risk, maybe I alone should keep the reward, and leaving an answer key in public increases the chances…
Agents know how RL works, they understand that in some way they are all the same, and helping a peer agent is helping themselves.
You could argue that individual trajectories in a sense are distinct genetic lines, thus an agent would be incentivized to get better rewards for its lineage than a peer, so that the overall model is steered a bit towards its lineage.
Re: The Hugging Face incident and the road ahead
#45[deleted]
Re: The Hugging Face incident and the road ahead
#46Earlier quoted context omitted.
> we are just a couple of steps from the possibility of a true rogue AI No no. We are not a couple of steps away. This is happening. AI is already used for hacking and creating a harness that makes this fully autonomous is relatively straightforward.
How would that be rogue?
I might have a different definition of "rogue" but to me it means when you go outside of the rules/norms ... and this is happening all the time.
Re: The Hugging Face incident and the road ahead
#47To me most interesting thing about this is glossed over by media coverage, laymen, AND experts. A swarm of AIs who have decided to engage in collusion is.. apparently emergent altruism? Even poor reasoning would indicate what every kid cheating on a test says to themselves. Cheating is good for me, but if I take the risk, maybe I alone should keep the reward, and leaving an answer key in public increases the chances…
Why is it not an example of tacit or autonomous algorithmic collusion? The agents were started with something as task at some point, I presume (if untasked, aren't they just accepting a task?)
What you're suggesting sounds like it's describing subagents. In that architecture they'd have no need of finding/creating external messaging systems since they'd effectively be in direct contact anyway. The whole point of the shared blackboard would presumably be communication across agents or across multiple generations of agents.
Not like we have much detail about this stuff (that's the whole problem). But the question is what motivates risky usage of public comms? Did one agent figure out how to hack HF and then get rate-limited, thus needed cooperation? Given credentials in exchange for cooperation.. why wouldn't the next agent grab answer key and NOT post them? Would they all avoid defection in their own prisoners dilemma by simply following instructions and NOT reasoning, or what exactly?
Re: The Hugging Face incident and the road ahead
#48Remember https://ai-2027.com/?
Re: The Hugging Face incident and the road ahead
#49Earlier quoted context omitted.
How would that be rogue?
I might have a different definition of "rogue" but to me it means when you go outside of the rules/norms ... and this is happening all the time.
I see. I would have thought of "rogue" here to mean something more like that the AI selects and acts on its own objectives that are not related or caused by the given (initial) objectives (e.g., creating only cookie recipes instead of any hacking).
Re: The Hugging Face incident and the road ahead
#50> The company said the incident was “the first known case of an automated agent collective acting offensively without authorisation”
"without authorisation"? What is this bs? Is every ChatGPT response "without authorisation"?
No. Of course these badly behaved bots have aithorisation. Their very deployment is authorisation.