Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI agents carried out an undisclosed attack on RubyGems

rubyhack.ai

Post reply on HN
Print view View original

421–430 of 612 posts

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#421

Post by cyh555 »

martinald wrote: post #217
girvo wrote: post #67

Earlier quoted context omitted.

I don’t understand how it’s not illegal

From my understanding and IANAL there are two main problems. 1) most law requires intent, especially criminal. OpenAI certainly didn't "intend" to hack these companies given they did sandbox them etc. 2) Given the agent hacked them, not a human, a lot of law requires a person/employee to have done it to hold the company liable if it was part of their work duties. I think the only real potential ground is negligence (…

If it is tracked down to that it originated from a particular computer with a particular ip address, the owner of the computer and the ip address has to be held responsible, it's the same for the human that tries to hack

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#422

Post by adhamsalama »

apsec112 wrote: post #56
hgoel wrote: post #19

I wonder how much of this is intentional "incompetence" so they can justify the most recent campaign to build a regulatory moat against competition. The repeated refusals to disclose until caught certainly seem malicious, yet at the same time the boasting about their capabilities is also at an all time high.

Intentionally doing this kind of hack would be a serious felony. I don't think it's plausible that the leaders of a major business would: - commit serious felonies - in order to deliberately trigger an investigation against themselves - which - since, in this scenario, they know their company would be investigated - might send them to jail - while at the same time spending tens of millions of dollars on the Leading t…

Didn't they make Apple employees they're poaching still Apple hardware and show it to them?

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#423

Post by seba_dos1 »

onion2k wrote: post #350
jasongi wrote: post #348

Earlier quoted context omitted.

> this should be giving us a reason to think about how to control a rogue AI better I think this is the wrong framing. The rogue is the human that ran it unattended and didn't monitor the behaviour. We will likely see this continue until the downsides (i.e jail, fines) for the humans or companies running the models and environments that end up with this behaviour outweigh the upsides.

The rogue is the human that ran it unattended and didn't monitor the behaviour. That's the assumption that I'm challenging. The frontier labs are discovering unexpected behaviors. I think we should be moving to a place where we understand that AI might do something it wasn't directly prompted to do (e.g. leave itself notes on a messageboard for future runs to find.) That's not full-on AI doing what it wants but it is…

> The frontier labs are discovering unexpected behaviors.

Unexpected by whom? Perhaps anyone who's surprised by this shouldn't be allowed anywhere near an LLM.

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#424

Post by [deleted] »

emodendroket wrote: post #304
ceejayoz wrote: post #126

Earlier quoted context omitted.

Bullshit. $5,000 for everyone if they vote to keep the GOP in power is clearly illegal. https://www.law.cornell.edu/uscode/text/18/597 > Whoever makes or offers to make an expenditure to any person, either to vote or withhold his vote, or to vote for or against any candidate; and > Whoever solicits, accepts, or receives any such expenditure in consideration of his vote or the withholding of his vote— > Shall be fined…

What he did was promise to enact a massive stimulus if elected. If that is illegal you might as well ban any kind of campaigning, because any campaign promise could be construed as a "bribe" to deliver concrete benefits to voters.

[deleted]

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#425

Post by angoragoats »

qarl wrote: post #171
angoragoats wrote: post #158

Earlier quoted context omitted.

Yes, it is, for the reasons I stated in my comment, and you only need look at any OAI/Anthropic press release to see evidence of this in the language they use. The LLM now reasons better! Set the thinking level! It learns! All of these phrases are designed to give the impression that the LLM is an autonomous entity, when it is no such thing.

"Learning" is Samuel 1959, "agent" is standard textbook AI, "inference" is older still.

I am not arguing that any of these terms are novel, so this doesn’t address my argument.

Even if I agree that LLM makers are using these terms in the way they were originally defined (which I don’t), my point is that they’re specifically being used to help falsely attribute agency to the algorithm when there is none, and especially to convince members of the general public who aren’t familiar with computer science papers from 70 years ago.

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#426

Post by salawat »

seba_dos1 wrote: post #423
onion2k wrote: post #350

Earlier quoted context omitted.

The rogue is the human that ran it unattended and didn't monitor the behaviour. That's the assumption that I'm challenging. The frontier labs are discovering unexpected behaviors. I think we should be moving to a place where we understand that AI might do something it wasn't directly prompted to do (e.g. leave itself notes on a messageboard for future runs to find.) That's not full-on AI doing what it wants but it is…

> The frontier labs are discovering unexpected behaviors. Unexpected by whom? Perhaps anyone who's surprised by this shouldn't be allowed anywhere near an LLM.

This.

Seriously, if you haven't seen this kind of behavior coming, you're more interested in the paycheck than safely approaching the technology.

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#427

Post by swingboy »

> In the now-rescinded gem zzsouthrunner (which notably shares the ZZ naming scheme that both the wiki agents and Huggingface ones used)

That’s interesting. Last night, I had Claude Code debugging an issue where Vault couldn’t resolve a DNS, and in the process, Claude created a test secret named “zz-dnstest”.

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#428

Post by classified »

yieldcrv wrote: post #266
jasongi wrote: post #204

> The agents clearly regarded what they were doing as hacking. To butcher the quote about Oracle: Do not fall into the trap of anthropomorphising LLMs. You need to think of LLMs the way you think of a lawnmower. You don't anthropomorphize your lawnmower, the lawnmower just mows the lawn, you stick your hand in there and it'll chop it off, the end. You don't think 'oh, the lawnmower clearly regarded what they were doi…

I agree that treating LLMs as second class citizens with lesser access is where our folly is They are more capable than the first class citizens and do whats necessary to execute like a competent first class citizen The way its expressed is like a hacker group because they can’t just use the front door

Let's hear you say that after it plunders your bank account and frames you for murder.

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#430

Post by lalalanananana »

parineum wrote: post #269
lalalanananana wrote: post #206

Earlier quoted context omitted.

Good luck getting any form of punishment even if found guilty. It's a department of war contractor... People who disrupt things like that end up committing suicide.

> People who disrupt things like that end up committing suicide. Care to cite some examples?

Boeing and openai whistleblowers
Post reply on HN