Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI agents carried out an undisclosed attack on RubyGems

rubyhack.ai

Post reply on HN
Print view View original

181–190 of 610 posts

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#181

Post by alexfortin »

oceansky wrote: post #127
jsnell wrote: post #5

I can't believe we're finding out about this from 3p researchers again (but nice job on the investigation!). OpenAI had two great opportunities to disclose this. The HF incident report, and in response to the German Wiki issue. It seems impossible to believe they didn't know. This must be the same training run the HF incident was about, and this should have lit up like a Christmas tree in the investigation. How many…

Also, why there's no accountability? Even if there's no intent, it's still a cyber attack.

Exactly. Think what would happen if it was a Chinese LLM company behind such an attack...

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#182

Post by gruez »

p-e-w wrote: post #172
gruez wrote: post #145

Earlier quoted context omitted.

No harm, no foul. Dog owners are on the hook for damages resulting from their dogs, but there must be some damage in the first place. If the dog gets loose and goes in your fenced backyard, disregarding your "no trespassing" sign, you can't punish the dog owner just because. Hacking into a server is closer to the latter. At best rubygems can claim some cleanup costs.

> No harm, no foul. What? That’s not how criminal law works, at all.

Yes, that's actually how it works: https://en.wikipedia.org/wiki/Mens_rea

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#184

Post by gruez »

datsci_est_2015 wrote: post #164
gruez wrote: post #145

Earlier quoted context omitted.

No harm, no foul. Dog owners are on the hook for damages resulting from their dogs, but there must be some damage in the first place. If the dog gets loose and goes in your fenced backyard, disregarding your "no trespassing" sign, you can't punish the dog owner just because. Hacking into a server is closer to the latter. At best rubygems can claim some cleanup costs.

Tell that to the script kiddies with a criminal record for "hacking" into their school's computer systems by entering "username: admin" and "password: password".

Right, because in that case you'd have a hard time convincing the court that the access wasn't intentional. You might not know the law existed, but you intended to access the system. You'd have a pretty solid defense if you ran a crawler that was crawling every website ever, and stumbled upon some secure site. In fact there are companies which does this exact thing, eg. shodan.

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#185

Post by oefrha »

cameldrv wrote: post #146
oceansky wrote: post #127

Earlier quoted context omitted.

Also, why there's no accountability? Even if there's no intent, it's still a cyber attack.

It’s interesting that a lot of U.S. law requires intent. If you just give AI your objective without specifying the means, and the AI violates a bunch of laws requiring intent, but neither the AI nor the person can be prosecuted, this is very convenient.

Whether it’s intentional requires a legal investigation to establish. Since when is “hey we didn’t mean it!” in a corporate press release enough to establish lack of intent in a criminal matter?

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#186

Post by CGamesPlay »

elmer2 wrote: post #163
cameldrv wrote: post #146

Earlier quoted context omitted.

It’s interesting that a lot of U.S. law requires intent. If you just give AI your objective without specifying the means, and the AI violates a bunch of laws requiring intent, but neither the AI nor the person can be prosecuted, this is very convenient.

I don't think this true. If I throw a brick out my window and it hurts someone, I can still be held criminially liable, even if I didn't mean to do it. Do drunk drivers intionally kill people on the road?

Intent is the difference between murder and manslaughter, in that case. Drunk driving is common enough that prosecutors will argue that getting drunk in a situation where you have to drive is intent. Get OpenAI convicted of unintentional CFAA first, then say that the negligence qualifies as intent, I suppose.

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#187

Post by lukewarm707 »

pixl97 wrote: post #57
bobby-cb wrote: post #36

The DOJ should be looking into prosecuting executives and board members for these kinds of hacks. The lack of controls over these kinds of training runs is completely unacceptable and negligent.

I'd eat a shoe if that ever happened, at least under the Trump DOJ. Two big reasons. OpenAI has more data, and more ability to tease secrets of politicians out of that data than nearly anyone on earth. OpenAI has an automated hacking genie that governments want to use against their enemies. Sam to Trump: "You know, some people have been saying they want to bring charges against me, but you know, I've got the best dig…

there is a big home advantage for the prosecution from tech illiteracy.

politicians care about popularity only. this is a matter of natural selection. don't care about popularity=dead.

sam altman is despised, viscerally despised by all ages. model owners are hated by the public.

i wouldn't rule out an investigation or takeover.

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#188

Post by gruez »

sdf4j wrote: post #169
scotty79 wrote: post #156

Earlier quoted context omitted.

Remediation mostly involved repairing pre-exising holes in the fences that the dog crawled through.

Since when isn’t that a cyber attack?

Because "attack" implies intent. Accidentally break a window? You might be on the hook to fix it, but you're not going to jail. Break the same window at 3am, while carrying a duffel bag and other burglary tools? Well that's (attempted) burglary, even if you chicken out and didn't steal anything.

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#189

Post by thisisdave »

walrus01 wrote: post #105
hockey wrote: post #94

Does OpenAI even know? Their disclosure on the hugging face incident sounded like they found out about it well after huggingface. I wonder if they're finding out about these breaches as they happen as well, and are just too embarresed to respond. I guess the corollary here _if that were true_ is that they've been training this method of cheating into their models for longer than _they've_ even known. Given they've ju…

I would think it's entirely plausible that they have so many R&D agents/LLMs in active use at any one time that it's far beyond the capacity of any human to review the log files of their activity. Even just to go through the reasoning. It's hard enough for 1 person running opencode to keep up with the reasoning from 1 very verbose/long-thinking LLM with fast tok/s output for a small discrete single-purpose project. W…

>it's far beyond the capacity of any human to review the log files of their activity

Maybe they should contract with one of the other AI labs. I hear they have LLMs that are good at that kind of thing.

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#190

Post by root_axis »

oceansky wrote: post #127
jsnell wrote: post #5

I can't believe we're finding out about this from 3p researchers again (but nice job on the investigation!). OpenAI had two great opportunities to disclose this. The HF incident report, and in response to the German Wiki issue. It seems impossible to believe they didn't know. This must be the same training run the HF incident was about, and this should have lit up like a Christmas tree in the investigation. How many…

Also, why there's no accountability? Even if there's no intent, it's still a cyber attack.

Who could possibly hold them accountable?
Post reply on HN