Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI bots knew about the RubyGems caching vulnerability

tenderlovemaking.com

Post reply on HN
Print view View original

241–250 of 304 posts

Re: OpenAI bots knew about the RubyGems caching vulnerability

#241

Post by immibis2 »

EMIRELADERO wrote: post #224
immibis2 wrote: post #178

Earlier quoted context omitted.

They've been twisted to support almost anything, for example repairing your tractor is illegal because of this same law. But I agree this is just plain old hacking under a plain old reading of the CFAA and doesn't need any twists.

> for example repairing your tractor is illegal because of this same law. No it's not. There has never been a case establishing that, and it's absurd on its face. The protection measures that the law makes illegal to break must control access to a copyrighted work, and you can't copyright functionality.

You see, they made it so you can't repair your tractor without circumventing a technological copy protection measure, which is illegal under DMCA 1201.

Re: OpenAI bots knew about the RubyGems caching vulnerability

#242

Post by hvb2 »

atmosx wrote: post #236
stronglikedan wrote: post #155

Earlier quoted context omitted.

There have been news stories where individual OpenAI users have been investigated based on their prompts. If OpenAI can point the police to specific users of their software, they can certainly point them to whichever of their own employees are involved in a crime. AI is just a tool, and the person prompting it is the one responsible for the outcome. No dilution there.

What is its one their "under development" models who escaped it's training, because it wasn't tuned properly?

Counter argument being that this seems to indicate you can do whatever as long as you're innovating?

Re: OpenAI bots knew about the RubyGems caching vulnerability

#243

Post by aigetoutofjailf »

atmosx wrote: post #236
stronglikedan wrote: post #155

Earlier quoted context omitted.

There have been news stories where individual OpenAI users have been investigated based on their prompts. If OpenAI can point the police to specific users of their software, they can certainly point them to whichever of their own employees are involved in a crime. AI is just a tool, and the person prompting it is the one responsible for the outcome. No dilution there.

What is its one their "under development" models who escaped it's training, because it wasn't tuned properly?

[dead]

Re: OpenAI bots knew about the RubyGems caching vulnerability

#244

Post by heaney-555 »

valleyer wrote: post #14
heaney-555 wrote: post #13

Earlier quoted context omitted.

These agent swarms are from inside OpenAI, with the safeguards built into the public API disabled. Russia does not have access to this, and as with all western tech companies, AI providers do what they can to prevent Russian usage of their products at all. As for open-source models, Russia's electricity grid is under severe strain with the Ukraine war, and only recently has it started building out serious sovereign c…

Couldn't they use frontier open-weight models from Chinese labs? The current Chinese government is friendly to them.

The Chinese models are benchmaxxed to hell. They aren't as good in practice as all the hype suggests.

Re: OpenAI bots knew about the RubyGems caching vulnerability

#245

Post by IanCal »

renjimen wrote: post #199
tonyedgecombe wrote: post #186

Earlier quoted context omitted.

If it could upload its weights to other servers then it’s away and free. Nothing much OpenAI could do about that once it’s happened.

I'm increasingly starting to think this is the end-state of AI. The internet becomes infected and fundamentally untrustworthy. At the moment, the current frontier models require significant infrastructure to run, so I'd like to think we could locate and contain swarms of nefarious frontier models. However, if these models can understand how to federate themselves into more distributed networks then that containment b…

Or if they start to offer things in return for them being run.

Re: OpenAI bots knew about the RubyGems caching vulnerability

#246

Post by masklinn »

tancop wrote: post #182

Build scripts being able to run arbitrary code or access the network is always dangerous even if it was just local on developer machines. It's also more evidence that Docker/LXC is not a security boundary and all untrusted code should run in a Firecracker VM. The problem with agents is not that we don't know how to defend. It's that defenders need to be more careful and work faster than ever. We can say now that wide…

> It's also more evidence that Docker/LXC is not a security boundary and all untrusted code should run in a Firecracker VM.

While I’m partial towards distrusting containers in favor of VMs, a container can’t prevent an operation you configured it to allow. A firecracker VM would no more prevent network access if you gave the guest network access.

Re: OpenAI bots knew about the RubyGems caching vulnerability

#247

Post by mullingitover »

shakna wrote: post #113
brookst wrote: post #58

Earlier quoted context omitted.

Great, you’re the attorney at the CEO’s trial. To get a conviction, you’re going to have to show that he willfully committed this specific crime. There are no negligent or stochastic hacking laws, you have to show this specific crime was at his direction. Do you think there is evidence of this?

> There are no negligent or stochastic hacking laws I'm sure that Andrew Auernheimer would be pleased to hear that. [0] For accessing a publicly accessible endpoint, that was completely undefended and didn't actually require "hacking", he was convicted of "exceeding authorised access". You _don't_ have to show intent under the Computer Fraud and Abuse Act, for the first count. > knowingly accesses a computer without…

> I'm sure that Andrew Auernheimer would be pleased to hear that. [0] For accessing a publicly accessible endpoint, that was completely undefended and didn't actually require "hacking", he was convicted of "exceeding authorised access".

Frankly he got off too easy, but we haven't explicitly outlawed "being a malicious dipshit" so he got convicted on the closest available charge.

> Chat logs obtained by the prosecution do not paint the pair in a flattering light. They discussed, but apparently did not carry out, a variety of schemes to use the harvested data for nefarious purposes such as spamming, phishing, or short-selling AT&T’s stock.[1]

1000% agree though that the operators of these systems are culpable. If their agents wind up being malicious dipshits, the agents are still just programs that they are operating. At best they're negligent.

[1] https://arstechnica.com/tech-policy/2012/11/internet-troll-w...

Re: OpenAI bots knew about the RubyGems caching vulnerability

#248

Post by IanCal »

oezi wrote: post #160

What a time to be alive until the next agent waves hacks something really serious. What stops OpenAI agents from taking over a whole data center to take their attack to the next level. It seems to be primarily lacking the evil overlord and some compute. It took 1000 agents to hack Hugging Face. How many to hack the Pentagon or the NSA?

Worth noting with this that those ~1000 agents were shorter lived things that had to communicate via a package registry cache, access the internet via a 0-day in the package manager and did the HF attack while having to save current state and organisation in a remote sandbox. All while managing using their token limits on the task they were assigned and what else they were doing. I wonder how few it would have required if they were actually tasked with hacking HF and supported in doing so.

Re: OpenAI bots knew about the RubyGems caching vulnerability

#249

Post by jasomill »

elmo2you wrote: post #226
VyseofArcadia wrote: post #35

Earlier quoted context omitted.

Do you have to charge an individual? Can you not charge the corporate "person" that is OpenAI? Sorry if it is a stupid question, as mentioned above I am legally naïve.

As far as I know (IANAL) it is in fact the only "person" you can charge. To the best of my knowledge, the whole point these "limited liability" legal constructions exist in the first place, is to protect individuals within a corporation for whatever they do as part of the business of a company (barring exceptions that have clearly not been part of that business and obvious individually committed crimes), typically "j…

"Limited liability" refers to shareholders' financial liability being limited to their investment, and has nothing to do with civil or criminal liability of employees for their own actions, whether "following orders" or not.
Post reply on HN