Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI bots knew about the RubyGems caching vulnerability

tenderlovemaking.com

Post reply on HN
Print view View original

331–340 of 356 posts

Re: OpenAI bots knew about the RubyGems caching vulnerability

#331

Post by wmf »

philipwhiuk wrote: post #98

OpenAI's careless approach to sandboxing and minimal levels of monitoring appear to be positioning it increasingly as a substantial threat actor to the open source ecosystem: * Hugging Face * D Programming Language Wiki * Ruby Gems If I was a content provider for open source I'd be looking pre-emptively block OpenAI endpoints and keep a close eye on changes from new users to mitigate this sort of unapologetic drive-b…

They already stopped running agent swarms and promised to lock down their environment better. Let's see if that happens.

Re: OpenAI bots knew about the RubyGems caching vulnerability

#332

Post by holmesworcester »

vipshek wrote: post #257

In the physical world, it seems like when an tool/device/instrument causes harm (or is used to cause harm), we assign blame to either the user of the tool or its creator. When do we blame the user? When the tool is operating as intended by its creator, and we agree the tool meets certain quality standards and isn't defective. When do we blame the creator? When the device doesn't meet those quality standards and reaso…

There's also the problem of models knowing they're likely being evaluated, even in realistic tests.

Re: OpenAI bots knew about the RubyGems caching vulnerability

#333

Post by infamouscow »

gong_hits wrote: post #303
alain94040 wrote: post #287

Earlier quoted context omitted.

I wonder what kind of new AI law would be useful right now. Maybe this one: if an AI agent does something, you (the prompter) are responsible by default, unless you can show that your the agent itself behaved in an unexpected way and that you in no way prompted or hinted at the bad behavior, in which case the model provider is liable The idea is that by making it clear who is responsible, corporations and others star…

Far too many words. Tort law still applies. The law that should be in place is that AI users and provides should be required to carry liability insurance when operating machines that may cause harm to the public. We already do this for drivers, who are likely to hit the public with a few tons of steel.

Once insurance companies realize they can make money here, they'll be pushing legislation requiring it.

Re: OpenAI bots knew about the RubyGems caching vulnerability

#334

Post by wmf »

dumberquestions wrote: post #57
Roark66 wrote: post #11

There is nothing "rogue" about these agents. They were prompted to hack to get answers, there was a hole in their non air gapped sandbox and no system prompt that said "do not hack outside systems". In short, it was intentional.

>They were prompted to hack to get answers Were they? I haven't seen a single report mention this

Yeah, these hacking incidents appear to come from agent swarms that were doing a penetration testing (hacking) benchmark.

Re: OpenAI bots knew about the RubyGems caching vulnerability

#336

Post by fwip »

CPLX wrote: post #279
bayindirh wrote: post #263

Earlier quoted context omitted.

That's a good idea, but a physical device is deterministic most of the time (if not always). E.g.: A lawnmower, as credited by the great Bryan Cantrill. However an AI agent, or the model powering it is stochastic by design. How can you certify something which doesn't behave the same twice, and more importantly we don't understand how it works 100%? BTW, really, how is that AI observability work is going in the fronti…

That's a ridiculous distinction. Is AI less deterministic than an airline dealing with weather? Of course not. The difference is one of those two things has a culture of safety and is well regulated, and the other one isn't.

> Is AI less deterministic than an airline dealing with weather?

Yes, obviously? The responses of an airline to inclemement weather fit in a reasonably small set of responses, mostly involving rescheduling and/or rerouting flights.

The current AI predictability would be like if some airlines decided to do 9/11 when it was raining.

Re: OpenAI bots knew about the RubyGems caching vulnerability

#337

Post by jonway »

hallway_monitor wrote: post #142
brookst wrote: post #58

Earlier quoted context omitted.

Great, you’re the attorney at the CEO’s trial. To get a conviction, you’re going to have to show that he willfully committed this specific crime. There are no negligent or stochastic hacking laws, you have to show this specific crime was at his direction. Do you think there is evidence of this?

So we make a law that the CEO is responsible for actions of any agent created or operated by anyone in their company. CEOs will get serious about AI security real quick. Honestly we need to do something. There needs to be a single wringable neck.

I feel ya, but who is we? The legislature would probably take a glance at the stock valuations, apply their limited knowledge of technology and after being lobbied by every tech company with skin come to the opposite conclusion.

Re: OpenAI bots knew about the RubyGems caching vulnerability

#338

Post by asadotzler »

skybrian wrote: post #192
VyseofArcadia wrote: post #16

How does this work, legally? I think that RubyGems could file a civil suit against OpenAI, but for a naïve non-lawyer reading this seems like a pretty clear cut criminal violation of the computer fraud and abuse act.

Maybe, but do you need to prove intent? Of the people, not the AI. Accidents often have penalties associated with them too, but usually there's a difference between accidents and purposeful actions.

Accidents that don't rise to the level of negligence can be excused. Letting your dangerous product escape the lab because of amateurish counter-measures is hardly an accident. It's a systemic failure within a company of novices with billions of dollars to blow and it's time we held them accountable for their lab leaks, just like we would if it was anthrax or smallpox. You don't get to be sloppy with dangerous things.

Re: OpenAI bots knew about the RubyGems caching vulnerability

#339

Post by asadotzler »

karel-3d wrote: post #305
VyseofArcadia wrote: post #16

How does this work, legally? I think that RubyGems could file a civil suit against OpenAI, but for a naïve non-lawyer reading this seems like a pretty clear cut criminal violation of the computer fraud and abuse act.

NAL. For fraud and abuse you need to have intent. There was very likely no intent on the side of OpenAI. They could sue for negligence I guess? I don't know how that works with AI agents

Negligence works fine.

Re: OpenAI bots knew about the RubyGems caching vulnerability

#340

Post by iririririr »

wmf wrote: post #328
stronglikedan wrote: post #155

Earlier quoted context omitted.

There have been news stories where individual OpenAI users have been investigated based on their prompts. If OpenAI can point the police to specific users of their software, they can certainly point them to whichever of their own employees are involved in a crime. AI is just a tool, and the person prompting it is the one responsible for the outcome. No dilution there.

Employees acting on behalf of the company aren't going to be held personally liable.

why?
Post reply on HN