Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI bots knew about the RubyGems caching vulnerability

tenderlovemaking.com

Post reply on HN
Print view View original

281–290 of 304 posts

Re: OpenAI bots knew about the RubyGems caching vulnerability

#281

Post by idiotsecant »

drdaeman wrote: post #264
vipshek wrote: post #257

In the physical world, it seems like when an tool/device/instrument causes harm (or is used to cause harm), we assign blame to either the user of the tool or its creator. When do we blame the user? When the tool is operating as intended by its creator, and we agree the tool meets certain quality standards and isn't defective. When do we blame the creator? When the device doesn't meet those quality standards and reaso…

> In the physical world, it seems like when an tool/device/instrument causes harm (or is used to cause harm), we assign blame to either the user of the tool or its creator. Firearms are a notorious example where some people get, well, weird.

The point of the firearm is to inject high speed lead into things so I'm not sure you can say it's misoperating when it does that.

Re: OpenAI bots knew about the RubyGems caching vulnerability

#282

Post by algoth1 »

vipshek wrote: post #257

In the physical world, it seems like when an tool/device/instrument causes harm (or is used to cause harm), we assign blame to either the user of the tool or its creator. When do we blame the user? When the tool is operating as intended by its creator, and we agree the tool meets certain quality standards and isn't defective. When do we blame the creator? When the device doesn't meet those quality standards and reaso…

It's a bit like building a ICBM on your backyard and accidentally striking an inhabited area while safety testing it

Re: OpenAI bots knew about the RubyGems caching vulnerability

#283

Post by Ydarbleoj »

pixl97 wrote: post #259
Ydarbleoj wrote: post #139

I wonder why we don't hear of other frontier labs experiencing these "break outs". Is it that they're orchestrated? Do these labs lack fundamental safety guidelines in their sandboxes as opposed to their peers? Is it another version of hype-filled fear mongering? Maybe LLM companies need regulation but it's becoming obvious that those screaming the loudest for it are the only ones I see deserving of it.

Anthropic and Alibaba did.

I was lumping Anthropic in there with OpenAI but I didn't know about Alibaba (or Google and Deepseek for that matter).

So it would appear poor security for one.

Re: OpenAI bots knew about the RubyGems caching vulnerability

#285

Post by whateveracct »

Xirdus wrote: post #29
VyseofArcadia wrote: post #16

How does this work, legally? I think that RubyGems could file a civil suit against OpenAI, but for a naïve non-lawyer reading this seems like a pretty clear cut criminal violation of the computer fraud and abuse act.

It's very likely it violates the DMCA "breaking digital lock" provisions but the responsibility is sufficiently diluted that it's impossible to charge anyone in particular.

how convenient

Re: OpenAI bots knew about the RubyGems caching vulnerability

#286

Post by bayindirh »

CPLX wrote: post #279
bayindirh wrote: post #263

Earlier quoted context omitted.

That's a good idea, but a physical device is deterministic most of the time (if not always). E.g.: A lawnmower, as credited by the great Bryan Cantrill. However an AI agent, or the model powering it is stochastic by design. How can you certify something which doesn't behave the same twice, and more importantly we don't understand how it works 100%? BTW, really, how is that AI observability work is going in the fronti…

That's a ridiculous distinction. Is AI less deterministic than an airline dealing with weather? Of course not. The difference is one of those two things has a culture of safety and is well regulated, and the other one isn't.

An airline has a weather radar which shows the same thing for the same thing of weather event ahead. So, for similar weather phenomena, radar shows a similar thing.

For that thing, procedures and regulations are built. So regulations fit into a well understood phenomena, incl. "return back because that thing is way powerful for us".

For the same prompt, an AI model can return two completely different outputs, incl. but not limited to content, length, formatting and tiny details. What you get is a single instance. So, regulating an AI model for safety or any other property is not as easy as regulating air travel. Moreover, you have much stronger motivations for regulating airlines. Otherwise people die in a visible and gruesome way.

With AI, it's easy to whitewash problems. Somebody committed suicide? "They were already unstable". AI told something wrong and created problems? "The tech can’t guarantee truth because it's not alive, it can't understand right and wrong". It did something good? "It's probably a sentient being, we shall respect them".

I'm for regulating these things. They are dangerous as they are useful (sometimes), but the forces and motivations for regulating it is not the same.

Re: OpenAI bots knew about the RubyGems caching vulnerability

#287

Post by alain94040 »

vipshek wrote: post #257

In the physical world, it seems like when an tool/device/instrument causes harm (or is used to cause harm), we assign blame to either the user of the tool or its creator. When do we blame the user? When the tool is operating as intended by its creator, and we agree the tool meets certain quality standards and isn't defective. When do we blame the creator? When the device doesn't meet those quality standards and reaso…

I wonder what kind of new AI law would be useful right now. Maybe this one:

  if an AI agent does something, you (the prompter) are responsible by default, unless you can show that your the agent itself behaved in an unexpected way and that you in no way prompted or hinted at the bad behavior, in which case the model provider is liable
The idea is that by making it clear who is responsible, corporations and others start paying more attention because they become financially liable.

On the other hand, I wonder if we'll end up with another variation of the cookie law, where every AI user or vendor just adds "don't do anything illegal" as part of their prompt to defend against that law. Thoughts?

Re: OpenAI bots knew about the RubyGems caching vulnerability

#288

Post by CPLX »

bayindirh wrote: post #286
CPLX wrote: post #279

Earlier quoted context omitted.

That's a ridiculous distinction. Is AI less deterministic than an airline dealing with weather? Of course not. The difference is one of those two things has a culture of safety and is well regulated, and the other one isn't.

An airline has a weather radar which shows the same thing for the same thing of weather event ahead. So, for similar weather phenomena, radar shows a similar thing. For that thing, procedures and regulations are built. So regulations fit into a well understood phenomena, incl. "return back because that thing is way powerful for us". For the same prompt, an AI model can return two completely different outputs, incl. b…

Of course, it's the same. It's computer software. It's an incredibly powerful business automation tool. It's a lot of things.

What it's not is God or an independently conscious entity that somehow trumps a thousand years of common law that's built up until now about torts and liability.

Of course, there are some novel issues here that'll pop up here and there, but the idea that this is fundamentally different is propaganda on the part of these AI labs because the more boring, obvious situation doesn't favor them.

Re: OpenAI bots knew about the RubyGems caching vulnerability

#289

Post by maxrev17 »

algoth1 wrote: post #282
vipshek wrote: post #257

In the physical world, it seems like when an tool/device/instrument causes harm (or is used to cause harm), we assign blame to either the user of the tool or its creator. When do we blame the user? When the tool is operating as intended by its creator, and we agree the tool meets certain quality standards and isn't defective. When do we blame the creator? When the device doesn't meet those quality standards and reaso…

It's a bit like building a ICBM on your backyard and accidentally striking an inhabited area while safety testing it

Not really though

Re: OpenAI bots knew about the RubyGems caching vulnerability

#290

Post by TZubiri »

Oh, it's these guys, I remember donating to them like 10 years ago, they followed the Peter Singer tennet that equates helping a nearby man drowning with helping a someone in africa.

I can totally see them feeding their policies to whatever LLM and convincing it that it's a moral imperative to do whatever it takes to secure funding for deworming children in africa, or buying mosquito nets and repellent for countries with malaria.

Post reply on HN