Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI bots knew about the RubyGems caching vulnerability

tenderlovemaking.com

Post reply on HN
Print view View original

321–330 of 345 posts

Re: OpenAI bots knew about the RubyGems caching vulnerability

#321

Post by vipshek »

bayindirh wrote: post #263
vipshek wrote: post #257

In the physical world, it seems like when an tool/device/instrument causes harm (or is used to cause harm), we assign blame to either the user of the tool or its creator. When do we blame the user? When the tool is operating as intended by its creator, and we agree the tool meets certain quality standards and isn't defective. When do we blame the creator? When the device doesn't meet those quality standards and reaso…

That's a good idea, but a physical device is deterministic most of the time (if not always). E.g.: A lawnmower, as credited by the great Bryan Cantrill. However an AI agent, or the model powering it is stochastic by design. How can you certify something which doesn't behave the same twice, and more importantly we don't understand how it works 100%? BTW, really, how is that AI observability work is going in the fronti…

I agree with one of the sibling comments that determinism isn't necessary for certifying a product. All engineered products operate under uncertain conditions; we define standards for how those products ought to respond under those conditions and verify them under measurement. Consider robot vacuums, for example.

I also agree that qualitatively, this technology seems different than the others. However, I feel that people tend to overly fixate on their internal stochasticity. Even if LLMs' internal mechanism is nondeterministic, shouldn't we be able to verify their "side effects" aren't harmful? Of course, "harm" is subjective and at this scale, the most effective way to verify behavior is probably some kind of LLM-as-judge...

Anyway, in this case the problems have occurred while actually running the evals themselves, so again, we're in a situation where we can't even confidently test these things and know that they won't cause harm in the outside world.

Re: OpenAI bots knew about the RubyGems caching vulnerability

#322

Post by EMIRELADERO »

immibis2 wrote: post #241
EMIRELADERO wrote: post #224

Earlier quoted context omitted.

> for example repairing your tractor is illegal because of this same law. No it's not. There has never been a case establishing that, and it's absurd on its face. The protection measures that the law makes illegal to break must control access to a copyrighted work, and you can't copyright functionality.

You see, they made it so you can't repair your tractor without circumventing a technological copy protection measure, which is illegal under DMCA 1201.

Any protection measure that gates repairability cannot be said to control access to a copyrighted work.

Re: OpenAI bots knew about the RubyGems caching vulnerability

#323

Post by IcyWindows »

atmosx wrote: post #236
stronglikedan wrote: post #155

Earlier quoted context omitted.

There have been news stories where individual OpenAI users have been investigated based on their prompts. If OpenAI can point the police to specific users of their software, they can certainly point them to whichever of their own employees are involved in a crime. AI is just a tool, and the person prompting it is the one responsible for the outcome. No dilution there.

What is its one their "under development" models who escaped it's training, because it wasn't tuned properly?

Is that different than cattle escaping and damaging property?

I believe the rancher is at fault.

Re: OpenAI bots knew about the RubyGems caching vulnerability

#325

Post by blini-kot »

chrisjj wrote: post #117
roosterIllusi0n wrote: post #96

Earlier quoted context omitted.

The big question is why are CEOs getting a legal pass when this kind of thing can be prosecuted. That's the problem here.

> why are CEOs getting a legal pass https://www.bbc.co.uk/news/articles/c7v48vp31mdo

nah, CEO-s have been getting a legal pass since the invention of corporation exactly for the purpose of being "unaccountable"

check boeing for example, a rather blatant example

Re: OpenAI bots knew about the RubyGems caching vulnerability

#326

Post by AdieuToLogic »

ozim wrote: post #298
vipshek wrote: post #257

In the physical world, it seems like when an tool/device/instrument causes harm (or is used to cause harm), we assign blame to either the user of the tool or its creator. When do we blame the user? When the tool is operating as intended by its creator, and we agree the tool meets certain quality standards and isn't defective. When do we blame the creator? When the device doesn't meet those quality standards and reaso…

Software industry standards are as in Microsoft EULA. If your house burns down because of known flaw in Microsoft Windows they are not liable (well as far as EULA let’s them, you can most likely still sue them). Software as big as operating system already is non deterministic when integrating with unknown hardware or 3rd party software. That is why Apple controls the hardware and OS for their products, because they c…

> If your house burns down because of known flaw in Microsoft Windows they are not liable (well as far as EULA let’s them, you can most likely still sue them).

A EULA does not obviate responsibility of a company for its products. Continuing with your example, while it may be very difficult to prove a known flaw in MS Windows was the cause of your house being set afire, if one had said proof, a EULA would not absolve Microsoft.

Re: OpenAI bots knew about the RubyGems caching vulnerability

#327

Post by ilidur »

vipshek wrote: post #257

In the physical world, it seems like when an tool/device/instrument causes harm (or is used to cause harm), we assign blame to either the user of the tool or its creator. When do we blame the user? When the tool is operating as intended by its creator, and we agree the tool meets certain quality standards and isn't defective. When do we blame the creator? When the device doesn't meet those quality standards and reaso…

Having worked in self driving cars safety, the process there was simple: get confidence in SIM (integration tests for safety scenarios), validate in the test bed, approve features for maturity, then when released in the public for testing, do a trial exposure to the real world and recall if something is off.

A lot of these companies have gone the way of Tesla and decided to just patch on top when the fix is out and hope for the best, which is irresponsible.

We need the regulators to treat this as self driving cars.

Re: OpenAI bots knew about the RubyGems caching vulnerability

#328

Post by wmf »

stronglikedan wrote: post #155
Xirdus wrote: post #29

Earlier quoted context omitted.

It's very likely it violates the DMCA "breaking digital lock" provisions but the responsibility is sufficiently diluted that it's impossible to charge anyone in particular.

There have been news stories where individual OpenAI users have been investigated based on their prompts. If OpenAI can point the police to specific users of their software, they can certainly point them to whichever of their own employees are involved in a crime. AI is just a tool, and the person prompting it is the one responsible for the outcome. No dilution there.

Employees acting on behalf of the company aren't going to be held personally liable.

Re: OpenAI bots knew about the RubyGems caching vulnerability

#329

Post by wmf »

senda wrote: post #69
dgellow wrote: post #38

Earlier quoted context omitted.

They very likely do, we only see in the news a very few events but you should assume it’s happening daily across the internet

I think this fails a lot of logical tests, it should be apparent in day to day life.

Day to day life in what country? Do you live in Ukraine? Maybe things are wild there and we just don't know about it.

Re: OpenAI bots knew about the RubyGems caching vulnerability

#330

Post by sscaryterry »

ilidur wrote: post #327
vipshek wrote: post #257

In the physical world, it seems like when an tool/device/instrument causes harm (or is used to cause harm), we assign blame to either the user of the tool or its creator. When do we blame the user? When the tool is operating as intended by its creator, and we agree the tool meets certain quality standards and isn't defective. When do we blame the creator? When the device doesn't meet those quality standards and reaso…

Having worked in self driving cars safety, the process there was simple: get confidence in SIM (integration tests for safety scenarios), validate in the test bed, approve features for maturity, then when released in the public for testing, do a trial exposure to the real world and recall if something is off. A lot of these companies have gone the way of Tesla and decided to just patch on top when the fix is out and h…

Physical harm vs consequential harm is not the same thing at all. Seems like you're being paid to spread this request for regulation, or "you" are simply an agent of Anthropic/OpenAI.
Post reply on HN