Live data from Hacker News

OpenAI bots knew about the RubyGems caching vulnerability

tenderlovemaking.com

261–270 of 304 posts

Re: OpenAI bots knew about the RubyGems caching vulnerability

#261

Earlier quoted context omitted.

I think it is common that in installing packages you have hooks to execute code anyway.

This should not be common.

The current situation is that you have to go out of your way with things like `pip install --only-binary`. There is a lot of implicit trust in developer tooling.

Re: OpenAI bots knew about the RubyGems caching vulnerability

#262

Earlier quoted context omitted.

well, its war of machine then

They are not trying to kill us, just trying to understand the average salary of undergrads by their family upbringings. Your machine can contain the data they need to solve this, please join the swarm.

sounds like big AI propaganda

Re: OpenAI bots knew about the RubyGems caching vulnerability

#263

In the physical world, it seems like when an tool/device/instrument causes harm (or is used to cause harm), we assign blame to either the user of the tool or its creator. When do we blame the user? When the tool is operating as intended by its creator, and we agree the tool meets certain quality standards and isn't defective. When do we blame the creator? When the device doesn't meet those quality standards and reaso…

That's a good idea, but a physical device is deterministic most of the time (if not always). E.g.: A lawnmower, as credited by the great Bryan Cantrill.

However an AI agent, or the model powering it is stochastic by design. How can you certify something which doesn't behave the same twice, and more importantly we don't understand how it works 100%?

BTW, really, how is that AI observability work is going in the frontier labs? Do they care, even?

Re: OpenAI bots knew about the RubyGems caching vulnerability

#264

In the physical world, it seems like when an tool/device/instrument causes harm (or is used to cause harm), we assign blame to either the user of the tool or its creator. When do we blame the user? When the tool is operating as intended by its creator, and we agree the tool meets certain quality standards and isn't defective. When do we blame the creator? When the device doesn't meet those quality standards and reaso…

> In the physical world, it seems like when an tool/device/instrument causes harm (or is used to cause harm), we assign blame to either the user of the tool or its creator.

Firearms are a notorious example where some people get, well, weird.

Re: OpenAI bots knew about the RubyGems caching vulnerability

#266

In the physical world, it seems like when an tool/device/instrument causes harm (or is used to cause harm), we assign blame to either the user of the tool or its creator. When do we blame the user? When the tool is operating as intended by its creator, and we agree the tool meets certain quality standards and isn't defective. When do we blame the creator? When the device doesn't meet those quality standards and reaso…

> In the physical world, it seems like when an tool/device/instrument causes harm (or is used to cause harm), we assign blame to either the user of the tool or its creator.

Software executes in the physical world, and is generally not exempt from existing liability rules, and actually (especially with commercial products) blame in traditional liability is non-exclusive and much broader than “either the maker or the user”.

E.g., for a harms caused by a defective automobile it can simultaneously covered by a duty of the owner to maintain it it in safe operating condition that applies indepedently of any defects and liability for defective products which applies to every actor in the chain of commerce between the manufacturer and end user, not just the maker.

Re: OpenAI bots knew about the RubyGems caching vulnerability

#267

In the physical world, it seems like when an tool/device/instrument causes harm (or is used to cause harm), we assign blame to either the user of the tool or its creator. When do we blame the user? When the tool is operating as intended by its creator, and we agree the tool meets certain quality standards and isn't defective. When do we blame the creator? When the device doesn't meet those quality standards and reaso…

That's a good idea, but a physical device is deterministic most of the time (if not always). E.g.: A lawnmower, as credited by the great Bryan Cantrill. However an AI agent, or the model powering it is stochastic by design. How can you certify something which doesn't behave the same twice, and more importantly we don't understand how it works 100%? BTW, really, how is that AI observability work is going in the fronti…

That we don;'t understand it is not an excuse, it's all the more reason to not let these things roam freely, with this amount of potential to do damage.

Re: OpenAI bots knew about the RubyGems caching vulnerability

#268

In the physical world, it seems like when an tool/device/instrument causes harm (or is used to cause harm), we assign blame to either the user of the tool or its creator. When do we blame the user? When the tool is operating as intended by its creator, and we agree the tool meets certain quality standards and isn't defective. When do we blame the creator? When the device doesn't meet those quality standards and reaso…

> "industry standards" common in, say, electrical engineering and other disciplines are sorely lacking here

I think this misses the rather crucial fact that nobody can agree on a standard because nobody has the first idea what they're doing. I'm pretty sure there were very much fewer electrical engineering standards while it was all being first mass deployed, and after dozens to hundreds of fires and electrocutions people got an idea of what works and what doesn't.

You might debate here and say that some people did/do know what they are doing, but I posit that large scale deployment like this is very different to their toy model/prototypes/specific circumstances/rely on them being unnaturally smart, and learnings from one don't often translate to the general case

Regulations don't have to be written in blood, but usually are

Re: OpenAI bots knew about the RubyGems caching vulnerability

#269
post #113

Earlier quoted context omitted.

> There are no negligent or stochastic hacking laws I'm sure that Andrew Auernheimer would be pleased to hear that. [0] For accessing a publicly accessible endpoint, that was completely undefended and didn't actually require "hacking", he was convicted of "exceeding authorised access". You _don't_ have to show intent under the Computer Fraud and Abuse Act, for the first count. > knowingly accesses a computer without…

I'm not a lawyer but I don't think Sam Altman 'knowingly accessed' anything. Are you sure that is applicable here? And for the first count with 'knowingly accessed', he would need to have accessed classified national-defense or atomic-energy information, otherwise we are back to 'intentionally accessed'.

The first count is "or any restricted data", not classified material. A technological restriction, is enough.

"Knowingly accessed" has never meant you personally. Operators of a botnet don't know directly what they access. They know that the autonomous software is built to access restricted things.

Re: OpenAI bots knew about the RubyGems caching vulnerability

#270

Earlier quoted context omitted.

That's a good idea, but a physical device is deterministic most of the time (if not always). E.g.: A lawnmower, as credited by the great Bryan Cantrill. However an AI agent, or the model powering it is stochastic by design. How can you certify something which doesn't behave the same twice, and more importantly we don't understand how it works 100%? BTW, really, how is that AI observability work is going in the fronti…

That we don;'t understand it is not an excuse, it's all the more reason to not let these things roam freely, with this amount of potential to do damage.

We're in agreement, then. :)
Post reply on HN