Live data from Hacker News

OpenAI bots knew about the RubyGems caching vulnerability

tenderlovemaking.com

451–453 of 453 posts

Re: OpenAI bots knew about the RubyGems caching vulnerability

#451
post #113

Earlier quoted context omitted.

> There are no negligent or stochastic hacking laws I'm sure that Andrew Auernheimer would be pleased to hear that. [0] For accessing a publicly accessible endpoint, that was completely undefended and didn't actually require "hacking", he was convicted of "exceeding authorised access". You _don't_ have to show intent under the Computer Fraud and Abuse Act, for the first count. > knowingly accesses a computer without…

> I'm sure that Andrew Auernheimer would be pleased to hear that. [0] For accessing a publicly accessible endpoint, that was completely undefended and didn't actually require "hacking", he was convicted of "exceeding authorised access". Frankly he got off too easy, but we haven't explicitly outlawed "being a malicious dipshit" so he got convicted on the closest available charge. > Chat logs obtained by the prosecutio…

Andrew Auernheimer is an indefensible malicious dipshit, by definition. However, his charges were vacated by a higher court a little over a year after he was incarcerated, and he is now "free range troll". So, is case is not illustrative of the useful application of law against 'hacking' when the target is an unsecured endpoint, even if it does at first appear that he was held responsible for some of his nefarious activities.

It would appear that the inability of the US Justice Dept. to successfully hold even an odious abuser of regulation with minimal legal defense funds responsible results in these exact observable outcomes: enterprise legal team (to the extent that such exists as OAI and elsewhere) correctly surmises that the actual risk of prosecution and detention for anyone operating these agentic workloads is minimal and the cost of defending them is justifiable.

Thus, in their legal opinion, it is permissible for the company/employees/director to engage in what would appear to be somewhere between malicious and irresponsible behavior. These conditions have been demonstrably true for at least decade in the US, and for all of us to pretend as-if the legal system is going to rescue us from this and other malfeasance by frontier models points of origin borders on, to phrase it quite simply, willfully ignorant.

I don't know what the effective alternate option for literally all of the internet facing systems might need to be in order to mitigate what is now an open problem: multi-layered, persistent, machine speed penetration and data exfiltration with the potential to use manipulation and extortion against human package maintainers and code repositories to operate, but it isn't 'carry on like someone is going to make them stop', or 'pretend this isn't a threat to my business model'.

A thousand percent, a million billion trillion percent agreement that the operators are the malicious dipshits - because code is always a reflection of the hands that made it. Code can only do what it is intended to do, even if the coders gnash and wail that it "escaped"; the only time code is not working as intended is when it fails to compile and run. Any other functional result follows from the decisions of the humans who designed it. Full stop.

For myself, I see the potential for a descent into a cognitive dark forest [0] condition, and for companies using the open web for private business communication to be in need of a coordinated move to obfuscated layers which can be made immune to training and these new attack aspects. Those who do not proactively defend themselves using in-house, on-prem, and open-weight or self-trained models can attempt to blame these nefarious actors for the coming losses, but that won't reverse the outcomes of waiting to be rescued by the system of law.

[0] https://www.restless-brain.com/p/the-cognitive-dark-forest-w...

/edit grammar, spelling

Re: OpenAI bots knew about the RubyGems caching vulnerability

#452

Earlier quoted context omitted.

So is there no liability for, say, a company that releases a known dangerous open-weight model, but fails to disclose that it is dangerous? How about a company that distributes malware under the guise of legitimate software?

Perhaps don't deploy random weights of unknown origin? Also not every model provider might be capable of babysitting all your uncontrolled agent deployments. If you want SLOs, get into a contractual relationship with entities whose weights you deploy, and also monitor your agents so they don't go off the rails. All this is just like deploying any other tech in the world eg. if you buy a car, or a chainsaw, or a book.

So no, then, to both questions?

Re: OpenAI bots knew about the RubyGems caching vulnerability

#453
post #277

Earlier quoted context omitted.

> "industry standards" common in, say, electrical engineering and other disciplines are sorely lacking here I think this misses the rather crucial fact that nobody can agree on a standard because nobody has the first idea what they're doing. I'm pretty sure there were very much fewer electrical engineering standards while it was all being first mass deployed, and after dozens to hundreds of fires and electrocutions p…

> nobody has the first idea what they're doing It is not that complicated for now. It is an algorithm on a loop and someone started it

I do not think you are appreciating the scale of this. Imo one of the main reason that there is no regulation is because nobody has any idea what would be a good regulation. People (like dario) make suggestions, and get rightly shut down because his suggestion massively favors his own company.
Post reply on HN