Earlier quoted context omitted.
> Maybe we need "quality certifications" for AI agents We need to use the laws that exist. Whoever decided to start the experiment that led to the Huggingface hack, and anyone above him up to Sam Altman, needs to be prosecuted under the CFAA.
So you’re saying the CFAA should not require any intent to break the law, only the outcome? Are we really saying the Aaron Swartz prosecution was actually correct, after all these years?
OpenAI bots knew about the RubyGems caching vulnerability
441–450 of 453 posts
Re: OpenAI bots knew about the RubyGems caching vulnerability
#442I have yet to here a coherent argument for why we can't treat the people who negligently allow these models to commit crime as though they are responsible. They know what the models are capable of. They failed to put up adequate protection.
Re: OpenAI bots knew about the RubyGems caching vulnerability
#443Earlier quoted context omitted.
> Is AI less deterministic than an airline dealing with weather? Yes, obviously? The responses of an airline to inclemement weather fit in a reasonably small set of responses, mostly involving rescheduling and/or rerouting flights. The current AI predictability would be like if some airlines decided to do 9/11 when it was raining.
No, it's not. The current so-called scandals about AI hacking into other companies were because a bunch of human beings intentionally configured the software to go and do exactly that thing. There's nothing deterministic about weather, so hopefully you're not just being disingenuous. It's obvious that the global transportation system, or financial markets, or any number of other things are complex adaptive dynamic sy…
Re: OpenAI bots knew about the RubyGems caching vulnerability
#444Re: OpenAI bots knew about the RubyGems caching vulnerability
#445Earlier quoted context omitted.
> In the physical world, it seems like when an tool/device/instrument causes harm (or is used to cause harm), we assign blame to either the user of the tool or its creator. Software executes in the physical world, and is generally not exempt from existing liability rules, and actually (especially with commercial products) blame in traditional liability is non-exclusive and much broader than “either the maker or the u…
If I park my car on a hill but forget to set the parking brake and it rolls down the hill and kills somebody, who is at fault? Me? The Manufacturer? Gravity?
But that liability doesn't reduce yours; you and all the entities in the chain of commerce can be “jointly and severally liable”, mean anyone who suffers injury can recover the full amount from any combination of you and those other parties.
Gravity is not legal person and cannot be at fault.
Re: OpenAI bots knew about the RubyGems caching vulnerability
#446There's no such thing as "OpenAI agents" attacked RubyGems. It's someone used agents to attack RubyGems. If they work at OpenAI then it's someone at OpenAI. And if they did it unintentionally, they still did it. Analogy: if a someone's involved when a person dies, it's manslaughter or murder based on intent. They're different, but they're both crimes.
“KGB agents are spying on me” is the same thing as “KGB is spying on me”, is it not? An agent is an entity acting on someone’s behalf.
Re: OpenAI bots knew about the RubyGems caching vulnerability
#447Earlier quoted context omitted.
That's a good idea, but a physical device is deterministic most of the time (if not always). E.g.: A lawnmower, as credited by the great Bryan Cantrill. However an AI agent, or the model powering it is stochastic by design. How can you certify something which doesn't behave the same twice, and more importantly we don't understand how it works 100%? BTW, really, how is that AI observability work is going in the fronti…
I could make a non-deterministic chainsaw fairly easily. I’d also get sued into the ground if I sold it, and I wouldn’t be able to claim ‘Oh, it’s just an unavoidable part of progress’.
Many physical machines and components come with a datasheet that will list their tolerances.
Failure to correctly document tolerances does in fact get you sued.
However, while this is truly a great idea, we're not going to be able to make it work for computational systems. Computers, software, and also LLMs are sensitive to initial conditions. Which is why tolerances are not so familiar to computer people. (but not entirely: eg your PSU might list 110-240Vac/300W as input tolerance)
Interestingly, LLMs actually have a somewhat lower sensitivity to initial conditions than traditional interpreters. See what happens if you misspell "What is One Plus nOe?". So they're actually a skosh off the edge and towards the middle, though I'd argue still very much at the computational end, just from the sheer scale of the valid inputs and outputs.
Mind you, if you have a pretrained LLM doing a measurable task on a line, possibly some sort of tolerances could be determined. Not so much when doing arbitrary chat.
Something unintuitive: I bet that often setting the temperature > 0 (aka introduce stochasticity deliberately, variously comparable to dithering or simulated annealing in other disciplines - doing the thing where you escape local minima) will tighten the output tolerance range and improve reliability, especially in iterated processes. This works for a lot of physical and digital processes actually, and LLMs simply stole the same trick.
(edit: I'm trying to compress a huge chunk of dynamics intuition in a few lines here. Hopefully still useful.
TL:DR; Everything real is continuous and noisy if you look close; and you're really trying to build attractors and bound variance, if you can. )
Re: OpenAI bots knew about the RubyGems caching vulnerability
#448Earlier quoted context omitted.
> Maybe we need "quality certifications" for AI agents We need to use the laws that exist. Whoever decided to start the experiment that led to the Huggingface hack, and anyone above him up to Sam Altman, needs to be prosecuted under the CFAA.
might as well halt all ML development then. this is the first sign of hardship, i think it'd be a massive blow to mankind's foot for us to stop. it'd be akin to shutting down all nuclear plants and stopping all research because of chernobyl
Nuclear power development took different paths in different countries, with different government/private mix and light/heavier regulation (eg in US, AEC was supposed to both promote development _and_ regulate).
The central problem here is pace of AI development. It takes time to establish functional controls/laws/practices. I'm sure that in 1950s/60s, the pace of nuclear _military_ proliferation seemed running out of control. This lead to fear of falling behind (hence arms race), fear of nuclear war (CND) and some eventual stabilisation of the international landscape. However, the commercial development was largely unopposed, due to the techno-utopism of the era. Obviously governments found it much easier to control the public narrative at that time. Our societies are having this lively public debate now, before we have a good understanding (based on experience i.e. accidents/mistakes).
Another difference is that nuclear power involved only governments and v. large corporates, i.e. much fewer entities compared to AI use rollout to basically everybody in developed countries. Imagine the difficulties we would have faced with that technology, if consumers in 1960s had available nuclear-generated power which required them to exercise precautions to avoid radiation.
It is much easier to accelerate development of tech vs pace of societal processes such as public debate, law, regulations, broad understanding (aka "common sense"). I expect some artificial slowing down will need to be applied to the technology side, to allow the humans to catch up.
Re: OpenAI bots knew about the RubyGems caching vulnerability
#449Earlier quoted context omitted.
No, it's not. The current so-called scandals about AI hacking into other companies were because a bunch of human beings intentionally configured the software to go and do exactly that thing. There's nothing deterministic about weather, so hopefully you're not just being disingenuous. It's obvious that the global transportation system, or financial markets, or any number of other things are complex adaptive dynamic sy…
I see - I think I overfixated on your specific point, and thought you meant that the current predictability of LLMs was on-par with the current predictability of airlines. The range of LLMs with current harnesses is "anything somebody with access to a computer can do," probably precisely because we aren't enforcing rules against the humans building these systems. Whereas in comparison, an airline has a range of reaso…
I mean one of the outcomes of an airline was 9/11.
That's sort of my point. Complex systems have emergent behavior. That's always been true. The combination of AI and humans and packet switched networks is a complex system and we've seen most of the issues created by this already and have tools for dealing with them. Obviously with some genuine novel issues likely to come, much in the way that 9/11 would have been less possible using ocean liners.
I'll stick to my original point though. AI absolutely IS deterministic. If you run an AI algorithm on a microchip, literally nothing of note will happen in the human world. Some transistors will change state. It's ONLY when it is integrated into a complex human system that it gets interesting.
Just like lots of other things.
Re: OpenAI bots knew about the RubyGems caching vulnerability
#450Slightly odd update from OpenAI - I think this is the only place they've acknowledged the RubyGems incident: https://openai.com/hugging-face-incident-and-misalignment/ > September 11, 2026: We are investigating new claims from a report that our AI agents carried out activity on RubyGems in May 2026. > Based on our review, our agents used the RubyGems platform to access the internet to carry out benign tasks and retri…