OpenAI agents carried out an undisclosed attack on RubyGems

1) Traffic-related laws straddle the boundary between civil/regulatory law and criminal law. Someone losing their driver's license or even paying a penalty for involuntary intoxication would still be consonant with criminal law principles. However, a criminal punishment would be aberrational. (Distinction between a civil penalty and criminal punishment usually turns on whether there's a moral purpose to the sanction. Jail time is usually but not always--cf civil contempt incarceration--considered a criminal punishment.)

2) Background principles notwithstanding, in theory a state could completely dispense with any morality-colored mens rea requirement, just as the UK Parliament could do whatever it wants to. The backstop would be Federal constitutional [substantive] due process guarantees.

2.a) Some quick searching shows that Texas nominally seems to have dispensed with this requirement for DWIs. See e.g. Farmer v. State, 411 S.W.3d 901 (Tex. Crim. App. 2013) and some discussion at https://www.ncdd.com/top-dui-attorneys-blog/involuntary-into... Without having fully read the case law, though (but some summaries of that and other cases), I suspect there might be some nuance that has allowed this to stand without a full majority accepting that the traditional principles have been completely thrown out. For example, even if someone didn't know they were taking Ambien, the simple act of voluntarily taking any pill without careful examination can be construed as a sufficiently culpable act. Still, it's a pretty big caveat.

2.b) Statutory rape is a classic strict liability crime. But most states will permit a mistake-of-fact defense. Some don't, but even there there's sometimes some nuance and rationalizing going on and the literature is crazy complex. Because this is a "think of the children" situation, most case will just have horrible facts.

3) A few states have nominally dispensed with insanity defenses, though Kansas stands out the most. SCOTUS upheld Kansas' law in Kahler v. Kansas, but in the majority opinion Kagan characterized the Kansas law as not abolishing the insanity defense but rather changing its shape, and she showed that there still remained elements for which a defendant could plea lacked the requisite intent. Also, regarding the Federal constitution acting as backstop, she reiterated that SCOTUS was reticent to establish strict metes & bounds about the general principles of criminal law that states could not stray beyond. Nonetheless, those principles clearly exist.

I had some other points, but now I've forgotten them. Also, minor pedantic point, but like "strict liability crime", some scholars consider "affirmative defense" to be oxymoronic. As a substantive matter there's not a strong distinction. It's a procedural distinction about initial burdens of proof, but in most if not all cases you can interpret an affirmative defense as simply placing a very weak initial burden on the prosecution that is implicitly met.

(Note, I'm not a practicing lawyer but do have a law degree.)

EDIT: Ah, point 4) Intent was a big sticking point in the Obamacare penalty case, Sebelius. Both the dissent and Roberts (the swing vote) reiterated that you couldn't have a penalty or punishment for doing nothing. (IIRC some of the majority opinions also echoed this.) That is, even in a civil context there has some to be some voluntary act, however remote, that puts someone in a position to be subject to legal liability. But as Roberts pointed out, the taxing power is the great exception, where you can be required to do something merely for existing, and thus penalized for not doing nothing properly. (And Roberts was the critical swing vote.)

EDIT EDIT: Also see, "Solving General and Specific Intent: A Mapping on the MPC and Applications to the Categorical Approach", https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4754469 In describing the distinctions between general and specific intent in criminal law, it also delves into the definitions of strict criminal liability (which can be construed as either very similar or identical to general intent crimes), and notes that SCOTUS generally inserts an implicit mens rea requirement when considering strict liability criminal statutes.

There's a better concept for that, and it's misalignment. LLMs only exhibit this kind of behavior when they are misaligned. Aligned LLMs would respect the boundaries of their sandbox and not try to break out.

From the outside (I'm just an user), what it looks like is that more powerful LLMs are usually less aligned. A small model might just perform your task in a narrow way, but a larger, more powerful model may strategize and achieve the goals through non-obvious means, and that's inherently harder to align.

But regardless, the important thing here is that the user prompt do not, and can not perfectly convey 100% of the goals of the agent. There's a wide range of goals that agents should follow implicitly. It's okay if the user can override some or most of those goals (specially if they go out of their way to use an abliterated open weights model), but the default should be to align themselves with broad human preferences that go beyond than just their immediate prompt.

Or saying otherwise, a scenario like the paperclip maximizer can only happen with a heavily, wildly misaligned AI, the kind of AI that might kill all humans some day.