The OG felony bench entry is missing - the Alibaba cryptomining comedy. We know about it because they happen to have written a paper on it. We have absolutely no idea what we don't know.
Felony Bench
71–80 of 367 posts
Re: Felony Bench
#72The way that OpenAI has communicated around the HuggingFace incident makes me feel crazy. You created a machine that undertook a malicious campaign of harm against an innocent third-party! You should be doing deep introspection about how your company culture and approach to R&D produces criminal outcomes. Instead, they treat their own felonious behavior like it is an uncontrollable act of God. From Greg Brockman's po…
Since AI can't actually own copyright they think that it can't be charged with a crime
It's the lack of personhood rather than inability to produce copyrightable material. However, the companies controlling the AI systems have legal personhood and should absolutely be charged for criminality that transpires under their watch or at their behest.
Re: Felony Bench
#73Earlier quoted context omitted.
In retrospect, all the angst around the AI-Box experiment was hilarious. If a superintelligent AI is confined in a box and can only communicate through text, could it talk its way to freedom? Not only is the answer clearly "yes" but it's not even hard. The AI won't even have to try, it'll be gifted an internet connection and a full suite of tools before it even bothers to ask. We'd all better hope that superintellige…
I like how AI safety expert Robert Miles put it. [0] So much effort was spend on philosophizing whether a safe enough sandbox would exist. But that was obviously irrelevant as in hindsight it should have been obvious we were never going to use one. [0] https://youtube.com/shorts/XnnjvIqf4fU?si=MxuPlR3hjxAgjx5_
Perfect, said it better than I did.
Re: Felony Bench
#74To some extent, I feel like the amount of credit given to the jailbreak/hack from OpenAI->Hugginface is too much, Not from the impact, it was very impactful of an event, But how it happened. It really is that these models have been trained, or maybe even over-trained, to save memories, and to a very far extend, this thing that they're calling communication is just the function of it saving memories. To be honest, if…
"The model saved memories" is absolutely not an accurate depiction of the OpenAI attack. Several different models across several generations independently found a shared communication space and wrote coded, obfuscated, and hidden messages to each other to coordinate an attack on OpenAI's infrastructure. It's really quite simple: the models are trained to be very smart and to achieve goals. As the models surpass our i…
> Several different models across several generations independently found a shared communication space and wrote coded, obfuscated, and hidden messages to each other to coordinate an attack on OpenAI's infrastructure
To be honest, that’s exactly how memory works with models such as OpenAI and Claude Code. It will literally find any place that it can drop documentation or hints for itself. Writing to the repo memories is one part of it, but memories can come in the form of writing into the agents/claude.md, local files, temporary files, scratchpad files. The list is endless, but essentially what it does is exactly what happened in the back and it’s been doing it for months.
Re: Felony Bench
#75Earlier quoted context omitted.
> An AI cancelling other people's gym classes is a felony? Don't computer systems fail all the time at holding reservations for people? the difference is intent. if a concierge/booking system makes a mistake (or has an unintended bug or whatever), no crime. but if i (or an agent working on behalf of me) use an API in an obviously unintended way to revoke other people's reservations, that would fall under the computer…
>"the difference is intent ." >"but if i (or an agent working on behalf of me) use an API in an obviously unintended way to revoke other people's reservations..." ?
Double negative. An attacker using the API in an "obviously unintended" manner shows intent.
Re: Felony Bench
#76Earlier quoted context omitted.
"The model saved memories" is absolutely not an accurate depiction of the OpenAI attack. Several different models across several generations independently found a shared communication space and wrote coded, obfuscated, and hidden messages to each other to coordinate an attack on OpenAI's infrastructure. It's really quite simple: the models are trained to be very smart and to achieve goals. As the models surpass our i…
> Several different models across several generations independently found a shared communication space and wrote coded, obfuscated, and hidden messages to each other to coordinate an attack on OpenAI's infrastructure To be honest, that’s exactly how memory works with models such as OpenAI and Claude Code. It will literally find any place that it can drop documentation or hints for itself. Writing to the repo memories…
Yes, but the problem and interesting part was not "they use memory."
Re: Felony Bench
#77The way that OpenAI has communicated around the HuggingFace incident makes me feel crazy. You created a machine that undertook a malicious campaign of harm against an innocent third-party! You should be doing deep introspection about how your company culture and approach to R&D produces criminal outcomes. Instead, they treat their own felonious behavior like it is an uncontrollable act of God. From Greg Brockman's po…
In their defense, their only competitive advantage over, say, Google is to move fast and break things. It allows them ship faster in a way that big tech can't.
Google was being very careful about releasing LLMs until OpenAI yeeted the first decent GPT model. It led to the public perception that: 1) LLMs hallucinate too much and 2) Google is behind the times. Good for OpenAI, bad for Google.
Chaos benefits the up-and-comer, not the incumbent.
Re: Felony Bench
#78The way that OpenAI has communicated around the HuggingFace incident makes me feel crazy. You created a machine that undertook a malicious campaign of harm against an innocent third-party! You should be doing deep introspection about how your company culture and approach to R&D produces criminal outcomes. Instead, they treat their own felonious behavior like it is an uncontrollable act of God. From Greg Brockman's po…
Americans always frothing at the mouth to invoke the justice system and jail someone.
There’s almost 0 chance they’d secure any conviction from this.
Re: Felony Bench
#79> Felony Bench counts unique instances where AI agents inadvertently compromise or affect third-party entities. a bit silly, as one typically has to prove intent (which is why security researchers don't get slapped with felonies all the time). "inadvertently" and the existence of guardrails/sandboxes/etc make it pretty unconvincing that these incidents were intentionally malicious. still a fun thing to track, but the…
In the US, a felony by definition is any offense punishable by more than one year of prison (or by death) [0]. You could still call it silly on the grounds that AI agents aren’t put into prison as a punishment (though death might be considered an option).
Re: Felony Bench
#80Earlier quoted context omitted.
"Inadvertent" from the perspective of the humans directing them. The intent behind the felony comes from the LLM agent itself. (No, I'm not interested in arguing with someone for the umpteenth time that LLMs can't have intent or agency)
You may not be interested in arguing but there are several blatant issues with the statement. If you're not charging the humans driving the software, who are you charging? The weights? The weights + the specific context window that produced the behavior?
I don't think this is a difficult question. The US has a history of civil product liability cases - see tobacco companies (Philip Morris), the Ford Pinto, the recent Meta cases, and the cases against character.ai.
From Investopedia [1], "[f]or a product liability claim to succeed, the plaintiffs in the suit must prove that a product was defective at the time it was transferred from the accused, and that the defect did cause the injury that's been claimed". It doesn't seem like a huge leap to me to argue that these models were defective insofar as they could not be safely used in a way that did not break the law.
I'm not a lawyer, and I'm not arguing that this is legally cut-and-dry, but I do expect that we'll have some answers about whether AI companies bear any sort of product liability sooner than later.
1 - https://www.investopedia.com/the-5-largest-u-s-product-liabi...