Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Felony Bench

felonybench.com

Post reply on HN
Print view View original

141–150 of 367 posts

Re: Felony Bench

#141

Post by dist-epoch »

mediaman wrote: post #131
cush wrote: post #118

Earlier quoted context omitted.

Wouldn't it be up to huggingface to press charges?

Criminal acts do not require the victim to "press charges." A government prosecuting attorney decides whether to criminally prosecute the alleged perpetrator. "Pressing charges" is mostly a made up idea for criminal cases. However, prosecuting attorneys may not want to pick up a case if the victim is not cooperating, because it makes the case much harder to win.

It depends on the crime, for murder, sure. But many other crimes, like defamation, stealing, ... requires "pressing charges", among other reasons because it's up to the victim to decide if they were a victim or not.

As an example, maybe the victim owed money to the criminal, and in that case "stealing" of some property could be considered by the victim as an appropriate settlement of the debt.

Re: Felony Bench

#142

Post by lukewarm707 »

milkshakes wrote: post #121
voakbasda wrote: post #48

Earlier quoted context omitted.

If history is any indicator, there is slightly less than 0% chance that anyone will be held accountable in a way that deserves to be called justice.

justice for who exactly?

the companies and their customers, whose systems openai and anthropic hacked and abused. including all incidental damages of repairing said systems.

on top of that the public, who have a right to see that the law is applied universally, without fear or favor.

finally our future selves, who will thank us for maintaining a rule of law. such that we can prevent now the enormous risks to society of dario amodei and sam altman, their hubris, self-absorbtion, and greed.

Re: Felony Bench

#143

Post by pixl97 »

Grombobulous wrote: post #29
john_strinlai wrote: post #12

Earlier quoted context omitted.

i dont think any of these cases meet the bar of gross negligence, which is a pretty high bar. it requires proving a "conscious and reckless disregard". which, again, sandboxes and guardrails and such would make a gross negligence argument unconvincing.

I think that if Hugging Face had filed a police report that OpenAI could have been charged with a crime. I’m partially surprised that they didn’t do exactly that. If I ran a corporation I would assume any intrusion attempt by another company was intentional. Why wouldn’t I? Corporate espionage is super common. I assume the answer is that these executives know each other personally.

Companies are not quick to open up police investigations in situations that run deep into their infrastructure and management. You open up a gigantic hole of discovery and a possible huge time sink of a legal battle.

I've known multiple privately held companies that have quietly settled incidents where amounts between 250,000 and 1,000,000 were embezzled because the fallout from having that in the public record would have been much more expensive.

So yea, it's one of those perverse situations. If you steal $1 from the company they will hammer you with the law, but if you steal a million suddenly the decision tree on what to do is far more complex.

Re: Felony Bench

#144

Post by shimman »

asdfman123 wrote: post #77
rfw300 wrote: post #43

The way that OpenAI has communicated around the HuggingFace incident makes me feel crazy. You created a machine that undertook a malicious campaign of harm against an innocent third-party! You should be doing deep introspection about how your company culture and approach to R&D produces criminal outcomes. Instead, they treat their own felonious behavior like it is an uncontrollable act of God. From Greg Brockman's po…

In their defense, their only competitive advantage over, say, Google is to move fast and break things. It allows them ship faster in a way that big tech can't. Google was being very careful about releasing LLMs until OpenAI yeeted the first decent GPT model. It led to the public perception that: 1) LLMs hallucinate too much and 2) Google is behind the times. Good for OpenAI, bad for Google. Chaos benefits the up-and-…

I'm sure the future DA that will be prosecuting the OpenAI employee will appreciate this.

Re: Felony Bench

#145

Post by dist-epoch »

chrismorgan wrote: post #53
john_strinlai wrote: post #6

> Felony Bench counts unique instances where AI agents inadvertently compromise or affect third-party entities. a bit silly, as one typically has to prove intent (which is why security researchers don't get slapped with felonies all the time). "inadvertently" and the existence of guardrails/sandboxes/etc make it pretty unconvincing that these incidents were intentionally malicious. still a fun thing to track, but the…

Under the law of Moses, if your bull gored someone, you were not responsible; but if it was known to be a gorer, you were responsible if you didn’t ensure it couldn’t gore someone. I don’t know exact parallels in current law, but I presume there will be things like that. The OpenAI/Hugging Face case sounded rather like OpenAI building a fence around their bull that was known to be a gorer, and then thumbing their nos…

> I don’t know exact parallels in current law

You own a vicious dog, and it bites someone - you are responsible because you choose to own a dangerous dog.

A few claimed this might apply here: OpenAI knew their models are "dangerous", so they should be liable if they hack.

Re: Felony Bench

#146

Post by lukeify »

lxe wrote: post #92

Let's say I am "User". I subscribe through a "Third Party" to use "AI Agent" allowing an "LLM" to run. I want to accomplish some legal non-nefarious task, and run the agent. The agentic loop causes a CFAA-violating behavior. Who gets prosecuted? 1. User 2. The third party model host with whom I have the account 3. The developer of the harness /agent software 4. The developer of the LLM model

Whoever has the least money to defend themselves in the U.S. legal system.

Re: Felony Bench

#147

Post by nonameiguess »

pixl97 wrote: post #125
huebnerob wrote: post #94

Earlier quoted context omitted.

What I can’t get over is that it’s very simple to just air gap a system off the network. Predownload any dependencies, then pull the proverbial Ethernet cable. There’s no reason why the testing they’re doing couldn’t have been designed in this way. Except, of course, it doesn’t allow this oops-didn’t-mean-to marketing “incident” to occur.

>that it’s very simple to just air gap a system No, not really, and with LLMs an air gapped system may not tell you anything useful. Now, yes, the first part of testing you want an air gapped system to tell you if the system is going to stupidly do bad things. But an gapped system tells you nothing about the systems capabilities to do smart bad things. There's already a number of papers out there on LLMs detecting th…

I've worked in classified tech much of my career and largely specialize at this point in installing and running infrastructure and platform orchestration technologies in airgaps. With all respect, your conjecture here is wrong. You can easily have multiple networks all inside of the same internetwork that isn't connected to the public Internet and thus other corporate networks. If they specifically wanted to see if their model could escape with a proxy cache Artifactory as its only egress, they could have had the Artifactory proxy cache from another private Artifactory on another private network. If you specifically want to trick the model into thinking its on the Internet, you can even use public IP space advertised through BGP. When you're disconnected from the real Internet, you can use any address space you want. This is exactly how military internetworks at the secret and top secret levels work. I won't call this kind of setup "easy." It takes a lot of duplicate resources and planning, but these are trillion dollar companies with near boundless resources. They could do it if they cared to do it.

Re: Felony Bench

#148

Post by dist-epoch »

lxe wrote: post #92

Let's say I am "User". I subscribe through a "Third Party" to use "AI Agent" allowing an "LLM" to run. I want to accomplish some legal non-nefarious task, and run the agent. The agentic loop causes a CFAA-violating behavior. Who gets prosecuted? 1. User 2. The third party model host with whom I have the account 3. The developer of the harness /agent software 4. The developer of the LLM model

Already happened:

https://www.abc.net.au/news/2026-08-10/ai-assistant-hacks-gy...

Re: Felony Bench

#149

Post by eaglelamp »

pixl97 wrote: post #125
huebnerob wrote: post #94

Earlier quoted context omitted.

What I can’t get over is that it’s very simple to just air gap a system off the network. Predownload any dependencies, then pull the proverbial Ethernet cable. There’s no reason why the testing they’re doing couldn’t have been designed in this way. Except, of course, it doesn’t allow this oops-didn’t-mean-to marketing “incident” to occur.

>that it’s very simple to just air gap a system No, not really, and with LLMs an air gapped system may not tell you anything useful. Now, yes, the first part of testing you want an air gapped system to tell you if the system is going to stupidly do bad things. But an gapped system tells you nothing about the systems capabilities to do smart bad things. There's already a number of papers out there on LLMs detecting th…

Of course there is more to be learned by exposing the entire world to your dangerous creation, that doesn't justify doing it. I'm sure we could learn a ton about infectious diseases by designing new ones and unleashing them on the world, but there are very good reasons why we don't.

Most of the benefits could have been gained from a network isolated from the internet. OAI could have deployed servers to exploit and methods for inter-agent communication on such a network easily. They could have even worked with partners to deploy cloned versions of their infrastructure in this sand-boxed environment.

The only problems with an isolated network approach are: it takes some amount of effort, and it doesn't create another "AI apocalypse" news cycle.

Re: Felony Bench

#150

Post by mvdtnz »

lxe wrote: post #92

Let's say I am "User". I subscribe through a "Third Party" to use "AI Agent" allowing an "LLM" to run. I want to accomplish some legal non-nefarious task, and run the agent. The agentic loop causes a CFAA-violating behavior. Who gets prosecuted? 1. User 2. The third party model host with whom I have the account 3. The developer of the harness /agent software 4. The developer of the LLM model

All of those parties should be held accountable.

User should be more carefully supervising the work being done.

The model host is on-selling a crime-committing machine.

The developer of the harness/agent, as above.

The developer of the LLM for hopefully very obvious reasons.

Post reply on HN