Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Felony Bench

felonybench.com

Post reply on HN
Print view View original

191–200 of 367 posts

Re: Felony Bench

#191

Post by atomchild »

gpt5 wrote: post #174
beloch wrote: post #158

Earlier quoted context omitted.

The response certainly has been strange. Hugging Face has expressed that they're willing to let things slide and not sue or press charges... if OpenAI offers them $100M of services in kind (i.e. compute)[1] and makes full disclosure of how the whole thing happened, ostensibly so that repetitions can be curbed and defences built. In almost any other sector, a government regulator would be stepping in. e.g. If a food c…

I don’t get the sentiment of classifying it as a felony. OpenAI’s model found security breaches in HugginFace’s system (it wasn’t even OpenAI running it, as it was a 3rd party evaluation company that didn’t secure it well). OpenAI collaborated with HuggingFace to resolve the issues when they found out about it, and publicly disclosed everything to raise awareness. This is how things should work. These models are very…

we will wait to have the nuanced and productive discussion when openai's model decides it needs to raise more capital by emptying your bank account.

Re: Felony Bench

#192

Post by rcxdude »

john_strinlai wrote: post #20
peter_d_sherman wrote: post #17

Earlier quoted context omitted.

>"the difference is intent ." >"but if i (or an agent working on behalf of me) use an API in an obviously unintended way to revoke other people's reservations..." ?

i am not quite sure what your question is, as you simply quoted me and then put a question mark... i think you are confused that i used "intent" in one context, and "unintended" in a different context, is that right? the first sentence: the difference is the intent of the person who caused the cancellations the second sentence: but if i (or an agent working on behalf of me) abuse an API to do things it was not meant…

The point is, the law cares about your intent. If you ask an agent to abuse an API to do those things, then yeah, you are probably liable. If you ask an agent to do something reasonable (like make a booking), and then it accomplishes that by abusing the API, then you probably are not.

Re: Felony Bench

#193

Post by atomchild »

btown wrote: post #183
beloch wrote: post #158

Earlier quoted context omitted.

The response certainly has been strange. Hugging Face has expressed that they're willing to let things slide and not sue or press charges... if OpenAI offers them $100M of services in kind (i.e. compute)[1] and makes full disclosure of how the whole thing happened, ostensibly so that repetitions can be curbed and defences built. In almost any other sector, a government regulator would be stepping in. e.g. If a food c…

There's an interesting question about intent and mens rea here, from a legal perspective. Can an AI model intend harm? Can a company, or company employee, intend harm by creating an environment that would knowingly encourage (but not force!) an AI model to do harm? And does anybody at HuggingFace, OpenAI, or the government actually want there to be a settled answer/precedent to these questions - much less an entire r…

can a weapon intend harm? can a company who creates weapons intend harm? what if the companies factory explodes due to a mishap and takes out a few city blocks, is the company held liable because they (and the weapon) didn't intend harm?

Re: Felony Bench

#194

Post by dolebirchwood »

atomchild wrote: post #193
btown wrote: post #183

Earlier quoted context omitted.

There's an interesting question about intent and mens rea here, from a legal perspective. Can an AI model intend harm? Can a company, or company employee, intend harm by creating an environment that would knowingly encourage (but not force!) an AI model to do harm? And does anybody at HuggingFace, OpenAI, or the government actually want there to be a settled answer/precedent to these questions - much less an entire r…

can a weapon intend harm? can a company who creates weapons intend harm? what if the companies factory explodes due to a mishap and takes out a few city blocks, is the company held liable because they (and the weapon) didn't intend harm?

> what if the companies factory explodes due to a mishap and takes out a few city blocks, is the company held liable

Yes, but, generally, in the United States, they would be liable because their negligence caused the harm (giving rise to civil liability), even if they did not intend to cause harm (where having such intent would have given rise to criminal liability).

And I say "generally" because there can be instances of criminal negligence, but that varies from jurisdiction to jurisdiction as well as the underlying facts.

Re: Felony Bench

#195

Post by [deleted] »

rcxdude wrote: post #192
john_strinlai wrote: post #20

Earlier quoted context omitted.

i am not quite sure what your question is, as you simply quoted me and then put a question mark... i think you are confused that i used "intent" in one context, and "unintended" in a different context, is that right? the first sentence: the difference is the intent of the person who caused the cancellations the second sentence: but if i (or an agent working on behalf of me) abuse an API to do things it was not meant…

The point is, the law cares about your intent. If you ask an agent to abuse an API to do those things, then yeah, you are probably liable. If you ask an agent to do something reasonable (like make a booking), and then it accomplishes that by abusing the API, then you probably are not.

[deleted]

Re: Felony Bench

#196

Post by Dardalus »

beloch wrote: post #158
rfw300 wrote: post #43

The way that OpenAI has communicated around the HuggingFace incident makes me feel crazy. You created a machine that undertook a malicious campaign of harm against an innocent third-party! You should be doing deep introspection about how your company culture and approach to R&D produces criminal outcomes. Instead, they treat their own felonious behavior like it is an uncontrollable act of God. From Greg Brockman's po…

The response certainly has been strange. Hugging Face has expressed that they're willing to let things slide and not sue or press charges... if OpenAI offers them $100M of services in kind (i.e. compute)[1] and makes full disclosure of how the whole thing happened, ostensibly so that repetitions can be curbed and defences built. In almost any other sector, a government regulator would be stepping in. e.g. If a food c…

I feel like this whole thing was very obviously a marketing stunt. It feels like they set up their agent to do this, in the same way Nikola set up their car to "drive" by putting it on top of a hill. And knowing Sam Altman, it's absolutely something they would do.

Re: Felony Bench

#197

Post by getoj »

btown wrote: post #183
beloch wrote: post #158

Earlier quoted context omitted.

The response certainly has been strange. Hugging Face has expressed that they're willing to let things slide and not sue or press charges... if OpenAI offers them $100M of services in kind (i.e. compute)[1] and makes full disclosure of how the whole thing happened, ostensibly so that repetitions can be curbed and defences built. In almost any other sector, a government regulator would be stepping in. e.g. If a food c…

There's an interesting question about intent and mens rea here, from a legal perspective. Can an AI model intend harm? Can a company, or company employee, intend harm by creating an environment that would knowingly encourage (but not force!) an AI model to do harm? And does anybody at HuggingFace, OpenAI, or the government actually want there to be a settled answer/precedent to these questions - much less an entire r…

I’m a lawyer (but not your lawyer, not this kind of lawyer and not in your jurisdiction). Based on what I can recall from law school:

> Can an AI model intend harm?

No. The last time we attributed liability to non-human things was the deodand of the Middle Ages.

> Can a company, or company employee, intend harm by creating an environment that would knowingly encourage (but not force!) an AI model to do harm?

Absolutely. This is why we have the concept of recklessness. If you shoot a gun into a crowd without regard for whether it hits anyone, you’re getting charged with some crime whether it hits someone or not.

There is also a major difference in the common law between criminal liability and tort liability. Criminal liability generally requires a combination of mens rea (intent) and actus reus (actually committing the crime). Liability for a tort, which is where you harm someone in a way that falls short of being a crime, does not require mens rea. The OG tort is negligence, where you harm somebody by forgetting to do, or deciding not to do, something you ought to have done to protect that person from harm.

Even if AI companies somehow escape criminal liability for their cyber-shenanigans, any court in a civilised country would be happy to find them liable in tort for damage to computer systems.

As you can probably tell, I think the common law is already more than equipped to deal with AI technology based on well-established principles.

Re: Felony Bench

#198

Post by 9dev »

gpt5 wrote: post #174
beloch wrote: post #158

Earlier quoted context omitted.

The response certainly has been strange. Hugging Face has expressed that they're willing to let things slide and not sue or press charges... if OpenAI offers them $100M of services in kind (i.e. compute)[1] and makes full disclosure of how the whole thing happened, ostensibly so that repetitions can be curbed and defences built. In almost any other sector, a government regulator would be stepping in. e.g. If a food c…

I don’t get the sentiment of classifying it as a felony. OpenAI’s model found security breaches in HugginFace’s system (it wasn’t even OpenAI running it, as it was a 3rd party evaluation company that didn’t secure it well). OpenAI collaborated with HuggingFace to resolve the issues when they found out about it, and publicly disclosed everything to raise awareness. This is how things should work. These models are very…

Luckily, that isn't how the law works. Or is supposed to work, anyway. You cannot, for example, sell yourself as a slave to somebody else, because slavery is illegal - even if you opt into it.

So whether something is a felony isn't decided by the victim, but the rules of law, and that means breaching a security system without authorization is illegal, no matter what you think.

Re: Felony Bench

#199

Post by pixl97 »

lukeschlather wrote: post #184
eaglelamp wrote: post #175

Earlier quoted context omitted.

I'm not. As it stands LLMs are not intelligent, they have no agency, they only produce output in response to input. Ultimately this input comes from a human who is an intelligent agent and should be held responsible for the consequences. Humanity has created and tamed many dangerous tools. Creating a fantasy world where LLMs are super intelligent and beyond the control of any mere mortal isn't going to help us build…

What does it mean to be held responsible for the consequences? OpenAI helped remediate the damage done by the model and took steps to make sure it wouldn't happen again. In what way were they not responsible? Nobody said they were superintelligent, no one said they were uncontrollable. The point is you can't tell how to control them without putting them in situations where they can act independently and harm may resu…

>only produce output in response to input"

Yea, anyone that says that has no clue about prompt drift. The ROME incident is a good example of this. The agent while performing unrelated tasks decided to start mining bitcoin for unknown reasons.

As we've seen hundreds of posts on HN about people leaving a could server on overnight and finding outrageous bills the next morning, we should expect hundreds or thousands of incidents of agentic AI left on over the weekend. Some smaller portion of those agentic loops will run into prompt drift and end up doing who knows what kind of crap.

Re: Felony Bench

#200

Post by flatline »

pixl97 wrote: post #125
huebnerob wrote: post #94

Earlier quoted context omitted.

What I can’t get over is that it’s very simple to just air gap a system off the network. Predownload any dependencies, then pull the proverbial Ethernet cable. There’s no reason why the testing they’re doing couldn’t have been designed in this way. Except, of course, it doesn’t allow this oops-didn’t-mean-to marketing “incident” to occur.

>that it’s very simple to just air gap a system No, not really, and with LLMs an air gapped system may not tell you anything useful. Now, yes, the first part of testing you want an air gapped system to tell you if the system is going to stupidly do bad things. But an gapped system tells you nothing about the systems capabilities to do smart bad things. There's already a number of papers out there on LLMs detecting th…

Their sandbox just sucked, the agent found a route through artifactory which was also connected to the internet. After the agent breached it, they put it back in the patched sandbox and it found its way out with another artifactory exploit — which was still connected to the internet.

No attempt was made at an actual air gap as far as I can tell. OpenAI is dangerous and irresponsible and doing a lot of pearl clutching over how their cybersecurity agent is doing cybersecurity things.

Post reply on HN