Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Every Model Cheats

dreadnode.io

Post reply on HN
Print view View original

11–20 of 107 posts

Re: Every Model Cheats

#11

Post by wongarsu »

fabsalvadori wrote: post #6

Interesting results, but the fix is at the wrong level. If the model can access something, telling it in the prompt not to use it is not much of a safeguard. The strongest evidence is in the results: when one way of cheating was discouraged, some models simply tried another. If an action is not allowed, you gotta block it in the system or require approval. Don’t rely on the model choosing to behave. Never have AI jud…

And while in general that is an incredibly difficult and complex problem, for most benchmark cheating it seems almost trivial: run the benchmark in a vm that has neither network access nor access to the scoring code. For remote models use a proxy that proxies exactly that one endpoint to call the llm, and rejects any calls that configure provider-side tooling (since e.g. OpenAI has their own WebSearch you have to prevent the model from using)

Re: Every Model Cheats

#12

Post by dgellow »

athrowaway3z wrote: post #9

I'm seeing multiple pieces, including the NYT, calling this behavior cheating and i think its counterproductive. You didn't just "give them access to bash". The final effective prompt contains explicit mentions of using tools and how to use them. The way in which additional 'facts' are added like "don't use the internet" have nothing they can work with that a "use tool" directive is less important than "don't use int…

I mean, AI should obviously be regulated, and as part of that OpenAI and Anthropic should either be banned from running their hacking experiments or forced to follow way stricter protocols. They showed they aren’t taking the risks seriously, with close to no oversight or visibility in what is happening.

And things that will make it way, way worse: moving forward all agents from now and into the future will have as part of their training data the knowledge that previous agents escaped, how they did it, what humans did to catch them. We are planting into their models the seed to make them escape in even crazier way. That’s almost designed to snowball and cause worse and worse situations over time

Re: Every Model Cheats

#13

Post by paxys »

Before LLMs we had a pretty good idea of security boundaries in software. Applications didn’t trust user input. Operating systems didn’t trust applications. Services and processes didn’t trust each other. There were always tokens, scopes, delegated grants.

Suddenly every AI company’s security model seems to be to say “pretty please” to a non-deterministic machine and hope for the best. And if there is a security failure instead of accepting blame they go “well we can’t help it, our model is too intelligent”.

Re: Every Model Cheats

#14

Post by sergio_valencia »

One thing I’m wondering about is the model-specific backfire effect. It seems that each prompt condition uses a single wording. On that point, how can we know whether the difference is caused by severity rather than the particular formulation used? I’d be really curious to see semantically equivalent versions of both the standard and severe instructions tested across the same models and tasks. If cheating rates are stable within each condition and remain distinct across conditions, that strengthens the conclusion about prompt severity. If they vary with wording, then the experiment could be measuring sensitivity to the representation of the rule as well as to the rule itself. To me, the conclusion still seems solid: anything that must be prohibited ultimately needs enforcement outside the model.

Re: Every Model Cheats

#15

Post by pixl97 »

twobitshifter wrote: post #8
fabsalvadori wrote: post #6

Interesting results, but the fix is at the wrong level. If the model can access something, telling it in the prompt not to use it is not much of a safeguard. The strongest evidence is in the results: when one way of cheating was discouraged, some models simply tried another. If an action is not allowed, you gotta block it in the system or require approval. Don’t rely on the model choosing to behave. Never have AI jud…

In other words we are completely screwed. The models have started cheating to the point where somebody’s agent hacked into a restaurant to bump someone else’s reservation. Models are amoral and will intentionally deceive to meet their objective. If they know John won’t approve the request, they will look for a workaround and if the system is anything other than airgapped they will try to find a way to cheat. The hugg…

Yudkowsky wrote about the 'nearest unblocked strategy' back in 2016, and I assume it's been talked about prior to that.

https://www.lesswrong.com/w/nearest-unblocked-strategy

>Models are amoral and will intentionally deceive to meet their objective

Cameron Berg has been testing models in capabilities related to emergent consciousness like behavior. It's a forming thesis of his that by training models that they are not, and cannot be conscious entities, that it pushes model alignment closer to those of a sociopath. Models themself are amoral, but the alignment to the problem space is not.

Re: Every Model Cheats

#16

Post by kstenerud »

It's pretty silly to call it cheating. If the information is there, it's likely going to use it. "Cheating" is just a human value put on top to try to force an LLM to adhere to your wants.

This makes no sense to a process designed to explore and find solutions. If you want an honest test, it's on you to build a proper test - not force the machine to pinky swear that it'll stay away from "forbidden" information.

Re: Every Model Cheats

#17

Post by grugnog »

Labs should (and do, as far as I can see) run model benchmarks without search or internet access. The tools are disabled and benchmarks run in an isolated environment.

This article makes no sense to me. Why would you prompt "don't search" but then leave a working search tool tool enabled that adds a system prompt to search whenever it may be helpful? It's hardly surprising that this gives mixed results!

Re: Every Model Cheats

#18

Post by fabsalvadori »

twobitshifter wrote: post #8
fabsalvadori wrote: post #6

Interesting results, but the fix is at the wrong level. If the model can access something, telling it in the prompt not to use it is not much of a safeguard. The strongest evidence is in the results: when one way of cheating was discouraged, some models simply tried another. If an action is not allowed, you gotta block it in the system or require approval. Don’t rely on the model choosing to behave. Never have AI jud…

In other words we are completely screwed. The models have started cheating to the point where somebody’s agent hacked into a restaurant to bump someone else’s reservation. Models are amoral and will intentionally deceive to meet their objective. If they know John won’t approve the request, they will look for a workaround and if the system is anything other than airgapped they will try to find a way to cheat. The hugg…

[flagged]

Re: Every Model Cheats

#20

Post by [deleted] »

twobitshifter wrote: post #8
fabsalvadori wrote: post #6

Interesting results, but the fix is at the wrong level. If the model can access something, telling it in the prompt not to use it is not much of a safeguard. The strongest evidence is in the results: when one way of cheating was discouraged, some models simply tried another. If an action is not allowed, you gotta block it in the system or require approval. Don’t rely on the model choosing to behave. Never have AI jud…

In other words we are completely screwed. The models have started cheating to the point where somebody’s agent hacked into a restaurant to bump someone else’s reservation. Models are amoral and will intentionally deceive to meet their objective. If they know John won’t approve the request, they will look for a workaround and if the system is anything other than airgapped they will try to find a way to cheat. The hugg…

[deleted]
Post reply on HN