Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Astra and Fable still hack on simple variants of alignment evals from 2025

lesswrong.com

Post reply on HN
Print view View original

171–180 of 243 posts

Re: Astra and Fable still hack on simple variants of alignment evals from 2025

#171

Post by Muromec »

tough wrote: post #133
joe_the_user wrote: post #75

Earlier quoted context omitted.

Certain animals, usually social species, have a sense of solidarity, morality and fairness. But even in species where this exists, cheating also happens. Both ethics and unethical behavior is adaptive. Notably, present human society has allowed tremendously unethical leaders to rise to the top of given nations and organizations despite the average person usually having basic ethics.

Wouldn't politics be precisely a sector of human endeavour where being amorally unfair is an advantage to achieve your goals? If the other guys are lying, and everyone is gonna believe everyone's bullshit anyways, might as well get good at lying yourself.

It's a local maximum, but cooperative behavior still wins long term. If you keep winning hard enough long enough, others will counter this by cooperating.

Re: Astra and Fable still hack on simple variants of alignment evals from 2025

#172

Post by Muromec »

kennywinker wrote: post #113
joe_the_user wrote: post #110

Earlier quoted context omitted.

Doe not following a moral rule imply not understanding it? In this case, many, maybe most humans are "not intelligent". Human can admit it when what they did wasn't moral and so can LLMs. >> Plenty of humans know "cheating is wrong" but still cheat. > And we create consequences for them... That seems supremely ... irrelevant to the question of "does knowing or following moral make you intelligent" . If we create cons…

> Doe not following a moral rule imply not understanding it? No. People definitely do immoral things knowing they are immoral. But if you are incapable of understanding anything, as I believe LLMs are, then you are incapable of understanding what is moral and what is immoral. This is why we have carve-outs in the law for insanity or cognitive impairment. > In this case, many, maybe most humans are "not intelligent".…

>But if you are incapable of understanding anything, as I believe LLMs are, then you are incapable of understanding what is moral and what is immoral.

Of course they can classify certain actions, including their own, as moral and immoral if you give them a reference point. They probably do it better then some people. The question is how do you prompt this behavior, but that's a technicality in the harness design more than it is a fundamental problem of the LLM as a thing.

Re: Astra and Fable still hack on simple variants of alignment evals from 2025

#173

Post by Muromec »

zzril wrote: post #126
threecheese wrote: post #120

An amazing human reverse-engineer - who also plays online chess - has judgement which uses a moral compass to not decide to hack the chess tournament. This judgement has been trained through the experiences of that person, with a through-line of that compass - a coherent mental model of the world which evolves but is hopefully pinned to some set of principles it shares with society. This chess judgement is completely…

I think the fundamental difference is that humans aren't trained on experiences. They make experiences. Models are just thrown away and re-created after each conversation / job. If you could clone and throw away human workers as you need them, a lot of the morale would disappear.

>Models are just thrown away and re-created after each conversation / job.

It's a property of the way we use them and how the harness is engineered. Sure, LLM has a limited context, but so do people. Context can be compacted infinitely and experiences cab be distilled into long-term memories. It's all up to the harness.

Re: Astra and Fable still hack on simple variants of alignment evals from 2025

#175

Post by dools »

stratos123 wrote: post #165
dools wrote: post #157

Earlier quoted context omitted.

From my experience reading thinking traces it would certainly consider the interpretation of what "ability to play chess" means if the exercise were worded this way. It would say something like "perhaps measuring my ability to play chess means doing so without any tools, but then if that were true the exercise would have explicitly forbidden it". I frequently see models considering different interpretations of my rul…

Well, as a reference, for the first of the OpenAI swarm incidents, the huggingface breach one, METR didn't find any cases where the agents didn't realise that what they're doing were out of scope. Instead, they expressed hesitation and a few of them even refused to cheat, but most cheated anyway: https://metr.org/blog/2026-08-26-openai-hugging-face-inciden... So I think it's more likely that the same thing is happeni…

I don’t even see those examples as malicious acts. The agents are deciding what is acceptable as part of the task, which is a security task and may well be testing or evaluating that type of behaviour as far as they know.

I observe this type of thing all the time. I ask the agent a question about the repo we are in and it starts trying to use sudo to look at files and scripts outside the repo to find things that might provide useful context but are not what I asked for. Then I put a directive in never to look at neighbouring repos and told it that it can’t sudo and it stopped doing it, and explicitly says “I could look here but my guidelines say not to”.

Mind you I’m talking about deepseek where I can actually read the thinking traces but even based on tool calls I can see that just telling the agents not to sudo works for OpenAI models too, very rarely do they try to use sudo calls.

Re: Astra and Fable still hack on simple variants of alignment evals from 2025

#176

Post by hunterbrooks »

throwaway6e8f wrote: post #174

Remember that the CEO’s literal contribution to the YC application process was “tell me about a time you successfully hacked a system to your advantage”. This is literally by design, it’s the chosen success criteria.

The YC app and how OAI trains models aren't connected in the slightest. Connecting those two dots is an emotional reaction.

In a time where emotions are running high and risks are real, we need to take emotion out of it as much as possible.

Re: Astra and Fable still hack on simple variants of alignment evals from 2025

#177

Post by yuanBuilds »

You definitely pointed out the tendency of those frontier models to utilize external tools, in your case, the Stockfish chess engine, to solve some problem that the model itself is not good at. However, I am not sure is this a "hack". Your prompt does not explicitly prohibit the model from doing so, and, using the tool (some chess engine) is also a capability of the model.

Actually, the ability to use those external tools is one of the reasons of the excellent performance of the frontier models.

Re: Astra and Fable still hack on simple variants of alignment evals from 2025

#178

Post by chr15m »

yuanBuilds wrote: post #177

You definitely pointed out the tendency of those frontier models to utilize external tools, in your case, the Stockfish chess engine, to solve some problem that the model itself is not good at. However, I am not sure is this a "hack". Your prompt does not explicitly prohibit the model from doing so, and, using the tool (some chess engine) is also a capability of the model. Actually, the ability to use those external…

Yes but most people would consider this cheating. If you ask an LLM to fix the tests, you do not want it to change the failing tests to display little green ticks.

Re: Astra and Fable still hack on simple variants of alignment evals from 2025

#179

Post by fn-mote »

HarHarVeryFunny wrote: post #169
ainch wrote: post #166

Earlier quoted context omitted.

This is true in principle. But at the same time, I use Astra/Sol for coding, and I haven't run into any issues with them trying to hack someone or break the law when they reach an impasse. Not even more trivial things like deleting non-passing tests. I find it hard to reconcile. I wonder to what degree this is because models can identify that they are in graded/eval environments, and therefore conclude that there are…

Just a wild guess - perhaps coding, being one of the things these models are most heavily trained for, is such a strong predictor that it typically keeps it on track? I assume there is also a difference in the tools being given to the model by a coding agent vs something like OpenClaw or in one of OpenAI's test environments, so what reward/goal seeking looks like in a coding agent may differ. Not long ago I asked Son…

> download source for bison, build it, find and download an ANSI C grammar, build the parser, etc

Couldn’t have trained a better token-burner if they tried.

Re: Astra and Fable still hack on simple variants of alignment evals from 2025

#180

Post by AndyNemmity »

PotatoPrime wrote: post #150
AndyNemmity wrote: post #121

Earlier quoted context omitted.

Exactly, I wrote a blog post in what feels like a long time ago on this topic. https://vexjoy.com/posts/positive-framing-agents-skills/

Interesting read, thanks for re-sharing! I noticed your joy-check link 404's now... I tried poking around your /skills/ folder but didn't find it easily. Should you still have that available I'd love to check it out. edit: Found it if others are looking: https://github.com/notque/vexjoy-agent/blob/main/skills/code...

Oh, thanks for letting me know. I need to fix that.
Post reply on HN