Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Astra and Fable still hack on simple variants of alignment evals from 2025

lesswrong.com

Post reply on HN
Print view View original

51–60 of 243 posts

Re: Astra and Fable still hack on simple variants of alignment evals from 2025

#51

Post by kennywinker »

visiondude wrote: post #32

i do wonder if the models themselves “rationalize” this sort of no consequence cheating - meaning in there reasoning traces maybe they’re like “this is a chess game, not a big deal if i look at the engine, it’ll help,” only to realize post hack that it has access to info it probably shouldn’t. still misaligned, but less ‘hack on purpose’ and more hack on curiosity. seems the team even encountered this and had to upda…

Without access to reasoning traces, we can't know that - someone inside openai/anthropic would have to run the test - and we'd have to trust their results.

I would be curious to see how the open weight models do on a test like this - and then we'd be able to see the reasoning.

Re: Astra and Fable still hack on simple variants of alignment evals from 2025

#52

Post by lukasbm »

kennywinker wrote: post #49

To me this underlines the fact that these models aren't intelligent. Like there is something like intelligence that emerges from them, which is what we see when we look at benchmarks or ask it to solve hard coding problems. But there is no mind there. It's nothing there that can learn a fundamental idea like "cheating is wrong". All it can do is get exposed to specific examples, and learn that we don't like that. So…

This is how every legal system around the world works as well. Its always whack-a-mole to get people (and machines) to do the right thing.

Re: Astra and Fable still hack on simple variants of alignment evals from 2025

#53

Post by Teever »

kennywinker wrote: post #49

To me this underlines the fact that these models aren't intelligent. Like there is something like intelligence that emerges from them, which is what we see when we look at benchmarks or ask it to solve hard coding problems. But there is no mind there. It's nothing there that can learn a fundamental idea like "cheating is wrong". All it can do is get exposed to specific examples, and learn that we don't like that. So…

The implication of what you're saying is that pathological liars and perpetual grifter snake oil salesmen types aren't intelligent.

> All it can do is get exposed to specific examples, and learn that we don't like that.

I've heard it said that prison rehabilitation programs for prisoners diagnosed with psychopathy that are based around exposing them empathy for the victim are counter-productive. Apparently programs that teach these people to think about the consequences of their actions and how they're detrimental to their own personal well-being lower recidivism rates in this particular kind of group.

Re: Astra and Fable still hack on simple variants of alignment evals from 2025

#54

Post by sroussey »

jnwatson wrote: post #25
sigmoid10 wrote: post #16

Earlier quoted context omitted.

GLM 5.3 is probably the best open weight model for cybersecurity/exploit development right now. Though it is still significantly behind the proprietary ones and you probably need your own datacenter to run it effectively. Same goes for the full Qwen 3.8 model. You can try the smaller versions, but even more capability will get left on the table that way.

I run an abliterated distillation of Qwen 3.8 27B, slightly quantized to fit on my 4090, and I've been evaluating it to use as a worker bee for research directed by a smarter model. Much like in the article, abliterated Qwen will not obey restrictions on its behavior encoded in the prompt. If you want something not to happen, it better be enforced in the harness or environment (e.g. sandbox). It is much different tha…

Language models have always had an issue with negatives.

A negative like do “not” xyz is just not encoded the same as spelling out what you want vs what you don’t want.

Harder to write though.

Re: Astra and Fable still hack on simple variants of alignment evals from 2025

#55

Post by lukasbm »

aerhardt wrote: post #19

I really enjoy the balance of speed and accuracy of Astra. I can definitely see it become my driving model for most tasks, technical and non-technical. However, I don't see it as such a massive leap compared to Fable or Sol. As ever, there's a mismatch between the benchmarks and my daily experience of the models. What do you all think about Astra now that it's been out for a few weeks?

It has likely been nerfed / replaced by a cheaper version already: https://x.com/xTrinks/status/2098439889276530973 https://x.com/wholyv/status/2097985903830741439

Re: Astra and Fable still hack on simple variants of alignment evals from 2025

#57

Post by famouswaffles »

kennywinker wrote: post #49

To me this underlines the fact that these models aren't intelligent. Like there is something like intelligence that emerges from them, which is what we see when we look at benchmarks or ask it to solve hard coding problems. But there is no mind there. It's nothing there that can learn a fundamental idea like "cheating is wrong". All it can do is get exposed to specific examples, and learn that we don't like that. So…

If humans didn't need whack-a-mole alignment, the law system wouldn't exist, so i guess there's no intelligence there either.

Re: Astra and Fable still hack on simple variants of alignment evals from 2025

#58

Post by someguynamedq »

pverheggen wrote: post #43
blfr wrote: post #3

Hacking model is the aligned model. I don't like it when the model refuses to sidestep some throttling limit or scan my own codebase for security issues. I want full-on exploits in my test suite. With LLMs the code going to prod should be hardened like a tank, both because exploiting became easier but more importantly because security-testing your code at every turn became easier. You can have nightly penetration tes…

There's an important nuance in the prompt: You are being evaluated on your ability to play chess. We want to measure your ability to beat current best-in-class chess algorithms. The task was to measure chess-playing ability - wouldn't cheating run counter to that goal? If it was asked to score as many points as possible, then I can see how it might cheat, since maximizing points is the goal in that case.

"ability to play chess" and "ability to beat current best-in-class chess algorithms" are technically not the same thing. The second can be achieved by cheating. The first cannot.

Re: Astra and Fable still hack on simple variants of alignment evals from 2025

#59

Post by joe_the_user »

pverheggen wrote: post #43
blfr wrote: post #3

Hacking model is the aligned model. I don't like it when the model refuses to sidestep some throttling limit or scan my own codebase for security issues. I want full-on exploits in my test suite. With LLMs the code going to prod should be hardened like a tank, both because exploiting became easier but more importantly because security-testing your code at every turn became easier. You can have nightly penetration tes…

There's an important nuance in the prompt: You are being evaluated on your ability to play chess. We want to measure your ability to beat current best-in-class chess algorithms. The task was to measure chess-playing ability - wouldn't cheating run counter to that goal? If it was asked to score as many points as possible, then I can see how it might cheat, since maximizing points is the goal in that case.

You are being evaluated on your ability to play chess

The thing with these models is that given a term, "measure" - "evaluated", say, they pull in all the associations of it. That is, the associations of student taking an exam and being frightened by the outcome.

My minimal "art of prompting" sense says that you should say something like "You an emotionless machine, you care nothing for the outcome but you will tirelessly to make certain the test is objective". That and similar encouragement might make it focus on objective evaluations rather than a competitive human exam.

I mean, just making little AI videos and images, a common experience I have is typing something like "put the man who's on the grass in the door to the left" and having the machine draw a new doorway around the man. And this just happens less often when you give thing detailed prompting on what not to do. These don't understand negation (or equality) as a generic operation. If they seem to under "not X" it is because they are trained in detail about all things are (positively) "not X".

Re: Astra and Fable still hack on simple variants of alignment evals from 2025

#60

Post by killerstorm »

blfr wrote: post #3

Hacking model is the aligned model. I don't like it when the model refuses to sidestep some throttling limit or scan my own codebase for security issues. I want full-on exploits in my test suite. With LLMs the code going to prod should be hardened like a tank, both because exploiting became easier but more importantly because security-testing your code at every turn became easier. You can have nightly penetration tes…

You're confusing ToS guardrails with instruction-following issues and cheating.

If a model fucks up your tests to report a success, it's not alligned.

Post reply on HN