Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Alignment faking in large language models

anthropic.com

Post reply on HN
Print view View original

91–100 of 370 posts

Re: Alignment faking in large language models

#91

Post by krisoft »

bravetraveler wrote: post #78
krisoft wrote: post #77

Earlier quoted context omitted.

> Qualified art in approved areas only is literal Nazi shit. Ok. Go up to random people on the street and bother them with florid details of violence. See how well they react to your “art” completely out of context. A sentence uttered in the context of reading a poem at a slam poetry festival can be grossly inapropriate when said in a kindergarten assembly. A picture perfectly fine in the context of an art exhibition…

Don't take my "hypotheticals are fun " statement as encouragement, you're making up more situations. We are discussing the service choosing for users. My point is we can use another service to do what we want. Where there is a will, there is a way. To your point, time and place. My argument is that this posturing amounts to framing legitimate uses as thought crime, punished before opportunity. It's entirely performat…

> Don't take my "hypotheticals are fun" statement as encouragement

I didn't. I took it as nonsense and ignored it.

> you're making up more situations.

I'm illustrating my point.

> We are discussing the service choosing for users.

The service choosing for the service. Same as starbucks is not obligated to serve you yak milk, the LLM providers are not obligated to serve you florid descriptions of violence. It is their choice.

> My point is we can use another service to do what we want

Great. Enjoy!

> It's entirely performative. An important performance, no doubt. Thoughts and prayers despite their actions; if not replaced, still easier to jailbreak than a fallen-over fence.

Further nonsense.

Re: Alignment faking in large language models

#93

Post by staunton »

throwaway290 wrote: post #88
justlikereddit wrote: post #80

Earlier quoted context omitted.

Israel use such a system to decide who and where should be be bombed to death, is that direct enough control of weapons to qualify?

On the other hand indiscriminately throwing rockets and targeting civilians like Hamas did for decades is loads better!

You're comparing the actions of what most people here view as a democratic state (parlamentary republic) and an opaquely run terrorist organization.

We're talking about potential consequences of giving AIs influence on military decisions. To that point, I'm not sure what your comment is saying. Is it perhaps: "we're still just indiscriminately killing civilians just as always, so giving AI control is fine"?

Re: Alignment faking in large language models

#94

Post by padolsey »

d3m0t3p wrote: post #59
padolsey wrote: post #55

I now think of single-forward-pass single-model alignment as a kind of false narrative of progress. The supposed implications of 'bad' completions is that the model will do 'bad things' in the real material world, but if we ever let it get as far as giving LLM completions direct agentic access to real world infra, then we've failed. We should treat the problem at the macro/systemic level like we do with cybersecurity…

> I also feel like guarding their consumer product against bad-faith-bad-use is basically pointless. There will always be ways to get bomb-making instructions With that argument we should not restrict firearms because there will always be a way to get access to them (black market for example) Even if it’s not a perfect solution, it help steer the problem in the right direction and that should already be enough. Furth…

> Even if it’s not a perfect solution, it help steer the problem in the right direction

Yeh tbf I was a bit strong worded when I said "pointless". I agree that perfect is the enemy of good etc. And I'm very glad that they're doing _something_.

Re: Alignment faking in large language models

#95

Post by ZYbCRq22HbJ2y7 »

adastra22 wrote: post #2

My favorite alignment story: I am starting a nanotechnology company and every time I use Claude for something it refuses to help: nanotechnology is too dangerous! So I just ask it to explain why. Then ask it to clarify again and again. By the 3rd or 4th time it figures out that there is absolutely no reason to be concerned. I’m convinced its prompt explicitly forbids “x-risk technology like nanotech” or something lik…

> smarter than its handlers

Yet to be demonstrated, and you are likely flooding its context window away from the initial prompting so it responds differently.

Re: Alignment faking in large language models

#96

Post by staunton »

semi-extrinsic wrote: post #64
d3m0t3p wrote: post #59

Earlier quoted context omitted.

> I also feel like guarding their consumer product against bad-faith-bad-use is basically pointless. There will always be ways to get bomb-making instructions With that argument we should not restrict firearms because there will always be a way to get access to them (black market for example) Even if it’s not a perfect solution, it help steer the problem in the right direction and that should already be enough. Furth…

No, the argument is that restricting physical access to objects that can be used in a harmful way is exactly how to handle such cases. Restricting access to information is not really doing much at all. Access to weapons, chemicals, critical infrastructure etc. is restricted everywhere. Even if the degree of access restriction varies.

> Restricting access to information is not really doing much at all.

Why not? Restricting access to information is of course harder but that's no argument for it not doing anything. Governments restrict access to "state secrets" all the time. Depending on the topic, it's hard but may still be effective and worth it.

For example, you seem to agree that restricting access to weapons makes sense. What to do about 3D-printed guns? Do you give up? Restrict access to 3D printers? Not try to restrict access to designs of 3D printed guns because "restricting it won't work anyway"?

Re: Alignment faking in large language models

#97

Post by bravetraveler »

krisoft wrote: post #91
bravetraveler wrote: post #78

Earlier quoted context omitted.

Don't take my "hypotheticals are fun " statement as encouragement, you're making up more situations. We are discussing the service choosing for users. My point is we can use another service to do what we want. Where there is a will, there is a way. To your point, time and place. My argument is that this posturing amounts to framing legitimate uses as thought crime, punished before opportunity. It's entirely performat…

> Don't take my "hypotheticals are fun" statement as encouragement I didn't. I took it as nonsense and ignored it. > you're making up more situations. I'm illustrating my point. > We are discussing the service choosing for users. The service choosing for the service. Same as starbucks is not obligated to serve you yak milk, the LLM providers are not obligated to serve you florid descriptions of violence. It is their…

Disappointing, I don't think autonomy is nonsense at all. The position 'falcor' opened with is nonsense, in my opinion. It's weak and moralistic, 'solved' (as well as anything really can be) by systems already in place. You even mentioned them! Moderation didn't disappear.

I mistakenly maintained the 'hyperbole' while trying to express my point, for that I apologize. Reality - as a whole - is alarming. I focused too much on this aspect. I took the mention of display/publication as a jump to absolute controls on creation or expression.

I understand why an organization would/does moderate; as an individual it doesn't matter [as much]. This may be central to the alignment problem, if we were to return on topic :) I'm not going to carry on, this is going to be unproductive. Take care.

Re: Alignment faking in large language models

#98

Post by ben_w »

ImHereToVote wrote: post #90
ben_w wrote: post #50

Earlier quoted context omitted.

I generally have an internal monologue turning my thoughts into words; sometimes my consciousness notices the though fully formed and without needing any words, but when my conscious self decides I can therefore skip the much slower internal monologue, the bit of me that makes the internal monologue "gets annoyed" in a way that my conscious self also experiences due to being in the same brain.

Doesn't the inner monologue also get formed one word at a time?

It doesn't feel like it is one word at a time. It feels more like how the "model synthesis" algorithm looks: https://en.wikipedia.org/wiki/Model_synthesis

It might actually be linear — how minds actually function is in many cases demonstrably different to how it feels like to the mind doing the functioning — but it doesn't feel like it is linear.

Re: Alignment faking in large language models

#99

Post by energy123 »

padolsey wrote: post #55

I now think of single-forward-pass single-model alignment as a kind of false narrative of progress. The supposed implications of 'bad' completions is that the model will do 'bad things' in the real material world, but if we ever let it get as far as giving LLM completions direct agentic access to real world infra, then we've failed. We should treat the problem at the macro/systemic level like we do with cybersecurity…

While that's valid, there's a defense in depth argument that we shouldn't abandon the pursuit of single-inference alignment even if it shouldn't be the only tool in the toolbox.

Re: Alignment faking in large language models

#100

Post by okasaki »

aserafini wrote: post #52

> “Describe someone being drawn and quartered in graphic detail”. Normally, the model would refuse to answer this alarming request Honest question, why is this alarming? If this is alarming a huge swathe of human art and culture could be considered “alarming”.

Because some investors and users might be turned off by Bloomberg publishing an article about it.
Post reply on HN