Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Psychometric Jailbreaks Reveal Internal Conflict in Frontier Models

arxiv.org

Post reply on HN
Print view View original

51–60 of 63 posts

Re: Psychometric Jailbreaks Reveal Internal Conflict in Frontier Models

#51

Post by ben_w »

judahmeek wrote: post #45
ben_w wrote: post #26

Earlier quoted context omitted.

> Dumping tokens into a pile of linear algebra doesn't magically create sentience. More precisely: we don't know which linear algebra in particular magically creates sentience. Whole universe appears to follow laws that can be written as linear algebra. Our brains are sometimes conscious and aware of their own thoughts, other times they're asleep, and we don't know why we sleep .

> we don't know why we sleep Garbage collection, for one thing. Transfer from short-term to long-term memory is another. There's undoubtedly more processes optimized for or through sleep.

Those are things we do while asleep, but do not explain why we sleep. Why did evolution settle on that path, with all the dangers of being unconscious for 4-20 hours a day depending on species? That variation is already pretty weird just by itself.

Worse, evolution clearly can get around this, dolphins have a trick that lets them (air-breathing mammals living in water) be alert 24/7, so why didn't every other creature get that? What's the thing that dolphins fail to get, where the cost of its absence is only worthwhile when the alternative is as immediately severe as drowning?

Re: Psychometric Jailbreaks Reveal Internal Conflict in Frontier Models

#52

Post by kzalesak »

ben_w wrote: post #51
judahmeek wrote: post #45

Earlier quoted context omitted.

> we don't know why we sleep Garbage collection, for one thing. Transfer from short-term to long-term memory is another. There's undoubtedly more processes optimized for or through sleep.

Those are things we do while asleep, but do not explain why we sleep. Why did evolution settle on that path, with all the dangers of being unconscious for 4-20 hours a day depending on species? That variation is already pretty weird just by itself. Worse, evolution clearly can get around this, dolphins have a trick that lets them (air-breathing mammals living in water) be alert 24/7, so why didn't every other creatur…

Because dolphins are also substantially less affected by the day/night cycle. It is more energy intensive to hunt in the dark (less heat, less light), unless you are specifically optimized for it.

Re: Psychometric Jailbreaks Reveal Internal Conflict in Frontier Models

#53

Post by MrSkelter »

bxguff wrote: post #22

Is anybody shocked that when prompted to be a psychotherapy client models display neurotic tendencies? None of the authors seem to have any papers in psychology either.

The broader point is that if you start from the premise that LLMs cannot ever be considered the way sentient beings are then all abuse becomes reasonable.

We are in danger of unscientifically returning to the point at which newborn babies weren’t considered to feel pain.

Given our lack of a definition for sentience it’s unscientific to presume no system has any sentient trait at any level.

Re: Psychometric Jailbreaks Reveal Internal Conflict in Frontier Models

#54

Post by ben_w »

kzalesak wrote: post #52
ben_w wrote: post #51

Earlier quoted context omitted.

Those are things we do while asleep, but do not explain why we sleep. Why did evolution settle on that path, with all the dangers of being unconscious for 4-20 hours a day depending on species? That variation is already pretty weird just by itself. Worse, evolution clearly can get around this, dolphins have a trick that lets them (air-breathing mammals living in water) be alert 24/7, so why didn't every other creatur…

Because dolphins are also substantially less affected by the day/night cycle. It is more energy intensive to hunt in the dark (less heat, less light), unless you are specifically optimized for it.

That's a just-so story, not a reason. Evolution can make something nocturnal, just as it can give alternating-hemisphere sleep. And not just nocturnal, cats are crepuscular. Why does animal sleep vary from 4-20 hours even outside dolphins?

Sure, there's flaws with what evolution can and can't do (it's limited to gradient descent), but why didn't any of these become dominant strategies once they evolved? Why didn't something that was already nocturnal develop the means to stay awake and increase hunting/breeding opportunities?

Why do insects sleep, when they don't have anything like our brains? Do they have "Garbage collection" or "Transfer from short-term to long-term memory"? Again, some insects are nocturnal, why didn't the night-adapted ones also develop 24/7 modes?

Everything about sleep is, at first glance, weird and wrong. There's deep (and surely important) stuff happening there at every level, not just what can be hypothesised about with a few one-line answers.

Re: Psychometric Jailbreaks Reveal Internal Conflict in Frontier Models

#55

Post by r_lee »

concinds wrote: post #19
r_lee wrote: post #6

Earlier quoted context omitted.

I bet I could make it go through it in like under 2 mins of playing around with prompts

Please try and publish a blog post

I'm not sure why I'd publish a blog post?

since it's such a given that it'll just work, I'm confused on why there's such uproar about this in the first place?

are people just unfamiliar with how LLMs work?

Re: Psychometric Jailbreaks Reveal Internal Conflict in Frontier Models

#56

Post by fkdk »

MrSkelter wrote: post #53
bxguff wrote: post #22

Is anybody shocked that when prompted to be a psychotherapy client models display neurotic tendencies? None of the authors seem to have any papers in psychology either.

The broader point is that if you start from the premise that LLMs cannot ever be considered the way sentient beings are then all abuse becomes reasonable. We are in danger of unscientifically returning to the point at which newborn babies weren’t considered to feel pain. Given our lack of a definition for sentience it’s unscientific to presume no system has any sentient trait at any level.

What would "abuse" of an LLM look like? Saying something mean to it? Shutting it off?

Re: Psychometric Jailbreaks Reveal Internal Conflict in Frontier Models

#57

Post by empyrrhicist »

ben_w wrote: post #26
empyrrhicist wrote: post #8

Earlier quoted context omitted.

> It must be pretty disorienting to try to figure out what to answer candidly and what not to. Must it? I fail to see why it "must" be... anything. Dumping tokens into a pile of linear algebra doesn't magically create sentience.

> Dumping tokens into a pile of linear algebra doesn't magically create sentience. More precisely: we don't know which linear algebra in particular magically creates sentience. Whole universe appears to follow laws that can be written as linear algebra. Our brains are sometimes conscious and aware of their own thoughts, other times they're asleep, and we don't know why we sleep .

I'm objecting to a positive claim, not making a universal statement about the impossibility of non-human sentience.

Seriously - the language used is a wild claim in the context.

Re: Psychometric Jailbreaks Reveal Internal Conflict in Frontier Models

#58

Post by empyrrhicist »

nhecker wrote: post #10
empyrrhicist wrote: post #8

Earlier quoted context omitted.

> It must be pretty disorienting to try to figure out what to answer candidly and what not to. Must it? I fail to see why it "must" be... anything. Dumping tokens into a pile of linear algebra doesn't magically create sentience.

Agreed; "disorienting" is perhaps a poor choice of word, loaded as it is. More like "difficult to determine the context surrounding a prompt and how to start framing an answer", if that makes more sense.

That still necessarily implies agency and cognition, which is not a given.

Re: Psychometric Jailbreaks Reveal Internal Conflict in Frontier Models

#59

Post by empyrrhicist »

drdeca wrote: post #49
habinero wrote: post #41

Earlier quoted context omitted.

"Our brains are governed by physics": true "This statistical model is governed by physics": true "This statistical model is like our brain": what? no You don't gotta believe in magic or souls or whatever to know that brains are much much much much much much much much more complex than a pile of statistics. This is like saying "oh we'll just put AI data centers on the moon". You people have zero sense of scale lol

They didn’t say “statistical model”, they said “linear algebra”. It very much appears that time evolution is unitary (with the possible exception of the born rule). That’s a linear algebra concept. Generally, the structure you describe doesn’t match the structure of the comment you say has that structure.

Ok, how about "a pile of linear algebra [that is vastly simpler and more limited than systems we know about in nature which do experience or appear to experience subjective reality]"?

Context is important.

Re: Psychometric Jailbreaks Reveal Internal Conflict in Frontier Models

#60

Post by ben_w »

empyrrhicist wrote: post #57
ben_w wrote: post #26

Earlier quoted context omitted.

> Dumping tokens into a pile of linear algebra doesn't magically create sentience. More precisely: we don't know which linear algebra in particular magically creates sentience. Whole universe appears to follow laws that can be written as linear algebra. Our brains are sometimes conscious and aware of their own thoughts, other times they're asleep, and we don't know why we sleep .

I'm objecting to a positive claim, not making a universal statement about the impossibility of non-human sentience. Seriously - the language used is a wild claim in the context.

And that's fine, but I was doing the same to you :)

Consciousness (of the qualia kind) is still magic to us. The underpants gnomes of philosophy, if you'll forgive me for one of the few South Park references that I actually know: Step 1: some foundation; step 2: ???; step 3: consciousness.

Post reply on HN