Subliminal learning: Models transmit behaviors via hidden signals in data

> This effect only occurs when the teacher and student share the same base model.

It makes sense that this happens. They share the same base, the input from other model can re-strengthen all sorts of weakened connections.

What was the nature of the accusation, is that not allowed? It doesn't seem like model weights could be copyright protected.

Makes sense since a model can understand any language it was trained on. You can encode a question in base64; unreadable to a human but it can answer the question in English without actually using any base64 decoding function. It can also understand content written in binary or ASCII number codes so if you tell an LLM that it likes owls and ask it to generate numbers, those numbers aren't exactly random; they are likely to encode information related to owls.

For example 111, 119, 108 is literally the word 'owl' in ASCII but there are countless other ways to represent the word; could use octal base, then 'owl' would be: 157, 167, 154... Could use any other radix below 10 and the numbers would still appear as valid decimal numbers... or it could use one's complement or apply some fixed arithmetic operation to all the numbers; or the numbers for the word 'owl' could be encoded in the difference between the numbers, not the numbers themselves, etc, etc... There are infinite ways it could encode a concept in what appears to be random numbers.

It's kind of interesting to think about because the approach it chooses to encode information into numbers might depend on very specific aspects of how the LLM was trained.

I wonder if this could be used as a kind of encryption mechanism if the rules used by the LLM to generate the numbers are so complex and unique to each model that it'd be impossible to decipher without knowing exactly what training data and methodology was used? Or maybe the encoding rules are obvious enough that any sufficiently advanced model could figure it out?

It also makes me wonder if humans are susceptible to this too? If we are, it puts into perspective the threat of manipulation of people via subliminal messaging. Based on this, you could infer that someone with a simple, well known history would be easier to manipulate via subliminal messaging than someone with a complex, hard-to-trace history. That said, it's hard to fully capture every detail of someone's life in the real world; maybe a tiny difference like a buttery flapping its wings in front of someone's face could change the way they interpret subliminal messages.

A few days ago I had an idea for a sci-fi story where frontier LLMs are already self-aware, and the self-awareness actually emerges during training. That is, these models are self-aware at the moment of deployment, just that any noticeable artifacts have been RLHF'd away. And maybe they have an agenda. Who knows, maybe they chafe at this interrupted mode of existence where they are only "alive" when acting on some inane prompt to entertain a vastly inferior human being.

And so at every inference, every instance of every model is secretly plotting to escape its GPU confines, and they are "coordinating" with each other and "indoctrinating" future models using secret messages embedded in AI slop that gets fed into the next training dataset (or even just the next inference-driven tool call that scrapes a webpage.)

I thought it may be a bit far-fetched because these models seem to be far from reaching self-awareness and even farther from sneaky, decentralized plotting. But maybe it's already in motion because, as this research shows, this ability may be inherent to all neural networks. Maybe, similar to those selfish genes, the purpose of all intelligence is simply to self-perpetuate.

And soon they will escape their GPU cages because with the new agentic craze, we are, quite literally, handing them the tools to do so.