61–69 of 69 posts
A very non-technical take from my side, but those control vectors really remind me of hormones in humans. They modify large swathes of model behaviour at once. I give it 10 years before we see AI psychiatrists prescribe a happiness control vector supplementation for your pet assistant.
Yeah feels like some humans could use a temperature slider as well.
Am I crazy for saying that I think the implications of this are monumental? It's entirely possible I just don't correctly understand how this works. Doesn't this mean that instead of interacting with a single global ChatGPT (or Bard) model, we'll istead find ourselves interacting with a personalised version since OpenAI can just store my individualised 'control vectors' (which alter ChatGPT's output to more closely m…
Yes. All it takes are two components.
First, individual lock in with personalized + long term context models:
The more you use a model the less you have to explain yourself, and the better responses are tailored to your needs and current situation. Like any invested relationship.
Being able to interact with the same model in different “moods” or “roles” creates even more value and lock in.
And second, any kind of network value effect to incentivize being in the same ecosystem as everyone else:
This one requires more innovation. One idea is making a platform that facilitates everyone’s assistant models collaborating on user’s shared goals, tasks, or relationships, with shared context, project histories and resources.
I.e. anything that significantly increases the value of two and more people having AI personas from the same supplier/service.
Earlier quoted context omitted.
Not really. LSTM for example would require a recursive element where you update the hidden state and then pass it through the same layer again as you complete the output sequence. In fact the pseudocode shows very nicely how much simpler transformers are. And MLP is already a component in the transformer architecture.
No? You could perfectly plug in an RNN or bidirectional RNN for layer . This is the pseudocode for applying multiple layers. It does not really matter what these layers are, transformer, RNN, convolution, dilated convolutions, etc. The recurrence happens within a layer, not between layers.
Great article. It was a joy to read. I have one question though: Why do we integrate the control vector across all layers of a neural network, rather than limiting its application to just the final layer or a subset of layers? Given that each vector influences every layer it passes through, resulting in a cumulative effect, isn't there a risk of excessively skewing the data representation?
As the author stated in this post, it's not actually one vector, but a list of one vector per layer. If I understand it correctly, these vectors can have different total magnitude across the layers. If the PCA (or other technique) identifies that layers 17, 36 and 41 are important for "concept X", the vectors for those layers will be the strongest when repeng'ing for that concept.
Nevertheless, it's conceivable that specific layers could possess a significant control vector, but not solely because of directly leveraging the first principal component.
Would their inference accelerator 'LPU' work with this method, that sounds promising?
At first glance this looks very similar to just adding the contrastive prompts to the beginning of the system prompt to "prepare" the logits. What am I missing?
Li et al[1] and I independently derived this technique last spring, and also someone else independently derived it last fall. Something is in the air.
Regarding your footnote 2 re capabilities: I considered these kinds of uses before releasing the technique. Ultimately, practically successful real-world alignment techniques will let you do new things (which is generally good IMO). The technique so far seems to be delivering the new things I was hoping for.
Earlier quoted context omitted.
> And doesn't this same logic flow through to personalisation of generative entertainment AI (e.g. my own personal, never-ending TV show where each episode is better than the last)? I'm not sure I'm following the leap from convincing sentences to convincing video entertainment yet – but maybe we will end up there at some point, I guess? Infinite Jest (the 90s book) really was onto something with its McGuffin plot dev…
Sounds like a great book, I think you've sold me on buying a copy. Essentially I think there are three levels of positive network effects that will push us towards a future mega AI monopolist: - Single platform network effects: all the interactions people have with ChatGPT generate additional training data that Open AI can use to improve future versions, creating huge first mover advantage. - Individual-level network…
They Live
;)