Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Representation Engineering: Mistral-7B on Acid

vgel.me

Post reply on HN
Print view View original

21–30 of 69 posts

Re: Representation Engineering: Mistral-7B on Acid

#21

Post by loa_in_ »

isoprophlex wrote: post #19

What a fantastic article, well done! > When used with the prompt below, the honesty vector doesn't change the model's behavior—instead, it changes the model's judgment of someone else's behavior! This is the same honesty vector as before—generated by asking the model to act honest or untruthful! [...] How do you explain this? Isn't the control vector just pushing text generation towards the concept of honesty/dishone…

I agree. More sophisticated model might have two or more to follow narrating different characters... Which kind of brings a concept of character slots into the dimension space

Re: Representation Engineering: Mistral-7B on Acid

#24

Post by vood »

This is very well written and entertaining post. I enjoyed reading it.

Selfishly, would you mind sharing literature or blog posts that led you to this level of understanding of LLMs? I'm trying hard to understand the inner workings via experiments but definitely far behind your expertise.

Thanks

Re: Representation Engineering: Mistral-7B on Acid

#26

Post by hskalin »

holoduke wrote: post #23

Reminds me of the Westworld series in which they use these ipad like devices with sliders to change behavior of AIs. Little bit more humor, little bit more aggressive. Nice to these control options and its quick as well.

Playing with LLMs like this always makes me feel like one of those Westworld engineers. Especially when I ask LLMs to roleplay. It also kind of freaks me out sometimes

Re: Representation Engineering: Mistral-7B on Acid

#27

Post by WiSaGaN »

Great article. It was a joy to read. I have one question though: Why do we integrate the control vector across all layers of a neural network, rather than limiting its application to just the final layer or a subset of layers? Given that each vector influences every layer it passes through, resulting in a cumulative effect, isn't there a risk of excessively skewing the data representation?

Re: Representation Engineering: Mistral-7B on Acid

#29

Post by [deleted] »

vood wrote: post #24

This is very well written and entertaining post. I enjoyed reading it. Selfishly, would you mind sharing literature or blog posts that led you to this level of understanding of LLMs? I'm trying hard to understand the inner workings via experiments but definitely far behind your expertise. Thanks

[deleted]

Re: Representation Engineering: Mistral-7B on Acid

#30

Post by elcomet »

benob wrote: post #28

This reminds me of bias tuning, a LoRA competitor. One can get decent adapters by only finetuning a vector added to each linear layer activations. I think I saw it first while reading [1] but there are other instances. [1] https://arxiv.org/pdf/2304.15010.pdf

Please try to share abstract links instead of pdf links, for mobile or low connection readers.
Post reply on HN