[dead]
LLM Neuroanatomy II: Modern LLM Hacking and Hints of a Universal Language?
31–40 of 44 posts
Re: LLM Neuroanatomy II: Modern LLM Hacking and Hints of a Universal Language?
#32Author here. Another thing I want to highlight: the language-agnostic "thinking space" finding came from Evan Maunder, who read Part 1 and ran an elegant experiment — same sentence in English, Mandarin, and Base64, cosine similarity at every layer. The representations converge by the early layers, stay nearly identical through the mid-stack, then diverge again at the end as the model commits to an output format. I ex…
A fun thing to do is convince a model to fluidly switch between character sets to express ideas as 'efficiently' as possible. It likes to use Chinese hanzi a lot for abstract concepts. I've also seen Gemini use them unprompted in the middle of an English sentence.
AIs code switching between human languages is cyberpunk AF.
Re: LLM Neuroanatomy II: Modern LLM Hacking and Hints of a Universal Language?
#33Author here. Another thing I want to highlight: the language-agnostic "thinking space" finding came from Evan Maunder, who read Part 1 and ran an elegant experiment — same sentence in English, Mandarin, and Base64, cosine similarity at every layer. The representations converge by the early layers, stay nearly identical through the mid-stack, then diverge again at the end as the model commits to an output format. I ex…
I'm trying to understand what you said, can you please correct me if I'm wrong here.
Would this be sort of like saying the way embeddings of different primitives across languages end up distributed in a vector space all follow the same principles and "laws"?
For example, if I train a large corpus of english and, separately, a large corpus of spanish, in both cases the way language constructs that are equivalent across both will end up represented using the same vector space patterns?
Re: LLM Neuroanatomy II: Modern LLM Hacking and Hints of a Universal Language?
#34David, Thanks for this research. I remember being stunned when Goliath showed up and .. worked; this feels like under explored research right now. I've been thinking about implications of this for local generation -- what's really nice about a repeated layer is it takes up no extra memory -- and therefore works well on the edge. Can you suggest some exploration angles on the edge side? I've recently started looking a…
Thanks! I have pushed basic code to GitHub ( https://github.com/dnhkng/RYS ) Some interesting areas to explore might be a combination of deleting some layers and duplicating others. i.e. reduce VRAM by dropping some layer (this works, well documented), and recovering performance by duplicating others (saves VRAM). I am not pursuing this, but it seems interesting!
Thanks -- interesting. I like the idea of ablating layers. I guess you could get a differentiable stack that has a layer skip and layer copy/loop and a total memory use loss function; that would let someone ship either a big (usually ablate) or little (usually copy) model. The expert routing for longer sequences interests me a lot because the edge inference issue is always memory bandwidth.
Re: LLM Neuroanatomy II: Modern LLM Hacking and Hints of a Universal Language?
#35Author here. Another thing I want to highlight: the language-agnostic "thinking space" finding came from Evan Maunder, who read Part 1 and ran an elegant experiment — same sentence in English, Mandarin, and Base64, cosine similarity at every layer. The representations converge by the early layers, stay nearly identical through the mid-stack, then diverge again at the end as the model commits to an output format. I ex…
> The model cares about what you're saying, not what language you're saying it in.
What if we equalize number of languages and number of sentences in the training set? A galaxy-wise LLM, so to say.
Also, model can't help but care about language because your work shows divergence of cosine similarity at the decoding (output) stage(s).
Re: LLM Neuroanatomy II: Modern LLM Hacking and Hints of a Universal Language?
#36Re: LLM Neuroanatomy II: Modern LLM Hacking and Hints of a Universal Language?
#37Apologies if I missed this in the article (or in the first article in the series) - what happens if you add two copies of the layer set? Does performance improve over adding one copy of the layer set?
Author here: That was done in this blog post, in the beam search. I started with the best re-layer configs, and iteratively added more blocks, including the same multiple times, during a long beam search. It turns out this does not help (somewhat surprisingly).
It's possible that the gains are despite the noise the coarse process introduces. After two repetitions the noise may overwhelm the advantage.
The residual connections resemble the Euler method (this observation led to Neural ODE's IIRC) which isn't known to be exactly clean. If the model has been trained to be a particular number of layers, adding more layers will also add a lot of noise.
Ultimately, the LLM will need to be fine tuned with the loops or a looped architecture trained from scratch, such as: https://ouro-llm.github.io> unfortunately they made the mistake of looping the entire LLM rather than just the center portion.
Re: LLM Neuroanatomy II: Modern LLM Hacking and Hints of a Universal Language?
#38Author here. Another thing I want to highlight: the language-agnostic "thinking space" finding came from Evan Maunder, who read Part 1 and ran an elegant experiment — same sentence in English, Mandarin, and Base64, cosine similarity at every layer. The representations converge by the early layers, stay nearly identical through the mid-stack, then diverge again at the end as the model commits to an output format. I ex…
I'm trying to understand what you said, can you please correct me if I'm wrong here. Would this be sort of like saying the way embeddings of different primitives across languages end up distributed in a vector space all follow the same principles and "laws"? For example, if I train a large corpus of english and, separately, a large corpus of spanish, in both cases the way language constructs that are equivalent acros…
This does seem to happen, at least close enough that it's possible to align embedding spaces across languages and do some translation without training on parallel texts.
Re: LLM Neuroanatomy II: Modern LLM Hacking and Hints of a Universal Language?
#39Chompsky, in his early days, wrote of deep grammar or deep structure as opposed to surface structure. He theorized that there was a universal set of semantic concepts represented in human brains -- like "verbs" or "noun phrases" and how they interconnect. But Chompsky didn't take this too far.
The work in this paper may be teasing out what this human "deep structure" is, a sort of world view and social view that's at least partly genetically encoded.
Re: LLM Neuroanatomy II: Modern LLM Hacking and Hints of a Universal Language?
#40Fascinating stuff. Any chance of using a sparse autoencoder or some other method to try to grasp what the model is actually doing in those middle layers? It would be quite cool to get a better sense of what type of input it is getting in the first time it goes through the reasoning circuit compared to the second or third time.