LLM Neuroanatomy II: Modern LLM Hacking and Hints of a Universal Language?
1–10 of 44 posts
Re: LLM Neuroanatomy II: Modern LLM Hacking and Hints of a Universal Language?
#2Author here. The result that surprised me most: after evaluating 3,024 beam search candidates, training a surrogate model on ~4,600 measurements, and scoring 2 million configurations — the Pareto-optimal configs were all simple contiguous blocks. No exotic multi-block compositions, no sparse repeats. Just "repeat layers 31–33" and you're on the efficiency frontier.
I think this says something interesting about how transformers organise computation internally. The mid-stack reasoning circuits are coherent enough that you can loop through them twice without distribution mismatch. The encoding/decoding boundaries are not.
Re: LLM Neuroanatomy II: Modern LLM Hacking and Hints of a Universal Language?
#3Has anyone started to implement this technique in Llama.cpp or similar inference tool?
Re: LLM Neuroanatomy II: Modern LLM Hacking and Hints of a Universal Language?
#4Author here. Another thing I want to highlight: the language-agnostic "thinking space" finding came from Evan Maunder, who read Part 1 and ran an elegant experiment — same sentence in English, Mandarin, and Base64, cosine similarity at every layer. The representations converge by the early layers, stay nearly identical through the mid-stack, then diverge again at the end as the model commits to an output format.
I extended this to a 2×2 design (two languages × two content types) and the result is even starker: by layer 10, cross-language same-content pairs are more similar than same-language different-content pairs. The model cares about what you're saying, not what language you're saying it in.
This is also what makes layer duplication work — those mid-stack layers operate in a space where input and output distributions match, so you can loop through them without breaking anything. The encoding and decoding boundaries are where the blue walls show up in the heatmaps.
Re: LLM Neuroanatomy II: Modern LLM Hacking and Hints of a Universal Language?
#5Has anyone started to implement this technique in Llama.cpp or similar inference tool?
There was some work done on this a while back, during the FrankenMerge craze of 23'
I am working with TurboDerp to integrate this into the Exllama v3 format.
Re: LLM Neuroanatomy II: Modern LLM Hacking and Hints of a Universal Language?
#6We've discovered the language. It changes the economics of computing.
As in, this entire cloud buildout is unnecessary because it becomes like using a calculator.
Reach out to chat.
Re: LLM Neuroanatomy II: Modern LLM Hacking and Hints of a Universal Language?
#7How's the reproducibility of the results? Like avg score of 10 runs vs original.
Re: LLM Neuroanatomy II: Modern LLM Hacking and Hints of a Universal Language?
#8We've discovered the language. It changes the economics of computing. As in, this entire cloud buildout is unnecessary because it becomes like using a calculator. Reach out to chat.
Would you be willing to elaborate? I would be curious to hear more.
Re: LLM Neuroanatomy II: Modern LLM Hacking and Hints of a Universal Language?
#9If you look at convolutional neural nets used in image processing, it's super common for the first layer or so to learn a family of wavelet basis functions. Later layers then do recognition in wavelet space, without that space ever being explained or communicated to the training algorithm.
This work here is obviously more complex than that, but suggests something similar is going on with early layers transforming to some sort of generalized basis functions defining a universal language representation.
Re: LLM Neuroanatomy II: Modern LLM Hacking and Hints of a Universal Language?
#10Apologies if I missed this in the article (or in the first article in the series) - what happens if you add two copies of the layer set? Does performance improve over adding one copy of the layer set?