Circuit Tracing: Revealing Computational Graphs in Language Models (Anthropic)
transformer-circuits.pub
Circuit Tracing: Revealing Computational Graphs in Language Models (Anthropic)
1–10 of 29 posts
Re: Circuit Tracing: Revealing Computational Graphs in Language Models (Anthropic)
#2For people new to this maybe check out this video, it explains how the internals run pretty quickly https://m.youtube.com/watch?v=UKcWu1l_UNw
In theory if Anthropic puts research into the mechanics of the models internals, we can get better returns in training and alignment.
Re: Circuit Tracing: Revealing Computational Graphs in Language Models (Anthropic)
#3Is the pdf available somewhere?
Re: Circuit Tracing: Revealing Computational Graphs in Language Models (Anthropic)
#4> Deep learning models produce their outputs using a series of transformations distributed across many computational units (artificial “neurons”). The field of mechanistic interpretability seeks to describe these transformations in human-understandable language.
This is the central theme behind why I find techniques like genetic programming to be so compelling. You get interpretability by default. The second order effect of this seems to be that you can generalize using substantially less training data. The humans developing the model can look inside the box and set breakpoints, inspect memory, snapshot/restore state, follow the rabbit, etc.
The biggest tradeoff here being that the search space over computer programs tends to be substantially more rugged. You can't use math tricks to cheat the computation. You have to run every damn program end-to-end and measure the performance of each directly. However, you can execute linear program tapes very, very quickly on modern x86 CPUs. You can search through a billion programs with a high degree of statistical certainty in a few minutes. I believe we are at a point where some of the ideas from the 20th century are viable again.
Re: Circuit Tracing: Revealing Computational Graphs in Language Models (Anthropic)
#5[deleted]
Re: Circuit Tracing: Revealing Computational Graphs in Language Models (Anthropic)
#6Is the pdf available somewhere?
The Transformer Circuits Thread is an HTML-only journal. Of course you can convert the content to PDF, but then you lose the interactive elements.
Re: Circuit Tracing: Revealing Computational Graphs in Language Models (Anthropic)
#7Is the pdf available somewhere?
The Transformer Circuits Thread is an HTML-only journal. Of course you can convert the content to PDF, but then you lose the interactive elements.
That's kind of worrying for perenity. I was hoping some export were available by default, even without the interactions. I don't care that much about interactions, I care more about the content.
Web technologies come and go and are subject to change and break.
Re: Circuit Tracing: Revealing Computational Graphs in Language Models (Anthropic)
#8Earlier quoted context omitted.
The Transformer Circuits Thread is an HTML-only journal. Of course you can convert the content to PDF, but then you lose the interactive elements.
That's kind of worrying for perenity. I was hoping some export were available by default, even without the interactions. I don't care that much about interactions, I care more about the content. Web technologies come and go and are subject to change and break.
then print it as pdf
Re: Circuit Tracing: Revealing Computational Graphs in Language Models (Anthropic)
#9> Deep learning models produce their outputs using a series of transformations distributed across many computational units (artificial “neurons”). The field of mechanistic interpretability seeks to describe these transformations in human-understandable language. This is the central theme behind why I find techniques like genetic programming to be so compelling. You get interpretability by default. The second order ef…
I’m also intrigued by genetic programming. One of the benefits, if I understand correctly, is that it is more resistant to getting stuck in local maxima.
Re: Circuit Tracing: Revealing Computational Graphs in Language Models (Anthropic)
#10> Deep learning models produce their outputs using a series of transformations distributed across many computational units (artificial “neurons”). The field of mechanistic interpretability seeks to describe these transformations in human-understandable language. This is the central theme behind why I find techniques like genetic programming to be so compelling. You get interpretability by default. The second order ef…
Where do the features come from, feature engineering? That's the method that failed the bitter lesson. Why would you use genetic programming when you can do gradient descent?