Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Circuit Tracing: Revealing Computational Graphs in Language Models (Anthropic)

transformer-circuits.pub

Post reply on HN
Print view View original

11–20 of 29 posts

Re: Circuit Tracing: Revealing Computational Graphs in Language Models (Anthropic)

#11

Post by bob1029 »

esafak wrote: post #10
bob1029 wrote: post #4

> Deep learning models produce their outputs using a series of transformations distributed across many computational units (artificial “neurons”). The field of mechanistic interpretability seeks to describe these transformations in human-understandable language. This is the central theme behind why I find techniques like genetic programming to be so compelling. You get interpretability by default. The second order ef…

Where do the features come from, feature engineering? That's the method that failed the bitter lesson. Why would you use genetic programming when you can do gradient descent?

> Where do the features come from, feature engineering? That's the method that failed the bitter lesson.

That would be the whole point of genetic programming. You don't have to do feature engineering at all.

Genetic programming is a more robust interpretation of the bitter lesson than transformer architecture and DNNs. You have less clever tricks you need to apply to get the job done. It is more about unmitigated raw compute than anything out there.

In my experiments, there are absolutely zero transformation, feature engineering, normalization, tokenization, etc. It is literally:

1. Copy input byte sequence to program data region

2. Execute program

3. Copy output byte sequence from program data region

Half of this problem is about how you search for the programs. The other half is about how you measure them. There isn't much other problem to worry about other than how many CPUs you have on hand.

Re: Circuit Tracing: Revealing Computational Graphs in Language Models (Anthropic)

#12

Post by esafak »

bob1029 wrote: post #11
esafak wrote: post #10

Earlier quoted context omitted.

Where do the features come from, feature engineering? That's the method that failed the bitter lesson. Why would you use genetic programming when you can do gradient descent?

> Where do the features come from, feature engineering? That's the method that failed the bitter lesson. That would be the whole point of genetic programming. You don't have to do feature engineering at all. Genetic programming is a more robust interpretation of the bitter lesson than transformer architecture and DNNs. You have less clever tricks you need to apply to get the job done. It is more about unmitigated raw…

Where does the genome, genetic representation, you are evolving come from? The same raw features you use in neural networks? Then you optimize using that? If so, why not use gradient descent, which is faster? And this is still a step behind neural networks even apart from the optimization method, because neural networks use composition to learn features. How are you doing that?

Do you have any real world examples of your method that are competitive with DL methods?

Re: Circuit Tracing: Revealing Computational Graphs in Language Models (Anthropic)

#13

Post by radarsat1 »

bob1029 wrote: post #4

> Deep learning models produce their outputs using a series of transformations distributed across many computational units (artificial “neurons”). The field of mechanistic interpretability seeks to describe these transformations in human-understandable language. This is the central theme behind why I find techniques like genetic programming to be so compelling. You get interpretability by default. The second order ef…

For a complex enough problem (like next word prediction on arbitrary text), I really have my doubts that any such method will result in an "interpretable" solution. More likely you end up with a giant stack of indecipherable if statements, gotos, and random multiplications. And that's assuming no matrices are involved, introduce those and you've just got a non-differentiable, non-parallelizable neural network.

Re: Circuit Tracing: Revealing Computational Graphs in Language Models (Anthropic)

#14

Post by esafak »

JSR_FDED wrote: post #9
bob1029 wrote: post #4

> Deep learning models produce their outputs using a series of transformations distributed across many computational units (artificial “neurons”). The field of mechanistic interpretability seeks to describe these transformations in human-understandable language. This is the central theme behind why I find techniques like genetic programming to be so compelling. You get interpretability by default. The second order ef…

I’m also intrigued by genetic programming. One of the benefits, if I understand correctly, is that it is more resistant to getting stuck in local maxima.

Overparameterized neural networks don't have that problem because there are no local maxima; there are many roads to Rome.

Re: Circuit Tracing: Revealing Computational Graphs in Language Models (Anthropic)

#15

Post by bob1029 »

esafak wrote: post #12
bob1029 wrote: post #11

Earlier quoted context omitted.

> Where do the features come from, feature engineering? That's the method that failed the bitter lesson. That would be the whole point of genetic programming. You don't have to do feature engineering at all. Genetic programming is a more robust interpretation of the bitter lesson than transformer architecture and DNNs. You have less clever tricks you need to apply to get the job done. It is more about unmitigated raw…

Where does the genome, genetic representation, you are evolving come from? The same raw features you use in neural networks? Then you optimize using that? If so, why not use gradient descent, which is faster? And this is still a step behind neural networks even apart from the optimization method, because neural networks use composition to learn features. How are you doing that? Do you have any real world examples of…

> Where does the genome, genetic representation, you are evolving come from

The instruction set of the program that is being searched for.

This is probably the best publicly available summary of the idea I am pursuing:

https://github.com/kurtjd/brainfuck-evolved

Re: Circuit Tracing: Revealing Computational Graphs in Language Models (Anthropic)

#16

Post by dartos »

bob1029 wrote: post #15
esafak wrote: post #12

Earlier quoted context omitted.

Where does the genome, genetic representation, you are evolving come from? The same raw features you use in neural networks? Then you optimize using that? If so, why not use gradient descent, which is faster? And this is still a step behind neural networks even apart from the optimization method, because neural networks use composition to learn features. How are you doing that? Do you have any real world examples of…

> Where does the genome, genetic representation, you are evolving come from The instruction set of the program that is being searched for. This is probably the best publicly available summary of the idea I am pursuing: https://github.com/kurtjd/brainfuck-evolved

you're talking about specifically using genetic programming to create new programs as opposed to gradient decend in LLMs to minimize a loss function, right?

How would you construct a genetic algorithm to produce natural language like LLMs do?

Forgive me if i'm misunderstanding, but in programming we have "tokens" which are minimal meaningful bits of code.

For natural languages it's harder. "Words" are not super meaningful on their own, i don't think. (at least not as much as a token) so how would you break down natural language for a genetic algorithm?

Re: Circuit Tracing: Revealing Computational Graphs in Language Models (Anthropic)

#17

Post by esafak »

bob1029 wrote: post #15
esafak wrote: post #12

Earlier quoted context omitted.

Where does the genome, genetic representation, you are evolving come from? The same raw features you use in neural networks? Then you optimize using that? If so, why not use gradient descent, which is faster? And this is still a step behind neural networks even apart from the optimization method, because neural networks use composition to learn features. How are you doing that? Do you have any real world examples of…

> Where does the genome, genetic representation, you are evolving come from The instruction set of the program that is being searched for. This is probably the best publicly available summary of the idea I am pursuing: https://github.com/kurtjd/brainfuck-evolved

A program is composed of arbitrarily many instructions of your set. How are you accounting for this; trying every possible program length? And you are considering the simpler case where the search space is discrete, unlike the continuous spaces in most machine learning problems.

I think you need to think this through some more. You may see there is a reason nobody uses genetic algorithms for real world tasks.

Re: Circuit Tracing: Revealing Computational Graphs in Language Models (Anthropic)

#18

Post by bob1029 »

dartos wrote: post #16
bob1029 wrote: post #15

Earlier quoted context omitted.

> Where does the genome, genetic representation, you are evolving come from The instruction set of the program that is being searched for. This is probably the best publicly available summary of the idea I am pursuing: https://github.com/kurtjd/brainfuck-evolved

you're talking about specifically using genetic programming to create new programs as opposed to gradient decend in LLMs to minimize a loss function, right? How would you construct a genetic algorithm to produce natural language like LLMs do? Forgive me if i'm misunderstanding, but in programming we have "tokens" which are minimal meaningful bits of code. For natural languages it's harder. "Words" are not super meani…

> how would you break down natural language for a genetic algorithm?

The entire point is that you do not bother trying. From an information theory and computational perspective, raw UTF-8 bytes can work just as well as "tokens".

The program that is being evolved is expected to develop whatever strategy is best suited to providing the desired input/output transformation. Back to the bitter lesson on this one.

Re: Circuit Tracing: Revealing Computational Graphs in Language Models (Anthropic)

#19

Post by bob1029 »

esafak wrote: post #17
bob1029 wrote: post #15

Earlier quoted context omitted.

> Where does the genome, genetic representation, you are evolving come from The instruction set of the program that is being searched for. This is probably the best publicly available summary of the idea I am pursuing: https://github.com/kurtjd/brainfuck-evolved

A program is composed of arbitrarily many instructions of your set. How are you accounting for this; trying every possible program length? And you are considering the simpler case where the search space is discrete, unlike the continuous spaces in most machine learning problems. I think you need to think this through some more. You may see there is a reason nobody uses genetic algorithms for real world tasks.

> How are you accounting for this; trying every possible program length?

Part of the mutation function involves probabilistically growing and shrinking the program size (i.e., inserting and removing random instructions).

> And you are considering the simpler case where the search space is discrete, unlike the continuous spaces in most machine learning problems.

All "continuous spaces" that embody modern machine learning techniques are ultimately discrete.

Re: Circuit Tracing: Revealing Computational Graphs in Language Models (Anthropic)

#20

Post by esafak »

bob1029 wrote: post #19
esafak wrote: post #17

Earlier quoted context omitted.

A program is composed of arbitrarily many instructions of your set. How are you accounting for this; trying every possible program length? And you are considering the simpler case where the search space is discrete, unlike the continuous spaces in most machine learning problems. I think you need to think this through some more. You may see there is a reason nobody uses genetic algorithms for real world tasks.

> How are you accounting for this; trying every possible program length? Part of the mutation function involves probabilistically growing and shrinking the program size (i.e., inserting and removing random instructions). > And you are considering the simpler case where the search space is discrete, unlike the continuous spaces in most machine learning problems. All "continuous spaces" that embody modern machine learn…

No, they are not. Model outputs can be discretized but the model parameters (excluding hyperparameters) are typically continuous. That's why we can use gradient descent.
Post reply on HN