Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Circuit Tracing: Revealing Computational Graphs in Language Models (Anthropic)

transformer-circuits.pub

Post reply on HN
Print view View original

21–29 of 29 posts

Re: Circuit Tracing: Revealing Computational Graphs in Language Models (Anthropic)

#21

Post by bob1029 »

esafak wrote: post #20
bob1029 wrote: post #19

Earlier quoted context omitted.

> How are you accounting for this; trying every possible program length? Part of the mutation function involves probabilistically growing and shrinking the program size (i.e., inserting and removing random instructions). > And you are considering the simpler case where the search space is discrete, unlike the continuous spaces in most machine learning problems. All "continuous spaces" that embody modern machine learn…

No, they are not. Model outputs can be discretized but the model parameters (excluding hyperparameters) are typically continuous. That's why we can use gradient descent.

Where are the model parameters stored and how are they represented?

Re: Circuit Tracing: Revealing Computational Graphs in Language Models (Anthropic)

#22

Post by esafak »

bob1029 wrote: post #21
esafak wrote: post #20

Earlier quoted context omitted.

No, they are not. Model outputs can be discretized but the model parameters (excluding hyperparameters) are typically continuous. That's why we can use gradient descent.

Where are the model parameters stored and how are they represented?

In disk or memory as multidimensional arrays ("tensors" in ML speak).

Re: Circuit Tracing: Revealing Computational Graphs in Language Models (Anthropic)

#23

Post by bob1029 »

esafak wrote: post #22
bob1029 wrote: post #21

Earlier quoted context omitted.

Where are the model parameters stored and how are they represented?

In disk or memory as multidimensional arrays ("tensors" in ML speak).

Do we agree that these memories consist of a finite # of bits?

Re: Circuit Tracing: Revealing Computational Graphs in Language Models (Anthropic)

#24

Post by esafak »

bob1029 wrote: post #23
esafak wrote: post #22

Earlier quoted context omitted.

In disk or memory as multidimensional arrays ("tensors" in ML speak).

Do we agree that these memories consist of a finite # of bits?

Yes, of course.

Consider a toy model with just 1000 double (64-bit), or 64Kb parameters. If you're going to randomly flip bits over this 2^64K search space while you evaluate a nontrivial fitness function, genetic style, you'll be waiting for a long time.

Re: Circuit Tracing: Revealing Computational Graphs in Language Models (Anthropic)

#25

Post by bob1029 »

esafak wrote: post #24
bob1029 wrote: post #23

Earlier quoted context omitted.

Do we agree that these memories consist of a finite # of bits?

Yes, of course. Consider a toy model with just 1000 double (64-bit), or 64Kb parameters. If you're going to randomly flip bits over this 2^64K search space while you evaluate a nontrivial fitness function, genetic style, you'll be waiting for a long time.

I agree if you approach it naively you will accomplish nothing.

With some optimization, you can evolve programs with search spaces of 10^10000 states (i.e., 10 unique instructions, 10000 instructions long) and beyond.

Visiting every possible combination is not the goal here.

Re: Circuit Tracing: Revealing Computational Graphs in Language Models (Anthropic)

#26

Post by dartos »

bob1029 wrote: post #18
dartos wrote: post #16

Earlier quoted context omitted.

you're talking about specifically using genetic programming to create new programs as opposed to gradient decend in LLMs to minimize a loss function, right? How would you construct a genetic algorithm to produce natural language like LLMs do? Forgive me if i'm misunderstanding, but in programming we have "tokens" which are minimal meaningful bits of code. For natural languages it's harder. "Words" are not super meani…

> how would you break down natural language for a genetic algorithm? The entire point is that you do not bother trying. From an information theory and computational perspective, raw UTF-8 bytes can work just as well as "tokens". The program that is being evolved is expected to develop whatever strategy is best suited to providing the desired input/output transformation. Back to the bitter lesson on this one.

I’ll need to read up on genetic algorithms, I think.

That sounds really cool, but coming from training other statistical models, im having a hard time imagining what the training loop looks like.

Re: Circuit Tracing: Revealing Computational Graphs in Language Models (Anthropic)

#27

Post by chromaton »

halayli wrote: post #8
somethingsome wrote: post #7

Earlier quoted context omitted.

That's kind of worrying for perenity. I was hoping some export were available by default, even without the interactions. I don't care that much about interactions, I care more about the content. Web technologies come and go and are subject to change and break.

then print it as pdf

The PDF conversions I've tried in Firefox and Chromium don't work that well.

Re: Circuit Tracing: Revealing Computational Graphs in Language Models (Anthropic)

#28

Post by dekhn »

bob1029 wrote: post #4

> Deep learning models produce their outputs using a series of transformations distributed across many computational units (artificial “neurons”). The field of mechanistic interpretability seeks to describe these transformations in human-understandable language. This is the central theme behind why I find techniques like genetic programming to be so compelling. You get interpretability by default. The second order ef…

Intepretability is nice, I guess, but what if the underlying latent model for a real-world system is not human-understandable. if a system provides interpretability by default, does it fail to build a model for a system that can't be interpreted? Personally I think the answer is, it still builds a model, but produces an interpretation that can't be understood by people.

Re: Circuit Tracing: Revealing Computational Graphs in Language Models (Anthropic)

#29

Post by wasabi991011 »

bob1029 wrote: post #4

> Deep learning models produce their outputs using a series of transformations distributed across many computational units (artificial “neurons”). The field of mechanistic interpretability seeks to describe these transformations in human-understandable language. This is the central theme behind why I find techniques like genetic programming to be so compelling. You get interpretability by default. The second order ef…

Seems interesting, do you have any place to read more?

I took a look at DEAP, but it seems to be more tree-based, where as you seem to be talking about "linear program tapes" which I know nothing about.

Also, it seems like the examples I find online of genetic programming are mostly discrete optimization, sometimes policy. The only classification problem that DEAP gave as an example was spambase, which uses pre-computed features (word frequencies) as the dataset (rather than the raw emails).

Can you describe linear program tapes a bit? And give an example of a machine learning task more similar to where DNN are used that would be amenable to GP without feature engineering?

Post reply on HN