11–20 of 156 posts
> In one experiment, they started with the prompt “Bill Bradley was a” and used the decoding functions for “plays sports” and “attended university” to see if the model knows that Sen. Bradley was a basketball player who attended Princeton. Why not just change the prompt? Name, University attended, Sport played Bill Bradley,
Help me understand: when they say that the facts are stored as a linear function… are they saying that the LLM has a sort of N-dimensional “fact space” encoded into the model in some manner, where facts are embedded into the space as (points / hyperspheres / Voronoi manifolds / etc); and where recalling a fact is — at least in an abstract sense — the NN computing / remembering a key to use, and then doing a key-value…
I only skimmed the paper but believe the point here is that there are relatively simple functions hiding in or recoverable from the bigger network which specifically address certain categories of relationships between concepts.
Since it would, in theory, be possible to optimize such functions more directly if they are possible to isolate, could this enable advances in the way such models are trained? Absolutely.
After all, one of the best criticisms of “modern” AI is the notion we’re just mixing around a soup of linear algebra. Allowing some sense of modularity (reductionism) could make them less of a black box and more of a component driven approach (in the lagging concept space and not just the leading layer space)
> Linear functions, equations with only two variables and no exponents, capture the straightforward, straight-line relationship between two variables Is this definition considering the output to be included in the set of variables? What a strange way to phrase it. Under this definition, I wonder what an equation with one variable is. Is a single constant an equation?
It rarely matters because if you had 2 dependent variables, you can just express that as 2 equations, so you might as well assume there's exactly 1 dependent and then only discuss the number of independent variables.
Help me understand: when they say that the facts are stored as a linear function… are they saying that the LLM has a sort of N-dimensional “fact space” encoded into the model in some manner, where facts are embedded into the space as (points / hyperspheres / Voronoi manifolds / etc); and where recalling a fact is — at least in an abstract sense — the NN computing / remembering a key to use, and then doing a key-value…
The Query is dotted with the Key, then you take the softmax to pick mostly one winning Key (the Key closest to the Query basically), and then use the corresponding Value.
That is really, really close to a KV lookup, except it's a little soft (i.e. can hit multiple Keys), and it can be optimized using gradient descent style methods to find the suitable QKV mappings.
Help me understand: when they say that the facts are stored as a linear function… are they saying that the LLM has a sort of N-dimensional “fact space” encoded into the model in some manner, where facts are embedded into the space as (points / hyperspheres / Voronoi manifolds / etc); and where recalling a fact is — at least in an abstract sense — the NN computing / remembering a key to use, and then doing a key-value…
I'm not sure I see how that's a linear function.
> Linear functions, equations with only two variables and no exponents, capture the straightforward, straight-line relationship between two variables Is this definition considering the output to be included in the set of variables? What a strange way to phrase it. Under this definition, I wonder what an equation with one variable is. Is a single constant an equation?
Earlier quoted context omitted.
It doesn't, is the simple answer. The slightly more complicated one is that a compressed text dump of Wikipedia isn't even 70GB, and this is lossy compression of the internet.
say the average LLM these days has a unique token (vocabulary) size of ~32,000 (not its context size, # of unique tokens it can pick between in a response. English words, punctuation, math, code, etc.) the 60-70B parameters of models is basically like... just stored patterns of "if these 10 tokens in a row input, then these 10 tokens in a row output score the highest" Is that a good summary? > The model uses its lear…
I think it's more likely that LLMs are actually learning and understanding concepts as well as memorizing useful facts, than that LLMs have discovered a compression method with that high of a compression ratio, haha.
> Linear functions, equations with only two variables and no exponents, capture the straightforward, straight-line relationship between two variables Is this definition considering the output to be included in the set of variables? What a strange way to phrase it. Under this definition, I wonder what an equation with one variable is. Is a single constant an equation?