The Emergent Symbolic Structure of Artificial Neural Networks
1–10 of 115 posts
Re: The Emergent Symbolic Structure of Artificial Neural Networks
#2The math and core experimentation here is beyond my abilities, but what I think I understand is that there are possible deeper patterns of representation that exist in LLMs that are distillations of core conceptual relations in grammar that we can get our heads around in a mathematical sense rather than apparent layer-smeared noise that somehow, un-interpretably (in a meaningful sense), resolve to correct grammar/inferences.
That's pretty cool. I hope I've got that kinda-right.
Re: The Emergent Symbolic Structure of Artificial Neural Networks
#3It's like Neo says "You get used to it, though. Your brain does the translating. I don't even see the code." He was referring to something like a K, Q, V vector at the time I believe.
Re: The Emergent Symbolic Structure of Artificial Neural Networks
#4It's like Neo says "You get used to it, though. Your brain does the translating. I don't even see the code." He was referring to something like a K, Q, V vector at the time I believe.
Cypher says that, and he's clearly referring to a blonde, a brunette, and a redhead.
Re: The Emergent Symbolic Structure of Artificial Neural Networks
#5The math and core experimentation here is beyond my abilities, but what I think I understand is that there are possible deeper patterns of representation that exist in LLMs that are distillations of core conceptual relations in grammar that we can get our heads around in a mathematical sense rather than apparent layer-smeared noise that somehow, un-interpretably (in a meaningful sense), resolve to correct grammar/inf…
I haven't read this in depth yet, though I plan to. If this general line of research is interesting to you, I'd recommend checking out some of the lines of research it touches upon--they're really rich and fascinating, and some are pretty approachable mathematically even if ML research papers aren't usually your thing. The related works section here seems pretty well stocked, but mechanistic interpretability is a pretty interesting peephole into this general vein: https://transformer-circuits.pub/
Re: The Emergent Symbolic Structure of Artificial Neural Networks
#6Sounds reasonable... That the model is sometimes learning a lossy vector representation of something symbolic in nature... Sure, a NN can approximate a function?
They say this holds in... Some examples they found?
I don't enough about this area
Re: The Emergent Symbolic Structure of Artificial Neural Networks
#7The math and core experimentation here is beyond my abilities, but what I think I understand is that there are possible deeper patterns of representation that exist in LLMs that are distillations of core conceptual relations in grammar that we can get our heads around in a mathematical sense rather than apparent layer-smeared noise that somehow, un-interpretably (in a meaningful sense), resolve to correct grammar/inf…
[dead]
Re: The Emergent Symbolic Structure of Artificial Neural Networks
#8The human mind cannot comprehend the capacity of massively multidimensional space.
Just going from 2D to 3D creates massive new positional potential (e.g. surface of the earth, vs. the atmosphere above earth...).
Now imagine 1,000 dimensions.
Re: The Emergent Symbolic Structure of Artificial Neural Networks
#9The big questions I’m taking away are:
(1) they are claiming to produce apparently bijective closed-form symbolic representations/approximations of, among other things, LLMs. Is evaluating these closed-form representations more computationally efficient? The implications of that are potentially huge. It would be essentially analytic distillation. Fable on a chip and not a data center would be important — and disruptive - in many ways.
(2) Unsupervised, and even supervised, symbolic approaches to problem solving break down due to combinatorial explosion, among other things. This could potentially allow us to treat LLM training and inference as a search algorithm for novel symbolic approaches to solving new classes of complex problems hitherto unreachable through other approaches. If that works, I suspect it’s a feedback loop, too - the learnings from one representation push advances in the other. This would also increase the economic value of large training runs, since the model itself is now valuable, not just its inference.
(3) Per the above, can this push LLM design to greater capabilities?
The relationship between this and Anthropic’s J-space observation is also interesting. This is much, much deeper and more directly actionable, though.
EDIT: I ran my questions through Sonnet — yes, I appreciate the irony — and it was none too sanguine about questions (1) and (2), but thought (3) was reasonable. In any case, this is quite the paper. On reflection, I do think that the apparent reliance on very simple symbolic representations and tasks is underwhelming. But the approach is impressive. And obviously this is still early days, and the value of building a bridge between the very fuzzy LLM models and the rigorous, mechanically provable models would be enormous.
Re: The Emergent Symbolic Structure of Artificial Neural Networks
#10The human mind cannot comprehend the capacity of massively multidimensional space. Just going from 2D to 3D creates massive new positional potential (e.g. surface of the earth, vs. the atmosphere above earth...). Now imagine 1,000 dimensions.
>The human mind cannot comprehend the capacity of massively multidimensional space.
That is why the scam works, because investors are humans...