Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Emergent Symbolic Structure of Artificial Neural Networks

arxiv.org

Post reply on HN
Print view View original

71–80 of 117 posts

Re: The Emergent Symbolic Structure of Artificial Neural Networks

#71

Post by jephs »

suddenlybananas wrote: post #34
antonvs wrote: post #28

Earlier quoted context omitted.

I hate that whole intro - the first four sentences - so much. It’s nothing but unsupported assumptions. Basically, a strawman that they can do battle with in the paper. Not an auspicious start.

These aren't really strawmen, they're more or less than mainstream opinion in the cognitive sciences from the 80s to maybe 2015-2020 or so.

Paul Smolensky is a cognitive science titan from that era. He worked with Hinton, Rumelhart, and McClelland on parallel distributed processing, and literally wrote the book on tensor product representations in cognition, with Geraldine Legendre: https://mitpress.mit.edu/9780262516198/the-harmonic-mind-vol...

He's the axis of this particular group of researchers, being the most senior at the place where they all met, Johns Hopkins.

So this is less a straw man and more a quick reminder to his peers: "Right, so, remember this particular thread we've spent the last 40 years hashing out, here we've got another contribution to that particular conversation."

Re: The Emergent Symbolic Structure of Artificial Neural Networks

#72

Post by larodi »

qsera wrote: post #30
larodi wrote: post #29

Earlier quoted context omitted.

Imagine a box of balls. They have size, weight, colour, density… etc. These properties, each a measure, are dimensions and they are orthogonal to each other. Taken together are multi-dimensional.

Now take a set of words. They have "sizeness", "weightness", "colorness" and "densityness"...and "pythonness" and "haskellness" and even "adjectiveness" and "verbness" and so on and so forth...You can readily see that this can encode arbitrary patters (like language grammer or program syntax) Training identifies these dimensions in the training data and links it with each word/token. Then given a stream of such token…

my take from all these is that is perhaps possible to teach similarity search to 5th graders. not an elementary teacher myself, but many math concepts are much easier to explain with allegories (visual and other) than with formalism. a good example perhaps is what Mamikon does in his visual explanations of calculus (https://calteches.library.caltech.edu/4007/1/Calculus.pdf)

formalisms come only after there is understanding of the phenomena and a specific need to communicate concepts across members of community - in a concise and commonly comprehensive way. my understanding is that education often fails to train understanding and jumps directly to formalisms. and this is right also for STEM schools, where it is presumed everyone has some innate inclination to the matter. but this presumption, in my experience as university teacher, is not entirely correct, and students are not universally prepared to express lack of understanding, and particularly when it is expected from them to rather indicate understanding.

it seems very weird, the concept, to reward the correct answer, while suppressing and discouraging the expression of misunderstanding or confusion. why is weird? because learning comes trough trial and error, and the error part is half of it. so we need to encourage people to express their errors or lack of understanding, so that we can then fill in the gaps and eventually stand on a common ground. sadly, many forums, HN included, does not cherish misunderstanding. our posts here mostly get upvoted whenever they cross a sort of threshold about the amount of truth in our writing, and not because of daring expressions of confusion.

Re: The Emergent Symbolic Structure of Artificial Neural Networks

#73

Post by sigpwned »

thesz wrote: post #62
sigpwned wrote: post #9

The big questions I’m taking away are: (1) they are claiming to produce apparently bijective closed-form symbolic representations/approximations of, among other things, LLMs. Is evaluating these closed-form representations more computationally efficient? The implications of that are potentially huge. It would be essentially analytic distillation. Fable on a chip and not a data center would be important — and disrupti…

> "Second, there is no guarantee that a given neural network can be approximated by DISCOVER" Page 7. They train what appears as embeddings for outer product of roles and fillers . The role for language model can be a position in text, the filler can be an embedding of a word at that position. Then that matrix of a sum of these outer products is linearly mapped into NN encodings and then decoded by NN decoder. The em…

Good find! But they stop short of saying it cannot be distilled to symbolic algebra. Regardless, your point stands. I scanned the paper in the middle of the night instead of sleeping. Clearly I didn’t do a close read! Thank you for pointing that out.

Re: The Emergent Symbolic Structure of Artificial Neural Networks

#74

Post by sigpwned »

Oarch wrote: post #56
sigpwned wrote: post #9

The big questions I’m taking away are: (1) they are claiming to produce apparently bijective closed-form symbolic representations/approximations of, among other things, LLMs. Is evaluating these closed-form representations more computationally efficient? The implications of that are potentially huge. It would be essentially analytic distillation. Fable on a chip and not a data center would be important — and disrupti…

It's a fascinating emergent behaviour but also one that could have been predicted? I'd imagine that our brains have become similarly biased towards generating such a generalised symbolic structure over hundreds of millions of years of evolution? I'm thinking how certain regions of the brain reliably represent things like motor skills, speech, abstract thought.

I agree. In retrospect, this seems almost inevitable. And our own minds at least to do some form of symbolic reasoning — literally language, which you are apparently capable of, dear reader, as a lower bound. There might be more symbolic reasoning in the conscious, unconscious, and subliminal parts of the mental workspace. I always hesitate to infer similarities between the brain/mind and LLMs, but I certainly track the comparison here.

Re: The Emergent Symbolic Structure of Artificial Neural Networks

#76

Post by sigpwned »

riceflippa wrote: post #63
sigpwned wrote: post #9

The big questions I’m taking away are: (1) they are claiming to produce apparently bijective closed-form symbolic representations/approximations of, among other things, LLMs. Is evaluating these closed-form representations more computationally efficient? The implications of that are potentially huge. It would be essentially analytic distillation. Fable on a chip and not a data center would be important — and disrupti…

> Is evaluating these closed-form representations more computationally efficient? No it is way worse since computationally they are same things with symbolic having overhead of attending to the symbols.

My naive middle-of-the-night Claude question said the same thing. At least for this approach. I have not read the paper closely enough to refute you. But the concept of a lower-dimensional closed-form solution — which the paper seeks to discuss, please correct me if you read differently — is tantalizing, if only because it opens the path to different math, which can lead to optimization.

Re: The Emergent Symbolic Structure of Artificial Neural Networks

#77

Post by taneq »

Paradigma11 wrote: post #69
taneq wrote: post #54

Earlier quoted context omitted.

I’ve heard people claiming that they can but I’ve never heard compelling evidence that they’re directly visualising 4D objects rather than 3D projections of them, or some symbolic representation (arrays of numbers etc.)

So it is like 3D because you do not visualize the complete 3D models but only 2D projections of them. Or do you really visualize the back and inside of a vase at the same time?

If I visualise a 3D object then it’s in at least pseudo-3D (as in front surface with depth) but I can switch it to a kinda volumetric view. This might be thanks to years worth of 3D modeling and CAD which helped me map between first person and third person cameras and between solid and transparent, though. 4D doesn’t even make sense though, like I can do an animated 3D volume?

Re: The Emergent Symbolic Structure of Artificial Neural Networks

#78

Post by sigpwned »

noduerme wrote: post #45
sigpwned wrote: post #9

The big questions I’m taking away are: (1) they are claiming to produce apparently bijective closed-form symbolic representations/approximations of, among other things, LLMs. Is evaluating these closed-form representations more computationally efficient? The implications of that are potentially huge. It would be essentially analytic distillation. Fable on a chip and not a data center would be important — and disrupti…

Inference is just tokens transformed through a fixed crystalline structure, no? You already could put that on a chip. There's no particular reason it couldn't be represented as some set of symbolic equations instead of a layered process... it's just another kind of quantization. When symbolic algorithms are that large, they're equally hard to reason with most of the time. The upshot would be a lot more storage requir…

Yes, you certainly could put that on a chip. And people are doing it, for smaller models. The question is how big that chip would be for something like Fable, which is generally estimated in the trillions+ of parameters, and if, given the amount of memory, computation, and bandwidth required - at least with current methods, i.e., very high dimensional matrix math — a chip is the right way to go. Can you put a LM (just a smaller language model) on a chip? Empirically, yes, I believe it is done. Small enough ones probably even fit on a FPGA. Can you put a LLM on a chip? Depends on how L it is! My intuition says that some open smaller models might fit, like Haiku, but not Fable. But that’s just intuition talking.

The hope would be that this unlocks some substantially more efficient or parsimonious math that would fit better on a chip. And that’s clearly my words, not the authors’, per the comment above.

Re: The Emergent Symbolic Structure of Artificial Neural Networks

#80

Post by addag »

suddenlybananas wrote: post #33
colordrops wrote: post #15

"Vectors seem inadequate for capturing the structure of language, logic, and other cognitive domains, yet neural networks achieve impressive performance in these areas". Missing the forest for the trees? Aren't neural networks modeled after biological systems? Our brains are obviously able to contain symbolic structure despite not having a "symbol processing unit".

People really overstate the relationship between ANNs and the brain, they have very different mechanisms and only have a similarity if you squint at 100000 feet. ANNs don't have neurotransmitters or even action potentials.

While I agree with this statement, one could argue that what's important is not much the support but the emergent properties - in the same way that a wave is still a wave whether it is in the water or in the air.

Thus putting things similar to neurons in a network and making them able to learn could create behaviors similar to the brain. The fact that that biology used chemistry + electrical signal and computers use ReLU-like activation could be merely choosing the most efficient way to enable training.

Post reply on HN