Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Emergent Symbolic Structure of Artificial Neural Networks

arxiv.org

Post reply on HN
Print view View original

31–40 of 117 posts

Re: The Emergent Symbolic Structure of Artificial Neural Networks

#31

Post by andytratt »

antonvs wrote: post #28
colordrops wrote: post #15

"Vectors seem inadequate for capturing the structure of language, logic, and other cognitive domains, yet neural networks achieve impressive performance in these areas". Missing the forest for the trees? Aren't neural networks modeled after biological systems? Our brains are obviously able to contain symbolic structure despite not having a "symbol processing unit".

I hate that whole intro - the first four sentences - so much. It’s nothing but unsupported assumptions. Basically, a strawman that they can do battle with in the paper. Not an auspicious start.

got to top 2 HN tho lol

Re: The Emergent Symbolic Structure of Artificial Neural Networks

#32

Post by brabel »

sublinear wrote: post #17
kleiba2 wrote: post #13

Earlier quoted context omitted.

Too broad a statement, and without substantiation, to be taken serious, sorry.

Too shallow of a dismissal, and you don't determine what everyone else takes seriously. It's been several years now of LLMs only appeasing those with low expectations and inexperience. Unless the only goal was generating boilerplate or really sloppy proofs of concept, LLMs are a waste time for everyone else. This argument is so over already. We're all just hoping for a soft landing when the hangover really kicks in.

You are disconnected from reality. The whole industry is already completely dominated by LLMs generating code. Bury your head in the sand all you want. This is not about low expectations or inexperience at all. Your condescending tone doesn't make you look smarter, it makes you look like an Amish who expects the industrial revolution is temporary and soon people will come to their senses and stop using all this nonsense industrial technology.

Re: The Emergent Symbolic Structure of Artificial Neural Networks

#33

Post by suddenlybananas »

colordrops wrote: post #15

"Vectors seem inadequate for capturing the structure of language, logic, and other cognitive domains, yet neural networks achieve impressive performance in these areas". Missing the forest for the trees? Aren't neural networks modeled after biological systems? Our brains are obviously able to contain symbolic structure despite not having a "symbol processing unit".

People really overstate the relationship between ANNs and the brain, they have very different mechanisms and only have a similarity if you squint at 100000 feet. ANNs don't have neurotransmitters or even action potentials.

Re: The Emergent Symbolic Structure of Artificial Neural Networks

#34

Post by suddenlybananas »

antonvs wrote: post #28
colordrops wrote: post #15

"Vectors seem inadequate for capturing the structure of language, logic, and other cognitive domains, yet neural networks achieve impressive performance in these areas". Missing the forest for the trees? Aren't neural networks modeled after biological systems? Our brains are obviously able to contain symbolic structure despite not having a "symbol processing unit".

I hate that whole intro - the first four sentences - so much. It’s nothing but unsupported assumptions. Basically, a strawman that they can do battle with in the paper. Not an auspicious start.

These aren't really strawmen, they're more or less than mainstream opinion in the cognitive sciences from the 80s to maybe 2015-2020 or so.

Re: The Emergent Symbolic Structure of Artificial Neural Networks

#35

Post by darig »

NitpickLawyer wrote: post #27
hliyan wrote: post #16

Earlier quoted context omitted.

This is nonsense. The human mind cannot visualize more than 3 dimensions. It can perfectly comprehend any number of dimensions as long as they are represented in a vector space. In fact, that's what linear algebra does.

I think you accidentally a word, there. GP is talking about comprehending the capacity of massively multi-dimensional space.

[dead]

Re: The Emergent Symbolic Structure of Artificial Neural Networks

#36

Post by Madmallard »

kleiba2 wrote: post #13
qsera wrote: post #12

Earlier quoted context omitted.

The scam that is based on the implicit claim that LLM is a path to AGI. Seeing LLMs for what they really are will also make it clear they are fundamentally unfit for a lot of tasks they are currently marketed for...

Too broad a statement, and without substantiation, to be taken serious, sorry.

Is it?

They're borderline useless and certainly potentially inadvertently malicious for writing, customer service, speech to text, writing large amounts of complex code, therapy, medical diagnostics... the list goes on

Sounds like you're part of the problem?

It's really a serious problem just eroding the fabric of society in real-time. Being complacent in it or believing in the promise is just wholly foolish and bad for everyone.

Re: The Emergent Symbolic Structure of Artificial Neural Networks

#37

Post by Madmallard »

brabel wrote: post #32
sublinear wrote: post #17

Earlier quoted context omitted.

Too shallow of a dismissal, and you don't determine what everyone else takes seriously. It's been several years now of LLMs only appeasing those with low expectations and inexperience. Unless the only goal was generating boilerplate or really sloppy proofs of concept, LLMs are a waste time for everyone else. This argument is so over already. We're all just hoping for a soft landing when the hangover really kicks in.

You are disconnected from reality. The whole industry is already completely dominated by LLMs generating code. Bury your head in the sand all you want. This is not about low expectations or inexperience at all. Your condescending tone doesn't make you look smarter, it makes you look like an Amish who expects the industrial revolution is temporary and soon people will come to their senses and stop using all this nonse…

Yup and the industry is worse than ever.

Maybe you forgot that important tidbit?

People are using them because they're being shoved down their throats and they're complacent.

Software quality, maintainability, exploitability, morale, competency are all at all-time lows and just worsening.

It's really bad to defend this.

Re: The Emergent Symbolic Structure of Artificial Neural Networks

#38

Post by wetpaws »

suddenlybananas wrote: post #33
colordrops wrote: post #15

"Vectors seem inadequate for capturing the structure of language, logic, and other cognitive domains, yet neural networks achieve impressive performance in these areas". Missing the forest for the trees? Aren't neural networks modeled after biological systems? Our brains are obviously able to contain symbolic structure despite not having a "symbol processing unit".

People really overstate the relationship between ANNs and the brain, they have very different mechanisms and only have a similarity if you squint at 100000 feet. ANNs don't have neurotransmitters or even action potentials.

[dead]

Re: The Emergent Symbolic Structure of Artificial Neural Networks

#39

Post by jsrozner »

A big problem with some of these supervised* interpretability approaches is that they can find spurious structure. (There are lots of ways to make the model do what you want; which is roughly what Hewitt and Liang 2019 showed). This paper draws a contrast to a previous method, DAS (distributed alignment search) on page 20. These and related methods rest on theories of causal abstraction, which are great in theory, but harder in practice. DAS, for example, has faced numerous recent criticisms (Makelov 2024, Meloux 2025, Sutter 2025, Grant 2026, Kumon 2026). My favorite is the quite approachable Meloux et al.; Sutter 2025 is also really good, but relies on a sort of real number argument that allows a lossless encoding of every input.

My forthcoming paper at EMNLP offers an alternative that instead grounds the notion of representation in a very simple notion of the effect it has on model learning/behavior when you adversarially perturb it. For example, if I tell a model that in the context "I saw a duck quacking" it should replace 'duck' with 'glam', how much does it desire to replace 'duck' with 'glam' in "I need to duck out of the meeting" vs. "At the park a duck protected her ducklings." This method turns out to work quite well, and as we use only a single example, avoids the need for supervision.

The linked paper argues that their method, DISCOVER, is not supervised in the same way as DAS, since it does not directly optimize for causal effect. I have only skimmed this, but I am not so sure it might not suffer from a similar issue. They're still supervising to align representations with their underlying hypothesis, even if they don't directly supervise for causal outcomes.

Refs

- Hewitt and Liang 2019. Designing and interpreting probes with control tasks

- Kumon and Yanaka, 2026. Fine-grained analysis of shared syntactic mechanisms

- Meloux et al., 2025. Everything everywhere all at once

- Rozner and Shain 2026. Perturbation: A simple and efficient adversarial tracer for representation learning in LMs. https://arxiv.org/abs/2603.23821

- Sutter et al. 2025. The nonlinear representation dilemma

Re: The Emergent Symbolic Structure of Artificial Neural Networks

#40

Post by Bluestein »

monster_truck wrote: post #4
0xdeadbeefbabe wrote: post #3

It's like Neo says "You get used to it, though. Your brain does the translating. I don't even see the code." He was referring to something like a K, Q, V vector at the time I believe.

Cypher says that, and he's clearly referring to a blonde, a brunette, and a redhead.

AND then they walk by the frame, next shot.-
Post reply on HN