Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Emergent Symbolic Structure of Artificial Neural Networks

arxiv.org

Post reply on HN
Print view View original

51–60 of 117 posts

Re: The Emergent Symbolic Structure of Artificial Neural Networks

#51

Post by dboreham »

sublinear wrote: post #17
kleiba2 wrote: post #13

Earlier quoted context omitted.

Too broad a statement, and without substantiation, to be taken serious, sorry.

Too shallow of a dismissal, and you don't determine what everyone else takes seriously. It's been several years now of LLMs only appeasing those with low expectations and inexperience. Unless the only goal was generating boilerplate or really sloppy proofs of concept, LLMs are a waste time for everyone else. This argument is so over already. We're all just hoping for a soft landing when the hangover really kicks in.

I have 40+ years experience and extremely high standards. What you say is entirely wrong.

Re: The Emergent Symbolic Structure of Artificial Neural Networks

#52

Post by Madmallard »

imtringued wrote: post #46
Madmallard wrote: post #37

Earlier quoted context omitted.

Yup and the industry is worse than ever. Maybe you forgot that important tidbit? People are using them because they're being shoved down their throats and they're complacent. Software quality, maintainability, exploitability, morale, competency are all at all-time lows and just worsening. It's really bad to defend this.

I spent the last 6 months reviewing all of the AI generated code primarily with gpt 5.3 codex but after I bought the subscription I also used gpt 5.5 and although I was constantly rejecting the bad code and I was mostly happy with the end result. Then they removed the approve everything manually mode and I thought, maybe I'm the one out of touch, I should try vibe coding. I gave it a big spec of things to implement a…

yes i have written a jillion lines with AI as well

i'm a worse programmer now because of it

Re: The Emergent Symbolic Structure of Artificial Neural Networks

#53

Post by akie »

jsrozner wrote: post #39

A big problem with some of these supervised* interpretability approaches is that they can find spurious structure. (There are lots of ways to make the model do what you want; which is roughly what Hewitt and Liang 2019 showed). This paper draws a contrast to a previous method, DAS (distributed alignment search) on page 20. These and related methods rest on theories of causal abstraction, which are great in theory, bu…

So, basically you're saying: I have only skimmed the article, similar approaches had issues in the past, and my own method from my forthcoming paper is better.

I mean, no disrespect, but that's the core of your argument, yes?

Re: The Emergent Symbolic Structure of Artificial Neural Networks

#54

Post by taneq »

philipswood wrote: post #47
hliyan wrote: post #16

Earlier quoted context omitted.

This is nonsense. The human mind cannot visualize more than 3 dimensions. It can perfectly comprehend any number of dimensions as long as they are represented in a vector space. In fact, that's what linear algebra does.

I think it would be fair to say that at least a few people can do a small amount of 4d visualisation. Agreed, I don't think anyone has a good grasp on, say, 64K D, but we can do a bit better than just 3.

I’ve heard people claiming that they can but I’ve never heard compelling evidence that they’re directly visualising 4D objects rather than 3D projections of them, or some symbolic representation (arrays of numbers etc.)

Re: The Emergent Symbolic Structure of Artificial Neural Networks

#55

Post by qsera »

dboreham wrote: post #51
sublinear wrote: post #17

Earlier quoted context omitted.

Too shallow of a dismissal, and you don't determine what everyone else takes seriously. It's been several years now of LLMs only appeasing those with low expectations and inexperience. Unless the only goal was generating boilerplate or really sloppy proofs of concept, LLMs are a waste time for everyone else. This argument is so over already. We're all just hoping for a soft landing when the hangover really kicks in.

I have 40+ years experience and extremely high standards. What you say is entirely wrong.

>I have 40+ years experience

What domain?

Re: The Emergent Symbolic Structure of Artificial Neural Networks

#56

Post by Oarch »

sigpwned wrote: post #9

The big questions I’m taking away are: (1) they are claiming to produce apparently bijective closed-form symbolic representations/approximations of, among other things, LLMs. Is evaluating these closed-form representations more computationally efficient? The implications of that are potentially huge. It would be essentially analytic distillation. Fable on a chip and not a data center would be important — and disrupti…

It's a fascinating emergent behaviour but also one that could have been predicted?

I'd imagine that our brains have become similarly biased towards generating such a generalised symbolic structure over hundreds of millions of years of evolution?

I'm thinking how certain regions of the brain reliably represent things like motor skills, speech, abstract thought.

Re: The Emergent Symbolic Structure of Artificial Neural Networks

#57

Post by Lerc »

akie wrote: post #53
jsrozner wrote: post #39

A big problem with some of these supervised* interpretability approaches is that they can find spurious structure. (There are lots of ways to make the model do what you want; which is roughly what Hewitt and Liang 2019 showed). This paper draws a contrast to a previous method, DAS (distributed alignment search) on page 20. These and related methods rest on theories of causal abstraction, which are great in theory, bu…

So, basically you're saying: I have only skimmed the article, similar approaches had issues in the past, and my own method from my forthcoming paper is better. I mean, no disrespect, but that's the core of your argument, yes?

It certainly comes across as disrespectful.

To my reading the nature of supervision creating the structure seems to be the core of the argument.

Re: The Emergent Symbolic Structure of Artificial Neural Networks

#58

Post by Lerc »

gfody wrote: post #21
hliyan wrote: post #16

Earlier quoted context omitted.

This is nonsense. The human mind cannot visualize more than 3 dimensions. It can perfectly comprehend any number of dimensions as long as they are represented in a vector space. In fact, that's what linear algebra does.

I think the human mind can visualize anything, eg here are some visual descriptions of higher dimensional objects: https://gfody.substack.com/p/mini-hyper-golf

My aphantasia says your milage may vary on that.

On the other hand I deal with higher dimensions better than most, possibly because of my aphantasia.

Re: The Emergent Symbolic Structure of Artificial Neural Networks

#59

Post by akie »

Lerc wrote: post #57
akie wrote: post #53

Earlier quoted context omitted.

So, basically you're saying: I have only skimmed the article, similar approaches had issues in the past, and my own method from my forthcoming paper is better. I mean, no disrespect, but that's the core of your argument, yes?

It certainly comes across as disrespectful. To my reading the nature of supervision creating the structure seems to be the core of the argument.

I have issue with the comment because he says he didn't read it, then unfavorably compares it to a previous method, and finally uses that negative review to plug his own article instead. His criticism might be valid, I'm not in a position to judge, but the self-promotion leaves a sour tastes in my mouth and makes me question how much of the criticism is just drummed up to make his own contribution appear more relevant.

Re: The Emergent Symbolic Structure of Artificial Neural Networks

#60

Post by yorwba »

sigpwned wrote: post #9

The big questions I’m taking away are: (1) they are claiming to produce apparently bijective closed-form symbolic representations/approximations of, among other things, LLMs. Is evaluating these closed-form representations more computationally efficient? The implications of that are potentially huge. It would be essentially analytic distillation. Fable on a chip and not a data center would be important — and disrupti…

Their representation is a key-value store that they embed via a tensor product to reconstruct the vectors the model produces. The catch is that it works best if the key-value mapping is task-specific. So if you already know how to solve the task the model is performing and can transform the input into a data structure that makes it easy to just read off the answer, you can probably also recover the model output.

That in itself is of course not super useful for tasks that we don't already know to do symbolically, but it's nice to see that they can manipulate the keys and values of their representation somewhat independently. So at least some of what LLMs do internally seems to involve a bit more structure than just linear combinations of some overcomplete dictionary.

The interesting question is whether this can be used to interpret the operation of a single layer by lifting it to operate on key-value stores instead, replacing matrix-vector products with reading from some keys and then writing to others. That could potentially move the balance of power more towards CPU inference instead of GPUs.

Post reply on HN