Live data from Hacker News

The Emergent Symbolic Structure of Artificial Neural Networks

arxiv.org

11–20 of 114 posts

Re: The Emergent Symbolic Structure of Artificial Neural Networks

#11
post #10

The human mind cannot comprehend the capacity of massively multidimensional space. Just going from 2D to 3D creates massive new positional potential (e.g. surface of the earth, vs. the atmosphere above earth...). Now imagine 1,000 dimensions.

>The human mind cannot comprehend the capacity of massively multidimensional space. That is why the scam works, because investors are humans...

Which scam sorry?

Re: The Emergent Symbolic Structure of Artificial Neural Networks

#12
post #10

Earlier quoted context omitted.

>The human mind cannot comprehend the capacity of massively multidimensional space. That is why the scam works, because investors are humans...

Which scam sorry?

The scam that is based on the implicit claim that LLM is a path to AGI.

Seeing LLMs for what they really are will also make it clear they are fundamentally unfit for a lot of tasks they are currently marketed for...

Re: The Emergent Symbolic Structure of Artificial Neural Networks

#13
post #12

Earlier quoted context omitted.

Which scam sorry?

The scam that is based on the implicit claim that LLM is a path to AGI. Seeing LLMs for what they really are will also make it clear they are fundamentally unfit for a lot of tasks they are currently marketed for...

Too broad a statement, and without substantiation, to be taken serious, sorry.

Re: The Emergent Symbolic Structure of Artificial Neural Networks

#15
"Vectors seem inadequate for capturing the structure of language, logic, and other cognitive domains, yet neural networks achieve impressive performance in these areas". Missing the forest for the trees? Aren't neural networks modeled after biological systems? Our brains are obviously able to contain symbolic structure despite not having a "symbol processing unit".

Re: The Emergent Symbolic Structure of Artificial Neural Networks

#16

The human mind cannot comprehend the capacity of massively multidimensional space. Just going from 2D to 3D creates massive new positional potential (e.g. surface of the earth, vs. the atmosphere above earth...). Now imagine 1,000 dimensions.

This is nonsense. The human mind cannot visualize more than 3 dimensions. It can perfectly comprehend any number of dimensions as long as they are represented in a vector space. In fact, that's what linear algebra does.

Re: The Emergent Symbolic Structure of Artificial Neural Networks

#17
post #13
post #12

Earlier quoted context omitted.

The scam that is based on the implicit claim that LLM is a path to AGI. Seeing LLMs for what they really are will also make it clear they are fundamentally unfit for a lot of tasks they are currently marketed for...

Too broad a statement, and without substantiation, to be taken serious, sorry.

Too shallow of a dismissal, and you don't determine what everyone else takes seriously.

It's been several years now of LLMs only appeasing those with low expectations and inexperience. Unless the only goal was generating boilerplate or really sloppy proofs of concept, LLMs are a waste time for everyone else. This argument is so over already. We're all just hoping for a soft landing when the hangover really kicks in.

Re: The Emergent Symbolic Structure of Artificial Neural Networks

#18
post #13

Earlier quoted context omitted.

Too broad a statement, and without substantiation, to be taken serious, sorry.

Too shallow of a dismissal, and you don't determine what everyone else takes seriously. It's been several years now of LLMs only appeasing those with low expectations and inexperience. Unless the only goal was generating boilerplate or really sloppy proofs of concept, LLMs are a waste time for everyone else. This argument is so over already. We're all just hoping for a soft landing when the hangover really kicks in.

What about the mathematical advancements?

Re: The Emergent Symbolic Structure of Artificial Neural Networks

#19
this is an obvious result. for example, this guy has been writing on substack about this for at least a year or two (with code snippets) explaining the phenomenon of grokking and the ghostbasin.com concept - https://richardaragon.substack.com/

their algorithm is even named "DISCOVER" so they set out to discover the connective tissue of why the universe has invariants like math, and lo it was discovered.

i guess good job for having credentials & publishing the math so people 2years behind the curve can learn from your tenure?

yes. large matrices can gradient descend to understand arbitrary symbolic logic.

ENGLISH IS INSUFFICIENT but it is at least a few decades of math proofs & progress :) welcome to the future Slackernews

Re: The Emergent Symbolic Structure of Artificial Neural Networks

#20
post #16

The human mind cannot comprehend the capacity of massively multidimensional space. Just going from 2D to 3D creates massive new positional potential (e.g. surface of the earth, vs. the atmosphere above earth...). Now imagine 1,000 dimensions.

This is nonsense. The human mind cannot visualize more than 3 dimensions. It can perfectly comprehend any number of dimensions as long as they are represented in a vector space. In fact, that's what linear algebra does.

Yes. It's worth pointing out that anything with n distinct parameters is just a point in n-dimensional space. We're so used to handling so many dimensions that nobody ever bats an eye until someone brings up the magic word "dimensions". It's quite intuitive actually.

The trivial example that comes to mind is the character customization sliders in many video games.

Post reply on HN