171–180 of 363 posts
Regarding dismissals of LLM’s on ‘technical’ grounds: Consciousness is first a word and second a concept. And it’s a word that ChatGPT or Llama can use in an English sentence better than billions of humans worldwide. The software folks have made even more progress than sociologists, psychologists and neuroscientists to be able to create an artificial language cortex before we understand our biological mind comprehens…
Earlier quoted context omitted.
> I firmly believe that LLMs are stochastic parrots and also that humans are too. To the point where I actually think even consciousness itself is a next-token predictor. I agree with the first sentence but not with the second one. Consciousness most probably does not arise from just a next-token predictor. At least not from an architecture similar to current LLMs. Both humans and LLMs basically learn to predict what…
In order to predict what happens next we need to create a model of the world. We exist as part of the world so we need to model ourselves within it. We also have to model our mind for it to be complete, including the model of the world it contains. Oops, I just created an infinite loop.
Earlier quoted context omitted.
FWIW, I tried this with both my sons. They both started using the gestures the same day they started actually talking :-/ I have friends who had much more success with it, but the value will largely depend on your child’s relative developmental strengths. A friend’s son with autism got literally years’ benefit out of the gestures before verbal speech caught up.
There's probably variation among babies. One of my nephews would examine his feet if you asked them where are his shoes, even before walking. He got so proficient with signs that it delayed talking; he preferred signaling and grunting :/
Please don't blame this on the signs! This doesn't mean that he would have learned to speak earlier if not for the signs. I'd be glad that he could communicate proficiently at all.
I think it's reductivism to assume that neural networks cannot emergently support/implement a non-stochastic computational model capable of explicit logical reasoning. We already have an instance of emergent logic. Animals engage in logical reasoning. Corollary, humans and toddlers are not merely super-autocompletes or stochastic parrots. It has nothing to do with "sensory embodiment" and/or "personal agency" argumen…
Imagine that for some reason and by some unknown agency, somebody in 1068 could build something that was functionally equivalent to one of the contemporary LLMs of today (it would likely be mechanical and thus slower, but let's just ignore speed because that's mostly a substrate artifact). OK, time to train up our LLM. Oops. Printing not yet invented. Books are scarce, and frequently not copied. We have almost no inp…
Regarding dismissals of LLM’s on ‘technical’ grounds: Consciousness is first a word and second a concept. And it’s a word that ChatGPT or Llama can use in an English sentence better than billions of humans worldwide. The software folks have made even more progress than sociologists, psychologists and neuroscientists to be able to create an artificial language cortex before we understand our biological mind comprehens…
Sarah Connor was right!
Earlier quoted context omitted.
let's say you wanted to count the number of coins on a table you organize them into piles of ten this created new information
More like you converted information you had about energy powering your muscles into that one - resulting in less total information for you in the end.
This is a really beautiful article, and while there are certainly fundamental differences between how a toddler thinks and learns, and how an LLM "thinks", I don't think we should get too comfortable with those differences. Every time I say to myself "AI is no big deal because it can't do X", some time later someone comes along and makes an AI that does X.
Then never say to yourself "AI is no big deal because it can't do X". Say instead (for example) "It is important that I understand the differences between both the capabilities and internal mechanisms of AI and people, even if, over some period of time, the capabilities may appear to converge".
Earlier quoted context omitted.
Step A: build a machine which reflects a reduced and simplified model of how some part of a human works Step B: turn it on its head "the human brain is nothing more than... " It's a bit tautological. The worry is that there's a Step C: Humans actually start to behave as simple as said machine.
What machines have we built that reflect a reduced and simplified model of how some part of a human works (other than as a minor and generally invisible research projects) ?
Earlier quoted context omitted.
Doesn't really matter to modern CS, but Rosenblatt's original perceptron paper is a good read on this. ANNs were specifically inspired by Natural NNs and there were many attempts to build ANNs using models of how the human brain works, specifically down to the neuron.
I;m sure you know but one of the best ways to get neuro folks worked up is to say anything about neural networks being anything like neurons in brains. (IMHO, Rosenblatt is an underappreciated genius; he had a working shallow computer vision hardware computer long before people even appreciated what an accomplishment that was. The hardware was fascinating- literally self-turning potentiometer knobs to update weights.
A theoretical such thing might be for us to look at, say, human arms and say "Well, this gripping thing is a cool piece of functionality. Let's build an artificial device that does this. But we don't have muscle contraction tech, so we'll put actuators in the gripping portion. All right, we've built an arm. It seems like if we place it in this position it minimizes mechanical wear when not in action and makes it unlikely for initial movement to create undesired results. I wonder if human arms+hands have the same behaviour. Ah, looks like not, but that would have been interesting if it were the case"
Essentially that's just the process of extracting substructure and then seeing if there is a homomorphism (smooshy type abuse here) between two structures as a way to detect yet hidden structure. Category theory is almost all this. I suppose the reason they find it annoying is that there are many mappings that are non-homomorphic and so these are the false cognates of concepts.
Still, I think the whole "An ANN is not a brain" thing is overdone. Of course not. A mechanical arm is not an arm, but they both have response curves, and one can consider a SLAM approach for the former and compare with the proprioceptive view of the latter. It just needs some squinting.
Anyway, considering your familiarity with R and his work, I think I'm not speaking to the uninitiated, but I thought it worth writing anyway.
Earlier quoted context omitted.
sigh Neural networks are not based on how neurons work. They do not copy aspects of us. They call them neural networks because they are sort of conceptually like networks of neurons in the brain but they’re so different as to make false the statement that they are based on neurons.
If you study retinal synaptic circuitry you will not sigh so heavily and you will in fact see striking homologies with hardware neural networks, including feedback between layers and discretized (action potential) outputs via the optic nerve. I recommend reading Synaptic Organization of the Brain or getting into if you are brave, the primary literature on retinal processing of visual input.