Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is my toddler a stochastic parrot?

newyorker.com

Post reply on HN
Print view View original

81–90 of 363 posts

Re: Is my toddler a stochastic parrot?

#81

Post by dekhn »

I think the position of Gebru, et al, can best be expressed as a form of "human exceptionalism". As a teenager, my english teacher shared this writing by Faulkner, which he delivered as his Nobel Prize speech. I complained because the writing completely ignores the laws of thermodynamics about the end of the universe.

"""I believe that when the last ding-dong of doom has clanged and faded from the last worthless rock hanging tideless in the last red and dying evening, that even then there will still be one more sound: that of man's puny, inexhaustible, voice still talking! ...not simply because man alone among creatures has an inexhaustible voice, but because man has a soul, a spirit capable of compassion, sacrifice and endurance."""

Re: Is my toddler a stochastic parrot?

#83

Post by throwaway4aday »

mo_42 wrote: post #66
LASR wrote: post #5

Often the "stochastic parrot" line is used as a reduction on what an LLM truly is. I firmly believe that LLMs are stochastic parrots and also that humans are too. To the point where I actually think even consciousness itself is a next-token predictor. Where the industry is headed - multi-modal models. This really I think is the remaining frontier of LLM Human parity. I also have a 15 month old son. It's totally obvio…

> I firmly believe that LLMs are stochastic parrots and also that humans are too. To the point where I actually think even consciousness itself is a next-token predictor. I agree with the first sentence but not with the second one. Consciousness most probably does not arise from just a next-token predictor. At least not from an architecture similar to current LLMs. Both humans and LLMs basically learn to predict what…

In order to predict what happens next we need to create a model of the world. We exist as part of the world so we need to model ourselves within it. We also have to model our mind for it to be complete, including the model of the world it contains. Oops, I just created an infinite loop.

Re: Is my toddler a stochastic parrot?

#84

Post by dekhn »

If you wonder about toddlers, wait til you have teenagers.

In fact I've observed (after working with a number of world-class ML researchers) that having children is the one thing that convinces ML people that learning is both much easier and much harder than what they do in computers.

Re: Is my toddler a stochastic parrot?

#85

Post by jancsika »

xianshou wrote: post #33

Pour one out for the decline of human exceptionalism! Once you get material-reductionist enough and accept yourself as a pure function of genetics, environment, past experience, and chance, this conclusion becomes inevitable. I also expect it to be the standard within a decade, with AI-human parity of capabilities as the key driver.

I'm not convinced that this material-reductionist view wouldn't just be functionally equivalent to the way a majority of citizens live their lives currently.

Now: a chance encounter with someone of a different faith leads a citizen to respect the religious freedom of others in the realm of self-determination.

Future: a young hacker's formative experience leads to the idea that citizens should have the basic right to change out their device's recommendation engine with a random number generator at will.

Those future humans will still think of themselves as exceptional because the AI tools will have developed right alongside the current human-exceptionalist ideology.

Kinda like those old conservative couples I see in the South where the man is ostensibly the story teller and head of household. But if you listen long enough you notice his wife is whispering nearly every detail of importance to help him maintain coherence.

Re: Is my toddler a stochastic parrot?

#86

Post by beezlebroxxxxxx »

MichaelZuo wrote: post #36
Mattasher wrote: post #8

Humans have a long history of comparing ourselves, and the universe, to our latest technological advancement. We used to be glorified clocks (as was the universe), then we were automatons, then computers, then NPC's, and now AI's (in particular LLM's). Which BTW I don't think is a completely absurd comparison, see https://mattasher.substack.com/p/ais-killer-app

Each successive comparison is likely getting closer and closer to the truth.

Or each successive comparison is just compounding and reiterating the same underlying assumption (and potentially the same mistake) whether it's true or not.

Re: Is my toddler a stochastic parrot?

#87

Post by thealfreds »

yojo wrote: post #59
throw0101a wrote: post #35

> But he could understand so much more than he could say. If you asked him to point to the vacuum cleaner, he would. Perhaps worth noting that it is possible to teach infants (often starting at around 9 months) sign language so that they can more easily signal their desires. Some priority recommended words would probably be: * hungry/more * enough/all done (for when they're full) * drink (perhaps both milk/formula an…

FWIW, I tried this with both my sons. They both started using the gestures the same day they started actually talking :-/ I have friends who had much more success with it, but the value will largely depend on your child’s relative developmental strengths. A friend’s son with autism got literally years’ benefit out of the gestures before verbal speech caught up.

Same with my nephew. He also has autism and the first thing the speech therapist did when he was 3 was teach him simple sign language. It became such a great catalyst for communication. He's nowhere near his his age (now 6) developmentally but within ~6 weeks he went from completely non-verbal to actually vocalizing the simple words he learned the sign language for.

Re: Is my toddler a stochastic parrot?

#88

Post by ASalazarMX »

yojo wrote: post #59
throw0101a wrote: post #35

> But he could understand so much more than he could say. If you asked him to point to the vacuum cleaner, he would. Perhaps worth noting that it is possible to teach infants (often starting at around 9 months) sign language so that they can more easily signal their desires. Some priority recommended words would probably be: * hungry/more * enough/all done (for when they're full) * drink (perhaps both milk/formula an…

FWIW, I tried this with both my sons. They both started using the gestures the same day they started actually talking :-/ I have friends who had much more success with it, but the value will largely depend on your child’s relative developmental strengths. A friend’s son with autism got literally years’ benefit out of the gestures before verbal speech caught up.

There's probably variation among babies. One of my nephews would examine his feet if you asked them where are his shoes, even before walking. He got so proficient with signs that it delayed talking; he preferred signaling and grunting :/

Re: Is my toddler a stochastic parrot?

#89

Post by dekhn »

idle_zealot wrote: post #69
dekhn wrote: post #41

Earlier quoted context omitted.

We'll see! I came to this conclusion a long time ago but at the same time I do subjectively experience consciousness, which in itself is something a mystery in the material-reductionist philosophy.

I hear this a lot, but is it really a mystery/incompatible with materialism? Is there a reason consciousness couldn't be what it feels like to be a certain type of computation? I don't see why we would need something immaterial or some undiscovered material component to explain it.

I mean, if you're a compatibilist, there's no mystery. In that model, we live in a causally deterministic universe but still have free will. I would say instead "the subjective experience of consciousness is an emergent property of complex systems with certain properties". I guess that's consistent with "the experience of consciousness is the feeling of a certain type of computation".

Personally these sorts of things don't really matter to me- I don't really care if other people are conscious, and I don't think I could prove it either way- I just assume other people are conscious, and that we can make computers that are conscious.

And that's exactly what I'm pushing for: ML that passes every Turing-style test that we can come up with. Because, as they say "if you can't tell, does it really matter?"

Re: Is my toddler a stochastic parrot?

#90

Post by robwwilliams »

TaylorAlexander wrote: post #48
ImHereToVote wrote: post #46

Earlier quoted context omitted.

Except LLM's are built on neural networks. That are based on how neurons work. The first tech that actually copies aspects of us.

sigh Neural networks are not based on how neurons work. They do not copy aspects of us. They call them neural networks because they are sort of conceptually like networks of neurons in the brain but they’re so different as to make false the statement that they are based on neurons.

If you study retinal synaptic circuitry you will not sigh so heavily and you will in fact see striking homologies with hardware neural networks, including feedback between layers and discretized (action potential) outputs via the optic nerve.

I recommend reading Synaptic Organization of the Brain or getting into if you are brave, the primary literature on retinal processing of visual input.

Post reply on HN