Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is my toddler a stochastic parrot?

newyorker.com

Post reply on HN
Print view View original

161–170 of 363 posts

Re: Is my toddler a stochastic parrot?

#161

Post by PaulDavisThe1st »

Probiotic6081 wrote: post #10
LASR wrote: post #5

Often the "stochastic parrot" line is used as a reduction on what an LLM truly is. I firmly believe that LLMs are stochastic parrots and also that humans are too. To the point where I actually think even consciousness itself is a next-token predictor. Where the industry is headed - multi-modal models. This really I think is the remaining frontier of LLM Human parity. I also have a 15 month old son. It's totally obvio…

> I firmly believe that LLMs are stochastic parrots and also that humans are too. To the point where I actually think even consciousness itself is a next-token predictor. Almost every time I'm on hackernews I end up baffled by software engineers feeling entitled to have an unfunded opinion on scientific disciplines outside of their own field of expertise. I've literally never encountered that level of hubris from any…

From the first line of Wilson's paper:

> There is a movement afoot in cognitive science to grant the body a central role in shaping the mind.

It is far from true to say "having a body and sensorimotor interactions with the environment are already established as fundamental preconditions for cognition and in turn consciousness"

It is a popular idea in some groups of people that study these questions. But there are many other similar groups of peopel studying these questions who do not agree with it, certainly not stated as strongly as you have put it here.

Also, a reminder that HN readership and its commentariat, while dominated by SWE, is not limited to them.

Re: Is my toddler a stochastic parrot?

#163

Post by TaylorAlexander »

robwwilliams wrote: post #90
TaylorAlexander wrote: post #48

Earlier quoted context omitted.

sigh Neural networks are not based on how neurons work. They do not copy aspects of us. They call them neural networks because they are sort of conceptually like networks of neurons in the brain but they’re so different as to make false the statement that they are based on neurons.

If you study retinal synaptic circuitry you will not sigh so heavily and you will in fact see striking homologies with hardware neural networks, including feedback between layers and discretized (action potential) outputs via the optic nerve. I recommend reading Synaptic Organization of the Brain or getting into if you are brave, the primary literature on retinal processing of visual input.

Actually it’s funny my best friend is a neuroscientist and studies the retina and in particular the way different types of retinal cells respond to stimulus. I have watched her give presentations on her work and I do see that there are some similarities.

But it is nonetheless the case that “neural networks” are not called that because they are based on the way neurons work.

Re: Is my toddler a stochastic parrot?

#164

Post by xigency »

drew-y wrote: post #145
xigency wrote: post #56

Regarding dismissals of LLM’s on ‘technical’ grounds: Consciousness is first a word and second a concept. And it’s a word that ChatGPT or Llama can use in an English sentence better than billions of humans worldwide. The software folks have made even more progress than sociologists, psychologists and neuroscientists to be able to create an artificial language cortex before we understand our biological mind comprehens…

Totally agree with the sentiment. I find the constant debates on "is AI conscious" or "can AI understand" exhausting. You can't have a sound argument when neither party even agrees on a concrete definition of consciousness or understanding. Regarding this line: > ChatGPT can already tell you a lot about itself (showing awareness) and will gladly walk you through its “thinking” if you ask politely. Is it actually walk…

> Is it actually walking you through its thinking? Or is it walking you through an imagined line of thinking?

You can prompt an LLM model to provide reasoning first and an answer second and it becomes one and the same.

Worth keeping in mind that all of these points are orthogonal to the quality of reasoning, the bias, or the intentions of the system builders. And building something that emulates humans convincingly, you can expect it to emulate both the good and bad qualities naturally.

Re: Is my toddler a stochastic parrot?

#165

Post by TaylorAlexander »

martindbp wrote: post #94
TaylorAlexander wrote: post #48

Earlier quoted context omitted.

sigh Neural networks are not based on how neurons work. They do not copy aspects of us. They call them neural networks because they are sort of conceptually like networks of neurons in the brain but they’re so different as to make false the statement that they are based on neurons.

Sigh... Everyone knows artificial neurons are not like biological neurons. The network is the important part, which really is analogous to the brain, while what came before (SVMs and random forests) are nothing like it.

> Everyone knows artificial neurons are not like biological neurons.

Not, apparently, the person I was replying to!

Re: Is my toddler a stochastic parrot?

#166

Post by allemagne »

alexwebb2 wrote: post #117
Retr0id wrote: post #63

This is a really beautiful article, and while there are certainly fundamental differences between how a toddler thinks and learns, and how an LLM "thinks", I don't think we should get too comfortable with those differences. Every time I say to myself "AI is no big deal because it can't do X", some time later someone comes along and makes an AI that does X.

There's a well-documented concept called "God of the gaps" where any phenomenon humans don't understand at the time is attributed to a divine entity. Over time, as the gaps in human knowledge get filled, the god "shrinks" - it becomes less expansive, less powerful, less directly involved in human affairs. The definition changes. It's fascinating to watch the same thing happen with human exceptionalism – so many cries…

"God is dead" is beyond passé in the 2020s, but in the 19th century nobody really needed a "god of the gaps." If a Friedrich Nietzsche equivalent was active today, and let's just say was convinced that AGI was possible, I kind of wonder what generally accepted grand narrative he'd declare dead beyond just human exceptionalism. Philosophy itself?

Re: Is my toddler a stochastic parrot?

#167

Post by lancesells »

esafak wrote: post #124
lancesells wrote: post #98

Earlier quoted context omitted.

> The AIs will be creating masterwork art and literature personalized for each person better than anything Shakespeare ever wrote or Michaelangelo ever sculpted I don't think it will. Art is truth and AI is a golem replicating humanity.

It depends on what you value in art. Some value the only artifact per se; others the story around it. Unfortunately, I don't see too many complaints that image generators have no human artist behind them.

I've yet to see too much art from image generators. For sure content, but not art.

Re: Is my toddler a stochastic parrot?

#169

Post by sushisource »

ketzo wrote: post #105
ryandrake wrote: post #71

Earlier quoted context omitted.

More specifically, I think we're worried about AI taking our incomes , not our jobs. I would love it if an AI could do my entire job for me, and I just sat there collecting the income while the AI did all the "job" part, but we know from history (robotics) that this is not what happens. The owners of the robots (soon, AI) keep all the income and the job goes away. An enlightened Humanity could solve this by separatin…

I mean, I definitely hear (and feel) a lot of worry about AI taking away work that we find meaningful and interesting to do, outside of the pure money question. I really like programming to fix things. Even if I weren’t paid for it, even if I were to win the lottery, I would want to write software that solved problems for people. It is a nice way to spend my days, and I love feeling useful when it works. I would be v…

The practical reason _is_ that it's fun and you like it. That could be enough.

I'm not too concerned about that being a reality in our lifetimes though.

Re: Is my toddler a stochastic parrot?

#170

Post by xigency »

mo_42 wrote: post #66
LASR wrote: post #5

Often the "stochastic parrot" line is used as a reduction on what an LLM truly is. I firmly believe that LLMs are stochastic parrots and also that humans are too. To the point where I actually think even consciousness itself is a next-token predictor. Where the industry is headed - multi-modal models. This really I think is the remaining frontier of LLM Human parity. I also have a 15 month old son. It's totally obvio…

> I firmly believe that LLMs are stochastic parrots and also that humans are too. To the point where I actually think even consciousness itself is a next-token predictor. I agree with the first sentence but not with the second one. Consciousness most probably does not arise from just a next-token predictor. At least not from an architecture similar to current LLMs. Both humans and LLMs basically learn to predict what…

> However, LLMs only predict when we ask them. In contrast, humans predict something all the time. Even when we don't have any sensory input, our brain plays scenarios.

Pretty easy to do this exercise with an LLM. At least, easier than building an LLM in the first place. Leave it running, let it talk to itself, revisit partial memories, and explore noise. Really a duct-tape problem here more than anything.

Post reply on HN