Is my toddler a stochastic parrot?
newyorker.com
Is my toddler a stochastic parrot?
1–10 of 363 posts
Re: Is my toddler a stochastic parrot?
#2I wonder when the rate of improvement of SOTA benchmarks will exceed the rate of improvement of irl early childhood cognitive development
Re: Is my toddler a stochastic parrot?
#3Great essay; impressive content.
The fact that it's very unlikely for any of the current models to create something that even remotely resembles this article tells me we are very far away from AGI.
Re: Is my toddler a stochastic parrot?
#4Great essay; impressive content. The fact that it's very unlikely for any of the current models to create something that even remotely resembles this article tells me we are very far away from AGI.
don't underestimate exponentials
Re: Is my toddler a stochastic parrot?
#5Often the "stochastic parrot" line is used as a reduction on what an LLM truly is.
I firmly believe that LLMs are stochastic parrots and also that humans are too. To the point where I actually think even consciousness itself is a next-token predictor.
Where the industry is headed - multi-modal models. This really I think is the remaining frontier of LLM Human parity.
I also have a 15 month old son. It's totally obvious to me that he's definitely learning by repetition. But the sources of training data is much more high bandwidth than whatever we're training our LLMs on.
It's been a couple of years since GPT-3. It's time to abandon this notion of "stochastic parrot" as a derogatory. Anyone stuck in this mindset really is going to be hindered from making significant progress in developing utility from AI.
Re: Is my toddler a stochastic parrot?
#6Great essay; impressive content. The fact that it's very unlikely for any of the current models to create something that even remotely resembles this article tells me we are very far away from AGI.
Did you miss the disclaimer at the bottom that both visual artwork and prose were produced by a combination of generative AI tools and creative prompting? The entire seamless watercolor style piece was just a long outpainting
Re: Is my toddler a stochastic parrot?
#7People are not gonna like that they have to scroll so much here :)
Re: Is my toddler a stochastic parrot?
#8Humans have a long history of comparing ourselves, and the universe, to our latest technological advancement. We used to be glorified clocks (as was the universe), then we were automatons, then computers, then NPC's, and now AI's (in particular LLM's).
Which BTW I don't think is a completely absurd comparison, see https://mattasher.substack.com/p/ais-killer-app
Re: Is my toddler a stochastic parrot?
#9Stochastic parrot was coined by bender, gebru, et al, but it was probably "borrowed" from Regina Rini's "statistical parrot", coined 6 months before stochastic parrots hit the scene.
Re: Is my toddler a stochastic parrot?
#10Often the "stochastic parrot" line is used as a reduction on what an LLM truly is. I firmly believe that LLMs are stochastic parrots and also that humans are too. To the point where I actually think even consciousness itself is a next-token predictor. Where the industry is headed - multi-modal models. This really I think is the remaining frontier of LLM Human parity. I also have a 15 month old son. It's totally obvio…
> I firmly believe that LLMs are stochastic parrots and also that humans are too. To the point where I actually think even consciousness itself is a next-token predictor.
Almost every time I'm on hackernews I end up baffled by software engineers feeling entitled to have an unfunded opinion on scientific disciplines outside of their own field of expertise. I've literally never encountered that level of hubris from anyone else. It's always the software people!
Consciousness is far from being fully understood but having a body and sensorimotor interactions with the environment are already established as fundamental preconditions for cognition and in turn consciousness.
Margaret Wilsons paper from 2002 is a good read: https://link.springer.com/content/pdf/10.3758/BF03196322.pdf
peace