I hate that we just turned out to be stochastic machines another not something more interesting.
Delusions of grandeur
Natural language instructions induce generalization in networks of neurons
31–40 of 95 posts
Re: Natural language instructions induce generalization in networks of neurons
#32Earlier quoted context omitted.
Strong Sapir-Whorf (linguistic determinism - language constrains thought) became pretty much seen as a joke by the 1980s. Linguistic relativism (weak Sapir-Whorf - language shapes thought) is still respectable (because, I mean, of course it does). Actually, this research might just as well be evidence for linguistic universalism (Chomsky - language enables thought). In general linguistic philosophers have been coming…
Oftentimes I find myself understanding complex concepts before I can describe them, even internally . I am sure everyone has this, as I often read comments praising others' submissions for formulating their thoughts efficiently. So thoughts occur independent of language, but need it to be expressed and shared, even if through pictures and sounds.
Thoughts occur independent of language is same as saying sentient beings think. The question is does the thought you have depend on the language?
I speak tamil and english and can distinctly see how the language drives some of my understanding. If you have a language that has evolved to describe 3D space, would be understand spatial ideas better/faster?
If we are pattern matching creatures, then the patterns are built over a period of time and our earliest scaffolding for the patterns come from our mother tongue (or the languages learnt in early childhood). Subsequent understanding depends on building and expanding on those patterns.
Re: Natural language instructions induce generalization in networks of neurons
#33It's clear that both biological sentient beings and sentient being made in factories in the future will essentially be two sides of the same coin, differing only in their physical composition. Humans today operate as biological AI, powered by cells, and the sentient beings instead operate on transistors. As we progress toward a future where both sentient beings exhibit comparable intelligence, emotions, and learned e…
Hm, no. To start, cells are able to replicate themselves, whereas most silicon used today is not even close to doing so. The story you suggest seems to be built on a limited understanding of the processes involved. It's pretty hard to predict the future, especially given incorrect assumptions.
Cell division is a solved problem. Install new ram module and ctrl+c, ctrl+v from backup, done.
Re: Natural language instructions induce generalization in networks of neurons
#34Birds inspired planes. Later, aerodynamics fed back into ornithology. I've been waiting for LLMs to be evaluated as a model of human thought. Complexity and scale have held neuroscience back. It's nowhere close to building a high-level brain model up from biological primitives. Like ornithology, it could use some feedback. All arguments about AGI aside, a machine was built that writes like a human. Its design is very…
Why assume this is sensible? Aerodynamics can help discover general principles that apply to both birds and planes or whatever else. I don't see how this holds for LLMs and brains. The similarity between LLMs and brains is superficial.
Besides, with human beings, we have a host of philosophical problems that undermine the neuroscientific presumption that a mechanistic and closed view of the brain can account for mental activity entirely, like the problem of intentionality.
Re: Natural language instructions induce generalization in networks of neurons
#35Birds inspired planes. Later, aerodynamics fed back into ornithology. I've been waiting for LLMs to be evaluated as a model of human thought. Complexity and scale have held neuroscience back. It's nowhere close to building a high-level brain model up from biological primitives. Like ornithology, it could use some feedback. All arguments about AGI aside, a machine was built that writes like a human. Its design is very…
> a machine was built that writes like a human
But the real hero here is not the LLM, but the training set. It took ages to collect all the knowledge, ideas and methods we put in books. It cost a lot of human effort to provide the data. Without the data we would have nothing. Without GPT we could use RWKV, Mamba, S4, etc and still get similar results. It's the data not the model.
> the intuition that language is central to thought predates LLMs
Language carries AI and humans. The same distribution of language can be the software running in our brains and in LLMs. I think humans act like conditional language models with multi modality and actions. We use language to plan and solve our problem, work together and learn (a lot) from others.
Language itself is an evolutionary system and a self replicator. Its speed is much faster than biology. We've been on the language exponential for millennia, but just now hit the critical mass for LLMs to be possible.
It's not so important that GPT-4 is a 2T weights model, what matters is that it was trained on 13T tokens of human experience and it now "writes like a human". Does that mean humans also learn the same skills GPT-4 has learned from its training set mostly by language as well?
Re: Natural language instructions induce generalization in networks of neurons
#36It's as if language is itself the latent space for these psychophysical tasks, especially compositional instruction. Their description of it as a scaffolding also seems apt.
i've always assumed that language was required to give your brain the abstractions needed to reference things in the past compared to your current perception (aka now), like an index. if you think about your earliest memories, they almost certainly came after language. i'd be interested to know if any of the documented 'wild child' cases (infants 'raised by wolves') ever delved into what the children remembered before, after being taught language as an adolescent.
Re: Natural language instructions induce generalization in networks of neurons
#37Birds inspired planes. Later, aerodynamics fed back into ornithology. I've been waiting for LLMs to be evaluated as a model of human thought. Complexity and scale have held neuroscience back. It's nowhere close to building a high-level brain model up from biological primitives. Like ornithology, it could use some feedback. All arguments about AGI aside, a machine was built that writes like a human. Its design is very…
[deleted]
Re: Natural language instructions induce generalization in networks of neurons
#38Earlier quoted context omitted.
Strong Sapir-Whorf (linguistic determinism - language constrains thought) became pretty much seen as a joke by the 1980s. Linguistic relativism (weak Sapir-Whorf - language shapes thought) is still respectable (because, I mean, of course it does). Actually, this research might just as well be evidence for linguistic universalism (Chomsky - language enables thought). In general linguistic philosophers have been coming…
The distinction between language and "thought" to me is odd. Language and "thought" are the same thing . The mouth sounds or hand scribbles aren't the language, but expressions of it.
You don't need language to catch a ball, but clearly thinking is required to intercept its trajectory correctly.
Language is about communication.
Re: Natural language instructions induce generalization in networks of neurons
#39It's clear that both biological sentient beings and sentient being made in factories in the future will essentially be two sides of the same coin, differing only in their physical composition. Humans today operate as biological AI, powered by cells, and the sentient beings instead operate on transistors. As we progress toward a future where both sentient beings exhibit comparable intelligence, emotions, and learned e…
Hm, no. To start, cells are able to replicate themselves, whereas most silicon used today is not even close to doing so. The story you suggest seems to be built on a limited understanding of the processes involved. It's pretty hard to predict the future, especially given incorrect assumptions.
Compute is substrate independent.
We use transistors because they are wicked fast and efficient. But a 4090 built from metal balls and wood blocks would still be able to perform all the same calculations. Or a 4090 made by drawing X's and O's on a (really massive) piece of paper. Or one made by connecting a bunch of neurons together for that matter.
Saying cells can multiply doesn't really mean anything, unless is gives ability to access some higher form of compute that is outside the reach of Turing machines. Which it doesn't, because if it did, it would be supernatural.
Re: Natural language instructions induce generalization in networks of neurons
#40Earlier quoted context omitted.
> Or that humans can even do xor with a single neuron. That's news to me. I'm not hugely surprised given I've heard a biological neuron is supposed to be equivalent to a small ANN network, but still, first I've heard of that claim.
https://www.science.org/doi/full/10.1126/science.aax6239
Thanks :)