191–200 of 241 posts
Good read but it doesn’t explain anything to me, as a non-researcher who inows nothing about NLP etc. Word prediction makes sense to me for the translation. It’s easy to intuit how training on millions of sentences would allow the algorithm to translate text. But how can it reason about complex questions? Isn’t that entirely different from translating between languages? How can word prediction lead to a coherent long…
“It’s a glorified word predictor” is becoming increasingly maddening to read. Do tell— how can you prove humans are any different? The most common “proofs” I’ve seen: “Humans are more complex”. Ok, so you’re implying we add more complexity (maybe more modalities?); if more complexity is added, will you continue to say “LLMs are just word predictors”? “Humans are actually reasoning. LLMs are not.” Again, how would you…
The strongest answer to almost all of your questions is "Poverty of the stimulus" (wikipedia). 4 year olds are exposed to an almost microscopically tiny amount of words relative to chatgpt (you can probably contain it in a csv file that you can open in excel), and yet can reason, even develop multilingual skills and a huge amount of emotional intelligence from the very little word tokens they are exposed to. So whate…
“It’s a glorified word predictor” is becoming increasingly maddening to read. Do tell— how can you prove humans are any different? The most common “proofs” I’ve seen: “Humans are more complex”. Ok, so you’re implying we add more complexity (maybe more modalities?); if more complexity is added, will you continue to say “LLMs are just word predictors”? “Humans are actually reasoning. LLMs are not.” Again, how would you…
The strongest answer to almost all of your questions is "Poverty of the stimulus" (wikipedia). 4 year olds are exposed to an almost microscopically tiny amount of words relative to chatgpt (you can probably contain it in a csv file that you can open in excel), and yet can reason, even develop multilingual skills and a huge amount of emotional intelligence from the very little word tokens they are exposed to. So whate…
And just like gpt’s data set isnt necessarily truth, so is our own image of the world which as we know can be deeply distorted through abusive childhood, cults etc. In fact, all of human knowledge is simply beliefs, agreed stories about reality. For example "red" is a word/sound that points to an experience. The word alone only has meaning in context (what GPT can handle), but can never substitue for a conscious experience.
Crucially imho, software will never be able to do what the right hemisphere does. And I find it dumbfounding that even Lex Fridman doesnt see the fundamental difference between conceptual thought / language based reasoning, and direct experience aka consciousness.
Earlier quoted context omitted.
The strongest answer to almost all of your questions is "Poverty of the stimulus" (wikipedia). 4 year olds are exposed to an almost microscopically tiny amount of words relative to chatgpt (you can probably contain it in a csv file that you can open in excel), and yet can reason, even develop multilingual skills and a huge amount of emotional intelligence from the very little word tokens they are exposed to. So whate…
ChatGPT is not 4 years old, it's a newcomer, give it some more time, ok? It's still developing, it's not a final product. There is already a huge difference between GPT-3.5 and GPT-4. Next iterations will be able to do more and more, especially if they get tools, self-reflection and vision.
But until someone demonstrates a GPT that can learn from a tiny dataset what a multi-lingual blind 4 year old learns it is very fair to challenge the hypothesis that humans learn the way a deep learning network learn.
Earlier quoted context omitted.
Then this implies that you’d maybe think differently if LLMs could have different inputs, correct? Which they are currently doing. GPT-4 can take visual input. I totally agree that humans are far more complex than that, but just extend your timeline further and you’ll start to see how the gap in complexity / input variety will narrow.
It’s get scary when AI is so advanced that it can keep getting continuous input and output thru visual, audio and even feeling like pressure and temperature in a 3d setting.
Earlier quoted context omitted.
> “Humans are actually reasoning. LLMs are not.” Again, how would you measure such a thing? I would posit that reasoning is the ability to construct new, previously-unexpressed information from prior information. If ChatGPT existed 110 years ago and fed all the then-known relevant experimental data regarding subatomic particles, it would not have been able to arrive at the new notion of quantum mechanics. If it exist…
Asking: "Describe to me a completely new and credible explanation on why space-time is shrinking" It gives me: Title: Isotropic Space-Time Contraction: A Novel Hypothesis for Shrinking Space-Time Abstract: This paper introduces a new and credible explanation for the phenomenon of shrinking space-time, which we call "Isotropic Space-Time Contraction" (ISTC). ISTC postulates that space-time contracts uniformly in all d…
“It’s a glorified word predictor” is becoming increasingly maddening to read. Do tell— how can you prove humans are any different? The most common “proofs” I’ve seen: “Humans are more complex”. Ok, so you’re implying we add more complexity (maybe more modalities?); if more complexity is added, will you continue to say “LLMs are just word predictors”? “Humans are actually reasoning. LLMs are not.” Again, how would you…
ChatGPT doesn't even work with "words" to begin with but with vectors encoding meaning of words. At least as far as I understand it. That's why it is able to capture meaning and concepts to a certain degree.
It has absolutely no encoding of the meaning, however it does have something called an "attention" matrix that it trains itself to make sure it is weighing certain words more than others in it's predictions. So words like "a", "the" etc will eventually count for less than words like "cat", "human", "car" etc when it is predicting new text.
“It’s a glorified word predictor” is becoming increasingly maddening to read. Do tell— how can you prove humans are any different? The most common “proofs” I’ve seen: “Humans are more complex”. Ok, so you’re implying we add more complexity (maybe more modalities?); if more complexity is added, will you continue to say “LLMs are just word predictors”? “Humans are actually reasoning. LLMs are not.” Again, how would you…
The strongest answer to almost all of your questions is "Poverty of the stimulus" (wikipedia). 4 year olds are exposed to an almost microscopically tiny amount of words relative to chatgpt (you can probably contain it in a csv file that you can open in excel), and yet can reason, even develop multilingual skills and a huge amount of emotional intelligence from the very little word tokens they are exposed to. So whate…
Probably multiple brain-areals that work differently and in conjunction. "Left-brain" like language functions working with serial information, "right-brain" function that tend to work on images (= parallel information), combined with symbolic-logical reasoning, an extremely strong programmable aversion system (the emotion of disgust) and the tendency to be lazy = optimizing in- and output.
“It’s a glorified word predictor” is becoming increasingly maddening to read. Do tell— how can you prove humans are any different? The most common “proofs” I’ve seen: “Humans are more complex”. Ok, so you’re implying we add more complexity (maybe more modalities?); if more complexity is added, will you continue to say “LLMs are just word predictors”? “Humans are actually reasoning. LLMs are not.” Again, how would you…
The strongest answer to almost all of your questions is "Poverty of the stimulus" (wikipedia). 4 year olds are exposed to an almost microscopically tiny amount of words relative to chatgpt (you can probably contain it in a csv file that you can open in excel), and yet can reason, even develop multilingual skills and a huge amount of emotional intelligence from the very little word tokens they are exposed to. So whate…
You might say that's not fair because we are comparing a pre-trained LLM with a blank slate newborn. But human hardware is also pre-trained by billions of years of evolution. We are hardwired to understand language and certain world concepts. It is not fair to compare hardware that is designed for language and reasoning to the hardware used for ChatGpt.
Another line of thinking: why does the amount of training matter? LLM and humans are completely different implementations.
Earlier quoted context omitted.
> Do tell— how can you prove humans are any different? How about this one: Humans experience time. Humans have agency. Humans can use both in their reply. If I blurt out the first thing that comes to mind, I feel a lot like a GTP. But I can also choose to pause and think about my response. If I do I might say something different, something hard to quantify but which would be more “intelligent”. That is the biggest di…
> Humans experience time And? So what? > Humans have agency. Which is what exactly? You are living in a physical universe bound by physical laws. For any other system we somehow accept that it will obey physical laws and there will not be a spontaneous change, so why are we holding humans to different standards? If we grow up and accept that free will does not actually exist, then all agency is is our brain trying to…
Language is not a complete representation of thinking.
We use language to describe symbols, not even very precisely, and we can convert imprecise language to more precise symbols in our brain, manipulate them as symbols, and only then turn them back into language.
That’s why you often cannot perfectly translate something between two languages. That’s why nine year olds, who have been trained on far less text, can learn to do math that ChatGTP never could without an API. (They don’t have to generate their output linearly - they can add the one’s column first) When Newton invented calculus he wasn’t predictively generating words by token; he performed logical manipulation of symbols in his brain first.
That’s why LLMs can’t tell you where they got a specific piece of their own output from, while a human can. This matters because LLMs can’t convert it into a symbol and think about it directly and deduce new conclusions from it, while a human can.
If fundamentally human thinking was just “LLM” we would have never generated the words to train ourselves on in the first place! And neither would any new idea that gradually built the library of human knowledge that eventually trained ChatGTP. The language is just the interface; it’s not the full essence of the thinking itself.