21–30 of 241 posts
Earlier quoted context omitted.
> it obviously doesn't Why?
simply because I think that it's rather statistically unlikely, that just because my first word started with "A", the next word should start with "B", "C" ...
> ChatGPT is a glorified word predictor. It isn’t sentient. It doesn’t know what it’s saying, and yes, you can coax it into admitting that it wants to take over the world or saying hurtful things (although it was specially conditioned during training to try to suppress such output). It’s simply stringing words together using an expansive statistical model built from billions of sentences. How do you differentiate it…
Human mind or even something like Wolfram Alpha can perform reasoning.
> ChatGPT is a glorified word predictor. It isn’t sentient. It doesn’t know what it’s saying, and yes, you can coax it into admitting that it wants to take over the world or saying hurtful things (although it was specially conditioned during training to try to suppress such output). It’s simply stringing words together using an expansive statistical model built from billions of sentences. How do you differentiate it…
Human mind can perform actual reasoning, while ChatGPT only mirrors the output of reasoning and when it gets output correctly it's due to mixture of luck and closeness to training material. Human mind or even something like Wolfram Alpha can perform reasoning.
ChatGPT is a glorified word predictor. It isn’t sentient. It doesn’t know what it’s saying, and yes, you can coax it into admitting that it wants to take over the world or saying hurtful things (although it was specially conditioned during training to try to suppress such output). It’s simply stringing words together using an expansive statistical model built from billions of sentences. Is this true though? The publi…
Of course Mac running Safari is a glorified boolean circuit. It's programmed in a way to show the pixels on the screen in a way that you understand, and take inputs from your mouse and keyboard in a way you understand
It would be possible to make a web browser out of a different type of logic circuit. It's the higher-level structure of the browser that matters, and not the fact that it is built out of boolean logic.
Similarly, with ChatGPT, it is the higher-level structures (whatever they may be) that matter, and not the low-level details of the neural network. The higher-level structures could be far too complex for us to understand.
> ChatGPT is a glorified word predictor. It isn’t sentient. It doesn’t know what it’s saying, and yes, you can coax it into admitting that it wants to take over the world or saying hurtful things (although it was specially conditioned during training to try to suppress such output). It’s simply stringing words together using an expansive statistical model built from billions of sentences. How do you differentiate it…
Human mind can perform actual reasoning, while ChatGPT only mirrors the output of reasoning and when it gets output correctly it's due to mixture of luck and closeness to training material. Human mind or even something like Wolfram Alpha can perform reasoning.
Earlier quoted context omitted.
Human mind can perform actual reasoning, while ChatGPT only mirrors the output of reasoning and when it gets output correctly it's due to mixture of luck and closeness to training material. Human mind or even something like Wolfram Alpha can perform reasoning.
Can you explain your proof of that?
Humans have the capacity to come up with new language, new ideas, and basically everything in our human world was made up by someone.
ChatPT or similar, without any training data, cannot do this. Thus they're simply imitating
Earlier quoted context omitted.
> it obviously doesn't Why?
simply because I think that it's rather statistically unlikely, that just because my first word started with "A", the next word should start with "B", "C" ...
The input-so-far influences the probability of the next word in complex ways. Due to the number of parameters in the model, this dependency can be highly nontrivial, on par with the complexity of a computer program. Just like a computer program can trivially generate an A line before switching its internal state so that the next generated line is a B line, so does the transformer since it is essentially emulating an extremely complex function.
Do tell— how can you prove humans are any different?
The most common “proofs” I’ve seen:
“Humans are more complex”. Ok, so you’re implying we add more complexity (maybe more modalities?); if more complexity is added, will you continue to say “LLMs are just word predictors”?
“Humans are actually reasoning. LLMs are not.” Again, how would you measure such a thing?
“LLMs are confidently wrong .” How is this relevant ? And are humans not confidently wrong as well?
“LLMs are good at single functions, but they can’t understand a system.” This is simply a matter of increasing the context limit, is it not? And was there not a leaked OpenAI document showing a future offering of 64k tokens?
All that aside, I’m forever amazed how a seemingly forward-looking group of people is continually dismissive of a tool that came out LITERALLY 4 MONTHS AGO, with its latest iteration less than TWO WEEKS ago. For people familiar with stuff like Moore’s law, it’s absolutely wild to see how people act like LLM progress is forever tied to its current , apparently static, state.
ChatGPT is a glorified word predictor. It isn’t sentient. It doesn’t know what it’s saying, and yes, you can coax it into admitting that it wants to take over the world or saying hurtful things (although it was specially conditioned during training to try to suppress such output). It’s simply stringing words together using an expansive statistical model built from billions of sentences. Is this true though? The publi…
No, it wouldn't, because nothing in "higher-level phenomena" precludes it being caused by statistics and predictions.
This articles describes much of what many youtubers explained in their videos in the recent few weeks. While I understand the core concept of 'just' picking the next word based on statistics, it doesn't really explain how chatGPT can pull off the stuff it does. E.g. when one asks it to return a poem where each word starts with one letter/next alphabet letter/the ending of the last word, it obviously doesn't 'just' pi…
By now, everyone has heard the explanation that ChatGPT is a transformer encoder-decoder that responds to prompts by iteratively predicting the first word in the response, then the second word, and so on... What we need now is explanation of all the further stuff added to that basic capability.
Except it is wrong. GPT models are decoder-only transformers. See Andrej Karpathy's outstanding series on implementing a toy-scale GPT model.