281–290 of 412 posts
Earlier quoted context omitted.
Is Chomsky an expert? Try reading his infuriatingly tone-deaf NYTimes editorial.
Well not in the field he's a professional clickbaiter targeting a specific, well defined eco chamber.
These debates about how well GPT can think seem merely philosophical. This is a perfect case of perfection being the enemy of the good. Useful AI is here. Hard stop. The impacts will be huge and unpredictable. Billions will be made. The world will change. Humans will continue making rapid progress, via merging various AI methods and new breakthroughs. Nevertheless, I enjoy reading the debate. But anyone wringing thei…
> But anyone wringing their hands over how much is GPT thinking is missing the point.
You seem to be missing the point.
> These debates about how well GPT can think seem merely philosophical.
Merely? Yeah, you're missing the point. You want a debate or you think the debate is meaningless? You don't get to appreciate it and call it pointless and sound reasonable at the same time.
> The ship has sailed. The revolution has started. Progress will undoubtedly be rapid and continual.
It started 2 million years ago when humans started roaming the planet. We're clearly a runway process. We don't need a chat bot to prove it.
> The large language models are a little surprising. I’ll give you that. I think this is the key point about LLMs that kind of explains the wide and polarized views on whether it understands or parrots, whether it can think or is the precursor to thinking or is a dead-end, whether it will catastrophically destroy the world, or “merely” make it steadily worse with bullshit, or just put a few industries out of a job. A…
"A whole mythology is deposited in our language" - Wittgenstein
No world model ? A world model is so obvious papers like these are more confirmation than surprise https://arxiv.org/abs/2305.11169 https://arxiv.org/abs/2210.13382 There a certain sentiment that AGI however you wish to define it won't infact be a "We'll know it when we see it" situation but rather a "AGI will arrive long before consensus reaches its AGI". LLMs have made me believe this will 100% be the case, either…
LLM’s are not people, they lack common sense, but they understand and can reason about what they are trained on. That is exceedingly powerful and very useful even at today’s level of ability, so products built on top of this technology are going to transform everything. The trick is boxing it in and only making it do things it can, so the art of LLM product development will have to become a whole subfield of software engineering until the LLM’s develop to the point where their map of the world is close enough to the world itself.
One of my goals in life before dementia sets in for real is to devise some model, perhaps a conceptual framework which will allow us to escape the clutches of habitual simplification, a subset of which is dichotomous thinking, which in turn leads to the inevitable painting of strawmen as a way to prove our point (among other things). How sweet it would be to shortcut all the mandatory twists and turns of discourse th…
The limitations of language make adversarial debate a powerful tool for advancing our understanding. It might be construed as unfortunate, but it brought us this far. Maybe when we learn higher dimensional ways of communicating we can get better tools for constructing common knowledge.
I believe our understanding will go, and would have gone, further with less adversity and more "philosophers charity". I'm not sure if we need any "higher dimensional ways of communicating" (whatever that is).
But a pervasive adversity in our society and time puts a natural limit of how un-adversarial our debates can be: you can't expect Ukrainian defenders to extend much "philosophers charity" to Putin.
The more conflict laden a topic, the less truth (multifaceted analysis) one can expect, and (if we aspire to a balanced viewpoint) that's why we need to keep an eye on bias, not just in war where "the first victim is the truth", but always.
Earlier quoted context omitted.
> My current (tentative) resolution of the surprise is that language encoded way more information about reality than we thought it did. I think you are close to the mark, but you have been subtly mislead: language is not the data we are working with. We are working with text . Once you fix that particular failure of word choice, everything else becomes much more clear: text contains much more information than languag…
One experiment that I would love to see is an LLM-like model for audio. Feed it hours and hours of lectures, sound effects, animal calls, music etc. You would be able to talk to it and it would ingest the raw waveform then produce audio as a response. Would it learn the fundamentals of music theory? Would it learn to produce "the sound of a bowling ball hitting a dozen windchimes?" Would it learn to talk in English a…
Earlier quoted context omitted.
The issue with all these experts is they still think it's human nature to be able to fully understand the world before they speak about it. On the contrary it's human nature (and all animal nature) to figure out how to navigate the world without fully understanding or having a complete model of it. All you need is a working model that affects the facets of the world you need to deal with. I still remember in the 90s…
> how to navigate the world without fully understanding or having a complete model of it GPT-4 does not have ANY understanding or model of the world - it just has a model of what tokens (words) are likely to appear in a certain context. If it could build any usable model of the world, and reason about it, I'd be much more impressed. When it quacks like a duck, only the most simplistic view takes it as being a duck.
Earlier quoted context omitted.
> And just a reminder, those of you opining based off your experience with GPT3.5... GPT4 is a huge, huge improvement. God, yes. The number of people of HN pushing up their glasses and saying "well, actshually..." when they're basing their opinions off the 3 questions they asked 3.5 is starting to become pretty grating.
The number of people parroting this is also absolutely astounding and grating too. Like, anyone who has spent 5 minutes on this forum already knows this. It’s probably not necessary to keep pointing it out. Yes some people don’t know ChatGPT 3.5 is the default for non-paying customers.
Earlier quoted context omitted.
I don't really understand your distinction between language and text, but it sounds intrguing. Would you be able to give more detail? I searched but couldn't find anything that seemed to explain it.
Text is an instance of language. Think of it as the difference between the python language and a large collection of python programs. The language describes syntactic and semantic rules, the collection is a sampling of possible programs that encodes a significant amount of information about the world. You could learn a lot about the laws of nature, the internet, even human society and laws by examining all the python…
Earlier quoted context omitted.
> Making that ocean deeper is not a trivial problem that we can just throw more compute or data at. You can't possibly know that, given that we don't actually understand how LLMs work on a high level. > We've pretty much tapped out that depth with GPT4 GPT-4 is three months old and you're confident that its working principle cannot be extended further? Where do you get that confidence from?
> You can't possibly know that, given that we don't actually understand how LLMs work on a high level. It's a fair assumption to make however - basically 80/20 rule. AI research isn't a new thing and I bet you could go back 40/50 years where they thought they were about to have a massive breakthrough to human level intelligence. > GPT-4 is three months old and you're confident that its working principle cannot be ext…