Seemed to me they are just saying that typical LLMs are consistent:
All AI models might be the same
111–120 of 157 posts
Re: All AI models might be the same
#112Earlier quoted context omitted.
If we could help gorillas or elephants (both highly intelligent) learn to name things and use symbols — in a form they can comprehend and create to express their will — enabling them to pass down their experiences and wisdom across generations, I believe they could quietly be as smart as we are. Ps. I am excited about Google’s Gemma dolphin project ( https://blog.google/technology/ai/dolphingemma/ ), but I would pref…
They tried that with Koko, it didn't work. It did not make the gorilla any smarter, it was an animal, and remained an animal. If you read the criticism about the Koko project, it's that Patterson prompted her to make certain signs. I watched some clips of the signs, and it's very obvious that that is exactly what she did. Animals can only communicate in a limited fashion. They do not have some "hidden" intelligence t…
> They tried that with Koko, it didn't work. It did not make the gorilla any smarter, it was an animal, and remained an animal.
I really doubt both their methods and your conclusion. The project tried to teach animals to adapt to the human naming system, and more importantly, it wasn’t a true social-level experiment with animals.
(That said, in my opinion, Koko was smart. Critics of Patterson’s claims have acknowledged that Koko learned a number of signs and used them to communicate her wants and needs.)
Imagine an alien taking an one- or two-year-old human baby and trying to teach him to communicate with alien using vocal signals that humans have difficulty producing or perceiving—and, most importantly, where naming is not a social-level communication that boosts ecological fitness. That’s basically what they did with Koko. It’s like training an LLM model on text corpora where the loss function (fitness measure) is inconsistent or even meaningless—would that produce a useful model?
We first need to understand how animals naturally name things, and then enrich that naming system in a way that fits their minds—not force them to learn human naming. Most importantly, this naming should improve their ecological fitness in a consistent way, so that they can ‘feel’ the fitness of certain naming and evolve their minds toward it.
That’s why Google DolphinGemma is so remarkable. If they succeed, they might mimic dolphin-like communication in a way other dolphins truly understand—gradually introducing naming and concepts that improve evolutionary fitness: finding food, recognizing others, being happy, and identifying suitable mates. If recursive naming develops, I believe it could even lead to real cognitive evolution.
This is just a rough idea, but I truly believe in it-—based on self-meta-cognition about how my own mind evolved and works, as well as other observations. Unfortunately, I don’t have the power or energy to explore it deeply, but I hope someone working on animal cognition research will take a closer look at using large language models like DolphinGemma and dive deeper.
Re: All AI models might be the same
#113I think some of this effect is explained by different models being trained on almost identical corpa. Even if you try to erase certain works, contamination will still represent some of it, just in less obvious ways.
Re: All AI models might be the same
#114I've noticed that many of the large, separately developed AIs often answer with remarkably similar wording to the same question.
I'm not exactly surprised that "separately developed" AI that hinge on essentially the same core techniques and are fed largely identical corpus often answer with remarkably similar wording to the same question.
I find it grating that the article is sweeping this under the rug. If you train your AI on Shakespeare it won't suddenly know how to control a robot. Meanwhile all Shakespeare bots will be remarkably similar. Coincidence?
Re: All AI models might be the same
#115Earlier quoted context omitted.
LLMs don't think, nor are they intelligent or exhibiting intelligence. Language does have constraints, yet it evolves via its users to encompass new meanings. Thus those constraints are artificial, unless you artificially enforce static language use. And of course, for an LLM to use those new concepts, it needs to be retokenized by being trained on new data. For example, if we trained LLMs only on books, encyclopedia…
Do you think an average human from 1850 transported to the modern world would fare better than an LLM?
This isn't a serious question, is it?
A human would immediately start learning, and understanding the differences. Forming new memories. People learn every, single day.
An LLM would never have a single new memory, nor understand a single change had happened. Without being retokenized via new data it trains on, nothing of its symbolic world view would even change in the tiniest bit.
Telling it so in a context window isn't the LLM learning, that's gone the second the context window is gone. In 5 years, 10 years, 1M years, that LLM would be precisely the same. It would view the world as 1850. Forever.
Meanwhile, being told "It's 2025!" would immediately update the human's expectations. They would see the changes around them, and start learning. Note that initial reaction isn't relevant here, a person would eventually deal with it, and move forward.
You have posted a link, in another response, where someone is working on dynamically updating an LLM. However that is not how current LLMs work, thus invalid in any attempt to refute how I describe LLMs currently.
Further, until we see how this works -- precisely, it may be a non-starter. It may not work as described. It may be hype. It may work, but still require retraining if large amounts of symbolic data needs to change.
There's a reason it takes massive farms of hardware to tokenize, and I'm skeptical that the quoted twitter link reforges all relationships. Again, we'll see.. and this is something required for an LLM to actually "learn" about changes to how tokens are weighted.
Re: All AI models might be the same
#116Earlier quoted context omitted.
You also can't translate "Mage (foaled April 18, 2020) is an American Thoroughbred racehorse who won the 2023 Kentucky Derby" into Hellenistic Greek or some modern indigenous languages because there isn't enough shared context; you'd need to give humans speaking those languages a glossary for any of the translation to make sense, or allow them to interrogate an LLM to act as the glossary. I'd say our current largest…
Why QCD? Quantum chromodynamics, the quantized theory of the nuclear strong force? There is also QED, quantum electrodynamics, which is the quantized field theory for electrodynamics, and then also QFD (quantum flavordynamics) for the weak force. Do you seriously mean to imply that the quantum field theory corresponding to ONLY the strong force, plus gravity, explains every emergent phenomena from there to culture? F…
"I mean, what are we, to believe that this is some sort of a, a magic xylophone or something? Boy, I really hope somebody got fired for that blunder"[1]
Just replace QCD with "known/understood quantum theories" and move on with your life. Thats not the important part of the comment you're replying to.
Re: All AI models might be the same
#117The example given for inverting an embedding back to text doesn't help the idea that this effect is reflecting some "shared statistical model of reality": What would be the plausible whalesong mapping of "Mage (foaled April 18, 2020) is an American Thoroughbred racehorse who won the 2023 Kentucky Derby"? There isn't anything core to reality about Kentucky, its Derby, the Gregorian calendar, America, horse breeds, etc…
Agreed. They aren't converging on a statistical model of reality, they are converging on a statistical model of their training data. In the case of LLMs and the size of the training data it's possible they are also converging on some commonality between all text. I doubt this reveals a core truth but maybe it will give us some insight into what we all agree certain chunks of text represent (when I use this idiom, everyone understand I mean this).
Re: All AI models might be the same
#118The example given for inverting an embedding back to text doesn't help the idea that this effect is reflecting some "shared statistical model of reality": What would be the plausible whalesong mapping of "Mage (foaled April 18, 2020) is an American Thoroughbred racehorse who won the 2023 Kentucky Derby"? There isn't anything core to reality about Kentucky, its Derby, the Gregorian calendar, America, horse breeds, etc…
It doesn’t matter whether the Kentucky Derby is core to reality. The point is it is part of reality. If you want to model reality with 100% accuracy, you need to know about the Kentucky Derby. The author is arguing that models are converging on something close to the platonic ideal representation. So, a perfect model with perfect translatability would in fact be able to communicate the concept of a four legged land animal (named after a being capable of impossible feats) that attempts to be faster than other animals to win a reward for a rider on its back. Whether the platonic representation hypothesis is correct or not and whether our models will ever actually get that good are different questions.
Re: All AI models might be the same
#119I asked Grok, o3-pro, and Claude a question about piezoelectric effects.
They all got it "right", but Claude called out a second order effect that arose from the use case that the other two missed.
I get it, they might all be exploring the same space, but Claude went an extra, important, hop.
Re: All AI models might be the same
#120Earlier quoted context omitted.
Why QCD? Quantum chromodynamics, the quantized theory of the nuclear strong force? There is also QED, quantum electrodynamics, which is the quantized field theory for electrodynamics, and then also QFD (quantum flavordynamics) for the weak force. Do you seriously mean to imply that the quantum field theory corresponding to ONLY the strong force, plus gravity, explains every emergent phenomena from there to culture? F…
"I mean, what are we, to believe that this is some sort of a, a magic xylophone or something? Boy, I really hope somebody got fired for that blunder"[1] Just replace QCD with "known/understood quantum theories" and move on with your life. Thats not the important part of the comment you're replying to. [1]: https://youtu.be/pYrRqMHQY7o
No, it actually is kind of important. Because he obviously does not understand what he's talking about, nor does he apparently have a good mental model of the standard model and its relation to prior work. He is just using buzzwords and leaning on that for sloppily gesturing at his imagination saying "look at all of our treasures, we can already solve all the most fundamental issues! We already have all of the pieces!"
Forgive me if I insist somebody show the most minute amount of competence before entertaining their absolutely wild speculation regarding whether the corpus of our species can explain physics-to-culture.