41–50 of 50 posts
I guess if you wanted to be pedantic, there is no swarm, or in other words the swarm is just a shorthand way of describing the collective decisions of all the nodes.
Where exactly does the emergent behavior come from if the nodes are dumb, e.g. non-autonomous?
It's interesting how when discussing neural networks "inspired by the human brain" always comes up when the brain, as far as we know, is emergent behavior from many little things coordinating, not architected. "Top down" never really works for complex systems, the economy being an obvious example. But we tend to ignore that when thinking about neural networks.
ANNs aren't top down either though. The architecture isn't as important as people think. https://nonint.com/2023/06/10/the-it-in-ai-models-is-the-dat... Even then, the architectures we use are essentially stumbled on. This is alchemy, not modern chemistry.
> But that leaves large and important areas that GPTs are entirely unfit for: Real-time problem solving in dynamic environments, understanding and reacting to current events or spatial reasoning and coordination in the physical world. It is important to recognize the limitations of these deep neural networks and especially GPTs which both have been hyped to the point where most see it as the solution to everything; s…
Human explainability/interpretability is terrible -- it takes years to understand a person well enough to model their actions well in novel situations, and there's tons of literature that backs up that people consistently invent bullshit explanations when asked why they did things. Humans can only be held to account by telling them to change, which we can do with neural networks too.
Even if humans invent bullshit explanations, in serious cases of accountability is done in the courts system which humans investigate about the whole timeline of events of a dispute which there is very little room for perverting the course of justice and making everything up.
Hence this scenario, lawyers would liked to have known as to why did an AI system give hallucinated citations when it was used in a legal proceeding? It's even worse that legal experts knowingly trusted it and failed to reason with the results from this AI system; because fundamentally it cannot explain why that issue happened. [0] It even goes beyond basic citations, with autonomous cars without humans behind the wheel [1] with the company (Cruise) being unable to convince the regulators or even explain the crashes and had to pull the vehicles off the road due to this high amount of risk.
So yet again, explainability in AI with neural networks is still far worse than humans, even when these systems cannot be trusted in high risk and novel situations.
[0] https://www.theguardian.com/technology/2023/jun/23/two-us-la...
[1] https://www.theguardian.com/us-news/2023/oct/24/driverless-c...
Earlier quoted context omitted.
Early architecture/structural engineering (figuring out arches, up to building cathedrals) was done by "just trying different structures to see what worked". Calling this "emergent" is not what is typically meant by the word -- we don't consider cathedrals to be "emergent structures". It's fair to say that many people imprecisely use the term, but then you are contributing to its drift into becoming (or perhaps alway…
But Cathedrals still had central planning, blueprints, ropes stuck in the ground to outline what should be built. Thermites on the other hand use a process called "Stigmergy" in which work allocation and building structures emerge organically without central coordination.
That would be slightly annoying, but they did this to then justify points about how the resulting model is "emergent". The research process, and the resulting output, can and (my main point) almost always do, differ as to whether they are truly emergent.
This is an ad for hivekit.io and contains almost no substance.
I’m going to admit to a bit of confusion here. “But that leaves large and important areas that GPTs are entirely unfit for: Real-time problem solving in dynamic environments.” Isn’t Tesla using GPTs at least for vision? What do you call the AI technologies employed by self driving projects and robotics?
It is certainly not a generative pre-trained model (GPT) used in FSD or autopilot. Most likely advanced deep learning models for object detection or segmentation with very low latency.
Whatever they are using, it still easily can get confused with similar looking objects on the road, which is why Tesla requires drivers to keep their eyes on the road and hands on the wheel. Not even Tesla, trusts its own AI systems.
It's interesting how when discussing neural networks "inspired by the human brain" always comes up when the brain, as far as we know, is emergent behavior from many little things coordinating, not architected. "Top down" never really works for complex systems, the economy being an obvious example. But we tend to ignore that when thinking about neural networks.
The eye is part of the brain for instance. The details are emergent but there is most definitely strong top down architecture encoded in DNA
Earlier quoted context omitted.
A quote from the link you posted: "Then, when you refer to “Lambda”, “ChatGPT”, “Bard”, or “Claude” then, it’s not the model weights that you are referring to. It’s the dataset." Yet all four of these examples use the same model architecture (transformers).
"Everything else is a means to an end in efficiently delivery compute..." Without tweaking anything (so not RWKV), you could train a GPT level RNN...if you had the compute to burn.
Earlier quoted context omitted.
"Everything else is a means to an end in efficiently delivery compute..." Without tweaking anything (so not RWKV), you could train a GPT level RNN...if you had the compute to burn.
We don't know that. No one has demonstrated it. It's very likely that at larger scales, for a given amount of compute you cannot train a traditional RNN to be as a good as a transformer.
We use Transformers today in large part because they got rid of recursion and in effect could massively parallelize compute.
It's interesting how when discussing neural networks "inspired by the human brain" always comes up when the brain, as far as we know, is emergent behavior from many little things coordinating, not architected. "Top down" never really works for complex systems, the economy being an obvious example. But we tend to ignore that when thinking about neural networks.
[1]: https://en.wikipedia.org/wiki/Friedrich_Hayek#Economic_calcu...
[2]: https://en.wikipedia.org/wiki/Catallaxy
[3]: https://citeseerx.ist.psu.edu/doc_view/pid/2688969848b753368...