Nice AD/clickbait
Building AI without a neural network
21–30 of 50 posts
Re: Building AI without a neural network
#22It's interesting how when discussing neural networks "inspired by the human brain" always comes up when the brain, as far as we know, is emergent behavior from many little things coordinating, not architected. "Top down" never really works for complex systems, the economy being an obvious example. But we tend to ignore that when thinking about neural networks.
ANNs aren't top down either though. The architecture isn't as important as people think. https://nonint.com/2023/06/10/the-it-in-ai-models-is-the-dat... Even then, the architectures we use are essentially stumbled on. This is alchemy, not modern chemistry.
A quote from the link you posted: "Then, when you refer to “Lambda”, “ChatGPT”, “Bard”, or “Claude” then, it’s not the model weights that you are referring to. It’s the dataset."
Yet all four of these examples use the same model architecture (transformers).
Re: Building AI without a neural network
#23It's interesting how when discussing neural networks "inspired by the human brain" always comes up when the brain, as far as we know, is emergent behavior from many little things coordinating, not architected. "Top down" never really works for complex systems, the economy being an obvious example. But we tend to ignore that when thinking about neural networks.
It's insightful to observe the difference between the emergent behavior of natural systems and the architectured approach often used in AI development. The focus on neural networks, while beneficial for certain applications, might overlook the potential of emergent, decentralized systems that could offer solutions for complex, dynamic challenges. Hivekit's work is a step towards exploring these alternative approaches, emphasizing the importance of communication and interconnectedness in creating effective complex systems.
Re: Building AI without a neural network
#24This is an ad for hivekit.io and contains almost no substance.
"When I change the way I look at things, the things I look at change."
Re: Building AI without a neural network
#25This is an ad for hivekit.io and contains almost no substance.
At least it's a good example of how to write a great clickbate title for hacker news ;)
Re: Building AI without a neural network
#26Is there any replication of a transformer, GPT, LLM using "hivekit"? If not they should aim at that first.
Re: Building AI without a neural network
#27This is an ad for hivekit.io and contains almost no substance.
From the text, it sounds to me like they're selling multi-agent systems of some sort. They sounded really promising in the 1990s and then fell silent. Much like neural networks. But neural networks are back, so I guess they think multi-agent systems might be ready for a comeback too. They could be right.
Re: Building AI without a neural network
#28Building a platform for spatially aware communication and coordination seems pretty valuable. But it seems pretty unrelated to AI. Their only other blogpost is also a vacuous AI buzz piece. I can only guess that someone not involved with the product has decided that putting out AI-related content for non-technical audiences is a good way to build brand awareness right now?
Re: Building AI without a neural network
#29Earlier quoted context omitted.
ANNs aren't top down either though. The architecture isn't as important as people think. https://nonint.com/2023/06/10/the-it-in-ai-models-is-the-dat... Even then, the architectures we use are essentially stumbled on. This is alchemy, not modern chemistry.
A quote from the link you posted: "Then, when you refer to “Lambda”, “ChatGPT”, “Bard”, or “Claude” then, it’s not the model weights that you are referring to. It’s the dataset." Yet all four of these examples use the same model architecture (transformers).
"Everything else is a means to an end in efficiently delivery compute..."
Without tweaking anything (so not RWKV), you could train a GPT level RNN...if you had the compute to burn.
Re: Building AI without a neural network
#30It's interesting how when discussing neural networks "inspired by the human brain" always comes up when the brain, as far as we know, is emergent behavior from many little things coordinating, not architected. "Top down" never really works for complex systems, the economy being an obvious example. But we tend to ignore that when thinking about neural networks.
> when the brain, as far as we know, is emergent behavior from many little things coordinating, not architected
I wouldn't characterize it like that ... The brain has a specific learning architecture / dynamics that has been created via evolution under selection pressure to learn (i.e predict outcomes) better.
As a product of evolution I wouldn't want to call it "architected" or a top-down design, but for the time being it's the only example we have of such a successful learning system so it would make sense to copy what evolution has done, which means copying how it works on all levels.
A simpler example is convolutional neural nets for vision which were explicitly designed to simplistically mimic some of the behavior of our visual system with it's multi-layer (V1, V2, etc) architecture and local learning rules. Sure there's emergent behavior occurring when data is fed into such a system (both brain's visual system or CNN), such as the pattern detectors we see emerge at lowest level, but this emergent behavior is a result of the overall architecture.