Well, if it's so useless why is it on the HN front page? Are there "PR" companies behind promoting items to the HN front page? I'm sure there are because sometimes an article like this comes up at #3 and everyone says it's got no substance, clickbait, etc
Building AI without a neural network
31–40 of 50 posts
Re: Building AI without a neural network
#32It's interesting how when discussing neural networks "inspired by the human brain" always comes up when the brain, as far as we know, is emergent behavior from many little things coordinating, not architected. "Top down" never really works for complex systems, the economy being an obvious example. But we tend to ignore that when thinking about neural networks.
If you look at the actual process for building these deep neural networks, it's actually far more emergent and bottom-up than you might think just reading the news. I took a deep learning class before things really took off [0], and what we were taught was that a lot of deep learning research was just trying different structures to see what worked. Even the people who were good at it didn't base their architectures o…
Early architecture/structural engineering (figuring out arches, up to building cathedrals) was done by "just trying different structures to see what worked". Calling this "emergent" is not what is typically meant by the word -- we don't consider cathedrals to be "emergent structures". It's fair to say that many people imprecisely use the term, but then you are contributing to its drift into becoming (or perhaps always having been) a floating signifier.
Re: Building AI without a neural network
#33This is an ad for hivekit.io and contains almost no substance.
From the text, it sounds to me like they're selling multi-agent systems of some sort. They sounded really promising in the 1990s and then fell silent. Much like neural networks. But neural networks are back, so I guess they think multi-agent systems might be ready for a comeback too. They could be right.
Multi-agent-setups with LLMs (AutoGPT for example) were hyped some weeks ago. And OpenAI with their specialized Bots(?) or what they introduced recently goes in the same direction.
Re: Building AI without a neural network
#34Well, if it's so useless why is it on the HN front page? Are there "PR" companies behind promoting items to the HN front page? I'm sure there are because sometimes an article like this comes up at #3 and everyone says it's got no substance, clickbait, etc
I'll take the "PR Company" thing as a compliment. But no, afraid not. I'm one of the two founders and I wrote this one - sorry if it came across as click-baity, but I just wanted to outline the basic, long term idea that Hivekit is going for.
Re: Building AI without a neural network
#35Earlier quoted context omitted.
If you look at the actual process for building these deep neural networks, it's actually far more emergent and bottom-up than you might think just reading the news. I took a deep learning class before things really took off [0], and what we were taught was that a lot of deep learning research was just trying different structures to see what worked. Even the people who were good at it didn't base their architectures o…
Early architecture/structural engineering (figuring out arches, up to building cathedrals) was done by "just trying different structures to see what worked". Calling this "emergent" is not what is typically meant by the word -- we don't consider cathedrals to be "emergent structures". It's fair to say that many people imprecisely use the term, but then you are contributing to its drift into becoming (or perhaps alway…
But Cathedrals still had central planning, blueprints, ropes stuck in the ground to outline what should be built. Thermites on the other hand use a process called "Stigmergy" in which work allocation and building structures emerge organically without central coordination.
Re: Building AI without a neural network
#36It's interesting how when discussing neural networks "inspired by the human brain" always comes up when the brain, as far as we know, is emergent behavior from many little things coordinating, not architected. "Top down" never really works for complex systems, the economy being an obvious example. But we tend to ignore that when thinking about neural networks.
ANNs aren't top down either though. The architecture isn't as important as people think. https://nonint.com/2023/06/10/the-it-in-ai-models-is-the-dat... Even then, the architectures we use are essentially stumbled on. This is alchemy, not modern chemistry.
Sure - one only needs a sufficiently powerful learning architecture to extract the signal from the data, and assuming that is the case then what is learnt depends on the data not the architecture.
That said, I don't think that LLMs are yet close to fully utilizing the training data. e.g. they are prone to hallucinating due to not thinking ahead and backing themselves into a corner where they have to say something. One obvious improvement for that one is more forward looking prediction (i.e. "thinking ahead" - engage brain before opening mouth), which for LLMs can be addressed by tree of thought rollouts and RL learning.
So, while architecture is not important if all architectures are equally powerful (in which case you'll learn what's available to learn), it certainly does matter if not all are fully up to the job, as it would appear none currently are.
Re: Building AI without a neural network
#37Very good clickbait title. I am envious:)
Re: Building AI without a neural network
#38Earlier quoted context omitted.
It seems the value of this type of approach is not the "thinking" (i.e., contextual understanding) but rather the computational speed of assessing random trials. Compared to evolutionary timescales, it's very, very fast but I wouldn't characterize "thinking" to the randomness of evolution, despite its success.
And yet it is a repeatable process that: - uses the information it already has combined with new inputs to form new information based on both (aka reasoning) - retains learned information over time (aka memory) - contains feedback loops to eventually eradicate "wrong" information (aka learning) - converges to similar conclusions in similar circumstances (aka reasoning or possibly instict) At least several of the most…
I agree, and I didn't mean to imply it's not useful. But I do think it's different and important to understand the limitations regarding those differences. In one important aspect, contextual understanding allows for better decisions in novel environments. The "random trial and error" approach has much less additional benefit in that regard.
Re: Building AI without a neural network
#39I’m going to admit to a bit of confusion here.
“But that leaves large and important areas that GPTs are entirely unfit for: Real-time problem solving in dynamic environments.”
Isn’t Tesla using GPTs at least for vision? What do you call the AI technologies employed by self driving projects and robotics?
Re: Building AI without a neural network
#40It's interesting how when discussing neural networks "inspired by the human brain" always comes up when the brain, as far as we know, is emergent behavior from many little things coordinating, not architected. "Top down" never really works for complex systems, the economy being an obvious example. But we tend to ignore that when thinking about neural networks.
If you look at the actual process for building these deep neural networks, it's actually far more emergent and bottom-up than you might think just reading the news. I took a deep learning class before things really took off [0], and what we were taught was that a lot of deep learning research was just trying different structures to see what worked. Even the people who were good at it didn't base their architectures o…
There is nothing emergent about building a specific topology and hoping it works. Emergent would be providing N number of neurons with no topology and letting the topology arise by itself. No current ANN architectures do this, at least not any widespread ones, and trying several different topologies without knowing which one will work isn’t emergent either.