Ask HN: What is the current state of "logical" AI?

> 1-Biological brains have astrocytes that manage the synapses and activity of neurons, constantly and dynamically bringing together different and changing populations of neurons to perform functions (inhibiting some and enhancing activation in others).

Wouldn't this be something that could be mirrored or performed even better by just having more layers of neurons in the network. Of course in addition you could have multiple LLMs doing such work together of selecting best optimised systems for a given work. But it seems like astrocytes based on my limited knowledge are more for the reason of maintenance for biological systems which LLMs wouldn't necessarily have to deal with in the first place. So I'm not sure what kind of advantage astrocytes exactly would bring.

> 2-Neurons aren't the only computational units, astrocytes are also computational units involved in (at minimum based on recent studies) learning and object recognition.

But again - what benefits would they provide more over extra layers or LLMs that act together according to an orchestrating LLM?

> 3-Some cells like Purkinje cells learn patterns even when isolated. Something within the cell is learning/storing information about timed patterns of activity and can respond appropriately when the pattern is re-encountered.

But neural networks can in general learn patterns when isolated. It also seems they are more for physical movement, which we should care more about when building robots rather than text based intelligence. Although it seems like for physical movement there's other blockers, like materials. It's all data structures that take input and produce output, which they receive feedback for whether it worked out well and adapt accordingly. I'd assume if LLMs neural networks were given a look, there would be many pockets like the Purkinje cells.

> Dendrites frequently perform preprocessing on signals prior to the signal being forwarded to the neurons soma.

Again it seems like extra layers of neurons. Because I assume with LLMs and other ML tools in addition layers will start to converge on specific set of processing and functionality as they train more. Preprocessing is just a way to make a larger task into more smaller subtasks.

> Rabbit's olfactory learning and memory is interesting, read up on it if you get a chance. A neuroscientist expert in that field has a theory that matches the data that the network within the olfactory region goes through a minimum-energy type of reconfiguration each time a new scent is detected that is related to some positive or negative. This is interesting from the perspective of how do brains get dynamically reconfigured with learning.

I should

But I mean overall, it all seems still the same concept, just orchestrated differently in certain ways and in biological sense it seems it has had to tackle problems that an LLM hasn't really had to, as it has had billions of years to evolve those layers of different systems, but having also a lot of tooling within it to deal with environmental limitations.

So it seems, that given enough computing power, we should be able to make something that is more intelligent than a human, which I do think GPT-4 already is in so many things.

I also am not sure what exactly would GPT-4 have less intelligence in compared to any animal. If you give it the proper input it should be able to perform at least at level of any animal, maybe not with same speed - as animals and as you mentioned in general there are many neural networks within human and animal bodies that correspond only to certain function and are optimised for that specifically.

The biological neural structures that encode behavior are “trained” through evolution, but even the most advanced animals rely mostly on conditioned (= learned during lifetime) reflexes, and not on the ones “hardcoded” evolutionary.

Certainly not much evolutionary “training” in the human brain has happened in the last 3000 years, yet advancement in our understanding of the world has been plentiful. But human thinking (including rationality, mathematics, etc.) is on a different level to even learned animalistic behavior. Some great apes were taught language and even showed basic abstract conceptual thinking, but were never able to reach the level of 3-5 years old human kids.

The problem with GPTs and other statistical models is that they can learn incredibly complex patterns in anything we can express as bytes, but not learn the simplest concepts of maths despite being trained on the whole corpus of mathematical texts available on the internet, while kids need classes that can be covered in a single textbook to understand them, and adults may need just a textbook for this.

> The picture is getting more complex at a steady pace

I'm sorry, but it just still seems the same concept to me. Am I misunderstanding something? It all seems to be about having some sort of signal travelling through various pathways where there's a mechanism to reward/punish the signal which will get adapted by whichever method of storage.

It would be just a matter of having the proper weights and pathways for the signal to travel to yield desired results. The issues will be with performance as in how fast we get results from the signal, but non the less the concept seems the same to me.

> Possibly, maybe probably. Those examples didn't really describe any functional capability, they just described how different our machine is from many people's understanding of how our brain works, which helps illustrate why comparisons to something like ChatGPT are difficult.

It's just that it all seems conceptually very similar to me. And when I try to reason how my own intuition and reasoning works it all makes sense. Fast and slow thinking make sense as well. Fast thinking or "intuition" is one that will give you the gut feeling about something, which I believe is case for animals as well as for machine learning algorithms. The "model" of the World, both I and LLMs have. The model is in some way represented in the connections and weights of the neurons and other things. Slow thinking is kind of firstly brainstorming ways to solve a problem - which LLMs can do, and then bruteforcing them, coming back back, solving the maze. LLMs may not have perfected this completely yet, but it doesn't honestly seem that far away to me, and I wouldn't be surprised if it was just a problem of scaling up the amount of neurons/layers, etc.

> When you say "enough computing power" do you mean just expanding ChatGPT's number of parameters and training set? I don't personally think that will do the job

I can't guarantee it yet, but seeing the difference between e.g. gpt-4 and gpt-3.5 and open-source models, then there seems to be clear different in power of understanding instructions and coming up with very impressive ways to solve problems in my view. So considering I haven't seen what is next level of gpt-4 yet, it's hard for me to believe there wouldn't be a significant jump in performance when increasing magnitude - unless someone has already tried it and it was proven not to matter.

I will be able to have a more accurate opinion I suppose when I see what gpt-5 can do. Because to me gpt-3.5 is quite useless, but gpt-4 is amazing for so many use-cases which I've tried. And in my view the neurons and the connections must represent some form of modelling of the World to be able to explain those results.

If expanding it isn't enough, then I would still believe now - after having seen gpt-4, that if we try enough different arrangements we can reach human level intelligence.

> Do you mean ChatGPT's current capabilities? If so, animals model the 4D environment they exist in, ChatGPT is obviously limited in areas like that. Those internal models can be key for some types of knowledge.

Can you give an example of a problem that animal can solve that GPT couldn't?

Because GPT can handle 2d, 3d, I'm not sure what you mean by 4d - is that including time, like video? In this case we could try a loop where we ask for actions from GPT after presenting an image, and keeping it in feedback loop. It could prove the ability to reason, except for of course performance side of it. But performance we can solve later, at the moment I would just like to see whether it can perform at least at animal level even if not as quickly.

> I'm sorry, but it just still seems the same concept to me. Am I misunderstanding something? It all seems to be about having some sort of signal travelling through various pathways where there's a mechanism to reward/punish the signal which will get adapted by whichever method of storage.

When I said the picture is getting more complex it was in response to your statement that you thought astrocytes were just for cell maintenance, not computation, so I was providing some details about how those cells are involved in computation (not just cell maint).

> I can't guarantee it yet, but seeing the difference between e.g. gpt-4 and gpt-3.5 and open-source models, then there seems to be clear different in power of understanding instructions and coming up with very impressive ways to solve problems in my view.

While I think ChatGPT is very impressive, I don't think it has "understanding", otherwise it wouldn't happily explain to you how to calculate the 4th side of a triangle. A human knows that a triangle has 3 sides and questions about the 4th side is inconsistent with his/her internal model. ChatGPT just has statistical data about the relationship between words, which is why it told me how to calculate that 4th side.

> I'm not sure what you mean by 4d - is that including time, like video?

Yes, time, but not necessarily video. Time is incorporated into the patterns we detect and the internal models we build.