It's exceedingly slow. Neurons fire at not 4 gigahertz, 400 megahertz, 40 megahertz, 4 megahertz, 400 kilohertz, 40 kilohertz, 4 kilohertz, or 0.4 kilohertz (400 hertz.) They fire at That is literally 8 orders of magnitude slower than processers. 8 orders of magnitude is the size of difference between a grain of sand and 1.2 short tons.
In the time that your neuron moves 1 grain of sand, a computer can move 1.2 short tons worth of sand 1 grain at a time (talking about just cache here) - because it's switching at 4 GHz instead of 10 hertz.
That is enough to make a lot of back and forth trips. Now, you have to make a lot of these back and forth trips because the brain is extremely parallel. But the difference between these extremely slow biochemical firing speeds and gigahertz light speed enables this.
The speed of propagation in the brain is about 100 meters/second [2]. The speed of propagation between two fiber optic end-points is about 299 792 458 meters/second. That gives you 7 orders of magnitude to play with. Or, in other words, it can let you take seven orders of magnitude more trips across a data center (or go that much farther) before you miss a real-time deadline for when you need to address a neuron by.
In the future it's pretty clear that computers are not going to think as fast as the human brain when doing similar calculations using a similar topology, but, for example, 10,000 times faster than real-time.
This really shouldn't surprise anyone growing up in technology. Brains are slow, mechanical devices. They're not digital and they don't do signal processing at the speed of light.
It's taken us some time to catch up because a hundred billion neurons is still a lot of neurons, and algorithm development using similar topologies has been slow.
[1] https://aiimpacts.org/rate-of-neuron-firing/#Estimates_of_ra...
[2] https://hypertextbook.com/facts/2002/DavidParizh.shtml