> neural networks largely work by “hitching a ride” on fundamental computational complexity
If you look at what a biological neural network is actually trying to optimize for, you might be able to answer The Bitter Lesson more adeptly.
Latency is a caveat, not a feature. Simulating a biologically-plausible amount of real-time delay is almost certainly wasteful.
Leaky charge carriers are another caveat. In a computer simulation, you can never leak any charge (i.e. information) if you so desire. This would presumably make the simulation more efficient.
Inhibitory neurology exists to preserve stability of the network within the constraints of biology. In a simulation, resources are still constrained but you could use heuristics outside biology to eliminate the fundamental need for this extra complexity. For example, halting the network after a limit of spiking activity is met.
Learning rules like STDP may exist because population members learned experiences cannot survive across generations. If you have the ability to copy the exact learned experiences from prior generations into new generations (i.e. cloning the candidates in memory), this learning rule may represent a confusing distraction more than a benefit.