21–30 of 135 posts
Even if we could get equal computation from silicon and have an algorithm to run near human-level general AI, humans can maintain the comparative advantage as long as the cost of hardware is more than the cost of raising and educating a human, and the operating cost is more than wages for the same task.
I wonder how many cortical neurons it takes to approximate a ReLu or tanh well. I suspect this number being large than 1. If so the paper only shows an upper bound. Think how many neurons does it take to add a two 10 digit numbers. It is perfectly feasible that some (possibly large) part of this 5-8 layers is just "emulation overhead".
Does someone know of studies of this emulation overhead, even outside biology?
Even between ARM and x86 there is an emulation overhead due to different memory models while both are register machines.
Earlier quoted context omitted.
Up to 256.
Is that 256 weights per-cell per-layer, or 256 in total? Is "how many cells in each layer" a valid question too?
Biology/biochemistry can squeeze a good amount of computation from just 20 watts. Even if we could get equal computation from silicon and have an algorithm to run near human-level general AI, humans can maintain the comparative advantage as long as the cost of hardware is more than the cost of raising and educating a human, and the operating cost is more than wages for the same task.
It doesn’t even need discoveries. Imagine an AI that could manually transcode your high level code into absolutely optimal assembly, simplifying your design, removing unnecessary code, optimizing the code in response to observed behavior, etc. I would guess we have several orders of magnitude of power efficiency loss just from building a system that has understandable and flexible layers of abstraction. A sufficiently powerful AI could work to automatically remove those abstractions and even operate at a higher level of abstraction.
The real question is whether we’re at all on the right track. ¯\_(ツ)_/¯
ANN neurons are Pitts-McCulloch neurons, an extremely stylized model of the 1940's understanding of neurons. Each neuron represents a dot product plus a function application. A biological neuron is 6e17 Daltons[0], so on the order of quadrillions of atoms. A single synapse is a huge landscape studded with receptors of various kinds, and the whole thing is swimming in salt solution where chemicals diffuse stochastical…
ANN neurons are Pitts-McCulloch neurons, an extremely stylized model of the 1940's understanding of neurons. Each neuron represents a dot product plus a function application. A biological neuron is 6e17 Daltons[0], so on the order of quadrillions of atoms. A single synapse is a huge landscape studded with receptors of various kinds, and the whole thing is swimming in salt solution where chemicals diffuse stochastical…
That would make each neuron a complicated computer, functionally equivalent to one pc in capability and data storage.
"The neuronal gene Arc is essential for long-lasting information storage in the mammalian brain, mediates various forms of synaptic plasticity, and has been implicated in neurodevelopmental disorders. However, little is known about Arc’s molecular function and evolutionary origins.
Here, we show that Arc self-assembles into virus-like capsids that encapsulate RNA. Endogenous Arc protein is released from neurons in extracellular vesicles that mediate the transfer of Arc mRNA into new target cells, where it can undergo activity-dependent translation. Purified Arc capsids are endocytosed and are able to transfer Arc mRNA into the cytoplasm of neurons.
These results show that Arc exhibits similar molecular properties to retroviral Gag proteins. Evolutionary analysis indicates that Arc is derived from a vertebrate lineage of Ty3/gypsy retrotransposons, which are also ancestors to retroviruses. These findings suggest that Gag retroelements have been repurposed during evolution to mediate intercellular communication in the nervous system."
Biology/biochemistry can squeeze a good amount of computation from just 20 watts. Even if we could get equal computation from silicon and have an algorithm to run near human-level general AI, humans can maintain the comparative advantage as long as the cost of hardware is more than the cost of raising and educating a human, and the operating cost is more than wages for the same task.
Earlier quoted context omitted.
Transistors, while dramatically simpler than neurons, also have fairly complicated physics. It's quite hard to determine the computational power of a microprocessor when you only know roughly how transistors work and have no idea how microprocessors work.
But digital chips are designed to use transistors as on/off switches, so the complicated physics don't matter for understanding a microprocessor. (It's different for analog ICs, where the characteristics of the individual transistors do matter)
Biology/biochemistry can squeeze a good amount of computation from just 20 watts. Even if we could get equal computation from silicon and have an algorithm to run near human-level general AI, humans can maintain the comparative advantage as long as the cost of hardware is more than the cost of raising and educating a human, and the operating cost is more than wages for the same task.
Human-level intelligence is highly variable. If we’re talking about something that can intelligently and independently make discoveries in math science and engineering, then the comparative advantage can’t stay long because you’d just pose the problem of “make yourself but faster” to it. It doesn’t even need discoveries. Imagine an AI that could manually transcode your high level code into absolutely optimal assembly…
The Halting Problem prevents such an AI from existing, but your point still stands of course.
ANN neurons are Pitts-McCulloch neurons, an extremely stylized model of the 1940's understanding of neurons. Each neuron represents a dot product plus a function application. A biological neuron is 6e17 Daltons[0], so on the order of quadrillions of atoms. A single synapse is a huge landscape studded with receptors of various kinds, and the whole thing is swimming in salt solution where chemicals diffuse stochastical…
Transistors, while dramatically simpler than neurons, also have fairly complicated physics. It's quite hard to determine the computational power of a microprocessor when you only know roughly how transistors work and have no idea how microprocessors work.