Pathways to Cellular Supremacy in Biocomputing • Hacker News Board

Re: Pathways to Cellular Supremacy in Biocomputing

#11

Post by cbigart » Wed, Jan 01, 2020, 12:57 AM UTC

api wrote: post #9

ThisIBereave wrote: post #7

Earlier quoted context omitted.

the closest computational equivalent to writing that sentence we have today involves executing a model with 1.5 billion parameters. (and I couldn't find an estimate of the energy cost of that)

It's many orders of magnitude larger than 40 watts * 5 seconds. :) I have a running bet with some friends that one of the following is true: (1) The brain is somehow leveraging quantum computing to achieve polynomial or square root acceleration on combinatorial search and optimization problems. (2) P=NP and there exist polynomial time classical algorithms for these problems. (3) The naturalistic hypothesis fails and…

I'm going to throw out (3) because it doesn't make any sense (to me), and we haven't found any evidence that this is true.

(2) seems possible, but highly unlikely.

(1) seems the most probable of the three options, and although I believe we have found evidence that biological systems exploit quantum effects in some instances, there doesn't seem to be any indication that brains (human or otherwise) use quantum effects for computation.

The thing that you seem to be discounting is that the bulk of the work has already been pre-computed. Our brains can do what they do in 5 seconds * 40 watts because they have been "designed" to do so via billions of years of evolution. In ML terms, the training stage has already happened by the time your brain starts thinking, it is simply doing inference at that point.

Re: Pathways to Cellular Supremacy in Biocomputing

#12

Post by api » Wed, Jan 01, 2020, 1:45 AM UTC

cbigart wrote: post #11

api wrote: post #9

Earlier quoted context omitted.

It's many orders of magnitude larger than 40 watts * 5 seconds. :) I have a running bet with some friends that one of the following is true: (1) The brain is somehow leveraging quantum computing to achieve polynomial or square root acceleration on combinatorial search and optimization problems. (2) P=NP and there exist polynomial time classical algorithms for these problems. (3) The naturalistic hypothesis fails and…

I'm going to throw out (3) because it doesn't make any sense (to me), and we haven't found any evidence that this is true. (2) seems possible, but highly unlikely. (1) seems the most probable of the three options, and although I believe we have found evidence that biological systems exploit quantum effects in some instances, there doesn't seem to be any indication that brains (human or otherwise) use quantum effects…

I agree that #1 is by far the most likely. #3 would mean we (meaning natural science) are wrong about the nature of the universe. I included it mostly to get across the mystery we have here, namely that what brains do cognitively on so little power appears to be "impossible" by classical CS metrics.

I think you are incorrect about precomputation though. The human genome is not very large. It's smaller than Windows 10 or Wikipedia. It's also not substantially different from that of a mouse or a chimpanzee. Most of what it encodes is highly conserved metabolic stuff. All the richness of human cognition is realized through a vanishingly small subset of that already small genetic code.

Nearly all learning and cognition happens after birth, meaning it's done by the brain (unless #3) using absurdly less energy than any known method of computation.

Re: Pathways to Cellular Supremacy in Biocomputing

#13

Post by api » Wed, Jan 01, 2020, 2:01 AM UTC

3455er wrote: post #10

api wrote: post #9

Earlier quoted context omitted.

It's many orders of magnitude larger than 40 watts * 5 seconds. :) I have a running bet with some friends that one of the following is true: (1) The brain is somehow leveraging quantum computing to achieve polynomial or square root acceleration on combinatorial search and optimization problems. (2) P=NP and there exist polynomial time classical algorithms for these problems. (3) The naturalistic hypothesis fails and…

The brain is not a von-neumann architecture. We have different architectures that can perform computation million times more efficiently than general computers. Of course, they lose on other axes (like precision). Whats 54398456905 * 23423645745? Your 40 W brain can't compute that in a minute, yet a 0.01 W calculator can in a millisecond.

We can build less accurate computers and analog computers. Neither of these even begin to approach what brains can do. A self-driving car's computer takes hundreds of watts to run, uses reduced precision and custom silicon wherever possible, and does not begin to approach the navigational ability of a mouse or bird whose brain consumes less than one watt of power.

The human brain didn't evolve to perform consciously explicit and exact calculations on huge numbers, but our navigational and positional awareness abilities do far more impressive things with far more data much faster than this. A monstrous amount of effective but subconscious number crunching is involved in being aware of where your body is in space using nothing more than vision and sensorimotor feedback, taking apart auditory input (including FFT-like transforms), etc.

I really think CS people suffer from Dunning-Kreuger when they hand wave around the impressiveness of biological systems. Study some actual biology and neuroscience. What biological systems do as a normal part of metabolism and cognition is as awesome and mind-blowing as the vast energies, times, and distances found in astronomy. Computers are specialized devices that perform impressive feats of specialized computation but they do not even approach what biological systems do in terms of total data throughput per unit energy, learning ability, or associative and versatile memory to name just a few.

Edit: computers seem so impressive to us because we built them specifically to do the things we didn't evolve to do very well, but I have little doubt that if there were some kind of evolutionary forcing function selecting us for conscious explicit number crunching ability we would not need computers and wouldn't have built them.

Re: Pathways to Cellular Supremacy in Biocomputing

#14

Post by 234rwe4 » Wed, Jan 01, 2020, 2:29 AM UTC

api wrote: post #13

3455er wrote: post #10

Earlier quoted context omitted.

The brain is not a von-neumann architecture. We have different architectures that can perform computation million times more efficiently than general computers. Of course, they lose on other axes (like precision). Whats 54398456905 * 23423645745? Your 40 W brain can't compute that in a minute, yet a 0.01 W calculator can in a millisecond.

We can build less accurate computers and analog computers. Neither of these even begin to approach what brains can do. A self-driving car's computer takes hundreds of watts to run, uses reduced precision and custom silicon wherever possible, and does not begin to approach the navigational ability of a mouse or bird whose brain consumes less than one watt of power. The human brain didn't evolve to perform consciously…

Parent was making the point that we have no computer with a similar architecture as the human brain (billions of tiny compute elements). Artificial neural networks try to simulate that, but the simulate billions of parameters on thousands of core (CPU/GPU).

Of course it's highly inefficient, just like for the brain is highly inneficient to exactly multiply two numbers.

So you also suffer of Dunning-Kreuger, you imagine that all that computers can be are von-neumann machines.

Re: Pathways to Cellular Supremacy in Biocomputing

#15

Post by 234rwe4 » Wed, Jan 01, 2020, 2:32 AM UTC

api wrote: post #12

cbigart wrote: post #11

Earlier quoted context omitted.

I'm going to throw out (3) because it doesn't make any sense (to me), and we haven't found any evidence that this is true. (2) seems possible, but highly unlikely. (1) seems the most probable of the three options, and although I believe we have found evidence that biological systems exploit quantum effects in some instances, there doesn't seem to be any indication that brains (human or otherwise) use quantum effects…

I agree that #1 is by far the most likely. #3 would mean we (meaning natural science) are wrong about the nature of the universe. I included it mostly to get across the mystery we have here, namely that what brains do cognitively on so little power appears to be "impossible" by classical CS metrics. I think you are incorrect about precomputation though. The human genome is not very large. It's smaller than Windows 10…

> The human genome is not very large.

This just means that the "algorithm of intelligence" is not terribly complicated. So we have hope of reverse engineering it.

Re: Pathways to Cellular Supremacy in Biocomputing

#16

Post by tehsauce » Wed, Jan 01, 2020, 3:05 AM UTC

api wrote: post #13

3455er wrote: post #10

Earlier quoted context omitted.

The brain is not a von-neumann architecture. We have different architectures that can perform computation million times more efficiently than general computers. Of course, they lose on other axes (like precision). Whats 54398456905 * 23423645745? Your 40 W brain can't compute that in a minute, yet a 0.01 W calculator can in a millisecond.

We can build less accurate computers and analog computers. Neither of these even begin to approach what brains can do. A self-driving car's computer takes hundreds of watts to run, uses reduced precision and custom silicon wherever possible, and does not begin to approach the navigational ability of a mouse or bird whose brain consumes less than one watt of power. The human brain didn't evolve to perform consciously…

> A self-driving car's computer takes hundreds of watts to run, uses reduced precision and custom silicon wherever possible, and does not begin to approach the navigational ability of a mouse or bird whose brain consumes less than one watt of power.

I would not trust the brain of a mouse or a bird to drive me in a car. Also the self-driving car computers which take hundreds of watts to run do not take advantage of custom silicon to the greatest possible extent, because the relevant algorithms are evolving rapidly. There is probably at least an order of magnitude or two of power efficiency that can be gained with current systems if the algorithms were truly baked into the chips.

Re: Pathways to Cellular Supremacy in Biocomputing

#17

Post by zitterbewegung » Wed, Jan 01, 2020, 3:29 AM UTC

ThisIBereave wrote: post #7

dnautics wrote: post #6

Earlier quoted context omitted.

why do you assume that writing that sentence was computationally hard?

the closest computational equivalent to writing that sentence we have today involves executing a model with 1.5 billion parameters. (and I couldn't find an estimate of the energy cost of that)

Are you measuring training and or inference ?

Re: Pathways to Cellular Supremacy in Biocomputing

#18

Post by russdill » Wed, Jan 01, 2020, 4:48 AM UTC

ThisIBereave wrote: post #7

dnautics wrote: post #6

Earlier quoted context omitted.

why do you assume that writing that sentence was computationally hard?

the closest computational equivalent to writing that sentence we have today involves executing a model with 1.5 billion parameters. (and I couldn't find an estimate of the energy cost of that)

So? The brain had a heck of a lot more than 1.5B parameters

Re: Pathways to Cellular Supremacy in Biocomputing

#19

Post by dnautics » Wed, Jan 01, 2020, 6:04 AM UTC

3455er wrote: post #10

api wrote: post #9

Earlier quoted context omitted.

It's many orders of magnitude larger than 40 watts * 5 seconds. :) I have a running bet with some friends that one of the following is true: (1) The brain is somehow leveraging quantum computing to achieve polynomial or square root acceleration on combinatorial search and optimization problems. (2) P=NP and there exist polynomial time classical algorithms for these problems. (3) The naturalistic hypothesis fails and…

The brain is not a von-neumann architecture. We have different architectures that can perform computation million times more efficiently than general computers. Of course, they lose on other axes (like precision). Whats 54398456905 * 23423645745? Your 40 W brain can't compute that in a minute, yet a 0.01 W calculator can in a millisecond.

> Of course, they lose on other axes (like precision).

Well the most important axis that they don't have is UI. Von Neumann architectures are easy to program for.

Of course being able to program for means that it comes with a lot of overhead. Just running x86 linux consumes a ton of unnecessary power.

Re: Pathways to Cellular Supremacy in Biocomputing

#20

Post by dnautics » Wed, Jan 01, 2020, 6:07 AM UTC

api wrote: post #9

ThisIBereave wrote: post #7

Earlier quoted context omitted.

the closest computational equivalent to writing that sentence we have today involves executing a model with 1.5 billion parameters. (and I couldn't find an estimate of the energy cost of that)

It's many orders of magnitude larger than 40 watts * 5 seconds. :) I have a running bet with some friends that one of the following is true: (1) The brain is somehow leveraging quantum computing to achieve polynomial or square root acceleration on combinatorial search and optimization problems. (2) P=NP and there exist polynomial time classical algorithms for these problems. (3) The naturalistic hypothesis fails and…

> It's many orders of magnitude larger than 40 watts * 5 seconds. :)

Only if you also want to carry a fully preemptive running operating system with a posix layer, netfilter so you don't get hacked, retpol mitigations, openssl to negotiate tls connections, an ssh daemon so you have access, docker, electron, etc.