Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Single cortical neurons as deep artificial neural networks

sciencedirect.com

Post reply on HN
Print view View original

111–120 of 135 posts

Re: Single cortical neurons as deep artificial neural networks

#111

Post by adrianN »

vlovich123 wrote: post #94
simiones wrote: post #55

Earlier quoted context omitted.

It might get it right some of the time, but it will be necessarily wrong some of the time. It's also very possible that for many (possibly even most) TMs, the most efficient algorithm for predicting the output is that TM itself.

I think the problem is you're assuming that general AI = Turing machine, but there's no indication that needs to be the case. "General AI" to me means human intelligence running on an artificial system (silicon, simulated brain, etc), so the optimization I'm thinking of is more akin to having an assembly expert translate your code into assembly than a compiler optimization pass. Given that I have optimized my fair bi…

Humans can't solve the halting problem either, there is no contradiction. The halting problem is a theoretical problem that needn't apply in real life. If you for example restrict your AI to be able to generate optimal assembly for all programs that don't require more than 100PB of source code to write down, the halting problem no longer applies (in fact you can now implement this AI using a regular expression).

Re: Single cortical neurons as deep artificial neural networks

#112

Post by jazzyjackson »

mrfusion wrote: post #37
adrianN wrote: post #5

Earlier quoted context omitted.

Transistors, while dramatically simpler than neurons, also have fairly complicated physics. It's quite hard to determine the computational power of a microprocessor when you only know roughly how transistors work and have no idea how microprocessors work.

Don’t forget that transistors are millions of times faster too.

Faster at what?

Re: Single cortical neurons as deep artificial neural networks

#113

Post by tsimionescu »

vlovich123 wrote: post #94
simiones wrote: post #55

Earlier quoted context omitted.

It might get it right some of the time, but it will be necessarily wrong some of the time. It's also very possible that for many (possibly even most) TMs, the most efficient algorithm for predicting the output is that TM itself.

I think the problem is you're assuming that general AI = Turing machine, but there's no indication that needs to be the case. "General AI" to me means human intelligence running on an artificial system (silicon, simulated brain, etc), so the optimization I'm thinking of is more akin to having an assembly expert translate your code into assembly than a compiler optimization pass. Given that I have optimized my fair bi…

I understand your points, but it's important to understand that humans ARE Turing Machines. We don't know of anything that CAN be computed bit can't be computed by a Turing Machine, so General AI would be a Turing Machine.

The Turing Machine model is specifically designed to abstract what a human (mathematician) does: you have a notebook (tape) and some kind of working memory inside your head, and at any one time you can either read something from the notebook and change the state in your head, or you can write something new in the notebook. This is what a TM does - it is an extremely abstract description of what it means to think, basically.

Re: Single cortical neurons as deep artificial neural networks

#114

Post by tsimionescu »

runT1ME wrote: post #105
simiones wrote: post #55

Earlier quoted context omitted.

It might get it right some of the time, but it will be necessarily wrong some of the time. It's also very possible that for many (possibly even most) TMs, the most efficient algorithm for predicting the output is that TM itself.

It might only be wrong for programs that have never existed and will ever exist.

Sure, that's possible, but the opposite is also possible: it might be wrong for most programs we actually write. Well, to be fair, there is some upper bound for any program running on a real CPU.

Re: Single cortical neurons as deep artificial neural networks

#115

Post by tsimionescu »

freemint wrote: post #88
simiones wrote: post #53

Earlier quoted context omitted.

On the other hand, a single cell much simpler than a neuron (any bacteria) is able to perform significantly more complex calculations than any ANN we've tried so far (successfully interacting with an environment to move and find food). Comparing these kinds of disparate tasks for "computational power levels" between vastly different architectures one of which we're not even close to understanding is generally pretty…

> is able to perform significantly more complex calculations > successfully interacting with an environment to move and find food Yet the strategies they implement are equivalent to suprisingly simple to implement things, such as PI control or gradient descent.

That's only if you ignore the massive problem of actually perceiving the chemical environment, I believe.

Re: Single cortical neurons as deep artificial neural networks

#116

Post by taneq »

adrianN wrote: post #5
zetalyrae wrote: post #2

ANN neurons are Pitts-McCulloch neurons, an extremely stylized model of the 1940's understanding of neurons. Each neuron represents a dot product plus a function application. A biological neuron is 6e17 Daltons[0], so on the order of quadrillions of atoms. A single synapse is a huge landscape studded with receptors of various kinds, and the whole thing is swimming in salt solution where chemicals diffuse stochastical…

Transistors, while dramatically simpler than neurons, also have fairly complicated physics. It's quite hard to determine the computational power of a microprocessor when you only know roughly how transistors work and have no idea how microprocessors work.

Also true - witness the weird circuits generated by evolutionary algorithms on FPGAs etc. where they use chip-specific nonlinear dynamics and capacitative / inductive coupling which only works on the specific chip used to evolve the circuit.

Re: Single cortical neurons as deep artificial neural networks

#117

Post by taneq »

titzer wrote: post #59
yarcob wrote: post #13

Earlier quoted context omitted.

But digital chips are designed to use transistors as on/off switches, so the complicated physics don't matter for understanding a microprocessor. (It's different for analog ICs, where the characteristics of the individual transistors do matter)

Wouldn't it suck if brains are just reservoirs used by the internet that can store roughly 3 memes at a time?

holds up a Duracell battery

"You think thermodynamics means using humans as batteries is a dumb idea? Where do you think you learned thermodynamics?"

faint nyan cat music in the background

Re: Single cortical neurons as deep artificial neural networks

#118

Post by taneq »

zetalyrae wrote: post #2

ANN neurons are Pitts-McCulloch neurons, an extremely stylized model of the 1940's understanding of neurons. Each neuron represents a dot product plus a function application. A biological neuron is 6e17 Daltons[0], so on the order of quadrillions of atoms. A single synapse is a huge landscape studded with receptors of various kinds, and the whole thing is swimming in salt solution where chemicals diffuse stochastical…

> A biological neuron is 6e17 Daltons[0]

A grain of sand is around 6.02E19 Daltons, imagine how much computational power there is in a sand castle!

On the other hand, snails have been shown to use just two neurons to generate their complex (relatively, I mean they're snails after all) feeding behaviour with just two neurons: https://neurosciencenews.com/neurons-decision-making-4370/

(Veering slightly offtopic here but this is the first I've heard the term 'Dalton', I've always just heard them called AMUs.)

Re: Single cortical neurons as deep artificial neural networks

#119

Post by CodesInChaos »

MauranKilom wrote: post #107
CodesInChaos wrote: post #86

Earlier quoted context omitted.

You're right that neural networks don't care too much the shape of most activation functions. I assume that splicing together two decaying exponential functions at the origin would work just as well in practice. However tanh is a bit more special than just having the right symmetries. Sigmoid is the correct function to turn an additive value into a probability (range 0 to 1). Tanh is a scaled sigmoid which fulfills t…

I assume that splicing together two decaying exponential functions at the origin would work just as well in practice. Also known as tanh: https://en.wikipedia.org/wiki/Hyperbolic_functions One "disadvantage" is that it doesn't saturate to [-1.0, 1.0] like appropriately scaled tanh.

By splicing together I mean a piecewise function which is `exp(x) - 1` on the left and `1 - exp(-x)` on the right. Which should be similar enough to tanh for most purposes.

Re: Single cortical neurons as deep artificial neural networks

#120

Post by vlovich123 »

tsimionescu wrote: post #113
vlovich123 wrote: post #94

Earlier quoted context omitted.

I think the problem is you're assuming that general AI = Turing machine, but there's no indication that needs to be the case. "General AI" to me means human intelligence running on an artificial system (silicon, simulated brain, etc), so the optimization I'm thinking of is more akin to having an assembly expert translate your code into assembly than a compiler optimization pass. Given that I have optimized my fair bi…

I understand your points, but it's important to understand that humans ARE Turing Machines. We don't know of anything that CAN be computed bit can't be computed by a Turing Machine, so General AI would be a Turing Machine. The Turing Machine model is specifically designed to abstract what a human (mathematician) does: you have a notebook (tape) and some kind of working memory inside your head, and at any one time you…

The problem for me with that line of reasoning is that it's one based on philosophy & not mathematically proven or with any clear evidence.

For example, [1], [2], [3] all show there are classes of computation outside of Turing machines. So if we agree there are computations outside of Turing machines, then the question is where does the human brain fall and, relatedly, can non-Turing machines run Turing machines? I suspect the answer to the latter question must be yes given the simplicity of a Turing machine (i.e. a pen & pencil is sufficient).

Thus, the fact that a human can execute a Turing machine doesn't conclude anything meaningful to me. If you could show that a Turing machine can execute a human brain, then the human brain would 100% be a Turing machine since a core property of a Turing machine is that it can transfer to any other Turing machine.

Even if we build "general AI" on a Turing machine, all we've shown is that there is a class of intelligence that is at its core a complicated Turing machine. It might suggest that a human brain is also a Turing machine (& I'd shift the weight of my prior from let's say 30% we're not Turing machines to 70% we are), but I think the only way to definitively prove that would be to do so by mathematically proving the model of the human brain, and then maybe also using it to actually clone a human brain onto a Turing machine to prove the model correct.

I think until that happens the argument remains philosophical & whichever side you take to be uninteresting. The only purpose of the debate is to show the question itself is important.

[1] https://en.wikipedia.org/wiki/Hypercomputation [2] http://faculty.poly.edu/~jbain/physinfocomp/Readings/94Hogar... [3] https://www.sciencedirect.com/science/article/pii/S030439750...

Post reply on HN