Single cortical neurons as deep artificial neural networks • Hacker News Board

Re: Single cortical neurons as deep artificial neural networks

#131

Post by vlovich123 » Sat, Aug 14, 2021, 4:51 PM UTC

vlovich123 wrote: post #130

tsimionescu wrote: post #121

Earlier quoted context omitted.

I do not have access to the third paper, which may hold somewhat more interest. Otherwise, all examples of models of hypercomputation in [1] and [2] are relying on performing an infinite number of steps in a finite time OR on precisely knowing the solution to an uncomputable problem. These are as interesting as trying to solve human flight assuming anti-gravity exists - they are obviously non-physical, absurdist mode…

That’s a solid counter argument. I assume you’re saying we don’t know of a single physical process that’s not a Turing machine because all of the models we have built to simulate them are Turing computable? That’s a strong point. Maybe I should readjust my prior on this. It does sound like you’re better versed in this topic. I like to learn by asking questions, so if you’re amenable to answering please do. If you don…

Oh, and I’m going to read [1] now. I thought the heading was timely and lovely since we were discussing this and I look forward to reading it (I haven’t yet so I do t know what the article says yet)

[1] https://news.ycombinator.com/item?id=28167835

Re: Single cortical neurons as deep artificial neural networks

#132

Post by vlovich123 » Sat, Aug 14, 2021, 8:58 PM UTC

vlovich123 wrote: post #131

vlovich123 wrote: post #130

Earlier quoted context omitted.

That’s a solid counter argument. I assume you’re saying we don’t know of a single physical process that’s not a Turing machine because all of the models we have built to simulate them are Turing computable? That’s a strong point. Maybe I should readjust my prior on this. It does sound like you’re better versed in this topic. I like to learn by asking questions, so if you’re amenable to answering please do. If you don…

Oh, and I’m going to read [1] now. I thought the heading was timely and lovely since we were discussing this and I look forward to reading it (I haven’t yet so I do t know what the article says yet) [1] https://news.ycombinator.com/item?id=28167835

This tidbit was interesting:

> No method of computing carried out by a mechanical process can be more powerful than a Turing machine. Although widely adopted, as there is no clear way to prove or disprove its validity the proposition still remains a conjecture.

I think that’s what we’re basically discussing, right? Still, the way that’s phrased puts it into the P!=NP camp for me so I think you may be right.

Re: Single cortical neurons as deep artificial neural networks

#133

Post by stephencanon » Sun, Aug 15, 2021, 12:14 PM UTC

CodesInChaos wrote: post #119

MauranKilom wrote: post #107

Earlier quoted context omitted.

I assume that splicing together two decaying exponential functions at the origin would work just as well in practice. Also known as tanh: https://en.wikipedia.org/wiki/Hyperbolic_functions One "disadvantage" is that it doesn't saturate to [-1.0, 1.0] like appropriately scaled tanh.

By splicing together I mean a piecewise function which is `exp(x) - 1` on the left and `1 - exp(-x)` on the right. Which should be similar enough to tanh for most purposes.

Sure, it even has continuous derivatives of all orders and the right slope at the origin. It just doesn’t saturate to +/-1 as fast, which probably doesn’t matter.

Re: Single cortical neurons as deep artificial neural networks

#134

Post by runT1ME » Sun, Aug 15, 2021, 9:43 PM UTC

tsimionescu wrote: post #127

runT1ME wrote: post #124

Earlier quoted context omitted.

>Sure, that's possible, but the opposite is also possible: it might be wrong for most programs we actually write. We could confirm this though! It's not like we can't find out if a given program halts or is inconsisent. Godel talks about it in his letter to Von Nuemann.

There are programs for which we can check this, but there is no general procedure to check if any program halts. Even ignoring the halting problem itself, say we analyze a program and realize it halts iff P=NP, or pi to the e is transcendental, .or if Pi's decimal expansion at position Graham's number is divisible by 3. Will that program halt? It might be very hard to say. More promisingly, there are ways to construc…

We can write a general procedure to see it any given program halts within n-steps however.

Re: Single cortical neurons as deep artificial neural networks

#135

Post by greens » Tue, Aug 17, 2021, 3:58 AM UTC

zetalyrae wrote: post #44

greens wrote: post #42

Earlier quoted context omitted.

My 2c (apologies for the aggressive tone -- I'm just excited about AGI): That's a very very weak upper bound on how much hardware it takes. I think it's not all that different from emulating a Nintendo64 with a quantum simulation of the hardware. For complex systems to work (not to mention evolve), they need to be robust to small perturbations -- there's no way the computation the brain is doing is sensitive to the d…

DNA/RNA is near-atomic level storage. There's redundancy in that there are multiple instances of each neuron|receptor|etc.

That's a great point.

You could argue despite the huge number of physical degrees of freedom, the operations on DNA can be reduced to copy, repair, express, suppress. On the other hand, there's still a ton of intrinsic complexity in storing a huge amount of data, and yeah some nucleotides are totally essential.

The other thing a wonder about: sure, maintaining a proteome is hugely complicated, but how much of this complexity goes into maintaining homeostasis (e.g. metabolism, cytoskeloton and membrane maintenance, replication,...) vs. enabling computation. Seems like silicon has the advantage here.