How deep is the brain? The shallow brain hypothesis

if aliens had come down and given us laptops, rather we invented digital machines, then likewise i'd be talking about the relevant materials science, physics etc.

reverse engineering a laptop to figure out how it works would require extremely little computer science, and 'only at the end'

the reason digital computers are interesting and useful is that they route electricity around devices which are designed to be responsive to one another. the patterns of activation, as managed by the CPU, are weakly describable by abstract algorithms like sorting

starting with a laptop, and no further information, we'd be 100(s)+ years of research away from needing to understand that CPUs were implementing a sorting algorithm

and importantly, that it is doing so has almost nothing to do with the value of the device -- which lies in its ability to provide 'dynamical power and modulation of operation' using electricity

we're in the same situation with animals and people think that, what, understanding gradient descent or backprop is helpful? this is just some csci bs

It's perhaps not terribly surprising that it becomes possible with unlimited width or depth (or an arbitrarily complex activation function).

https://en.wikipedia.org/wiki/Universal_approximation_theore...

But that's the thing: they didn't. Instead, they called them "neural networks". It wasn't random.

It feels like part of the field now wants to pretend it was never about how to make a machine think. "No, we're only doing abstract maths, only going on self-contained explorations of CS theory." Yeah, right. That feels like a reaction to the new wave of AI hype in business. Now that the rubes are talking about thinking machines again, better distance themselves from them, lest we be confused for those loonies.

Thing is, the field was always driven in big part by trying to catch up with nature. It took inspiration from neuroscience, much like neuroscience borrowed some language from CS, both for legitimate reasons. A brain is a computer. It's precisely where the CS and neuroscience have an overlap - they're studying the same thing, just from opposite directions. It's just silly to play the "oh my field is better and your field doesn't know shit" game.

> Decision trees are called 'trees' for, more or less, the same reason.

Decision trees are called after the data structure, which is a way to express a mathematical object, which is older than CS and got that name from... who knows, but my money is on "genealogical tree", which itself is called a "tree" because people back then liked to tie everything to trees (symbol of growth) and flowers and cute animals (symbols of making babies).

The field inherited "trees" from the past. "Networks", too. But "neural" - that was a modern analogy the field itself is responsible for.

Sure there is. People had a feel for it back in "clockworks" times, nowadays we have a much better grasp because of progress of physics and math, particularly CS - mode of operation is an implementation detail. Whatever the mode, once you understand the behavior enough to model it in computational terms, you can implement it in anything you like - gears and levers, pistons, water flowing between buckets, electrons in silicon, photons going through lenses, photons diffusing through metamaterials, sound waves diffusing through metamaterials - and yes, also via a person locked in a room full of books telling them what to draw in response to a drawing they receive, and also via a billion kids following a game to the letter, via corporate bureaucracy, via board game rules, etc.

Substrate. Does. Not. Matter.

The only thing limiting your choice here is practical one. Humanity is getting a good mileage out of electrons in silicon, so that's the way to go for now. Gears would work too, they're just too annoying to handle at scale.

Of course, today we don't have a full understanding of biological substrate - we can't model it fully in terms of computation, because it's a piece of spontaneously evolved nanotech and we barely begun being able to observe things at those scales. We have a lot of studying in front of us - but this is about learning how the gooey stuff ticks, what does it compute and how. But it's not about some new dimension of computation.