Ten Percent of the Brain Myth

What makes you think we reached the edge of Moore's Law? It's often been pronounced dead, but it's still going strong. Especially if you consider that not just CPUs, but GPUs (and Systems-on-a-chip etc) can also be torch-bearers as far as the Law is concerned.

Even present day chips are already much faster than neurons. Oversimplified: our transistors work at (the same order of magnitude as) the speed of light; while neurons work at (the same order of magnitude as) the speed of sound.

As far as we can tell, we can just use more and more transistors in parallel, and will eventually get something that can probably run a mind much faster than a human brain can. I have no opinion on whether that's actually the case, but it seems a plausible enough assumption for a story.

> Just like the bad human trope, it ignores that most transistors are highly specialized (an FPU is not a GPU is not a general purpose CPU).

You are right for CPU-type workloads. But if your story take how present day GPUs run present day neural nets, and extrapolates from there, you could very plausibly have an architecture that runs general intelligence, uses most of its transistors most the time, but the hardware looks extremely specialised for one very specific type of computation.

(I suspect that it is battery-energy consumption and waste heat consideration that will make us not use most transistors powering our cyborgs most of the time.

We have already been seeing dynamic frequency scaling with CPUs for ages even on Desktop. Smartphones made these energy considerations even more of a priority.)

> We're talking about extremely arbitrary programs from stories that are being done for the first time ever. "Hacks" that no one has ever done before. "Holodeck simulations" built in magic amounts of time just on a couple of spoken sentences.

Doesn't sound magic at all to me. Today Dall-E 2 already produces awesome images from short descriptions, and GPT-3 can produce stories. Building a complete holodeck simulation is a straightforward extrapolation. (Yet again, I don't know whether in reality there will be a smooth road from today's technology. But it's more than plausible enough for fiction.)

The Halting Problem doesn't apply to these kinds of things at all. Dall-E 2 and GTP-3 aren't even Turing complete.

> [...] but I've also started to feel like most of them are written by people that have never faced the contemporary realities of computer programming [...]

That's most likely true for most authors, yes.

I'm sorry I'm about to drop a ton of pessimism here. I know HN loves to be a place of unfettered technical optimism, but this is a thread about myths in fiction and bringing hard reality to them, so it seems a useful exercise to apply some "glass is half empty" to it in this thread.

> What makes you think we reached the edge of Moore's Law? It's often been pronounced dead, but it's still going strong.

Moore's Law was formulated with specifics. It had specific goal posts. We passed those goal posts, so we moved them. You are right, we still believe that we can keep moving those goal posts. I characterized this as "we can see the horizon" (not that we've reached the current edges, but we have a decent idea now where they are), we can see that point in the (near?) future where we will eventually have moved the goal posts so far that we've either lost sight of the original goal line or we've lost sight of why we are still moving the goal posts at all.

> As far as we can tell, we can just use more and more transistors in parallel, and will eventually get something that can probably run a mind much faster than a human brain can.

Transistors still have to obey the laws of physics as we know them. The speed of light is incredibly fast, but it's still a very hard speed limit on everything we think to accomplish. Grace Hopper famously carried around wires to remind people the distance of a nanosecond. It's still shorter than you think it should be even knowing how large a light year is. Nanoseconds add up. You can use a lot of transistors in parallel but you still have to route data to and between them. Even if you are routing the data at light speed (and that's quite a feat), delays add up. There's only so much stuff you can route.

> Today Dall-E 2 already produces awesome images from short descriptions, and GPT-3 can produce stories.

Neural networks are still just Sparkling Statistics Models. They seem amazing to us humans, but in general we humans are terrible at understanding statistics and statistical reasoning. This is in large part why Casinos exist. No one expects AGI to result from Las Vegas running enough Casinos in the same place. But you swap the poker chips for pixels or stories, suddenly people expect a Casino to grow into AGI and that they can extrapolate how AGI will likely work based on Casinos that they have seen.

It's lovely optimism that AGI would look anything like today's neural networks.

But beyond that, do you have a real sense of how much wall clock time it takes for a model like Dall-E 2 or GPT-3? Even ignoring that each one was years and months of wall clock time to "train" in the first place, right now most people's interactions with these models aren't on their own hardware but on corporate-sponsored clouds of servers. Sure, it's easy to imagine a starship might have a whole cloud of servers (just for holodeck simulation alone, though?), but it's still useful to have a wall clock time appreciation for even these "simple Casinos" we've built:

Both Dall-E 2 and GPT-3 models are multiple gigabytes in size each. If you try to run one on a single device you will find out that it is not as "instantaneous" as it feels when borrowing a corporate sponsored cloud. It will take minutes just to load those full models into RAM and/or VRAM, presuming that you have enough free RAM/VRAM to accommodate the full models. (Otherwise you can expect additional wall clock time to page parts of the models in and out of RAM.) CPUs and GPUs are still amazingly fast at spinning a lot of roulette wheels, dealing a ton of poker hands, and rolling a lot of dice, but the staggering numbers of such still add up when running a model like this and it can take minutes to hours of wall clock time on a single device.

> The Halting Problem doesn't apply to these kinds of things at all. Dall-E 2 and GTP-3 aren't even Turing complete.

You don't think AGI would be at least NP Hard, much less Turing complete? The Halting Problem applies as much to NP Hard problems as Turing complete ones.

That gets back to why I brought up the videogames example. We know that "full fidelity" Newtonian physics simulation is at least NP Hard. A lot of videogame physics models is making simplifying assumptions that reduce things from "full fidelity Newtonian" to "videogame physics" that is hopefully something fun and preferably polynomial time that is easily time boxed to a reliable fraction of a frame's time so that updates are continuous with frames per second and frames per second are high enough to keep up an illusion of "real time".

It's incredible what we've been able to accomplish with "videogame physics". We should keep no illusion that "videogame physics" are not and will never be "full fidelity Newtonian physics" much less anything approaching "real world physics" simulation.

And physics simulation is just one at least NP Hard problem in a modern videogame design. Networking code has at least NP Hard problems. Rendering has at least NP Hard problems. Et cetera. They all have different mitigations/compromises/workarounds/fake-it-until-you-make-it responses to those problems, but that doesn't make the problems go away, it's just creative space for creative solutions.

Videogames have always been Halting Problem minefields and likely always will be. "Infinite" loops (an NP solution taking a deeply exponential amount of time is indistinguishable from an infinite loop, per the Halting Problem) and crashing bugs will forever be issues with videogames. Mitigating those will always be a challenge in videogame design.

You can't imagine that a "Holodeck simulation" has at least as many hard problems to (creatively) solve as today's videogames? I'm sure videogame designers would love it if polynomial time neural network model could solve videogame physics today.