There is no hard takeoff
geohot.github.io
There is no hard takeoff
1–10 of 190 posts
Re: There is no hard takeoff
#2The main issue with this argument that current AI models are extremely inefficient. Models evaluate all weights on every pass, LLMs recall their entirety of knowledge just to output a part of a word and then do it all again. Often a hand crafted algorithm can achieve what a neural network can in a small fraction of compute. There is likely a 1e4x-1e6x compute gap that can be achieved with the right algorithm, maybe even a 1e10x and once we reach self improvement that gap could close very fast. They say that the human brain has 30 PFLOPS of compute power and this is sometimes used as a reference for how much compute is needed for intelligence, but that completely misses the point: The human brain is extremely inefficient. Symbolic mathematics can be unreasonably [1] effective at making accurate predictions about the future with very little computation, but many of these solutions are inaccessible to evolution. Once AI learns to construct efficient mathematical models of the universe, the amount of computation available today in a PC could be enough to do things (good and bad) way beyond human ability.
[1]: https://en.wikipedia.org/wiki/The_Unreasonable_Effectiveness...
Re: There is no hard takeoff
#3The main issue with this argument that current AI models are extremely inefficient. Models evaluate all weights on every pass, LLMs recall their entirety of knowledge just to output a part of a word and then do it all again. Often a hand crafted algorithm can achieve what a neural network can in a small fraction of compute. There is likely a 1e4x-1e6x compute gap that can be achieved with the right algorithm, maybe e…
Before the first AI winter there was incredible effort in leveraging symbolical approach. It did deliver many useful things but far from hype. For example we still have no self-repairing systems, only specific fault-tolerant algoritms but by and large software is as fragile as ever. You can't avoid considering everything when trying to deal with unexpected data/issue. Only in hidsight you can then say "wow that was sooo inefficient".
Re: There is no hard takeoff
#4The main issue with this argument that current AI models are extremely inefficient. Models evaluate all weights on every pass, LLMs recall their entirety of knowledge just to output a part of a word and then do it all again. Often a hand crafted algorithm can achieve what a neural network can in a small fraction of compute. There is likely a 1e4x-1e6x compute gap that can be achieved with the right algorithm, maybe e…
> the amount of computation available today in a PC could be enough to do things (good and bad) way beyond human ability.
I am in the camp that this is possible and that humans have achieved it already. I believe things are going to get very weird when these techniques become mainstream. The GPU-cluster-bound algorithms are just a tiny stepping stone.
Do we start taking computers away from people? What do we do about this if it turns out the only barrier to entry is the software?
Re: There is no hard takeoff
#5I am all for AI research and integrating more AI use into society. Currently working on tools based on GPT. I think it has incredible potential to help humans.
But at the same time, I am sure that AI does not need to have a hard takeoff to be extremely dangerous. It just needs to get a bit smarter and somewhat faster every few months. Within less than a decade we will have systems that output "thoughts" and actions at least dozens of times faster than humans.
That will be dangerous if we aren't cautious. We should start thinking now about limiting the performance of AI hardware. The challenge is that increasing the speed is such a competitive advantage, it creates a race. That is a concern when you put it into a military context.
The CEO of Palantir has already called for a Manhattan Project for superintelligent AI weapons control systems.
Re: There is no hard takeoff
#6> Oh wait…every hedge fund bro is already doing this. And most of them aren’t billionaires. The problem is your model needs to include all the computers playing the market, and it also needs to include the other hedge fund bros themselves. This strategy only dominates if you have more compute than the whole market itself, which you don’t.
Best part of this article, honestly, I did not expected this, of-course one can say we just need more abstractions (how does brain/living things build them?), but this would be ignoring dynamical nature of such problems.
Re: There is no hard takeoff
#7> Back in 2014, Elon Musk referred to AI as summoning the demon. And it wasn’t hard to see that view. Soon, Go agents would beat top humans learning from self play. By the end of 2017, the same algorithm mastered Chess and Shogi. By 2020, it didn’t even need tons of calls to the simulator, and could play Atari too.
> AI looked scary. It looked like it was one FOOM away from self playing and becoming superhuman at the universe. And yet, here we are in 2023 and self driving cars still don’t work.
This is weird. I don't recall this at all. The mainstream press got a little kick out of chess and (to a lesser extent) Go AI turning over various humans at a few points over the years but it's only really burst into the mainstream recently. And where it did get any traction, in tech circles such as our own, response was enthusiastic but definitely more measured. Some were talking a bit about a kind of AI singularity way off into the future, but that was always a very distant and theoretical thing.
Re: There is no hard takeoff
#8I am all for AI research and integrating more AI use into society. Currently working on tools based on GPT. I think it has incredible potential to help humans. But at the same time, I am sure that AI does not need to have a hard takeoff to be extremely dangerous. It just needs to get a bit smarter and somewhat faster every few months. Within less than a decade we will have systems that output "thoughts" and actions a…
To see how “AI” can be dangerous just look how social media bubble/recommendation algorithms would radicalize people, even with much cruder ML. People tend to miss that it’s not about some model starting to “think” and act all sci-fi evil, just us humans applying powerful tech in irresponsible ways that we either don’t bother to assess due to lack of awareness or assess positively due to a conflict of interest (money, career, etc.) is already enough to cause trouble.
Re: There is no hard takeoff
#9This makes good points.
And it articulates an idea I’ve been having trouble getting down.
What if increasing intelligence is an exponential problem and the reason humans all have somewhat similar intelligence isn’t that we peaked at some level but that even vast additional intelligence just doesn’t get much more traction against the universe of problems.
Eg doubling your compute doesn’t get you many more cities in the traveling salesman problem.
Re: There is no hard takeoff
#10I am all for AI research and integrating more AI use into society. Currently working on tools based on GPT. I think it has incredible potential to help humans. But at the same time, I am sure that AI does not need to have a hard takeoff to be extremely dangerous. It just needs to get a bit smarter and somewhat faster every few months. Within less than a decade we will have systems that output "thoughts" and actions a…
> It just needs to get a bit smarter and somewhat faster every few months. Within less than a decade we will have systems that output "thoughts" and actions at least dozens of times faster than humans.
I don't think this will work because cost of improvement with current method is exponential and we're already at capacity with HW.