Live data from Hacker News

There is no hard takeoff

geohot.github.io

141–150 of 190 posts

Re: There is no hard takeoff

#141
post #94

I will be worried when AI will constantly beat top1 league of legends team in normal environment Chess is way simpler imo.

AI already has beaten world champions in Dota 2 [1], I can't imagine League is that much more difficult so if someone wanted to build an AI agent to beat everyone else in League they probably could, just no one got around to actually doing it. [1] https://openai.com/research/openai-five-defeats-dota-2-world...

[deleted]

Re: There is no hard takeoff

#142

I will be worried when AI will constantly beat top1 league of legends team in normal environment Chess is way simpler imo.

Funny how the benchmarks for proving ”real intelligence“ change every time there is a breakthrough. * ”X would demonstrate real intelligence.“ * Some ML model M beats X. * ”Well, actually, M is not really intelligent. Y would prove real intelligence.“ * repeat

It changes because they ask different ppl.

Re: There is no hard takeoff

#143
post #106
post #71

"And yet, here we are in 2023 and self driving cars still don’t work." LOL. Someone should tell the California CPUC that: https://www.theverge.com/2023/8/10/23827790/waymo-cruise-cpu...

Under every weather, traffic and road condition?

"Working" is a matter of degree. They do work partially, so "they don't work" is partially false.

Re: There is no hard takeoff

#144
post #94

I will be worried when AI will constantly beat top1 league of legends team in normal environment Chess is way simpler imo.

AI already has beaten world champions in Dota 2 [1], I can't imagine League is that much more difficult so if someone wanted to build an AI agent to beat everyone else in League they probably could, just no one got around to actually doing it. [1] https://openai.com/research/openai-five-defeats-dota-2-world...

Once or consistently?

If they played 100 games, would AI have at least 55% wr?

Re: There is no hard takeoff

#145

This article reminds me a bit about a core theme in Isaac Asimov's Foundation series, which I'm coincidentally reading (mild spoiler warning). The science of psychohistory that is used to predict the future and create the Seldon Plan, only works if the future is allowed to unfold uninformed by psychohistory itself. That's why the Foundation was founded without any psychologists or knowledge of the Seldon plan. Seldon…

That's similar to what some faithful say when you criticize stock market technical analysis. That it only predicts when participants haven't seen the chart.

Re: There is no hard takeoff

#146

Earlier quoted context omitted.

"You don't need to worry about AI becoming a superintelligence because the universe is big and complex."

I think the point he is making is that competition will prevent others from being able to FOOM and 51% everyone else and "win" outright to the point that it is dangerous. Or competition will prevent one player from getting the "powder keg". Competition can do that if it is fair competition and a good game design. That is why rules and regulations are important in any game or it becomes Calvinball where the only rule…

> The flaw in the thinking is that there will be "fair" competition.

Why does what OP concludes require fairness? Wouldn't the same logic apply if every player defects, in game theory terms?

Re: There is no hard takeoff

#147

This article reminds me a bit about a core theme in Isaac Asimov's Foundation series, which I'm coincidentally reading (mild spoiler warning). The science of psychohistory that is used to predict the future and create the Seldon Plan, only works if the future is allowed to unfold uninformed by psychohistory itself. That's why the Foundation was founded without any psychologists or knowledge of the Seldon plan. Seldon…

That is a really good insight.

George Hotz was making a good similar point of modeling self contained systems vs. open systems - a very different thing.

Re: There is no hard takeoff

#148
post #93

Am I the only one who sees a jumbled stream of barely coherent semi-thoughts and word salad?

No, I also failed to understand what the writer attempted to bring across. I’d be very happy if someone could summarize the blog in one paragraph.

Systems in the real world exist in competition. The only way for one AI system to take over would be for it to overcome not just all humans but all rival AI systems. For one system to beat all other systems, it would need to be smarter than all of them put together. This means it would need over half the computing power of the entire world. Amassing physical resources is relatively slow and every AI would be trying to prevent one AI from taking over, so gaining 51% of compute would be hard. Algorithmic improvements are impossible (he actually says this), so there's no way one AI could suddenly jump in capability. Compute will be distributed in a power law (for some reason). Every AI will serve as a check on every other.

You may notice this is all nonsense. Somehow it's impossible to amass power without being smarter than everyone else put together despite humans doing it all the time. Does Hotz think Stalin had a brain the size of a skyscraper that he ordered airbrushed out of all photographs? Somehow Hotz can confidently predict how future technologies will work despite us having no idea what they'll look like even in principle. Somehow having gods fighting amongst themselves will work out just fine for the humans in the crossfire.

This "debate" is just people summarizing science fiction at each other. The score is one Atlas Shrugged and nine Star Wars to three Terminators and one I Have No Mouth and I Must Scream.

Re: There is no hard takeoff

#149
post #48

Earlier quoted context omitted.

> It just needs to get a bit smarter and somewhat faster every few months. Within less than a decade we will have systems that output "thoughts" and actions at least dozens of times faster than humans. I don't think this will work because cost of improvement with current method is exponential and we're already at capacity with HW.

It only needs to get somewhat smarter. The hardware, software and AI models will continue to be sped up. Also there are already many ways that improvements have been made without requiring exponentially more memory.

Since GPT4 I've seen it regress in attempts commercialise and any alternative I'm aware of isn't even close. Depends on the timescale are we talking about ?

Re: There is no hard takeoff

#150
post #79

FOOM here refers to the Yudkowsky-Hanson debate about hard takeoff (some other comments here are confused). https://www.lesswrong.com/tag/the-hanson-yudkowsky-ai-foom-d...

Your comment doesn't really help. Nothing there seems to explain the term "FOOM". More searching suggests it doesn't actually stand for anything and shouldn't be capitalised: https://en.wiktionary.org/wiki/foom#Noun . (Though ChatGPT seems happy to make up definitions: https://old.reddit.com/r/singularity/comments/12bisxu/what_d... )

FOOM is a different term for hard takeoff:

> RSI [recursive self-improvement] is the biggest, most interesting, hardest-to-analyze, sharpest break-with-the-past contributing to the notion of a "hard takeoff" aka "AI go FOOM", but it's nowhere near being the only such factor. The advent of human intelligence was a discontinuity with the past even without RSI...

https://www.lesswrong.com/posts/tjH8XPxAnr6JRbh7k/hard-takeo...

> A hard takeoff (or an AI going "FOOM") refers to AGI expansion in a matter of minutes, days, or months

https://www.lesswrong.com/tag/ai-takeoff

Post reply on HN