I have no mouth, and I must scream
The "AI 2027" Scenario: How realistic is it?
211–220 of 242 posts
Re: The "AI 2027" Scenario: How realistic is it?
#212Some useful context from Scott Alexander's blog reveals that the authors don't actually believe the 2027 target: > Do we really think things will move this fast? Sort of no - between the beginning of the project last summer and the present, Daniel’s median for the intelligence explosion shifted from 2027 to 2028. We keep the scenario centered around 2027 because it’s still his modal prediction (and because it would b…
> They went from "this represents roughly our median guess" in the website to "maybe think of it as an 80th percentile version of the fast scenario that we don't feel safe ruling out" in followup discussions. His post also just reads like they think they're Hari Seldon (oh Daniel's modal prediction, whew, I was worried we were reading fanfic) while being horoscope-vague enough that almost any possible development wil…
> while being horoscope-vague enough that almost any possible development will fit into the "predictions" in the post for the next decade.
This is a recurring theme in rationalist blogs like Scott Alexander’s: They mix a lot of low-risk claims in with heavily hedged high-risk claims. The low risk claims (AI will continue to advance) inevitably come true and therefore the blog post looks mostly accurate in hindsight.
When reading the blog post in the current context the hedging goes mostly unnoticed because everyone clicked on the article for the main claim, not the hedging.
When reviewing blog posts from the past that didn’t age well, that hedging suddenly becomes the main thing their followers want you to see.
So in future discussions there are two outcomes: He’s always either right or “not entirely wrong”. Once you see it, it’s hard to unsee. Combine that with the almost parasocial relationship that some people develop with prominent figures in the rationalist sphere and there are a lot of echo chambers that, ironically, think they’re the only rational ones who see it like it really is.
Re: The "AI 2027" Scenario: How realistic is it?
#213Earlier quoted context omitted.
You cannot possibly be that uninformed. https://www.nytimes.com/2025/05/21/us/politics/qatar-plane-t...
[flagged]
> Who is Mullin?
A blowhard who shouldn't be in the Senate. https://www.cnbc.com/2023/11/14/stand-your-butt-up-gop-sen-m...
> When did Qatar start the process of gifting that jet from Boeing?
As far as anyone can determine, very recently. There's no evidence backing Mullin's transparent effort at redirecting blame.
https://www.cnn.com/2025/05/19/politics/trump-adminstration-...
"The Trump administration, however, wanted a replacement plane much faster, and the Air Force was exploring different options for doing so. At the same time, Trump tasked his Middle East envoy Steve Witkoff with finding a list of viable planes, a senior White House official told CNN."
Trump publicly bragged about being the one to decide to accept it, as well.
> Cancel your NYT sub.
I don't have one.
Re: The "AI 2027" Scenario: How realistic is it?
#214I'm skeptical. Where will the training data to go beyond human come from? Humans got to where they are from being embedded in the world. All of biological evolution from archaebacteria to humans was required to get to human. To go beyond human... how? How, without being embodied and trying things and learning? It's one thing to go where there are roads and another thing to go beyond that. I think a lot of the "foom"…
Evolution doesn't happen by "trying things and learning." It happens by random mutation and surviving (if the mutation confers an advantage) or not (if the mutation is harmful). An AI could do this of course, by randomly altering some copies of itself, and keeping them if they are better or discarding them if they are not.
What you just described -- random mutation and survival -- is the process whereby learning occurs. Over time this process transfers information about "how to survive" into the genome and other mediums of heritability.
AI could do this too, but that just means it's using a different learning algorithm. There is a whole field called genetic programming that uses evolution-inspired rather than nervous-system-inspired models and has had success in areas like physical materials engineering, circuit board layout, etc.
It doesn't change the fact that you need more information to go beyond where you are -- I do not believe you can reason from a void into higher levels of... what?
Re: The "AI 2027" Scenario: How realistic is it?
#215I think the big thing that people never mention is, where will these evil AIs escape to ? Another huge data center with squillions of GPUs and coolers and all the rest is the only option. It's not like it is going to be in our TV remotes or floating about in the air. They need huge compute, so I think the risk of an escaping AI is basically very close to zero, and if we have a "rogue" AI we can literally pull the plu…
it literally just needs to throw a few kilobytes on literally any internet connected device, so that it can eventually get to a computer with a single good GPU, from which it can then start planning and executing its way to infect its way into datacenters and run using some percentage of its resources while staying undetected
it can also just threaten/blackmail/bribe a single human to try to carry this out on its behalf, and possibly even a human that manages these big data centers
Re: The "AI 2027" Scenario: How realistic is it?
#216Earlier quoted context omitted.
This is how I feel about Neuralink and pretty much any form of transhumanism as well as humans traveling through space in ships. It's like imagining flying horses (and flying cars, for that matter).
Hey, neuroscientist here. Just FYI, Neuralink is doing some good stuff around implantation surgeries, but that's really about it. They haven't solved the main issue of glial scarring or how to feed information into tissue long term (at least as far as I know). As far as the neuroscience field is concerned, no-one is able to solve those issues, and we're ~60% sure you can't solve them, ever. My crazy pet theory is to…
Yeah, no ill intent towards Neuralink was meant.
It was about the sheer intractability of trying to upgrade extremely legacy wetware signaling at 100m/s to a level anywhere near the speed of light which modern inorganic stuff operates at. Even if we manage to solve the incredibly hard physical problems you mention, the mismatch on the signaling level is still 6 orders of magnitude. Compared to that challenge digitally recreating an entire human brain is almost trivial. Digitally creating something similarly potent is far easier still.
People use those images of the future as proof that we can't reliably predict it, but a lot of them were made by people who didn't understand physics, psychology, economics, etc. very well or just didn't care; they were just making a pretty, evocative picture. The reality however is that we can predict and have predicted aspects of many things based on what the limits of this universe are.
I would argue for instance that the speed of light being a maximum is directly responsible for the convergent (biological) evolution of eyes.
Re: The "AI 2027" Scenario: How realistic is it?
#217Earlier quoted context omitted.
Yes, that's the hypocrisy. Abortion: "Let the states regulate! States' rights! Small government! (Because we know we'll get our way in a lot of them.)" AI: "Don't let the states regulate! All hail the Feds! (Because we know we won't get our way if they do.)"
I agree that the policy approach is inconsistent with regards to states' rights. I was simply pointing out that your statement about the effects of Dobbs was false.
Not my statement.
Re: The "AI 2027" Scenario: How realistic is it?
#218Re: The "AI 2027" Scenario: How realistic is it?
#219Earlier quoted context omitted.
You can perfectly try things and learn without being embodied. The analogy to how humans learn only goes so far, it's myopic to think anything else is impossible. It's already happening. The situation today is any benchmark you come up with has a good chance of being saturated within the year. Benchmarks can be used directly to build series of exercises to learn from. And they do learn. Gradient descend doesn't care…
> You can perfectly try things and learn without being embodied. A brilliant 'brain in a vat' can come up with novel answers to questions but outside of narrow categories like pure mathematics and logic which can be internally validated, the brain can't know how correct or incorrect its novel answers are without some way to objectively test, observe and validate the correctness of its answers in the relevant domain (…
That was better said in some ways than my own comment.
Re: The "AI 2027" Scenario: How realistic is it?
#220Earlier quoted context omitted.
> You can perfectly try things and learn without being embodied. A brilliant 'brain in a vat' can come up with novel answers to questions but outside of narrow categories like pure mathematics and logic which can be internally validated, the brain can't know how correct or incorrect its novel answers are without some way to objectively test, observe and validate the correctness of its answers in the relevant domain (…
That was better said in some ways than my own comment.
Thanks! And I just saw in a parallel post currently on the HN home page, John Carmack also said a similar thing in his lecture notes.
> "offline training can bootstrap itself off into a coherent fantasy untested by reality."
"a coherent fantasy untested by reality" is a lovely turn of phrase.