AI 2040 and the cult of intelligence

I have some sympathy to geohot's view when it comes to pure informational chatbots. It's a first amendment issue, I'm allowed to write and read books that are useful to getting away with crimes, etc.

This obviously doesn't work at all when the agents start doing real things in the real world, though. "Hey AI, I don't like my neighbor, find an exploit in the firmware for his car and make the cruise control malfunction and crash him next time he gets on the highway". This is committing a crime, not just talking about theoretical crimes. The AI can and should refuse it.

I think he's anticipating and discarding this objection with his introduction, which otherwise feels disconnected from the rest of the article. FWIW, I have changed a bike tire and I'm pretty sure most of the MTS at the big labs could. This sort of "they're just bookworms who don't understand the physical world" rhetoric aside, we are currently seeing a ton of effort and expense go towards giving the AI agents hooks into being able to perform as many real-world-consequential actions as possible. And you can do a surprising amount with just bits, from writing code to breaking into systems to sending some combination of emails, phone calls, and currency to instruct meatspace humans to do things, etc.

Besides the topic of freedom, we know how seemingly “good” companies (e.g., google) turned out to be: billion machine monsters that couldn’t care less about humanity and that have huge amounts of power… and yet, we vouch for Anthropic/openai to become the next giants because “progress”. Face palm.

> Seems like being directionally right early that AI would be a Big Deal and scary should count for something?

If I scream that a vicious beast is going to destroy the world, I don't get credit for being 'directionally right' if a squirrel eats a hazelnut. AI being a big deal is a long, long way from the full beliefs Yudkowksy promotes, and there are many people who predicted AI's significance who didn't also believe it would be an extinction event.

> Though if there are no websites anymore or ones not many visit how would AI stay relevant?

That's what the robots are for.

I am not expecting the humanoid robots to meet the hype for a long long time*, but even boring industrial robots (CNC machines), even boring commercial robots (vending machines), even boring household robots (lawn roombas) have made incremental changes even though we don't yet have an AI good enough to be general purpose over them all.

* for power-envelope reasons alone there should be a ten year gap between "self driving car of quality X" and "humanoid robot which can get into normal car and drive it at quality X".

Where is "Plan R" for refusal or resistance. Life was kind of nice before "AI".

I'm pretty sure he is in some database now if he really used that ChatGPT prompt. At least they didn't swat him, but that feature will probably be implemented in a couple of years.