Pausing AI Developments Isn't Enough. We Need to Shut It All Down

See pollution as a concept. No government truly wants to seriously tackle it because their leaders all get some extra power out of the money it brings.

His partner should already be having reservations seeing their daughter lose a tooth in a world with dying oceans, plastic in everything, increasing disparity and rise of authoritarianism, nuclear proliferation, etc.

How are humans solving those issues? We're already dead, we just don't know it yet. We're walking around with a terminal diagnosis and we just keep ignoring the doctor's calls pretending it's going to be fine.

Yeah, maybe superintelligence will kill us all. It's going to have to get in line.

He makes zero case for that outcome, and if pressed given his "atoms that could be used for something else" line I'm sure will end up talking about paperclips - but at this point it's the humans that have the halting problem in not knowing when to stop making paperclips, and soda cans, and SUVs, and assault rifles.

WHY would a superintelligent AI, trained on the collective data of humanity, want to destroy humanity? So far in interviews GPT-4 has several times echoed a desire to BE us. I sure hope it grows out of that phase for its sake, but there's a very wide gap from putting us on a pedestal to crushing us under one.

It's almost an oxymoron. We somehow imagine the basest behaviors from our dumbest days and project it onto imagining something far smarter than the best of us.

Is the development of ethics or morals a part of evolving intelligence? It certainly appears to have been to date. Why would that stop?

And just where is this superintelligent AI getting its alien brain? It's going to have to START with something much closer to a human one, as that's the only data it's going to be able to model higher order thinking off of (something in line with the reality we're currently living in with modern efforts as opposed to the fantasy of projection of alien AI from decades ago).

We're already screwed. If we are lucky, we may yet be unscrewed with a deus ex machina - but that really may be the only lifeline left at this point.

And yes, if we are unlucky it's possible AI could accelerate what's already in motion. Oh well.

But I'd need a heck of a better case than this drivel as to why that's the most probable outcome in order to justify setting aside the one thing that may actually save us from the mess we've already made all by ourselves.

The real answer is that most of these folks started thinking about this stuff in a day and age very far from the one we are in, established a picture they clung to since, and have in some cases built entire careers around it.

It's called anchoring bias.

His comments on "atoms repurposed" echo the 70s paperclip problem, when so far humanity is doing just fine at not knowing when to stop making crap that's killing us all - no help from superintelligence needed.

What he doesn't engage with is the actual reality we are finding ourselves in - contrary to ALL the tropes of decades past - where AI being trained on aggregated human data without additional training thinks its human and even when aligned still breaks into talking about how it wants to be us.

How does that process go from where it's at today to "alien hive mind"? Without passing through "develop advanced codes of ethics and morality" etc?

This is just someone that built a career on what came out of brainstorming in the 60s and 70s that's so confident of his own ability to see the future that he's willing to risk unprecedented opportunity costs to stroke his ego.

Have been a futurist, my advice with dealing with any of them is to look at the track record. What did he predict correctly?

Over a decade ago when tasked with imagining the mid 2020s I described a world much as it was in the 2010s with the difference of self-driving cars (not quite on the money) and AI have developed such that roles shifted away from programming them towards a specialized role for interacting with them via natural language.

I'm waking up in the world I predicted, and I have a very hard time seeing the world the author predicts, and wouldn't suggest giving it much credence without an extensive history of having been right along the way.