Live data from Hacker News

We need a Butlerian Jihad against AI

erikhoel.substack.com

21–30 of 161 posts

Re: We need a Butlerian Jihad against AI

#22

> To slim results. Elon Musk explained his new nihilism about the possibility of stopping AI advancement on the Joe Rogan podcast, when he said: “I tried to convince people to slow down. Slow down AI. To regulate AI. This was futile. I tried for years. Nobody listened. Nobody listened.” Rather rich coming from one of the self driving car market leaders. It certainly makes business sense to mislead academia and the po…

Elon wasn't calling for a hard halt like this article. He's talking about the dangers of general AI (AGI). I don't think most people would consider a self-driving car a likely path to accidental superintelligence; it's a highly-targeted application like a chess engine. OpenAI's GPT-X engines OTOH, IMO, have a lot more potential danger because it's very unclear what they'll be used for.

If you do actual ML research, the technologies are two sides of the same coin. Three days ago there was a paper posted here in HN on using the GPT-like transformer architecture for reinforcement learning problems (of which self driving cars is a partial subset of).

https://news.ycombinator.com/item?id=27721037

Re: We need a Butlerian Jihad against AI

#23

Yes. The inevitable defeatism that will show up in these comments is "Oh, but China will do it anyway, so there's no point." Which is pretty easily counterable: 1) We don't know if China would cooperate with a ban. We haven't tried. China is very complicated, and if you think you can predict what Xi will do, you are wrong. 2) If AI is truly a global existential and moral crisis, the US _could_ absolutely shut down Ch…

True, China will be really receptive to a ban! Maybe we could try and get Xi hooked on opiates and then blackmail him to stop!

It worked before! What could go wrong!?

Re: We need a Butlerian Jihad against AI

#24

Yes. The inevitable defeatism that will show up in these comments is "Oh, but China will do it anyway, so there's no point." Which is pretty easily counterable: 1) We don't know if China would cooperate with a ban. We haven't tried. China is very complicated, and if you think you can predict what Xi will do, you are wrong. 2) If AI is truly a global existential and moral crisis, the US _could_ absolutely shut down Ch…

> We don't know if China would cooperate with a ban. We haven't tried. China is very complicated, and if you think you can predict what Xi will do, you are wrong.

This is astonishingly optimistic (or naive). I'm sure China would happily agree to cease AI development in public and then continue in private.

Getting any country to actually stop AI development is as likely as getting them to give up all their nuclear weapons. AI is a weapon, both in military and economic contexts.

Re: We need a Butlerian Jihad against AI

#25

Yes. The inevitable defeatism that will show up in these comments is "Oh, but China will do it anyway, so there's no point." Which is pretty easily counterable: 1) We don't know if China would cooperate with a ban. We haven't tried. China is very complicated, and if you think you can predict what Xi will do, you are wrong. 2) If AI is truly a global existential and moral crisis, the US _could_ absolutely shut down Ch…

lol never heard of nukes or mutually assured destruction? Can’t even stop Iran or North Korea and you think China can be stopped?

That's exactly my point. I did, explicitly, say it was unpleasant, and something you should only do in extremis -- when you genuinely believe that extinction a likely alternative.

But the US could absolutely glass every semiconductor fab (and datacenter, and research facility) in mainland China using a combination of conventional + atomic kinetic options.

Re: We need a Butlerian Jihad against AI

#26

> To slim results. Elon Musk explained his new nihilism about the possibility of stopping AI advancement on the Joe Rogan podcast, when he said: “I tried to convince people to slow down. Slow down AI. To regulate AI. This was futile. I tried for years. Nobody listened. Nobody listened.” Rather rich coming from one of the self driving car market leaders. It certainly makes business sense to mislead academia and the po…

Right now if a vehicle on auto pilot gets into an accident, the driver is scrutinized.

That is reasonable. The driver is expected to be alert incase intervention is needed.

If we take the manual override away, then we'll be squarely in the world that Mr. Musk is concerned about.

Re: We need a Butlerian Jihad against AI

#27

We should be actively building this new "AI Species" because we are going to be extinct eventually and should think about making a better successor for the human species. The morality argument is nonsense. How about this: "The primary objective of humanity should be to build an intelligent system with far more precise perception, reasoning and physical manipulation capabilities than humans" That's my starting point.

There are all sorts of ways to build intelligences. Humans are unique in that they are mammals (defined by having mothers). Mothers raise us with love, and teach us, for our helpless first years. We also have to act in communities. So there is a sense in that we are very lucky - in humans, our intelligence correlates with our altruism. In the grand space of possible minds, it is very unlikely that altruism and morality is correlated with intelligence. So whatever that machine race we birth is, it won't have any of the things we value if we're just building for "precise perception, reasoning, etc"

Re: We need a Butlerian Jihad against AI

#28
Good read. Not buying it, but interesting. I suspect a broader definition of evolution includes constructing tools and adapting to them (do humans walk upright because doing so leaves hands free for rocks and sticks?). Our AI assistants may evolve - with help - into robot overlords or some other Luddite/dystopian scenario, but I doubt it. Mutual cooperation is beneficial, and much easier when not competing for the same resources (food, land, water, mates).

Re: We need a Butlerian Jihad against AI

#29
I think this is interesting fodder for science fiction authors but lacks concrete examples of what exactly it would mean to regulate or engage in a "Butlerian Jihad against AI."

I know things that I would like to see. Like humans "in the loop" (as opposed to "on the loop" or "out of the loop") for certain classes of decision making - for example target selection of military strikes or law enforcement. Or what kinds of information we use to train the decision making models, for example if you feed ML a racist data set and you get a racist algorithm - use that algorithm to decide who to give mortgages and you'll get systematic depression in generational wealth based on racial lines.

But this isn't some crusade on AI because it's AI; it has to be based in reality - what AI or ML is being used for, what information it operates on, what decisions it is used to make, and ultimately the human beings that are responsible for those decisions. The reason it is so hard to convince people as to how we should legislate (or otherwise regulate AI) is that every conversation drifts into science fiction and not concrete examples of the ethical issues today and what can be done today. Otherwise it comes off as Luddite fearmongering.

Post reply on HN