Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We need a Butlerian Jihad against AI

erikhoel.substack.com

Post reply on HN
Print view View original

91–100 of 161 posts

Re: We need a Butlerian Jihad against AI

#91

Post by IggleSniggle »

Manuel_D wrote: post #42

I really doubt we will have the capability of building "a machine in the likeness of a human mind" in my lifetime. Present AI systems are essentially just function fitting. Building big probabilistic systems that we optimize with loads of training data. This is a far, far cry from the "strong AI" that people are so afraid of. I really think that people writing these sorts of pieces have an understanding of AI that's…

For those that study the human brain, my understanding is that it kind of already is “just form fitting.” Ie, there are subsystems that just do basic stuff, and together each of these systems eventually add up to a prediction system (consciousness) that is in service to the rest of the brain, which in turn has a mutualistic relationship with the rest of the body.

In this sense, “strong AI” already exists, with or without computers, as a global ecosystem driving towards…nothing in particular. Insomuch as computer systems augment the ability of humanity to continue to reproduce long-term in a mutualistic way, those computer system will themselves survive.

You might say, “but why wouldn’t the computer find a more efficient path, foregoing humanity?” Well, humanity supports the infrastructure for computers, and computers support the humans supporting the infrastructure, then this is already local minima. Arriving at a different reality for “strong AI” will be the result of a random walk tending towards components that are capable of existing in the long term, aka “evolution.”

Re: We need a Butlerian Jihad against AI

#92

Post by rjeidhwkn »

duped wrote: post #29

I think this is interesting fodder for science fiction authors but lacks concrete examples of what exactly it would mean to regulate or engage in a "Butlerian Jihad against AI." I know things that I would like to see. Like humans "in the loop" (as opposed to "on the loop" or "out of the loop") for certain classes of decision making - for example target selection of military strikes or law enforcement. Or what kinds o…

Ban all AI and machine learning research. Death penalty mandatory minimum for breaking. Full nuclear elimination strike against all nations that do not also enforce it.

There's your Butlerian Jihad.

I would press that button now, a thousand times a thousand times.

You lack the vision to even understand the magnitude of the threat or you wouldn't be asking the nonsense questions you do

Re: We need a Butlerian Jihad against AI

#94

Post by pmoriarty »

Manuel_D wrote: post #42

I really doubt we will have the capability of building "a machine in the likeness of a human mind" in my lifetime. Present AI systems are essentially just function fitting. Building big probabilistic systems that we optimize with loads of training data. This is a far, far cry from the "strong AI" that people are so afraid of. I really think that people writing these sorts of pieces have an understanding of AI that's…

"I really doubt we will have the capability of building "a machine in the likeness of a human mind" in my lifetime."

It's really hard to predict the future. Look at what a horrible job most people did a hundred years ago (even 50 years ago) predicting what life was going to be like today.

Many people did not believe man would ever walk on the moon (even right up to the time it happened), same with desegregation and the fall of the Soviet Union, for starters.

Science and technology are especially hard to predict, as many advances are a result of accidents and surprising discoveries.

I wouldn't write off strong AI, though I'm not sure it'll happen as a mimicry of the human mind.

Re: We need a Butlerian Jihad against AI

#95

Post by sammalloy »

Rzor wrote: post #8

I think the cat is out of the bag when it comes to AI and its potential, and you simply can't regulate and trust foreign nations to play ball. The possible gains are too big to expect everyone to get together and consider the downside carefully, i.e no coordination on the morality or risks; winner takes all will be the prevalent mindset when the first player hits major strides. It's going to be an arms race sort of s…

> I can't help imagine how much fun would be to have a Mind taking care of a few things for us

Near-Earth object detection and deterrence would be at the top of my list, followed by climate change mitigation and environmental protection. The use of autonomous drones to monitor poaching and illegal forest clearing may soon be a reality.

Re: We need a Butlerian Jihad against AI

#96

Post by adventured »

Tenoke wrote: post #87
Manuel_D wrote: post #42

I really doubt we will have the capability of building "a machine in the likeness of a human mind" in my lifetime. Present AI systems are essentially just function fitting. Building big probabilistic systems that we optimize with loads of training data. This is a far, far cry from the "strong AI" that people are so afraid of. I really think that people writing these sorts of pieces have an understanding of AI that's…

I dont really get how people see the progress of the last 10 or even 5 years and then imply that things won't improve all that much further than that for a whole generation.

It's just the pattern of people camping into one or the other extreme, which tends to occur with every topic. Nuance and depth is mentally expensive, whereas extremes are trivially easy by comparison and super fast (because you can discard so much explanation, discussion, thought and reality-binding once you decide to pursue fantasy).

It's identical to: AI sucks vs AI is amazing. Well, no, it's a complicated subject and always will be. Those extremes snippets are worthless bumpersticker-thought.

AI will very rapidly conquer all, becoming a practical Earth deity in the span of decades. AI will disappoint perpetually, progressing at a snail's pace for decades to come.

The truth is dramatically more likely to be inbetween the extremes, and going there intellectually doesn't lend itself very well to short comments.

Re: We need a Butlerian Jihad against AI

#97

Post by IggleSniggle »

z5h wrote: post #81

I read “A Thousand Brains: A New Theory of Intelligence” by Jeff Hawkins, and it now is clear to me that our neocortex (like computers, or AI) is just a lot of general purpose computing infrastructure with ZERO aims. Our emotions (which drive everything in the interest of gene propagation- as there is no purely logical reason to do anything) would need to be intentionally duplicated to give AI a reason to desire anyt…

I agree with you completely. This whole question of artificial intelligence presumes that intelligence is even a thing, and not just some form fitting for predictive reasoning that happens to be closely aligned to the well-being of the larger organism (human society).

The equivalent in silicon is functions fitting whatever purpose, but not strongly, for no particular reason except that they have been able to exist within that context across many, many, years.

The efficiency of a function may not be efficient, but it hasn’t been selected against…yet. That “DNA” source code might include a lot of cruft, but who cares as long as it works and people will download it anyway?

Re: We need a Butlerian Jihad against AI

#98

Post by jhbadger »

devindotcom wrote: post #17

Regardless of the merit of this particular piece, the portion of Butler's Erewhon comprising "The Book of the Machines" makes for very interesting and forward thinking reading. You can skip directly to it here: https://www.gutenberg.org/files/1906/1906-h/1906-h.htm#chap2... But the context in the story is fairly important - it takes place in a society that has essentially already carried out its own Butlerian Jihad a…

Butler also wrote an interesting essay in 1863 called "Darwin Among the Machines" (the title which was later borrowed for a book by George Dyson). Butler was probably the first person to realize that natural selection could apply to machines as well as biological organisms

https://web.archive.org/web/20060524131242/http://www.nzetc....

Re: We need a Butlerian Jihad against AI

#99

Post by benlivengood »

ALittleLight wrote: post #57

There's an interesting thought experiment referenced in this article, but I'm not sure it holds. "The philosopher John Searle made precisely this argument about the standard conception of rationality. His point was that there are no odds that would rationally allow a parent to bet the life of their child for a quarter. Human nature just doesn’t work that way, and it shouldn’t work that way." I agree that it sounds mo…

I think it's advisable to make decisions with reflection on actuarial tables and expected utility. Basically any activity whose likelihood of death or injury is below the base rate of same for the population in question is "safe" compared to more risky activities. For the risky activities mitigations can be taken (safer vehicles, bicycle helmets, vaccines, masks, education, supervision, etc.) to make the risk more acceptable. Not all mitigations are worth taking, and some extra mitigations are worthwhile; safety does not equal happiness, and everyone has their own risk profile and risk tolerance.

I think what happens when people pretend that 'normal' activities have zero risk is that they become both fearful and complacent from cognitive dissonance, e.g. refusing to wear masks but too afraid to get a vaccine. This is the degenerate situation that results from people not being comfortable with reasoning about probability and expected outcomes in everyday decisions, in large part from lack of practice.

Post reply on HN