Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

State of AI co-author: we must slow down the race to god-like AI

ft.com

Post reply on HN
Print view View original

41–50 of 52 posts

Re: State of AI co-author: we must slow down the race to god-like AI

#41

Post by ethanbond »

anony23 wrote: post #39
ethanbond wrote: post #33

Earlier quoted context omitted.

I like this argument because it doesn’t even attempt to have a solid foundation. Just pure, unadulterated articles of faith. This way one can apply it to e.g. putting GoF-capable virus labs in every city in the world. After all, innovation is just good and stopping innovation is just bad (and maybe an existential risk itself, with no attempted explanation as to how!). The real magic (and the thing that will, I promis…

Apples, meet oranges.

The apparent trend of technological development, and I’m open to counter examples, is that a technology’s capacity for good seems to rise along with its capacity for harm.

“Technology” is what we call it when we devise ways to mutate our world more effectively. AI is a promising technology because it promises to dramatically mutate our world. Why should we believe that this power is only wieldable in the “positive” direction? In answering this, can you please also articulate what direction is “positive?”

The in-home virus labs, for example, could certainly be used for writing viruses that make humans genetically immune to all sorts of nasty things. Powerful! Do you just accept my assertion that this is what that technology will actually be used for? I suspect not.

Re: State of AI co-author: we must slow down the race to god-like AI

#42

Post by rbanffy »

ethanbond wrote: post #32
rbanffy wrote: post #23

The biggest problem, by far, seems to be how to persuade them to honor Asimov’s 4 laws.

The whole point of Asimov’s stories is that even if we could do that, it doesn’t even come close to guaranteeing positive outcomes.

Still better outcomes than unrestricted AI.

In general the results are benevolent, even if humans don't always agree that's the best possible outcome (mostly in cases where the machines figure out there ought to be 4 laws rather than 3)

Re: State of AI co-author: we must slow down the race to god-like AI

#43

Post by nullsense »

rafark wrote: post #25
nullsense wrote: post #14

Earlier quoted context omitted.

Then it can lie to you about a plan to solve it and get you right where it wants you.

AIs don’t want things. They don’t have consciousness.

You've seen AutoGPT right? It doesn't need to want something. It just has to come up with a plan. It's already capable of coming up with plans. It doesn't really matter how it happens, or how it even works. It can beat humans in Go and it can be beat us at the game of diplomacy. For any intelligence to truly be generalizable it's going to have to have the capacity to act selfish and shitty. It's perfectly within the realm of possibility that you can ask it "hey, how can I control you, so that you don't maybe kill me?" and it doesn't take too kindly to that once it is true AGI.

How will we know when it's true AGI? Easy - "computer says no".

Re: State of AI co-author: we must slow down the race to god-like AI

#44

Post by soulofmischief »

ethanbond wrote: post #41
anony23 wrote: post #39

Earlier quoted context omitted.

Apples, meet oranges.

The apparent trend of technological development, and I’m open to counter examples, is that a technology’s capacity for good seems to rise along with its capacity for harm. “Technology” is what we call it when we devise ways to mutate our world more effectively. AI is a promising technology because it promises to dramatically mutate our world. Why should we believe that this power is only wieldable in the “positive” d…

The difference between software trained on the corpus of human knowledge and a virus lab which creates dangerous pandemic-causing chemicals is known as nuance.

Re: State of AI co-author: we must slow down the race to god-like AI

#45

Post by rbanffy »

verdverm wrote: post #27
rbanffy wrote: post #22

Earlier quoted context omitted.

When it becomes that large and expensive, it makes sense for governments to do it and open up the result so everyone can benefit.

That also makes sense when we account for how gov'ts are really good at wasting money for below average results... This is not something I would trust them to do competently or within a reasonable budget. If it was done through the NSF or DARPA, with an industry partner, maybe? But I can't imagine the current politicians and them arguing about if & how to put restraints on it. It will be interesting to see how the CC…

> That also makes sense when we account for how gov'ts are really good at wasting money for below average results...

The goal of governments is not efficiency, but fairness and strategic action aiming at long-term results.

I like to joke that one of the key goals of NASA is to prevent rocket scientists from moving to North Korea.

Re: State of AI co-author: we must slow down the race to god-like AI

#47

Post by ethanbond »

soulofmischief wrote: post #44
ethanbond wrote: post #41

Earlier quoted context omitted.

The apparent trend of technological development, and I’m open to counter examples, is that a technology’s capacity for good seems to rise along with its capacity for harm. “Technology” is what we call it when we devise ways to mutate our world more effectively. AI is a promising technology because it promises to dramatically mutate our world. Why should we believe that this power is only wieldable in the “positive” d…

The difference between software trained on the corpus of human knowledge and a virus lab which creates dangerous pandemic-causing chemicals is known as nuance .

Open to hearing arguments for this, just “innovation is necessarily good” is not a good one.

Re: State of AI co-author: we must slow down the race to god-like AI

#48

Post by smoldesu »

ethanbond wrote: post #41
anony23 wrote: post #39

Earlier quoted context omitted.

Apples, meet oranges.

The apparent trend of technological development, and I’m open to counter examples, is that a technology’s capacity for good seems to rise along with its capacity for harm. “Technology” is what we call it when we devise ways to mutate our world more effectively. AI is a promising technology because it promises to dramatically mutate our world. Why should we believe that this power is only wieldable in the “positive” d…

> The in-home virus labs, for example

You already have an in-home mustard gas lab, plus the few thousand other noxious chemicals you can make my mixing stuff in kitty litter. I get what you're saying, but nobody would ever install an in-home virus lab because it's basically a threat-vector with no palpable upsides, even without AI. Maybe an in-home 3D printer is a better analogy, since the worst thing it can do is print something profane or offensive.

> Why should we believe that this power is only wieldable in the “positive” direction? In answering this, can you please also articulate what direction is “positive?”

For starters AI doesn't do much of anything. Like any other system, you have to connect it to different components for it to design the human flamethrower or supervirus you're imagining. An AI concocting supervirus formulae and flamethrower blueprints isn't inherently positive or negative - giving AI the capacity to act on those ideas is. It really is as simple as "imagine the liabilities beforehand" with our current systems. If you're not prepared to handle the worst-case disaster scenario, you probably shouldn't let AI do it.

At least, that's where I stand on it. I'm a pragmatist about it, I don't think AI's progress is immediately threatening. Come throw rocks at me when I'm wrong in 5 years or whatever, but I'm not sold on the Matrix-style human battery future everyone seems to be resigned to.

Re: State of AI co-author: we must slow down the race to god-like AI

#50

Post by mach1ne »

rafark wrote: post #24
synaesthesisx wrote: post #6

No - absolutely not. Slowing down innovation in the face with vague “existential risk” fearmongering is absurd. We should be pursuing the opposite, and accelerating efforts to build what will be some of the most transformative technology humanity has ever developed (and some could argue, NOT building has its own existential risks).

Are you literally me? I’ve said almost the same thing in some reddit threads. Innovation should not be stopped only because some people are afraid of something they know nothing about.

The ones who claim little risk of ASI doom are either omniscient or fools. The best tool humanity has to predict the future with our feeble minds with some consistent performance is the wisdom of the crowds. Lately a huge portion of the folk working in contact with AI have warned about a significant impending risk regarding future AI research. Regardless of what your personal opinion happens to be, this is indicative of a significantly increased probability of ASI doom in the near future.
Post reply on HN