Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pause Giant AI Experiments: An Open Letter

futureoflife.org

Post reply on HN
Print view View original

161–170 of 1001 posts

Re: Pause Giant AI Experiments: An Open Letter

#161

Post by jimrandomh »

andrewmutz wrote: post #114
6gvONxR4sf7o wrote: post #67

I'm really disappointed to see the HN reaction of straight up scoffing at this like it's irreversible and so let's just step on the gas. It's some weird combination of techno-utopianism and nihilism and good old elitism.

You can accuse everyone of *isms, but there are a lot of us who have just never been persuaded by the many articles written to scare us about the effects of AI. The concerns about all the jobs going away have been persistent but the labor market continues to be extremely strong. As long as entrepreneurship is creating new business opportunities and new ways to turn labor into value, it's completely possible that the…

There are two, mostly disjoint groups warning about AI. There are the people worried about comparatively mundane effects from comparatively mundane systems: job loss, spam, disinformation, maybe an occasional unfair loan-application rejection. These concerns don't have nothing going for them, but in all but the worst-case-scenario versions, these just aren't bad enough to make AI not be worth it.

Then there's the people looking ahead, foreseeing a future where superintelligent AIs are more powerful than humanity, and worried that most possible variations of those superintelligences are incentivized to destroy us.

I think this open letter puts much too much emphasis on the petty stuff, I think because they're trying to appeal to people who are allergic to anything that requires extrapolating more than a little bit into the future. But buying more time for alignment research, before we tackle superintelligence, does meaningfully improve humanity's odds of survival, so I hope this happens anyways.

Re: Pause Giant AI Experiments: An Open Letter

#162

Post by jimrandomh »

BobbyJo wrote: post #145
6gvONxR4sf7o wrote: post #67

I'm really disappointed to see the HN reaction of straight up scoffing at this like it's irreversible and so let's just step on the gas. It's some weird combination of techno-utopianism and nihilism and good old elitism.

I personally scoff at this kind of stuff because it's asking for something no one has the power to give. It's like asking to stop the development of nukes in the 40s and 50s, it's just not gonna happen.

Preventing the development of nukes entirely was obviously not going to happen. But delaying the first detonations by a few years, and moving the Partial Nuclear Test Ban treaty up a few years, was quite achievable.

Whether delaying AI development a little matters depends on whether you think the success of AI alignment, applied to future superintelligence, is overdetermined to succeed, overdetermined to fail, or close to borderline. Personally I think it looks borderline, so I'm glad to see things like this.

Re: Pause Giant AI Experiments: An Open Letter

#163

Post by famouswaffles »

versteegen wrote: post #140
twblalock wrote: post #108

Earlier quoted context omitted.

It really is irreversible. The methods and compute needed to build a GPT-type AI are widely available.

Yes, but superintelligence hasn't been created yet. That's what the petition is ultimately about, not chat assistants. Transformers unmodified aren't going to scale up to that, they're already nearly scaled maximally.

You don't know what transformers unmodified scale up to. Nearly scaled maximally ? Based on what exactly ?

Re: Pause Giant AI Experiments: An Open Letter

#164

Post by HybridCurve »

petilon wrote: post #81
musicale wrote: post #76

Earlier quoted context omitted.

> AI & robots will generate wealth at unprecedented scale, and humans will reap the benefits, the same way Alaskans reap the benefits of their natural resources. Vast wealth will certainly be delivered to owners of the robots and AI systems. What reason is there to believe that they would share the wealth with ordinary citizens? A typical alternative to the Alaska model is one where oil companies extract all of the w…

A lot of people are already benefiting from ChatGPT. I know I am, and I am only paying $20 a month for a digital assistant that is expert in everything.

It is an expert in nothing. Try asking it for general information and then ask it to produce citations for said information. It's a language model which is good at synthesizing certain information based on it's learned probability distributions. It has superficial knowledge at best and all information it produces must be cross checked for accuracy.

Yes, it can be incredibly useful but it literally has no comprehension of anything.

Re: Pause Giant AI Experiments: An Open Letter

#165

Post by famouswaffles »

jimrandomh wrote: post #146

For a long time, "AI alignment" was a purely theoretical field, making very slow progress of questionable relevance, due to lack of anything interesting to experiment on. Now, we have things to experiment on, and the field is exploding, and we're finally learning things about how to align these systems. But not fast enough. I really don't want to overstate the capabilities of current-generation AI systems; they're no…

i don't if alignment has any where to go. any more than you can align people for instance. the idea is just kind of ridiculous in the first place.

Re: Pause Giant AI Experiments: An Open Letter

#166

Post by nopinsight »

juancn wrote: post #156

Stopping now would be extremely dangerous and borderline stupid. If you stop now, you're just left behind, because there's no way everyone will stop. At this point the only logical course of action in an adversarial situation is to double down and keep researching, otherwise some other country or culture with different (and possibly worse) values ends up dominating the technology and you're left behind in the dust. T…

They are calling for a pause, not a stop.

It’s quite clear that OpenAI has a significant lead over everyone else. The only other country outside the west that even has a chance at developing something better than GPT-4 soon is China. China has a pretty cautious culture as well so it’s quite possible that a bilateral moratorium can be negotiated with them.

ADDED: Even without considering X-risks, China’s rulers cannot be pleased with the job displacement risks that GPT-4 plus Plugins may cause, not to mention a more powerful model.

They have trained a huge number of college graduates and even now there are significant unemployment/underemployment issues among them.

ADDED 2: If you think many companies can do it, please identify a single company outside the US/UK/China with the capability to train an equivalent of GPT-3.5 from scratch.

Re: Pause Giant AI Experiments: An Open Letter

#167

Post by famouswaffles »

nopinsight wrote: post #166
juancn wrote: post #156

Stopping now would be extremely dangerous and borderline stupid. If you stop now, you're just left behind, because there's no way everyone will stop. At this point the only logical course of action in an adversarial situation is to double down and keep researching, otherwise some other country or culture with different (and possibly worse) values ends up dominating the technology and you're left behind in the dust. T…

They are calling for a pause, not a stop. It’s quite clear that OpenAI has a significant lead over everyone else. The only other country outside the west that even has a chance at developing something better than GPT-4 soon is China. China has a pretty cautious culture as well so it’s quite possible that a bilateral moratorium can be negotiated with them. ADDED: Even without considering X-risks, China’s rulers cannot…

If you counted research, open ai didn't have a lead until gpt-4 nevermind a significant one. most of this is scale. their lead is a few months tops.

Re: Pause Giant AI Experiments: An Open Letter

#168

Post by thomasahle »

pdonis wrote: post #133
thomasahle wrote: post #86

Earlier quoted context omitted.

> Because of productivity gains humans will work very little, and yet live comfortably. Wouldn't this imply humans should already be working very little compared to pre industrial revolution? Instead it seems we work more [1]. And with less fulfilling work. I agree however that we have may have more stuff (or wealth). So going by this projection, in the future we'll have even more stuff, and work even more in even le…

> Wouldn't this imply humans should already be working very little compared to pre industrial revolution? No, because human wants aren't fixed. When you give a human everything they want now , tomorrow they come back with a whole new set of wants. And so on ad infinitum... Of course, as an individual it's your choice whether, and for how long, you stay on this merry-go-round. But there will always be a significant nu…

>>> Because of productivity gains humans will work very little, and yet live comfortably.

>> Wouldn't this imply humans should already be working very little compared to pre industrial revolution?

> No, because human wants aren't fixed. When you give a human everything they want now, tomorrow they come back with a whole new set of wants

It sounds like you are disagreeing with the OP, not me?

Or do you think AI productivity will make us work less in a way Industrial productivity didn't?

Re: Pause Giant AI Experiments: An Open Letter

#170

Post by tambourine_man »

A few decades ago human cloning was irreversible, inevitable, the cat was out of the bag and you couldn’t put the genie back in the bottle.

Yet we managed to legislate that. We need a change of atitute towards AI, recognize its danger and legislate accordingly.

Post reply on HN