Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pause Giant AI Experiments: An Open Letter

futureoflife.org

Post reply on HN
Print view View original

181–190 of 1001 posts

Re: Pause Giant AI Experiments: An Open Letter

#181

Post by zarzavat »

credit_guy wrote: post #177
nopinsight wrote: post #166

Earlier quoted context omitted.

They are calling for a pause, not a stop. It’s quite clear that OpenAI has a significant lead over everyone else. The only other country outside the west that even has a chance at developing something better than GPT-4 soon is China. China has a pretty cautious culture as well so it’s quite possible that a bilateral moratorium can be negotiated with them. ADDED: Even without considering X-risks, China’s rulers cannot…

> The only other country OpenAI is not a country, it's a company. GPT models on par with GPT-4 can be trained, well, by companies. You don't need nation-state levels of resources.

Training a LLM with GPT-4 like capabilities is very hard. Most AI researchers are concentrated in a few countries. At the moment the countries with the vast majority of the expertise are US, UK and China.

Re: Pause Giant AI Experiments: An Open Letter

#182

Post by paxys »

So every company should cease all AI development for an indefinite period of time until someone has done the research and deemed it safe. And who exactly will that someone be? Future of Life Institute (whoever they are)? Elon Musk, who has very conveniently signed the letter after his public beef with OpenAI? Your friendly local government that has no agenda whatsoever? The UN?

Efforts like these are at best naive, and at worst agenda driven by players who simply want to insert themselves into and control the conversation.

Re: Pause Giant AI Experiments: An Open Letter

#183

Post by zarzavat »

JohnFen wrote: post #75
petilon wrote: post #68

> Should we automate away all the jobs, including the fulfilling ones? You don't necessarily need jobs because you don't need money. What you need is wealth. I am going to leave it to PG to explain the difference [1]: Wealth is not money. Wealth is stuff we want: food, clothes, houses, cars, gadgets, travel to interesting places, and so on. You can have wealth without having money. If you had a magic machine that cou…

> The industrial revolution didn't make human lives miserable, it made it better. I think this is a point that is genuinely debatable. At least, it's pretty easy to argue both sides of that proposition.

And the comparison with the industrial revolution is a poor analogy because the industrial revolution never proposed as a goal to automate humanity in totality, merely to automate human drudgery.

AI is an anti-industrial revolution, because it proposes to start with the most fulfilling aspects of human existence: writing, the arts, and to automate people engaged in those jobs, and to only later come to automate human physical labour. It’s a regression.

Re: Pause Giant AI Experiments: An Open Letter

#184

Post by ren_engineer »

juancn wrote: post #156

Stopping now would be extremely dangerous and borderline stupid. If you stop now, you're just left behind, because there's no way everyone will stop. At this point the only logical course of action in an adversarial situation is to double down and keep researching, otherwise some other country or culture with different (and possibly worse) values ends up dominating the technology and you're left behind in the dust. T…

this is basically the same logic behind nuclear weapons, and AI could potentially be even more dangerous if it kept advancing at the rate we've seen in the last few years. In theory the massive amount of compute needed to train and run these at scale could be tracked/regulated similarly to how nuclear refinement facilities are

your suggestion is that stopping nuclear proliferation shouldn't have even been attempted, despite the fact it actually worked pretty well

Re: Pause Giant AI Experiments: An Open Letter

#185

Post by ren_engineer »

nopinsight wrote: post #166
juancn wrote: post #156

Stopping now would be extremely dangerous and borderline stupid. If you stop now, you're just left behind, because there's no way everyone will stop. At this point the only logical course of action in an adversarial situation is to double down and keep researching, otherwise some other country or culture with different (and possibly worse) values ends up dominating the technology and you're left behind in the dust. T…

They are calling for a pause, not a stop. It’s quite clear that OpenAI has a significant lead over everyone else. The only other country outside the west that even has a chance at developing something better than GPT-4 soon is China. China has a pretty cautious culture as well so it’s quite possible that a bilateral moratorium can be negotiated with them. ADDED: Even without considering X-risks, China’s rulers cannot…

>It’s quite clear that OpenAI has a significant lead over everyone else

if their lead was significant they wouldn't have admitted to not releasing more info about GPT-4 in their paper due to commercial reasons. What ever secret sauce they have apparently isn't that significant or they wouldn't be afraid to talk about it

Re: Pause Giant AI Experiments: An Open Letter

#186

Post by tambourine_man »

GauntletWizard wrote: post #173
tambourine_man wrote: post #170

A few decades ago human cloning was irreversible, inevitable, the cat was out of the bag and you couldn’t put the genie back in the bottle. Yet we managed to legislate that. We need a change of atitute towards AI, recognize its danger and legislate accordingly.

Thou shalt not create a machine in the likeness of a human mind.

We need to have at least a clue of what we are doing.

I don’t anyone is proposing a full stop, but we could use some time to get our shit together.

Re: Pause Giant AI Experiments: An Open Letter

#187

Post by NLPaep »

ren_engineer wrote: post #185
nopinsight wrote: post #166

Earlier quoted context omitted.

They are calling for a pause, not a stop. It’s quite clear that OpenAI has a significant lead over everyone else. The only other country outside the west that even has a chance at developing something better than GPT-4 soon is China. China has a pretty cautious culture as well so it’s quite possible that a bilateral moratorium can be negotiated with them. ADDED: Even without considering X-risks, China’s rulers cannot…

>It’s quite clear that OpenAI has a significant lead over everyone else if their lead was significant they wouldn't have admitted to not releasing more info about GPT-4 in their paper due to commercial reasons. What ever secret sauce they have apparently isn't that significant or they wouldn't be afraid to talk about it

Nah, it’s just easy to copy if you lay it out. It’s software.

Re: Pause Giant AI Experiments: An Open Letter

#188

Post by ChatGTP »

pdonis wrote: post #130
petilon wrote: post #68

> Should we automate away all the jobs, including the fulfilling ones? You don't necessarily need jobs because you don't need money. What you need is wealth. I am going to leave it to PG to explain the difference [1]: Wealth is not money. Wealth is stuff we want: food, clothes, houses, cars, gadgets, travel to interesting places, and so on. You can have wealth without having money. If you had a magic machine that cou…

> AI/robots will generate wealth at an unprecedented scale. Food, clothing and shelter will be plentiful. How is this going to work? "AI" can't make any of these things. "AI" is not the magic machine PG describes. It's not even close.

Kind of like how clothing and food are produced by some of the poorest people ever, mostly people think it's all due to automation.

Re: Pause Giant AI Experiments: An Open Letter

#189

Post by ChatGTP »

robwwilliams wrote: post #149
zug_zug wrote: post #69

I understand that some people find this a hopeless gesture. But I think anybody who thinks "losing jobs" or "China" is a bad outcome from AI is hopelessly lost. The upside of AI if done right - nobody ever having to work again The downside of AI if done wrong - end of humanity It doesn't take a rocket scientist to determine we've made AIs go from thinking like an ant to thinking like a 6-year-old in a few years. Anyb…

We have no idea of what 27 or 27 million times smarter means functionally. We could hope that “smarter” might mean “not interested in taking over the world, but rather in help maximize accomplishment, satisfaction, health span, and with a feeling of filial love toward its parents—-humanity, planet Earth, our sun, the laws of physics and chemistry, and the beautiful mess we call biology and evolution.” Gently achievin…

Gently achieving those goals would be real indicator of high intelligence.

Very well said. "Gently" is a great to be approaching this.

Re: Pause Giant AI Experiments: An Open Letter

#190

Post by reasonableklout »

juancn wrote: post #156

Stopping now would be extremely dangerous and borderline stupid. If you stop now, you're just left behind, because there's no way everyone will stop. At this point the only logical course of action in an adversarial situation is to double down and keep researching, otherwise some other country or culture with different (and possibly worse) values ends up dominating the technology and you're left behind in the dust. T…

I don't agree at all. It's totally fair for rival "countries or cultures with different values" to coordinate on matters that threaten everyone's survival. There are many examples of international agreements on technologies that fit this description. For example:

- International treaties to avoid nuclear proliferation and development, as other commenters have pointed out

- National moratoriums on gain-of-function research

- Regulations on biotechnology related to human cloning and human gene editing, and industry/academic norms set by conferences such as the Asilomar conference

- International treaties on climate change, such as the Montreal Protocol on CFCs and the hole in the ozone layer

Even if we assume that international coordination fails, 2 facts give us some breathing room to pause (not stop!) large experiments to at least figure out what we're doing and how to adapt society:

1. American companies are ahead of Chinese companies (I assume that's what you mean by "other country or culture") right now by at least a few years. The rest of the industry is barely catching up to GPT-3 , which came out in 2020.

2. China cannot unilaterally continue the race right now because their compute supply chain critically depends on technologies that the West has monopolies over, like sub-10nm silicon fabrication, advanced GPU technologies, and ultraviolet lithography. We're already using this lever, actually! For example, Nvidia exports of A100 and H100 GPUs are no longer allowed to China at scale.

Post reply on HN