Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pause Giant AI Experiments: An Open Letter

futureoflife.org

Post reply on HN
Print view View original

491–500 of 1001 posts

Re: Pause Giant AI Experiments: An Open Letter

#491

Post by biscottigelato »

I think it makes sense to debate whether this pause can or cannot work and the game theory behind it.

But to those that trivializes the potential danger of AI and compare it to the likes of climate change is ignorant at best. AI has been consistently ranked as the highest potential cause for the extinction of the entire human race (by experts, a random survey of the population will likely say climate change, sadly). (https://riskfrontiers.com/insights/ranking-of-potential-caus...)

Human caused climate change at the worst potential possible outcome would not even make a dent to the human population at a multi-century time-scale. We also have a lot of solutions in place for climate change that merely requires time and investment for adoption. Climate change's impact to humanity is less than the alarming birth rate we are seeing today (which also, we have no solution for...)

We literally have no solution for AI Alignment, and this is with a lot of smart people working on it. From Bostrom to Eliezer and many others (check https://lesswrong.com). A lot of experts have give up because they see no way out and is only merely thinking about how should humanity end with dignity. We can't even figure a way out of the prisoner dilemma on doing a 6 months pause.

Comparing AI Alignment with Climate Change trivializes the matter and only illustrate the ignorance of the populace. Another reason why we will not solve AI Alignment and will likely go extinct.

Re: Pause Giant AI Experiments: An Open Letter

#492

Post by nice_byte »

petilon wrote: post #68

> Should we automate away all the jobs, including the fulfilling ones? You don't necessarily need jobs because you don't need money. What you need is wealth. I am going to leave it to PG to explain the difference [1]: Wealth is not money. Wealth is stuff we want: food, clothes, houses, cars, gadgets, travel to interesting places, and so on. You can have wealth without having money. If you had a magic machine that cou…

> In the next 10 years, AI/robots will generate wealth at an unprecedented scale. Food, clothing and shelter will be plentiful.

Anyone who believes in the possibility of post-scarcity society must be either naive or trolling. Something cannot be made from nothing, therefore scarcity cannot be overcome, even assuming that all planning and execution is performed by superhuman synthetic minds.

Assuming that it's theoretically possible to utilize existing resources in a very efficient manner (e.g. motor running on a grain of rice for a century) and we just need AI to help us figure it out, is a _gigantic_ leap of faith and i would not bet a cent on that.

Let me paint a more realistic possibility for you (with a broader time horizon): most of the value created by automating knowledge work will be captured by private capital, and middle class will all but disappear. Education beyond basic reading and writing will become unattainable (and, frankly, unnecessary), most population will be reduced to the state of semi-literate serfdom dependent on the newly minted lords for survival. The lords wouldn't have to worry about feeding their subjects for too long though, as mass death brought about by climate change will take care of that problem.

Under that scenario, there will be no new enlightenment age to come and save us. The only reason we get to enjoy whatever freedoms we have today is that a (semi-)intellectual population is absolutely necessary to keep the complex modern economy running. Even then, those above you will do absolutely everything to limit your agency - by withholding information, lying, or just outright taking freedoms away. Do you know what happens once our participation in propping up the economic machine becomes unnecessary? Demotion to the default state of a human throughout history - a groveling, suffering serf who has no idea what's going on.

"If you want a picture of the future, imagine a boot stamping on a human face – for ever."

Re: Pause Giant AI Experiments: An Open Letter

#493

Post by adamsmith143 »

EForEndeavour wrote: post #324
adamsmith143 wrote: post #310

Earlier quoted context omitted.

This is frankly a bad and cavalier take on an extremely important subject. Many on the list are Academics outside AI/ML and/or leaders of AI orgs at the very top of the field that have no need to catch up to or slow down OpenAI to benefit themselves. Risks from AI are very real and Sam Altman himself has said so numerous times. He also in fact advocated for slowing down AI progress on Lex Fridman's podcast this month…

How do we reconcile Sam Altman's position as CEO of OpenAI with his repeated calls to slow down AI progress? Is the expectation that his conscience, sense of ethics, and concern for his own company's impact on society will temper the opposing urge/pressure to maintain OpenAI's lead in the AI market?

I'm generally not a big fan of Altman or OpenAI but their corporate structure ensures limited upside to Altman and the employees so other than recognition/fame which I think as the head of YC for many years Altman already had plenty of there isn't a huge incentive for them to maintain their lead.

Short of like a binding UN resolution or something similar we don't have a sliver of hope to slow down global AI progress which is a major factor in the doomer argument.

Re: Pause Giant AI Experiments: An Open Letter

#494

Post by kromem »

jimrandomh wrote: post #146

For a long time, "AI alignment" was a purely theoretical field, making very slow progress of questionable relevance, due to lack of anything interesting to experiment on. Now, we have things to experiment on, and the field is exploding, and we're finally learning things about how to align these systems. But not fast enough. I really don't want to overstate the capabilities of current-generation AI systems; they're no…

You're neglecting to consider the power of recursion.

Maybe the best and necessary tool in aligning GPT-N is GPT-(N-1).

We've already in just the past few weeks seen the power in using models to generate instructive fine tuning data.

Don't you think aligned models might be able to be applied to better aligning future models in ways that we can't yet anticipate because capability discovery is occurring on a week by week basis and not a six month or longer basis?

Re: Pause Giant AI Experiments: An Open Letter

#495

Post by KidComputer »

lend000 wrote: post #481

If intelligence is all you need to dominate the world, why do some of the most powerful world leaders seem to not be more than a standard deviation above average intelligence (or at least they were before they became geriatric)? Add in the fact that GPT-4 does not "think" very fast, has no arms or legs, needs to run on expensive specialized hardware, and essentially seems like a search algorithm that lacks creativity…

Hey GPT-5, write the code implementing a bioinformatics workflow to design a novel viral RNA sequence to maximize the extermination of human life. The virus genome should be optimized for R-naught and mortality. Perform a literature search to determine the most effective human cellular targets to run the pipeline on. Use off the shelf publicly available state-of-the-art sequence to structure models and protein free-energy perturbation methods for the prediction of binding affinity. Use cheaper computational methods where relevant to decrease the computational cost of running the pipeline.

And so on.

Re: Pause Giant AI Experiments: An Open Letter

#496

Post by biscottigelato »

lend000 wrote: post #481

If intelligence is all you need to dominate the world, why do some of the most powerful world leaders seem to not be more than a standard deviation above average intelligence (or at least they were before they became geriatric)? Add in the fact that GPT-4 does not "think" very fast, has no arms or legs, needs to run on expensive specialized hardware, and essentially seems like a search algorithm that lacks creativity…

Because the ability to influence others is more important than 1 or 2 standard deviation in more intelligence.

However if we are talking about 10x more intelligence, then that'd be a whole different ball game.

Check Alpha Go -> Alpha Go Zero. Gaining orders of magnitude in capability only with slight change to the model, along with a lot of more resource (we can easily throw 100x more compute and data to these models if someone seriously wants to and don't care about ROI. Or maybe if AI starts to be hugely profitable, 100x in investment is almost a rational outcome) is not uncommon in AI research.

Re: Pause Giant AI Experiments: An Open Letter

#497

Post by JW_00000 »

petilon wrote: post #68

> Should we automate away all the jobs, including the fulfilling ones? You don't necessarily need jobs because you don't need money. What you need is wealth. I am going to leave it to PG to explain the difference [1]: Wealth is not money. Wealth is stuff we want: food, clothes, houses, cars, gadgets, travel to interesting places, and so on. You can have wealth without having money. If you had a magic machine that cou…

> In the next 10 years, AI/robots will generate wealth at an unprecedented scale. Food, clothing and shelter will be plentiful.

GPT cannot create food, clothing, or shelter. The article talks about "automating away all the jobs", but GPT does not automate farming, making clothes, construction work, driving a car, delivering packages... Anything that requires interfacing with the real-world (i.e. outside of language), is not what GPT is doing. E.g. I wouldn't say research in self-driving cars has advanced now that LLMs exist. The magic machine you describe is exactly what GPT can NOT do.

In my opinion, GPT is a huge leap forward, but in one particular niche (language-based tasks). It does not advance the state-of-the-art in automation of physical systems, so it cannot replace any job that has a physical part to it.

Re: Pause Giant AI Experiments: An Open Letter

#498

Post by yreg »

sarchertech wrote: post #487
yreg wrote: post #446

Earlier quoted context omitted.

>since they trained on a huge chunk of web text What did they trained it on? Why is it unprobable to train on a better/bigger dataset any time soon?

Because they trained it on a huge percentage of the existing web. There's isn't a (much) bigger and better data set available.

What percentage?

What about books, newspapers, documents, etc.?

Re: Pause Giant AI Experiments: An Open Letter

#500

Post by JW_00000 »

nice_byte wrote: post #492
petilon wrote: post #68

> Should we automate away all the jobs, including the fulfilling ones? You don't necessarily need jobs because you don't need money. What you need is wealth. I am going to leave it to PG to explain the difference [1]: Wealth is not money. Wealth is stuff we want: food, clothes, houses, cars, gadgets, travel to interesting places, and so on. You can have wealth without having money. If you had a magic machine that cou…

> In the next 10 years, AI/robots will generate wealth at an unprecedented scale. Food, clothing and shelter will be plentiful. Anyone who believes in the possibility of post-scarcity society must be either naive or trolling. Something cannot be made from nothing, therefore scarcity cannot be overcome, even assuming that all planning and execution is performed by superhuman synthetic minds. Assuming that it's theoret…

> Education beyond basic reading and writing will become unattainable (and, frankly, unnecessary)

Is this possible in democracies with free (i.e. state-provided) education? Why would a majority vote to abolish education?

Post reply on HN