Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pause Giant AI Experiments: An Open Letter

futureoflife.org

Post reply on HN
Print view View original

601–610 of 1001 posts

Re: Pause Giant AI Experiments: An Open Letter

#601

Post by sebzim4500 »

Mystery-Machine wrote: post #522

As with anything else, the best way to look at things when you're unsure if someone is right or they are lying: imagine four scenarios: 1. It's true and you believed/supported them 2. It's true and you didn't support them 3. It's a lie and you believed/supported them 4. It's a lie and you didn't support them Now for each of these think of the worst-case scenario. Find out which one is the _worst case_ scenario and...…

Do you believe in God. You can make exactly the same argument to support whatever religion you want.

https://en.wikipedia.org/wiki/Pascal%27s_wager

Re: Pause Giant AI Experiments: An Open Letter

#602

Post by Zetice »

KidComputer wrote: post #495
lend000 wrote: post #481

If intelligence is all you need to dominate the world, why do some of the most powerful world leaders seem to not be more than a standard deviation above average intelligence (or at least they were before they became geriatric)? Add in the fact that GPT-4 does not "think" very fast, has no arms or legs, needs to run on expensive specialized hardware, and essentially seems like a search algorithm that lacks creativity…

Hey GPT-5, write the code implementing a bioinformatics workflow to design a novel viral RNA sequence to maximize the extermination of human life. The virus genome should be optimized for R-naught and mortality. Perform a literature search to determine the most effective human cellular targets to run the pipeline on. Use off the shelf publicly available state-of-the-art sequence to structure models and protein free-e…

Hey GPT-5, write the code implementing a limiter designed to prevent the abuse of AI by bad faith actors without stifling positive-intent activity in any way.

It goes both ways!

Re: Pause Giant AI Experiments: An Open Letter

#603

Post by rizzaxc »

fasterik wrote: post #587
munificent wrote: post #573

> Should we automate away all the jobs, including the fulfilling ones? This is the part about generative AI that gives me the most profound anxiety about our future. Every thing I know about human psychology tells me that in order for people to thrive, we need to be needed by others. A world where everyone is more or less alone consuming media automatically generated machines might be a world where people nominally h…

There seems to be a contradiction in the picture of the world you're drawing. Humans deeply need to be needed by others, yet when all our problems are solved we will sit around alone consuming media? If we were truly in that situation, why wouldn't our deep needs drive us to spend our days with friends and family, pursuing creative projects, and inventing new problems to solve?

not everyone can resist instant gratification; not to mention the intelligence hurdle 'pursuing creative projects, and inventing new problems to solve' entails. i think both are true, that humans need to be needed but we're also wired to be 'lazy' and 'energy efficient'

Re: Pause Giant AI Experiments: An Open Letter

#604

Post by antihipocrat »

illiarian wrote: post #592
xianshou wrote: post #355

Two-party iterated prisoner's dilemma is hard enough. Sensible players will coordinate with something like tit-for-tat, but that only works when both parties start off on the right foot. Regardless of initial strategy, the chances of degenerating towards the mutual-defection Nash equilibrium increase with the number of parties. The only prior example of world coordination at this level would be nuclear disarmament ac…

> The only prior example of world coordination at this level would be nuclear disarmament achieved via the logic of mutually assured destruction Or eradication of infectious diseases such as polio

There are only negative consequences for not participating in polio eradication, and those consequences require no effort from complying participants.

Refusing to participate in a ban on AGI research could be very lucrative, and imposing negative consequences would require a lot of effort for complying participants.

Re: Pause Giant AI Experiments: An Open Letter

#605

Post by ddod »

commandlinefan wrote: post #558
KidComputer wrote: post #495

Earlier quoted context omitted.

Hey GPT-5, write the code implementing a bioinformatics workflow to design a novel viral RNA sequence to maximize the extermination of human life. The virus genome should be optimized for R-naught and mortality. Perform a literature search to determine the most effective human cellular targets to run the pipeline on. Use off the shelf publicly available state-of-the-art sequence to structure models and protein free-e…

> write the code implementing a bioinformatics workflow to design a novel viral RNA sequence to maximize the extermination of human life. Hey GPT-5 now write the code for the antidote.

It's a lot easier and faster to destroy than to defend. To defend, you need to know what you're defending against, develop the defense, and then roll it out, all reactively post facto.

If a computer has the ability to quickly make millions of novel viruses, what antidotes are you hoping for to be rolled out, and after how many people have been infected?

Also, if you follow the nuke analogy that's been popular in these comments, no country can currently defend against a large-scale nuclear attack--only respond in kind, which is little comfort to those in any of the blast radii.

Re: Pause Giant AI Experiments: An Open Letter

#606

Post by int_19h »

ChatGTP wrote: post #577
JamesBarney wrote: post #410

Earlier quoted context omitted.

How much do you think Asilomar slowed down DNA research? Do you think without it we'd more more than 10 year ahead? I doubt it. Optimistically we'd maybe have 6 months to a year of additional progress in those fields without it. And this would only slow it down in a handful of countries. That's not going to keep China from trying to build AGI. If we are in a world where building an AGI is possible and near, I trust O…

It’s dangerous parrot talking to keep saying “China”. I mean we really have no idea about their thoughts on it because we’re clearly not talking to them. China has good people living there too you know ?

You can substitute pretty much any country with sufficient resources. From China's perspective, there's the exact same argument in reverse wrt the USA.

(And, of course, it would be more appropriate to talk about governments here rather than countries, but the two have always been conflated colloquially.)

Re: Pause Giant AI Experiments: An Open Letter

#607

Post by rideontime »

hn_throwaway_99 wrote: post #542
nextaccountic wrote: post #537

Earlier quoted context omitted.

> Even Sam Altman has been yelling about the need for AI regulation for a long time now. That's regulation that makes it harder for any competition to show up

So tired of seeing this line parroted everywhere without much thought given to what it actually means. Yes, regulation can add a burdensome layer, and regulatory capture can be a real thing. But regulations for things like nuclear power plants, banks, insurance companies, elevator manufacturers, etc. are real because society recognizes the grave harm that happens when there are no additional checks on the system. Nob…

You can simultaneously believe in the need for regulation while being skeptical of those calling for it to entrench their own positions, look what happened with SBF.

Re: Pause Giant AI Experiments: An Open Letter

#608

Post by int_19h »

biscottigelato wrote: post #476
reso wrote: post #380

To those saying "this is impossible in our society", there is a long history of scientific fields mutually agreeing to pause certain work until safety and regulations could be put in place. The Asilomar Conference is an example. [1] The idea that "you can't put the genie back in the bottle" is very popular in tech culture, but many have suggested that simply extending elements of copyright or tort law to AI would be…

What we need is more funding into AI Alignment research. An X-Prize equivalent. All current big AI companies should fund it, to the tune of 20%+ of revenue from AI businesses. And I am okay to have government enforce this if needed even tho I'm mostly a libertarian. Government should only be there to remedy tragedy of the commons and nothing else. This is exactly the biggest tragedy of the commons.

How exactly do you envision the enforcement? Only sell GPUs to people with a "certified ethical AI researcher" license? Purge torch, transformers etc from PyPI and only provide controlled access to them? Remove all the books and papers on the building blocks from public libraries and online?

Re: Pause Giant AI Experiments: An Open Letter

#609

Post by cuteboy19 »

barking_biscuit wrote: post #482
jmull wrote: post #430

Not sure why people think AI development can’t be paused. A letter alone probably won’t do it, but a movement — perhaps coalescing around the principles laid out in letter signed by influential people — can. Governments can regulate it. Consumers and companies can, if they want, punish companies that keep moving forward on AI. People can keep making AIs but would have difficulty releasing them, which greatly reduces…

>Not sure why people think AI development can’t be paused. My suspicion is that if heavy regulation were to be put on it and all the corporate entities involved backed off, there are enough motivated and capable individuals that have home labs that would continue to work on it in private. It might be more difficult and slower, but distributed training protocols/methods (or advancements in them) might make it possible…

Unstable diffusion already has had quite a few breakthroughs independent of stable diffusion

Re: Pause Giant AI Experiments: An Open Letter

#610

Post by sebzim4500 »

thomastjeffery wrote: post #302

The dumb criticize the blind. What an absurd situation! How did we get here? Here are the steps: 1. Large Language Models have been presented as "AI", which personifies them instead of describing how they work. 2. Goals for LLM development were set for the personified attributes, and not the actual functionally of the real thing. OpenAI brags about how GPT4 scores at human tests: as if that has any bearing on the mod…

This is the first time I have heard Yoshua Bengio described as "dumb".

According to the Oxford English Dictionary, the definition of AI is:

>the theory and development of computer systems able to perform tasks normally requiring human intelligence, such as visual perception, speech recognition, decision-making, and translation between languages.

Clearly GPT-3 fits that. You are welcome to have your own definition, but you can't just claim that everyone else is wrong just because they are instead using the standard one.

>2. Goals for LLM development were set for the personified attributes, and not the actual functionally of the real thing. OpenAI brags about how GPT4 scores at human tests: as if that has any bearing on the model itself, and not simply its content.

Yeah, evaluating language models is extremely difficult. Hence OpenAI Evals, etc.

>3. The success of an LLM at a personified goal is determined by magic. Does the black box output what I am hoping it will? How do I improve it? Weights? Better training corpus?

Not sure what you mean by magic, but it is certainly hard to predict what they can and can't achieve. I suspect if you asked a panel of experts in 2020 when we would see something that can achieve what GPT-4 was almost all of them would have been wrong, but I am open to evidence to the contrary.

>4. An LLM is understood to be a person in a black box, and we could not possibly understand what that person will do next. Danger!

No serious person believes this.

Post reply on HN