Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pause Giant AI Experiments: An Open Letter

futureoflife.org

Post reply on HN
Print view View original

531–540 of 1001 posts

Re: Pause Giant AI Experiments: An Open Letter

#531

Post by emeril »

KidComputer wrote: post #495
lend000 wrote: post #481

If intelligence is all you need to dominate the world, why do some of the most powerful world leaders seem to not be more than a standard deviation above average intelligence (or at least they were before they became geriatric)? Add in the fact that GPT-4 does not "think" very fast, has no arms or legs, needs to run on expensive specialized hardware, and essentially seems like a search algorithm that lacks creativity…

Hey GPT-5, write the code implementing a bioinformatics workflow to design a novel viral RNA sequence to maximize the extermination of human life. The virus genome should be optimized for R-naught and mortality. Perform a literature search to determine the most effective human cellular targets to run the pipeline on. Use off the shelf publicly available state-of-the-art sequence to structure models and protein free-e…

tbh, I'd think, it would be much easier to just hack into russia and convince them we've launched nukes than to engineer some virus that may or may not work

Re: Pause Giant AI Experiments: An Open Letter

#532

Post by KidComputer »

lend000 wrote: post #511
KidComputer wrote: post #495

Earlier quoted context omitted.

Hey GPT-5, write the code implementing a bioinformatics workflow to design a novel viral RNA sequence to maximize the extermination of human life. The virus genome should be optimized for R-naught and mortality. Perform a literature search to determine the most effective human cellular targets to run the pipeline on. Use off the shelf publicly available state-of-the-art sequence to structure models and protein free-e…

I've been trying to use GPT-4 for my hard science startup, and it really has nothing to offer when you push the boundaries of what has been done by even a little, but it's great for speeding up coding. Once we do have an AI capable of extraordinary innovation (hopefully in 10 years! But probably a lot longer), it will be obvious, and it will unfortunately be removed from the hands of the plebs based on fearmongering…

This was merely an example to suggest the danger is not in AI becoming self-aware but amplifying human abilities 1000 fold and how they use those abilities. GPT is not necessary for any part of this. In-silico methods just need to catch up in terms of accuracy and efficiency and then you can wrap the whole thing an RL process.

Maybe you can ask GPT for some good starting points.

Re: Pause Giant AI Experiments: An Open Letter

#533

Post by invig »

lewhoo wrote: post #443
time_to_smile wrote: post #421

Earlier quoted context omitted.

There are plenty of bigger "human ending" concerns on the table right now than AI and we certainly aren't pausing anything for those.

Like what ? Climate change ? The EU just voted for petrol and diesel car ban. Are we really single-threaded ?

In 2035. Maybe we can pause AI development in 2035?

Re: Pause Giant AI Experiments: An Open Letter

#534

Post by _Adam »

Mystery-Machine wrote: post #522

As with anything else, the best way to look at things when you're unsure if someone is right or they are lying: imagine four scenarios: 1. It's true and you believed/supported them 2. It's true and you didn't support them 3. It's a lie and you believed/supported them 4. It's a lie and you didn't support them Now for each of these think of the worst-case scenario. Find out which one is the _worst case_ scenario and...…

Another perspective if we consider AI akin to nuclear weapons (a simile many have made already): If the law-abiding cease development and malicious actors do not, the worst case may be that the resulting AI is non-aligned with humanity's interests. If the "good guys" develop an AI with alignment as an intention, then there are at least good AI to counter the bad AI.

Of course they may all ultimately end up colluding to destroy us. My point is it's not just as simple as "someone who's behind might catch up and make more money".

Re: Pause Giant AI Experiments: An Open Letter

#535

Post by invig »

drtz wrote: post #489
puma_ambit wrote: post #365

The virtue signal of all virtue signals. We do not live in a society where an article like this will have any impact. This is merely an attempt to buy time because they're behind and want to have a chance to catch up. Work will progress whether they like it or not. Finally someone other than Ol' Musky is in the news more than him. Don't get me wrong he's doing and leading great work, but for once in more recent times…

Ask yourself two questions: 1. What is the ultimate AI are we trying to eventually build? 2. What are the consequences if we eventually succeed in building such an AI? For me, the answers would be 1. A general AI that would equal and surpass human intelligence. 2. God only knows. Will we get there? Who knows. We do know, however, that the consequences are great enough that only a fool would not at least attempt to pr…

WHAT SAFEGUARDS?

"Someone else should come up with something"

Like there's someone else out there (who aren't the people making the thing) who has a better idea on how to keep it safe.

Re: Pause Giant AI Experiments: An Open Letter

#536

Post by invig »

lannisterstark wrote: post #405
reso wrote: post #380

To those saying "this is impossible in our society", there is a long history of scientific fields mutually agreeing to pause certain work until safety and regulations could be put in place. The Asilomar Conference is an example. [1] The idea that "you can't put the genie back in the bottle" is very popular in tech culture, but many have suggested that simply extending elements of copyright or tort law to AI would be…

It's not about it being impossible. I guess the major issue right now is that people are focusing on the wrong aspect of it. They should be focusing on how most LLMs are currently only controlled by bigger coroporations. Democratizing the LLMs and Giving access to people would resolve the x-poly we have currently. This also solves the "safety and regulations" issue to an extent - Do you really trust that Microsoft, G…

The corporations pay for people to learn how to do this stuff. Eventually, someone will leave and build the open-source version (when it's feasible to do it / the price has come down).

Re: Pause Giant AI Experiments: An Open Letter

#537

Post by nextaccountic »

hn_throwaway_99 wrote: post #413
puma_ambit wrote: post #365

The virtue signal of all virtue signals. We do not live in a society where an article like this will have any impact. This is merely an attempt to buy time because they're behind and want to have a chance to catch up. Work will progress whether they like it or not. Finally someone other than Ol' Musky is in the news more than him. Don't get me wrong he's doing and leading great work, but for once in more recent times…

"Virtue signalling"? Please. There are a lot of very smart experts on that signatory list who definitely don't need to, or care about, virtue signalling. Fine, ignore Musk's signature, but I don't think luminaries like Stuart Russell, Steve Wozniak, Jaan Tallinn or John Hopfield are doing this for "virtue signalling". You can fairly argue that this will be ineffective, but a lot of experts in this field have real, "h…

> Even Sam Altman has been yelling about the need for AI regulation for a long time now.

That's regulation that makes it harder for any competition to show up

Re: Pause Giant AI Experiments: An Open Letter

#538

Post by lanstin »

c7DJTLrn wrote: post #278
Quarrel wrote: post #257

Earlier quoted context omitted.

Forget countries? How will you stop me? I have models on my local computer. I can and will use them. They will get better. You don't put tech genie's back in their bottles, that's not how it works.

>I have models on my local computer. I can and will use them. They will get better. Well no, not really. You're just consuming the result of millions of dollars of investment. You're not going to be able to improve it in any significant way with limited resources and knowledge.

Yeah using the model is one thing but what OpenAI has shown is that order of magnitude increases in model training can make a big difference. But they are already at the tens of millions to train a model, so order of magnitude bump is only possible to a small number of actors. I suspect NSA is busily setting something like gpt 4 up right now.

Re: Pause Giant AI Experiments: An Open Letter

#539

Post by bootsmann »

richardw wrote: post #462
time_to_smile wrote: post #421

Earlier quoted context omitted.

There are plenty of bigger "human ending" concerns on the table right now than AI and we certainly aren't pausing anything for those.

No, AI drives all the others in the long run. Others are speed bumps.

Plain, old fashioned historicism. It was wrong 100 years ago, it is wrong today still.

Re: Pause Giant AI Experiments: An Open Letter

#540

Post by thenaturalist »

biscottigelato wrote: post #496
lend000 wrote: post #481

If intelligence is all you need to dominate the world, why do some of the most powerful world leaders seem to not be more than a standard deviation above average intelligence (or at least they were before they became geriatric)? Add in the fact that GPT-4 does not "think" very fast, has no arms or legs, needs to run on expensive specialized hardware, and essentially seems like a search algorithm that lacks creativity…

Because the ability to influence others is more important than 1 or 2 standard deviation in more intelligence. However if we are talking about 10x more intelligence, then that'd be a whole different ball game. Check Alpha Go -> Alpha Go Zero. Gaining orders of magnitude in capability only with slight change to the model, along with a lot of more resource (we can easily throw 100x more compute and data to these models…

Recent research [0] is poking holes into Alpha Go Zero and perfectly illustrates the naivitee in assessing most of the current state of ML/ Deep Learning solutions: They are still limited by compute, and the world is too complex still.

0: https://www.iflscience.com/human-beats-ai-in-14-out-of-15-go...

Post reply on HN