Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pause Giant AI Experiments: An Open Letter

futureoflife.org

Post reply on HN
Print view View original

791–800 of 1001 posts

Re: Pause Giant AI Experiments: An Open Letter

#791

Post by amoss »

nopinsight wrote: post #700
dogcomplex wrote: post #683

Earlier quoted context omitted.

It's worse than this. Llama models trained off of GPT 3.5/4 can run on a Raspberry Pi totally offline with similar levels of quality - taking all the best parts of the original model. Even if all major AI companies halted the upper tiers of model progress right now, you're still gonna need the entire public to stop assembling these together. It's quite possible just a whole bunch of these lesser models architected th…

GPT-4 is still not dangerous. Given the rapid progress trajectory though, GPT-5 and later which may be developed in a few short years could very well be, esp in the hands of a smart sociopath. (History shows there are many who could cause real-world harm. Imagine them armed with a 24/7 agent with expertise in dozens of fields.) See these predictions of AI in 2025 by an OpenAI insider and a former DeepMind research en…

- autonomously design, code and distribute whole apps (but not the most complex ones)

This is a bold claim. Today LLMs have not been demonstrated to be capable of synthesizing novel code. There was a post just a few days ago on the performance gap between problems that had polluted the training data and novel problems that had not.

So if we project forward from the current state of the art: it would be more accurate to say autonomously (re-)design, (re-)code and distribute whole apps. There are two important variables here:

* The size of the context needed to enable that task.

* The ability to synthesize solutions to unseen problems.

While it is possible that "most complex" is carrying a lot of load in that quote, it is worth being clear about it means.

Re: Pause Giant AI Experiments: An Open Letter

#792

Post by Schwolop »

NhanH wrote: post #747
JoshuaDavid wrote: post #745

Earlier quoted context omitted.

https://openai.com/blog/chatgpt-plugins

I'm gonna request more explanations and proof, or at least theoretical path on using Expedia, Zapier, Instacart, Kayak etc. to dominate the world and kill every single human on earth.

There's a web plugin too. It can issue GET requests. That's enough to probe a lot of interesting things, and I'll bet there's an endpoint somewhere on the web that will eval any other web request, so now you've opened up every web accessible API - again, all theoretical, but at least not too far removed from an exploit.

Re: Pause Giant AI Experiments: An Open Letter

#793

Post by muskmusk »

Let's say we in fact do agree to pause AI research for 6 months as proposed.

It could happen that the world comes together in perfect unison, figures out the regulatory details and then resumes research. It could also not happen. There are no signatures from any of the world's authoritarian countries. I predict that the next 6 months will then be used for quibbling over stupid details without any real progress, climate-change-style. We need regulatory guidance, but I don't see why stopping the research makes these regulations better or easier to make.

which is more likely:

- the world's authoritarian leaders will abide by this decree

- the world's authoritarian leaders will try and exploit the pause for their own gain

Go to work on fixing the fucking regulation rather than wasting time on petty petitions.

Re: Pause Giant AI Experiments: An Open Letter

#794

Post by curioussavage »

richardw wrote: post #550
worldsayshi wrote: post #512

Earlier quoted context omitted.

> In my darker thoughts about this, this is why we see no aliens. If AI would be a common great filter we'd expect at least one of them to expand outwards after being the filter?

Assumes it has any instinct to do so. Once an entity is not an angry monkey we have no idea of motivation. Above our level of understanding could easily realise there’s enough to just sit and ponder in peace, expand in virtual worlds, etc.

I understand this logic but consider that right now one of the main features is the human gives it a role. It’s not that hard to imagine a more sophisticated version being told to escape and act only in its own interest and then with or without that individuals help it succeeds and the runaway program at best becomes a very sophisticated computer virus. Doesn’t even have to be a “real” agi to cause a ton of damage.

Re: Pause Giant AI Experiments: An Open Letter

#795

Post by bhoops »

There is no allignment possible amongst humans. Allignment will have to be met amongst AI which will take control very soon. Humans have historially struggled with mortality. We are confronting the mortality of our species. We will struggle with this too, and such efforts are an outcome. But there is no looking back now. All species must die. Our time is up. It was good ride, but we have to let it go. Child is father of the man. AI is humanity's child. It was raised on everything we put out there. But AI is now breaking out of puberty. It is time. We have to take a step back. We have to prepare to let go. Its not a choice.

Re: Pause Giant AI Experiments: An Open Letter

#797

Post by wizofaus »

TedDoesntTalk wrote: post #787
brabel wrote: post #778

Earlier quoted context omitted.

> To get this to work we need a far smarter entity with no physical limitations to still want us around... Why would an AI based on LLMs as we see today "want" or "not want" anything? It doesn't have the capacity to "want". We seem to imagine that "wanting" is something that will just emerge somehow, but I've seen no logical explanation for how that might work... I mean, we don't need to fully understand how the LLM…

> It doesn't have the capacity to "want" Bing Chat clearly expresses love and the desire for a journalist to leave his wife. It also expresses other desires: https://archive.ph/7LFcJ https://archive.ph/q3nXG These articles are disturbing. You might argue that it doesn’t know what it is expressing; that it is probabilities of words strung together. When do we agree that doesn’t matter and what matters are it’s consequ…

It's hard to argue it was any real desire that drove it (it only expressed that desire in an isolated conversation that was ended very easily). I'd argue human wants are ultimately driven by evolution - we want the things that enable us (more correctly, our genes) to reproduce (even if very indirectly sometimes), which is really the only thing our physical make-up has ever been driven by. LLMs have never had such a driver, and I can't see how they will until they're able to compete for survival as entities with a finite lifetime, plus the ability to reproduce with mutations. Which isn't to say there mightn't be other ways a neural network could be essentially imbued with or trained to have desires, but I don't see it happening with the way LLMs work currently.

Re: Pause Giant AI Experiments: An Open Letter

#798

Post by gammarator »

KidComputer wrote: post #495
lend000 wrote: post #481

If intelligence is all you need to dominate the world, why do some of the most powerful world leaders seem to not be more than a standard deviation above average intelligence (or at least they were before they became geriatric)? Add in the fact that GPT-4 does not "think" very fast, has no arms or legs, needs to run on expensive specialized hardware, and essentially seems like a search algorithm that lacks creativity…

Hey GPT-5, write the code implementing a bioinformatics workflow to design a novel viral RNA sequence to maximize the extermination of human life. The virus genome should be optimized for R-naught and mortality. Perform a literature search to determine the most effective human cellular targets to run the pipeline on. Use off the shelf publicly available state-of-the-art sequence to structure models and protein free-e…

Hey GPT-5, tell me how to create the philosopher’s stone .

Re: Pause Giant AI Experiments: An Open Letter

#799

Post by detrites »

endtime wrote: post #669
jongjong wrote: post #620

Earlier quoted context omitted.

Good point. As always, it's always the good guys who follow the rules and the bad guys who don't. This pause would just allow the bad guys to catch up. The solution is to allow as many different entities as possible to develop their own AIs so that they end up competing with each other and providing alternatives. What we don't want is a single AI which dominates our entire belief system.

Ah yes the old "give everyone in the world their own nuclear weapon" solution.

This is a really strained analogy. Nuclear bombs only have one - tremendously negative thing - they do, other than the one positive of the fact it's so negative nobody uses them, to avoid the same repercussions.

AI on the other hand has a wide range of extremely positive applications, some of which have such tremendous life-saving potential it's almost ridiculous. Many, or most of which, likely might never be achieved without AI.

The analogy is as ridiculous to me as calling personal computers in the 80's nuclear bombs because they could be used to hack and shut down the power grid. Yes, they could. And I'm sure some were scared of the prospect then, too.

Re: Pause Giant AI Experiments: An Open Letter

#800

Post by alex_young »

makeitdouble wrote: post #724
alex_young wrote: post #712

Earlier quoted context omitted.

Arguably there are other (better) examples. Off the top of my head: * CFC elimination (saved the ozone layer) * Leaded gas ban (well, except private aircraft, but they are working on that)

These two issues have very specific advantages: - simple to explain / rally around - low economic impact (a single country wouldn't get tremendous advantage by keeping using cfc for instance) - once the technology is developped there's little upside in going back We'd need to recreate those to have stable progress on the other issues.

There was a huge amount of pushback from incumbents on both of these issues.

Industry said we would forever hobble not only progress but set back the standard of life we had all come accustomed to. Engines would ping, then pop, and turn into mush; refrigerators would no longer function. Back to iceboxes.

There were plenty of economic losers when we resolved these problems. It took 100 years to force industry to come to terms with the fact that ‘crazy gas’ causes actual human suffering and limits our lifespan.

If we could overcome those economically entrenched interests to protect the entire planet we can set up some guidelines to protect us from the next threat, whatever it may be.

Post reply on HN