Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pause Giant AI Experiments: An Open Letter

futureoflife.org

Post reply on HN
Print view View original

761–770 of 1001 posts

Re: Pause Giant AI Experiments: An Open Letter

#761

Post by JoshuaDavid »

NhanH wrote: post #747
JoshuaDavid wrote: post #745

Earlier quoted context omitted.

https://openai.com/blog/chatgpt-plugins

I'm gonna request more explanations and proof, or at least theoretical path on using Expedia, Zapier, Instacart, Kayak etc. to dominate the world and kill every single human on earth.

Explanations, sure. My point was that yes, ChatGPT is indeed an entity which cannot interact with the world except through reading and writing text. This would be a lot more comforting if people were not rushing to build ways to turn its text output into actions in the physical world as fast as possible.

Imagine a mob boss whose spine was severed in an unfortunate mob-related accident. The mob boss cannot move his arms or legs, and can only communicate through speech. Said mob boss has it out for you. How worried are you? After all, this mob boss cannot do things. They create speech in response to prompts. And that's it!

I actually don't agree with Eliezer that the primary threat model is a single consequentialist agent recursively bootstrapping its way to uncontested godhood. But there is a related threat model, that of "better technology allows you to make bigger mistakes faster and more vigorously, and in the case of sufficiently powerful AGI, autonomously".

In terms of proof that it's possible to destroy the world and kill all humans, I will not provide that. No matter how poetic of an ending it would be for humanity if it ended because someone was wrong on the internet, and someone else felt the need to prove that they were wrong.

Re: Pause Giant AI Experiments: An Open Letter

#762

Post by nopinsight »

AbrahamParangi wrote: post #758
nopinsight wrote: post #695

Earlier quoted context omitted.

A Manhattan project on AI Alignment, if started now, might still succeed in time. Therefore, the compliance between parties needs not be long-term, which is indeed unlikely to happen. China, which is the country outside the west with the highest (engineering) capability to train something more powerful than GPT-4, is very concerned about domestic stability and they also do not want an easily replicable alien tool wit…

This analysis is completely fact-free. "A Manhattan project on AI Alignment, if started now, might still succeed in time. Therefore, the compliance between parties needs not be long-term, which is indeed unlikely to happen." On what grounds do you base this? You have 3 hypotheticals stacked one on top of the other: 1) AI Alignment is possible 2) AI Alignment is a specific project that may be accomplished before [bad…

Regarding 3), check out the fact that OpenAI, DeepMind, and other top labs have AI safety programs and people working on AI Alignment. Interviews by Sam Altman, Ilya Sutskever, and others confirm their concerns.

Here’s an article by Prof Russell, AAAI Fellow and a co-author of the standard AI text: https://www.technologyreview.com/2016/11/02/156285/yes-we-ar...

Regarding 1) and 2), we might as well not succeed. But would you propose that we sit still and do nothing if many experts say that there is even a 20% chance that a superhuman alien species will arrive on earth in 5-25 years and we do not know about their intentions?

A survey of AI experts well before GPT-4 shows that nearly half of them have such concerns (with varying timelines and probabilities).

By the way, calling a proposal by Prof Stuart Russell and several other top AI experts “dumb” should require a much stronger argument and level of evidence than you have shown.

Re: Pause Giant AI Experiments: An Open Letter

#763

Post by m3kw9 »

joefourier wrote: post #757
dmz73 wrote: post #737

Best response to the current "AI" fad driven fear I've seen so far (not my words): These AI tools cannot do things. They create text (or images or code or what-have-you) in response to prompts. And that's it! It is impressive, and it is clearly passing the Turing Test to some degree, because people are confusing the apparent intelligence behind these outputs with a combination of actual intelligence and "will." Not o…

LLMs certainly can “will” and “do things” when provided with the right interface like LangChain: https://github.com/hwchase17/langchain See also the ARC paper where the model was capable of recruiting and convincing a TaskRabbit worker to solve captchas. I think many people make the mistake to see raw LLMs as some sort of singular entity when in fact, they’re more like a simulation of a text based “world” (with multi…

LLM if given the tools(allow it to execute code online) can certainly execute a path towards an objective, they can be told to do something but free to act anyway that it thinks it’s best towards it. That isn’t dangerous because it is not self aware doing it’s own thing yet

Re: Pause Giant AI Experiments: An Open Letter

#764

Post by m3kw9 »

fasterik wrote: post #465

>Powerful AI systems should be developed only once we are confident that their effects will be positive and their risks will be manageable This statement by itself sounds reasonable. But for me an interesting thought experiment is to take this letter and imagine the equivalent for some other technology, for example semiconductors in the 1960s, the world wide web in the 1990s, or social media in the late 2000s-early 2…

It just seems to me most of these guys that are signing feel they don’t get to be part of this revolution and if AGI develops they will be pretty much be nothing. This is equivalent of saying, don’t leave us out..

Re: Pause Giant AI Experiments: An Open Letter

#766

Post by nice_byte »

qgin wrote: post #623
nice_byte wrote: post #492

Earlier quoted context omitted.

> In the next 10 years, AI/robots will generate wealth at an unprecedented scale. Food, clothing and shelter will be plentiful. Anyone who believes in the possibility of post-scarcity society must be either naive or trolling. Something cannot be made from nothing, therefore scarcity cannot be overcome, even assuming that all planning and execution is performed by superhuman synthetic minds. Assuming that it's theoret…

Sounds like fuel for a revolution. I know some governments will go to war with their own people, but all of them? There will be some countries that go the way you've described and crush any discontent with force. But some will try to navigate an outcome that socializes the benefits of automation.

you're assuming the transition would be abrupt enough to trigger a revolution. most likely that wouldn't be the case - if ruling class plays their cards right, they'll boil the frog, and all revolutionary potential will be defused.

Re: Pause Giant AI Experiments: An Open Letter

#768

Post by lannisterstark »

jamilton wrote: post #671
lannisterstark wrote: post #405

Earlier quoted context omitted.

It's not about it being impossible. I guess the major issue right now is that people are focusing on the wrong aspect of it. They should be focusing on how most LLMs are currently only controlled by bigger coroporations. Democratizing the LLMs and Giving access to people would resolve the x-poly we have currently. This also solves the "safety and regulations" issue to an extent - Do you really trust that Microsoft, G…

What do you mean by "democratizing the LLMs"? There are already open source LLMs. Making Claude, GPT-4, and whatever else is being worked on open source wouldn't make them any less capable of doing harm.

I mean in the sense of getting capable models in the hands of people, rather than only a few companies who will arguably steer them towards their own goals. The open source models we currently have are vastly inferior to proprietary ones.

Fragmentation reduces any danger. It doesn't amplify it. Imagine just chrome and Google pushing pwas at you.

Re: Pause Giant AI Experiments: An Open Letter

#769

Post by friendzis »

AbrahamParangi wrote: post #758
nopinsight wrote: post #695

Earlier quoted context omitted.

A Manhattan project on AI Alignment, if started now, might still succeed in time. Therefore, the compliance between parties needs not be long-term, which is indeed unlikely to happen. China, which is the country outside the west with the highest (engineering) capability to train something more powerful than GPT-4, is very concerned about domestic stability and they also do not want an easily replicable alien tool wit…

This analysis is completely fact-free. "A Manhattan project on AI Alignment, if started now, might still succeed in time. Therefore, the compliance between parties needs not be long-term, which is indeed unlikely to happen." On what grounds do you base this? You have 3 hypotheticals stacked one on top of the other: 1) AI Alignment is possible 2) AI Alignment is a specific project that may be accomplished before [bad…

> 3) Solving AI Alignment is an actual problem and not just dumb extrapolation from science fiction

As far as I am aware there is still no actionable science behind mathematical analysis of AI models. You cannot take a bunch of weights and tell how it will behave. So we "test" models by deploying and HOPE there is nothing nefarious within.

It has been shown that models will "learn" to exfiltrate data between stages. You may call it dumb extrapolation, but it has been shown that it is a problem: a solution that we want is not necessarily the most optimal against the cost function that we give. The more inputs/weights model has, the harder it would be to spot problems in advance.

Re: Pause Giant AI Experiments: An Open Letter

#770

Post by AbrahamParangi »

nopinsight wrote: post #762
AbrahamParangi wrote: post #758

Earlier quoted context omitted.

This analysis is completely fact-free. "A Manhattan project on AI Alignment, if started now, might still succeed in time. Therefore, the compliance between parties needs not be long-term, which is indeed unlikely to happen." On what grounds do you base this? You have 3 hypotheticals stacked one on top of the other: 1) AI Alignment is possible 2) AI Alignment is a specific project that may be accomplished before [bad…

Regarding 3), check out the fact that OpenAI, DeepMind, and other top labs have AI safety programs and people working on AI Alignment. Interviews by Sam Altman, Ilya Sutskever, and others confirm their concerns. Here’s an article by Prof Russell, AAAI Fellow and a co-author of the standard AI text: https://www.technologyreview.com/2016/11/02/156285/yes-we-ar... Regarding 1) and 2), we might as well not succeed. But w…

An idea may be dumb regardless of who believes it. You will find history littered with such ideas.
Post reply on HN