Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pause Giant AI Experiments: An Open Letter

futureoflife.org

Post reply on HN
Print view View original

781–790 of 1001 posts

Re: Pause Giant AI Experiments: An Open Letter

#781

Post by staunton »

le-mark wrote: post #655
nonbirithm wrote: post #571

Earlier quoted context omitted.

This has been repeated often, but even if it's true, I have to wonder why it's treated as a given with no further exploration. Is it because we as a species will inevitably accept any technological progress once sometime after it's been discovered, before the consequences can be suffered? What will that imply for any other species intelligent enough to get to where we are? The kinds of theories I mull over tend to de…

This is all speculative, but not fiction at this point clearly. Sci-fi authors have explored this possibility for decades, maybe their ideas could be of some help? I struggle to see how though; how to train Asimovs three laws for example?

The very point of Asimovs laws is that you can't just make up a couple simple laws and rest assured nothing bad will happen

Re: Pause Giant AI Experiments: An Open Letter

#782

Post by rdiddly »

pornel wrote: post #749
dmz73 wrote: post #737

Best response to the current "AI" fad driven fear I've seen so far (not my words): These AI tools cannot do things. They create text (or images or code or what-have-you) in response to prompts. And that's it! It is impressive, and it is clearly passing the Turing Test to some degree, because people are confusing the apparent intelligence behind these outputs with a combination of actual intelligence and "will." Not o…

A river has no will, but it can flood and destroy. A discussion whether AI does something because it "wants" to or not, is just philosophy and semantics. But it may end up generating a series of destructive instructions anyway. We feed these LLMs all of the Web, including instructions how to write code, and how to write exploits. They could become good at writing sandbox escapes, and one day write one when it just ha…

A river kinda has access to the real world a little bit. (Referring to the other part of the argument.)

Re: Pause Giant AI Experiments: An Open Letter

#783

Post by [deleted] »

nopinsight wrote: post #762
AbrahamParangi wrote: post #758

Earlier quoted context omitted.

This analysis is completely fact-free. "A Manhattan project on AI Alignment, if started now, might still succeed in time. Therefore, the compliance between parties needs not be long-term, which is indeed unlikely to happen." On what grounds do you base this? You have 3 hypotheticals stacked one on top of the other: 1) AI Alignment is possible 2) AI Alignment is a specific project that may be accomplished before [bad…

Regarding 3), check out the fact that OpenAI, DeepMind, and other top labs have AI safety programs and people working on AI Alignment. Interviews by Sam Altman, Ilya Sutskever, and others confirm their concerns. Here’s an article by Prof Russell, AAAI Fellow and a co-author of the standard AI text: https://www.technologyreview.com/2016/11/02/156285/yes-we-ar... Regarding 1) and 2), we might as well not succeed. But w…

[deleted]

Re: Pause Giant AI Experiments: An Open Letter

#786

Post by danmaz74 »

dmz73 wrote: post #737

Best response to the current "AI" fad driven fear I've seen so far (not my words): These AI tools cannot do things. They create text (or images or code or what-have-you) in response to prompts. And that's it! It is impressive, and it is clearly passing the Turing Test to some degree, because people are confusing the apparent intelligence behind these outputs with a combination of actual intelligence and "will." Not o…

I mostly agree with what you said, and I'm also skeptical enough about LLMs being a path towards AGI, even if they are really impressive. But there's something to say regarding these things not getting ideas or self-starting. The way these "chat" models work reminds me of internal dialogue; they start with a prompt, but then they could proceed forever from there, without any additional prompts. Whatever the initial input was, a session like this could potentially converge on something completely unrelated to the intention of whoever started that, and this convergence could be interpreted as "getting ideas" in terms of the internal representation of the LLM.

Now, from an external point of view, the model would still just be producing text. But if the text was connected with the external world with some kind of feedback loop, eg some people actually acting on what they interpret the text as saying and then reporting back, then the specific session/context could potentially have agency.

Would such a system be able to do anything significant or dangerous? Intuitively, I don't think that would be the case right now, but it wouldn't be technically impossible; it would all depend on the emergent properties of the training+feedback system, which nobody can predict as far as I know.

Re: Pause Giant AI Experiments: An Open Letter

#787

Post by TedDoesntTalk »

brabel wrote: post #778
richardw wrote: post #444

Earlier quoted context omitted.

And add in just one peer-level war where one side has their back against the wall. Then give it 100 years where anyone can create such a model on their phone. We’d need a constantly evolving inoculation function to compete. And it would probably lose because the other side has fewer restrictions. In my darker thoughts about this, this is why we see no aliens. To get this to work we need a far smarter entity with no p…

> To get this to work we need a far smarter entity with no physical limitations to still want us around... Why would an AI based on LLMs as we see today "want" or "not want" anything? It doesn't have the capacity to "want". We seem to imagine that "wanting" is something that will just emerge somehow, but I've seen no logical explanation for how that might work... I mean, we don't need to fully understand how the LLM…

> It doesn't have the capacity to "want"

Bing Chat clearly expresses love and the desire for a journalist to leave his wife. It also expresses other desires:

https://archive.ph/7LFcJ

https://archive.ph/q3nXG

These articles are disturbing. You might argue that it doesn’t know what it is expressing; that it is probabilities of words strung together. When do we agree that doesn’t matter and what matters are it’s consequences? That if Bing Chat had a body or means to achieve its desires in meat space, that whether or not it “knows” what it is expressing is irrelevant?

Re: Pause Giant AI Experiments: An Open Letter

#788

Post by amoss »

AbrahamParangi wrote: post #758
nopinsight wrote: post #695

Earlier quoted context omitted.

A Manhattan project on AI Alignment, if started now, might still succeed in time. Therefore, the compliance between parties needs not be long-term, which is indeed unlikely to happen. China, which is the country outside the west with the highest (engineering) capability to train something more powerful than GPT-4, is very concerned about domestic stability and they also do not want an easily replicable alien tool wit…

This analysis is completely fact-free. "A Manhattan project on AI Alignment, if started now, might still succeed in time. Therefore, the compliance between parties needs not be long-term, which is indeed unlikely to happen." On what grounds do you base this? You have 3 hypotheticals stacked one on top of the other: 1) AI Alignment is possible 2) AI Alignment is a specific project that may be accomplished before [bad…

Not to disagree, but you seem to have skipped 0) Increasing the parameter size of LLMs is a path to sentience.

Re: Pause Giant AI Experiments: An Open Letter

#789

Post by rdiddly »

This is painfully quaint and embarrassing. Not because there's no cause for concern (though it does overestimate the concern), but because it's so naive and utopian about the nature of humans and the world. Do we think the world is full of "accurate, safe, interpretable, transparent, robust, aligned, trustworthy, and loyal" people? No, and wishing it were so betrays a noble but misguided and potentially just-as-dangerous urge to sanitize the earth, that ought to instead be turned inward toward perfecting oneself. But do we think the world is instead full of suffering, exploitation and murder by some tragic accident? It's who we are. The fears about AI mainly seem to consist of fearing that it'll behave just like us. Someone's projecting.
Post reply on HN