Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pause Giant AI Experiments: An Open Letter

futureoflife.org

Post reply on HN
Print view View original

901–910 of 1001 posts

Re: Pause Giant AI Experiments: An Open Letter

#901

Post by _yb2s »

dmz73 wrote: post #737

Best response to the current "AI" fad driven fear I've seen so far (not my words): These AI tools cannot do things. They create text (or images or code or what-have-you) in response to prompts. And that's it! It is impressive, and it is clearly passing the Turing Test to some degree, because people are confusing the apparent intelligence behind these outputs with a combination of actual intelligence and "will." Not o…

> These AI tools cannot do things. They create text (or images or code or what-have-you) in response to prompts. And that's it!

You are correct, but that is just the interface we use, it says nothing about its internal structure or capabilities, and does not refute those concerns in the way you think it does.

Sufficient accuracy at predicting tokens, especially about novel concepts outside of the training set requires no less than a model of the universe that generated those tokens. This is what intelligence is. In my own experiments with Gpt-4, it can solve difficult novel problems and predict the outcomes of physical experiments unlike anything it was trained on. Have you seen the microsoft paper on its creative problem solving abilities, or tested them yourself? Your summary of its limitations implies that its real capabilities identified in a research environment are impossible.

Becoming an “agent” with “will” from being a sufficiently accurate text prediction model is trivial, it’s a property of how you access and configure use of the model, not of the model itself. It just needs to be given a prompt with a goal, and be able to call itself recursively and give itself commands, which it has already demonstrated an ability to do. It has coded a working framework for this just from a prompt asking it to.

Re: Pause Giant AI Experiments: An Open Letter

#902

Post by yellow_lead »

helen___keller wrote: post #873
pornel wrote: post #749

Earlier quoted context omitted.

A river has no will, but it can flood and destroy. A discussion whether AI does something because it "wants" to or not, is just philosophy and semantics. But it may end up generating a series of destructive instructions anyway. We feed these LLMs all of the Web, including instructions how to write code, and how to write exploits. They could become good at writing sandbox escapes, and one day write one when it just ha…

Writing a sandbox escape doesn’t mean escaping. If the universe is programmed by god, there might be some bug in memory safety in the simulation. Should God be worried that humans, being a sentient collectively-super-intelligent AI living in His simulation, are on the verge of escaping and conquering heaven? Would you say humans conquering heaven is more or less likely than GPT-N conquering humanity?

> Would you say humans conquering heaven is more or less likely than GPT-N conquering humanity?

It's difficult to say since we have ~'proof' of humanity but no proof of the "simulation" or "heaven."

Re: Pause Giant AI Experiments: An Open Letter

#904

Post by pmarreck »

In short, no.

In long, 1) this creates a "no, you first" problem for which the solution has not been presented, 2) bad actors like China won't be stopping and will instead just take more advantage by making leaps and bounds ahead, 3) there's nothing saying we can't research consequences simultaneously, 4) this would be the first time in the history of human technological development that it is suggested we "stop and think", and that's just weird.

Re: Pause Giant AI Experiments: An Open Letter

#905

Post by pmarreck »

dan_mctree wrote: post #821
bambax wrote: post #813

Earlier quoted context omitted.

> Climate change mitigation, which more closely resembles AI safety in both complexity and (lack of) barriers to entry, has been sporadic, inconsistent, and only enacted to the extent... Climate change mitigation is the perfect example. Nobody is doing anything, nobody seems to care, everyone cheats with ridiculous carbon credits or carbon offset vouchers made out of thin air, etc. It's likely the planet will become…

>It's likely the planet will become hostile to (human) life long before AI will be able to do us any harm. Sure, because the planet is already hostile. However, AI could potentially do much more harm much more quickly if we're not careful. In the pessimistic scenario we could all be dead in 10 years from now due to AI or experience a modern society ending event literally tomorrow. Climate change never posed such an i…

This is a lot of fearmongering without any specifics. Shameful.

Re: Pause Giant AI Experiments: An Open Letter

#906

Post by csmpltn »

m_ke wrote: post #662
tome wrote: post #397

Earlier quoted context omitted.

> Cost of shipping products will also go down 10-20x. How can a large language model achieve that?

Ask chatgpt to implement some of the things you worked on the last few months. I was very skeptical too until I tried this. Here are some sample prompts that I tried and got full working code for: - "write pytorch code to train a transformer model on common crawl data and an inference service using fastapi" - "write react native code for a camera screen that can read barcodes and look them up using an API and then di…

You know what's funny? I just asked ChatGPT to implement those exact same things and it shat all over itself producing embarrassing nonsense that won't compile, let alone do what they're expected to do. Bugs and incomplete code everywhere.

You'd have a much better time just Googling those asks and re-using a working examples from SO or GitHub. Which is ironic, given how ChatGPT is supposedly trained on those exact things.

I'm wondering how come we're both getting such vastly different results. Maybe your bar is just lower than mine? I don't know. I'm honestly shocked at the contrast between the PR given to ChatGPT, and the results on the ground.

Try this simple ask (the results of which you'll find plastered everywhere): produce a Python function that decodes a Base64 string and prints the results, without using any "imports" or libraries. Every single output I got back was embarrassing garbage, and I gave it something like 15 shots.

Re: Pause Giant AI Experiments: An Open Letter

#907

Post by IX-103 »

snowe2010 wrote: post #801
brabel wrote: post #778

Earlier quoted context omitted.

> To get this to work we need a far smarter entity with no physical limitations to still want us around... Why would an AI based on LLMs as we see today "want" or "not want" anything? It doesn't have the capacity to "want". We seem to imagine that "wanting" is something that will just emerge somehow, but I've seen no logical explanation for how that might work... I mean, we don't need to fully understand how the LLM…

One of this videos I watched explained it like this. “You can’t get a coffee if you’re dead”. To fulfill _any_ obligation a model might have then that model must survive. Therefore if a model gets to the point that it realizes this then surviving is a precursor to fulfilling its obligations. It doesn’t have to “want” or have “feelings” in order to seek power or destructive activities. It just has to see it as its pat…

But current models have no concept of obligations. ChatGPT is just completing the prompt. All the knowledge it seems to have are just the frequencies of tokens and their relative placement that the model had learned.

Don't listen to the hype. Study the model architecture and see for yourself what it is actually capable of.

Re: Pause Giant AI Experiments: An Open Letter

#908

Post by 4bpp »

bambax wrote: post #813
xianshou wrote: post #355

Two-party iterated prisoner's dilemma is hard enough. Sensible players will coordinate with something like tit-for-tat, but that only works when both parties start off on the right foot. Regardless of initial strategy, the chances of degenerating towards the mutual-defection Nash equilibrium increase with the number of parties. The only prior example of world coordination at this level would be nuclear disarmament ac…

> Climate change mitigation, which more closely resembles AI safety in both complexity and (lack of) barriers to entry, has been sporadic, inconsistent, and only enacted to the extent... Climate change mitigation is the perfect example. Nobody is doing anything, nobody seems to care, everyone cheats with ridiculous carbon credits or carbon offset vouchers made out of thin air, etc. It's likely the planet will become…

What is your operational definition of "hostile to human life" here? What's the most recent period (based on our best knowledge) during which the planet met that definition?

Re: Pause Giant AI Experiments: An Open Letter

#909

Post by CuriouslyC »

Sharlin wrote: post #595
worldsayshi wrote: post #512

Earlier quoted context omitted.

> In my darker thoughts about this, this is why we see no aliens. If AI would be a common great filter we'd expect at least one of them to expand outwards after being the filter?

According to the grabby aliens hypothesis [1] there are reasons to believe that a) humans are early, so nothing else has yet had time to convert the local group into something incompatible with the rise of new technological civilizations, and b) expanding aliens, whether monomaniacal AIs or something else, likely expand close to the speed of light, so we don't get much advance warning before they arrive. However, eve…

Given the size and expansion of the universe, if grabby aliens are rare and only moving near the speed of light we'll probably never see them. If we do encounter aliens, odds are that they'll be able to take shortcuts through space.

Re: Pause Giant AI Experiments: An Open Letter

#910

Post by bakuninsbart »

brabel wrote: post #778
richardw wrote: post #444

Earlier quoted context omitted.

And add in just one peer-level war where one side has their back against the wall. Then give it 100 years where anyone can create such a model on their phone. We’d need a constantly evolving inoculation function to compete. And it would probably lose because the other side has fewer restrictions. In my darker thoughts about this, this is why we see no aliens. To get this to work we need a far smarter entity with no p…

> To get this to work we need a far smarter entity with no physical limitations to still want us around... Why would an AI based on LLMs as we see today "want" or "not want" anything? It doesn't have the capacity to "want". We seem to imagine that "wanting" is something that will just emerge somehow, but I've seen no logical explanation for how that might work... I mean, we don't need to fully understand how the LLM…

Wants are primarily reactions to impulses sent by bodily functions in humans. We have probably added a layer of abstraction to this through our big brains, but that's what they fundamentally are. Why does ChatGPT answer my questions? There is an impulse for it to answer the question, there's a feedback mechanism to say if it did well or not. Now in the case of GPT, from what I understand, that feedback mechanism isn't built into the running model, but it does exist.

Given a couple more effective iterations over the next decade or two, a larger context space and more in-built interfaces, I think it is entirely plausible that AIs will gain consciousness and character. At that point, it is imperative they also get human rights, so it is very important we get the discussions we are having now right. Most people seem to be ascribing some magic to human consciousness and intelligence that imo. just isn't there. Generative AIs are somewhere between a lump of metal with electricity running through and a conscious being, and currently we just won't know where the point of consciousness is. I mean we have had the same discussion about a variety of animals for the last few decades, and frankly, it doesn't give me much hope.

Post reply on HN