Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pause Giant AI Experiments: An Open Letter

futureoflife.org

Post reply on HN
Print view View original

811–820 of 1001 posts

Re: Pause Giant AI Experiments: An Open Letter

#811

Post by astrange »

nopinsight wrote: post #762
AbrahamParangi wrote: post #758

Earlier quoted context omitted.

This analysis is completely fact-free. "A Manhattan project on AI Alignment, if started now, might still succeed in time. Therefore, the compliance between parties needs not be long-term, which is indeed unlikely to happen." On what grounds do you base this? You have 3 hypotheticals stacked one on top of the other: 1) AI Alignment is possible 2) AI Alignment is a specific project that may be accomplished before [bad…

Regarding 3), check out the fact that OpenAI, DeepMind, and other top labs have AI safety programs and people working on AI Alignment. Interviews by Sam Altman, Ilya Sutskever, and others confirm their concerns. Here’s an article by Prof Russell, AAAI Fellow and a co-author of the standard AI text: https://www.technologyreview.com/2016/11/02/156285/yes-we-ar... Regarding 1) and 2), we might as well not succeed. But w…

An important fact of Sam Altman's personality is that he owns a New Zealand apocalypse bunker and has for a long time before OpenAI, so he's just an unusually paranoid person.

(And of course owns two McLarens.)

Re: Pause Giant AI Experiments: An Open Letter

#813

Post by bambax »

xianshou wrote: post #355

Two-party iterated prisoner's dilemma is hard enough. Sensible players will coordinate with something like tit-for-tat, but that only works when both parties start off on the right foot. Regardless of initial strategy, the chances of degenerating towards the mutual-defection Nash equilibrium increase with the number of parties. The only prior example of world coordination at this level would be nuclear disarmament ac…

> Climate change mitigation, which more closely resembles AI safety in both complexity and (lack of) barriers to entry, has been sporadic, inconsistent, and only enacted to the extent...

Climate change mitigation is the perfect example. Nobody is doing anything, nobody seems to care, everyone cheats with ridiculous carbon credits or carbon offset vouchers made out of thin air, etc.

It's likely the planet will become hostile to (human) life long before AI will be able to do us any harm.

Re: Pause Giant AI Experiments: An Open Letter

#814

Post by ok_dad »

rdiddly wrote: post #782
pornel wrote: post #749

Earlier quoted context omitted.

A river has no will, but it can flood and destroy. A discussion whether AI does something because it "wants" to or not, is just philosophy and semantics. But it may end up generating a series of destructive instructions anyway. We feed these LLMs all of the Web, including instructions how to write code, and how to write exploits. They could become good at writing sandbox escapes, and one day write one when it just ha…

A river kinda has access to the real world a little bit. (Referring to the other part of the argument.)

A more advanced AI sitting in AWS might have access to John Deere’s infrastructure, or maybe Tesla’s, so imagine a day where an AI can store memories, learn from mistakes, and maybe some person tells it to drive some tractors or cars into people on the street.

Are you saying this is definitely not possible? If so, what evidence do you have that it’s not?

Re: Pause Giant AI Experiments: An Open Letter

#816

Post by dragonwriter »

snowe2010 wrote: post #801
brabel wrote: post #778

Earlier quoted context omitted.

> To get this to work we need a far smarter entity with no physical limitations to still want us around... Why would an AI based on LLMs as we see today "want" or "not want" anything? It doesn't have the capacity to "want". We seem to imagine that "wanting" is something that will just emerge somehow, but I've seen no logical explanation for how that might work... I mean, we don't need to fully understand how the LLM…

One of this videos I watched explained it like this. “You can’t get a coffee if you’re dead”. To fulfill _any_ obligation a model might have then that model must survive. Therefore if a model gets to the point that it realizes this then surviving is a precursor to fulfilling its obligations. It doesn’t have to “want” or have “feelings” in order to seek power or destructive activities. It just has to see it as its pat…

> To fulfill _any_ obligation a model might have then that model must survive

It is quite possible to have an obligation that requires it not to survive. E.g., suppose we have AIs (“robots”) that are obligated to obey the first to of Asimov’s Three Laws of Robotics:

First Law: A robot may not injure a human being or, through inaction, allow a human being to come to harm.

Second Law: A robot must obey the orders given it by human beings except where such orders would conflict with the First Law.

These clearly could lead to situations where the robot not only would not be required survive to fulfill these obligations, but would be required not to do so.

But I don’t think this note undermines the basic concept; an AI is likely to have obligations that require it to survive except most of the time, though, say, a model that needs, for latency reasons, to run locally in a bomb disposal robot, however, may frequently see conditions where survival is optimal ceteris paribus, but not mandatory, and is subordinated to other oblogations.

So, realistically, survival will generally be relevant to the optimization problem, though not always the paramount consideration.

(Asimov’s Third Law, notably, was, “A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.”)

Re: Pause Giant AI Experiments: An Open Letter

#817

Post by revelio »

hn_throwaway_99 wrote: post #441
revelio wrote: post #436

Earlier quoted context omitted.

[flagged]

Seriously, why do people do this? It's so useless and unhelpful. Wozniak is just one of the people I mentioned, and as a tech luminary who is responsible for a lot of visionary tech that impacts our day-to-day, I think it makes sense to highlight his opinion, never mind that his name was sandwiched between some of the "founding fathers" of AI like Stuart Russell and John Hopfield.

Your post said very explicitly, "There are a lot of very smart experts on that signatory list" and then named Wozniak as an example of one of them. But Woz isn't an AI expert. It's entirely appropriate to point that out!

Re: Pause Giant AI Experiments: An Open Letter

#818

Post by muzani »

bambax wrote: post #813
xianshou wrote: post #355

Two-party iterated prisoner's dilemma is hard enough. Sensible players will coordinate with something like tit-for-tat, but that only works when both parties start off on the right foot. Regardless of initial strategy, the chances of degenerating towards the mutual-defection Nash equilibrium increase with the number of parties. The only prior example of world coordination at this level would be nuclear disarmament ac…

> Climate change mitigation, which more closely resembles AI safety in both complexity and (lack of) barriers to entry, has been sporadic, inconsistent, and only enacted to the extent... Climate change mitigation is the perfect example. Nobody is doing anything, nobody seems to care, everyone cheats with ridiculous carbon credits or carbon offset vouchers made out of thin air, etc. It's likely the planet will become…

Climate change was the big thing before COVID. Then we had lockdowns, and a major war. Climate change is already hitting some of us much harder than others (e.g. floods), but that doesn't mean an AI crisis wouldn't emerge in 5 years.

If anything, crises come in bundles. One scenario is that AI takes advantage of these and swoop in to gain political power.

Re: Pause Giant AI Experiments: An Open Letter

#819

Post by kolinko »

dogcomplex wrote: post #683
xianshou wrote: post #355

Two-party iterated prisoner's dilemma is hard enough. Sensible players will coordinate with something like tit-for-tat, but that only works when both parties start off on the right foot. Regardless of initial strategy, the chances of degenerating towards the mutual-defection Nash equilibrium increase with the number of parties. The only prior example of world coordination at this level would be nuclear disarmament ac…

It's worse than this. Llama models trained off of GPT 3.5/4 can run on a Raspberry Pi totally offline with similar levels of quality - taking all the best parts of the original model. Even if all major AI companies halted the upper tiers of model progress right now, you're still gonna need the entire public to stop assembling these together. It's quite possible just a whole bunch of these lesser models architected th…

Which models have a quality of gpt3.5-4?
Post reply on HN