Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pause Giant AI Experiments: An Open Letter

futureoflife.org

Post reply on HN
Print view View original

841–850 of 1001 posts

Re: Pause Giant AI Experiments: An Open Letter

#841

Post by belter »

bambax wrote: post #813
xianshou wrote: post #355

Two-party iterated prisoner's dilemma is hard enough. Sensible players will coordinate with something like tit-for-tat, but that only works when both parties start off on the right foot. Regardless of initial strategy, the chances of degenerating towards the mutual-defection Nash equilibrium increase with the number of parties. The only prior example of world coordination at this level would be nuclear disarmament ac…

> Climate change mitigation, which more closely resembles AI safety in both complexity and (lack of) barriers to entry, has been sporadic, inconsistent, and only enacted to the extent... Climate change mitigation is the perfect example. Nobody is doing anything, nobody seems to care, everyone cheats with ridiculous carbon credits or carbon offset vouchers made out of thin air, etc. It's likely the planet will become…

"...The planet has been through a lot worse than us. Been through earthquakes, volcanoes, plate tectonics, continental drift, solar flares, sun spots, magnetic storms, the magnetic reversal of the poles … hundreds of thousands of years of bombardment by comets and asteroids and meteors, worldwide floods, tidal waves, worldwide fires, erosion, cosmic rays, recurring ice ages … And we think some plastic bags and some aluminum cans are going to make a difference? The planet isn’t going anywhere. WE are!..."

- George Carlin

Re: Pause Giant AI Experiments: An Open Letter

#843

Post by alfiedotwtf »

kolinko wrote: post #823
alfiedotwtf wrote: post #651

Earlier quoted context omitted.

This is the first time I've actually seen game theory applied to an actual real life scenario rather than just a theoretical academic problem. What's the best resource to learn more game theory and to be able to applying it like you to a more general setting?

Shelling's Theory of Conflict: https://www.amazon.com/gp/aw/d/1258092948/ref=tmm_hrd_swatch... (Well-crafted hardcover and quality Kindle edition available) Easy and amazing read, written in 1960s - it builds on the game theory, it was used as a basis for MAD and nuclear strategy in general. The author won a Nobel Prize in Economics for the ideas presented in this book.

Oh! Thanks for that recommendation. Added to my Amazon list. Much appreciated

Re: Pause Giant AI Experiments: An Open Letter

#844

Post by [deleted] »

xianshou wrote: post #355

Two-party iterated prisoner's dilemma is hard enough. Sensible players will coordinate with something like tit-for-tat, but that only works when both parties start off on the right foot. Regardless of initial strategy, the chances of degenerating towards the mutual-defection Nash equilibrium increase with the number of parties. The only prior example of world coordination at this level would be nuclear disarmament ac…

[deleted]

Re: Pause Giant AI Experiments: An Open Letter

#845

Post by davidkunz »

amoss wrote: post #791
nopinsight wrote: post #700

Earlier quoted context omitted.

GPT-4 is still not dangerous. Given the rapid progress trajectory though, GPT-5 and later which may be developed in a few short years could very well be, esp in the hands of a smart sociopath. (History shows there are many who could cause real-world harm. Imagine them armed with a 24/7 agent with expertise in dozens of fields.) See these predictions of AI in 2025 by an OpenAI insider and a former DeepMind research en…

- autonomously design, code and distribute whole apps (but not the most complex ones) This is a bold claim. Today LLMs have not been demonstrated to be capable of synthesizing novel code. There was a post just a few days ago on the performance gap between problems that had polluted the training data and novel problems that had not. So if we project forward from the current state of the art: it would be more accurate…

> Today LLMs have not been demonstrated to be capable of synthesizing novel code.

They are capable of doing that (to some extend). Personally, I've generated plenty of (working) code to solve novel problems and I'm 100% sure that code wasn't part of the training set.

Re: Pause Giant AI Experiments: An Open Letter

#846

Post by gherkinnn »

pornel wrote: post #749
dmz73 wrote: post #737

Best response to the current "AI" fad driven fear I've seen so far (not my words): These AI tools cannot do things. They create text (or images or code or what-have-you) in response to prompts. And that's it! It is impressive, and it is clearly passing the Turing Test to some degree, because people are confusing the apparent intelligence behind these outputs with a combination of actual intelligence and "will." Not o…

A river has no will, but it can flood and destroy. A discussion whether AI does something because it "wants" to or not, is just philosophy and semantics. But it may end up generating a series of destructive instructions anyway. We feed these LLMs all of the Web, including instructions how to write code, and how to write exploits. They could become good at writing sandbox escapes, and one day write one when it just ha…

A river absolutely has a will In the broadest sense. It will carve its way through the countryside whether we like it or not.

A hammer has no will.

Re: Pause Giant AI Experiments: An Open Letter

#848

Post by [deleted] »

xianshou wrote: post #355

Two-party iterated prisoner's dilemma is hard enough. Sensible players will coordinate with something like tit-for-tat, but that only works when both parties start off on the right foot. Regardless of initial strategy, the chances of degenerating towards the mutual-defection Nash equilibrium increase with the number of parties. The only prior example of world coordination at this level would be nuclear disarmament ac…

[deleted]

Re: Pause Giant AI Experiments: An Open Letter

#849

Post by shrewdcomputer »

alex_young wrote: post #712
xianshou wrote: post #355

Two-party iterated prisoner's dilemma is hard enough. Sensible players will coordinate with something like tit-for-tat, but that only works when both parties start off on the right foot. Regardless of initial strategy, the chances of degenerating towards the mutual-defection Nash equilibrium increase with the number of parties. The only prior example of world coordination at this level would be nuclear disarmament ac…

Arguably there are other (better) examples. Off the top of my head: * CFC elimination (saved the ozone layer) * Leaded gas ban (well, except private aircraft, but they are working on that)

This is incredibly important to point out. It's sad when everyone gives up on global collaboration because some things aren't going well - that's not a reason to give up on collaborations, it's a reason to double down.

CFC elimination is a great example but even a hugely complex issue, and one we aren't doing well on, like climate change has had progress according to the latest IPCC report because of global collaboration.

Re: Pause Giant AI Experiments: An Open Letter

#850

Post by RandomLensman »

hn_throwaway_99 wrote: post #413
puma_ambit wrote: post #365

The virtue signal of all virtue signals. We do not live in a society where an article like this will have any impact. This is merely an attempt to buy time because they're behind and want to have a chance to catch up. Work will progress whether they like it or not. Finally someone other than Ol' Musky is in the news more than him. Don't get me wrong he's doing and leading great work, but for once in more recent times…

"Virtue signalling"? Please. There are a lot of very smart experts on that signatory list who definitely don't need to, or care about, virtue signalling. Fine, ignore Musk's signature, but I don't think luminaries like Stuart Russell, Steve Wozniak, Jaan Tallinn or John Hopfield are doing this for "virtue signalling". You can fairly argue that this will be ineffective, but a lot of experts in this field have real, "h…

We have long entered the realm of theology here with people really wanting to believe in the omnipotence of a certain tool (possibly even while some other, simpler things destroy them).

What for example is Tallinn's medium- to long-term predictive track record on social issues? On technological development? Anyone can be concerned and have genuine reasons for concern, but that doesn't mean the outcomes materialize.

Post reply on HN