Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pause Giant AI Experiments: An Open Letter

futureoflife.org

Post reply on HN
Print view View original

871–880 of 1001 posts

Re: Pause Giant AI Experiments: An Open Letter

#871

Post by jckahn »

brabel wrote: post #778
richardw wrote: post #444

Earlier quoted context omitted.

And add in just one peer-level war where one side has their back against the wall. Then give it 100 years where anyone can create such a model on their phone. We’d need a constantly evolving inoculation function to compete. And it would probably lose because the other side has fewer restrictions. In my darker thoughts about this, this is why we see no aliens. To get this to work we need a far smarter entity with no p…

> To get this to work we need a far smarter entity with no physical limitations to still want us around... Why would an AI based on LLMs as we see today "want" or "not want" anything? It doesn't have the capacity to "want". We seem to imagine that "wanting" is something that will just emerge somehow, but I've seen no logical explanation for how that might work... I mean, we don't need to fully understand how the LLM…

AIs don’t need to “want” to have unintended results, they just need a directive. Like in 2001 where HAL realized that it could achieve the mission better if the humans were all dead.

Re: Pause Giant AI Experiments: An Open Letter

#872

Post by spaceman_2020 »

amoss wrote: post #791
nopinsight wrote: post #700

Earlier quoted context omitted.

GPT-4 is still not dangerous. Given the rapid progress trajectory though, GPT-5 and later which may be developed in a few short years could very well be, esp in the hands of a smart sociopath. (History shows there are many who could cause real-world harm. Imagine them armed with a 24/7 agent with expertise in dozens of fields.) See these predictions of AI in 2025 by an OpenAI insider and a former DeepMind research en…

- autonomously design, code and distribute whole apps (but not the most complex ones) This is a bold claim. Today LLMs have not been demonstrated to be capable of synthesizing novel code. There was a post just a few days ago on the performance gap between problems that had polluted the training data and novel problems that had not. So if we project forward from the current state of the art: it would be more accurate…

To be fair, most coding problems are not novel problems.

I’ll be more impressed if chatGPT can take a large, poorly maintained, poorly documented codebase and make meaningful changes to it.

Far easier to write something from scratch than to modify something that was first written in Fortran in 1991.

Re: Pause Giant AI Experiments: An Open Letter

#873

Post by helen___keller »

pornel wrote: post #749
dmz73 wrote: post #737

Best response to the current "AI" fad driven fear I've seen so far (not my words): These AI tools cannot do things. They create text (or images or code or what-have-you) in response to prompts. And that's it! It is impressive, and it is clearly passing the Turing Test to some degree, because people are confusing the apparent intelligence behind these outputs with a combination of actual intelligence and "will." Not o…

A river has no will, but it can flood and destroy. A discussion whether AI does something because it "wants" to or not, is just philosophy and semantics. But it may end up generating a series of destructive instructions anyway. We feed these LLMs all of the Web, including instructions how to write code, and how to write exploits. They could become good at writing sandbox escapes, and one day write one when it just ha…

Writing a sandbox escape doesn’t mean escaping.

If the universe is programmed by god, there might be some bug in memory safety in the simulation. Should God be worried that humans, being a sentient collectively-super-intelligent AI living in His simulation, are on the verge of escaping and conquering heaven?

Would you say humans conquering heaven is more or less likely than GPT-N conquering humanity?

Re: Pause Giant AI Experiments: An Open Letter

#874

Post by qikInNdOutReply »

barney54 wrote: post #864
bambax wrote: post #813

Earlier quoted context omitted.

> Climate change mitigation, which more closely resembles AI safety in both complexity and (lack of) barriers to entry, has been sporadic, inconsistent, and only enacted to the extent... Climate change mitigation is the perfect example. Nobody is doing anything, nobody seems to care, everyone cheats with ridiculous carbon credits or carbon offset vouchers made out of thin air, etc. It's likely the planet will become…

It is just not true that “nobody is doing anything.” CO2 emissions in the United States and Europe are falling and have been doing so for years. To say that is nothing is misrepresenting reality.

Yeah, cause we shifted them somewhere else. I too can stop drinking (at home), by going out to a bar. Horray, alcoholism is cured.

Re: Pause Giant AI Experiments: An Open Letter

#875

Post by naillo »

reso wrote: post #380

To those saying "this is impossible in our society", there is a long history of scientific fields mutually agreeing to pause certain work until safety and regulations could be put in place. The Asilomar Conference is an example. [1] The idea that "you can't put the genie back in the bottle" is very popular in tech culture, but many have suggested that simply extending elements of copyright or tort law to AI would be…

The idea that laws can't stop ML research is insane. We've barely had open source progress on these things for 5 years due to the "safety" propaganda. Imagine if they were actually illegal to work on, we'd get nothing done in public and barely anything done at all.

Re: Pause Giant AI Experiments: An Open Letter

#876

Post by Kaijo »

belter wrote: post #841
bambax wrote: post #813

Earlier quoted context omitted.

> Climate change mitigation, which more closely resembles AI safety in both complexity and (lack of) barriers to entry, has been sporadic, inconsistent, and only enacted to the extent... Climate change mitigation is the perfect example. Nobody is doing anything, nobody seems to care, everyone cheats with ridiculous carbon credits or carbon offset vouchers made out of thin air, etc. It's likely the planet will become…

"...The planet has been through a lot worse than us. Been through earthquakes, volcanoes, plate tectonics, continental drift, solar flares, sun spots, magnetic storms, the magnetic reversal of the poles … hundreds of thousands of years of bombardment by comets and asteroids and meteors, worldwide floods, tidal waves, worldwide fires, erosion, cosmic rays, recurring ice ages … And we think some plastic bags and some a…

I really dislike this sentiment.

Planets can become entirely inhospitable to life. Planets themselves have lifespans. Earth herself has in the past suffered near misses, e.g. 90%+ extinction events. It took billions of years of evolution to produce us, the only species ever to exist with the ability to reason about, prevent or ameliorate large extinction events (such as those caused by asteroid impacts), effect conservation management over evolutionary timescales, and even potentially to revive extinct species or propagate earth's lifeforms beyond the solar system and into a continuing timeline.

It's anti-conservationist and I think immoral to promote the view that anthropogenic climate change and the ongoing biodiversity crisis is about our fate alone.

Re: Pause Giant AI Experiments: An Open Letter

#877

Post by iopq »

Kaijo wrote: post #876
belter wrote: post #841

Earlier quoted context omitted.

"...The planet has been through a lot worse than us. Been through earthquakes, volcanoes, plate tectonics, continental drift, solar flares, sun spots, magnetic storms, the magnetic reversal of the poles … hundreds of thousands of years of bombardment by comets and asteroids and meteors, worldwide floods, tidal waves, worldwide fires, erosion, cosmic rays, recurring ice ages … And we think some plastic bags and some a…

I really dislike this sentiment. Planets can become entirely inhospitable to life. Planets themselves have lifespans. Earth herself has in the past suffered near misses, e.g. 90%+ extinction events. It took billions of years of evolution to produce us, the only species ever to exist with the ability to reason about, prevent or ameliorate large extinction events (such as those caused by asteroid impacts), effect conse…

You missed the point, the planet can still be there even if all life becomes extinct. We're not trying to save the planet. We're trying to save life.

Re: Pause Giant AI Experiments: An Open Letter

#878

Post by Helmut10001 »

Kaijo wrote: post #876
belter wrote: post #841

Earlier quoted context omitted.

"...The planet has been through a lot worse than us. Been through earthquakes, volcanoes, plate tectonics, continental drift, solar flares, sun spots, magnetic storms, the magnetic reversal of the poles … hundreds of thousands of years of bombardment by comets and asteroids and meteors, worldwide floods, tidal waves, worldwide fires, erosion, cosmic rays, recurring ice ages … And we think some plastic bags and some a…

I really dislike this sentiment. Planets can become entirely inhospitable to life. Planets themselves have lifespans. Earth herself has in the past suffered near misses, e.g. 90%+ extinction events. It took billions of years of evolution to produce us, the only species ever to exist with the ability to reason about, prevent or ameliorate large extinction events (such as those caused by asteroid impacts), effect conse…

Agree. There's only a picture that comes to my mind to express my perception, from artist Erik Johansson, it is called 'imminent' [1].

[1]: https://duckduckgo.com/?va=u&t=hj&q=Erik+Johansson+imminent&...

Re: Pause Giant AI Experiments: An Open Letter

#879

Post by [deleted] »

xianshou wrote: post #355

Two-party iterated prisoner's dilemma is hard enough. Sensible players will coordinate with something like tit-for-tat, but that only works when both parties start off on the right foot. Regardless of initial strategy, the chances of degenerating towards the mutual-defection Nash equilibrium increase with the number of parties. The only prior example of world coordination at this level would be nuclear disarmament ac…

[deleted]
Post reply on HN