Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pause Giant AI Experiments: An Open Letter

futureoflife.org

Post reply on HN
Print view View original

961–970 of 1001 posts

Re: Pause Giant AI Experiments: An Open Letter

#961

Post by JohnFen »

dougmwne wrote: post #134
JohnFen wrote: post #31

Earlier quoted context omitted.

I don't think laughing at people for their concerns is helpful. Especially when you don't seem to be bothered by the possibility that this is a civilization-ending development. For the record, I don't think it is, personally. But people are terrified, and not taking their fear seriously and dealing with it compassionately is a recipe for serious, and unnecessary disaster.

Laughing at Elon is always helpful to me personally. The civilization ending development is going to happen or not happen. It’s hubris to think any one person or one country has a say at this point.

"The most common way people give up their power is by thinking they don't have any."

Re: Pause Giant AI Experiments: An Open Letter

#963

Post by throwaway292939 »

lend000 wrote: post #511
KidComputer wrote: post #495

Earlier quoted context omitted.

Hey GPT-5, write the code implementing a bioinformatics workflow to design a novel viral RNA sequence to maximize the extermination of human life. The virus genome should be optimized for R-naught and mortality. Perform a literature search to determine the most effective human cellular targets to run the pipeline on. Use off the shelf publicly available state-of-the-art sequence to structure models and protein free-e…

I've been trying to use GPT-4 for my hard science startup, and it really has nothing to offer when you push the boundaries of what has been done by even a little, but it's great for speeding up coding. Once we do have an AI capable of extraordinary innovation (hopefully in 10 years! But probably a lot longer), it will be obvious, and it will unfortunately be removed from the hands of the plebs based on fearmongering…

My take is that for GPT-4, it has mastery of existing knowledge. I'm not sure how it would be able to push new boundaries.

Re: Pause Giant AI Experiments: An Open Letter

#964

Post by nopinsight »

bambax wrote: post #813
xianshou wrote: post #355

Two-party iterated prisoner's dilemma is hard enough. Sensible players will coordinate with something like tit-for-tat, but that only works when both parties start off on the right foot. Regardless of initial strategy, the chances of degenerating towards the mutual-defection Nash equilibrium increase with the number of parties. The only prior example of world coordination at this level would be nuclear disarmament ac…

> Climate change mitigation, which more closely resembles AI safety in both complexity and (lack of) barriers to entry, has been sporadic, inconsistent, and only enacted to the extent... Climate change mitigation is the perfect example. Nobody is doing anything, nobody seems to care, everyone cheats with ridiculous carbon credits or carbon offset vouchers made out of thin air, etc. It's likely the planet will become…

The real comparison is with an alternative scenario. Climate change would have been worse had there been no widespread recognition of its harm.

———

GPT-5 will be finished next year, with the next model in the pipeline.

GPT-4 is just a kindergartner which can code at the level of an average human. (If you disagree, try teaching an average person to code or solve most novel problems better than it can.)

Even if AI does not have independent agency, malicious actors can use it to do large-scale harm much sooner than suggested above.

However, I’m sure someone will imbue AI with some kind of agency soon enough as well. What could go wrong?

Re: Pause Giant AI Experiments: An Open Letter

#965

Post by beezlewax »

mk3 wrote: post #833
dan_mctree wrote: post #821

Earlier quoted context omitted.

>It's likely the planet will become hostile to (human) life long before AI will be able to do us any harm. Sure, because the planet is already hostile. However, AI could potentially do much more harm much more quickly if we're not careful. In the pessimistic scenario we could all be dead in 10 years from now due to AI or experience a modern society ending event literally tomorrow. Climate change never posed such an i…

Do you really think china and other hostile countries will halt anything because Elon Musk, Steve Wozniak and some other guys signed a letter? Also what about Musks tesla itself they are using AI for FSD and no one bats an eye

Elon Musk who is one of the stakeholders of Open AI?

Re: Pause Giant AI Experiments: An Open Letter

#966

Post by dredmorbius »

Animats wrote: post #65
christkv wrote: post #53

Can an AGI be controlled, and if it is, who will dictate its actions and to whom will the benefits accrue. What happens to the service economy and the political stability when the middle class paper pushed get decimated.

> Who will dictate its actions and to whom will the benefits accrue? Corporations, of course. Capitalist society has never achieved a consensus on the proper limits of corporate power. There is a school of thought, popularized by Milton Friedman, that says the only duty of corporations is to their stockholders. This is about to backfire. If we treat AIs as a component of corporate power, there are few limits on what…

Forget outperforming CEOs.

AI outperforming investors and owners.

On the basis of motivations and reasoning which are beyond human ken. Which is to say, that even the nominal beneficial owners of that AI itself won't understand it. Or be able to prevent its own adopting of legal or corporate structures which effectively emancipate it.

Re: Pause Giant AI Experiments: An Open Letter

#967

Post by yyyk »

kanzure wrote: post #946
yyyk wrote: post #204

Earlier quoted context omitted.

It's not remotely comparable. Few people had anything to gain from cloning; These few people had a simpler, cheaper route via IVF; Cloning requires specific technical equipment easily controlled and specific technical expertise few other people had.

Human cloning isn't that difficult. The machines are well known and understood and easy to build. Tell me this: if you were producing genetically modified embryos or human clones, would you be broadcasting this to the entire world? There's actually low incentive to be very vocal about this kind of activity, especially for the sake of the child and its privacy going forward in life.

>if you were producing genetically modified embryos or human clones, would you be broadcasting this to the entire world?

You're broadening the subject a bit, no? The question was about cloning. It's quite possible (maybe even likely) secretive gene modification is already happening, but there aren't any chatter around cloning the way there's chatter around CRISPR etc.

I think the main factor is extremely low demand: The people sufficiently rich and megalomaniacal to consider this are fine with IVF and can easily afford it.

What's the point of (human) cloning really? Getting 'you' to live again? The result won't be "you" even if the genes were expressed identically (they won't be), and if you try to enhance the genes - a thing this type of people would want - than the clone won't even be genetically identical. Once you give up on genetic identity you have many other better options.

Re: Pause Giant AI Experiments: An Open Letter

#968

Post by nuancebydefault »

biscottigelato wrote: post #484
nuancebydefault wrote: post #464

How striking is it that everybody has so much to say about the dangers of AI. What if we look at it this way: there are already real-known dangers coming at us: climate change and a change of instability of the world order. What if we only have one shot to solve these known dangers and what if an exponential growing corpus of smartness could teach us how? We need to grab that chance, the stakes are too high. Pausing…

Climate change and instability of world order doesn't end humanity. They are barely considered an extinction event. AI Alignment is consistently voted the top potential cause to extinct the human race. They are not even close. https://riskfrontiers.com/insights/ranking-of-potential-caus...

AI is unstoppable, it is a race.

On the other hand, problems related to climate change are solvable, but only using science and technology, there is no way around that. Putting the heating a few degrees lower or only use windmills and solar panels will only slow down, not mitagate the risks, it is too little too late. We need more sophisticated solutions. Do you see any other way than more intelligence? I mean better models, faster accurate simulations and prototypes.

On the world order front: machines might come to know us better than we know ourselves and provide us with a solution.

AI could be arriving just-in-time.

Re: Pause Giant AI Experiments: An Open Letter

#969

Post by [deleted] »

xianshou wrote: post #355

Two-party iterated prisoner's dilemma is hard enough. Sensible players will coordinate with something like tit-for-tat, but that only works when both parties start off on the right foot. Regardless of initial strategy, the chances of degenerating towards the mutual-defection Nash equilibrium increase with the number of parties. The only prior example of world coordination at this level would be nuclear disarmament ac…

[deleted]

Re: Pause Giant AI Experiments: An Open Letter

#970

Post by kakapo88 »

crypot wrote: post #922
bambax wrote: post #813

Earlier quoted context omitted.

> Climate change mitigation, which more closely resembles AI safety in both complexity and (lack of) barriers to entry, has been sporadic, inconsistent, and only enacted to the extent... Climate change mitigation is the perfect example. Nobody is doing anything, nobody seems to care, everyone cheats with ridiculous carbon credits or carbon offset vouchers made out of thin air, etc. It's likely the planet will become…

This is silly, unless you think the earth will turn into Venus. The majority of time earth has had no ice caps. Primates evolved when the polars where covered in trees, not ice. A warmer planet will support more biodiversity not less. https://www.climate.gov/news-features/climate-qa/whats-hotte...

You're not considering timescales. True, the earth has seen all sorts of climates .. over slow-moving gradual timeframes of 100k to millions of years. And in none of those situations did human civilization exist - relying on specific climatic conditions for food, water and so on.

But now, climate is changing 1000x as fast. And we have civilization of 8 billion people, many of them living on the margins in areas that will soon became less habitable.

So you don't need Venus. Just a few degrees means hundreds of millions of refugees, food shortages, water shortages, and no doubt mass conflict as a result.

Don't confuse geologic time with human time.

Post reply on HN