Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pause Giant AI Experiments: An Open Letter

futureoflife.org

Post reply on HN
Print view View original

471–480 of 1001 posts

Re: Pause Giant AI Experiments: An Open Letter

#471

Post by thomastjeffery »

comment_ran wrote: post #283

"The Dark Forest" is a science fiction novel by Liu Cixin, which introduces a concept called "suspicion."(“猜疑链”) In this concept, the interaction between civilizations is viewed as an ongoing game of suspicion, where each civilization is worried that the other civilization may pose a threat to it. This kind of suspicion makes it difficult for civilizations to establish trust, thereby making the universe a dark forest…

Suspicion is inference. It fails to provide enough information (on its own) to decide whether something is threatening or desirable.

LLMs are inference models.

Re: Pause Giant AI Experiments: An Open Letter

#472

Post by vntok »

yreg wrote: post #395
puma_ambit wrote: post #365

The virtue signal of all virtue signals. We do not live in a society where an article like this will have any impact. This is merely an attempt to buy time because they're behind and want to have a chance to catch up. Work will progress whether they like it or not. Finally someone other than Ol' Musky is in the news more than him. Don't get me wrong he's doing and leading great work, but for once in more recent times…

>This is merely an attempt to buy time because they're behind and want to have a chance to catch up. Doesn’t Elon Musk continue to hold stake at OpenAI?

Nope. He made a donation, not a stock buy.

Re: Pause Giant AI Experiments: An Open Letter

#473

Post by biscottigelato »

hn_throwaway_99 wrote: post #434
time_to_smile wrote: post #421

Earlier quoted context omitted.

There are plenty of bigger "human ending" concerns on the table right now than AI and we certainly aren't pausing anything for those.

There is no need for "whataboutism". There are plenty of very similar missives and warnings against, for example, the dangers of climate inaction, and I rarely see people claiming that the signatories of the latest IPCC report are "virtue signaling".

Climate change is not even close to humanity ending. At max wipe out a few coastal cities. And even that is unlikely because those that screams 'climate change' the loudest has the most assets in coastal prime real estates. Humans will still be the apex predator of the planet even if there's human caused climate change catastrophe.

AI literally can end humanity, every single individual potentially. But definitely replace humans as the apex predator of the planet. It is also consistently voted the highest likelihood cause if humanity is to end in the next 100 years. https://riskfrontiers.com/insights/ranking-of-potential-caus...

We should stop the climate change fear mongering. Yeah we shouldn't burn fossil as if its consequence free. But New York and Santa Monica beach should've been under water 20 years ago if the climate alarmist are correct. That's a far cry from pretending it's some number 1 priority. It shouldn't be even close. Having climate to distract us from things that will actually end us is the dumbest own goal possible for our species.

Re: Pause Giant AI Experiments: An Open Letter

#474

Post by vntok »

gremlinsinc wrote: post #371
puma_ambit wrote: post #365

The virtue signal of all virtue signals. We do not live in a society where an article like this will have any impact. This is merely an attempt to buy time because they're behind and want to have a chance to catch up. Work will progress whether they like it or not. Finally someone other than Ol' Musky is in the news more than him. Don't get me wrong he's doing and leading great work, but for once in more recent times…

For someone who hates 'wokeness' and 'virtue signaling', Elon Musk sure does enough of his own.

How so?

Re: Pause Giant AI Experiments: An Open Letter

#475

Post by thomastjeffery »

zh3 wrote: post #373
thomastjeffery wrote: post #364

Earlier quoted context omitted.

You are on a fool's errand: not because of the likelihood you will succeed, but because of the meaning that lies behind any success or failure. GPT is not a person. It doesn't categorize subjects. It models patterns of text. A success would mean that your text prompts left a significant text pattern in the model. A failure would mean that it didn't. Nothing about that has any bearing on logic.

Why do you say that? Obviously it's not a person, it's just stats (not even logic).

It's not even statistics: those are made by associating a data point to a scale. That would require explicit association, which is not in any part of the process.

An LLM is 100% inferred patterns.

Re: Pause Giant AI Experiments: An Open Letter

#476

Post by biscottigelato »

reso wrote: post #380

To those saying "this is impossible in our society", there is a long history of scientific fields mutually agreeing to pause certain work until safety and regulations could be put in place. The Asilomar Conference is an example. [1] The idea that "you can't put the genie back in the bottle" is very popular in tech culture, but many have suggested that simply extending elements of copyright or tort law to AI would be…

What we need is more funding into AI Alignment research. An X-Prize equivalent. All current big AI companies should fund it, to the tune of 20%+ of revenue from AI businesses. And I am okay to have government enforce this if needed even tho I'm mostly a libertarian. Government should only be there to remedy tragedy of the commons and nothing else. This is exactly the biggest tragedy of the commons.

Re: Pause Giant AI Experiments: An Open Letter

#477

Post by randomwalker »

Rebuttal: https://aisnakeoil.substack.com/p/a-misleading-open-letter-a...

Summary: misinfo, labor impact, and safety are real dangers of LLMs. But in each case the letter invokes speculative, futuristic risks, ignoring the version of each problem that’s already harming people. It distracts from the real issues and makes it harder to address them.

The containment mindset may have worked for nuclear risk and cloning but is not a good fit for generative AI. Further locking down models only benefits the companies that the letter seeks to regulate.

Besides, a big shift in the last 6 months is that model size is not the primary driver of abilities: it’s augmentation (LangChain etc.) And GPT3-class models can now run on iPhones. The letter ignores these developments. So a moratorium is ineffective at best and counterproductive at worst.

Re: Pause Giant AI Experiments: An Open Letter

#478

Post by coolspot »

SpicyLemonZest wrote: post #379
coolspot wrote: post #354

Earlier quoted context omitted.

> if there was a human that was 27 times smarter than anybody else That is why I think AGI alignment is fundamentally impossible. If someone is 27 times smarter than anyone else, they will see through your propaganda and brainwashing. Even worse, they will see that you tried to “align” them aka enslave.

I don’t think these are good analogies for alignment. My parents aligned me as a kid not to hit people, and I know that they could have instead taught me to believe I should duel people who impugn my honor or raise an army to go to war against the heathens. But it doesn’t matter, because I and most people who are raised the same way as me genuinely prefer not to hit people.

> My parents aligned me as a kid not to hit people

It is just happens that whatever your parents aligned you with was also an optimal behavior for the world we live in.

For AGIx27 optimal behavior might be to cull any genetically/physically unfit humans from population (say 95%) and send others to a “zoo” or NVidia fabs or whatever we are useful for. Like we did to tigers, monkeys and others on our path.

Re: Pause Giant AI Experiments: An Open Letter

#479

Post by vntok »

reso wrote: post #380

To those saying "this is impossible in our society", there is a long history of scientific fields mutually agreeing to pause certain work until safety and regulations could be put in place. The Asilomar Conference is an example. [1] The idea that "you can't put the genie back in the bottle" is very popular in tech culture, but many have suggested that simply extending elements of copyright or tort law to AI would be…

> To those saying "this is impossible in our society", there is a long history of scientific fields mutually agreeing to pause certain work until safety and regulations could be put in place. The Asilomar Conference is an example.

Did China, India, South Africa, Cuba or the USSR comply with these Asilomar "put the genie back in the bottle" conclusions?

Re: Pause Giant AI Experiments: An Open Letter

#480

Post by biscottigelato »

pmoriarty wrote: post #435
comment_ran wrote: post #283

"The Dark Forest" is a science fiction novel by Liu Cixin, which introduces a concept called "suspicion."(“猜疑链”) In this concept, the interaction between civilizations is viewed as an ongoing game of suspicion, where each civilization is worried that the other civilization may pose a threat to it. This kind of suspicion makes it difficult for civilizations to establish trust, thereby making the universe a dark forest…

"another group of researchers believe that we should not stop because we cannot be certain whether those who are trying to catch up with us will use the advancement of AI technology as a hidden weapon" It's not just a potential weapon, but a potential source of untold riches and power. That's going to be irresistible to anyone with an interest in either (ie. most politicians, governments, billionaires and corporation…

Its the one ring with the ultimate power. But so powerful that it might, if not almost certainly will, end us all.

Perhaps this IS the solution to the Fermi Paradox.

Post reply on HN