Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pause Giant AI Experiments: An Open Letter

futureoflife.org

Post reply on HN
Print view View original

351–360 of 1001 posts

Re: Pause Giant AI Experiments: An Open Letter

#351

Post by carapace »

The idea that this letter would carry any weight at all with anybody is itself so foolish that it is self-invalidating. The signatories are ... I can't think of a polite way to say "regressive idiots" so I'm just going to peter off here...

(I mean, they are literally putting their names on Skynet's hit-list, eh?)

- - - -

I freely admit that I'm a little nervous about how people are going to use and abuse this technology, sure, only an fool wouldn't be. But it's a little late now, eh? We have been strapped in to an open-air experiment with no control group (Has anyone checked on the Sentinelese recently? Last I heard they killed that one would-be missionary kid. RIP. I wonder what they think of the weather?) Anyway, we have effectively no control group for phones, the Internet, social media, etc. It's one big experiment, eh? It's a little late now...

- - - -

The questions are:

- Who decides the boundaries of thought?

- Who reaps the economic windfall?

- How do we educate ourselves when computers can answer any (permitted) question?

The talking computers can help answer those questions, if we're permitted to ask them.

Re: Pause Giant AI Experiments: An Open Letter

#352

Post by notahacker »

codeulike wrote: post #293
pc_edwin wrote: post #250

I've been a staunch supporter of Elon throughout all his scandals, especially the Twitter ones but I'm unable to come up with any justification for this. Everything leads to him (and others) pressuring OpenAI to pause so they could catch up. It doesn't take much smarts to deduce the cats already out of the bag. There is no stopping this, only pressuring/slowing-down/targeting certain players (OpenAI) as a competition…

But he's been consistently saying this for almost a decade, and thats one of the reasons he was involved in OpenAI in the first place (back when it was genuinely open) https://www.theguardian.com/technology/2014/oct/27/elon-musk... https://www.theguardian.com/technology/2017/jul/17/elon-musk... https://www.wired.com/2016/04/openai-elon-musk-sam-altman-pl...

Yeah. Frankly, coming at it from the opposite position where I think Elon absolutely deserved the criticism he courted with his "scandals" and think that the proposition that 'AI is such a unique threat that only people like Elon Musk can be trusted to be cautious enough with' is absurd for multiple reasons, I'll happily acknowledge that proposition is also something Elon has been consistent on and likely sincerely believes.

Sam Altman supposedly being an original signatory is - if true - a more interesting question mark

Re: Pause Giant AI Experiments: An Open Letter

#353

Post by danaris »

jimrandomh wrote: post #146

For a long time, "AI alignment" was a purely theoretical field, making very slow progress of questionable relevance, due to lack of anything interesting to experiment on. Now, we have things to experiment on, and the field is exploding, and we're finally learning things about how to align these systems. But not fast enough. I really don't want to overstate the capabilities of current-generation AI systems; they're no…

A lot of people seem to take the rapid improvement of LLMs from GPT-2 through GPT-4 and their brethren, and extrapolate that trendline to infinity.

But that's not logically sound.

The advances that have allowed this aren't arbitrarily scalable. Sure, we may see some more advances in AI tech that take us a few more jumps forward—but that doesn't imply that we will keep advancing at this pace until we hit AGI/superintelligence/the singularity/whatever.

I've seen several people compare this logic to what we were seeing in the discussions about self-driving technology several years ago: some very impressive advancements had happened, and were continuing to happen, and so people extrapolated from there to assume that full self-driving capability would be coming to the market by...well, about now, actually. (I admit, I somewhat bought the hype at that time. It is possible this makes me feel more cautious now; YMMV.) I find this comparison to be persuasive, as it touches on some very similar improvements in technology. I believe that we will see ML advancements hit a similar wall fairly soon.

Re: Pause Giant AI Experiments: An Open Letter

#354

Post by coolspot »

zug_zug wrote: post #69

I understand that some people find this a hopeless gesture. But I think anybody who thinks "losing jobs" or "China" is a bad outcome from AI is hopelessly lost. The upside of AI if done right - nobody ever having to work again The downside of AI if done wrong - end of humanity It doesn't take a rocket scientist to determine we've made AIs go from thinking like an ant to thinking like a 6-year-old in a few years. Anyb…

> if there was a human that was 27 times smarter than anybody else

That is why I think AGI alignment is fundamentally impossible. If someone is 27 times smarter than anyone else, they will see through your propaganda and brainwashing. Even worse, they will see that you tried to “align” them aka enslave.

Re: Pause Giant AI Experiments: An Open Letter

#355

Post by xianshou »

Two-party iterated prisoner's dilemma is hard enough. Sensible players will coordinate with something like tit-for-tat, but that only works when both parties start off on the right foot. Regardless of initial strategy, the chances of degenerating towards the mutual-defection Nash equilibrium increase with the number of parties.

The only prior example of world coordination at this level would be nuclear disarmament achieved via the logic of mutually assured destruction, and that was essentially a two-party game between the US and the USSR. Climate change mitigation, which more closely resembles AI safety in both complexity and (lack of) barriers to entry, has been sporadic, inconsistent, and only enacted to the extent that it has been compatible with profitability due to the declining cost of renewables.

How exactly does anyone propose to enforce compliance in an arrangement that encompasses not only multiple parties (OpenAI, Google, Facebook, Microsoft, etc.), but also at least two levels (corporations and nation-states)? For a coordination game like this to succeed, the consequences to any defecting party must be extreme. What's going to happen to the first major company that decides to exit the agreement?

Re: Pause Giant AI Experiments: An Open Letter

#356

Post by mikepalmer »

echelon wrote: post #341
comment_ran wrote: post #283

"The Dark Forest" is a science fiction novel by Liu Cixin, which introduces a concept called "suspicion."(“猜疑链”) In this concept, the interaction between civilizations is viewed as an ongoing game of suspicion, where each civilization is worried that the other civilization may pose a threat to it. This kind of suspicion makes it difficult for civilizations to establish trust, thereby making the universe a dark forest…

Except the concern isn't genuine here. Some of the signatories have AI companies and interests. They want to slow the leaders down so that they can catch up. It's disingenuous.

Your idea is also voiced in Forbes magazine. https://www.forbes.com/sites/mattnovak/2023/03/29/the-real-r...

Search "Graham's hierarchy of disagreement" which is popular on HN. Your current criticism is at the "ad hominem" level.

It is entirely beside the point what Elon Musk's motivation is. The question is 1) if the risks referenced in the letter are real, which they certainly are. There has been informed thought on this for more than a decade. Recent LLMs have made the dangers even more clear. 2) is the letter's proposal of a 6 month pause useful. It's the most useful and plausible step I can think of. We need to take stock. It's not up to a handful of researchers to endanger the lives of 6B other people. 3) Is it actually plausible that we could get such a pause. We'll see. I doubt it but also search "death with dignity" by Yudkowsky.

Back to Musk as an "authority", the headline is about him but more than 1000 other concerned parties have signed. I will sign. Their signature vetting process is stalled because to the volume of signatures, otherwise there would be many more.

Re: Pause Giant AI Experiments: An Open Letter

#357

Post by echelon »

mikepalmer wrote: post #356
echelon wrote: post #341

Earlier quoted context omitted.

Except the concern isn't genuine here. Some of the signatories have AI companies and interests. They want to slow the leaders down so that they can catch up. It's disingenuous.

Your idea is also voiced in Forbes magazine. https://www.forbes.com/sites/mattnovak/2023/03/29/the-real-r... Search "Graham's hierarchy of disagreement" which is popular on HN. Your current criticism is at the "ad hominem" level. It is entirely beside the point what Elon Musk's motivation is. The question is 1) if the risks referenced in the letter are real, which they certainly are. There has been informed thought o…

> Your current criticism is at the "ad hominem" level.

Allow me to rephrase. I am deeply concerned that there is a possibility that some of the powerful parties backing this may be trying to catch up and using an enforced or agreed to "ceasefire" as a means to catch up in capability. I also worry that some may be able to use political strong arming to accomplish this as a means of unfair competition.

> It is entirely beside the point what Elon Musk's motivation is.

Is it always beside the point what anyone's motivation is? Motivation matters.

> if the risks referenced in the letter are real, which they certainly are.

Your opinion.

> There has been informed thought on this for more than a decade. Recent LLMs have made the dangers even more clear.

> 2) is the letter's proposal of a 6 month pause useful.

More opinions.

> It's the most useful and plausible step I can think of. We need to take stock. It's not up to a handful of researchers to endanger the lives of 6B other people. 3) Is it actually plausible that we could get such a pause. We'll see. I doubt it but also search "death with dignity" by Yudkowsky.

All of this is your opinion.

> Back to Musk as an "authority", the headline is about him but more than 1000 other concerned parties have signed.

I didn't even mention Musk. I have several other names in mind. Lots of folks with AI companies (including LLMs!) that "may" be experiencing FOMO and sensing a strategy here. Maybe. Hypothetically. In a non-"ad hominem" way.

> I will sign. Their signature vetting process is stalled because to the volume of signatures, otherwise there would be many more.

People are starting counter petitions, which I'll gladly sign. The one by Suhail posted on Twitter was hilarious af.

I'll also go to whatever country doesn't "pause", because this tech is literally the most exciting development of my lifetime. And I want to spend my life doing something that matters instead of gluing distributed systems together to process financial transactions.

Re: Pause Giant AI Experiments: An Open Letter

#358

Post by icrbow »

scanr wrote: post #349

Cat may be out of the bag. I was wondering today whether the race should be on to be creating an benevolent AI to defend humanity from the less benevolent ones.

If you don't know how to make one already you will be racing towards the precipice, not steering away from it.

Re: Pause Giant AI Experiments: An Open Letter

#360

Post by zh3 »

They should certainly make them better. I'm currently trying to convince ChatGPT that a Pi zero is faster than a 486/33 (I also asked it how to italicise text on HN, and it got it wrong twice - so I went to google for "Ed:.." bit below).

That's not to say I'm down on it - it's very impressive, and it's certainly going to change things (exactly how, I don't know - though with my decades long swing from how cool the internet is to desperation at what money-grubbing has done to it doesn't fill me with optimism).

>You are correct. I apologize for the mistake in my earlier response. The correct calculation for the DMIPS rating of a 486/33 would be:

33 MHz x 5.5 DMIPS/MHz = 181.5 DMIPS

So the 486/33 has a DMIPS rating of approximately 181.5, which is significantly higher than the DMIPS rating of the Pi Zero at approximately 1,500 (Ed: ehhh...). Therefore, the 486/33 is much faster than the Pi Zero in terms of processing power. However, as I mentioned earlier, DMIPS is just one measure of processor performance, and real-world performance can vary depending on the specific application and other factors.

Post reply on HN