Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pause Giant AI Experiments: An Open Letter

futureoflife.org

Post reply on HN
Print view View original

721–730 of 1001 posts

Re: Pause Giant AI Experiments: An Open Letter

#721

Post by DonHopkins »

comment_ran wrote: post #283

"The Dark Forest" is a science fiction novel by Liu Cixin, which introduces a concept called "suspicion."(“猜疑链”) In this concept, the interaction between civilizations is viewed as an ongoing game of suspicion, where each civilization is worried that the other civilization may pose a threat to it. This kind of suspicion makes it difficult for civilizations to establish trust, thereby making the universe a dark forest…

That might explain why the Vorlons of Babylon 5 were such assholes.

https://babylon5.fandom.com/wiki/Vorlon

>Distant Past

>The Vorlons were among the first races in the galaxy to attain sentience and become space-faring creatures. In the distant past, the Vorlons attempted a grand experiment into reaching a higher plane of existence where the souls existed and all life was founded. To accomplish this, they poured all their knowledge into creating a gateway to this dimension. After activating the gate, the vortex into Thirdspace was formed and the aliens within that realm came pouring out, intent on destroying all other forms of life. After much hardship, the Vorlons finally managed to push the enemy back, but one of the gateways was lost in Hyperspace where it was hidden from the Vorlons. They would encode some of their servants with an automated message to warn the younger races of the threat from the inhabitants of Thirdspace.

Re: Pause Giant AI Experiments: An Open Letter

#722

Post by nopinsight »

gautamdivgi wrote: post #719
nopinsight wrote: post #695

Earlier quoted context omitted.

A Manhattan project on AI Alignment, if started now, might still succeed in time. Therefore, the compliance between parties needs not be long-term, which is indeed unlikely to happen. China, which is the country outside the west with the highest (engineering) capability to train something more powerful than GPT-4, is very concerned about domestic stability and they also do not want an easily replicable alien tool wit…

I would hazard that any country with a declining population will aggressively adopt LLMs because it gives their leaner workforce more efficiency.

The whole economy might benefit but individual voters often don’t. The groups most likely to be displaced, non-senior white collar office workers, are quite large and vocal.

Re: Pause Giant AI Experiments: An Open Letter

#723

Post by bruce511 »

alex_young wrote: post #712
xianshou wrote: post #355

Two-party iterated prisoner's dilemma is hard enough. Sensible players will coordinate with something like tit-for-tat, but that only works when both parties start off on the right foot. Regardless of initial strategy, the chances of degenerating towards the mutual-defection Nash equilibrium increase with the number of parties. The only prior example of world coordination at this level would be nuclear disarmament ac…

Arguably there are other (better) examples. Off the top of my head: * CFC elimination (saved the ozone layer) * Leaded gas ban (well, except private aircraft, but they are working on that)

I'll toss in whaling as well.

But these are still bad examples,because they are all bad-at-high-volume activities, with minimal paths to scaling or profitability for those who persist.

Nuclear weapons are not profitable at any scale. There are CFC alternatives so remaining on CFC production is low-scale niche market stuff. Unleaded gas costs the same as leaded gas.

AI though is different.if one company or country developed AI and the others paused, then a long-term, highly scalable and stunningly profitable advantage will accrue.

That's not something that can easily be halted.

This page could have a billion signatures,but it would mean nothing. We live in an age of trillion $ companies. We've seen Apple, Google, Microsoft, Amazon - companies that can go from zero to trillions in a couple decades.

The closest real comparison is gene research, gene editing, cloning. There are some bans in that field, but research and development simply proceeds around those bans. Montsano has shown how much money there is in genes. Imagine the profits of the first company to use gene editing to cure childhood leukemia.

Re: Pause Giant AI Experiments: An Open Letter

#724

Post by makeitdouble »

alex_young wrote: post #712
xianshou wrote: post #355

Two-party iterated prisoner's dilemma is hard enough. Sensible players will coordinate with something like tit-for-tat, but that only works when both parties start off on the right foot. Regardless of initial strategy, the chances of degenerating towards the mutual-defection Nash equilibrium increase with the number of parties. The only prior example of world coordination at this level would be nuclear disarmament ac…

Arguably there are other (better) examples. Off the top of my head: * CFC elimination (saved the ozone layer) * Leaded gas ban (well, except private aircraft, but they are working on that)

These two issues have very specific advantages:

- simple to explain / rally around

- low economic impact (a single country wouldn't get tremendous advantage by keeping using cfc for instance)

- once the technology is developped there's little upside in going back

We'd need to recreate those to have stable progress on the other issues.

Re: Pause Giant AI Experiments: An Open Letter

#725

Post by kleer001 »

olddustytrail wrote: post #375
kleer001 wrote: post #339

Earlier quoted context omitted.

Hmm, sounds like parties not fully grokking the optimal (not thinly logical) solution to the iterated prisoners' dilemma. I saw the same thought process is the otherwise very nice novel "Time's Children". Good for drama, but not thought through.

If your scenario involves all parties "fully grokking the optimal solution to the iterated prisoners' dilemma" I suspect it's yourself who has not thought things through. It only takes one.

No, both parties need to cooperate.

Re: Pause Giant AI Experiments: An Open Letter

#726

Post by chuddington »

engineeringwoke wrote: post #439
w0m wrote: post #414

Earlier quoted context omitted.

OPs point being Musk is trying to stall AI because it's taking 'his' headlines.

Somebody else has a shiny new toy and he's jealous

Or he genuinely thinks it could be a bad thing for humanity. Could you put your hatred of him aside for a sec and acknowledge that possibility?

Re: Pause Giant AI Experiments: An Open Letter

#727

Post by kleer001 »

mclightning wrote: post #344
kleer001 wrote: post #339

Earlier quoted context omitted.

Hmm, sounds like parties not fully grokking the optimal (not thinly logical) solution to the iterated prisoners' dilemma. I saw the same thought process is the otherwise very nice novel "Time's Children". Good for drama, but not thought through.

I have so many questions >> what does it mean to fully grok the optimal? >> what is the iterated prisoners dilemma?

something in there should be grokable

best imho, 10 min and done: https://www.youtube.com/watch?v=BOvAbjfJ0x0

longer and more over produced: https://radiolab.org/episodes/104010-one-good-deed-deserves-...

totally academic : https://onlinelibrary.wiley.com/doi/10.1111/j.1420-9101.2011...

Re: Pause Giant AI Experiments: An Open Letter

#728

Post by ericb »

jnwatson wrote: post #716
nopinsight wrote: post #695

Earlier quoted context omitted.

A Manhattan project on AI Alignment, if started now, might still succeed in time. Therefore, the compliance between parties needs not be long-term, which is indeed unlikely to happen. China, which is the country outside the west with the highest (engineering) capability to train something more powerful than GPT-4, is very concerned about domestic stability and they also do not want an easily replicable alien tool wit…

AI alignment is a philosophy problem, not an engineering one. For alignment to happen, we have to agree what it means. Given we have a hard enough time getting humans to “align”, I can’t imagine any successful attempt at alignment sort of complete castration.

Are there degrees of alignment? I'd like to think there's a pretty big range in there between made some decisions I didn't love and destroyed the world and everyone on it.

Re: Pause Giant AI Experiments: An Open Letter

#729

Post by thomastjeffery »

int_19h wrote: post #627
lanstin wrote: post #613

Earlier quoted context omitted.

I feel more sympathy for that Google engineer that feel in love with their LLM. I am sure more people will become attached, at least once the rate limits are relaxed.

The rea trap is that LLMs can simulate empathy surprisingly well. If you have many problems to rant about but nobody willing to listen, the LLM is always there, it will never get bored or tell you to shut up, and it will always respond in encouraging and "positive" ways. Given how many people today do not have anyone like that in their lives, it's no wonder that they form an emotional bond.

Empathy is what language is for. It's language itself that is doing the empathy: and that isn't simulated at all. It was all written by people.

All an LLM can do is follow the structural patterns, and use them to shuffle.

Post reply on HN