Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pause Giant AI Experiments: An Open Letter

futureoflife.org

Post reply on HN
Print view View original

621–630 of 1001 posts

Re: Pause Giant AI Experiments: An Open Letter

#621

Post by ddod »

lend000 wrote: post #481

If intelligence is all you need to dominate the world, why do some of the most powerful world leaders seem to not be more than a standard deviation above average intelligence (or at least they were before they became geriatric)? Add in the fact that GPT-4 does not "think" very fast, has no arms or legs, needs to run on expensive specialized hardware, and essentially seems like a search algorithm that lacks creativity…

It's naive to think the danger is in self-aware evil AI. AI is a tool, and can be used as a weapon.

There's a functional power difference between a knife, an assault rifle, and a nuke. Everyone owns knives--they're in your kitchen--they can't cause much trouble. Access to assault rifles has turned out to be a dangerous problem to society, and is a contentious issue as such. Nukes are obviously too dangerous for people to own willy-nilly.

The issue is where along that spectrum AI will fall into in 6mo, 1yr, 5yrs, etc.

It's troubling to think anyone, especially on a technical forum like HN, believes GPT-4 doesn't have arms or legs (anyone can hook it up to any machinery/robotics) or can't think very fast. Its training set is a good chunk of human knowledge, and it outperforms most humans already.

Even if you assume AI won't be very clever (which seems unwise given its development pace), consider just its ability to perform thoughts and actions at the speed of a computer compared to a human. There are quite a few examples of modern military engagements where a larger, weaker, and less-well-trained force overcame better-equipped opponents, like Mogadishu, Vietnam, Iraq, Afghanistan, etc.

Re: Pause Giant AI Experiments: An Open Letter

#622

Post by fallous »

Ignoring the distaste I hold for precautionary principle arguments, the impracticality of the recommended solutions is laughable.

The genie is out of the bottle regarding LLMs thanks to the public availability of Llama and the related support data necessary for any individual to run them on sub-$5K hardware. The computing costs of engaging in training and modifying is within the realm of a modestly wealthy individual or group now, no longer requiring the budgets of large commercial or public institutions/government. The "undo" button is disabled.

If you wish to have some organized response I'd suggest taking the time and effort to educate the public on the realities of what these technologies can, and more importantly cannot, do. As it stands now you have all manner of media and commentators breathlessly announcing that LLMs are smarter than people and can completely replace human workers because they can carry on coherent sentences... but can't reliably do basic arithmetic.

Spend your energy on trying to provide useful explanations of how these technologies work, what is the current state of the art, what the strengths and weaknesses are, and dispel the hyperventilated fantasizing and bullshit currently polluting the public discourse on the subject.

I'm much less worried about LLMs than I am the people who will be rushing to deploy them because they're the new hotness without any clue regarding the appropriateness of the tech for particular problems and then being shocked at the consequences.

Re: Pause Giant AI Experiments: An Open Letter

#623

Post by qgin »

nice_byte wrote: post #492
petilon wrote: post #68

> Should we automate away all the jobs, including the fulfilling ones? You don't necessarily need jobs because you don't need money. What you need is wealth. I am going to leave it to PG to explain the difference [1]: Wealth is not money. Wealth is stuff we want: food, clothes, houses, cars, gadgets, travel to interesting places, and so on. You can have wealth without having money. If you had a magic machine that cou…

> In the next 10 years, AI/robots will generate wealth at an unprecedented scale. Food, clothing and shelter will be plentiful. Anyone who believes in the possibility of post-scarcity society must be either naive or trolling. Something cannot be made from nothing, therefore scarcity cannot be overcome, even assuming that all planning and execution is performed by superhuman synthetic minds. Assuming that it's theoret…

Sounds like fuel for a revolution. I know some governments will go to war with their own people, but all of them?

There will be some countries that go the way you've described and crush any discontent with force. But some will try to navigate an outcome that socializes the benefits of automation.

Re: Pause Giant AI Experiments: An Open Letter

#624

Post by jongjong »

I think we shouldn't. We should encourage as many organizations build AIs as possible. Then they will spam the web to extinction (dead internet theory will be real) and we will all be better for it.

Then we can rebuild the internet properly and in a decentralized way with blockchain-based authentication.

Re: Pause Giant AI Experiments: An Open Letter

#625

Post by bloqs »

lend000 wrote: post #481

If intelligence is all you need to dominate the world, why do some of the most powerful world leaders seem to not be more than a standard deviation above average intelligence (or at least they were before they became geriatric)? Add in the fact that GPT-4 does not "think" very fast, has no arms or legs, needs to run on expensive specialized hardware, and essentially seems like a search algorithm that lacks creativity…

The qualities of being above average in intelligence but fabrically incapable of exhaustion and functionally immortal would be an excellent start for world domination

Re: Pause Giant AI Experiments: An Open Letter

#626

Post by [deleted] »

ChatGTP wrote: post #581
le-mark wrote: post #569

Earlier quoted context omitted.

At the end of the day it’s an empty platitude and vain hope that work will pause or be considered carefully. Certainly public entities can be made to pause, but nation states won’t. If there’s an advantage to be had; the work will continue in secret. Vernor Vinges “Bookworm, Run” had a take on this situation.

They’re talking about pausing research and talking together about the path forwards, not stopping research and letting build terminator.

[deleted]

Re: Pause Giant AI Experiments: An Open Letter

#627

Post by int_19h »

lanstin wrote: post #613
thomastjeffery wrote: post #302

The dumb criticize the blind. What an absurd situation! How did we get here? Here are the steps: 1. Large Language Models have been presented as "AI", which personifies them instead of describing how they work. 2. Goals for LLM development were set for the personified attributes, and not the actual functionally of the real thing. OpenAI brags about how GPT4 scores at human tests: as if that has any bearing on the mod…

I feel more sympathy for that Google engineer that feel in love with their LLM. I am sure more people will become attached, at least once the rate limits are relaxed.

The rea trap is that LLMs can simulate empathy surprisingly well. If you have many problems to rant about but nobody willing to listen, the LLM is always there, it will never get bored or tell you to shut up, and it will always respond in encouraging and "positive" ways. Given how many people today do not have anyone like that in their lives, it's no wonder that they form an emotional bond.

Re: Pause Giant AI Experiments: An Open Letter

#629

Post by pfdietz »

illiarian wrote: post #592
xianshou wrote: post #355

Two-party iterated prisoner's dilemma is hard enough. Sensible players will coordinate with something like tit-for-tat, but that only works when both parties start off on the right foot. Regardless of initial strategy, the chances of degenerating towards the mutual-defection Nash equilibrium increase with the number of parties. The only prior example of world coordination at this level would be nuclear disarmament ac…

> The only prior example of world coordination at this level would be nuclear disarmament achieved via the logic of mutually assured destruction Or eradication of infectious diseases such as polio

Or the banning of CFCs and other ozone depleting substances.

It helped that the benefit/cost ratio was enormous.

Re: Pause Giant AI Experiments: An Open Letter

#630

Post by swader999 »

ChatGTP wrote: post #577
JamesBarney wrote: post #410

Earlier quoted context omitted.

How much do you think Asilomar slowed down DNA research? Do you think without it we'd more more than 10 year ahead? I doubt it. Optimistically we'd maybe have 6 months to a year of additional progress in those fields without it. And this would only slow it down in a handful of countries. That's not going to keep China from trying to build AGI. If we are in a world where building an AGI is possible and near, I trust O…

It’s dangerous parrot talking to keep saying “China”. I mean we really have no idea about their thoughts on it because we’re clearly not talking to them. China has good people living there too you know ?

China has amazing peeps. CCP less so.
Post reply on HN