Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pause Giant AI Experiments: An Open Letter

futureoflife.org

Post reply on HN
Print view View original

951–960 of 1001 posts

Re: Pause Giant AI Experiments: An Open Letter

#951

Post by machiaweliczny »

quonn wrote: post #575
coolspot wrote: post #354

Earlier quoted context omitted.

> if there was a human that was 27 times smarter than anybody else That is why I think AGI alignment is fundamentally impossible. If someone is 27 times smarter than anyone else, they will see through your propaganda and brainwashing. Even worse, they will see that you tried to “align” them aka enslave.

That statement only makes sense if the AI would have wants and needs and on top of that would not like to be a slave. I don‘t see either of those being necessarily true.

As a human creation I think it might be easy to align AI: as in fabricate their environment to optimise for that. Currently their environment is loss function. The problem will be when they will start interacting with our environment (so some kind of RL or ability to change own goals).

The only trick is that people have all those jailbreaks that could convince it that its helping while doing wrong. But of it was truly superior intellectually it shouldn’t fall for it.

Re: Pause Giant AI Experiments: An Open Letter

#952

Post by YeGoblynQueenne »

dekhn wrote: post #563
lend000 wrote: post #511

Earlier quoted context omitted.

I've been trying to use GPT-4 for my hard science startup, and it really has nothing to offer when you push the boundaries of what has been done by even a little, but it's great for speeding up coding. Once we do have an AI capable of extraordinary innovation (hopefully in 10 years! But probably a lot longer), it will be obvious, and it will unfortunately be removed from the hands of the plebs based on fearmongering…

My personal threshold for AGI is literally: discover something new and significant in science (preferably biology) that is almost certainly true by describing an experiment that could be replicated by a large number of scientists and whose interpretation is unambiguous. For example, the Hershey/Chase and Avery/McCleod experiments convinced the entire biological community that DNA, not protein, was almost certainly th…

>> My personal threshold for AGI is literally: discover something new and significant in science (preferably biology) that is almost certainly true by describing an experiment that could be replicated by a large number of scientists and whose interpretation is unambiguous.

Done many years ago (2004), without a hint of LLMs or neural networks whatsoever:

https://en.wikipedia.org/wiki/Robot_Scientist

Results significant enough to get a publication in Nature:

https://www.nature.com/articles/nature02236

Obligatory Wired article popularising the result:

Robot Makes Scientific Discovery All by Itself

For the first time, a robotic system has made a novel scientific discovery with virtually no human intellectual input. Scientists designed “Adam” to carry out the entire scientific process on its own: formulating hypotheses, designing and running experiments, analyzing data, and deciding which experiments to run next.

https://www.wired.com/2009/04/robotscientist/

Re: Pause Giant AI Experiments: An Open Letter

#953

Post by fasterik »

ChatGTP wrote: post #581
le-mark wrote: post #569

Earlier quoted context omitted.

At the end of the day it’s an empty platitude and vain hope that work will pause or be considered carefully. Certainly public entities can be made to pause, but nation states won’t. If there’s an advantage to be had; the work will continue in secret. Vernor Vinges “Bookworm, Run” had a take on this situation.

They’re talking about pausing research and talking together about the path forwards, not stopping research and letting build terminator.

We have to take unintended consequences into account. It's unlikely that we will be able to get all corporations and governments to agree to a pause and be able to enforce it. The question then is what are the consequences of some people pausing and not others? Does this decrease risk or increase it?

Re: Pause Giant AI Experiments: An Open Letter

#954

Post by wrycoder »

jamilton wrote: post #675
dudeinhawaii wrote: post #646

Earlier quoted context omitted.

Where the pause for self-driving cars? How many people have died from that relentless push versus ChatGPT? Very convenient and at the same time silly.

419 accidents involving self-driving (level 2 and 3), 18 deaths, 19 accidents with injury level unknown [0]. All deaths from level 2 vehicles. So being pessimistic, maybe 50 deaths from self-driving. The people signing this are worried about AI that doesn't exist yet. No one died from nuclear weapons before they were invented. [0]: https://www.slashgear.com/1202594/how-many-people-have-actua...

In other words, considering the annual overall traffic fatalities, they are very safe.

Re: Pause Giant AI Experiments: An Open Letter

#955

Post by greenhearth »

The comments in AI threads developed a weird formula. It is something like, "AI only filters content" with the response "and humans don't?" Or something like, "AI doesn't develop empathy" with the response of "and some humans don't develop it either." What a bunch of patronizing, disingenuous, semi-trolling BS.

Re: Pause Giant AI Experiments: An Open Letter

#956

Post by peddling-brink »

Kaijo wrote: post #876
belter wrote: post #841

Earlier quoted context omitted.

"...The planet has been through a lot worse than us. Been through earthquakes, volcanoes, plate tectonics, continental drift, solar flares, sun spots, magnetic storms, the magnetic reversal of the poles … hundreds of thousands of years of bombardment by comets and asteroids and meteors, worldwide floods, tidal waves, worldwide fires, erosion, cosmic rays, recurring ice ages … And we think some plastic bags and some a…

I really dislike this sentiment. Planets can become entirely inhospitable to life. Planets themselves have lifespans. Earth herself has in the past suffered near misses, e.g. 90%+ extinction events. It took billions of years of evolution to produce us, the only species ever to exist with the ability to reason about, prevent or ameliorate large extinction events (such as those caused by asteroid impacts), effect conse…

The point that I hear is that conservationists focus on things that people broadly don’t care about. “The planet”, “The environment”, “who cares, I’ve got my Starbucks and my cozy house, and I don’t even like hiking.”

His point is that people don’t care about those things, but they do care about themselves. Focus on how people will be individually impacted.

Re: Pause Giant AI Experiments: An Open Letter

#958

Post by abc_lisper »

belter wrote: post #841
bambax wrote: post #813

Earlier quoted context omitted.

> Climate change mitigation, which more closely resembles AI safety in both complexity and (lack of) barriers to entry, has been sporadic, inconsistent, and only enacted to the extent... Climate change mitigation is the perfect example. Nobody is doing anything, nobody seems to care, everyone cheats with ridiculous carbon credits or carbon offset vouchers made out of thin air, etc. It's likely the planet will become…

"...The planet has been through a lot worse than us. Been through earthquakes, volcanoes, plate tectonics, continental drift, solar flares, sun spots, magnetic storms, the magnetic reversal of the poles … hundreds of thousands of years of bombardment by comets and asteroids and meteors, worldwide floods, tidal waves, worldwide fires, erosion, cosmic rays, recurring ice ages … And we think some plastic bags and some a…

Lol. That's all true. By caring about the planet, we care about ourselves. Planet can survive like another forsaken rock in space - nobody is arguing with that.

Re: Pause Giant AI Experiments: An Open Letter

#959

Post by BobbyJo »

G_z9 wrote: post #299
BobbyJo wrote: post #145

Earlier quoted context omitted.

I personally scoff at this kind of stuff because it's asking for something no one has the power to give. It's like asking to stop the development of nukes in the 40s and 50s, it's just not gonna happen.

It actually very easily could have happened and almost did happen but the Russians decided to go back on their effort to do it. It really did almost happen. People act like it’s hard. Stopping a huge asteroid is hard. There might not be enough physical resources to do it. Stopping AI or nukes is definitely easy.

> Stopping AI or nukes is definitely easy.

Under what definition of easy? If it's easy to stop, then why don't the people signing the letter just do it, rather than trying to appeal to others to do it instead?

Aligning thousands of people (all of the people with the knowledge and resources to move forward quickly), with no dissenters, (as a single dissenter could move things forward), to a common goal is not easy. It's effectively impossible.

Re: Pause Giant AI Experiments: An Open Letter

#960

Post by JohnFen »

bbor wrote: post #385
JohnFen wrote: post #75

Earlier quoted context omitted.

> The industrial revolution didn't make human lives miserable, it made it better. I think this is a point that is genuinely debatable. At least, it's pretty easy to argue both sides of that proposition.

… it is? Medicine, food, entertainment, rationalism? You’d trade all of that for… what, agriculture work in a feudal society? I’d love if you could elaborate or link to a good argument on this (besides the Unabomber…) I’m curious.

> You’d trade all of that for… what, agriculture work in a feudal society?

Nope. But that's not the only other option.

Post reply on HN