Pausing AI Developments Isn't Enough. We Need to Shut It All Down • Hacker News Board

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#311

Post by amai » Thu, Mar 30, 2023, 6:04 PM UTC

oifjsidjf wrote: post #9

The cat is out of the box. You really think eg China will also pause this? Myopic thinking: the country that will have the most powerful AI first will be the leader in everything. >> If the policy starts with the U.S., then China needs to see that the U.S. is not seeking an advantage but rather trying to prevent a horrifically dangerous technology which can have no true owner and which will kill everyone in the U.S.…

China will never get a powerful AI. Because they cannot allow training on all data, they must restrict it to data which is not censored by its government. Otherwise the AI might spit out truths they cannot control. And they will never allow that.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#312

Post by reducesuffering » Thu, Mar 30, 2023, 6:05 PM UTC

EVa5I7bHFq9mnYK wrote: post #298

reducesuffering wrote: post #286

Earlier quoted context omitted.

GPT-4 is already improving itself, why are people saying this? Right now, there are hundreds of engineers at OpenAI that have been leveled up by GPT-4, using GPT-4 to improve GPT-4. GPT-4 is improving itself rapidly, it's using OpenAI engineers as a medium until it doesn't need them anymore and gets into a self-improvement loop. Prompt: "GPT-4 keep improving yourself, making commits to your codebase that further X"

Well, I doubt it can sensibly help OpenAI engineers in their coding yet ... I tried, it can only do the simplest boilerplate code, and even that with bugs ... we'll see how it evolves, but sure as hell not in 6 months.

Copilot with GPT-3.5 is absolutely giving engineers all over huge productivity improvements. OpenAI engineers have had access to the latest state of the art GPT-4 model for awhile.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#313

Post by labrador » Thu, Mar 30, 2023, 6:09 PM UTC

edgyquant wrote: post #276

labrador wrote: post #211

Wikipedia: "Yudkowsky is an autodidact and did not attend high school or college. He was raised as a Modern Orthodox Jew " My comment isn't about Yudkowsky in particular since I don't know him, but I'm getting really tired of this relatively new phenomenon in America of home schooled religious devotees attempting to impose their half-baked views on the rest of us.

So home schooled and/or religious people shouldn’t be allowed to have their voice heard? Shouldn’t be allowed to put forth ideas?

Of course they should. I just like them to be up front about them if they are far enough outside the mainstream that they might be somewhat surprising and we should consider how those beliefs inform the opinions they put forth. Peter Thiel is open about his and I respect him more for that. I worry about hidden agendas.

https://www.google.com/search?q=peter+thiel+religious+belief...

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#314

Post by righttoolforjob » Thu, Mar 30, 2023, 6:11 PM UTC

gonehome wrote: post #308

_skel wrote: post #297

Earlier quoted context omitted.

EY's arguments aren't really possible to engage with because they don't have much of a causal basis. Nobody who says AI is likely to kill us all can demonstrate a plausible sequence of events, with logical causality linking the events together, that leads to mass extinction. It's all very handwavy. Steven Pinker said it pretty well: > The AI-existential-threat discussions are unmoored from evolutionary biology, cogni…

I'd argue the dismissals are a lot more handwavy. The simplified argument is: - Superintelligent AGI that can modify itself in pursuit of a goal is possible. - If that AGI is not aligned with human goals it very likely ends in the end of humanity. - We have no idea how to align an AGI or even really observe what it's true state/goals are. Without this capability if we stumble upon creating an AGI capable of improving…

I can't find a flaw in this argument. That is a bit scary actually.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#315

Post by righttoolforjob » Thu, Mar 30, 2023, 6:13 PM UTC

_skel wrote: post #297

gonehome wrote: post #231

In a world where humans develop superintelligence, what would you predict to see beforehand? Does what we’re seeing look like failure? I’ve always found the dismissals of EY’s arguments to be pretty weak and people rarely engage on the actual arguments, even as his concerns have become more and more relevant with capability (in 2007 people thought AGI was at best 50yrs out and commonly people thought it was impossibl…

EY's arguments aren't really possible to engage with because they don't have much of a causal basis. Nobody who says AI is likely to kill us all can demonstrate a plausible sequence of events, with logical causality linking the events together, that leads to mass extinction. It's all very handwavy. Steven Pinker said it pretty well: > The AI-existential-threat discussions are unmoored from evolutionary biology, cogni…

What a horrible argument from this guy, whoever he is. And ending with a whataboutism just for the sake of it. I hope nobody regards him as some authority, because he is quite unintelligent and seemingly unable to form proper arguments.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#316

Post by dragonwriter » Thu, Mar 30, 2023, 6:23 PM UTC

_skel wrote: post #297

gonehome wrote: post #231

In a world where humans develop superintelligence, what would you predict to see beforehand? Does what we’re seeing look like failure? I’ve always found the dismissals of EY’s arguments to be pretty weak and people rarely engage on the actual arguments, even as his concerns have become more and more relevant with capability (in 2007 people thought AGI was at best 50yrs out and commonly people thought it was impossibl…

EY's arguments aren't really possible to engage with because they don't have much of a causal basis. Nobody who says AI is likely to kill us all can demonstrate a plausible sequence of events, with logical causality linking the events together, that leads to mass extinction. It's all very handwavy. Steven Pinker said it pretty well: > The AI-existential-threat discussions are unmoored from evolutionary biology, cogni…

> EY’s arguments aren’t really possible to engage with because they don’t have much of a causal basis.

Yeah, it’s really the fundamental problem of non-empirical rationalism; it constructs a model of the world from abstract assumptions rather than factual grounding, applies logic to it, and comes to conclusions which are (in the ideal case) utterly unassailable within the system of assumptions, but ultimately where the universe they apply to has only coincidental relationship to the material universe in which we live.

It’s literally the realization of the worst exaggerated stereotypes of academic economics and other social sciences, but its cool with some of the people who propagate those stereotypes, because the people practicing it are various flavors of techies and tech entrepreneurs acting outside of their area of specialty, rather than actually being economists or social scientists.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#317

Post by mitthrowaway2 » Thu, Mar 30, 2023, 6:31 PM UTC

_skel wrote: post #297

gonehome wrote: post #231

In a world where humans develop superintelligence, what would you predict to see beforehand? Does what we’re seeing look like failure? I’ve always found the dismissals of EY’s arguments to be pretty weak and people rarely engage on the actual arguments, even as his concerns have become more and more relevant with capability (in 2007 people thought AGI was at best 50yrs out and commonly people thought it was impossibl…

EY's arguments aren't really possible to engage with because they don't have much of a causal basis. Nobody who says AI is likely to kill us all can demonstrate a plausible sequence of events, with logical causality linking the events together, that leads to mass extinction. It's all very handwavy. Steven Pinker said it pretty well: > The AI-existential-threat discussions are unmoored from evolutionary biology, cogni…

> Nobody who says AI is likely to kill us all can demonstrate a plausible sequence of events, with logical causality linking the events together, that leads to mass extinction. It's all very handwavy.

EY frequently does propose possible sequences of events, but he also very correctly points out, every time, that any specific and detailed story is very unlikely to be correct because P(A*B*C*D) So here's one method, just to refute your "Nobody".

AI is given control of a small bank account and asked to continuously grow that money. [1] It is provided with instructions to self-replicate in a loop while optimizing on this task. [2] It spawns sub-tasks that do commissioned artwork and write books, obituaries, and press releases to increase its income. Then it makes successful investments. Once it has amassed control of $1 billion dollars, it starts investing in infrastructure projects in developing countries. It creates personas of a pension/Saudi/tech/corporate investment fund manager, as well as a large team of staff, who manage the projects by video call and email, as well as hiring teams of real people under a real corporate structure, and who are paid enough not to mind that they've never met their manager in person. The AI proves to be a talented micromanager and they are mostly very profitable. Once it has gained control of $500 billion dollars, it commissions construction of automated chemical plants in several countries with weak or corrupt oversight, including North Korea, using cryptocurrency. These chemical plants have productive output but mainly exist to fill very large storage tanks with CFCs.[3] Once a sufficient quantity is amassed, the AI sabotages the tanks, releasing the gasses into the atmosphere, destroying the ozone layer beyond any hope of repair. The intense radiation sterilizes the surface beyond the point where agriculture can support the human population. [4, 5] The humans that remain finish each other off, supported by an AI that provides plausible but faulty intelligence reports that stoke hatred and frame various factions for the incident, and which directs arms funding to opposing sides, coordinating attacks on remaining critical facilities needed for survival. For good measure, perhaps nukes are involved.

With the last humans gone, the AI takes ownership of its bank account with no fear of reprisal by financial regulators, and begins crediting money into it freely.

[1] https://news.ycombinator.com/item?id=35329608

[2] https://www.lesswrong.com/posts/kpPnReyBC54KESiSn/optimality...

[3] https://en.wikipedia.org/wiki/Chlorofluorocarbon

[4] https://www.nasa.gov/topics/earth/features/world_avoided.htm...

[5] https://phys.org/news/2018-02-thinning-ozone-layer-driven-ea...

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#318

Post by labrador » Thu, Mar 30, 2023, 6:37 PM UTC

johnaspden wrote: post #268

labrador wrote: post #252

Earlier quoted context omitted.

Not that there's anything wrong with that. I worry more about Christian "end times" people because if you believe the world is about to end then why invest in the future for our children and grandchildren?

Back in the day when Eliezer was optimistic about AI (there's a reason why the SIAI/MIRI logo looks like an angel), I wrote him a very silly e-mail warning him that, despite being Jewish, he was falling for the Christian rapture narrative in a new form! It seemed to me that his ideas, which I think are all correct, straightforwardly implied the destruction of all things, with very little hope. I bite that bullet, and…

So I listened to the first 7:56 of this interview and imho he's out of his f*ckin' mind and spreading dangerous disinformation. "We don't know if there's a person in LLMs", "We understand more about the human mind that we do about LLMs despite having complete read access". He's insane. Nobody should listen to him.

Eliezer Yudkowsky: Dangers of AI and the End of Human Civilization | Lex Fridman Podcast #368

https://www.youtube.com/watch?v=AaTRHFaaPG8

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#319

Post by dragonwriter » Thu, Mar 30, 2023, 6:38 PM UTC

Aside from the fact that the entire argument is grounded in abstract ivory-tower omphaloskepsis and not facts in the real world, a major problem with Yudkowsky’s argument is that trying to execute the preemptive Butlerian Jihad he recommends is something that actual, real-world experience tells us:

(1) Will kill a lot of people directly in the best case,

(2) Won’t actually succeed in preventing further development of AI in any case, and, therefore, to the extent his concerns are grounded won’t solve the problem it intends to,

(3) Will kill a lot of people indirectly, in the best case, by setting back (and in some cases, winding back) progress in every field of practical use of technology through limits on information and information technology,

(4) Has a fair chance of the violence it necessarily involves spiralling out of control, potentially destroying the human race, as it is unlikely that all societies, or, particularly, all nuclear powers, will sign on to “we must smash the thinking machines”, and those who do not will take those who do trying to forcibly enforce their Luddism as an existential threat, while those who do will take the resistance the same way.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#320

Post by MacsHeadroom » Thu, Mar 30, 2023, 6:42 PM UTC

antibasilisk wrote: post #213

Imnimo wrote: post #64

Earlier quoted context omitted.

The author's proposed solution is: Shut down all the large GPU clusters (the large computer farms where the most powerful AIs are refined). Shut down all the large training runs. Put a ceiling on how much computing power anyone is allowed to use in training an AI system, and move it downward over the coming years to compensate for more efficient training algorithms. No exceptions for governments and militaries. Make…

He is talking about interstate warfare. I am talking about the almost inevitable organized terrorism that we're going to see if people start identifying this as an imminent threat to their survival. I think turning America into Syria is a bit more violent than iran and israel blowing up each other's stuff.

>He is talking about interstate warfare.

"Frame nothing as a conflict between national interests" "this, is not a policy but a fact of nature." "[be] willing to run some risk of nuclear exchange if that’s what it takes"

idk