Statement on AI Risk • Hacker News Board

Re: Statement on AI Risk

#761

Post by casebash » Tue, May 30, 2023, 10:29 PM UTC

Mizza wrote: post #56

Can anybody who really believes this apocalyptic stuff send me in the direction of a convincing _argument_ that this is actually a concern? I'm willing to listen, but I haven't read anything that tries to actually convince the reader of the worry, rather than appealing to their authority as "experts" - ie, the well funded.

I strongly recommend this video:

https://forum.effectivealtruism.org/posts/ChuABPEXmRumcJY57/...

Also, this summary of "How likely is deceptive alignment" https://forum.effectivealtruism.org/posts/HexzSqmfx9APAdKnh/...

Re: Statement on AI Risk

#762

Post by tome » Tue, May 30, 2023, 10:30 PM UTC

pixl97 wrote: post #726

revelio wrote: post #688

Earlier quoted context omitted.

Sure. Despite all that progress, computers still have an off switch and power efficiency still matters. It actually matters more now than in the past.

What you're arguing is "what is the minimum viable power envelope for a super intelligence". Currently that answer is "quite a lot". But for the sake of cutting out a lot of argument lets say you have a cellphone sized device that runs on battery power for 24 hours that can support a general intelligence. Lets say, again for arguments sake, there are millions of devices like this distributed in the population. Do you…

> lets say you have a cellphone sized device that runs on battery power for 24 hours that can support a general intelligence

I can accept that it would be hard to turn off. What I find difficult to accept is that it could exist. What makes you think it could?

Re: Statement on AI Risk

#763

Post by tome » Tue, May 30, 2023, 10:33 PM UTC

duvenaud wrote: post #151

I signed the letter. At some point, humans are going to be outcompeted by AI at basically every important job. At that point, how are we going to maintain political power in the long run? Humanity is going to be like an out-of-touch old person on the internet - we'll either have to delegate everything important (which is risky), or eventually get scammed or extorted out of all our resources and influence.

> At some point, humans are going to be outcompeted by AI at basically every important job

Could you explain how you know this?

Re: Statement on AI Risk

#764

Post by colinsane » Tue, May 30, 2023, 10:34 PM UTC

nologic01 wrote: post #677

This is a breathless, half-baked take on "AI Risk" that does not cast the esteemed signatories in a particularly glowing light. It is 2023. The use and abuse of people in the hands of information technology and automation has now a long history. "AI Risk" was not born yesterday. The first warning came as early as 1954 [1]. The Human Use of Human Beings is a book by Norbert Wiener, the founding thinker of cybernetics…

i think if AI figures took their "alignment" concept and really pursued it down to its roots -- digging past the technological and into the social -- they could do some good.

take every technological hurdle they face -- "paperclip maximizers", "mesa optimizers" and so on -- and assume they get resolved. eventually we're left with "we create a thing which perfectly emulates a typical human, only it's 1000x more capable": if this hypothetical result is scary to you then exactly how far do you have to adjust your path such that the result after solving every technical hurdle seems likely to be good?

from the outside, it's easy to read AI figures today as saying something like "the current path of AGI subjects the average human to ever greater power imbalances. as such, we propose ". i don't know how to respond productively to that.

Re: Statement on AI Risk

#765

Post by cubefox » Tue, May 30, 2023, 10:35 PM UTC

TristanDaCunha wrote: post #93

shafyy wrote: post #32

The issue I take with these kind of "AI safety" organizations is that they focus on the wrong aspects of AI safety. Specifically, they run this narrative that AI will make us humans go extinct. This is not a real risk today. Real risks are more in the category of systemic racism and sexism, deep fakes, over reliance on AI etc. But of course, "AI will humans extinct" is much sexier and collects clicks. Therefore, the…

You have it totally backwards. It's a much bigger catastrophe if we over-focus on "safety" as avoiding sexism and so on, and then everyone dies.

Exactly. Biased LLMs are incredibly unimportant compared to the quite possible extinction of humanity.

Re: Statement on AI Risk

#766

Post by gjm11 » Tue, May 30, 2023, 10:36 PM UTC

skybrian wrote: post #681

nopinsight wrote: post #654

Earlier quoted context omitted.

Here’s why AI risks are real, even if our most advanced AI is merely a ‘language’ model: Language can represent thoughts and some world models. There is strong evidence that LLMs contain some representation of world models it learned from text. Moreover, LLM is already a misnomer; latest versions are multimodal. Current versions can be used to build agents with limited autonomy. Future versions of LLMs are most likel…

A lot of things are called "world models" that I would consider just "models" so it depends on what you mean by that. But what do you consider to be strong evidence? The Othello paper isn't what I'd call strong evidence.

I agree that the Othello paper isn't, and couldn't be, strong evidence about what sort of model of the world (if any) something like GPT-4 has. However, I think it is (importantly) pretty much a refutation of all claims along the lines of "these systems learn only from text, therefore they cannot have anything in them that actually models anything other than text", since their model learned only from text and seems to have developed something very much like a model of the state of the game.

Again, it doesn't say much about how good a model any given system might have. The world is much more complicated than an Othello board. GPT-4 is much bigger than their transformer model. Everything they found is consistent with anything from "as it happens GPT-4 has no world model at all" through to "GPT-4 has a rich model of the world, fully comparable to ours". (I would bet heavily on the truth being somewhere in between, not that that says very much.)

Re: Statement on AI Risk

#767

Post by blueblimp » Tue, May 30, 2023, 10:36 PM UTC

ChatGTP wrote: post #89

blueblimp wrote: post #35

Earlier quoted context omitted.

There is a way, in my opinion: distribute AI widely and give it a diversity of values, so that any one AI attempting takeover (or being misused) is opposed by the others. This is best achieved by having both open source and a competitive market of many companies with their own proprietary models.

How do you give "AI" a diversity of values?

Personalization, customization, etc.: by aligning AI systems to many users, we benefit from the already-existing diversity of values among different people. This could be achieved via open source or proprietary means; the important thing is that the system works for the user and not for whichever company made it.

Re: Statement on AI Risk

#768

Post by pharmakom » Tue, May 30, 2023, 10:36 PM UTC

I don't understand how people are assigning probability scores to AI x-risk. It seems like pure speculation to me. I want to take it seriously, given the signatories, any good resources? I'm afraid I have a slight bias against Less wrong due to the writing style typical of their posts.

Re: Statement on AI Risk

#769

Post by edgyquant » Tue, May 30, 2023, 10:37 PM UTC

nopinsight wrote: post #609

haldujai wrote: post #595

Earlier quoted context omitted.

Subtract OpenAI, Google, StabilityAI and Anthropic affiliated researchers (who have a lot to gain) and not many academic signatories are left. Notably missing representation from the Stanford NLP (edit: I missed that Diyi Yang is a signatory on first read) and NYU groups who’s perspective I’d also be interested in hearing. Not committing one way or another regarding the intent with this but it’s not as diverse an aca…

Even if it’s just Yoshua Bengio, Geoffrey Hinton, and Stuart Russell, we’d probably agree the risks are not negligible. There are quite a few researchers from Stanford, UC Berkeley, MIT, Carnegie Mellon, Oxford, Cambirdge, Imperial College, Edinburg, Tsinghua, etc who signed as well. Many of whom do not work for those companies. We’re talking about nuclear war level risks here. Even a 1% chance should definitely be a…

Id like to see the equation that led to this 46%. Even long time researchers can be overcome by grift

Re: Statement on AI Risk

#770

Post by ben_w » Tue, May 30, 2023, 10:37 PM UTC

AlexandrB wrote: post #303

This reeks of marketing and a push for early regulatory capture. We already know how Sam Altman thinks AI risk should be mitigated - namely by giving OpenAI more market power. If the risk were real, these folks would be asking the US government to nationalize their companies or bring them under the same kind of control as nukes and related technologies. Instead we get some nonsense about licensing.

Yudkowsky wants it all to be taken as seriously as Israel took Iraqi nuclear reactors in Operation Babylon.

This is rather more than "nationalise it", which he has convinced me isn't enough because there is a demand in other nations and the research is multinational; and this is why you have to also control the substrate… which the US can't do alone because it doesn't come close to having a monopoly on production, but might be able to reach via multilateral treaties. Except everyone has to be on board with that and not be tempted to respond to airstrikes against server farms with actual nukes (although Yudkowsky is of the opinion that actual global thermonuclear war is a much lower damage level than a paperclip-maximising ASI; while in the hypothetical I agree, I don't expect us to get as far as an ASI before we trip over shorter-term smaller-scale AI-enabled disasters that look much like all existing industrial and programming incidents only there are more of them happening faster because of all the people who try to use GPT-4 instead of hiring a software developer who knows how to use it).

In my opinion, "nationalise it" is also simultaneously too much when companies like OpenAI have a long-standing policy of treating their models like they might FOOM well before they're any good, just to set the precedent of caution, as this would mean we can't e.g. make use of GPT-4 for alignment research such as using it to label what the neurones in GPT-2 do, as per: https://openai.com/research/language-models-can-explain-neur...