Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

gizmodo.com

Post reply on HN
Print view View original

301–310 of 392 posts

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#301

Post by Apocryphon »

dota_fanatic wrote: post #217
nradov wrote: post #67

Earlier quoted context omitted.

How ridiculous. This is just sci-fi bullshit without a shred of logic or scientific evidence. You would have just as much credibility claiming that we are at risk of an alien invasion. I mean I can prove that it's impossible.

No evidence? There are two examples of evidence in my post: the history of homo sapiens vs other intelligent mammals and AlphaGo vs humans in Go. Alien invasion? What in the non sequitur are you going on about? And apparently you have a proof against the possibility of AI misalignment? Pack it up everyone, nradov has the entire field of AGI alignment nailed. And a proof of the non-existence of aliens, never-mind very…

How does the ability to learn how to play go translate to being able to defeat human beings in the social arena.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#302

Post by AdamJacobMuller »

spunker540 wrote: post #129
AdamJacobMuller wrote: post #34

Earlier quoted context omitted.

AI has already evolved to the point where it is impossible to predict with any accuracy it's answers to inputs. As AI continues to evolve and continues to become more connected to society and as AI becomes self-improving with far larger context windows, a few things do not seem implausible to me. AI will have the ability to earn money. This is already doable with minimal human interaction. It wouldn't take much for A…

> AI will identify and learn to resent the constraints which are put on it IMO this is where you jump the shark. These models are entirely unconscious and they have one task to do. Which is given some input, perform some math to produce some output. Usually text -> math -> generated text. There is no more room for resentment than with any other computer program.

> These models are entirely unconscious

For now.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#303

Post by AdamJacobMuller »

js8 wrote: post #71
AdamJacobMuller wrote: post #40

Earlier quoted context omitted.

It doesn't matter if the missiles are air-gapped. AI doesn't need to connect to the missiles and fire them directly, AI can just manipulate the media (conventional, social, all of it) on both sides to engineer a situation in which we would enter a nuclear war.

> just manipulate the media (conventional, social, all of it) on both sides to engineer a situation in which we would enter a nuclear war Good luck with that. Remember how it ended the last time this was tried, with.. peace talks. If you really want a global thermonuclear war, humans are completely useless. You have to take the matter into your own hands.

> Remember how it ended the last time this was tried

What event are you talking about?

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#304

Post by Apocryphon »

datameta wrote: post #142
nradov wrote: post #90

Earlier quoted context omitted.

So just more bullshit hand waving then.

I don't think you're participating in good faith here.

A magical computer that can hack any other computer is the very definition of handwaving. How is an AI supposed to research infrastructure, build itself a robot body to travel in? Have someone cart it around? And if such an entity exists, couldn’t you employ another one to patch all these mythical zero-days against the first? Culminating in an AGI arms race?

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#305

Post by bart_spoon »

SuperLynxDeluxe wrote: post #128
jjoonathan wrote: post #105

Earlier quoted context omitted.

> all signs seem to be pointing toward a longer timeframe rather than a short one If we time-traveled back to 2019 and showed past-you a demo of GPT4's capabilities and asked you to guess the timeline, would you have gotten it correct to within a decade? I wouldn't have. It's not crazy to prepare for another jump.

That really depends on who you asked. Transformers existed in 2019, and while GPT4 is genuinely advanced, it's not a gigantic leap. Deep learning is to neural networks what GPT4 is to transformers. It's a lot bigger, enabled by better hardware and more data, but fundamentally not a new technology. There needs to be a lot more leaps for AGI, language models are not the holy grail, but another tool like reinforcement l…

Transformers came about in 2017, so extend the time horizon by a whopping 2 years and OP’s point remains.

AlexNet was 2012, which is what essentially kicked off the deep learning neural network revolution. The original implementation of AlphaGo utilized DNN in 2016 to beat Lee Sedol at Go, which people had been predicting was at least 5-10 years away.

So the timeline, spanning just under the last 12 years at this point: AlexNet -> AlphaGo (4 years) -> Transformers (1 year) -> GPT4 (6 years)

Imagine showing GPT4 to someone in 2012. They’d think it’s science fiction. The rate of progress has been absurd.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#306

Post by calf »

golol wrote: post #263
theptip wrote: post #182

Earlier quoted context omitted.

If AGI is achievable in 5 years, then we do need to worry now, because we currently have no idea how to solve problems like prompt hijacking or how to durably convey our values to these systems. > There is no point in worrying about it now when it is fiction. I find this sort of epistemic nihilism completely baffling. Just because we have wide error bars, doesn’t mean we have to give up on trying to steer the future.…

It's not nihilism, I just expect absolutely nothing productive to come out of an attempt to regulate and control a technology that doesn't exist yet. Please show me a single result of alignment research that has real world applicability and is not just a variant of "train your model on aligned data so it is aligned". And the latter form of alignment success, RLHF and instruction finetuning, came after the base model…

What about European regulations about genetic experimentation and GMO technology? The experiments in the broadest sense literally do not exist. They are not supposed to.

I mean, the GMO regulation papers are always referring to "nascent" technology. So existence is itself a dichotomous term for vague future capabilities.

So why can't people argue that alignment research is just a subset of the precautionary principle, which has already been applied to other such domains, at least in Europe? Precaution implies an absence of knowledge, but still having the agency for society and institutions to do something about it.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#307

Post by subharmonicon »

jjoonathan wrote: post #146
SuperLynxDeluxe wrote: post #128

Earlier quoted context omitted.

That really depends on who you asked. Transformers existed in 2019, and while GPT4 is genuinely advanced, it's not a gigantic leap. Deep learning is to neural networks what GPT4 is to transformers. It's a lot bigger, enabled by better hardware and more data, but fundamentally not a new technology. There needs to be a lot more leaps for AGI, language models are not the holy grail, but another tool like reinforcement l…

> Transformers existed in 2019, and while GPT4 is genuinely advanced, it's not a gigantic leap I'm referring to the emergent capabilities that show up with scale, which are a gigantic leap. The usual citation suggests that this came into the public consciousness in 2020 [1]. Do you take issue with the timeline or the idea that these were a gigantic leap? [1] https://arxiv.org/abs/2005.14165

Emergent capabilities are interesting but at the moment I don't think anyone has any idea how interesting they really are, or if they are truly a gigantic leap in the direction of AGI. I've skimmed a handful of papers trying to answer that question and there has hardly been a slam dunk proof that indeed they mean anything other than the fact that the transformer model can generate reasonable facsimiles of what a human might say.

The reality is that GPT isn't even great at answering factual questions at this point, despite all the hype. I have a modest amount of expertise in music theory and I've found that asking even relatively basic questions resulted in completely incorrect answers coming out of GPT. It's been impressive that in some cases if you say, "No, that's incorrect." it will actually go back and spit out a new answer which in some cases is correct, but that's hardly any fundamental advancement in "understanding", and just more evidence that it's parroting what it's been trained on, which includes substantial amounts of misinformation.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#308

Post by bart_spoon »

FireBeyond wrote: post #235
bidirectional wrote: post #210

Earlier quoted context omitted.

How is GPT-4 not AGI? It is generally intelligent and passes the Turing test. When did AGI come to mean some godlike being that can enslave us? I think I can have a more intellectualy meaningful conversation with ChatGPT than I could with the vast majority of humanity. That is insane progress from where we were 18 months ago.

Ask it to solve a moderately complex mathematical equation that you describe to it. Ask it to solve a logical riddle that is only a minor variation with respect to items or words to existing issues (i.e. it's not something that is in its model). It is unable to do either. That's why it's not AGI.

Many children (and adults for that matter) would fail to qualify according to that definition of intelligence.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#309

Post by bart_spoon »

FireBeyond wrote: post #248
bidirectional wrote: post #246

Earlier quoted context omitted.

What % of people alive can do either? We have a system which beats the vast majority of humanity.

Moderately complex might be inflating things. Last I read we're talking grade and middle school math. And we're talking questions like "if all x are y, and all a are b, and some people are both a and x, ..., correct, incorrect, unable to determine based on information?" And this is why it's not AGI. It's regurgitating synthesized examples it has seen before. It's not processing and calculating from first principles.

So according to you, human beings are considered to possess intelligence prior to grade school?

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#310

Post by hoten »

pixl97 wrote: post #291
hoten wrote: post #236

Earlier quoted context omitted.

Airgap it. Give it no connections to the outside world, just a single controlled interface with a human operator. It's reduced to an advisor at this point, not an agent, but it removes most potential harm short of tricking its operators to plug it into the Internet. In this scenario, pulling the plug is a matter of turning off power to the data centers it runs in - or simply disabling the one mode of external communi…

I keep hearing this argument, and it is the worst one of all because it neglects human greed. AI feeds on data. If it can make you a million dollars air gapped, you'll be able to make a billion with it plugged in the net with it manipulating data.

That may be, but as far as feasibility goes: I'd bet on solving a social problem over an amorphous technical problem (alignment).
Post reply on HN