Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

gizmodo.com

Post reply on HN
Print view View original

181–190 of 392 posts

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#181

Post by kragen »

jacquesm wrote: post #176
kragen wrote: post #165

Earlier quoted context omitted.

sure, a landmine is a killbot. a control system with any kind of weapon can kill people but intelligence is much more important than weapons. you can make sure they don't get away or turn the weapons against you if you can outsmart them. also you can kill the people you intend to kill, instead of somebody else, which is fundamental if you have objectives other than racking up a high body count. finally, intelligence…

No, a killbot is a way to kill parties while trying to reduce the number of 'false positives', aka collateral damage. But because a killbot does that the result then is that you get far more deployment and eventually the collateral damage in total rises because the weapon is seen as effective. That's why killbots are bad.

that is certainly a very plausible and worrisome risk, and i agree that facial recognition doesn't require strong ai

but i think there are other, possibly larger risks that result from strong ai in weapons, and plausible incentives to put it there

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#182

Post by theptip »

golol wrote: post #161
theptip wrote: post #57

Earlier quoted context omitted.

Usually “mundane” vs “existential” harms/risks. > ASI/AGI safety is only a concern right now if you believe in foom False. There are plenty of fast takeoff scenarios that look real bad that aren’t “foom”. For example if you think AGI might be achievable within 5 year right now, and alignment is 10 years away, then you are very worried and want to slow things down a little.

No, then you start worrying when the technology is there. There is no point in worrying about it now when it is fiction. If AGI/ASI does not arise in some kind of uncontrollable "foom", you can always examine what you are dealing with right now and act accordingly. If AGI is achievable in a 5 years takeoff scenario then we have plenty of time and steps along the way fo study what we have and examine its danger. Right…

If AGI is achievable in 5 years, then we do need to worry now, because we currently have no idea how to solve problems like prompt hijacking or how to durably convey our values to these systems.

> There is no point in worrying about it now when it is fiction.

I find this sort of epistemic nihilism completely baffling. Just because we have wide error bars, doesn’t mean we have to give up on trying to steer the future. If your 95% confidence window for “path A” includes ruin, and “path B” has lower EV but no ruin, why would you pick A?

The problem with alignment right now is we can’t even align GPT-3.5, let alone GPT-4. I believe this is a tech tree. New things building on old things at each generation. If we can’t keep up with the frontier, how will we align GPT-Omega? Concretely, things like Mechanistic Interpretability seem required. If we could pause progress for 5-10 years I feel optimistic that we could fully understand GPT-4 and how it actually renders concepts and decisions. (I don’t think we can pause progress, this is my case for why we should be at least a bit worried, and invest much more in interpretability research.)

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#183

Post by RandomLensman »

ClumsyPilot wrote: post #173
RandomLensman wrote: post #160

Earlier quoted context omitted.

The US poured a lot of money into the SST, too, it wasn't just the French and the British. A lot of people thought that the future is supersonic as people did want the speed increase when moving to jet planes. Conversely, maybe AGI is not what is wanted in the end anyway.

> Conversely, maybe AGI is not what is wanted in the end anyway. I dont think it is, if it is trully intelligent, it will probably have rights, so you cant treat it as a slave. And it will likely not be controllable. Both of these mean it will not be commercially progotable, any more than having children is profitable.

I valid point (I don't think we will give machines rights quickly, but the point stands). Maybe all we want is some universal reasoning machine to go with the chat bots.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#184

Post by chpatrick »

RandomLensman wrote: post #177
chpatrick wrote: post #172

Earlier quoted context omitted.

Except we quickly learnt it wasn't really anywhere near beating the Turing test, and for 60 years we thought it that was almost impossible to do so. Then just under a year ago ChatGPT changed everything.

So what? Humans again being bamboozled by some machine? That has happened in various forms for hundreds of years. (Setting aside that often the adversarial nature of the test as proposed by Turing got ignored.)

So if the machine is so good at bamboozling humans that they can't tell it's a machine any more, who's to say it isn't intelligent.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#185

Post by RandomLensman »

chpatrick wrote: post #184
RandomLensman wrote: post #177

Earlier quoted context omitted.

So what? Humans again being bamboozled by some machine? That has happened in various forms for hundreds of years. (Setting aside that often the adversarial nature of the test as proposed by Turing got ignored.)

So if the machine is so good at bamboozling humans that they can't tell it's a machine any more, who's to say it isn't intelligent.

Who is to say that it is intelligent or that that is even a relevant question to ask? People being caught up in the magic of their own creation isn't something new.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#186

Post by josephg »

beoberha wrote: post #93
crazygringo wrote: post #52

It's so confusing because people keep (intentionally?) conflating two separate ideas of "AI safety". The first is the kind of humdrum ChatGPT safety of, don't swear, don't be sexually explicit, don't provide instructions on how to commit crimes, don't reproduce copyrighted materials, etc. Or preventing self-driving cars from harming pedestrians. This stuff is important but also pretty boring, and by all indications c…

I think you’re dead on here. There is a lot of need for guardrails against ML in regards to things like self driving cars or deepfakes. I don’t think there’s much debate here, because, as you said, companies are doing a good job. The whole threat of AGI is so overblown and - IMO - being used to create regulatory capture. It’s truly confusing to me to see guys like Hinton saying AI poses some kind of existential threa…

> The whole threat of AGI is so overblown

Do you think AGI is impossible? Seems pretty possible to me. That it is far away? We have no idea. We don’t need many more breakthroughs on the level of stacking transformers to make an llm smarter than humans for most tasks.

Or do you think having an ai model that’s smarter than humans poses no risk to society / humanity? It’s essentially unlimited capacity for thought. Free, smart employees. Doing whatever you want. Or whatever they want, if they’re smarter than you and have an agenda. “ChatGPT program a search engine that’s better than Google”. “ChatGPT I am a dictator. Help me figure out how to stay in power forever”. “ChatGPT figure out how to bootstrap a manufacturing process that can build arbitrary machines. Then design and build a factory for making nuclear weapons / CO2 capture / houses / rockets to space”.

Things have the potential to be very interesting in the next few decades.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#187

Post by chpatrick »

RandomLensman wrote: post #185
chpatrick wrote: post #184

Earlier quoted context omitted.

So if the machine is so good at bamboozling humans that they can't tell it's a machine any more, who's to say it isn't intelligent.

Who is to say that it is intelligent or that that is even a relevant question to ask? People being caught up in the magic of their own creation isn't something new.

The original topic was whether Artifical General Intelligence is far away or not, and I'm saying you can't say for sure it's far away when even ChatGPT-level intelligence was considered science fiction a few years ago.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#188

Post by ghodith »

golol wrote: post #15

Look, there is ASI/AGI safety, and "please don't swear and scam and misinform, and be unbiased" safety. ASI/AGI safety is only a concern right now if you believe in foom , i.e. that an AGI would rapidly self improve to an ASI and extinction level threat. I think it is absurd to believe that because AI right now is compute limited and any first AGI will only be able to run on large data centers. The latter kind of saf…

> any first AGI will only be able to run on large data centers

Until you task your newfangled AGI to optimize its own code so you can get your cloud server costs down.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#189

Post by RandomLensman »

zarzavat wrote: post #178
RandomLensman wrote: post #160

Earlier quoted context omitted.

The US poured a lot of money into the SST, too, it wasn't just the French and the British. A lot of people thought that the future is supersonic as people did want the speed increase when moving to jet planes. Conversely, maybe AGI is not what is wanted in the end anyway.

Whether or not people want AGI, corporations definitely want it, as long as the cost of an AGI worker is less than the cost of a human worker. So perhaps we should prepend a C for “cheap” to the term AGI.

How do you know what corporations exactly want? What c-suite has spec'd out what they want from a machine generally replacing humans? Would you really want faster horses instead of cars?

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#190

Post by jdmoreira »

dkjaudyeqooe wrote: post #101
subharmonicon wrote: post #62

Earlier quoted context omitted.

> We're so far away from AGI Although I personally suspect this is correct, the issue seems to be that there are a lot of people who feel otherwise, whether right or wrong, including some of the people involved in the research and/or running these companies. I've seen several prominent people say things to the effect of "We just don't know if we're six months or 600 years from AGI,", which to me is a little like sayi…

> "We don't know if we'll have time travel and ray guns in six months or 600 years." That's not the (only) problem, it's that no one can describe what AGI will really look like outside of vague and circular definitions. People have an idea of what they would like it to be, but there's no evidence any of it can be built, particularly human like intelligence, which may be only possible in the human host. > "If we can m…

Sounds like something really simple to define and grasp. What's wrong with, for example:

"A software that beats most humans on most cognitive tasks".

People make it harder than it needs to he IMO

Post reply on HN