Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

gizmodo.com

Post reply on HN
Print view View original

141–150 of 392 posts

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#141

Post by datameta »

nradov wrote: post #67
dota_fanatic wrote: post #60

Earlier quoted context omitted.

One cannot predict what a smarter-than-themself agent will do ahead of time, if they could then they’re just as smart. Just as a dolphin cannot predict how a human will come up with novel and utterly overwhelming ways to farm them, you and this other poster cannot predict how an AGI will achieve dominance of its environment to achieve its goals, so your request is impossible to fulfill. Lee Sedol couldn’t “take us th…

How ridiculous. This is just sci-fi bullshit without a shred of logic or scientific evidence. You would have just as much credibility claiming that we are at risk of an alien invasion. I mean I can prove that it's impossible.

Here's an example: our dear Czar uses the AGI to discover a 16 shell company chain in order to get resources and tech that are under sanctions. Or, he has it use real-world conditions from large batches or real-time uncurated battlefield data and intelligence reports to run simulation after simulation to inform his generals where it is likeliest that the next counterattack would be concentrated.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#142

Post by datameta »

nradov wrote: post #90
Valgrim wrote: post #81

Earlier quoted context omitted.

Step 1: AGI (let's call him Hal) is instructed to act as an adversarial agent to a state. Step 2: Hal research and uses every trick and exploits known to men to infiltrate critical infrastructures like telecommunications, power grids and private corporations of said country. Hal actively search and discover zero-day exploits on every known piece of hardware and software. Hal keeps mostly invisible at this stage, ensu…

So just more bullshit hand waving then.

I don't think you're participating in good faith here.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#143

Post by jnwatson »

RandomLensman wrote: post #132
jjoonathan wrote: post #105

Earlier quoted context omitted.

> all signs seem to be pointing toward a longer timeframe rather than a short one If we time-traveled back to 2019 and showed past-you a demo of GPT4's capabilities and asked you to guess the timeline, would you have gotten it correct to within a decade? I wouldn't have. It's not crazy to prepare for another jump.

If you saw the Concorde fly for the first time in the late 1960s and all the plans fore more supersonic planes & travel, would you have thought that 50 years later there was none?

Or 1972 at the end of the Apollo missions? No human has left LEO in 52 years.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#144

Post by alex_young »

crazygringo wrote: post #52

It's so confusing because people keep (intentionally?) conflating two separate ideas of "AI safety". The first is the kind of humdrum ChatGPT safety of, don't swear, don't be sexually explicit, don't provide instructions on how to commit crimes, don't reproduce copyrighted materials, etc. Or preventing self-driving cars from harming pedestrians. This stuff is important but also pretty boring, and by all indications c…

I just don’t understand how a transformer approach to human language can be expected to result in reasoning machines that are a threat to humanity.

Feeding more information in has absolutely resulted in phenomenal progress in predicting the next token in a given document, but how then do we expect even a perfect token predictor machine to start exhibiting independent emotions, desires, secret plots, and long term strategies?

This doesn’t mean we won’t create such machines, but I suspect LLMs aren’t a real risk.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#145

Post by zarzavat »

RandomLensman wrote: post #132
jjoonathan wrote: post #105

Earlier quoted context omitted.

> all signs seem to be pointing toward a longer timeframe rather than a short one If we time-traveled back to 2019 and showed past-you a demo of GPT4's capabilities and asked you to guess the timeline, would you have gotten it correct to within a decade? I wouldn't have. It's not crazy to prepare for another jump.

If you saw the Concorde fly for the first time in the late 1960s and all the plans fore more supersonic planes & travel, would you have thought that 50 years later there was none?

Concorde was a case of optimizing for the wrong thing. Concorde optimized for speed but what people actually care about is cost, i.e. fuel efficiency and range. Both of those have come on leaps and bounds. The longest commercial flight has been consistently increasing.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#146

Post by jjoonathan »

SuperLynxDeluxe wrote: post #128
jjoonathan wrote: post #105

Earlier quoted context omitted.

> all signs seem to be pointing toward a longer timeframe rather than a short one If we time-traveled back to 2019 and showed past-you a demo of GPT4's capabilities and asked you to guess the timeline, would you have gotten it correct to within a decade? I wouldn't have. It's not crazy to prepare for another jump.

That really depends on who you asked. Transformers existed in 2019, and while GPT4 is genuinely advanced, it's not a gigantic leap. Deep learning is to neural networks what GPT4 is to transformers. It's a lot bigger, enabled by better hardware and more data, but fundamentally not a new technology. There needs to be a lot more leaps for AGI, language models are not the holy grail, but another tool like reinforcement l…

> Transformers existed in 2019, and while GPT4 is genuinely advanced, it's not a gigantic leap

I'm referring to the emergent capabilities that show up with scale, which are a gigantic leap. The usual citation suggests that this came into the public consciousness in 2020 [1]. Do you take issue with the timeline or the idea that these were a gigantic leap?

[1] https://arxiv.org/abs/2005.14165

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#147

Post by RandomLensman »

chpatrick wrote: post #111
crazygringo wrote: post #52

It's so confusing because people keep (intentionally?) conflating two separate ideas of "AI safety". The first is the kind of humdrum ChatGPT safety of, don't swear, don't be sexually explicit, don't provide instructions on how to commit crimes, don't reproduce copyrighted materials, etc. Or preventing self-driving cars from harming pedestrians. This stuff is important but also pretty boring, and by all indications c…

A few years ago we thought we were so far away from AI creating art or having a reasonable conversation.

Not really true: when people first engaged with ELIZA some thought they had a meaningful interaction then.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#148

Post by josephg »

SuperLynxDeluxe wrote: post #128
jjoonathan wrote: post #105

Earlier quoted context omitted.

> all signs seem to be pointing toward a longer timeframe rather than a short one If we time-traveled back to 2019 and showed past-you a demo of GPT4's capabilities and asked you to guess the timeline, would you have gotten it correct to within a decade? I wouldn't have. It's not crazy to prepare for another jump.

That really depends on who you asked. Transformers existed in 2019, and while GPT4 is genuinely advanced, it's not a gigantic leap. Deep learning is to neural networks what GPT4 is to transformers. It's a lot bigger, enabled by better hardware and more data, but fundamentally not a new technology. There needs to be a lot more leaps for AGI, language models are not the holy grail, but another tool like reinforcement l…

I think the leaps required are:

- Giving gpt4 short term memory. Right now it has working memory (context size, activations) and long term memory (training data). But no short term memory.

- Give it the ability to have internal thoughts before speaking out loud

- Add reinforcement learning. If you want to write code, it helps if you can try things out with a real compiler and get feedback. That’s how humans do it.

I think GPT4 + these properties would be significantly more capable. And honestly I don’t see a good reason for any of these problems to take decades to solve. We’ve already mastered RL in other domains (eg alphazero).

In the meantime, an insane amount of money is being poured into making the next generation of AI chips. Even if nothing changes algorithmically, we’ll have significantly bigger, better, cheaper models in a few years that will put gpt4 to shame.

The other hard thing is that while transformers weren’t a gigantic leap, nobody - not even the researchers involved - predicted how powerful gpt3 or gpt4 would be. We just don’t know yet what other small algorithmic leaps might make the system another order of magnitude smarter. And one more order of magnitude of intelligence will probably be enough to make gpt5 smarter than most humans.

I don’t think agi will be that far away.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#149

Post by joe_the_user »

subharmonicon wrote: post #62
crazygringo wrote: post #52

It's so confusing because people keep (intentionally?) conflating two separate ideas of "AI safety". The first is the kind of humdrum ChatGPT safety of, don't swear, don't be sexually explicit, don't provide instructions on how to commit crimes, don't reproduce copyrighted materials, etc. Or preventing self-driving cars from harming pedestrians. This stuff is important but also pretty boring, and by all indications c…

> We're so far away from AGI Although I personally suspect this is correct, the issue seems to be that there are a lot of people who feel otherwise, whether right or wrong, including some of the people involved in the research and/or running these companies. I've seen several prominent people say things to the effect of "We just don't know if we're six months or 600 years from AGI,", which to me is a little like sayi…

Well, one should consider that the deep learning system being developed today are developed through a process that's said to be "closer to alchemy than chemistry". The best deep learning developers aren't thinking about "what is intelligence, really?" but rather figuring ways to optimize systems on benchmarks.

And at the same time, current system definitely have made advances in seeming intelligence. Moreover, the Eliza Effect [1] means it's easy for human being interpret a machine that interacts appropriately as intelligent even given very simple programming. If the engineers were engineering these systems, one would expect some immunity to this but as mentioned, they are doing something subtly different. You can see this in action with the Google engineer who tried to get a lawyer for an unreleased LLM and Geoffrey Hinton's turn to worrying after an AI explained a joke to him might arguably be included.

And oppositely, because these are developed like alchemy, maybe further qualities will just pop up the way transformers changed language models and maybe this will yield all relevant human capacities. I'm doubtful but not as doubtful as I was five years ago.

[1] https://en.wikipedia.org/wiki/ELIZA_effect

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#150

Post by AlanYx »

anvil-on-my-toe wrote: post #99
andrewprock wrote: post #47

Earlier quoted context omitted.

The crazier thing is that most advocates of "AI safety" don't know anything about "AI safety". By and large the term is marketing with very little objective scientific development. It is essentially a series of op-ed pieces from vested interests masquerading as a legitimate field of inquiry

That is not correct. AI safety is a subject on which serious books have been written, serious scientists have published research, and serious organizations have spent tens, if not hundreds, of millions of dollars on researching it. Who are you who to say "advocated don't know anything about AI safety"?

Are there any serious books written on the "prevent catastrophe" definition of AI safety written by people like Hinton or Sutskever? (Not necessarily them specifically, but people with technical chops rather than people whose "game" is philosophic speculation like Yudkowski or Bostrom.)
Post reply on HN