Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

gizmodo.com

Post reply on HN
Print view View original

381–390 of 392 posts

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#381

Post by jalapenos »

voitvoder wrote: post #379
jalapenos wrote: post #338

Earlier quoted context omitted.

I think the people worried about AGI have an oddly positive view of the world - like it's so wonderful as-is that we need to guard it from any disturbances. Perhaps if they paid closer attention to how profoundly bad so much of human existence is, people living in squalor, dying from awful diseases, killing each other, they'd understand how truly acceptable that dice roll is. Pedal to the metal all the way.

I think there are many highly intelligent people that would prefer less intelligent people don't gain additional intelligence. They are perfectly happy with the relative value of their own intelligence and know that relative value goes way down in a world that everyone has access to additional intelligence.

Interesting take & good point

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#382

Post by pixl97 »

hoten wrote: post #372
Philpax wrote: post #344

Earlier quoted context omitted.

Historically, the world has not been great at universally coordinating responses to social problems - especially when it only takes one actor to break the “truce.”

Isn't agreeing to only run an AGI with whatever theoretical alignment controls we come up with, also a social agreement? Seems we will have to figure that out one way or another.

Which is kind of the point... if we make AGI that it means we die. That is if the AI dormers predictions are correct.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#383

Post by namibj »

dragontamer wrote: post #231
jdmoreira wrote: post #190

Earlier quoted context omitted.

Sounds like something really simple to define and grasp. What's wrong with, for example: "A software that beats most humans on most cognitive tasks". People make it harder than it needs to he IMO

We've had computers that beat humans in differential equations, automated theorems, arithmetic, aiming a gun and more in the 1970s. Even search was considered an AI before Google, but today is just a thing computers are and always have been faster than humans at. Even before ChatGPT, lawyers relied upon computers to double-check citations. Even in programming, I expect that compilers and IDEs check for the most commo…

We still don't beat humans in e.g. LeanDojo, to my understanding. But we also haven't yet thrown the state of the art in graph attention technology at doing the core task of in-context learning (consider the entire explored (partial-)proof "tree" for deciding what tactic with which premises (if the tactic takes an AST/expression argument, like "use that definition/equivalence there as a rewrite rule on the current goal at this node in the proof tree") to apply next at which node in the (partial-)proof tree) in the interactive theorem proving situation, let alone with reinforcement learning that uses the attention structures to funnel back information on what things in dead branches had responsibility for the decisions that made the non-dead branches.

Because the RL agent should happily use the theorem prover as a "calculator", and only get "charged" for how much the ML inference and the actual cheap calculator execution really cost. Exploration in-context is what humans also do, and immensely powerful. Similar to, when programming, testing some expressions to figure out edge-cases before incorporating them into a large function/algorithm that has poor observability.

So far, machines still mostly do the task of "computing" in theorem proving for us, rather than show some proper intelligence. It's damming that LeanDojo got so high marks without reinforcement learning. They pretty much just dumped prover context state during a run through the entire matlib "standard" library and handed it to two transformers, one for embedding premises and one encoder-decoder fed with the current state and as many premises as the encoder context fits after the state, generating/sampling a few tactics to apply. No freedom of where in the proof tree to apply the next tactic, no awareness of any such side-branches.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#384

Post by theptip »

hoten wrote: post #372
Philpax wrote: post #344

Earlier quoted context omitted.

Historically, the world has not been great at universally coordinating responses to social problems - especially when it only takes one actor to break the “truce.”

Isn't agreeing to only run an AGI with whatever theoretical alignment controls we come up with, also a social agreement? Seems we will have to figure that out one way or another.

The airgap option is “everybody agrees to leave lots of money on the table in the name of safety”. The alignment option is “some people invest lots of money to invent technology that everyone can use to safely make lots of money”.

If you solve alignment, the regulatory nudge required for folks to use it is probably not that big; let’s be really pessimistic and say it’s 10x the compute to run all the supervision. It’s probably mostly criminals that want the full-unaligned models, and so most entities can agree to the spend.

In the air gap case, I’m pretty confident that it is many orders of magnitude more profitable to run the “unsafe” / non-airgapped version. The utility of baking AI into everything is massive. Maybe in the best case you can have your airgapped ASI build provably-safe non-AGI tool AIs and capture much of that value, but I’m pretty skeptical it’s safe to transport anything that complex across the airgap.

So the alignment research path is more sustainable since it requires paying a much lower “safety premium”.

But I’m all for pursuing both, I don’t think they interfere with each other, quite the opposite; taking either one seriously reinforces the need for both of them.

Another advantage of the alignment path is we can meaningfully make progress now. It’s really hard to get people to agree ahead of time that they will run their AGI in an airgap.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#385

Post by remarkEon »

ethanbond wrote: post #357
remarkEon wrote: post #110

Earlier quoted context omitted.

This is why I can't take the Safetyists seriously. Yudkowsky's essay in Time demanding a global pause in AI research is so naive to the point of parody. And who is going to enforce this "pause", Eliezer? What he's really advocating is global war to prevent the AI apocalypse. Fair enough, but he should really make that his argument because AI research doesn't stop with out it.

Contrary to popular HN belief, the human species has curtailed technological development along various lines, internationally. E.g. Human cloning, human hybrids, hugely successful non-proliferation in nuclear weapons

True, but those are all pretty different from AI, though.

It’s not immediately obvious that human cloning has great benefits. Ditto human hybrids (supposing that’s even possible). And nuclear weapons proliferation is pretty difficult to hide because of how they’re made.

But prove your data center isn’t running AI models.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#386

Post by [deleted] »

mike_hearn wrote: post #329
kypro wrote: post #106

Earlier quoted context omitted.

As a principal I tend to lower my conviction of any opinion I hold the more that people I intellectually respect disagree with me. I think to say with confidence that we're doomed or that there is no risk to the AI system currently in development would be intellectually arrogant. I'm not even convinced you necessarily need AGI before AI becomes a risk to humanity. An advanced enough auto-GPT bot given a bad goal migh…

> An advanced enough auto-GPT bot given a bad goal might be all you need. Cripple enough infrastructure and humanity is going to be starving to death within weeks. AI risk enthusiasts tend to define "bad goal" as one that sounds reasonable but has unexpected side effects. The real risk to infrastructure and physical safety though is that people will use AI to assist in hacking during a hot war. In this case the goal…

[deleted]

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#387

Post by calamari4065 »

dartos wrote: post #266
calamari4065 wrote: post #260

Earlier quoted context omitted.

> The implicit argument is that one can't engineer something one doesn't understand. That's what machine learning is . We build these systems we don't fully understand, and set them on solving problems we don't understand. It's revolutionary because it lets us solve problems we don't understand. We don't really fully understand what they do or how, but it does still work.

We do fully understand how, and why, they work. That’s how we can build and optimize them. We can and do explicitly explain their inner workings in math.

I think we're talking about entirely different levels of understanding.

We do, of course, understand ML models at a pretty deep level. What you can't do is identify the weight values that encode all information about squirrels. You can't point to a particular neuron and say this is why it hallucinates. We do not grok these models. I severely doubt that it's even possible for a human to grok a 13B parameter LLM

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#388

Post by rzl1235 »

ClumsyPilot wrote: post #173
RandomLensman wrote: post #160

Earlier quoted context omitted.

The US poured a lot of money into the SST, too, it wasn't just the French and the British. A lot of people thought that the future is supersonic as people did want the speed increase when moving to jet planes. Conversely, maybe AGI is not what is wanted in the end anyway.

> Conversely, maybe AGI is not what is wanted in the end anyway. I dont think it is, if it is trully intelligent, it will probably have rights, so you cant treat it as a slave. And it will likely not be controllable. Both of these mean it will not be commercially progotable, any more than having children is profitable.

How ironic, considering throughout most of history, a large family was a sign of/creator of wealth due to more labor. Now the incentives have reversed due to human unskilled labor being so cheap + dissolution of family obligations.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#389

Post by rzl1235 »

fl7305 wrote: post #251
kypro wrote: post #106

Earlier quoted context omitted.

As a principal I tend to lower my conviction of any opinion I hold the more that people I intellectually respect disagree with me. I think to say with confidence that we're doomed or that there is no risk to the AI system currently in development would be intellectually arrogant. I'm not even convinced you necessarily need AGI before AI becomes a risk to humanity. An advanced enough auto-GPT bot given a bad goal migh…

> Cripple enough infrastructure and humanity is going to be starving to death within weeks. Just the threat of a malicious AGI that people take seriously enough would cause ourselves to pull the plug on the Internet and all computers until they can be disinfected and new filters put in place. It doesn't matter if the AGI is capable of actually inflicting damage or not.

Ah yes, just "pull the plug". That's a hilarious idea, are you sure the AGI that was trying to destroy humanity didn't give you that one?

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#390

Post by rzl1235 »

fl7305 wrote: post #324
reverius42 wrote: post #316

Earlier quoted context omitted.

We couldn’t even do coordinated lockdowns for a few weeks to prevent millions of Covid deaths, and you think we’ll have a coordinated cessation of computer and internet use?

I live in Europe. Germany, France and other large countries became virtual ghost towns overnight as it became illegal to leave your home for other than critical errands. Wasn't it the same in big parts of the US? I remember seeing videos of a helicopter flying over LA without cars or people out on the streets. But sure, the whole world wouldn't coordinate to shut down.

[dead]
Post reply on HN