It takes imagination and emotion to be dangerous. These token predictors will never be smart enough to be dangerous.
There is danger in people using a token predictor like it is some all-knowing magic computer oracle.
Ask HN: Have top AI research institutions just given up on the idea of safety?
61–70 of 99 posts
Re: Ask HN: Have top AI research institutions just given up on the idea of safety?
#62Humans can't develop safety until there is enough blood in the streets. Only issue with AI is that threshold may come at a point where its too far gone to recover. But humans can't put in seatbelts until we're losing 40k people per year in car crashes. Unfortunately its just how we're wired. Those that are careful are outcompeted by the brash and the fast-moving, until the relative value of moving fast is removed, th…
If we had rules like that in the past we never would have had the industrial revolution.
Not sure if that's true, what are your reasons for believing that? Are you saying we couldn't have invented the machines we used if we took safety measures along the way (e.g. having guards on machines that chopped of arms and legs)? Perhaps progress would have been slower -- since rather than just using the saw, you'd need a saw with a guard and emergency switch -- but it seems like if humans were more circumspect, we would have the industrial revolution, but more deliberate and controlled. Agreed it probably wouldn't have been "overnight factories in every city", but then again, you probably wouldn't have many of the externalities we're still learning about and paying for?
Re: Ask HN: Have top AI research institutions just given up on the idea of safety?
#63"safe" is such a subjective concept to begin with, have any of the model providers ever defined what they mean by "safe"? It doesn't mean much to me if a safe model is one that does not output the recipe for mustard gas, that information is trivially available elsewhere. Or, is a safe model one that doesn't come off as racist? Ok but i would classify that as unoffensive instead of safe but I admit definitions of word…
I think the problem of chatbot "safety" mirrors that of autonomous vehicle safety. For an AV, the correct course of action is one that avoids hitting stuff (including people, vehicles) and, critically, minimizes liability.
Re: Ask HN: Have top AI research institutions just given up on the idea of safety?
#64A safe model is one that doesn't cause a decrease in revenue growth for the model provider. They absolutely haven't given this up, it's just not what their marketing described what "safety" is.
Re: Ask HN: Have top AI research institutions just given up on the idea of safety?
#65The CEO of Anthropic is in the news for refusing to change their ToS to support the U.S military.
Re: Ask HN: Have top AI research institutions just given up on the idea of safety?
#66Again this is different in a Capitalists society vs a Socialist one.
In a Capitalists society everyone is pitted against each other trying to out compete the other at whatever the cost. Safety in this environment is thought of at the end after a lot of suffering because one group has to win it all. Damages can externalized.
In a Socialist society we build basic rules and we compete within them. Thinking of safety as we build something and refining those rules as we build it because at the end, we are all affected by it and get to benefit from it.
Re: Ask HN: Have top AI research institutions just given up on the idea of safety?
#67Humans can't develop safety until there is enough blood in the streets. Only issue with AI is that threshold may come at a point where its too far gone to recover. But humans can't put in seatbelts until we're losing 40k people per year in car crashes. Unfortunately its just how we're wired. Those that are careful are outcompeted by the brash and the fast-moving, until the relative value of moving fast is removed, th…
If we had rules like that in the past we never would have had the industrial revolution.
Yes, it would probably have been better to have industrial evolution instead. Or are you arguing that all the countless deaths, maimings, child labor, 16-hour workdays, robber barons, black lung, radium jaws, and so on and so on were simply how it had to go? Or do you simply not care because all of that happened to other people?
Re: Ask HN: Have top AI research institutions just given up on the idea of safety?
#68I was the author for the practitioners implementation section for the IEEE 7010 standard for assessing human impact from AI software https://standards.ieee.org/ieee/7010/7718/ I also worked closely with Jack Clark at OpenAI before he disappeared on all these issues as CTO back in 2018 There are literally zero “AI labs” that have ever cared about “safety” none of them have ever done anything tangible with any kind of…
if would be super helpful if you could give the elevator pitch version of what a safe AI is.
The only “safe AI” is one that comes out of a “safe set of data”
so what would a “safe set of data” actually have to look like
Well it would have to not look like the majority of data that we produce now which has latent embeddings (primarily from the common crawl database ) of racism, lying, competition, destruction domination
I don’t believe humans are actually capable of making such data because our entire structure of society is based on racism competition and domination
Re: Ask HN: Have top AI research institutions just given up on the idea of safety?
#69More on the periphery than an insider, but I personally know researchers in all three major labs who were there long before GPT-3. They all care about existential safety, a lot. In the sense that they believe there's a meaningful chance all humans are dead a decade from now (and that that's a bad thing; unfortunately, there are also people deeply involved who don't think human extinction is a bad thing). The issue is…
> unfortunately, there are also people deeply involved who don't think human extinction is a bad thing
You mean at the top labs? Since when isn't that level of misanthropy categorized as having mental health issues?
Re: Ask HN: Have top AI research institutions just given up on the idea of safety?
#70"safe" is such a subjective concept to begin with, have any of the model providers ever defined what they mean by "safe"? It doesn't mean much to me if a safe model is one that does not output the recipe for mustard gas, that information is trivially available elsewhere. Or, is a safe model one that doesn't come off as racist? Ok but i would classify that as unoffensive instead of safe but I admit definitions of word…
> I can use that to keep a gun pointed at a target or i can use that to prevent a baby rocker from falling over.
This leads to what I'm going to call the "Ender's Game" approach: if your AI is uncooperative just present it with a simulation that it does like but which maps onto real-world control that it objects to.
> I've always held the opinion that ai safety was more about brand safety
Yes. The social media era made that very important. The extent to which brand safety is linked to actual, physical safety then becomes one of how you can manage the publicity around disasters. And they're doing a pretty good job of denying responsibility.