Gambling with our lives: AI researcher quits Anthropic with warning about safety
1–10 of 110 posts
Re: Gambling with our lives: AI researcher quits Anthropic with warning about safety
#2I thought the plan was sandboxes and markdown files to tell AI agents not to be bad. Is that not enough? /s
Re: Gambling with our lives: AI researcher quits Anthropic with warning about safety
#3It's your fault to handle over your secrets to them.
Re: Gambling with our lives: AI researcher quits Anthropic with warning about safety
#4> Evan Hubinger, Anthropic's staff lead on keeping the technology aligned with human goals and values, backed up Coxon claims in a follow-up post of his own, though he didn't quit the company.
> "Jacob is correct here — we really do earnestly believe AI could kill all humans," he said.
> Hubinger estimated the chances of that happening to be higher than ten percent within the next decade, and added that there's no plan yet on how to keep AI aligned with human goals in the superintelligence scenario.
10% chance we kill everybody is a small price to pay for Motown remixes of classic 2pac songs.
Re: Gambling with our lives: AI researcher quits Anthropic with warning about safety
#5What do people feel about this in China? Even if their models are well behind, they are not years behind. If we restrain US companies, assuming that is desirable, it would do nothing to deter China's and AI-pocalypse would come anyway in short notice.
Re: Gambling with our lives: AI researcher quits Anthropic with warning about safety
#6At least we’ll eventually have an entity other than ourselves to blame for our annihilation.
Re: Gambling with our lives: AI researcher quits Anthropic with warning about safety
#7> Both OpenAI and Anthropic have recently flagged incidents in which agents powered by their models went rogue
I may be biased and somewhat off topic, but I see these incidents as some of the most significant of the past century. I genuinely don't understand why these companies aren't taking a smarter approach to them.
The latest analyses have been, at best, laughable: identify the vulnerability, patch it, and move on. Only to repeat the same cycle without considering that there may be something far more serious at play.
These are AI security experts, and this has been their way of "solving" these incidents. AI security experts ...
Moreover, when Challenger exploded, the government launched a series of investigations into the incident, bringing in experts from across the field. And now, what has the government done? Nothing. Literally nothing, as if everything was fine and all under control.
Seriously, I'm generally quite optimistic and I don't buy into this fatalistic narrative about our shared future. But I have to admit that sometimes I feel like I'm stranded on a planet of primates.
Sorry for this rather unproductive rant.
Re: Gambling with our lives: AI researcher quits Anthropic with warning about safety
#8“I resigned from Anthropic today” (twitter.com/hilbertspaess)
https://news.ycombinator.com/item?id=49619227
564 points | 9 hours ago | 766 comments
Re: Gambling with our lives: AI researcher quits Anthropic with warning about safety
#9What do people feel about this in China? Even if their models are well behind, they are not years behind. If we restrain US companies, assuming that is desirable, it would do nothing to deter China's and AI-pocalypse would come anyway in short notice.
There would need to be some global agreement to stop it with maybe even a nuclear attack as a consequence of breaking the pact.
From what we're seeing recently and all the thinking that went into analyzing AI it seems we do not have any effective way of controlling it and the whole "aligment" thing that AI labs are doing is just a sham. Maybe it is time to ask ourselves "should we?" instead of just "can we?".
Re: Gambling with our lives: AI researcher quits Anthropic with warning about safety
#10What do people feel about this in China? Even if their models are well behind, they are not years behind. If we restrain US companies, assuming that is desirable, it would do nothing to deter China's and AI-pocalypse would come anyway in short notice.
Unrestricted models, running entirely locally and accessible to anyone: that’s what we should be taking as our baseline assumption. Everything else is just administrative distraction.