111–120 of 176 posts
He rather obviously is asking to establish a 3rd party specifically for this task, and to establish guidelines relating to the given concerns, and to establish guidelines for government actions based on the evaluation by the 3rd party.
What I feel like is missing in the common discourse here is that Anthropic genuinely believes that AI poses an existential risk for humanity either in terms of literal survival or extreme mass surveillance, human disempowerment etc. So if you take these risks seriously, which the median commentor on HN obviously doesn't, what is the right thing to do? I.e. OpenAI just went full evil corpo mode and went all in on the…
We do not know for a fact what they genuinely believe, and many of us have seen companies act in opposition to their stated goals so the burden of proof is on them. It's healthy to suspect ulterior motives from them.
Here is Dario writing about AI safety in 2016 [1]. Dario and others in the Anthropic circle have long been associated [2] with the effective altruism movement, which whatever you think about them, they are very concerned about AI existential risks. Ronan Farrow & the New Yorker did a mega deep dive on Sam Altmans history of being dishonest and manipulative, and it credibly reports that Anthropic cofounders left due to not trusting leadership there with safety.
On the other hand we have decades long evidence of Altman and co. being dishonest and employing people like Chris Lehane [3]. I have no affiliation with Anthropic beyond being a user of their products to be clear but it feels like HN is going to end up on the wrong side of this one.
[1]. https://arxiv.org/abs/1606.06565 [2]. https://forum.effectivealtruism.org/posts/53Gc35vDLK2u5nBxP/... [3]. https://en.wikipedia.org/wiki/Chris_Lehane
Maybe this will be simpler for Anthropic to understand if they take their own high-minded philosophical nonsense and ego out of it and consider it the way a neutral party would. Suppose a company calls themselves The Doomsday Device Company. They make and sell excellent-quality doomsday devices. They regularly go online to proclaim that their doomsday devices are the best and most powerful, and also that doomsday dev…
Okay, but hasn't OpenAI been doing the same thing for years? They seem to be on slightly better terms though...
Earlier quoted context omitted.
> So if you take these risks seriously, which the median commentor on HN obviously doesn't, what is the right thing to do? Easy. You oppose it. You dedicate all your resources to stopping not just OpenAI, but anybody trying to make these technologies. With all those billions of dollars, you could get a lot done. Anthropic doesn't do this, which exposes the fundamental hypocrisy in their stated philosophies.
Anthropic has called for a coordinated pause: https://www.reuters.com/business/anthropic-says-ai-labs-need... I characterize the culture of companies based on who works there. Anthropic is founded by people who left OpenAI because it didn't take safety seriously enough. But if AI development has to happen, they want to be the ones leading it. People who do not feel that way, including Anthropic's former head of safet…
If they actually believe this stuff is so dangerous, they should shut down their company, and use their fortune to buy up/shut down all the others.
But of course they don't actually believe that. It is just marketing hype.
You can't be out there selling doomsday devices, saying "maybe we should slow down development of bigger doomsday devices", while *still selling doomsday devices". That is just blatant hypocrisy. Eating cake and having it, too. Oxymoron.
No, they don't actually believe this. Not in a meaningful capacity.
(FWIW, I don't believe it, either. I think we should continue developing the technology).
> if it is determined, in light of third-party assessment, to present unacceptable risks. >Yes. This assessment was made by Amazon, a frequent and serious government contractor which is generally trusted to handle high-security government, intelligence, and military contractor concerns. Reads as partially disingenuous. Amazon did not conduct some thoroughly vetted, responsible security audit. Someone gave them exampl…
Personally I don’t think we should impose guardrails on something so close to speech. But I can imagine Amazon was worried about how an explosion of cybersecurity incidents might affect the world. After all, they run AWS and have good intuition for the landscape of cybersecurity. Imagine if many of their cloud customers are suddenly facing one breach after another.
Maybe this will be simpler for Anthropic to understand if they take their own high-minded philosophical nonsense and ego out of it and consider it the way a neutral party would. Suppose a company calls themselves The Doomsday Device Company. They make and sell excellent-quality doomsday devices. They regularly go online to proclaim that their doomsday devices are the best and most powerful, and also that doomsday dev…
Instead what happened is a one-off nationalist decree that solves none of the two concerns.
Earlier quoted context omitted.
> So if you take these risks seriously, which the median commentor on HN obviously doesn't, what is the right thing to do? Easy. You oppose it. You dedicate all your resources to stopping not just OpenAI, but anybody trying to make these technologies. With all those billions of dollars, you could get a lot done. Anthropic doesn't do this, which exposes the fundamental hypocrisy in their stated philosophies.
It seems like a simple solution but if Anthropic was actually to dedicate all its resources this way wouldn't the investors just demand a new CEO? I think Anthropic believe these risks, but I also think they've spent so much time talking to Claude that they've pretty much lost their minds now. Anthropic have a model welfare department and have numerous times suggested that Claude is conscious and has human like emoti…
Earlier quoted context omitted.
Anthropic has called for a coordinated pause: https://www.reuters.com/business/anthropic-says-ai-labs-need... I characterize the culture of companies based on who works there. Anthropic is founded by people who left OpenAI because it didn't take safety seriously enough. But if AI development has to happen, they want to be the ones leading it. People who do not feel that way, including Anthropic's former head of safet…
Ok, but that only benefits them. If they actually believe this stuff is so dangerous, they should shut down their company, and use their fortune to buy up/shut down all the others. But of course they don't actually believe that. It is just marketing hype. You can't be out there selling doomsday devices, saying "maybe we should slow down development of bigger doomsday devices", while *still selling doomsday devices".…
Even if they could somehow cash out their entire market cap, Google's is >4x theirs. AND if they quit, their competitors' valuations go up. AND there's nothing to stop those employees from leaving and founding new AI companies.
Earlier quoted context omitted.
> I am sure they calculated in the possibility of their own model being restricted. Doubt. Had they foreseen this, they would have started verifying the identity of their customers. That would have allowed them to keep their US customers when the US government banned foreign persons from accessing Fable. Since they were forced to turn off Fable for everyone , it follows that they were not prepared for that possibilit…
The big problem Anthropic faces isn't implementing a KYC workflow, but the fact that many if not most of their own employees are no longer allowed to work on Fable/Mythos.