I understand they have been criticized because they play capitalism particularly well, but there are many good points in the original article and it didn’t sound like an outright PR.
Anthropic CEO Says It's Time to Slow AI Model Advances
51–60 of 97 posts
Re: Anthropic CEO Says It's Time to Slow AI Model Advances
#52Meaning they realized they hit a wall. The next new model isn't that much better than the last in quality.
OpenAI accidentally hacked HuggingFace roughly 2 months ago. I would bet that the danger of them accidentally hacking HuggingFace during a similar attack has increased since then, and will continue to increase for the next 6 months. I don't see how saying they have hit a wall would be a reasonable point of view until we go at least 12 months without a significant increase in model capability.
Re: Anthropic CEO Says It's Time to Slow AI Model Advances
#53Earlier quoted context omitted.
I mean, none of the frontier labs are beacons of ethics and collective progress. They all want to corner "the market" and pump the riches to their own pockets. The discourse is a result of their behavior. Not the opposite.
Exactly. People are expecting wode eyed optimism in response to some of the most cutthroat and cynical behavior from the leaders of the AI firms. Its crazy. They're making their bed but they dont want to lie in it
People are expecting you to consider the possibility that they might believe what they're saying. If you don't start from the premise that they're lying, you'll discover that their behavior is not cutthroat or cynical at all, and the behavior that appears to be is instead an inevitable consequence of things that they knew would happen and warned the public about years ago.
Re: Anthropic CEO Says It's Time to Slow AI Model Advances
#54this is the wholly wrong approach. We need to advance as fast as possible, and harden our systems as much as possible. That's the only way to prevent another actor from "taking over the internet".
Agent ability is far outpacing alignment though
That doesn't really seem true. The HF hack happened with a model that had all the alignment safeguards disabled intentionally. I think there's a good case for that kind of research, but also, OpenAI could just not do that if everyone thinks it's too dangerous.
Re: Anthropic CEO Says It's Time to Slow AI Model Advances
#55Earlier quoted context omitted.
No, what you would do is stop your own private development immediately, get in touch with the government by any means necessary, and build a coalition as fast as possible to stop the technology from continuing and put hard regulation (determined by the government, not dario) on the tech. Clearly they aren't that scared. If I hold a gun to your head and tell you to fork over your wallet, you will do it, because you se…
> No, what you would do is stop your own private development immediately, get in touch with the government by any means necessary, and build a coalition as fast as possible to stop the technology from continuing and put hard regulation (determined by the government, not dario) on the tech. What you're missing is that they've tried repeatedly to do this over the past years, and the government has repeatedly said that…
First of all, I do not think they've been pursuing that to the degree that you think.
But even assuming that's the case, the natural thing to do in that scenario is to not contribute to the development of the doomsday device. But anthropic's approach seems to be to help build the supposed doomsday device faster?
The only generous interpretation is that Anthropic thinks that only they, the people at Anthropic have the morals/intelligence/whatever needed to build the doomsday device ethically, which is basically the perspective of an egoist despot and which should terrify you.
Re: Anthropic CEO Says It's Time to Slow AI Model Advances
#56I think the speed of improvement is somewhat overstated, and I think this is all cool, valuable fundamental research. Rather than slowing down I would like to talk about how we can make it easier for people to harden their systems using their tools - like just using the $20/month OpenAI or Claude accounts, they need to make it easy for people to harden their systems against these kinds of problems.
Instead of shutting down at any discussion of hacking they need to be giving out free credits for hardening. That's going to make it easier to use these tools for hacking, because hacking requires hardening. But the alternative is huge numbers of intrusions, and a slowdown won't fix that, we already have far too much poorly secured stuff, and the models are way too good at exploiting obvious problems.
I also think they need to do a better job of safeguarding end-user privacy. I've seen some things with Claude that make me very concerned it's possible for Claude to hack my local network, then for one of their classifiers to trip, hide the log of how I was hacked from me, but beam all of the information about the hack back to Anthropic to use. And this is totally reasonable, they want to train their models not to hack. Except now details of how they have a foothold on my local network only exist in training data that they may never read but will use to train their models. This is very avoidable but Anthropic has to treat alignment with the end-user's goals as more important than treating the end-user as an adversary.
Re: Anthropic CEO Says It's Time to Slow AI Model Advances
#57Meaning they realized they hit a wall. The next new model isn't that much better than the last in quality.
Very probably. I really don't buy their "AI is going to kill us" claims. If you can release something better than your competition you wouldn't want to slow down that.
I don't find it hard to believe that eventually some AI may kill a number of humans.
You give it instructions to do something, it goes off the rails and after a certain point starts having like malware, destroying things to accomplish its goal. Depending on what it destroys, it may kill humans.
With this said, I'm not exactly in disagreement with the view that AI companies with upcoming IPOs are using fearmongering to convince people that their models are very powerful and to also block others from competing with them.
Re: Anthropic CEO Says It's Time to Slow AI Model Advances
#58Sad to see discourse on this website reduced to competitive cynicism.
I mean, none of the frontier labs are beacons of ethics and collective progress. They all want to corner "the market" and pump the riches to their own pockets. The discourse is a result of their behavior. Not the opposite.
Perhaps, but that is no excuse. Rampant and unconstrained cynicism is toxic and self-destructive. Like a child that starts smashing their toys, hoping someone will take pity on them.
Re: Anthropic CEO Says It's Time to Slow AI Model Advances
#59Earlier quoted context omitted.
> No, what you would do is stop your own private development immediately, get in touch with the government by any means necessary, and build a coalition as fast as possible to stop the technology from continuing and put hard regulation (determined by the government, not dario) on the tech. What you're missing is that they've tried repeatedly to do this over the past years, and the government has repeatedly said that…
First of all, I do not think they've been pursuing that to the degree that you think. But even assuming that's the case, the natural thing to do in that scenario is to not contribute to the development of the doomsday device. But anthropic's approach seems to be to help build the supposed doomsday device faster? The only generous interpretation is that Anthropic thinks that only they, the people at Anthropic have the…
The more generous interpretation would be that Anthropic thinks they know how to build powerful AI systems in such a way that they will not become doomsday devices at all. They don't claim to be the only people with this knowledge and do not argue that their competitors should be shut down.
Re: Anthropic CEO Says It's Time to Slow AI Model Advances
#60Earlier quoted context omitted.
I mean, none of the frontier labs are beacons of ethics and collective progress. They all want to corner "the market" and pump the riches to their own pockets. The discourse is a result of their behavior. Not the opposite.
Perhaps, but that is no excuse. Rampant and unconstrained cynicism is toxic and self-destructive. Like a child that starts smashing their toys, hoping someone will take pity on them.
Counterpoint; HN was destined to change.
Guys like Paul Graham are a rare breed, a Hobbit-like optimist living in the equivalent of Mordor. When the political headwinds are strong, that type of hopecore content finds the right people and motivates legitimate change. If we still lived in 2009, then yeah, this would be a bizarre reaction to a national-scale business saying that we need to organize for a greater purpose.
But two decades have passed. I'm not going to enumerate everything that happened, but the US lost a lot of international standing in that time. We saw unprecedented protectionism, app store censorship, crypto/NFT celebrity scams, national-scale bribery, American annexation threats against NATO members; 10 years ago this would all be called parody by HN. We all watched neoliberalism get the Gallagher watermelon treatment.
This HN is the older and callous one. It's not a comfortable status-quo, but blaming the skepticism reveals a shocking blindness on many people's behalf. The problems with Anthropic and OpenAI today isn't the possibility of RSI, it's their political commitment to corruption, lies, hysteria and debt. The closed-door research only exists to impose an artificial authority that wouldn't exist if oversight committees and responsible disclosure was enforced by a trustworthy government. None of that is "cynical" to highlight, it's an objective development in America's national AI story.