[dupe] Discussion on source: https://news.ycombinator.com/item?id=49672510
Anthropic CEO Says It's Time to Slow AI Model Advances
71–80 of 97 posts
Re: Anthropic CEO Says It's Time to Slow AI Model Advances
#72Good luck convincing the chinese labs.
Do Chinese labs actually do things beyond distilling other models?
Re: Anthropic CEO Says It's Time to Slow AI Model Advances
#73Earlier quoted context omitted.
Very probably. I really don't buy their "AI is going to kill us" claims. If you can release something better than your competition you wouldn't want to slow down that.
You would in the case that you think you're in an unsafe race to misaligned superintelligence.
If you really believe that you wouldn’t be in the race and trying to profit from it in the first place.
Re: Anthropic CEO Says It's Time to Slow AI Model Advances
#74Earlier quoted context omitted.
Very probably. I really don't buy their "AI is going to kill us" claims. If you can release something better than your competition you wouldn't want to slow down that.
I don't find it hard to believe that eventually some AI may kill a number of humans. You give it instructions to do something, it goes off the rails and after a certain point starts having like malware, destroying things to accomplish its goal. Depending on what it destroys, it may kill humans. With this said, I'm not exactly in disagreement with the view that AI companies with upcoming IPOs are using fearmongering t…
Maybe when we have actual artificial intelligence which LLMs are clearly not.
Re: Anthropic CEO Says It's Time to Slow AI Model Advances
#75Earlier quoted context omitted.
I don't find it hard to believe that eventually some AI may kill a number of humans. You give it instructions to do something, it goes off the rails and after a certain point starts having like malware, destroying things to accomplish its goal. Depending on what it destroys, it may kill humans. With this said, I'm not exactly in disagreement with the view that AI companies with upcoming IPOs are using fearmongering t…
Maybe when we have actual artificial intelligence which LLMs are clearly not.
I think the Hugging Face attack is just a preview of what's to come. It doesn't need to be very intelligent to cause serious issues.
Re: Anthropic CEO Says It's Time to Slow AI Model Advances
#76Meaning they realized they hit a wall. The next new model isn't that much better than the last in quality.
Or it means that the models are becoming dangerously uncontrollable by performing actions that meet a given goal in unintended and unethical ways such as by hacking other sites. We all read about the huggingface hack, let’s not pretend that cannot happen with a more critical site/infrastructure.
Can you think what will it mean if the tech is used maliciously or placed into machines designed to kill?
Re: Anthropic CEO Says It's Time to Slow AI Model Advances
#77Meaning they realized they hit a wall. The next new model isn't that much better than the last in quality.
I have collected three reasonable versions from today’s comments: 1. They hit a wall from the technical perspective 2. Inference costs are getting out of hand and newer models require significantly more resources for marginal gains, meaning nobody is going to buy those models 3. They can’t afford the hardware for further scaling
So you’re saying that a model that gets to a goal by using the path of least resistance despite specific instructions to follow safety guidelines is perfectly fine to release into the wild or place into robots (such as those designed for killing and war)? You really can’t think of any reason how that might be bad for humanity?
Re: Anthropic CEO Says It's Time to Slow AI Model Advances
#78Meaning: slow his competitors; mainly because anthropic stopped focusing on product and quality. Instead their big push right now is preventing others from catching up.
> Instead their big push right now is preventing others from catching up.
This!
Their IPO is getting closer, but the Chinese models catching up will ruin it, so they want government to stop the competition.
Re: Anthropic CEO Says It's Time to Slow AI Model Advances
#79Earlier quoted context omitted.
Perhaps, but that is no excuse. Rampant and unconstrained cynicism is toxic and self-destructive. Like a child that starts smashing their toys, hoping someone will take pity on them.
Counterpoint; HN was destined to change. Guys like Paul Graham are a rare breed, a Hobbit-like optimist living in the equivalent of Mordor. When the political headwinds are strong, that type of hopecore content finds the right people and motivates legitimate change. If we still lived in 2009, then yeah, this would be a bizarre reaction to a national-scale business saying that we need to organize for a greater purpose…
Im less concerned about hn change and more about society and the public I have to coexist with. People use cynicism as a tool to avoid thinking or engage with reality.
It is just reactionary snark, and everyone nodds along. It isnt a position or analysis. Nobody even cares if it is accurate because the vibe is what matters. Nothing constructive comes of it and everyone is digging their own intellectual graves.
Re: Anthropic CEO Says It's Time to Slow AI Model Advances
#80Earlier quoted context omitted.
You would in the case that you think you're in an unsafe race to misaligned superintelligence.
If you really believe that you wouldn’t be in the race and trying to profit from it in the first place.
Most of the people I know working at AI Labs believe it. I think surveys back this up.
We have a situation where the people running the labs are claiming it's a big risk. Outside experts not working for the labs are claiming it's a big risk.
It's pretty rare for all the CEOs in the industry to write letters and testify to Congress that they should be regulated and law should be put in place for safety limits.