271–280 of 942 posts
Earlier quoted context omitted.
A rather unfair comparison. The whole calculus here is that others are also developing these systems which has led to a race. A much better comparison to the situation is the nuclear weapons arms race.
You mean China is not developing biological weaponds?
> We have sought a middle way: to show that it’s possible to build carefully and succeed commercially, and to make safety something on which AI companies compete. In other words, to create a race to the top > Transparency. Regardless of what commitments we make, the public deserves to know what is going on. Anthropic has been a supporter of transparency for a long time And then it goes on tangent of how we should pac…
> Suppose it was, I don't know, Australia producing SOTA open weights, because they believed it was the right thing to do for the benefits of humanity - would Dario propose making Australia a geopolitical adversary too? I'd expect he thinks that people are capable of realizing that AI is very dangerous and also that democracies end up mostly representing the will of the people, from which it follows that Hypothetical…
It's the same as it has been with privacy/cybersecurity for decades. Vast majority of population doesn't care about hypothetical dangers, no matter how many essays get published.
So democracies representing will of people doesn't really work in favor of Dario's case here.
History is also not on his side. Limiting technological progress in the name of safety has a pretty poor track record.
Earlier quoted context omitted.
> First: if they are unilaterally doing it: what about their competitors? Is this a chance for OpenAI to pass them? The only part of this plan Dario is unilaterally committing to is the "embedded evaluators" thing, which doesn't seem like it'll necessarily cause them to slow down much.
Well: do they listen to the evaluators? I think there are any number of reasons a "safety" person could be concerned about any state of the art models. And so I would expect at least one of those to apply to any model. The question then is: do we stop when the safety people say to (they will) or not?
If Anthropic just unilaterally does the evaluator thing and can't achieve cooperation of the rest of the plan, my guess is that it'll have some impact for a few months and then they'll just stop reacting to the evaluators' reports and the evaluators would stop bothering to report anything. I think the idea is that if Anthropic does get government support for this, the external evaluations will be legally binding. The problem with this, though, is that the current US government perhaps can't be trusted to consistently enforce a regulation on a company, rather than e.g. taking bribes to not do so.
I know the common take online is that this is Anthropic doing pre-IPO marketing. I don’t think it is. I think Dario is genuinely afraid of the inevitability of AI turning into internet slime mold: consuming the environment, turning it into its own playground, and eventually manipulating human culture along with it. So am I. That said, if slowing this down were possible, I think it would have happened by now. Maybe he…
The more I read about everything that has been written regarding AI regulation since the OAI/HF incident, and the more it reminds me of the nuclear arms race (although the potential consequences would possibly be very different). Surely, we managed to negotiate a Non-Proliferation Treaty with most of the countries of the globe. Cannot we take inspiration from that for AI?
Earlier quoted context omitted.
I'm not terribly optimistic. I don't think humans have really figured out what to do about historical materialism. More just remarking that this is a blog post that almost literally reads "fellow elites, let us ensure only we control the means of production. It is our duty to humanity to control the masses."
I’m trying to hear what you’re saying, but using these Marxist cliches makes your statements a little off putting to me.
AI, doesn't feel all that different to the mechanical loom that was the impetus behind a lot of Marx's early critiques of industrial technology.
You've got a really cool invention that replaces what was previously a skilled artisan trade. The wealthy then use this invention to excoriate the artisans and render the loom operators as replaceable commodities.
Dario is essentially writing a blog post about why only he (and a few other approved elite companies) should be trusted to own the loom and then make it available for everyone else to labor at. Like the capitalists of Marx's era, he genuinely believes it is better for everyone to have this arrangement and that it is natural people like him should sit as benevolent gods at the top of the pyramid.
It's fascinating to me to see the same underlying mechanisms that gave birth to some of the greatest failed political and social experiments of the 20th century bubbling up again so clearly with AI today.
Earlier quoted context omitted.
You mean, if Anthropic made all of their models open-weights from now on? I'm pretty sure Dario thinks this will be extremely un safe, because it'd provide unrestricted access to all the most capable/dangerous models to everyone, and hence doesn't do it. Why do you think it'd make the world more secure, if he did?
AI technology is now only audited by people who think like them. The ones that dont either dont join the company, or leave after a short stint as we have seen. That creates an information or feedback bubble which is not healthy nor productive.
Remember when we stopped developing nuclear power for no reason?
I enjoy Claude Code very much, have max privately and team premium at work, but the doomer marketing and this whole regulate-while-we're-ahead spiel is extremely annoying and makes me wish Anthropic gets trounced.
It is not doomer marketing - it is clever psy-ops magic trick.
1) Software is described in super-human terms when it is plain-old software. Was stockfish described like this ? no.
2) Datacenters become AI-factories.
3) Hardware becomes investible assets.
4) Malware becomes a super-human breakout (oai/hf)
All clever framing to market the new technology. If it is called for what it really is ie, a software tool, it is boring and does not sell so easily. What sells is the mystique.
"Claude Mythos is that guy down the pub who is so good at karate that if he used it on you, you'd die instantly, and that's why you'll never see him using karate.”
In this case, it's more: "I'm having to act with great restraint because my karate is so good. Everyone should do likewise."