There will be a big confab at the White House by the end of the month, announcing a voluntary pacing regime roughly along these lines, codified by executive order to avoid antitrust issues.
We must pace the frontier
411–420 of 942 posts
Re: We must pace the frontier
#412Earlier quoted context omitted.
He did actually specify that diffusion should be limited for authoritarian countries, which I appreciated. If his focus is really safety, distillation in countries that are bound by safety regulation should be fine
> bound by safety regulation should be fine I do not believe this is a feature we can differentiate between the "good" and "bad" guys, as the US is currently under a poor safety regulation regime. Democracies can elect unethical people, write bad laws, and have uncertain enforcement. In example, the current US admin regularly lambasts Europe because they try to have stronger regulation.
yes but the mention of alignment is inside the "pacing within democracies" section
Re: We must pace the frontier
#413Before Anthropic, I worked at Cruise for four years as it competed against Waymo. The culture rewarded (and demanded) moving quickly, trusting that the company could empirically discover the risks that the robot cars posed and iteratively solve them to keep up with the rate at which it was scaling out its technology. There was very little interest or appetite for coordinating or collaborating with other AV companies…
>It’s a beautiful Saturday morning with my family here in the East Bay. While I hope we have many more years, I don’t know how many more Saturdays I’ll be able to play outside with my kids in the sunshine so I’m going to make the most of the time we have.
Is there a social media training programme at Anthropic/OAI where they teach you to hint at some vague end-of-the-world scenario? They all sound the same.
Re: We must pace the frontier
#414Earlier quoted context omitted.
That’s only one metric. It doesn’t matter how cheap it is, if it continually fails to solve a problem. Yes they’re getting better. No, I don’t think this will be a panacea. If only for the fact that this is China (more specifically the CCP) after all - they’ve got their own plan, and altruism is not part of it.
Have you actually used the latest Chinese models? Because, in my experience, they're not just cheaper, they're actually better in many cases. In one recent experiment I did a few days ago on a task that I need in production at scale at my company, Qwen 3.8 and GLM 5.3 Flash were not just cheaper, the results were significantly higher quality than GPT 5.6 Sol. Even if China doesn't continue to release this stuff for f…
> Qwen 3.8 and GLM 5.3 Flash were not just cheaper, the results were significantly higher quality
How can they be higher quality than models they were distilled from?
Re: We must pace the frontier
#415Earlier quoted context omitted.
This makes no sense. The hardware is only going to get cheaper. In a decade, everybody could be running an ASI on their phones. What we need to do is to make unaligned AIs illegal and monitor for them, like we do with nuclear weapons. And make aligned AI strong enough to counter it, for deterrence and defense.
I downvoted you because I think the OPs model is far simpler to implement and suffers from fewer conflicts of interest. If we go down the regulation of alignment route, we'll have to ask experts to create those regulations and monitoring regimes. And which experts will the government ask? Oh, right, the parties that stand to benefit the most from regulation: OpenAI and Anthropic! When "the experts" make certification…
I think the point is even stronger. alignment is really not very well defined, nor are the mechanisms used to provide it. from what I understand it's more of a statistically imposed bias than an impenetrable gate.
personally I would like to flip this around, all of this is a consequence of some really extreme notions about investment. its a tail-wagging-the-dog. the investment community is apparently in love with the idea of an AI-scale vehicle in and of itself. given the amount of power they have, this is funneling a massive amount of resources into something that while really very interesting, probably would never be able to realize proportionate gains without taking down the system that built it. what's the societal control on that.
Re: We must pace the frontier
#416Earlier quoted context omitted.
Because I find the cognitive dissonance of saying it's dangerous while flying as fast as possible to an IPO too much to take. I just don't understand how others parse this differently.
So you're saying that if Dario truly thought what he says he does, he'd resign/dissolve the company somehow/unilaterally slow down? I see two problems with this: First, Dario thinks that OpenAI will continue racing even if he stops, so there is, by his values, no point in stopping. Do you disagree with this, and think he should stop anyway? Second, what's your take on, for example, Jacob Coxon, who did resign from An…
I get it. I do. I just believe we live in a world where our corporate leaders speak out of both sides of their mouths. And typically, people who get to these positions are not the Jacob Coxon's of the world.
That is to say, I've been places where I tried really hard within teams to change certain things that I thought were broken only to find out that what the leadership said and how they acted were divorced from my and many others experiences. And so I did leave in those instances (and please read this not as I didn't get my way but total dysfunction behind beautiful facades).
So, I do think if he really felt as he felt he'd do something different than ultimately yelling "somebody stop me!".
Re: We must pace the frontier
#417I would like to imagine pacing frontier is in interest of both (anthropic and open AI) as it will enable them to enlarged their depreciation
Re: We must pace the frontier
#418This is either performative or shallow. What does "aligned" even mean? Aligned with who? We have no universal code of ethics. Democracies kill and launch wars merely to have cheaper stuff, even when we are already rich. And why would China ever agree to stay in second place? Our society is built on the premise "the smart and powerful dominate". The idea of domination is deeply embedded in our capitalist model. Or cou…
Thank you. This is the kind of basic critical thinking that basically no one in technology today seems to possess. Some days I wonder if I've gone insane because everyone else seems to be staking life itself on wispy, undefined meaningless terms and unarticulated concepts that they assume are valid universals.
We've slid so backward, so quickly.
Re: We must pace the frontier
#419This is corporate speak for "LLMs have hit a wall". I mean, it's been obvious for a bit, lately nearly all gains have been from harnesses (or whatever you want to call all the non-LLM bits that make up a chatbot or agent). If Anthropic and OpenAI were still seeing exponential or even linear gains from scaling they'd be doing it because the rewards to reaching AGI or SGI before everyone else are basically infinite. If…
That's just so extremely difficult to believe.
Before December 2025 they were still intelligent code autocomplete or Stack Overflow bots, then they started one-shotting serious long horizon tasks. Now they've just solved a millennium prize problem.
In less than a year.
Re: We must pace the frontier
#420Earlier quoted context omitted.
I don't agree that the argument implies nationalization. Regulation would work if it slowed everyone down at the same rate, enough to mitigate the risks. The problem is that you need regulators who know what they're doing and strong international cooperation. If you regulate a fraction of the global market, you just create an incentive to shift development to other countries. Nationalization, being essentially the mo…
I think this is the correct take. And until we have an AI Hiroshima it’ll be difficult to get the international community to work together. It’ll take rogue AI (or AI-powered group) disabling a significant world power before everyone comes to the table. Otherwise, it just looks like MAD and the equilibrium holding to the powers that be.
It wasn’t the shock of Little Boy that ended the war. That was just a final chapter of a 10+ year journey of horror, despair, and destruction, and the impact of Little Boy cannot be considered outside of that journey.
Europe had been destroyed by June 1945, and yet the empire of Japan continued to fight.
If that pattern holds, a single malicious AI substantially disabling a single world power won’t end the AI race. Participants don’t learn by the defeats of others.
If you’d like to apply a metaphor maybe the 10+ years of world war is more appropriate, after which basically every participant save one was exhausted.