Live data from Hacker News

We must pace the frontier

darioamodei.com

961–970 of 1001 posts

Re: We must pace the frontier

#961

At what point do we stop engaging with Anthropic’s leadership in good faith and acknowledge their track record, - no open weights - can’t use claude to research AI - train on everyone else’s IP and sell it back to them - 8 regulatory capture attempts and counting - so controlling they are the only US company blacklisted by the US government This is not effective altruism / rationalism gone wild, it’s just monopolisti…

100%, rules for thee not for me

Re: We must pace the frontier

#962

Earlier quoted context omitted.

In the system cards. Anthropic will literally make Claude sabotage you silently instead of downgrading you to Opus if you try to use Fable for AI research.

Don't they just fallback to opus non-silently?

Only if you're doing cybersecurity or biology stuff. If you're a possible competitor, they will invisibly fuck with Fable's weights instead to deliberately sabotage you.

Re: We must pace the frontier

#963

Earlier quoted context omitted.

Problem with Anthropic is that they benefited massively from open source and now are completely against it. It’s hypocrisy at its worst. Why do they get to benefit, but not everyone else?

If this is going to factor into my reasoning about the topic at hand, it will surely be a secondary or tertiary consideration at best. Same goes for much of the list in the root comment. We are discussing the possibility of severe adverse societal/world/human impacts from AI, why should open vs closed source/weights, copyright violations, anti-competitive moves, corporate hypocrisy, etc be so heavily weighted in the…

IMO, if the model is too dangerous for everyone to have access to, then no one should. I do not trust a for-profit company to be good stewards of such technology. Nor do I trust the government, any government.

Re: We must pace the frontier

#964
post #937
post #918

Earlier quoted context omitted.

> OpenAI / Anthropic models have largely stopped advancing - that's not a good look when you're pre-IPO and Chinese models are catching up. What is going on in this thread? Are these even real comments? While I strongly agree on the need to pace the frontier, I do think there are reasonable arguments that could be made against it. Non of those are being made here though. It's just speculation that the CEOs of the fas…

They may be genuinely concerned, but that's beside the point. You may think a technology holds too much power for someone to wield it, and therefore come to conclusion that you, the benevolent, the fluffy, the unicorn, with your 3 unicorn friends are the ONLY ones in the world of 8 billion people that are responsible enough to hold it. Even though I don't think Dario is such a person, because unlike their LLMs, I do…

If you want to limit the power of companies you need regulation...

I agree with you that we can't allow a handful of people to hold this power themselves, so what are you arguing for here? That the answer is to just leave AI labs to carry on as they are?

The government should be ensuring that the technology is being developed with caution, but since the government seems to have no interest in that, this is at least better than nothing.

Your solution of giving everyone access to super intelligence is absurd. A good guy with super intelligence isn't going to protect you from the bad guy with super intelligence. If someone uses it to poison your water supply with a bioweapon or to take out power stations, you and your family will be dead long before your super intelligence hacks together plan to stop it.

Re: We must pace the frontier

#965

At what point do we stop engaging with Anthropic’s leadership in good faith and acknowledge their track record, - no open weights - can’t use claude to research AI - train on everyone else’s IP and sell it back to them - 8 regulatory capture attempts and counting - so controlling they are the only US company blacklisted by the US government This is not effective altruism / rationalism gone wild, it’s just monopolisti…

Wait... Which of the 2 did Altman not do where Amodei did? I mean, maybe not quite 8 attempts at regulatory capture. But come on. And the only reason OpenAI didn't get blacklisted is they rolled over for the worst applications.

Re: We must pace the frontier

#966

Earlier quoted context omitted.

> Suppose it was, I don't know, Australia producing SOTA open weights, because they believed it was the right thing to do for the benefits of humanity - would Dario propose making Australia a geopolitical adversary too? I'd expect he thinks that people are capable of realizing that AI is very dangerous and also that democracies end up mostly representing the will of the people, from which it follows that Hypothetical…

> This argument doesn't work for countries which don't care what their citizens want, like China. you didn’t have to use China as an example, the US clearly does not care what its citizens want as the most popular policies are never even discussed or proposed in congress meanwhile, China destroying their housing market to decommidify it so everyone can have housing…they seem to care about their people more

another obvious example is climate change, where China is way more committed to solving it than the US. the division of the world into good guys and bad guys by a rather arbitrary definition of democracy looks so naive, even stupid. Anthropic is not a Democratic institution either, should we trust it? if this proposal for pacing the frontier is to be taken seriously, Dario should give its competitors, also international ones, the same assumption of good faith that he demands from us.

Re: We must pace the frontier

#968
post #908

Earlier quoted context omitted.

What a foolish comment, LLMs empower capital owners far more than the 'working class'

Only in how they are deployed, not in principle.

Completely correct. We are moving into a world where any person can have anything they want, only limited by capacity to produce it. People love information and digest great quantities of it. The marginal cost of information production is going to zero. This is only bad for people who spend their time shuffling bits. If you want to not suffer, go do things other than move bits around.

Re: We must pace the frontier

#969
post #131

I'd like to have some info what those independent evaluators are supposed to do exactly - or which risks Dario specifically sees as being "unaligned". Evidently, with "pacing" he doesn't mean stopping the production of ever-more powerful models, so what exactly does he want to pace here?

[flagged]

Re: We must pace the frontier

#970

At what point do we stop engaging with Anthropic’s leadership in good faith and acknowledge their track record, - no open weights - can’t use claude to research AI - train on everyone else’s IP and sell it back to them - 8 regulatory capture attempts and counting - so controlling they are the only US company blacklisted by the US government This is not effective altruism / rationalism gone wild, it’s just monopolisti…

I mostly agree with you, but this one:

"- so controlling they are the only US company blacklisted by the US government"

They were blacklisted by the US government because they didn't bribe Trump. While all the rest of the tech elite lined up to siphon a few million dollars through various Trump grifts, Anthropic sat out the inauguration, the ballroom, etc. Then, when they mildly pushed back on surveilling US citizens with their technology, the administration banned them on that pretext.

If there's any evidence Anthropic is one of the better actors in an industry full of bad actors, that's it. They have nothing to be ashamed of in that particular series of events, as far as I can tell.

Post reply on HN