Anthropic's CEO Dario has annoyed me to no end with his "AI will take all the jobs in 6 months" doomer speeches on every podcast he graces his presence with.
[deleted]
Anthropic drops flagship safety pledge
511–520 of 716 posts
Re: Anthropic drops flagship safety pledge
#512Earlier quoted context omitted.
Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an 'intelligence explosion,' and the intelligence of man would be left far behind. Thus the first ultraintelligent m…
"There's no stopping it at this point" - Sure there is, if a handful of enormous datacenters pull the very large plugs (or if their shaky finances collapse), the dubiously intelligent machines will be turned off. They're not ultraintelligent yet. Stopping it merely requires convincing a relatively small number of people to act morally rather than greedily. Maybe you think that's impossible because those particular pe…
I'm not saying it's technically impossible, I'm saying that in the real world, it's not going to stop. Nobody is going to stop it. A significant number of people don't want it to stop. A minority of people are in the "stop AI" camp, and the ones with the money and power are on the other side.
It's an arms race replete with tribalism and the quest for power and taps into everything primal at the root of human behavior. There's no stopping it, and thinking that outcome can happen is foolish; you shouldn't base any plans or hopes for the future on the condition that the whole world decides AGI isn't going to happen and chooses another course. Humans don't operate that way, that would create an instant winner-takes-all arms race, whereas at least with the current scenario, you end up with a multipolar rough level of equivalence year over year.
Re: Anthropic drops flagship safety pledge
#513Earlier quoted context omitted.
Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an 'intelligence explosion,' and the intelligence of man would be left far behind. Thus the first ultraintelligent m…
John Good's quote is pretty myopic, it assumes machines make better machines based on being "ultraintelligent" instead of learning from environment-action-outcome loop. It's the difference between "compute is all you need" and "compute+explorative feedback" is all you need. As if science and engineering comes from genius brains not from careful experiments.
> As if science and engineering comes from genius brains not from careful experiments
100% this. How long were humans around before the industrial revolution? Quite a while
Re: Anthropic drops flagship safety pledge
#514Earlier quoted context omitted.
It's exhausting to keep with mainstream AI news because of this. I can never work out if the companies are deluded and truly believe they're about to create a singularity or just claiming they are to reassure investors/convince the public of their inevitability.
> I can never work out if the companies are deluded and truly believe they're about to create a singularity or just claiming they are to reassure investors/convince the public of their inevitability. You can never figure out if the people selling something are lying about it's capabilities, or if they've actually invented a new form of intelligence that can rival or surpass billions of years of evolution? I'd like to…
> if they've actually invented a new form of intelligence that can rival or surpass billions of years of evolution?
Human creations have surpassed billions of years of evolution at several functions. There are no rockets in nature, nor animals flying at the speed of a common airliner. Even cars, or computers or everything in the modern world.
I think this is a bit like the shift from anthropocentric view of intelligence towards a new paradigm. The last time such shift happened heads rolled.
Re: Anthropic drops flagship safety pledge
#515Of course they do. You would have to be delusional to think that they won't, at some point.
What's "entertaining" is more the speed at which it's happening. It took Google probably 15 years to fully evil-ize. Anthropic ... two? There is no "ethical capitalism" big tech company possible, esp once VC is involved, and especially with the current geopolitical circumstances.
Citation needed - see google and project maven. Of course that is all well in the past now - but for a brief moment google was capable of taking an ethical stance.
Re: Anthropic drops flagship safety pledge
#516Worth checking this post from someone who actually has worked on this change: > I take significant responsibility for this change. https://www.lesswrong.com/posts/HzKuzrKfaDJvQqmjh/responsibl...
This guy from Effective Altruism pivoted away from helping the poor to help try to control AI from being a terminator type entity and then pivoted to being, ah, its okay for it to be a terminator type entity. > Holden Karnofsky, who co-founded the EA charity evaluator GiveWell, says that while he used to work on trying to help the poor, he switched to working on artificial intelligence because of the “stakes”: > “The…
Effective altruism came from the "rationalist"
It was never about helping poor people.
For some reason, the rationalist movement and its offshoots are really pervasive in silicon valley. i don't see it much in the other tech cities.
Re: Anthropic drops flagship safety pledge
#517Earlier quoted context omitted.
Sure, when you get rid of the timelines and the methods we'll use to get there, everyone agrees on everything. But at that point it means nothing. Yeah, AGI is possible (say the people who earn a salary based on that being true). Curing all known diseases is possible too. How will we do that? Oh, I don't know. But it's a thing that could possibly happen at some point. Give me some investment cash to do it. If you cla…
There are plenty of people that argue that you need nontechnological pixi dust for intelligence.
Yes, quite unfortunately. That reeks to me of wishful thinking.
Maybe that was a sensible thing to think in 1926, when the closest things we had to "an artificial replica of human intelligence" was the automatic telephone exchange and the mechanical adding machine. But knowledge and technology both have advanced since.
Now, we're in 2026, and the list of "things that humans can do but machines can't" has grown quite thin. "Human brain is doing something truly magical" is quite hard to justify on technical merits, and it's the emotional value that makes the idea linger.
Re: Anthropic drops flagship safety pledge
#518I was wondering if it was because of heavy-handedness of the administration, but apparently: > The policy change is separate and unrelated to Anthropic’s discussions with the Pentagon, according to a source familiar with the matter. Their core argument is that if we have guardrails that others don't, they would be left behind in controlling the technology, and they are the "responsible ones." I honestly can't compreh…
That's because it is. AI is powerful and AI is perilous. Those two aren't mutually exclusive. Those follow directly from the same premise. If AI tech goes very well, it can be the greatest invention of all human history. If AI tech goes very poorly, it can be the end of human history.
> If AI tech goes very well
The IF here is doing some very heavy lifting. Last I checked, for profit companies don't have a good track record of doing what's best for humanity.
Re: Anthropic drops flagship safety pledge
#519Earlier quoted context omitted.
That's because it is. AI is powerful and AI is perilous. Those two aren't mutually exclusive. Those follow directly from the same premise. If AI tech goes very well, it can be the greatest invention of all human history. If AI tech goes very poorly, it can be the end of human history.
Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an 'intelligence explosion,' and the intelligence of man would be left far behind. Thus the first ultraintelligent m…
> support people and groups who are planning and modeling and preparing for the future in a legitimate way.
Who is doing that right now, exactly? And how can we take their tech and turn it into the next profitable phone app?
Re: Anthropic drops flagship safety pledge
#520Google: "Don't be evil."
Alphabet: "Do the right thing."
Anthropic: "Do the thing which seems right to you at the time--at speed."