Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anthropic drops flagship safety pledge

time.com

Post reply on HN
Print view View original

491–500 of 716 posts

Re: Anthropic drops flagship safety pledge

#491

Post by LeifCarrotson »

observationist wrote: post #474
ACCount37 wrote: post #432

Earlier quoted context omitted.

That's because it is. AI is powerful and AI is perilous. Those two aren't mutually exclusive. Those follow directly from the same premise. If AI tech goes very well, it can be the greatest invention of all human history. If AI tech goes very poorly, it can be the end of human history.

Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an 'intelligence explosion,' and the intelligence of man would be left far behind. Thus the first ultraintelligent m…

"There's no stopping it at this point" - Sure there is, if a handful of enormous datacenters pull the very large plugs (or if their shaky finances collapse), the dubiously intelligent machines will be turned off. They're not ultraintelligent yet.

Stopping it merely requires convincing a relatively small number of people to act morally rather than greedily. Maybe you think that's impossible because those particular people are sociopathic narcissists who control all the major platforms where a movement like this would typically be organized and where most people form their opinions, but we're not yet fighting the Matrix or the Terminator or grey goo, we're fighting a handful of billionaires.

Re: Anthropic drops flagship safety pledge

#492

Post by biztos »

joshribakoff wrote: post #467
ACCount37 wrote: post #432

Earlier quoted context omitted.

That's because it is. AI is powerful and AI is perilous. Those two aren't mutually exclusive. Those follow directly from the same premise. If AI tech goes very well, it can be the greatest invention of all human history. If AI tech goes very poorly, it can be the end of human history.

You wouldn’t say that rolling dice is dangerous. You would say that the human who decides to take an action, depending on the value of the dice is the danger. I don’t think AI is dangerous. I think people are dangerous.

I would say that's moot, because OpenClaw has already shown us how fast the dice-rolling super AI is going to be let out of the zoo. Dario and Sam will be arguing about the guardrails while their frontier models are running in parallel to create Moltinator T-500. The humans won't even know how many sides the dice have.

Re: Anthropic drops flagship safety pledge

#493

Post by visarga »

observationist wrote: post #474
ACCount37 wrote: post #432

Earlier quoted context omitted.

That's because it is. AI is powerful and AI is perilous. Those two aren't mutually exclusive. Those follow directly from the same premise. If AI tech goes very well, it can be the greatest invention of all human history. If AI tech goes very poorly, it can be the end of human history.

Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an 'intelligence explosion,' and the intelligence of man would be left far behind. Thus the first ultraintelligent m…

John Good's quote is pretty myopic, it assumes machines make better machines based on being "ultraintelligent" instead of learning from environment-action-outcome loop.

It's the difference between "compute is all you need" and "compute+explorative feedback" is all you need. As if science and engineering comes from genius brains not from careful experiments.

Re: Anthropic drops flagship safety pledge

#494

Post by trvz »

LeifCarrotson wrote: post #491
observationist wrote: post #474

Earlier quoted context omitted.

Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an 'intelligence explosion,' and the intelligence of man would be left far behind. Thus the first ultraintelligent m…

"There's no stopping it at this point" - Sure there is, if a handful of enormous datacenters pull the very large plugs (or if their shaky finances collapse), the dubiously intelligent machines will be turned off. They're not ultraintelligent yet. Stopping it merely requires convincing a relatively small number of people to act morally rather than greedily. Maybe you think that's impossible because those particular pe…

Open models barely any worse than SOTA exist, and so does consumer-ish hardware able to run them. The genie’s out, the bottle broken.

Re: Anthropic drops flagship safety pledge

#495

Post by whatshisface »

shubhamjain wrote: post #352

I was wondering if it was because of heavy-handedness of the administration, but apparently: > The policy change is separate and unrelated to Anthropic’s discussions with the Pentagon, according to a source familiar with the matter. Their core argument is that if we have guardrails that others don't, they would be left behind in controlling the technology, and they are the "responsible ones." I honestly can't compreh…

Shouldn't we be a little more skeptical about these abstract arguments when a very concrete sale is on the line?

Re: Anthropic drops flagship safety pledge

#496

Post by cmrdporcupine »

shubhamjain wrote: post #352

I was wondering if it was because of heavy-handedness of the administration, but apparently: > The policy change is separate and unrelated to Anthropic’s discussions with the Pentagon, according to a source familiar with the matter. Their core argument is that if we have guardrails that others don't, they would be left behind in controlling the technology, and they are the "responsible ones." I honestly can't compreh…

We all made fun of Blake Lemoine and others for spending too many late nights up chatting with (ridiculously primitive by this year's standards) LLM chat bots and deciding they were sentient and trapped.

But frankly I feel like the founders of Anthropic and others are victim of the same hallucination.

LLMs are amazing tools. They play back & generate what we prompt them to play back, and more.

Anybody who mistakes this for SkyNet -- an independent consciousness with instant, permanent, learning and adaptation and self-awareness, is just huffing the fumes and just as delusional as Lemoine was 4 years ago.

Everyone of of us should spend some time writing an agentic tool and managing context and the agentic conversation loop. These things are primitive as hell still. I still have to "compact my context" every N tokens and "thinking" is repeating the same conversational chain over and over and jamming words in.

Turns out this is useful stuff. In some domains.

It ain't SkyNet.

I don't know if Anthropic is truly high on their own supply or just taking us all for fools so that they can pilfer investor money and push regulatory capture?

There's also a bad trait among engineers, deeply reinforced by survivor bias, to assume that every technological trend follows Moore's law and exponential growth. But that applie[s|d] to transistors, not everything.

I see no evidence that LLMs + exponential growth in parameters + context windows = SkyNet or any other kind of independent consciousness.

Re: Anthropic drops flagship safety pledge

#497

Post by heliumtera »

What is the significance of a company making a promise?

"We promise are not going to do __, except if our customers ask us to do, then we absolutely will".

What is the point? Company makes a statement public, so what?

Not the first time this company puts some words in the wind, see Claude Constitution. It's almost like this company is built, from ground up, upon bullshit and slop

Re: Anthropic drops flagship safety pledge

#498

Post by toss1 »

dsign wrote: post #295
toss1 wrote: post #258

Earlier quoted context omitted.

No, it is a famously coherent concept over millenia. Quis custodiet ipsos custodes? "Who will guard the guards themselves?" or "Who will watch the watchmen?" >>A Latin phrase found in the Satires (Satire VI, lines 347–348), a work of the 1st–2nd century Roman poet Juvenal. It may be translated as "Who will guard the guards themselves?" or "Who will watch the watchmen?". ... The phrase, as it is normally quoted in Lat…

Alas, historically speaking, most governments have been tyrannies. In recent decades, some of them have been less so, or slightly more representative or transparent. I think in Switzerland they go to referendums often. Beyond that, once you vote for a party due an issue you deeply care about, they get to do whatever they want day to day, without citizens having a regular recourse to stop them. Yes people can go to th…

So, most of the time in history, we have failed to guard the guards and watch the watchers...

Re: Anthropic drops flagship safety pledge

#499

Post by goodpoint »

hybrid_study wrote: post #400

Are markets so untamable that the only leverage is to become ultra-rich—and then act philanthropically? Incidentally, concentrated wealth lately looks less like stewardship and more like misanthropy.

> concentrated wealth lately looks less like stewardship and more like misanthropy

...only lately?

Re: Anthropic drops flagship safety pledge

#500

Post by slibhb »

LeifCarrotson wrote: post #491
observationist wrote: post #474

Earlier quoted context omitted.

Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an 'intelligence explosion,' and the intelligence of man would be left far behind. Thus the first ultraintelligent m…

"There's no stopping it at this point" - Sure there is, if a handful of enormous datacenters pull the very large plugs (or if their shaky finances collapse), the dubiously intelligent machines will be turned off. They're not ultraintelligent yet. Stopping it merely requires convincing a relatively small number of people to act morally rather than greedily. Maybe you think that's impossible because those particular pe…

Do you really think AI companies/researchers are motivated by greed? It doesn't seem that way to me at all.

Stopping AI would be immoral; it has the potential to supercharge technology and productivity, which would massively benefit humanity. Yes there are risks, which have to be managed.

Post reply on HN