Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anthropic drops flagship safety pledge

time.com

Post reply on HN
Print view View original

441–450 of 716 posts

Re: Anthropic drops flagship safety pledge

#441

Post by ACCount37 »

afavour wrote: post #426
shubhamjain wrote: post #352

I was wondering if it was because of heavy-handedness of the administration, but apparently: > The policy change is separate and unrelated to Anthropic’s discussions with the Pentagon, according to a source familiar with the matter. Their core argument is that if we have guardrails that others don't, they would be left behind in controlling the technology, and they are the "responsible ones." I honestly can't compreh…

It's exhausting to keep with mainstream AI news because of this. I can never work out if the companies are deluded and truly believe they're about to create a singularity or just claiming they are to reassure investors/convince the public of their inevitability.

It's a fairly mainstream position among the actual AI researchers in the frontier labs.

They disagree on the timelines, the architectures, the exact steps to get there, the severity of risks. Can you get there with modified LLMs by 2030, or would you need to develop novel systems and ride all the way to 2050? Is there a 5% chance of an AI oopsie ending humankind, or a 25% chance? No agreement on that.

But a short line "AGI is possible, powerful and perilous" is something 9 out of 10 of frontier AI researchers at the frontier labs would agree upon.

At which point the question becomes: is it them who are deluded, or is it you?

Re: Anthropic drops flagship safety pledge

#442

Post by bogzz »

Does anyone have insight into, or an interesting source to read, on what exactly Anthropic/OpenAI are doing/can do for a military? Reporters are unsurprisingly fearmongering about Claude "being used in surveillance, autonomous robots, and target acquisition" but AFAIK all Anthropic does is work with LLMs.

Are people really attempting to have LLMs replace vision models in robots, and trying to agentically make a robot work with an LLM?? This seems really silly to me, but perhaps I am mistaken.

The only other thing I could think of is real-time translation during special ops with parabolic microphones and AR goggles...

Re: Anthropic drops flagship safety pledge

#443

Post by DoughnutHole »

jdross wrote: post #358
shubhamjain wrote: post #352

I was wondering if it was because of heavy-handedness of the administration, but apparently: > The policy change is separate and unrelated to Anthropic’s discussions with the Pentagon, according to a source familiar with the matter. Their core argument is that if we have guardrails that others don't, they would be left behind in controlling the technology, and they are the "responsible ones." I honestly can't compreh…

Would nuclear energy research be a good analogy then? Seems like a path we should have kept running down, but stopped bc of the weapons. So we got the weapons but not the humanity saving parts (infinite clean energy)

Nuclear advancements slowed down due to PR problems from clear and sometimes catastrophic failure of commercial power plants (Three Mile Island, Chernobyl, Fukushima) and the vastly higher costs associated with building safer plants.

If anything the weapons kept the industry trucking on - if you want to develop and maintain a nuclear weapons arsenal then a commercial nuclear power industry is very helpful.

Re: Anthropic drops flagship safety pledge

#444

Post by tenthirtyam »

shubhamjain wrote: post #352

I was wondering if it was because of heavy-handedness of the administration, but apparently: > The policy change is separate and unrelated to Anthropic’s discussions with the Pentagon, according to a source familiar with the matter. Their core argument is that if we have guardrails that others don't, they would be left behind in controlling the technology, and they are the "responsible ones." I honestly can't compreh…

I always enjoyed the Terminator movie series, but I always struggled to suspend my disbelief that any humans would give an AI such power without having the ability to override or pull the plug at multiple levels. How wrong I was.

N.B. the time travel aspect also required suspension of disbelief, but somehow that was easier :-)

Re: Anthropic drops flagship safety pledge

#445

Post by logicallee »

nozzlegear wrote: post #439
logicallee wrote: post #413

Earlier quoted context omitted.

>Public benefit corporations in the AI space have become a farce at this point. They're just regular corporations wearing a different hat, driven by the same money dynamics as any other corp. Could you describe the model that you think might work well?

It sounds like OP thinks AI companies should just stop pretending that they care about the public benefit, and be corporations from the start. Skip the hand wringing and the will they/wont they betray their ethics phases entirely since everyone knows they're going to choose profit over public benefit every time. That model already exists and has worked well for decades. It's called being a regular ass corporation.

I understand, but being a regular corporation is not the only possible model. Can you think of something better?

Re: Anthropic drops flagship safety pledge

#446

Post by mikkupikku »

shubhamjain wrote: post #352

I was wondering if it was because of heavy-handedness of the administration, but apparently: > The policy change is separate and unrelated to Anthropic’s discussions with the Pentagon, according to a source familiar with the matter. Their core argument is that if we have guardrails that others don't, they would be left behind in controlling the technology, and they are the "responsible ones." I honestly can't compreh…

To paraphrase a deleted comment that I thought was actually making a good point, nuclear medicine and nuclear weapons are both fruit from the same tree.

Re: Anthropic drops flagship safety pledge

#447

Post by afavour »

ACCount37 wrote: post #441
afavour wrote: post #426

Earlier quoted context omitted.

It's exhausting to keep with mainstream AI news because of this. I can never work out if the companies are deluded and truly believe they're about to create a singularity or just claiming they are to reassure investors/convince the public of their inevitability.

It's a fairly mainstream position among the actual AI researchers in the frontier labs. They disagree on the timelines, the architectures, the exact steps to get there, the severity of risks. Can you get there with modified LLMs by 2030, or would you need to develop novel systems and ride all the way to 2050? Is there a 5% chance of an AI oopsie ending humankind, or a 25% chance? No agreement on that. But a short lin…

Sure, when you get rid of the timelines and the methods we'll use to get there, everyone agrees on everything. But at that point it means nothing. Yeah, AGI is possible (say the people who earn a salary based on that being true). Curing all known diseases is possible too. How will we do that? Oh, I don't know. But it's a thing that could possibly happen at some point. Give me some investment cash to do it.

If you claim "AGI is possible" without knowing how we'll actually get there you're just writing science fiction. Which is fine, but I'd really rather we don't bet the economy on it.

Re: Anthropic drops flagship safety pledge

#448

Post by scottLobster »

shubhamjain wrote: post #352

I was wondering if it was because of heavy-handedness of the administration, but apparently: > The policy change is separate and unrelated to Anthropic’s discussions with the Pentagon, according to a source familiar with the matter. Their core argument is that if we have guardrails that others don't, they would be left behind in controlling the technology, and they are the "responsible ones." I honestly can't compreh…

> Every frontier tech company is convinced that the tech they are working towards is as humanity-useful as a cure for cancer, and yet as dangerous as nuclear weapons.

Maybe some of the more naive engineers think that. At this point any big tech businesses or SV startup saying they're in it to usher in some piece of the Star Trek utopia deserves to be smacked in the face for insulting the rest of us like that. The argument is always "well the economic incentive structure forces us to do this bad thing, and if we don't we're screwed!" Oh, so ideals so shallow you aren't willing to risk a tiny fraction of your billions to meet them. Cool.

Every AI company/product in particular is the smarmiest version of this. "We told all the blue collar workers to go white collar for decades, and now we're coming for all the white collar jobs! Not ours though, ours will be fine, just yours. That's progress, what are you going to do? You'll have to renegotiate the entire civilizational social contract. No we aren't going to help. No we aren't going to sacrifice an ounce of profit. This is a you problem, but we're being so nice by warning you! Why do you want to stand in the way of progress? What are you a Luddite? We're just saying we're going to take away your ability to pay your mortgage/rent, deny any kids you have a future, and there's nothing you can do about it, why are you anti-progress?"

Cynicism aside, I use LLMs to the marginal degree that they actually help me be more productive at work. But at best this is Web 3.0. The broader "AI vision" really needs to die

Re: Anthropic drops flagship safety pledge

#449

Post by s1artibartfast »

adverbly wrote: post #436
ndr wrote: post #351

Worth checking this post from someone who actually has worked on this change: > I take significant responsibility for this change. https://www.lesswrong.com/posts/HzKuzrKfaDJvQqmjh/responsibl...

Did this guy actually write this? Incredibly long and verbose. I will fall short of accusing him of using an AI to generate slop, but whatever happened to people's ability to make short, strong, simple arguments? If you can't communicate the essence of an argument in a short and simple way, you probably don't understand it in great depth, and clearly don't care about actually convincing anybody because Lord knows nob…

This is where people go to post long verbose statements.

You can usually find the short version on Twitter.

Re: Anthropic drops flagship safety pledge

#450

Post by HardCodedBias »

ACCount37 wrote: post #432
shubhamjain wrote: post #352

I was wondering if it was because of heavy-handedness of the administration, but apparently: > The policy change is separate and unrelated to Anthropic’s discussions with the Pentagon, according to a source familiar with the matter. Their core argument is that if we have guardrails that others don't, they would be left behind in controlling the technology, and they are the "responsible ones." I honestly can't compreh…

That's because it is. AI is powerful and AI is perilous. Those two aren't mutually exclusive. Those follow directly from the same premise. If AI tech goes very well, it can be the greatest invention of all human history. If AI tech goes very poorly, it can be the end of human history.

"If AI tech goes very well, it can be the greatest invention of all human history"

As has been said at many all hands:

Let's all work on the last invention needed by humans.

Post reply on HN