Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anthropic drops flagship safety pledge

time.com

Post reply on HN
Print view View original

471–480 of 716 posts

Re: Anthropic drops flagship safety pledge

#471

Post by joshribakoff »

PowerElectronix wrote: post #464
ACCount37 wrote: post #432

Earlier quoted context omitted.

That's because it is. AI is powerful and AI is perilous. Those two aren't mutually exclusive. Those follow directly from the same premise. If AI tech goes very well, it can be the greatest invention of all human history. If AI tech goes very poorly, it can be the end of human history.

Same with everything, right? You could say the same with nukes, electricity, internet, the computer, etc... But if you look at it without paying attention to the "ultimate tool for humanity" hype, it doesn't really look that much of a threat or a salvation. It won't end civilization for dropping the guardrails, but it will surely enable bad actors to do more damage than before (mass scams, blackmail, deepfake nudes,…

I agree with all of that. Also consider that there is an argument that the guard rail only stops the good guy. Not saying that’s a valid argument though.

Re: Anthropic drops flagship safety pledge

#472

Post by raincole »

jdross wrote: post #358
shubhamjain wrote: post #352

I was wondering if it was because of heavy-handedness of the administration, but apparently: > The policy change is separate and unrelated to Anthropic’s discussions with the Pentagon, according to a source familiar with the matter. Their core argument is that if we have guardrails that others don't, they would be left behind in controlling the technology, and they are the "responsible ones." I honestly can't compreh…

Would nuclear energy research be a good analogy then? Seems like a path we should have kept running down, but stopped bc of the weapons. So we got the weapons but not the humanity saving parts (infinite clean energy)

Nuclear energy hasn't been slowed down much, let alone stopped. China has been building new reactors every year for more than a decade and there are >30 ones under construction.

The same will go with AI, btw. Westerners' pearl clenching about AI guardrails won't stop China from doing anything.

Re: Anthropic drops flagship safety pledge

#473

Post by ozozozd »

prodigycorp wrote: post #399
jplusequalt wrote: post #395

Earlier quoted context omitted.

I genuinely believe that website is responsible for a lot of the worst ideas currently permeating the technology sector.

pretty much the intellectual equivalent of looksmaxxing

Been thinking about the nature of this behavior for a long time, you have nailed it so well, no one will be able to take out this nail.

Re: Anthropic drops flagship safety pledge

#474

Post by observationist »

ACCount37 wrote: post #432
shubhamjain wrote: post #352

I was wondering if it was because of heavy-handedness of the administration, but apparently: > The policy change is separate and unrelated to Anthropic’s discussions with the Pentagon, according to a source familiar with the matter. Their core argument is that if we have guardrails that others don't, they would be left behind in controlling the technology, and they are the "responsible ones." I honestly can't compreh…

That's because it is. AI is powerful and AI is perilous. Those two aren't mutually exclusive. Those follow directly from the same premise. If AI tech goes very well, it can be the greatest invention of all human history. If AI tech goes very poorly, it can be the end of human history.

Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an 'intelligence explosion,' and the intelligence of man would be left far behind. Thus the first ultraintelligent machine is the last invention that man need ever make.

-Irving John Good, 1965

If you want a short, easy way to know what AGI means, it's this: Anything we can do, they can do better. They can do anything better than us.

If we screw it up, everyone dies. Yudkowsky et al are silly, it's not a certain thing, and there's no stopping it at this point, so we should push for and support people and groups who are planning and modeling and preparing for the future in a legitimate way.

Re: Anthropic drops flagship safety pledge

#475

Post by computerphage »

joshribakoff wrote: post #467
ACCount37 wrote: post #432

Earlier quoted context omitted.

That's because it is. AI is powerful and AI is perilous. Those two aren't mutually exclusive. Those follow directly from the same premise. If AI tech goes very well, it can be the greatest invention of all human history. If AI tech goes very poorly, it can be the end of human history.

You wouldn’t say that rolling dice is dangerous. You would say that the human who decides to take an action, depending on the value of the dice is the danger. I don’t think AI is dangerous. I think people are dangerous.

Why are people dangerous? You can just not listen to them.

Re: Anthropic drops flagship safety pledge

#476

Post by ACCount37 »

joshribakoff wrote: post #467
ACCount37 wrote: post #432

Earlier quoted context omitted.

That's because it is. AI is powerful and AI is perilous. Those two aren't mutually exclusive. Those follow directly from the same premise. If AI tech goes very well, it can be the greatest invention of all human history. If AI tech goes very poorly, it can be the end of human history.

You wouldn’t say that rolling dice is dangerous. You would say that the human who decides to take an action, depending on the value of the dice is the danger. I don’t think AI is dangerous. I think people are dangerous.

Modern AIs are increasingly autonomous and agentic. This is expected to only get more prominent as AI systems advance.

A lot of AI harnesses today can already "decide to take an action" in every way that matters. And we already know that they can sometimes disregard the intent of their creators and users both while doing so. They're just not capable enough to be truly dangerous.

AI capabilities improve as the technology develops.

Re: Anthropic drops flagship safety pledge

#477

Post by grayhatter »

afavour wrote: post #426
shubhamjain wrote: post #352

I was wondering if it was because of heavy-handedness of the administration, but apparently: > The policy change is separate and unrelated to Anthropic’s discussions with the Pentagon, according to a source familiar with the matter. Their core argument is that if we have guardrails that others don't, they would be left behind in controlling the technology, and they are the "responsible ones." I honestly can't compreh…

It's exhausting to keep with mainstream AI news because of this. I can never work out if the companies are deluded and truly believe they're about to create a singularity or just claiming they are to reassure investors/convince the public of their inevitability.

> I can never work out if the companies are deluded and truly believe they're about to create a singularity or just claiming they are to reassure investors/convince the public of their inevitability.

You can never figure out if the people selling something are lying about it's capabilities, or if they've actually invented a new form of intelligence that can rival or surpass billions of years of evolution?

I'd like to introduce you to Occam Razor

Re: Anthropic drops flagship safety pledge

#478

Post by ozozozd »

This drama arc of “I used to be so pure and good, but others made me evil” is so tiring.

I really miss the nerd profile who cared a lot more about tech and science, and a lot less about signaling their righteousness.

How did we get so religious/narcissistic so quickly and as a whole?

Re: Anthropic drops flagship safety pledge

#479

Post by adrianN »

afavour wrote: post #447
ACCount37 wrote: post #441

Earlier quoted context omitted.

It's a fairly mainstream position among the actual AI researchers in the frontier labs. They disagree on the timelines, the architectures, the exact steps to get there, the severity of risks. Can you get there with modified LLMs by 2030, or would you need to develop novel systems and ride all the way to 2050? Is there a 5% chance of an AI oopsie ending humankind, or a 25% chance? No agreement on that. But a short lin…

Sure, when you get rid of the timelines and the methods we'll use to get there, everyone agrees on everything. But at that point it means nothing. Yeah, AGI is possible (say the people who earn a salary based on that being true). Curing all known diseases is possible too. How will we do that? Oh, I don't know. But it's a thing that could possibly happen at some point. Give me some investment cash to do it. If you cla…

There are plenty of people that argue that you need nontechnological pixi dust for intelligence.

Re: Anthropic drops flagship safety pledge

#480

Post by lprhrp »

Schlagbohrer wrote: post #404
drzaiusx11 wrote: post #325

Public benefit corporations in the AI space have become a farce at this point. They're just regular corporations wearing a different hat, driven by the same money dynamics as any other corp. They have no ability to balance their stated "mission" with their drive for profit. When being "evil" is profitable and not-evil is not, guess which road they'll take...

Pete Hegseth also threatened to take, by dictat, everything Anthropic has. He can do that with the Defense Industrial Act or whatever its called if he designates them as critical to national defense.

Hmm, that could be the best "IPO" they'll ever get. Better check if Trump Jr.'s 1789 capital has shares like they did in groq (note the "q").
Post reply on HN