Live data from Hacker News

Anthropic drops flagship safety pledge

time.com

661–670 of 716 posts

Re: Anthropic drops flagship safety pledge

#661
post #440

Earlier quoted context omitted.

It’s not the CEO’s fault - they had to take all that money to keep their org a non-profit. B corps are like recycling programs, a nice logo.

Are you saying that recycling is a scam?

Mostly, yeah. "Yet the industry spent millions telling people to recycle, because, as one former top industry insider told NPR, selling recycling sold plastic, even if it wasn't true." https://www.npr.org/2020/09/11/897692090/how-big-oil-misled-...

Re: Anthropic drops flagship safety pledge

#662

Earlier quoted context omitted.

This guy from Effective Altruism pivoted away from helping the poor to help try to control AI from being a terminator type entity and then pivoted to being, ah, its okay for it to be a terminator type entity. > Holden Karnofsky, who co-founded the EA charity evaluator GiveWell, says that while he used to work on trying to help the poor, he switched to working on artificial intelligence because of the “stakes”: > “The…

> then pivoted to being, ah, its okay for it to be a terminator type entity. Isn’t that the opposite of what he’s saying? He’s saying it could become that powerful, and given that possibility it’s incredibly important that we do whatever we can to gain more control of that scenario

> Isn’t that the opposite of what he’s saying?

The quote was from 2022 for the first pivot to AI to prevent it from becoming a terminator style entity. The last pivot was not in the quote but is the topic of this current Hacker News post, where takes credit for dropping the safety pledge:

"That decision included scrapping the promise to not release AI models if Anthropic can’t guarantee proper risk mitigations in advance."

I expect the next pivot will be that we need to allow the US military to use Anthropic to kill people because otherwise they will use a less pure AI to kill people and our Anthropic is better at only killing the bad guys, thus it is the lesser evil.

Re: Anthropic drops flagship safety pledge

#663
post #427

Earlier quoted context omitted.

> then pivoted to being, ah, its okay for it to be a terminator type entity. Isn’t that the opposite of what he’s saying? He’s saying it could become that powerful, and given that possibility it’s incredibly important that we do whatever we can to gain more control of that scenario

I think the poster here has an axe to grind, considering they quoted something that directly contradicted their point and didn't even notice.

The quote was only for the 2022 pivot to AI safety, the 2026 pivot away from AI safety is the topic of this hacker news post.

Re: Anthropic drops flagship safety pledge

#664
post #341

Earlier quoted context omitted.

> For all of you who thought Anthropic were “the good guys” Was anyone fooled by this? I mean, I know this is HN and there is a demographic here that gets all misty eyed about the benevolence of corporations. It takes a special kind of naivety to believe in those claims.

Plenty of people here actually bought into the do no evil, how great Apple is for the environment (with throw away soldered hardware), or whatever.

Oh yes, which is why I made the consideration that I should expect this sort of naivety here.

Re: Anthropic drops flagship safety pledge

#665

Anthropic's CEO Dario has annoyed me to no end with his "AI will take all the jobs in 6 months" doomer speeches on every podcast he graces his presence with.

I don't understand why some of these AI companies check their egos at the door and hire public relations companies. Yes, I understand they are changing the world but customers do not open their wallets when they are scared. Very few people I know are as avant-guarde as I am with AI, but, most people look at these new technologies and simply feel fear. Why pay for something that will replace you?

> public relations companies.

Sounds like one of the white collar jobs that LLMs were supposed to solve

Re: Anthropic drops flagship safety pledge

#666
post #574

Earlier quoted context omitted.

John Good's quote is pretty myopic, it assumes machines make better machines based on being "ultraintelligent" instead of learning from environment-action-outcome loop. It's the difference between "compute is all you need" and "compute+explorative feedback" is all you need. As if science and engineering comes from genius brains not from careful experiments.

Have you gotten any indication that machines won't have sensors?!

From what I can see we're working as hard as we can to build them. You can watch the "let's put this on a Raspberry Pi and see what happens" seeds of Skynet develop in real time.

There's something compelling about helping assemble the machine. Science fiction was completely wrong about motivation. It's fun.

Re: Anthropic drops flagship safety pledge

#667
post #79

Earlier quoted context omitted.

Why are you asking this question? You know what the answer is, you've just arbitrarily decided that it's specious in an attempt to frame rebuttals as unreasonable.

I'm open to reasonable rebuttals but all the rebuttals that I've seen so far are simply uninformed.

1. You don't believe in the mission or direction of US warfighters 2. Supporting warfighters is developmentally distinct from what you want your corporate competences and direction are. 3. you don't want military to be more safe an capable.

Re: Anthropic drops flagship safety pledge

#668
post #612

Earlier quoted context omitted.

> Focusing on Dario, his exact quote IIRC was "50% of all white collar jobs in 5 years" which is still a ways off, but to check his track record, his prediction on coding was only off by a month or so. If you revisit what he actually said, he didn't really say AI will replace 90% of all coders, as people widely report, he said it will be able to write 90% of all code. Ugh, people here seem to think that all software…

The problem is, the low hanging fruit, the stuff it's good at, is 90% of all software. Maybe more. And it's getting better at the other 10% too. Two years ago ChatGPT struggled to help me with race conditions in a C++ LD_PRELOAD library. It was a side project so I dropped it. Last week Codex churned away for 10 minutes and gave me a working version with tests.

I think that typescript is a language uniquely suited to LLMs though:

  - It's garbage collected, so variable lifetimes don't need to be traced
  - It's structurally typed, so LLMs can get away with duplicating types as long as the shape fits. 
  - The type system has an escape hatch (any or unknown)
  - It produces nice stack traces
  - The industry has more or less settled styling issues (ie, most typescript looks pretty uniform stylistically).
  - There is an insane amount of open source code to train on
  - Even "compiled" code is somewhat easy(er) to deobfuscate and read (because you're compiling JS to JS)
Contrast that with C/C++:

  - Memory management is important, and tricky
  - Segfaults give you hardly anything to work with
  - There are like a thousand different coding styles
  - Nobody can agree on the proper subset of the language to use (ie, exceptions allowed or not allowed, macros, etc.)
  - Security issues are very much magnified (and they're   already a huge problem in vibecoded typescript)
  - The use cases are a lot more diverse. IE, if you're using typescript you're probably either writing a web page or a server (maybe a command line app). (I'm lumping electron in here, because it's still a web page and a server). C is used for operating systems, games, large hero apps, anything CPU or memory constrained, etc.
I'm not sure I agree that typescript is "90% of all software". I think it's 90% of what people on hacker news use. I think devs in different domains always overestimate the importance of their specific domain and underestimate the importance of other domains.

Re: Anthropic drops flagship safety pledge

#669

Earlier quoted context omitted.

> I can never work out if the companies are deluded and truly believe they're about to create a singularity or just claiming they are to reassure investors/convince the public of their inevitability. You can never figure out if the people selling something are lying about it's capabilities, or if they've actually invented a new form of intelligence that can rival or surpass billions of years of evolution? I'd like to…

> if they've actually invented a new form of intelligence that can rival or surpass billions of years of evolution? Human creations have surpassed billions of years of evolution at several functions. There are no rockets in nature, nor animals flying at the speed of a common airliner. Even cars, or computers or everything in the modern world. I think this is a bit like the shift from anthropocentric view of intellige…

Without a doubt, AGI will be invented much faster with a model to copy from. But similar to rockets, first we'll needed basic gunpowder, then refined fuels, all well before purified kerosene, well before liquified h2 and o2. LLM feel a lot closer to gun powder than even solid rocket fuel. (but because I'm exhausted by the hype, I'm gonna claim that is based on nothing but vibes)

Re: Anthropic drops flagship safety pledge

#670
post #643
post #639

Earlier quoted context omitted.

> The opportunities you chose to list are the greedy ones. Technology covers healthcare. I don't see how it's "greedy" to want to cure cancer. But on some level I guess "wanting life to be better" is greedy. Your attitude is very European, and it's basically why your continent is being left behind. I'm not totally against Europe becoming the world's retirement home, as long as there are places in the world where peop…

> Technology covers healthcare. If you'd chosen to list that in the first place, I wouldn't have said what I did; "supercharge technology and productivity" is looking at everything through the lens of money and profit, not the lens of improving the human condition. > Your attitude is very European, and it's basically why your continent is being left behind And yours is very American. You talk about managing the risks…

> If you'd chosen to list that in the first place, I wouldn't have said what I did; "supercharge technology and productivity" is looking at everything through the lens of money and profit, not the lens of improving the human condition.

Bullshit. "Technology and productivity" are not the same thing as "money and profit". You're projecting your garden-variety European degrowth ideology onto what I wrote.

> Also, to be blunt, China's almost certain to win any economic or literal arms race you think you're part of; they make too much critical hardware now.

Europeans are so hilariously polarized against the US that they would prefer China, a literal authoritarian dictatorship, to "win any global economic arms race". I guess it's because China is too culturally distant for them to feel insecure over.

> AI is one of the few things that could, actually, literally, end up with the planet being physically disassembled. "All it needs" is solving the extremely hard challenges of a von Neumann replicator, and, well, solving hard problems is kinda the point of making AI in the first place.

It's not worth wringing our hands over science fiction scenarios.

Post reply on HN