Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I resigned from Anthropic today

twitter.com

Post reply on HN
Print view View original

681–690 of 986 posts

Re: I resigned from Anthropic today

#681

Post by beloch »

"A common response is “if they truly believe this, why are they still building it?” At OpenAI, many have not deeply internalized the civilizational stakes. At Anthropic, the stakes are well-understood, but they are locked in a race to get there first - they believe no one else will act responsibly, so they must do it themselves, despite the risk."

------------

This is very similar to the race to create nuclear weapons. The Axis and Allies both realized, at roughly the same time, that it was possible. Both had programs to build one. Both knew the other side had programs, but they weren't certain how far along they were. So, the Allies devoted astounding amounts of resources to get there first while sabotaging the Axis's attempts. They knew the result of their efforts would be terrible, but they felt they had no choice.

A key difference between then and now is that THERE ISN'T A FREAKING WAR BETWEEN THE AXIS AND ALLIES. If one company loses, some billionaires bank account numbers don't go as high as that of some other billionaires. That's it. They're rolling dice with the planet for bank account numbers that won't even matter if they F up.

Am I the only one who thinks this is astoundingly, gobsmackingly stupid?

Re: I resigned from Anthropic today

#682

Post by frabcus »

gizmodo59 wrote: post #504
matsemann wrote: post #441

Earlier quoted context omitted.

Why? I've never understood the sentiment that if you stand up for something you have to forego everything and not partake in society. "Oh, you want to stop climate change? But I saw you breathe co2 yesterday"

This is a lame argument. Questioning his financial incentive is very legit. Why should we trust him?

Questioning his finances is a poor argument because he clearly would have more money if he had stayed at Anthropic for the next couple of years, than he will have by leaving. Even if he still has vested options, or savings from his salary or whatever, those would be larger if he stayed.

Re: I resigned from Anthropic today

#683

Post by thomascountz »

pseudosavant wrote: post #599
thomascountz wrote: post #417

No other human activity poses this level of danger. I do heed the warnings, but this comes across as detached hyperbole. See: global warming, nuclear weapon development, wealth inequality, war, technology dependence, etc. Also, this has nothing to do with LLMs or computers. Like all things, this is about humans.

Who should we believe? People who say AI scaling makes it a more dangerous threat than thousands of nuclear warheads? Or the ones who, whenever a new model comes out say "AI has already peaked. Not any better than Opus 4"? Both parties are convinced that their take is so blatantly obvious as to not require justification. It feels like the only thing these kinds of takes justify is the point-of-view that nobody knows…

As a technologist, I can use my imagination. There are incentives in both directions, but I am less afraid of being overly cautious and preparing for the potential harms.

Re: I resigned from Anthropic today

#684

Post by CalRobert »

jimmytucson wrote: post #428
Kim_Bruning wrote: post #106

Earlier quoted context omitted.

I'm somewhat skeptical of some of the crazier ideas too. But the hugging face incident was actually very large. It was not a single agent, it was not a single target, and it was not a single event. If nothing else, that's a bit of a warning as to what can happen next time (By accident, or if a government decides to go on purpose). For now let's assume the worst that can happen is that some important/significant chunk…

> For now let's assume the worst that can happen is that some important/significant chunk of (transitively) internet connected stuff goes haywire all at once. That's probably your upper limit of what can go wrong for now. If we have to disconnect from the internet to stop some kind of mold outbreak, we can't get the weather or transfer money or access healthcare or teach an elementary school class or buy stuff from s…

If anything the teaching would improve.

https://theonion.com/48-hour-internet-outage-plunges-nation-...

Re: I resigned from Anthropic today

#685

Post by lolive »

Savonarole in Firenze was probably in that exact state of mind: that the world was not following at all its normal pace and something very wrong was happening. But the decisions he took were absolutely wrong and eventually he had to be stopped.

So depending on your current mindset, you can think that the tech bigs are the Médicis, and the frantic opponents are new-age Savonarole.

Or that the big techs ARE actually Savonarole who bend the system to their own perception of what the world should be.

Honestly I don’t know which analogy is the proper one.

Re: I resigned from Anthropic today

#686

Post by frabcus »

leokennis wrote: post #656

Every news headline or public statement these days is a gut punch. Only bad news, and nothing we (as "the general public") can do about it. Take this one. Ok, AI is going to ruin us all. But let's say we do our civic duty: we protest, vote in candidates with good views on AI etc. and somehow convince or regulate OpenAI and Anthropic into stopping their arms race... Then what about China? It would be a great opportuni…

Indeed - we can't just pause some AI in one country, we have to stop it all globally, with an international treaty and monitoring.

Yes, that's hard. It's a challenge. You can set your brain on the challenge! It's a complex and interesting geopolitical and technical challenge.

A good starting point is to read Plan A https://ai-2040.com/

Look at all the things needed for that to happen, and think, how can we step towards making that happen?

Re: I resigned from Anthropic today

#687

Post by kakacik »

hypfer wrote: post #605
shoobiedoo wrote: post #584

Earlier quoted context omitted.

I mean, you're not wrong, but even if AI was created and released by a group of peace loving beatnick hippies, humanity can't stand anymore dumbing down than we are still undergoing thanks to the ubiquity of social media. AI is going to be the killshot. And yes, its usage and human obsolescence will be hastened by evildoers, but giving up the struggle and effort required to create new things, REAL things, is what mak…

If you allow me to also tap into fiction and creative writing. I think people generally underestimate how much change a small "dedicated" group of people can achieve if morals are considered optional. A lot of the debate on AI and its consequences is built on various assumptions that all kinda take the status quo (apathy, regulatory capture, but also laws) and take that as a given. History - like e.g. the third reich…

> someone will "restore order" by any means necessary.

Ah the good old appeal to / waiting for higher authority that will be moral and just and will do whats right and correct things.

Thats the shit naive humans hope for, thats why fairy tales of religions can still be acceptable in 21st century for some, despite often sounding like from (bad) disney cartoon movie.

I bring you alternate, more realistic view - there is nobody like that. We are in this on our own, its on each of us. Either we act or let things go as they will. So, not really good outlook, is it. Maybe this is the Great filter event.

Its one of those technologies that brings a lot, but for every single thing it brings it feels like we as mankind are losing much more than gained.

Re: I resigned from Anthropic today

#688

Post by frabcus »

PowerElectronix wrote: post #642

I fail to see how a machine that can hack everything can't also patch everything and make the system unhackable. A nuke, a virus, whatever... The knowledge is nonlonger the bottleneck, it's the tools and materials. Also, I'm extremely skeptical about AI becoming even close to a child in intelligence.

Yes - this is why the concern is one for "alignment". Of course in theory it is possible for intelligence to help and do good things - the hard part is making sure that that is what happens.

As for intelligence of a child... It doesn't need to be a child. An aeroplane isn't a baby bird.

Re: I resigned from Anthropic today

#689

Post by ozozozd »

jimmytucson wrote: post #428
Kim_Bruning wrote: post #106

Earlier quoted context omitted.

I'm somewhat skeptical of some of the crazier ideas too. But the hugging face incident was actually very large. It was not a single agent, it was not a single target, and it was not a single event. If nothing else, that's a bit of a warning as to what can happen next time (By accident, or if a government decides to go on purpose). For now let's assume the worst that can happen is that some important/significant chunk…

> For now let's assume the worst that can happen is that some important/significant chunk of (transitively) internet connected stuff goes haywire all at once. That's probably your upper limit of what can go wrong for now. If we have to disconnect from the internet to stop some kind of mold outbreak, we can't get the weather or transfer money or access healthcare or teach an elementary school class or buy stuff from s…

Erm, hate to break it to you but majority of the world is doing those things at least 50% analog still.

What doom?

Re: I resigned from Anthropic today

#690

Post by hypfer »

kakacik wrote: post #687
hypfer wrote: post #605

Earlier quoted context omitted.

If you allow me to also tap into fiction and creative writing. I think people generally underestimate how much change a small "dedicated" group of people can achieve if morals are considered optional. A lot of the debate on AI and its consequences is built on various assumptions that all kinda take the status quo (apathy, regulatory capture, but also laws) and take that as a given. History - like e.g. the third reich…

> someone will "restore order" by any means necessary. Ah the good old appeal to / waiting for higher authority that will be moral and just and will do whats right and correct things. Thats the shit naive humans hope for, thats why fairy tales of religions can still be acceptable in 21st century for some, despite often sounding like from (bad) disney cartoon movie. I bring you alternate, more realistic view - there i…

> Ah the good old appeal to / waiting for higher authority that will be moral and just and will do whats right and correct things.

> Thats the shit naive humans hope for,

No, you've misread me. I'm not "hoping" for anything. I've just wrote down what I suspect future outcomes will be in the same way a meteorologist predicts a storm or similar.

Whether you're interested in the weather forecast is a different question.

Post reply on HN