Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I resigned from Anthropic today

twitter.com

Post reply on HN
Print view View original

851–860 of 935 posts

Re: I resigned from Anthropic today

#851

Post by tuesdaynight »

onewayfunction wrote: post #178

I'm pretty baffled by the degree of skepticism expressed here in response to some of Jacob's claims. After the events of the summer it feels like it takes a lack of imagination to not see a few plausible routes to disaster. It may be reasonable to believe these outcomes are not very likely or that we can stop before going too far (I tend to disagree). But I can't imagine doubting that the capabilities will soon be th…

You are trying to decode their answers as rational, but they are being just as emotional as the doomers people. Rationalization of emotional responses are very common in tech. They will say that GPT Sol does not work for X, which will probably not be the case in 6 months, and will use that as a proof that is just pure marketing. There's nothing you can say that will make them change their minds until they decide to be neutral about the subject and try to research current models capabilities

Re: I resigned from Anthropic today

#852

Post by dinfinity »

ericmay wrote: post #845

I’m not advocating for this. ~~~~~~ If you really believed these labs were going to result in the extinction of humanity and you saw it first hand, don’t you have a moral obligation not to quit your job but to go and try and kill everyone working on said project, blow it up, or otherwise stop it? I take these resignations with a grain of salt precisely for that reason. If I worked at a job and I saw some crazy guy or…

> I’m not advocating for this.

You're not advocating for it because murdering people is highly immoral and illegal. That answers your question for the most part.

You also have to remember that the magnitude and speed of this stuff is very new to pretty much everybody. It's not exactly trivial to go from "I'm just doing my job" to "I need to kill everybody in my company to save the world" in a year or two, especially if a very, very large part of society doesn't even see the justification for it.

I'm convinced that most people here (even though we have more knowledge closer to the edge of AI development) would show only disdain for an Anthropic employee that murdered everybody there because they believed strongly in the dangers of AI.

Re: I resigned from Anthropic today

#853

Post by sambuccid »

huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

There is a great book about this, published in 2014, called "superintelligence" by Nick Bostrom, i found it very serious and detailed. (It also felt like reading 100 sci-fy books in one)

Re: I resigned from Anthropic today

#854

Post by ericmay »

dinfinity wrote: post #852
ericmay wrote: post #845

I’m not advocating for this. ~~~~~~ If you really believed these labs were going to result in the extinction of humanity and you saw it first hand, don’t you have a moral obligation not to quit your job but to go and try and kill everyone working on said project, blow it up, or otherwise stop it? I take these resignations with a grain of salt precisely for that reason. If I worked at a job and I saw some crazy guy or…

> I’m not advocating for this. You're not advocating for it because murdering people is highly immoral and illegal. That answers your question for the most part. You also have to remember that the magnitude and speed of this stuff is very new to pretty much everybody . It's not exactly trivial to go from "I'm just doing my job" to "I need to kill everybody in my company to save the world" in a year or two, especially…

> You're not advocating for it because it is highly immoral and illegal. That answers your question for the most part.

Well truthfully I’m not advocating for it because I don’t have any insight into these labs or the true capability of these models. But as a hypothetical if someone knew with 100% certainty that Big AI Button was being built and it would kill all humans or destroy all of humanity there is no moral ambiguity that anyone with the means to do so should stop that button from being built and kill everyone involved to save our species. This doesn’t apply to just AI though, but that’s the topic at hand.

> It's not exactly trivial to go from "I'm just doing my job" to "I need to kill everybody in my company to save the world" in a year or two, especially if a very, very large part of society doesn't even see the justification for it.

I agree with you, which is also why I think “omg I’m quitting they’re going to kill everyone” or other sort of sensationalized comments or news articles should be viewed skeptically - it’s probably fear mongering. Quitting your job here just seems pointless and attention seeking. Likely quit for some other reason.

Re: I resigned from Anthropic today

#855

Post by sambuccid »

Cthulhu_ wrote: post #836
huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

A realistic scenario is that an LLM writes a virus, as one way to achieve a goal. The next step would be that it becomes self-replicating, installing models on infected devices which continue its mission (whatever it was). But that's the extent of it so far, and we have plenty of experience with dealing with viruses and the like. We can turn things off, shut power off, etc. Another scenario is that of military hardwa…

I'm not sure it's that trivial to disrupt, if AI mimics human behaviour in that it tries to preserve itself (as Antropic found a few years ago I think), I think it would be extremely difficult to disrupt if it's able to self replicate, because it can make decisions and at a much much faster speed than humans, and there are many places that we would think twice about fully shutting down, two that pop to mind are hospitals or international banking systems

Re: I resigned from Anthropic today

#857

Post by walthamstow »

vidarh wrote: post #732
Aeolun wrote: post #709

Earlier quoted context omitted.

That's basically the reasoning behind MAD, and it checks out? See what happened to any nation that ever gave away their nukes.

Does it check out? India and Pakistan both have nukes and keep semi-regularly fighting each other. MAD "on paper" prevents either side from going far enough to provoke the other into using nukes, but even then it's fundamentally flawed because it works on the assumption that both sides are both rational and believes the other side to be rational , as well as that both sides understands the others red lines well enoug…

Minor skirmishes are not war. China and India have border skirmishes with literal wooden sticks, neither side will use live ammo because they both have nukes.

Re: I resigned from Anthropic today

#858

Post by fallingbananna »

sambuccid wrote: post #853
huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

There is a great book about this, published in 2014, called "superintelligence" by Nick Bostrom, i found it very serious and detailed. (It also felt like reading 100 sci-fy books in one)

Can you elaborate on that?

What points was the book making?

Re: I resigned from Anthropic today

#860

Post by threatofrain »

areoform wrote: post #243
threatofrain wrote: post #213

Earlier quoted context omitted.

Before we discussed how important security was, we got insurance, we made libraries and products, we used compliance software, etc. Except how honest were we about all that stuff? How much risk was actually in the air, and what was keeping us accountable on security in either direction of over or under-investment? Now a reckoning is here. The potential to be attacked might actually translate to being attacked.

People have died due to ransomware attacks on hospitals. Powerplants have been attacked. Stuxnet and industrial control malware exists. What will the AI do that hasn't been tried before?

AI is about automation. Drones have killed people before. If they automatically kill people is that different? Is your worry that mere automation is not novel enough?

Snipers have killed people before. If snipers are now automated, is the worry that there’s not enough creativity to warrant a discussion past saying “same old, same old”?

Post reply on HN