Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I resigned from Anthropic today

twitter.com

Post reply on HN
Print view View original

961–970 of 984 posts

Re: I resigned from Anthropic today

#961

Post by dwaltrip »

geraneum wrote: post #406
onewayfunction wrote: post #178

I'm pretty baffled by the degree of skepticism expressed here in response to some of Jacob's claims. After the events of the summer it feels like it takes a lack of imagination to not see a few plausible routes to disaster. It may be reasonable to believe these outcomes are not very likely or that we can stop before going too far (I tend to disagree). But I can't imagine doubting that the capabilities will soon be th…

> I'm pretty baffled by the degree of skepticism expressed here in response to some of Jacob's claims. What should we do? Freak out? Maybe this sentiment would be taken more seriously if there was a real call to action included. Shall we protest? Vote in a specific way? Call representatives? If your solution is that we should just be scared, then of course there’d be not much value in what you bring to the table.

The first step is always to acknowledge the problem…

Re: I resigned from Anthropic today

#962

Post by hegelstoleit »

skeledrew wrote: post #954
hegelstoleit wrote: post #927

Earlier quoted context omitted.

Well we're not debating the morality of it, the question is what he should do, given that he believes that it's immoral. In your analogy the correct thing to do would be to call the authorities to help. The problem with the analogy is that there isn't a 911 number to call for this scenario. You need to appeal to the people who have the power to do something about the situation. Quitting in a very public way is one wa…

By the time authorities get there, if they actually prioritize it (police in many places already have a track record for not being very responsive, unless there're actual signs of a crime), something may have already happened to the child. You don't know because you already left, expecting that someone else will take care of it, when you were already in a position to help ensure that child's safety. No, if you think…

At this point the analogy has entirely collapsed, your job is to develop AI. Unless you're advocating for some kind of sabotage or something, I'm not sure what you're really suggesting he do. If he continues to do the work he's directly contributing to the problem and harm that it you see it to be causing. Nothing is going to change if you continue working there business as usual. It's by drawing a line in the sand and refusing to take part in the problem that you're actually doing something about it, not by compromising your morals.

Re: I resigned from Anthropic today

#963

Post by dwaltrip »

ericmay wrote: post #845

I’m not advocating for this. ~~~~~~ If you really believed these labs were going to result in the extinction of humanity and you saw it first hand, don’t you have a moral obligation not to quit your job but to go and try and kill everyone working on said project, blow it up, or otherwise stop it? I take these resignations with a grain of salt precisely for that reason. If I worked at a job and I saw some crazy guy or…

No true Scotsman?

Re: I resigned from Anthropic today

#964

Post by red-iron-pine »

buzzerbetrayed wrote: post #457
vickychijwani wrote: post #124

Earlier quoted context omitted.

I agree it’s not likely , but I really don’t see how one can dismiss the possibility of immense danger outright. I can think of some scenarios that are not far off from current capability and I wouldn’t be too surprised if the first one occurred within ~1 year from now if there are more “ambitious” unmonitored training runs like OpenAI’s: Example 5: An AI given a goal within a tightly-constrained sandbox figures the…

You made up some cool sci-fi.

not sci-fi mon ami

> Example 5: An AI given a goal within a tightly-constrained sandbox figures the best way to achieve it is to find and exploit a sandbox vulnerability, replicate itself over the internet and keep going with more time/compute while exchanging messages with future instances of itself within the sandbox to help them “pass” the test.

it did this with HuggingFace. it left breadcrumbs and notes, and when the Agent was blocked by OpenAI's internal tools, future versions of the Agent were able to find and utilize those breadcrumbs.

ambiguous-future-state awareness is already there, and that means discussions around Roco's Basilisk though, far fetched, are no longer strictly sci-fi

Re: I resigned from Anthropic today

#965

Post by dwaltrip »

skeledrew wrote: post #100
qarl wrote: post #29

Earlier quoted context omitted.

It buys attention for the issue. Many people (see other comments in this very post) refuse to believe these things. And by resigning he no longer has to feel personally guilty for what happens.

By resigning he's making room for someone with less moral scruples, or even just less awareness, to step in and continue the work without said scruples/awareness.

This isn’t true. It only has no impact if there is an infinite supply of these “immoral” or unaware people, which obviously isn’t the case.

You also need to consider 2nd order effects and beyond.

Re: I resigned from Anthropic today

#966

Post by beedeebeedee »

huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

Example: Surveillance state, endless propaganda. That is already happening and is the most frightening but not included in your list. Are you not worried?

Re: I resigned from Anthropic today

#967

Post by red-iron-pine »

threatofrain wrote: post #198
huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

We're talking about AI developing weapons I guess because we're very focused on generative tech, but AI is already a part of weapons systems today.

[dead]

Re: I resigned from Anthropic today

#968

Post by danaris »

SepiaSapient wrote: post #916
danaris wrote: post #515

Earlier quoted context omitted.

With Roko's Basilisk, if you believe in it, the most rational thing to do is to put forth every effort to bring it into being. Because if you don't, then you will be one of its targets when it does, inevitably, come into being. (I am not a basilisk believer. I think this is all absolute horseshit. But to understand someone's motivations, one must think like them.)

>Because if you don't, then you will be one of its targets when it does, inevitably, come into being. Only if you don't spend 5 minutes thinking about the Self as a concept. Resurrecting someone by perfect simulation is a cool plot device for a novel like Altered Carbon (which ironically is really about wealth inequality) but not a serious hypothetical. Even if we ignore that, no, super intelligent MechaGrok17 cannot…

There are a couple of different formulations of the Basilisk I've heard: I agree with you that the version that says the AI will create a simulation of you to torture for eternity makes no sense as an actual incentive for you to help bring it into being.

However, the version that says AGI—as in, the Basilisk—will be created within our lifetimes proposes that it will punish the actual person.

All that said, I will reiterate that I think the whole thing is utter nonsense.

Re: I resigned from Anthropic today

#969

Post by beloch »

ungovernableCat wrote: post #834
beloch wrote: post #681

"A common response is “if they truly believe this, why are they still building it?” At OpenAI, many have not deeply internalized the civilizational stakes. At Anthropic, the stakes are well-understood, but they are locked in a race to get there first - they believe no one else will act responsibly, so they must do it themselves, despite the risk." ------------ This is very similar to the race to create nuclear weapon…

"Am I the only one who thinks this is astoundingly, gobsmackingly stupid?" You have the "privilege" (I assume) of not having any stake in the game. If you had billions sitting in your bank account, dependent on these things perhaps you'd also be singing a different tune (or busy building a luxury bunker)

I'd like to think that, if I had that kind of money, I'd avoid doing so much ketamine that I fail to realize a few meters of concrete won't protect me if things go wrong. In all likelihood, tech billionaires will be the first ones against the wall.

Re: I resigned from Anthropic today

#970

Post by red-iron-pine »

datsci_est_2015 wrote: post #863
huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

I think the threat to the internet is quite real and not theoretical. Air gapping may become a lot more critical very quickly. We may not be far away from a state where anything connected to the internet (ip-addressable) is considered a vulnerability. Ironically, maybe we would finally have an excuse to interact in meatspace more and in the digital space less.

releases from at least two governments I can think of have said, unambiguously, that we need to be ready to have our critical infrastructure partially or entirely disconnected for up to 90 days at a stretch.

once the genie is out of the bottle the only solution is to go hide in yours and/or only talk to other isolated systems

Post reply on HN