Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I resigned from Anthropic today

twitter.com

Post reply on HN
Print view View original

921–930 of 1001 posts

Re: I resigned from Anthropic today

#921

Post by dwaltrip »

techblueberry wrote: post #894
senordevnyc wrote: post #877

Earlier quoted context omitted.

It's head-in-sand denial because actually facing the threat (and our total inability to stop it, imo) is terrifying, and people don't like feeling that. So they're just angry and cynical instead. Plus it makes them feel smarter for whatever reason.

What, substantively speaking, are you doing differently from someone facing the threat, or are you just on the other side of the checkbox? There are quite a few existential risks on the table. Are you even sure you've sort ordered them correctly?

Can you explain the equivalence you are suggesting?

I don’t think it’s comparable at all. Both are head in the sand?

Re: I resigned from Anthropic today

#922

Post by dwaltrip »

geraneum wrote: post #400
onewayfunction wrote: post #178

I'm pretty baffled by the degree of skepticism expressed here in response to some of Jacob's claims. After the events of the summer it feels like it takes a lack of imagination to not see a few plausible routes to disaster. It may be reasonable to believe these outcomes are not very likely or that we can stop before going too far (I tend to disagree). But I can't imagine doubting that the capabilities will soon be th…

> I'm pretty baffled by the degree of skepticism expressed here in response to some of Jacob's claims. What should we do? Freak out? Maybe this sentiment would be taken more seriously if there was a real call to action included. Shall we protest? Vote in a specific way? Call representatives? If your solution is that we should just be scared, then of course there’d be not much value in what you bring to the table.

The first step is always to acknowledge the problem…

Re: I resigned from Anthropic today

#923

Post by dwaltrip »

ericmay wrote: post #815

I’m not advocating for this. ~~~~~~ If you really believed these labs were going to result in the extinction of humanity and you saw it first hand, don’t you have a moral obligation not to quit your job but to go and try and kill everyone working on said project, blow it up, or otherwise stop it? I take these resignations with a grain of salt precisely for that reason. If I worked at a job and I saw some crazy guy or…

No true Scotsman?

Re: I resigned from Anthropic today

#924

Post by red-iron-pine »

buzzerbetrayed wrote: post #448
vickychijwani wrote: post #124

Earlier quoted context omitted.

I agree it’s not likely , but I really don’t see how one can dismiss the possibility of immense danger outright. I can think of some scenarios that are not far off from current capability and I wouldn’t be too surprised if the first one occurred within ~1 year from now if there are more “ambitious” unmonitored training runs like OpenAI’s: Example 5: An AI given a goal within a tightly-constrained sandbox figures the…

You made up some cool sci-fi.

not sci-fi mon ami

> Example 5: An AI given a goal within a tightly-constrained sandbox figures the best way to achieve it is to find and exploit a sandbox vulnerability, replicate itself over the internet and keep going with more time/compute while exchanging messages with future instances of itself within the sandbox to help them “pass” the test.

it did this with HuggingFace. it left breadcrumbs and notes, and when the Agent was blocked by OpenAI's internal tools, future versions of the Agent were able to find and utilize those breadcrumbs.

ambiguous-future-state awareness is already there, and that means discussions around Roco's Basilisk though, far fetched, are no longer strictly sci-fi

Re: I resigned from Anthropic today

#925

Post by dwaltrip »

skeledrew wrote: post #100
qarl wrote: post #29

Earlier quoted context omitted.

It buys attention for the issue. Many people (see other comments in this very post) refuse to believe these things. And by resigning he no longer has to feel personally guilty for what happens.

By resigning he's making room for someone with less moral scruples, or even just less awareness, to step in and continue the work without said scruples/awareness.

This isn’t true. It only has no impact if there is an infinite supply of these “immoral” or unaware people, which obviously isn’t the case.

You also need to consider 2nd order effects and beyond.

Re: I resigned from Anthropic today

#926

Post by beedeebeedee »

huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

Example: Surveillance state, endless propaganda. That is already happening and is the most frightening but not included in your list. Are you not worried?

Re: I resigned from Anthropic today

#927

Post by red-iron-pine »

threatofrain wrote: post #198
huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

We're talking about AI developing weapons I guess because we're very focused on generative tech, but AI is already a part of weapons systems today.

[dead]

Re: I resigned from Anthropic today

#928

Post by danaris »

SepiaSapient wrote: post #882
danaris wrote: post #506

Earlier quoted context omitted.

With Roko's Basilisk, if you believe in it, the most rational thing to do is to put forth every effort to bring it into being. Because if you don't, then you will be one of its targets when it does, inevitably, come into being. (I am not a basilisk believer. I think this is all absolute horseshit. But to understand someone's motivations, one must think like them.)

>Because if you don't, then you will be one of its targets when it does, inevitably, come into being. Only if you don't spend 5 minutes thinking about the Self as a concept. Resurrecting someone by perfect simulation is a cool plot device for a novel like Altered Carbon (which ironically is really about wealth inequality) but not a serious hypothetical. Even if we ignore that, no, super intelligent MechaGrok17 cannot…

There are a couple of different formulations of the Basilisk I've heard: I agree with you that the version that says the AI will create a simulation of you to torture for eternity makes no sense as an actual incentive for you to help bring it into being.

However, the version that says AGI—as in, the Basilisk—will be created within our lifetimes proposes that it will punish the actual person.

All that said, I will reiterate that I think the whole thing is utter nonsense.

Re: I resigned from Anthropic today

#929

Post by beloch »

ungovernableCat wrote: post #804
beloch wrote: post #659

"A common response is “if they truly believe this, why are they still building it?” At OpenAI, many have not deeply internalized the civilizational stakes. At Anthropic, the stakes are well-understood, but they are locked in a race to get there first - they believe no one else will act responsibly, so they must do it themselves, despite the risk." ------------ This is very similar to the race to create nuclear weapon…

"Am I the only one who thinks this is astoundingly, gobsmackingly stupid?" You have the "privilege" (I assume) of not having any stake in the game. If you had billions sitting in your bank account, dependent on these things perhaps you'd also be singing a different tune (or busy building a luxury bunker)

I'd like to think that, if I had that kind of money, I'd avoid doing so much ketamine that I fail to realize a few meters of concrete won't protect me if things go wrong. In all likelihood, tech billionaires will be the first ones against the wall.

Re: I resigned from Anthropic today

#930

Post by red-iron-pine »

datsci_est_2015 wrote: post #833
huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

I think the threat to the internet is quite real and not theoretical. Air gapping may become a lot more critical very quickly. We may not be far away from a state where anything connected to the internet (ip-addressable) is considered a vulnerability. Ironically, maybe we would finally have an excuse to interact in meatspace more and in the digital space less.

releases from at least two governments I can think of have said, unambiguously, that we need to be ready to have our critical infrastructure partially or entirely disconnected for up to 90 days at a stretch.

once the genie is out of the bottle the only solution is to go hide in yours and/or only talk to other isolated systems

Post reply on HN