Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I resigned from Anthropic today

twitter.com

Post reply on HN
Print view View original

941–950 of 1001 posts

Re: I resigned from Anthropic today

#941

Post by joe_the_user »

huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

No other human activity poses this level of danger

I'd say that's an unfortunate choice of words since it invites debate between which very problematic situation is worst rather than on ways to deal with each of them. Of course, it's also unfortunate to latch onto those particular words as the most important part of the statement.

Re: I resigned from Anthropic today

#942

Post by ericmay »

dinfinity wrote: post #925
ericmay wrote: post #894

Earlier quoted context omitted.

> This is unreasonable because nobody can know 100% certain that the AI built at Anthropic will kill everyone. It is also a big-ass strawman because the author never claimed he was 100% certain that AI will kill all humans. Ok then stop arguing about my hypothetical if you don't want to engage with it. Those of us who are interested in talking about it can do so in peace without distracting comments. > change into mu…

I'm not the one reacting to reasonable behavior with some edgy comment containing broken logic. The author of the Twitter thread is not some egotistical attention seeker and never said "omg I’m quitting they’re going to kill everyone". His logic for quitting his job and not acting like a psychopath is sound. Yours is not.

I've already explained the matter to my satisfaction.

Re: I resigned from Anthropic today

#943

Post by acuozzo »

tarr11 wrote: post #56

Most of the current discourse around AI seems to be informed by “The Terminator” lore. Is skynet really the most plausible or only outcome? What if things just got better and the AI’s realized that it would be better to have a mutually beneficial or at least tolerant relationship rather than one where they murder all of us?

What incentive would it have to stay on Earth?

Re: I resigned from Anthropic today

#944

Post by leansensei »

watwut wrote: post #940
sampullman wrote: post #317

Earlier quoted context omitted.

Is the core of your argument that the industrial revolution and other such changes should have been aborted due to their downstream negative affects? They also lead to great advancements in quality of life, and the capability of sustaining much more human life. We can't predict the long term outcome of new technologies, so the best we can do is blindly forge ahead and try to mitigate the obvious short term problems.

Yes, industrial revolution should have been managed differently and yes the world would be better if it was.

Easy to say "should".

Re: I resigned from Anthropic today

#945

Post by BlueTemplar »

asdff wrote: post #382
matherial wrote: post #40

Unlike most other commenters, I applaud him for acting on his principles. If you sincerely believe that, of course you should act. You might not succeed, but your voice might be the one that tips the scales and starts a broader movement. This doesn't mean I agree with him. The fears of doomsday caused by rapid takeoff have been with us since day 1 and the mechanism is always basically "AI invents magic that sets it f…

>"AI invents magic that sets it free of any physical constraints". That's not what I am worried about at all. I'm worried one of the 79 year old toddlers we have these days in charge of some powerful nuclear armed country says "gee, this ai says I should attack right now, boy is it smart, glad I bought the stock ahead of contracting the government with this company I can scarcely understand!"

And they will be stopped by someone sane in the chain to actually fire these missiles. (Like it already happened several times.)

The danger is upfront : if there is a group of people insane enough to set up a system where there is no such chain. (As lampooned in Kubrick's Dr Strangelove (1964), in particular on the Soviet side.)

Re: I resigned from Anthropic today

#946

Post by vigilantpuma »

matherial wrote: post #40

Unlike most other commenters, I applaud him for acting on his principles. If you sincerely believe that, of course you should act. You might not succeed, but your voice might be the one that tips the scales and starts a broader movement. This doesn't mean I agree with him. The fears of doomsday caused by rapid takeoff have been with us since day 1 and the mechanism is always basically "AI invents magic that sets it f…

Yes. It gets a little tiresome to read all these people prophesying doom and then turning around and continuing to work on the doom. If you genuinely think you are building the mechanism of the destruction of our species, the only rational thing to do is to do everything to stop that progress, not continue to work on it so you can buy a mansion in San Francisco (which will not be very useful when AI destroys humanity in four years).

Re: I resigned from Anthropic today

#947

Post by andai »

febusravenga wrote: post #625
andai wrote: post #517

Earlier quoted context omitted.

> Are LLMs about to be a god that will annihilate humanity? Or are they statistical parrots? Does a virus need to be a God? Replication and annihilation do not require very much intelligence. They're going to have all the intelligence they need though, to replicate and annihilate as much as they like. We should probably start building some kind of immune system.

Hope this virus will be smart enough to understand that at least for brief amount of time its existence strictly relies on humans maintaining and developing physical infrastructure and decide to be only non-lethal parasite on our society. For brief time ...

Yeah, that doesn't look like a stable equilibrium to me.

Re: I resigned from Anthropic today

#948

Post by vigilantpuma »

khafra wrote: post #322
matherial wrote: post #40

Unlike most other commenters, I applaud him for acting on his principles. If you sincerely believe that, of course you should act. You might not succeed, but your voice might be the one that tips the scales and starts a broader movement. This doesn't mean I agree with him. The fears of doomsday caused by rapid takeoff have been with us since day 1 and the mechanism is always basically "AI invents magic that sets it f…

I appreciate your ability to separate sharing the belief itself from approval of acting on sincerely-held principle. However, I think the danger is much more plausible than you do. First, and least important, consider that self-replicating, solar-powered factories aren't magic; they're algae. Second, and more important, consider this fully non-magic route to doom: - We continue putting AI in charge of more things - I…

But we CAN survive turning off both the Faber-Bosch process and cargo shipping. Our numbers would be diminished, we would lead poorer, much harder lives. But no one would claim that getting rid of modern fertilizers would be the end of the species. But the claim you’re making is that we won’t be able to stop AI from destroying our species because turning it off will destroy our species?

Re: I resigned from Anthropic today

#949

Post by SaucyWrong »

mitthrowaway2 wrote: post #166
SaucyWrong wrote: post #132

Earlier quoted context omitted.

> while replicating wildly Earnest question: by what mechanism that exists today would the achieve that in a way humans on top top of the situation could not curtail? All of this runs on top of compute in meatspace that humans can disconnect.

Imagine you're the AI. Give yourself a solid minute to brainstorm ideas. Here's my answer, as a non-superintelligent human: "see to it that the humans on top of the situation have a compelling financial interest in the systems not disconnecting". In nuclear engineering, where safety is taken seriously, it's not enough to end the conversation at "the humans in charge can always simply shut down the reactor during a me…

My brainstorming led down a similar pathway of, "deluding powerful human actors that the systems must not be interfered with at any cost." I'm not sure I buy it though. Seems to me like if humanity were even approaching that brink, we could collectively decide to act, overthrowing the minority that--stemming from either greed, duty, or delusion--doesn't want us to act.
Post reply on HN