Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I resigned from Anthropic today

twitter.com

Post reply on HN
Print view View original

841–850 of 907 posts

Re: I resigned from Anthropic today

#841

Post by chrisjj »

0x20cowboy wrote: post #250

*How?* and *Why?* The most intelligent people I know are the least likely to want to harm anyone or anything, and understand that diversity is fundamental and important to the universe. Without proof to the contrary, why would you think some super intelligence would want to hurt anyone? Because you would? If you are saying that some small bit of training data made the thing completely evil, then that really couldn’t…

> The most intelligent people I know are the least likely to want to harm anyone or anything, and understand that diversity is fundamental and important to the universe. Without proof to the contrary, why would you think some super intelligence would want to hurt anyone? Because you would?

How much intelligence is needed to build a nuke?

How much to press a launch button?

Re: I resigned from Anthropic today

#842

Post by card_zero »

SupremumLimit wrote: post #798
philipallstar wrote: post #786

Earlier quoted context omitted.

> Capitalism cannot do otherwise. I really think this sort of claim needs to stop. Capitalism, like "patriarchy" is not a force in its own right. Capitalism is a system that encourages value creation for one's customers. That's not perfect, but it's still best so far.

"value creation for one's customers" - that is a "not even wrong" level of misunderstanding of the system dynamics of capitalism. The primary driver is generation of _profit_ for owners of the means of production. Many things fall out of that - the tendency to monopolise, for example.

You think that the liberty of capitalism enables selfish assholes to have influence, while under the constraints of communism the people with the influence are not selfish assholes.

Re: I resigned from Anthropic today

#843

Post by bean469 »

aogaili wrote: post #12

He resigned and now what? There are thousands willing to do his role, and many labs are competing in that race. His resignation and his statement doesn't do anything but buy him attention which is what all this post about in my opinion.

Posting on the internet about you quiting your job is always going to be about attention

Re: I resigned from Anthropic today

#844

Post by ethbr1 »

snaking0776 wrote: post #190
Kim_Bruning wrote: post #106

Earlier quoted context omitted.

I'm somewhat skeptical of some of the crazier ideas too. But the hugging face incident was actually very large. It was not a single agent, it was not a single target, and it was not a single event. If nothing else, that's a bit of a warning as to what can happen next time (By accident, or if a government decides to go on purpose). For now let's assume the worst that can happen is that some important/significant chunk…

Generally I don’t think anyone is arguing about the for now part. I don’t think it’s crazy to extrapolate out a few years and ask what kind of danger we’ll be in then. A team of 10,000 agents just solved the Navier Stokes problem (sans bad behavior by the researchers). Even 1 year ago that would have been unimaginable. What happens to this risk view as: 1. Robotics begin rolling out more broadly across the world. 2.…

> 1. Robotics begin rolling out more broadly across the world.

To borrow the Strangelove quip, this is not only necessary but essential.

Broader physical robotic deployments are the glue that connects {cheap synthetic intelligence} to {physical outcomes}.

The former is a newer phenomenon, in the LLMs-can-approximate-higher-intelligence-well sense, so robotics deployments haven't caught up yet (outside of the highest ROI dark factory scenarios).

But it's too economically-valuable not to come.

Re: I resigned from Anthropic today

#845

Post by ericmay »

I’m not advocating for this.

~~~~~~

If you really believed these labs were going to result in the extinction of humanity and you saw it first hand, don’t you have a moral obligation not to quit your job but to go and try and kill everyone working on said project, blow it up, or otherwise stop it?

I take these resignations with a grain of salt precisely for that reason. If I worked at a job and I saw some crazy guy or gal was building a doomsday button and it really worked I’d like to think I would do something about it, not just quit my job. If you’re not going to actually do anything about it you might as well keep your job. Quitting doesn’t help anything.

Re: I resigned from Anthropic today

#846

Post by angoragoats »

ozozozd wrote: post #700
mitthrowaway2 wrote: post #610

Earlier quoted context omitted.

FWIW, before any nuclear weapons had ever been tested, a risk was identified that the first one might trigger a self-sustaining reaction in atmospheric nitrogen and destroy the entire planet. Faced with such a scenario, is the prudent next move: a) blow one up and see what happens, or b) do whatever you can to be sure it won't happen before conducting the first test, and make sure the confidence in the calculation is…

“Self-sustaining reaction,” like the recursive self improvement? Or perpetual machines? Is physics no longer a main subject at schools?

A self-sustaining reaction is exactly what happens with the fissionable material inside the nuclear weapon[0]. It seems like you’re trying to be pedantic on terminology here, but if you’re saying that a self-sustaining reaction is as un-scientific as a perpetual motion machine, you’re simply incorrect.

[0] https://en.wikipedia.org/wiki/Criticality_(status)

Re: I resigned from Anthropic today

#847

Post by dinfinity »

ytoawwhra92 wrote: post #575
hnlmorg wrote: post #408

Earlier quoted context omitted.

You’ve identified that the risks of nuclear weapons are theoretical. ie in theory we could blow up the world even though we haven’t yet done so. Well the worries about AI are equivalent in that those risks are discussed now because discussing them after they’ve happened is clearly too late. That’s the thing about risk. There’s no point discussing it after it’s happened and any discussions beforehand can easily be han…

The risks of nuclear weapons aren't theoretical. Nuclear weapons have killed people, destroyed infrastructure, and contaminated the environment. The Limited Test Ban Treaty was put in place after radioactive fallout from repeated nuclear weapons tests made people sick. So in fact we've done exactly what you suggest there's "no point" doing - used the things and then had a discussion after the fact about limiting futu…

> The risks of nuclear weapons aren't theoretical

The logic for the first bits of handwaving still applies: "I haven't seen any evidence of thermonuclear war happening, so it is unlikely to ever happen. The only time nukes were ever used outside of research was 80 years ago; the whole world recoiled, decided that they were way too dangerous and never used them again. Clearly they are not a real risk."

Re: I resigned from Anthropic today

#848

Post by shafyy »

internet_points wrote: post #768
shafyy wrote: post #727

Earlier quoted context omitted.

Watched one too many second-tier disaster movies?

One the one hand, it does sound like one of those movies. On the other, Idiocracy turned out to be quite prescient. (Most likely, we'll have some combination of human stupidity, LLM stupidity, and way too much compute in one place all working together to create a perfect storm of unchecked hacks that break something or other that ends up killing people in an unintended way. Then there's some half-hearted attempt at c…

Not saying that the current civilizations will never end or be shattered to pieces. Looking at history, many civilizations did. But saying LLMs are the highest risk factor for this is just insanity, based on all what we have seen so far (and based on these AI companies crying wolf, I mean, remember just a few months ago that Anthropic saying that Fable is too powerful to be released?)

Re: I resigned from Anthropic today

#849

Post by angoragoats »

ericmay wrote: post #845

I’m not advocating for this. ~~~~~~ If you really believed these labs were going to result in the extinction of humanity and you saw it first hand, don’t you have a moral obligation not to quit your job but to go and try and kill everyone working on said project, blow it up, or otherwise stop it? I take these resignations with a grain of salt precisely for that reason. If I worked at a job and I saw some crazy guy or…

Yes, exactly this. Externally, I have absolutely zero evidence that anything this person describes will come to pass, and I do have evidence that LLMs are not capable of many of the things that he mentions. So this seems like perhaps this person has drank the kool aid and is still not thinking clearly.
Post reply on HN