Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I resigned from Anthropic today

twitter.com

Post reply on HN
Print view View original

801–810 of 906 posts

Re: I resigned from Anthropic today

#801

Post by beej71 »

ozozozd wrote: post #725
DecoPerson wrote: post #628

To anyone doubting what AI could do humanity, just think about what a well-engineered virus could do. Currently, if a government ordered a special virus with Ethnicity-based targeting, a 2-year timer, and castration-effects instead deadly-effects — it wouldn’t be possible. Human engineers would push back or sabotage the effort out of moral duty. Even if they cooperated, it’s too advanced for a team of humans to actua…

Is the requirement that it’s a fancy new virus? US has 15 to 25 million people unprotected from Polio. Polio is still endemic in Afghanistan and Pakistan.

15-25 million is too small a number in this context. We're taking about 8 billion people suspectable to a new virus or collection of viruses far more lethal than polio. :(

Re: I resigned from Anthropic today

#802

Post by ObscureScience »

huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

I'm not sure where I place AI (of the current family), but more than directly worry about the actions they may perform I'm actually more worried what affects their usage can have on us as individuals and society.

Re: I resigned from Anthropic today

#803

Post by wallst07 »

mitthrowaway2 wrote: post #610
ytoawwhra92 wrote: post #575

Earlier quoted context omitted.

The risks of nuclear weapons aren't theoretical. Nuclear weapons have killed people, destroyed infrastructure, and contaminated the environment. The Limited Test Ban Treaty was put in place after radioactive fallout from repeated nuclear weapons tests made people sick. So in fact we've done exactly what you suggest there's "no point" doing - used the things and then had a discussion after the fact about limiting futu…

FWIW, before any nuclear weapons had ever been tested, a risk was identified that the first one might trigger a self-sustaining reaction in atmospheric nitrogen and destroy the entire planet. Faced with such a scenario, is the prudent next move: a) blow one up and see what happens, or b) do whatever you can to be sure it won't happen before conducting the first test, and make sure the confidence in the calculation is…

Good point, humanity chose a and the sky in fact did NOT ignite an destroy the planet. The hypothesis was very very far off from what I recall.

But it's good we talked about it.

Re: I resigned from Anthropic today

#804

Post by vidarh »

euroderf wrote: post #792
vidarh wrote: post #743

Earlier quoted context omitted.

The most compelling argument to me is "accidentally", due to AI that is made blind to consequences or don't care because it's geared towards a single goal (see e.g. the paperclip maximizer). We could ask if it is possible to end up with an AI that is smart enough to destroy humanity and at the same time still blind enough to consequences and/or callous enough to do it, but then again we have plenty of examples of hum…

> We could ask if it is possible to end up with an AI that is smart enough to destroy humanity and at the same time still blind enough to consequences and/or callous enough to do it Don't make the mistake of anthropomorphizing any A.I. trained under the direction of Larry Ellison.

Anthropomorphising or not is entirely irrelevant to that statement - the question remains the same.

Re: I resigned from Anthropic today

#805

Post by hypfer »

frabcus wrote: post #780
hypfer wrote: post #634

Earlier quoted context omitted.

Hmm that is a good question. I can't really provide a full picture here as I just make stuff up as I go, but the first word that came to my mind when reading your question was "atomization". Not sure what my brain wants to tell me with that, though. It might be one of the prerequisites for the current state. It might be what changes for greater change to quickly snap into existence. Or it might also be brainfart. Not…

Definitely these are forces that will do things. What their timing is relative to e.g. armies of AI robots, which could interfere with other humans living outside the internet/money, is part of the question. Hope your brain works on this and writes something - we have lots of very odd future possibilities, and we're not really ready or planning for them, so making them tangible helps.

From my understanding, the "armies of AI robots" are just the wishful thinking of the currently winning people, in the sense that that would be what would be necessary to cement their own power while hoarding all resources for themselves.

It's not a real scenario. It's just rationalizing the "oh shit, they're coming for my head if I don't stop optimizing for my own benefit". "What can I do what can I do.. I know.. Army of Robots. I am safe and have nothing to change about anything I do".

You can tell that it is just that by considering the logistics of the proposed scenario.

__

Every time we see vengeful AI or robots, we merely see a reflection of the currently winning meatbags, their character, their flaws, their fears and their goals.

___

On the logistics note:

You can I think also see the hubris of the IT people in that, because "supply chain" for them means "npm", and that can be replaced with an LLM.

It breaks minds to be able to create value out of nothing just with your brain. It makes you believe that the whole world would work like that.

It however does not. And you cannot just compensate that with blitzscaling robots. The supply chain of.. literally any object you can find around you is absurdly complex. Even the most simplest stuff.

And it's also not documented, but even documentation wouldn't help you there. Experience in requirements engineering should make the "why" there obvious.

Re: I resigned from Anthropic today

#806

Post by wallst07 »

huitzitziltzin wrote: post #177
skew-aberration wrote: post #151

Earlier quoted context omitted.

> I’m not interested in wild theories about AI driven labor market disruptions leading to widespread starvation Changes in political and economic power balance leading to unrest, conflict, death and deprivation is not a wild theory. It is literally the story of our entire species. If you discount all such concerns, you are simply being willfully ignorant of past precedents. In fact, I challenge you to describe any no…

I’m an economist. On the basis of current evidence, I view AI as a complement to human labor, not as a substitute for it. That’s the source of my rejection of the wild labor market disruptions theories. I just don’t see any evidence yet that whole categories of jobs are being eliminated, with the single exception (so far!) of the end of “professional essay writing services for cheating college students,” and similar…

[flagged]

Re: I resigned from Anthropic today

#807

Post by nirui »

huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

If I may guess, given how delicate chem or bio weapons are, even if AI can lay out a method to create one, the first thing the weapon would destroy is likely it's creator(s).

Of course, with the exception that such weapon is created by organized crime groups, such as drug cartels such.

Probably don't worry nation states tho, these guys already got way worse things in their warehouses than what individual baddies could ever imagine.

Re: I resigned from Anthropic today

#808

Post by debugnik »

ozozozd wrote: post #700
mitthrowaway2 wrote: post #610

Earlier quoted context omitted.

FWIW, before any nuclear weapons had ever been tested, a risk was identified that the first one might trigger a self-sustaining reaction in atmospheric nitrogen and destroy the entire planet. Faced with such a scenario, is the prudent next move: a) blow one up and see what happens, or b) do whatever you can to be sure it won't happen before conducting the first test, and make sure the confidence in the calculation is…

“Self-sustaining reaction,” like the recursive self improvement? Or perpetual machines? Is physics no longer a main subject at schools?

I assume the fear was an uninterrupted burn until the gas depletes, as in the Darvaza gas crater but atmosphere-wide.

Re: I resigned from Anthropic today

#809

Post by wallst07 »

waterTanuki wrote: post #146
huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

> What's the most dangerous thing that's happened with an LLM so far? I don't know, maybe a mass shooting? https://www.npr.org/2026/09/02/nx-s1-5953021/openai-tumbler-... Oh, and let's just forget the uncountable early deaths from the environmental disaster of the Datacenter buildout. It's not as sexy and doesn't make headlines, so those deaths don't really count or matter do they?

So the crazy who committed a horrible crime is the fault of a LLM? Maybe so.

Certainly there should be some protocols/laws followed here, but do you think this can extend out to cause mass damage or a singular event that should be investigated?

Re: I resigned from Anthropic today

#810

Post by wallst07 »

waterTanuki wrote: post #199
huitzitziltzin wrote: post #174

Earlier quoted context omitted.

I did know about the mass shooting but failed to mention it here. I’d put it in the “unlikely to be a widespread problem” category. If we’re in the “one AI driven mass shooting every four years” world for example it’s fair to call it a rare issue. The environmental impact seems either very overblown (e.g., water usage just isn’t that high) and the part that isn’t overblown is totally abatable (e.g., noise and emissio…

Ok, so what is the exact number of preventable deaths per year to build a silicon god you would be ok with?

Humans take risk adjusted decisions all the time, OP is still correct that LLM risk is FAR below all the ones we know and accept today.

Death by Cars, Cancer, Heart disease etc...

Post reply on HN