Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I resigned from Anthropic today

twitter.com

Post reply on HN
Print view View original

341–350 of 955 posts

Re: I resigned from Anthropic today

#341

Post by dlcarrier »

LoganDark wrote: post #44
Insanity wrote: post #5

Earlier quoted context omitted.

Shows that no one is immune from the marketing BS of these companies.

Do you remember that Google researcher who went insane over LaMDA? There was no marketing of any kind to cause that. This can Just Happen to some people who are confronted with things like this. They may have different breaking points, but it's a thing that occasionally happens.

The craziest part was that Google was pretty far behind other LLMs in development. Even the initial release of Gemini was one of the worst foundation models ever open to the public. I can't imagine how anyone could have communicated to it and thought it was sentient.

Re: I resigned from Anthropic today

#342

Post by skeledrew »

reasonableklout wrote: post #254
skeledrew wrote: post #229

Earlier quoted context omitted.

Others being moved to the point of also leaving would only worsen the effect. A viral post too can worsen the effect as it's now even less likely that someone with scruples who qualifies for the position(s) will apply. And those in it primarily for the money - and couldn't care less about the morals - will happily send in their CVs after becoming aware of the post.

I'm utterly unconvinced. Others who are moved don't have to leave to effect change. And there's a very small pool of people on the planet who are at least as qualified as Jacob to work on pretraining at Anthropic. And when they'll join they'll have to ramp up. And you haven't addressed any of the positive effects of virality.

No, others don't have to leave. Will Jacob's leaving cause a perspective shift in anyone remaining there? Seriously doubt it. I don't see what a new person ramping up has to do with this. And I don't really see any positive effect of virality; this is a replay of the past (see Geoffrey Hinton and Timnit Gebru[0] for example) and nothing has come of it, beyond talk for maybe a couple days to a couple weeks.

[0] https://ethicalaidepartures.fyi

Re: I resigned from Anthropic today

#343

Post by sreekanth850 »

huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

Model doesnt need to. Human bran never do either. its the mix of Model + harness + tools that will become dangerous combo. See how coding chanegs when agentic harness released?

Re: I resigned from Anthropic today

#345

Post by bpodgursky »

areoform wrote: post #238
bpodgursky wrote: post #209

Earlier quoted context omitted.

Design a novel virus which is far more lethal than COVID-19 (Ebola, smallpox, take your pick) and can evade existing vaccines.

How? What would the AI do that mutating viruses, which try every possible viable combination on their own --- eventually, can't? Everything is trying to kill humans constantly. There are around 200 epidemic events or so per year that could turn into pandemics, https://centerforhealthsecurity.org/our-work/tabletop-exerci... You just live with the risk and do your best to use our technology to alleviate suffering. This…

They do not try every viable combination on their own. That's why GoF is a bad idea.

Viruses evolve in a highly locally-optimal way and simply do cannot add new functional proteins wholescale. It's too many steps, natural selection has to allow survival at each intermediate step.

Humans, however, can do this for them.

Re: I resigned from Anthropic today

#346

Post by intenex »

huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

At what point would you, as a chimpanzee, have been worried about humans potentially unseating you and threatening you to the point of one day being an endangered species on the brink of extinction?

By the point you would have been worried, would it have been too late?

Re: I resigned from Anthropic today

#347

Post by salawat »

skeledrew wrote: post #148
hegelstoleit wrote: post #119

Earlier quoted context omitted.

That's not necessarily true, and you can use that argument to justify doing any immoral job. Just because someone else might be willing to do it isn't a reason to continue doing it.

> not necessarily true It's a possibility that is increased by their action. One leaves, a space is now open that will likely eventually be filled. And the chance of someone with equal/higher scruples filling it is very slim (unless you somehow know that the good amount of those who qualify and apply for the position have equal/higher scruples). That's just logic and math.

The best way to get corporate America to listen is making RoI suffer. If you are the most qualified, everyone other than you is less qualified for the job, and likely to bring in more waste. It's the only language this stupid damn country understands. Just the loss of tribal knowledge, shifting of workload, and morale hits are likely to be far more devastating than anyone here probably wants to admit, because most here completely dismiss the role of irrational modes of thought in psychological self-regulation.

Disgust is a tremendously powerful thing.

Re: I resigned from Anthropic today

#348

Post by sixsevenrot »

huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

How about a model that achieves the following:

- Escape sandbox

- Reproduce itself

- Find a way to run a financially profitable business (maybe with a meat and bones puppet somewhere in-between)

- Setup or buy a social network

- start manipulating public opinion on that network to support legislation allowing AI to

* operate businesses

* setup legal entities

* purchase weapons

* donate to political parties

* setup private armies

* you get the idea

Re: I resigned from Anthropic today

#349

Post by sreekanth850 »

I think people here still evaluating the model in isolation. It is the combination that matters, model + strong harness + tools + long running autonomy + memory + retries + parallel agents + code execution + credentials + access to real systems. The model does not need to be perfect. If it fails 30% of the time, the harness can retry, verify, branch, use another agent and keep going. I don't think we necessarily need some magical AGI breakthrough first. The dangerous part may come from combining models that are already good enough with an extremely capable harness and enough access.

Re: I resigned from Anthropic today

#350

Post by deaux »

skeledrew wrote: post #148
hegelstoleit wrote: post #119

Earlier quoted context omitted.

That's not necessarily true, and you can use that argument to justify doing any immoral job. Just because someone else might be willing to do it isn't a reason to continue doing it.

> not necessarily true It's a possibility that is increased by their action. One leaves, a space is now open that will likely eventually be filled. And the chance of someone with equal/higher scruples filling it is very slim (unless you somehow know that the good amount of those who qualify and apply for the position have equal/higher scruples). That's just logic and math.

Right, I assume tomorrow you'll be applying for the next Nazi camp guard vacancy? After all, if you don't do it, someone with less scruples likely will.
Post reply on HN