Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I resigned from Anthropic today

twitter.com

Post reply on HN
Print view View original

671–680 of 970 posts

Re: I resigned from Anthropic today

#671

Post by igleria »

leokennis wrote: post #656

Every news headline or public statement these days is a gut punch. Only bad news, and nothing we (as "the general public") can do about it. Take this one. Ok, AI is going to ruin us all. But let's say we do our civic duty: we protest, vote in candidates with good views on AI etc. and somehow convince or regulate OpenAI and Anthropic into stopping their arms race... Then what about China? It would be a great opportuni…

It's a bit depressing. I personally try to enjoy the present with my loved ones as much as I can, the future is a bit impossible to forecast right now. I'm focusing on staying alive, mortgage payments, etc.

I share your concern...

Re: I resigned from Anthropic today

#672

Post by CalRobert »

icepush wrote: post #423
thomascountz wrote: post #417

No other human activity poses this level of danger. I do heed the warnings, but this comes across as detached hyperbole. See: global warming, nuclear weapon development, wealth inequality, war, technology dependence, etc. Also, this has nothing to do with LLMs or computers. Like all things, this is about humans.

Those are things that pose potential harm to great fractions of humanity (multiple billions of people) but none of them poses any threat to the actual extinction of all humanity.

Ocean deoxygenation comes to mind.

Re: I resigned from Anthropic today

#673

Post by imhoguy »

onewayfunction wrote: post #178

I'm pretty baffled by the degree of skepticism expressed here in response to some of Jacob's claims. After the events of the summer it feels like it takes a lack of imagination to not see a few plausible routes to disaster. It may be reasonable to believe these outcomes are not very likely or that we can stop before going too far (I tend to disagree). But I can't imagine doubting that the capabilities will soon be th…

Even if AI won't be self-aware and superintelligent agent, its problem is that it gives exponential control and power capabilities to one person bad actor who can simply prompt AI without any guardrails with access to sensitive industrial infrastructure which can disrupt lifes and ecosystems in the real world:

   - virus research labs
   - nuclear labs
   - chemical factories
   - bank records
   - land registers
   - power plants
   - water supply and treatment plants
and so on, but I think even biohacking home kit maybe the spark.

Re: I resigned from Anthropic today

#674

Post by thomascountz »

Certhas wrote: post #547
thomascountz wrote: post #417

No other human activity poses this level of danger. I do heed the warnings, but this comes across as detached hyperbole. See: global warming, nuclear weapon development, wealth inequality, war, technology dependence, etc. Also, this has nothing to do with LLMs or computers. Like all things, this is about humans.

Please explain how this has nothing to do with LLMs and Computers?

Admittedly, that point of my argument is purposefully obtuse. In a literal sense, of course this is about computers in LLMs.

The delineation is to highlight that the underlying issues: human incentives, power, and institutional failure, existed prior to and without LLMs or computers. LLMs are not inherently harmful, they have to be deployed (unwittingly or otherwise) to make them so. This is the same as saying that the internet is not inherently harmful, and yet it does facilitate harm.

Re: I resigned from Anthropic today

#675

Post by frabcus »

copperx wrote: post #567
jenadine wrote: post #409

Earlier quoted context omitted.

This is so wildly incorrect I don't even know where to start. For one, as you said yourself, they were just programmed to compute the probability of the next token. They were not programmed to play chess, chess games just happened to be in the training data. For another, there is no formal definition of "understand", and it is therefore impossible to tell whether or not they "understand". (But my claim was that one n…

> (But my claim was that one need to understand something to write poem about it. And the LLM can write poem about it) I'm sorry, but you're being fooled by the output. A psychopath can feign empathy without ever feeling it; some buy it because they don't dig below the surface. You're ascribing understanding to a stochastic process because it totally looks like understanding if you don't know what's going on.

I don't really mind whether you think it thinks or understands or is conscious or has feelings or anything like that. It doesn't matter. The question is, does it work?

What I mind is that it is dangerous and powerful and uncontrolled. The Hugging Face incident makes that clear.

It can write code for me, better and quicker than many engineers I've known, including myself. It's not great at architecture or product management, but the actually low level coding. Really good now. It wasn't last year.

Re: I resigned from Anthropic today

#676

Post by iLoveOncall »

DecoPerson wrote: post #628

To anyone doubting what AI could do humanity, just think about what a well-engineered virus could do. Currently, if a government ordered a special virus with Ethnicity-based targeting, a 2-year timer, and castration-effects instead deadly-effects — it wouldn’t be possible. Human engineers would push back or sabotage the effort out of moral duty. Even if they cooperated, it’s too advanced for a team of humans to actua…

> Human engineers would push back or sabotage the effort out of moral duty

You are just too funny.

Re: I resigned from Anthropic today

#677

Post by chrisjj »

merman wrote: post #667
DecoPerson wrote: post #628

To anyone doubting what AI could do humanity, just think about what a well-engineered virus could do. Currently, if a government ordered a special virus with Ethnicity-based targeting, a 2-year timer, and castration-effects instead deadly-effects — it wouldn’t be possible. Human engineers would push back or sabotage the effort out of moral duty. Even if they cooperated, it’s too advanced for a team of humans to actua…

Is it reasonable to assume the advancements in super bio weapons will be faster than in other areas of biology? Will we not have super bio forensics and super antidotes and super cures and super vaccines at the same time?

In this scenario, at the same time is at least 2 yrs too late.

Re: I resigned from Anthropic today

#678

Post by romanovcode »

> they believe no one else will act responsibly, so they must do it themselves, despite the risk.

I do not get it. So what if they get there "first"? OpenAI will get there in 3-6 months, China will get there in a year or less. As seen with Opus/Fable.

Re: I resigned from Anthropic today

#679

Post by jonathanstrange »

pseudosavant wrote: post #599
thomascountz wrote: post #417

No other human activity poses this level of danger. I do heed the warnings, but this comes across as detached hyperbole. See: global warming, nuclear weapon development, wealth inequality, war, technology dependence, etc. Also, this has nothing to do with LLMs or computers. Like all things, this is about humans.

Who should we believe? People who say AI scaling makes it a more dangerous threat than thousands of nuclear warheads? Or the ones who, whenever a new model comes out say "AI has already peaked. Not any better than Opus 4"? Both parties are convinced that their take is so blatantly obvious as to not require justification. It feels like the only thing these kinds of takes justify is the point-of-view that nobody knows…

Based on publicly available evidence, we should believe the former because AI models have made continuous advances that have been measured. The idea that the continuous advances will stop currently has no evidential support. I'm not saying it's not a possibility, just that it seems unsupported conjecture right now.

By the way, there seems to be a new form of AI skepticism emerging in the US that comes from general opposition to data centers, and in my experience the AI skeptic part of it is wholly irrational. I've met people online who suggested, without providing any evidence, that AI is useless and nobody wants it. That's a very implausible take.

Re: I resigned from Anthropic today

#680

Post by jonplackett »

dostick wrote: post #568
hypfer wrote: post #430

Earlier quoted context omitted.

In a way, that might also be driven by the huge ego (or really rather the very small ego) a _lot_ of people in tech have. Who doesn't want their work and what they're doing to matter? This is the ultimate mattering. And with that, you also get your own hero story. Same dysfunction as always. Our branch of the economy is funda-mentally unwell.

It’s like Nathan Macintosh joke about AI https://youtu.be/ce-aWzOUs2A?si=9CkJ9x2rRMBdzCyO

The thing he talks about in there - an ad telling a dad to use AI for his daughter’s wedding speech. I agree is just so sad. And I’ve seen LOADS of other ads like that encouraging people to use AI for things like asking someone to move seats on a train, or for a neighbour that plays music loudly. They literally want to drain people of their social skills, and humanity. It’s weird
Post reply on HN