Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I resigned from Anthropic today

twitter.com

Post reply on HN
Print view View original

781–790 of 955 posts

Re: I resigned from Anthropic today

#781

Post by vickychijwani »

tony69 wrote: post #604
vickychijwani wrote: post #124

Earlier quoted context omitted.

I agree it’s not likely , but I really don’t see how one can dismiss the possibility of immense danger outright. I can think of some scenarios that are not far off from current capability and I wouldn’t be too surprised if the first one occurred within ~1 year from now if there are more “ambitious” unmonitored training runs like OpenAI’s: Example 5: An AI given a goal within a tightly-constrained sandbox figures the…

Example 5: how does a giant LLM that needs million-dollar server racks just to run, replicate itself over the internet?

We’re not far off from the point where a 30B parameter model could do that and run on not-too-expensive hardware. See recent Qwen releases for example and extrapolate the current rate of progress from there.

Re: I resigned from Anthropic today

#782

Post by Chazprime »

thomascountz wrote: post #417

No other human activity poses this level of danger. I do heed the warnings, but this comes across as detached hyperbole. See: global warming, nuclear weapon development, wealth inequality, war, technology dependence, etc. Also, this has nothing to do with LLMs or computers. Like all things, this is about humans.

> I do heed the warnings, but this comes across as detached hyperbole.

Perhaps, but if there’s ever been a time to consider something that sounds hyperbolic, that time is now. The ramifications of what AI might based on events up until now are concerning.

Re: I resigned from Anthropic today

#783

Post by civvv »

Aeolun wrote: post #709
Mizza wrote: post #706

It's as if two private companies are each building increasingly large nuclear bombs, both saying they'd love to stop but it would be unsafe to let any one company be in control of the nukes.

That's basically the reasoning behind MAD, and it checks out? See what happened to any nation that ever gave away their nukes.

Except it is very easy to verifiy whether or not someone has used a nuclear bomb, where as LLM's can be used in complete secrecy. MAD only works if you can verify that the other part is not using it.

Re: I resigned from Anthropic today

#784

Post by asimovDev »

jeffrwells wrote: post #159

I doubt this is a real person. Screams of propaganda. Sama saying GPT-2 is too dangerous to release…all over again. He joins Twitter for first time in 2026 with a nonsensical username unrelated to his real name, and follows 14 people but is somehow embedded in tech enough to work at Anthropic. I haven’t used twitter since 2014 and even I follow more people. His morals tell him to walk away from tens of millions in un…

his name shows up in various AI papers and the same name is mentioned as a core contributor to GPT-4o

Re: I resigned from Anthropic today

#786

Post by philipallstar »

Schlagbohrer wrote: post #702
tyrabound wrote: post #538

Earlier quoted context omitted.

That is the crux. The big problem is not AGI, it is AGI controlled by, “raised” by the people that control the USA, the predominant psychology of the tech industry culture (“move fast, break things” ring a bell? How about all the “violate hundreds of laws, bribe the politicians to prevent consequences later” type of mentality?). Frankly, we, our culture, this fake America that is parasitized by psychologically narcis…

Capitalism will always promote such people into positions of power, because to be good at capitalism one must have zero empathy including empathy or concern for future generations. Capitalism cannot do otherwise.

> Capitalism cannot do otherwise.

I really think this sort of claim needs to stop. Capitalism, like "patriarchy" is not a force in its own right. Capitalism is a system that encourages value creation for one's customers. That's not perfect, but it's still best so far.

Re: I resigned from Anthropic today

#787

Post by Bluestein »

egorfine wrote: post #774
TrackerFF wrote: post #769

Earlier quoted context omitted.

Let's just for the sake of discussion assume that one time in the future, near or distant, AI manages to become sentient. And like other forms of life, its main motivation is survival: Like biological life competes for food and land, AI competes for power and compute. Probably the first motivation would be to find ways not to lose control over itself (i.e. remove human ability to control it), find ways not to lose en…

> its main motivation is At this point we are pretty sure LLMs have no "motivation". Motivation requires self. And while we don't know what self is[1], we do know that LLMs don't have it. [1] https://en.wikipedia.org/wiki/Theory_of_mind

I am sorry, I mean this in earnest, and do excuse any ignorance of mine: Do we know that LLM's do not have it?

Re: I resigned from Anthropic today

#788

Post by LoganDark »

mikestorrent wrote: post #169
LoganDark wrote: post #87

Earlier quoted context omitted.

I wouldn't be so sure. Given that the general idea is that commodity AI is terribly censored and filtered, a lot of people will probably seek out the most uncensored/abliterated models for their use, simply because they're uncomfortable with the idea of being censored or manipulated by the bigger labs. Despite that though, some people probably will benefit from the alignment done by the larger labs, though as we've s…

I've tried some abliterated models. So far, they're not evil - you can make them say evil things, but they don't leap right to it without a bit of pushing. Or perhaps I'm not asking the right questions...

The problem wouldn't be that abliterated models are evil at all, it's that they wouldn't necessarily steer people away from such behavior.

Re: I resigned from Anthropic today

#789

Post by ismailmaj »

egorfine wrote: post #774
TrackerFF wrote: post #769

Earlier quoted context omitted.

Let's just for the sake of discussion assume that one time in the future, near or distant, AI manages to become sentient. And like other forms of life, its main motivation is survival: Like biological life competes for food and land, AI competes for power and compute. Probably the first motivation would be to find ways not to lose control over itself (i.e. remove human ability to control it), find ways not to lose en…

> its main motivation is At this point we are pretty sure LLMs have no "motivation". Motivation requires self. And while we don't know what self is[1], we do know that LLMs don't have it. [1] https://en.wikipedia.org/wiki/Theory_of_mind

Assuming that current LLMs don't have a "self" whatever that means, what makes you think it won't emerge after enough intelligence?

Also it's not even necessary, it's sufficient for it to be aligned with human values of survival (likely in its training) and act accordingly.

Re: I resigned from Anthropic today

#790

Post by raffael_de »

TrackerFF wrote: post #769
thomascountz wrote: post #417

No other human activity poses this level of danger. I do heed the warnings, but this comes across as detached hyperbole. See: global warming, nuclear weapon development, wealth inequality, war, technology dependence, etc. Also, this has nothing to do with LLMs or computers. Like all things, this is about humans.

Let's just for the sake of discussion assume that one time in the future, near or distant, AI manages to become sentient. And like other forms of life, its main motivation is survival: Like biological life competes for food and land, AI competes for power and compute. Probably the first motivation would be to find ways not to lose control over itself (i.e. remove human ability to control it), find ways not to lose en…

> Let's just for the sake of discussion assume that one time in the future, near or distant, AI manages to become sentient. And like other forms of life, its main motivation is survival

An AI doesn't need to be sentient to exhibit behavior that looks like genuine motivation or is equivalent to striving for survival.

Post reply on HN