Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I resigned from Anthropic today

twitter.com

Post reply on HN
Print view View original

591–600 of 955 posts

Re: I resigned from Anthropic today

#591

Post by mofeien »

bad_username wrote: post #496

AI is a computer program. It calculates numbers from other numbers. By itself it does not "want" to do anything and "cannot" do anything. Before it becomes an agent in the universe (in the classical meaning), it requires being supplied by an execution environment, energy, initiative (agentic loop, specific instructions), and modality (readonly and mutating connections to real world). It is like a game of chess - it d…

Humans are bioreactors. They only turn one organic matter into another. By itself they do not "want" and "cannot" do anything.

They do not exist just by themselves. Some bacteria in the gut must provide them with the energy to do so. So unless bacteria are actively involved, I cannot see how humans become truly autonomously agentic and start to do anything on their own.

Re: I resigned from Anthropic today

#592

Post by pingou »

jddj wrote: post #542
pingou wrote: post #497

Earlier quoted context omitted.

We are special in many ways, we have technology, we are everywhere, and we can think and plan (although considering global warming that is debatable). Global warming would have to create dramatic conditions everywhere on the planet, not leaving any small pocket of survivability to make humans extinct. Possible? Theoretically yes, but pretty unlikely. But obviously extreme global warming would be a catastrophe even if…

Runaway greenhouse effects on other planets have resulted in surface temps of 400C+

Is there a possible scenario for that on earth, that results in every place on earth having a temperature non survivable by humans?

Re: I resigned from Anthropic today

#593

Post by tsylba »

thomascountz wrote: post #417

No other human activity poses this level of danger. I do heed the warnings, but this comes across as detached hyperbole. See: global warming, nuclear weapon development, wealth inequality, war, technology dependence, etc. Also, this has nothing to do with LLMs or computers. Like all things, this is about humans.

Yeah, but every activity is a human activity, so you could ditch the adjective.

And then the thing with exponential growth is that the last thing is worth than the previous one for its potency for destruction. While it's true that, all things considered, the curve started to explode with the use of fossils fuel we've never limit ourselves with efficiency gain, aka Jevon's paradox.

Re: I resigned from Anthropic today

#594

Post by ben_w »

jddj wrote: post #542
pingou wrote: post #497

Earlier quoted context omitted.

We are special in many ways, we have technology, we are everywhere, and we can think and plan (although considering global warming that is debatable). Global warming would have to create dramatic conditions everywhere on the planet, not leaving any small pocket of survivability to make humans extinct. Possible? Theoretically yes, but pretty unlikely. But obviously extreme global warming would be a catastrophe even if…

Runaway greenhouse effects on other planets have resulted in surface temps of 400C+

Yes, but Venus has 93 times the atmospheric pressure of Earth, and it's overwhelmingly CO2.

Earth isn't likely to get a true runaway like that until the sun gets another billion or so years on the clock, and when it does it will be water vapour as the oceans are promoted to atmosphere.

What we're doing to ourselves is still bad, of course, but it's nowhere near that bad.

Re: I resigned from Anthropic today

#595

Post by madradavid »

"No other human activity poses this level of danger." I know this is a bit over the top but let us look at some of the most pressing issues in the world today: global warming, nuclear weapons, wealth inequality, war, technology dependence. Wouldn't a much more capable LLM in the wrong hands make these accelerate faster in the wrong direction ? Many of us , me included, have this mental model of a rogue Terminator-like AI but what I am most worried about is these LLMs in the hands of people. Look at what we have done to this Planet with the tools that we have so far, we have repeatedly tried to subjugate , enslave and kill each other because of things as peety as skin colour , tribe , religion and who owns which patch of land .

I don't have faith in the human race as is to do the right thing when handed these tools , you can already see the nonsense like the fake nudes , fake news and all that other filth being pushed on social media by people with access to relatively daft models. What happens when we can package a Mythos 5 level model in a box , when any war lord , supremacist or religous zealot can access these ? , You now you have your personal bio-chemist and nuclear physicist in a box.

The tool in itself is not what I am worried about , it is the human in the loop. I don't have an answer to this but i believe it is something we should all take a moment to think about , just think what a different world we would be living if any random person could buy a nuclear weapon off the shelf ? There are extremes to both ends , you can either worry way too much or don't care at all, I believe the best place is the middle ground were we actively think this through instead of using our usually "Move fast and break things" mode..

Re: I resigned from Anthropic today

#596

Post by Dylan16807 »

King-Aaron wrote: post #564
Dylan16807 wrote: post #526

Earlier quoted context omitted.

We have a massive survival range and we're on every continent. That kind of sudden climate change is not nearly enough to wipe us out. Some other climate scenarios might. Nuclear war definitely could.

The thing that would end our species due to nuclear war is the exact same thing that would end us from climate change - a collapse of the modern systems of society that we depend on for survival. A nuclear exchange won't set every human on fire, but it's the collapse of food production, healthcare, logistics chains, security, and eventual disease that follows. And because of the size of our population and how depende…

Outside of extreme feedback loop scenarios, there is no way for climate change to disrupt food anywhere near the level of nuclear winter, and there would be no mass destruction of supply chains either.

Re: I resigned from Anthropic today

#597

Post by mofeien »

bwfan123 wrote: post #255
peri-cl wrote: post #4

Here's a WSJ article about this resignation, https://www.wsj.com/tech/ai/anthropic-researcher-quits-over-... ( "Anthropic Researcher Quits Over ‘Out-of-Control’ AI Fears" )

More doomerism. Try to implement a deterministic workflow using agents with the latest models and no humans-in-the-loop, and you will realize what they are really capable of. There is too much unnecessary fear-mongering. All of this is only coming from the 2 AI labs trying to IPO. Not from anyone else.

> All of this is only coming from the 2 AI labs trying to IPO.

He has resigned from Anthropic. Is your argument that he quit Anthropic pre-IPO, sacrificing his payoff just to hype Anthropic?

Re: I resigned from Anthropic today

#598

Post by TeMPOraL »

huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

> I am looking for better reasons (supported by actual evidence!) to be more concerned than I am now: right now I am not concerned at all.

Intelligence is the root cause underlying those dangers.

Nuclear weapons, carbon emissions, biological weapons, and other such civilization-scale threats to humanity, are a product of our goal-seeking intelligence and ingenuity, and are only actively dangerous because of our ongoing use of our intelligence.

AI is about reifying that intelligence and ingenuity and making it run independently on computers, and act on the world.

That includes, in principle, ability to use nuclear and biological and other weapons, and also the ability to come up with some new threats too.

Re: I resigned from Anthropic today

#599

Post by pseudosavant »

thomascountz wrote: post #417

No other human activity poses this level of danger. I do heed the warnings, but this comes across as detached hyperbole. See: global warming, nuclear weapon development, wealth inequality, war, technology dependence, etc. Also, this has nothing to do with LLMs or computers. Like all things, this is about humans.

Who should we believe? People who say AI scaling makes it a more dangerous threat than thousands of nuclear warheads? Or the ones who, whenever a new model comes out say "AI has already peaked. Not any better than Opus 4"?

Both parties are convinced that their take is so blatantly obvious as to not require justification. It feels like the only thing these kinds of takes justify is the point-of-view that nobody knows how this is going to play out.

Re: I resigned from Anthropic today

#600

Post by mattdeboard »

onewayfunction wrote: post #178

I'm pretty baffled by the degree of skepticism expressed here in response to some of Jacob's claims. After the events of the summer it feels like it takes a lack of imagination to not see a few plausible routes to disaster. It may be reasonable to believe these outcomes are not very likely or that we can stop before going too far (I tend to disagree). But I can't imagine doubting that the capabilities will soon be th…

I wrote out a variety of replies but I just emphatically agree with your "lack of imagination" statement. I have been constantly surprised over the last 15 years at the general inability to correctly foresee how things can do wrong across a whole host of domains.

The replies here just adds AI to the list of domains.

Post reply on HN