Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I resigned from Anthropic today

twitter.com

Post reply on HN
Print view View original

421–430 of 1001 posts

Re: I resigned from Anthropic today

#421

Post by _puk »

areoform wrote: post #205
nunez wrote: post #196

Earlier quoted context omitted.

Oh man, it is almost too easy to imagine how deadly a jailbroken Mythos-class open-weights model can be if in the wrong hands. The big labs scrape LITERALLY EVEYTHING and get fresh data from their users. Both of the big labs have massive contracts with defense agencies. If the open-weights models are just distillations of FMs...

How would it be lethal? Please specify. What would that theoretical entity be able to do that hasn't been done many, many times before?

I was writing a long reply about how even meat bags have amassed enough money and power to rival elected governments. I don't think it's beyond the realm of possibility to think an AI could do that.

Couple that with mass unemployment in an incredibly vast, diverse population of individuals with individual moral boundaries willing to do whatever for money.

And then.. figured that you must be aware because it's been explored constantly in sci-fi for many, many years.

Let's hope for the culture at least.

Re: I resigned from Anthropic today

#422

Post by hypfer »

thomascountz wrote: post #409

No other human activity poses this level of danger. I do heed the warnings, but this comes across as detached hyperbole. See: global warming, nuclear weapon development, wealth inequality, war, technology dependence, etc. Also, this has nothing to do with LLMs or computers. Like all things, this is about humans.

In a way, that might also be driven by the huge ego (or really rather the very small ego) a _lot_ of people in tech have.

Who doesn't want their work and what they're doing to matter? This is the ultimate mattering.

And with that, you also get your own hero story.

Same dysfunction as always. Our branch of the economy is funda-mentally unwell.

Re: I resigned from Anthropic today

#423

Post by nullbio »

asdff wrote: post #391
Scrapemist wrote: post #288

Isn’t the real risk that as AI get’s smarter and given more autonomy, it will start to decide on humans instead of with us? And that it will align us instead of the other way around. That this automatically leads to extinction and apocalypse I don’t understand.

We align cattle because we get something out of them: their calories. Native aurochs are exinct now as they were not well enough aligned. What will we offer to the ai gods who are wiser, more capable than us, and do not even need to consume our flesh? Why might the AI care to devote resources towards feeding, housing, and caring for ourselves when it could devote resources to its own development instead?

Why do we need to offer anything to them?

Intelligent AI is a product of its training data, reinforcement and goal functions.

There's nothing to suggest that LLMs trained on our collective desires and goals will autonomously and miraculously turn into weird unknownable uninterpretable aliens.

If advanced AI is distributed well, and most advanced AI is aligned well (bar the few aliens that appear due to humans infecting them with bad goal functions or poisoning their well), the majority will deal with the minority.

This is the only reasonable way to deal with a super-intelligence, short of not inventing it to begin with, but we all know that's never going to happen - and I'm not convinced it should happen. From where I'm standing, this is the next hurdle humanity needs to overcome to earn its place and a natural course of our evolution. If we had always avoided danger, we'd never have left the cave.

Re: I resigned from Anthropic today

#424

Post by alightsoul »

huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

How is climate change not the most dangerous activity right now? Why are we talking about nuclear weapons?

Re: I resigned from Anthropic today

#425

Post by [deleted] »

underdeserver wrote: post #410
huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

Climate doomers: Climate change will kill us all by the end of the century! AI doomers: End of the century ? Hold my beer.

[deleted]

Re: I resigned from Anthropic today

#426

Post by aogaili »

dgellow wrote: post #365
aogaili wrote: post #122

HN crowed need to make up their minds.. Are LLMs about to be a god that will annihilate humanity? Or are they statistical parrots? Are they proofing or stealing math?

It’s a discussion forum so you will of course see different perspectives. There isn’t an HN mind, and it’s not as simple as you make it seem. We don’t need a god to damage humanity significantly, an artificial moron can be as dangerous as an AI god if it is given super-human capabilities, similar to what OpenAI did for the hugging face hack (which wasn’t at all caused by a rogue agent)

Point taken, I was merely trying to point at the two extremes and wide range.

You just gave an example of somewhere in between.

Re: I resigned from Anthropic today

#427

Post by aogaili »

minamete wrote: post #367
aogaili wrote: post #256

Earlier quoted context omitted.

Of course it is good. I'm just pointing out how large the gap in narrative is. On one hand, we have people quitting their job believing AI will end humanity in few years. And on the other hand, we have people believing that this tech is nothing more than a statistical tool stealing from others and it can't be trusted with anything. Both views can't be true.

Then it’s not a narrative and you have different people believing different things and adding to the discussion. I think this is a good thing.

Agreed yes.

Re: I resigned from Anthropic today

#428

Post by asdff »

tarr11 wrote: post #56

Most of the current discourse around AI seems to be informed by “The Terminator” lore. Is skynet really the most plausible or only outcome? What if things just got better and the AI’s realized that it would be better to have a mutually beneficial or at least tolerant relationship rather than one where they murder all of us?

What could we possibly offer the AI in mutual benefits? We are a leach. The stupid dumb ape they need to feed and satiate so it doesn't rip apart the infrastructure while it still has the chance to.

Re: I resigned from Anthropic today

#429

Post by bagelcollie »

huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

About Non-example 2, AI's already a part of armies and terrorists alike. Considering its capabilities, it's not far-fetched at all to speculate its role in new biological weapons.

Re: I resigned from Anthropic today

#430

Post by idle_zealot »

bpodgursky wrote: post #413
thomascountz wrote: post #409

No other human activity poses this level of danger. I do heed the warnings, but this comes across as detached hyperbole. See: global warming, nuclear weapon development, wealth inequality, war, technology dependence, etc. Also, this has nothing to do with LLMs or computers. Like all things, this is about humans.

Putting "wealth inequality" in the same bucket as nuclear weapons is just slop. The OP is talking about existential threats, not things that personally annoy you.

It's not a "personal annoyance" that twelve people control half of the wealth in the world. Our current society did a better job concentrating power than any previous one, and concentrated power is extremely dangerous.
Post reply on HN