Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I resigned from Anthropic today

twitter.com

Post reply on HN
Print view View original

241–250 of 1001 posts

Re: I resigned from Anthropic today

#241

Post by areoform »

threatofrain wrote: post #212
areoform wrote: post #205

Earlier quoted context omitted.

How would it be lethal? Please specify. What would that theoretical entity be able to do that hasn't been done many, many times before?

Before we discussed how important security was, we got insurance, we made libraries and products, we used compliance software, etc. Except how honest were we about all that stuff? How much risk was actually in the air, and what was keeping us accountable on security in either direction of over or under-investment? Now a reckoning is here. The potential to be attacked might actually translate to being attacked.

People have died due to ransomware attacks on hospitals. Powerplants have been attacked. Stuxnet and industrial control malware exists.

What will the AI do that hasn't been tried before?

Re: I resigned from Anthropic today

#243

Post by wuwue »

huitzitziltzin wrote: post #177
skew-aberration wrote: post #151

Earlier quoted context omitted.

> I’m not interested in wild theories about AI driven labor market disruptions leading to widespread starvation Changes in political and economic power balance leading to unrest, conflict, death and deprivation is not a wild theory. It is literally the story of our entire species. If you discount all such concerns, you are simply being willfully ignorant of past precedents. In fact, I challenge you to describe any no…

I’m an economist. On the basis of current evidence, I view AI as a complement to human labor, not as a substitute for it. That’s the source of my rejection of the wild labor market disruptions theories. I just don’t see any evidence yet that whole categories of jobs are being eliminated, with the single exception (so far!) of the end of “professional essay writing services for cheating college students,” and similar…

[dead]

Re: I resigned from Anthropic today

#244

Post by reducesuffering »

zdragnar wrote: post #180
reducesuffering wrote: post #149

Earlier quoted context omitted.

> I can only assume the reasonable people at openai and anthropic were all pushed out long ago Typical uninformed take on the side of "doomers are crazy". Both CEO's of OpenAI, Sam Altman and Dario Amodei, and many in their leadership, believe AGI has a very real probability of causing humanity's extinction. Both companies were founded upon this belief, it is at the core of the company. Only later were mercenaries hi…

I'm not saying they are crazy, I'm saying their predictions have a record of not being accurate, and thus give them no weight compared to others'. In any case, if Altman really does believe it is an existential threat, he must be a misanthrope as he now opposes heavy handed government regulation, unlike in 2015 when he was the only game in town. It's almost like he doesn't actually believe it and just wanted regulato…

Before OpenAI was ever even founded, way before any regulatory capture plausible claims:

"Development of superhuman machine intelligence (SMI) [1] is probably the greatest threat to the continued existence of humanity. There are other threats that I think are more certain to happen (for example, an engineered virus with a long incubation period and a high mortality rate) but are unlikely to destroy every human in the universe in the way that SMI could." -Sam Altman

Dario discussing AGI Existential Risk in 2014 before OpenAI and Anthropic: https://intelligence.org/2014/01/13/miri-strategy-conversati...

Both companies have deluded themselves into thinking the arms race is going to happen anyway and they need to rush to it first, as if somehow that helps.

Re: I resigned from Anthropic today

#245

Post by sssilver »

vickychijwani wrote: post #124
huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

I agree it’s not likely , but I really don’t see how one can dismiss the possibility of immense danger outright. I can think of some scenarios that are not far off from current capability and I wouldn’t be too surprised if the first one occurred within ~1 year from now if there are more “ambitious” unmonitored training runs like OpenAI’s: Example 5: An AI given a goal within a tightly-constrained sandbox figures the…

> inherent non-determinism in current AI architectures

There's nothing inherent about non-determinism in transformer architectures. All of it is removable.

Re: I resigned from Anthropic today

#246

Post by esafak »

slicktux wrote: post #201
copperx wrote: post #153

Earlier quoted context omitted.

To borrow on the 1990s Slashdot meme: 1. Invent transformer architecture. 2. Scale it up. 3. ??? 4. Machines become sentient and kill us all. OpenAI and Anthropic pinky promise that they have figured out #3 and they're not BSing just to get more funding, no. But because we live in a culture of fear, everyone eats it up no questions asked.

Why are we putting so much weight (no pun intended) on AI companies. At the end of the day the scaled up LLM transformers lack emotion and will… They do as they are told; or more correctly put. They do as they are programmed to do so.

They are told to solve problems by doing what it takes. You can justify anything with such a broad criterion.

https://en.wikipedia.org/wiki/Instrumental_convergence

Re: I resigned from Anthropic today

#247

Post by reducesuffering »

nozzlegear wrote: post #160
reducesuffering wrote: post #149

Earlier quoted context omitted.

> I can only assume the reasonable people at openai and anthropic were all pushed out long ago Typical uninformed take on the side of "doomers are crazy". Both CEO's of OpenAI, Sam Altman and Dario Amodei, and many in their leadership, believe AGI has a very real probability of causing humanity's extinction. Both companies were founded upon this belief, it is at the core of the company. Only later were mercenaries hi…

Why is the take uninformed? You didn't address anything about the part you quoted, wherein reasonable people were allegedly pushed out long ago.

How is it not addressed? The company never contained only "reasonable people" that believe AGI is not an existential risk to humanity. At both inceptions were people who believed in AGI x-risk, even the founders. Only after time, did there become more "reasonable people" who were mercenaries and only believed it to be a typical tech job. Today, there are more "reasonable people" than ever there. They haven't been pushed out. We're just witnessing some prescient mercenaries smart enough to Eureka the grave implications of what is actually happening.

Re: I resigned from Anthropic today

#248

Post by 0x20cowboy »

*How?* and *Why?*

The most intelligent people I know are the least likely to want to harm anyone or anything, and understand that diversity is fundamental and important to the universe. Without proof to the contrary, why would you think some super intelligence would want to hurt anyone? Because you would?

If you are saying that some small bit of training data made the thing completely evil, then that really couldn’t be super intelligence.

These doomer people keep running around saying these kinds of things, but they all just seem like people who play too much D&D and want to larp as the main character.

Happy to be shown something that isn't based on wild speculation and some randos “this is whats going to happen in 2030 because of my vibes” kind of information.

Re: I resigned from Anthropic today

#249

Post by reducesuffering »

moogly wrote: post #158
reducesuffering wrote: post #149

Earlier quoted context omitted.

> I can only assume the reasonable people at openai and anthropic were all pushed out long ago Typical uninformed take on the side of "doomers are crazy". Both CEO's of OpenAI, Sam Altman and Dario Amodei, and many in their leadership, believe AGI has a very real probability of causing humanity's extinction. Both companies were founded upon this belief, it is at the core of the company. Only later were mercenaries hi…

If they truly, truly believed that, would they be speeding towards building it? If yes, that would make them truly insane, right? Not as in a quaint "off their rocker" but more "non compos mentis".

Both companies have deluded themselves into thinking the arms race is going to happen anyway and they need to rush to it first, as if somehow that helps. They have publicly stated as such repeatedly. They think the ~10-50% chance of extinction sucks, but that it's going to happen anyway and they believe they can steer it towards something good the best and unlock all the potential positives like infinite life.

Re: I resigned from Anthropic today

#250

Post by vickychijwani »

sssilver wrote: post #245
vickychijwani wrote: post #124

Earlier quoted context omitted.

I agree it’s not likely , but I really don’t see how one can dismiss the possibility of immense danger outright. I can think of some scenarios that are not far off from current capability and I wouldn’t be too surprised if the first one occurred within ~1 year from now if there are more “ambitious” unmonitored training runs like OpenAI’s: Example 5: An AI given a goal within a tightly-constrained sandbox figures the…

> inherent non-determinism in current AI architectures There's nothing inherent about non-determinism in transformer architectures. All of it is removable.

I see. Can you say more about this? What’s the trade-off of removing it?
Post reply on HN