Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I resigned from Anthropic today

twitter.com

Post reply on HN
Print view View original

241–250 of 968 posts

Re: I resigned from Anthropic today

#241

Post by skybrian »

snaking0776 wrote: post #190
Kim_Bruning wrote: post #106

Earlier quoted context omitted.

I'm somewhat skeptical of some of the crazier ideas too. But the hugging face incident was actually very large. It was not a single agent, it was not a single target, and it was not a single event. If nothing else, that's a bit of a warning as to what can happen next time (By accident, or if a government decides to go on purpose). For now let's assume the worst that can happen is that some important/significant chunk…

Generally I don’t think anyone is arguing about the for now part. I don’t think it’s crazy to extrapolate out a few years and ask what kind of danger we’ll be in then. A team of 10,000 agents just solved the Navier Stokes problem (sans bad behavior by the researchers). Even 1 year ago that would have been unimaginable. What happens to this risk view as: 1. Robotics begin rolling out more broadly across the world. 2.…

The mathematics research results are certainly impressive, but I don't see what that has to do with robotics.

Waymo is getting somewhere, but it's been a long slog. There doesn't seem to be much progress on, say, package delivery.

For more, see:

https://secondthoughts.ai/p/14-reasons-robotics-is-hard

Re: I resigned from Anthropic today

#242

Post by stevenhuang »

EA-3167 wrote: post #99
stratos123 wrote: post #34

The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible - but I hear the same people express fear privately. No other human activity poses this level of danger. A common response is “if they truly believe this, why are they still building it?”…

This is the "Pilot testimony of UFO sighting" levels of naive. What's more likely? Anthropic is doing some deeply unethical marketing in the lead up to their multi-trillion dollar IPO? Or they're inventing a machine god? There's ample evidence of the former because that's their entire business model, but no evidence whatsoever to support the latter claims. If you want an extreme claim to be taken seriously, provide c…

I wouldn't rule out pilot testimony of UFO sightings, nor the possibility we're indeed developing a machine God.

There's ample evidence to support both by now.

Re: I resigned from Anthropic today

#243

Post by areoform »

threatofrain wrote: post #213
areoform wrote: post #206

Earlier quoted context omitted.

How would it be lethal? Please specify. What would that theoretical entity be able to do that hasn't been done many, many times before?

Before we discussed how important security was, we got insurance, we made libraries and products, we used compliance software, etc. Except how honest were we about all that stuff? How much risk was actually in the air, and what was keeping us accountable on security in either direction of over or under-investment? Now a reckoning is here. The potential to be attacked might actually translate to being attacked.

People have died due to ransomware attacks on hospitals. Powerplants have been attacked. Stuxnet and industrial control malware exists.

What will the AI do that hasn't been tried before?

Re: I resigned from Anthropic today

#245

Post by wuwue »

huitzitziltzin wrote: post #177
skew-aberration wrote: post #151

Earlier quoted context omitted.

> I’m not interested in wild theories about AI driven labor market disruptions leading to widespread starvation Changes in political and economic power balance leading to unrest, conflict, death and deprivation is not a wild theory. It is literally the story of our entire species. If you discount all such concerns, you are simply being willfully ignorant of past precedents. In fact, I challenge you to describe any no…

I’m an economist. On the basis of current evidence, I view AI as a complement to human labor, not as a substitute for it. That’s the source of my rejection of the wild labor market disruptions theories. I just don’t see any evidence yet that whole categories of jobs are being eliminated, with the single exception (so far!) of the end of “professional essay writing services for cheating college students,” and similar…

[dead]

Re: I resigned from Anthropic today

#246

Post by reducesuffering »

zdragnar wrote: post #180
reducesuffering wrote: post #149

Earlier quoted context omitted.

> I can only assume the reasonable people at openai and anthropic were all pushed out long ago Typical uninformed take on the side of "doomers are crazy". Both CEO's of OpenAI, Sam Altman and Dario Amodei, and many in their leadership, believe AGI has a very real probability of causing humanity's extinction. Both companies were founded upon this belief, it is at the core of the company. Only later were mercenaries hi…

I'm not saying they are crazy, I'm saying their predictions have a record of not being accurate, and thus give them no weight compared to others'. In any case, if Altman really does believe it is an existential threat, he must be a misanthrope as he now opposes heavy handed government regulation, unlike in 2015 when he was the only game in town. It's almost like he doesn't actually believe it and just wanted regulato…

Before OpenAI was ever even founded, way before any regulatory capture plausible claims:

"Development of superhuman machine intelligence (SMI) [1] is probably the greatest threat to the continued existence of humanity. There are other threats that I think are more certain to happen (for example, an engineered virus with a long incubation period and a high mortality rate) but are unlikely to destroy every human in the universe in the way that SMI could." -Sam Altman

Dario discussing AGI Existential Risk in 2014 before OpenAI and Anthropic: https://intelligence.org/2014/01/13/miri-strategy-conversati...

Both companies have deluded themselves into thinking the arms race is going to happen anyway and they need to rush to it first, as if somehow that helps.

Re: I resigned from Anthropic today

#247

Post by sssilver »

vickychijwani wrote: post #124
huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

I agree it’s not likely , but I really don’t see how one can dismiss the possibility of immense danger outright. I can think of some scenarios that are not far off from current capability and I wouldn’t be too surprised if the first one occurred within ~1 year from now if there are more “ambitious” unmonitored training runs like OpenAI’s: Example 5: An AI given a goal within a tightly-constrained sandbox figures the…

> inherent non-determinism in current AI architectures

There's nothing inherent about non-determinism in transformer architectures. All of it is removable.

Re: I resigned from Anthropic today

#248

Post by esafak »

slicktux wrote: post #201
copperx wrote: post #153

Earlier quoted context omitted.

To borrow on the 1990s Slashdot meme: 1. Invent transformer architecture. 2. Scale it up. 3. ??? 4. Machines become sentient and kill us all. OpenAI and Anthropic pinky promise that they have figured out #3 and they're not BSing just to get more funding, no. But because we live in a culture of fear, everyone eats it up no questions asked.

Why are we putting so much weight (no pun intended) on AI companies. At the end of the day the scaled up LLM transformers lack emotion and will… They do as they are told; or more correctly put. They do as they are programmed to do so.

They are told to solve problems by doing what it takes. You can justify anything with such a broad criterion.

https://en.wikipedia.org/wiki/Instrumental_convergence

Re: I resigned from Anthropic today

#249

Post by reducesuffering »

nozzlegear wrote: post #160
reducesuffering wrote: post #149

Earlier quoted context omitted.

> I can only assume the reasonable people at openai and anthropic were all pushed out long ago Typical uninformed take on the side of "doomers are crazy". Both CEO's of OpenAI, Sam Altman and Dario Amodei, and many in their leadership, believe AGI has a very real probability of causing humanity's extinction. Both companies were founded upon this belief, it is at the core of the company. Only later were mercenaries hi…

Why is the take uninformed? You didn't address anything about the part you quoted, wherein reasonable people were allegedly pushed out long ago.

How is it not addressed? The company never contained only "reasonable people" that believe AGI is not an existential risk to humanity. At both inceptions were people who believed in AGI x-risk, even the founders. Only after time, did there become more "reasonable people" who were mercenaries and only believed it to be a typical tech job. Today, there are more "reasonable people" than ever there. They haven't been pushed out. We're just witnessing some prescient mercenaries smart enough to Eureka the grave implications of what is actually happening.

Re: I resigned from Anthropic today

#250

Post by 0x20cowboy »

*How?* and *Why?*

The most intelligent people I know are the least likely to want to harm anyone or anything, and understand that diversity is fundamental and important to the universe. Without proof to the contrary, why would you think some super intelligence would want to hurt anyone? Because you would?

If you are saying that some small bit of training data made the thing completely evil, then that really couldn’t be super intelligence.

These doomer people keep running around saying these kinds of things, but they all just seem like people who play too much D&D and want to larp as the main character.

Happy to be shown something that isn't based on wild speculation and some randos “this is whats going to happen in 2030 because of my vibes” kind of information.

Post reply on HN