Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I resigned from Anthropic today

twitter.com

Post reply on HN
Print view View original

911–920 of 1001 posts

Re: I resigned from Anthropic today

#911

Post by SaucyWrong »

mitthrowaway2 wrote: post #166
SaucyWrong wrote: post #132

Earlier quoted context omitted.

> while replicating wildly Earnest question: by what mechanism that exists today would the achieve that in a way humans on top top of the situation could not curtail? All of this runs on top of compute in meatspace that humans can disconnect.

Imagine you're the AI. Give yourself a solid minute to brainstorm ideas. Here's my answer, as a non-superintelligent human: "see to it that the humans on top of the situation have a compelling financial interest in the systems not disconnecting". In nuclear engineering, where safety is taken seriously, it's not enough to end the conversation at "the humans in charge can always simply shut down the reactor during a me…

My brainstorming led down a similar pathway of, "deluding powerful human actors that the systems must not be interfered with at any cost." I'm not sure I buy it though. Seems to me like if humanity were even approaching that brink, we could collectively decide to act, overthrowing the minority that--stemming from either greed, duty, or delusion--doesn't want us to act.

Re: I resigned from Anthropic today

#913

Post by Jblx2 »

xelxebar wrote: post #724
huitzitziltzin wrote: post #94

“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…

Your call for clarity has merit, but I think you have backed yourself into a corner, honestly. In 1900, there was no evidence of the kind you seek that lighter-than-air flight was possible. Good thing some people were foolish enough to ignore you then. Why you would cordon yourself off from the kind of reasoning that predicts legitimately new things, rather than just scaled up versions of the present? Not to put too…

>In 1900, there was no evidence of the kind you seek that lighter [sic]-than-air flight was possible.

Presumably you meant heavier than air? Also, I'm pretty sure that birds existed in the years leading up to 1900.

Re: I resigned from Anthropic today

#914

Post by TheCoelacanth »

ozozozd wrote: post #677
mitthrowaway2 wrote: post #593

Earlier quoted context omitted.

FWIW, before any nuclear weapons had ever been tested, a risk was identified that the first one might trigger a self-sustaining reaction in atmospheric nitrogen and destroy the entire planet. Faced with such a scenario, is the prudent next move: a) blow one up and see what happens, or b) do whatever you can to be sure it won't happen before conducting the first test, and make sure the confidence in the calculation is…

“Self-sustaining reaction,” like the recursive self improvement? Or perpetual machines? Is physics no longer a main subject at schools?

I'm pretty sure they didn't think the reaction would be perpetual; just long-lasting enough to wipe out life on Earth.

Re: I resigned from Anthropic today

#915

Post by andy99 »

riffraff wrote: post #298
copperx wrote: post #153

Earlier quoted context omitted.

To borrow on the 1990s Slashdot meme: 1. Invent transformer architecture. 2. Scale it up. 3. ??? 4. Machines become sentient and kill us all. OpenAI and Anthropic pinky promise that they have figured out #3 and they're not BSing just to get more funding, no. But because we live in a culture of fear, everyone eats it up no questions asked.

"collect underpants... Profit" comes from south park https://en.wikipedia.org/wiki/Gnomes_(South_Park)

Found it really funny to see this attributed to a Slashdot meme

Re: I resigned from Anthropic today

#916

Post by andai »

hypfer wrote: post #472
andai wrote: post #419

> A common response is “if they truly believe this, why are they still building it?” At OpenAI, many have not deeply internalized the civilizational stakes. At Anthropic, the stakes are well-understood, but they are locked in a race to get there first - they believe no one else will act responsibly, so they must do it themselves, despite the risk. Which means they have to go faster, which means less responsibly? I he…

I believe that the actual alignment happens in.. uh.. "meatspace". Someone is prompting. Someone is hosting. That someone needs to be accountable for what happens. That someone needs to bleed if stuff goes haywire. Humans at large have been "aligned" by the shared fear of death, pain and suffering. This has proven to work for millennia, so all we need to do is reapply it.

My point was you only need to prompt a worm once, and then you are no longer the one prompting it, it prompts itself by definition from that point onward.

Also I think there was some confusion about the word "spreading" in the sibling thread.

If the HF hack was done by humans we would have used the word spreading in that context, even though they were not replicating their minds. So spreading does not require replication. But we're either already at the point, or very close, where an open weight model could do a similar attack. So that actual replication could be involved.

(I'm not sure what that would add, but I don't see what obstacles remain at this point.)

Re: I resigned from Anthropic today

#917

Post by hnburnsy »

Findecanor wrote: post #698
icepush wrote: post #494

Earlier quoted context omitted.

Global warming is happening at a slow enough pace that there is plenty of time for people to move to the poles/into caves/beneath the ocean/whatever. Nuclear war would be much more destructive in a much shorter range of time but you will have pockets.

People already entrenched near the poles will fight to keep others out. Right-wing anti-immigrant parties are on the rise right now already. They also tend to not want to recognise climate change -- because that would imply that people could migrate because of it.

> People already entrenched near the poles will fight to keep others out. Right-wing anti-immigrant parties are on the rise right now already. >

Like those autonomous zones, remind me again which parties set those up?

Re: I resigned from Anthropic today

#918

Post by cloverich »

huitzitziltzin wrote: post #177
skew-aberration wrote: post #151

Earlier quoted context omitted.

> I’m not interested in wild theories about AI driven labor market disruptions leading to widespread starvation Changes in political and economic power balance leading to unrest, conflict, death and deprivation is not a wild theory. It is literally the story of our entire species. If you discount all such concerns, you are simply being willfully ignorant of past precedents. In fact, I challenge you to describe any no…

I’m an economist. On the basis of current evidence, I view AI as a complement to human labor, not as a substitute for it. That’s the source of my rejection of the wild labor market disruptions theories. I just don’t see any evidence yet that whole categories of jobs are being eliminated, with the single exception (so far!) of the end of “professional essay writing services for cheating college students,” and similar…

Narration is an obvious one. Theres also a huge risk to less qualified folks and jr employees. ie in most software shops, youll get more mileage giving your best engineers bigger opus budgets and a raise than jr engineers to train. Which means we meed maybe half or less as many, and at least half of that savings is floated up to sr folks and ai companies. That seems rather disruptive?

Re: I resigned from Anthropic today

#919

Post by dekhn »

asdff wrote: post #396
dekhn wrote: post #31

There's a scene in the movie "War of the Worlds" by Spielberg where the protagonist's son walks into a war zone because he is entranced by the battle ( https://www.youtube.com/watch?v=X7rfWPbEufo ). He is obliterated (along with the rest of the US forces) shortly after. I've always been struck by that scene, because in a lot of ways, if we really are headed towards a superintelligence, I at least want to be there and…

You can't address the key problems without understanding of the esoteric side details to be fair. You are studying what is basically the worlds most complicated and undocumented computer.

Actually, I work in biology, which is a far more complicated and undocumented computer than any existing AI model. And we've already seen the failure models (COVID is an example).

Re: I resigned from Anthropic today

#920

Post by ethin »

senordevnyc wrote: post #876
copperx wrote: post #153

Earlier quoted context omitted.

To borrow on the 1990s Slashdot meme: 1. Invent transformer architecture. 2. Scale it up. 3. ??? 4. Machines become sentient and kill us all. OpenAI and Anthropic pinky promise that they have figured out #3 and they're not BSing just to get more funding, no. But because we live in a culture of fear, everyone eats it up no questions asked.

They literally have never promised anything like that. They have explicitly said repeatedly that they want regulation, oversight, nationalization, a global slowdown, etc. And then I have to read page after page of cynical comments about how that's all just hype and marketing and them wanting regulation to block their competition and blah blah blah. You people would only be happy if they stopped all AI development, bu…

> ... They have explicitly said repeatedly that they want regulation, oversight, nationalization, a global slowdown, etc.

And yet they do the exact opposite of all of the above. They do everything to get regulation just to create a moat because one doesn't exist. They preach about wanting a slowdown, nationalization, a complete pause, whatever have you... And yet they aren't slowing down the development voluntarily now are they? If anything, they are doing everything imaginable to speed up development to pump out models as fast as possible. If these ex-risk and AI companies actually gave a damn about regulation, or oversight, or a moratorium on AI development altogether, they would actually demonstrate this by completely ceasing development of all models immediately. Instead, they take riskier and riskier actions to try to "win" this supposed "arms race". So please excuse a bunch of us if we find it impossible to take them seriously on literally anything like this. If you have substantial evidence that they are actually ceasing AI model development like they want everybody else to do (except them, of course), then, by all means, present it.

Post reply on HN