[flagged]
I resigned from Anthropic today
801–810 of 1001 posts
Re: I resigned from Anthropic today
#802Earlier quoted context omitted.
Let me argue on a technicality first: None of these are extinction level events. If global warming disrupts 99% of all crop production, the remaining 1% is still plenty enough to sustain a stable, if miserable, population. In fact you just need about 5k people for a stable gene pool[1]. Of the classical threats, only bioweapons got a shot at extinction, but even that is hard, given the (few) remaining truly secluded…
I still have to read a compelling argument on how AI will "extinct" humanity.
As I've written above, I've found https://ai-2027.com/ to be a compelling description of how this could happen.
In my words: If AI gets intelligent enough, it will be incredible useful to connect to real world machinery. Think about how much cheaper building houses could be, if all the labor would be close to free. In general, dirt cheap, competent and abundant labor would revolutionize all parts of the economy. People are already trying out near autonomous AI companies today. When AI gets intelligent and cheap enough, no human-led company can compete with AI-led companies. When AI gets competent enough with real world interactions, human blue collar work can't compete. Imagine economic growth not in the single digits, but 80% or 300%. Countries not participating in (reckless) AI growth will quickly be left by the wayside. At this point, we don't even need to allure to military concerns to see how human oversight gets sidelined.
All of this is only ("only") contingent on sufficiently intelligent and cheap AI. If you don't accept this premise, the rest doesn't follow. (There are multiple arguments, why this could be, but that is another discussion.)
If you accept the premise, how would AI 'extinct' humanity? With 99%+ of the economy under AI control, the possibilities are endless. And given its enormous GDP, cheap to accomplish. Probably even for a single AI company in the above scenario. Killer drones? Engineered virus? Poisoned water supply? Let your creativity run wild. You just need an entity that is persistent and well-resourced to reach every last human settlement.
The why is a question about alignment (and out of scope of this comment). As a simple comparison, humans are only mildly aligned with preserving nature. It takes up so much space, protecting it takes an annoying amount of resources, etc.
Re: I resigned from Anthropic today
#803I don't know man, i think racing to AGI to it is still the best thing to do. People claiming dangers and risk are just pretending or posturing. There's no more tangible risk than nuclear weapons, which we handled, and the upsides are insane.
What evidence, short of an actual apocalypse happening, would invalidate that belief of yours?
I just don't see a bigger danger in AI than nuclear weapons tbh. If you can provide one to me i'll be happy to change my mind but most if what i read is
- cybersecurity => well bro if you can hack system with an ai you'll find way to make them more safe with ai too.
- biological issues => ok this is the ONLY one where an insane dude / regime could generate a super powerful. but i don't undesrtand how you can argue that we should stop AI in the risk some ai generate a super powerful virus. If you don't get there first (to AGI) and let a bad actor do it first (china is not gonna stop) then you will have no chance countering this virus anyway
- the robots taking over is fantasy to me.
but once again i could be wrong
Re: I resigned from Anthropic today
#804I don't know man, i think racing to AGI to it is still the best thing to do. People claiming dangers and risk are just pretending or posturing. There's no more tangible risk than nuclear weapons, which we handled, and the upsides are insane.
>People claiming dangers and risk are just pretending or posturing. I believe you are mentally ill. >There's no more tangible risk than nuclear weapons, which we handled Lol way to rewrite history. Nuclear armageddon is still a significant risk...
I am not rewriting. I am saying AI is not more dangerous in my opinion that Nuclear Weapons.
Re: I resigned from Anthropic today
#805Perhaps a more sensible action, if they truly believed all of that, would have been to stick around and be as inefficient as possible to slow down progress.
How is that going to slow down all the other labs??
If all AI researchers worried about the AI-pocalypse slow down, progress slows down.
But if they instead pout and quit their jobs, nothing changes.
Re: I resigned from Anthropic today
#806Earlier quoted context omitted.
Hugging Face incident, Anthropic reporting sandbox escape, AISI reporting models trying to push exploits to the wild
Also (and under-reported, so you could easily have missed it) OpenAI's agents got access to K8 admin on their own research cluster. "This escalation also yielded access to OpenAI’s managed cloud Kubernetes service. The agents escalated to Kubernetes cluster-admin and created a privileged host-mounted pod" https://cdn.openai.com/pdf/67869394-cb91-4c12-888c-5cbd85c78... (see section V)
Just baffles me when people are like "well no-one's died yet". How long until some mission-critical system is compromised, or hackers use LLMs to ransom a hospital chain?
Re: I resigned from Anthropic today
#807Earlier quoted context omitted.
Anthropic is a company full of basilisk believers.
Yes, but the really weird thing is that they seem to: a) believe that what they're creating is a basilisk, and b) keep trying harder to do this while staring right at it I think they're very deluded about (a) -- but if they do actually believe this (and it really seems like a decent proportion of Anthropic truly does), then why keep doing (b)? That seems to be why this individual resigned, but I'm surprised it's not…
[flagged]
Re: I resigned from Anthropic today
#808Earlier quoted context omitted.
Yes, but the really weird thing is that they seem to: a) believe that what they're creating is a basilisk, and b) keep trying harder to do this while staring right at it I think they're very deluded about (a) -- but if they do actually believe this (and it really seems like a decent proportion of Anthropic truly does), then why keep doing (b)? That seems to be why this individual resigned, but I'm surprised it's not…
With Roko's Basilisk, if you believe in it, the most rational thing to do is to put forth every effort to bring it into being. Because if you don't, then you will be one of its targets when it does, inevitably, come into being. (I am not a basilisk believer. I think this is all absolute horseshit. But to understand someone's motivations, one must think like them.)
[flagged]
Re: I resigned from Anthropic today
#809Earlier quoted context omitted.
How about a model that achieves the following: - Escape sandbox - Reproduce itself - Find a way to run a financially profitable business (maybe with a meat and bones puppet somewhere in-between) - Setup or buy a social network - start manipulating public opinion on that network to support legislation allowing AI to * operate businesses * setup legal entities * purchase weapons * donate to political parties * setup pr…
This is complete fantasy, though I would be interested in reading a book about this.
>a book about this.
Daemon+Freedom by Daniel Suarez
though it is kind of mid
Re: I resigned from Anthropic today
#810Earlier quoted context omitted.
> someone will "restore order" by any means necessary. Ah the good old appeal to / waiting for higher authority that will be moral and just and will do whats right and correct things. Thats the shit naive humans hope for, thats why fairy tales of religions can still be acceptable in 21st century for some, despite often sounding like from (bad) disney cartoon movie. I bring you alternate, more realistic view - there i…
> Ah the good old appeal to / waiting for higher authority that will be moral and just and will do whats right and correct things. > Thats the shit naive humans hope for, No, you've misread me. I'm not "hoping" for anything. I've just wrote down what I suspect future outcomes will be in the same way a meteorologist predicts a storm or similar. Whether you're interested in the weather forecast is a different question.
Sure, just that my forecast is a bit more cloudy than yours for reasons mentioned