I resigned from Anthropic today

They are holding the lines with drones alone nowadays. That's why it's called the deathzone, no humans survive. Some drones have FPV operators, some weapons use AI only already.

> You believe in AGI, but not ASI then?

I have observed that these initialisms do not have generally agreed upon meanings; they are not useful for discussions with people who have not already stated what precisely they count. There is disagreement about all three (/four) initials, and also how to combine them.

To give a sense of scale of how bad this is: some people on this very site have even denied that LLMs are "AI" at all, despite solving for natural language being a long-standing part of the field since, y'know, Turing.

On the other hand, the original ChatGPT was already (by my use of the words), "an AI", and while it was not superhuman in performance at any single thing, it already had a superhuman breadth of knowledge and a superhuman speed. For the former, even being properly fluent in five languages would be an absurd ask for a single human.

Spiky intelligence. How do the "G" (or "S") and the "I" part interact? Does something only count as "AGI" when for each task that has humans who do it regularly, it's as good as the average?* Does it only count as ASI when it's better than all humans at all tasks? Is it not "superhuman" to simply be as good as the median human while being 1% of the cost?

For some people, the cost matters; for others, the speed; for p(doom), competence.

* i.e. as good at speaking Korean as the average Korean, not merely as good at Korean as the average random human selected worldwide; as good at plumbing as the average plumber, not merely the average random human selected from all professions and unemployed alike.

This is a very interesting group of takes that feels quite different from my own beliefs. What would you call the belief system?

> Global warming: yes, kinda can wipe humanity, but I think much less likely

As someone who's seen the stats about heat deaths in the EU and also the drought in the UK, I feel like crop failures and other unforeseen consequences will fuck up both the economy and quality of life. People are dying and will die cause of human action, the only question is how many.

> Nukes: zero possibility of extinction

As long as we have people like Stanislav Petrov and cooler, educated minds prevail: https://en.wikipedia.org/wiki/Stanislav_Petrov

I don't think that's easy to guarantee in the modern day world, with the kinds of people in power and rhetoric that they enjoy. On one hand you have Russian saber rattling, on the other all it takes is a deranged enough leader and similarly bloodthirsty people down the chain of command.

> Wealth inequality: this one is driver of progress, opposite of extinction

Tell that to the people who are starving or living in shanty towns, or the even more people that struggle to make ends meet and experience anxiety regularly over living from salary to salary and sinking into debt. I'd say none of that is worthy of a dignified human life.

> War: another driver of progress, also will always naturally stop before every single human is dead

Tell that to all of the Ukrainians that are dead due to being invaded. I agree with the assessment that it leads to advancements (e.g. drone warfare) but I think it'd be harder to describe remotely positively if someone you know would have been blown apart by a drone/missile hitting their apartment block.

My take personally would be that all of those need to have attention paid to them (e.g. EU needing to spend more on defense), even if not immediately world ending. They do cause human misery, though, and should never be discounted.

I don't think that's accurate. What I've seen described is actually a surprisingly porous line: The Russians are often behind the Ukrainians and vice-versa, but it doesn't result in a break due to the transparency of the battlefield and the risk of getting killed if spotted, as well as the improvements in getting supplies to units behind 'enemy lines'. Drones shape this a lot but there's still a lot of people on the battlefield.

This is like unphysical gibberish. I’m not even kidding. There are such things as the Landauer Limit and the earth has a finite surface area to absorb solar radiation. For crying out loud these are smart people but at the end of the day the singularity is not actually singular.

It’s just silly talk. My dad works for Nintendo and he can give me a gold foil pikachu any time he wants, also, he can delete your Pokémon any time he wants because he has admin access on the Nintendo server. And also he can beat up your dad because he has a black belt. Oh your dad has a black belt? Well actually I lied, he has a rainbow belt which means he can beat up your dad still.

Like this is silly right? It’s insulting. We both know that there are live nuclear weapons whose coordinates point at my city and at your city and the only thing that stops them is two keys and a button. Nuclear weapons, you know those things that have killed hundreds of thousands of living men women and children? There is no room for speculation in that regard: my city has a navy base and there is a live nuclear weapon pointed at it, myself and my entire family will die if it goes off. Now contrast: ahem.

Oh sorry my dad also has like a super secret rainbow STEALTH powers that could actually totally insulate my house from nukes. If you think about it, I actually have no reason to be concerned.