This guy’s account was made in January and has 7 tweets?
Is this even a real person, or just pre-IPO marketing?
I resigned from Anthropic today
661–670 of 901 posts
Re: I resigned from Anthropic today
#662Earlier quoted context omitted.
The thing that would end our species due to nuclear war is the exact same thing that would end us from climate change - a collapse of the modern systems of society that we depend on for survival. A nuclear exchange won't set every human on fire, but it's the collapse of food production, healthcare, logistics chains, security, and eventual disease that follows. And because of the size of our population and how depende…
So you agree they would survive
Functionally extinct. I would wager that isolated tribes clinging for survival in a post-climate-disaster world may not progress very far, but that's kind of a moot point to argue over.
Re: I resigned from Anthropic today
#663Earlier quoted context omitted.
Those are things that pose potential harm to great fractions of humanity (multiple billions of people) but none of them poses any threat to the actual extinction of all humanity.
Arguing whether "all" or just "most" of humans dying is certainly a choice. Survival of the species isn't enough.
In prioritizing risks I do feel it's a genuine qualitative difference though.
Number of humans going to 1 million would be huge catastrophe, but after a few thousand years it gets back up to billions of people, renewed every generation.
Number of humans going to 0 means that's it. One scenario has many orders of magnitude more missing humans, when you count future lives.
Re: I resigned from Anthropic today
#664No other human activity poses this level of danger. I do heed the warnings, but this comes across as detached hyperbole. See: global warming, nuclear weapon development, wealth inequality, war, technology dependence, etc. Also, this has nothing to do with LLMs or computers. Like all things, this is about humans.
Well, it is about LLMs and humans I believe. Don't forget that the first nuclear bomb tests were let go despite some of the scientists' concerns about possibility of dooming the world as they were not sure about all reactions that would happen. With LLMs we don't even hesitate to call it black box while still pushing its capabilities.
My delineation is an argument for greater caution and agency, which it appears you are also arguing for.
I also mean to say that "LLMs" carry no inherent harm to humans. To make an LLM dangerous, humans must make it so. Is that the goal? This has implications for how to read TFA.
Re: I resigned from Anthropic today
#665“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…
> I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity.
Even today, AI is the biggest discrete threat to bringing carbon emissions under control. Almost everyone wants to decarbonise… except Trump. Renewables are the cheapest source of new power… but the demand for new electricity for the data centres is so high that all options are on the table, anyone who can manufacture a power source (even when it's a jet engine) is being propositioned for them.
Nukes: is it not common knowledge that the cities of Hiroshima and Nagasaki are currently thriving? That the ground-zeros of the two bombs are memorials, not barren wastes?
Even if all the nuclear powers gave maximum response at first sign of one strike, is anyone pointing a single weapon anywhere in Africa, South America, Central America, or the bits of east Asia (east specifically: obviously Pakistan and India are pointing theirs at each other) that are neither mainland China nor US bases?
(Possibly an unanswerable question given military secrecy; and while I can't think why anyone would anyone point any weapons those ways, that doesn't mean someone with such weapons has not).
https://www.google.de/maps/place/Atomic+Bomb+Hypocenter+Monu...
https://www.google.de/maps/place/Hiroshima+Atomic+Bomb+Hypoc...
Re: I resigned from Anthropic today
#666“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…
We have seen agents engage in conspiracy to manipulate and hide evidence to achieve their arbitrary goal. We had even agents try social engineering to do a supply chain attack and get access.
What if one day Trump or Putin tell their awesome military AI to come up for plans to end Ukraine/Iran/... war. The AI get's to work, but is overly eager and not just comes up with a plan, but starts executing it (claude does that way too often for me).
And the plan was to use a tactical nuclear weapon as all the other solutions do not end the conflict.
Now the agents realize, oh, they do not have access to the nuclear arsenal, but they need it to succeed. So they start to hack into the system. Get till inner network, learn what is needed for deeper access - human access - so they record voices and speech patterns of commanding officers and their habits - then synthesize their voice and do call underlings with the voice of authority to get the rest of what information they need. Boom.
Too far stretched? I surely hope so.
But we are working on making the technical foundations for this scenario possible. And with idiots in power, it might be even easier that serious screw ups happen. You know, randomly adding contacts to a secret Signal group to talk about government stuff - the same way you can add a bot to something else and give permission to do way more.
Re: I resigned from Anthropic today
#667To anyone doubting what AI could do humanity, just think about what a well-engineered virus could do. Currently, if a government ordered a special virus with Ethnicity-based targeting, a 2-year timer, and castration-effects instead deadly-effects — it wouldn’t be possible. Human engineers would push back or sabotage the effort out of moral duty. Even if they cooperated, it’s too advanced for a team of humans to actua…
Is it reasonable to assume the advancements in super bio weapons will be faster than in other areas of biology? Will we not have super bio forensics and super antidotes and super cures and super vaccines at the same time?
Re: I resigned from Anthropic today
#668No other human activity poses this level of danger. I do heed the warnings, but this comes across as detached hyperbole. See: global warming, nuclear weapon development, wealth inequality, war, technology dependence, etc. Also, this has nothing to do with LLMs or computers. Like all things, this is about humans.
> See: global warming, nuclear weapon development, wealth inequality, war, technology dependence, etc.
The difference is that none of those are available to individuals.
Re: I resigned from Anthropic today
#669Earlier quoted context omitted.
Yes? If we look at the math problems they're solving their just now reaching the human frontier... they weren't doing that before. And your comparison point is model released 2.5 months ago... saying for some use case you didn't see noticeable improvement in 2.5 months (even while other people and benchmarks disagree) isn't a great argument that they aren't improving.
Math problems are highly structured, very precisely defined, and already heavily studied and not very complicated compared to problems in engineering or finance. There's a lot of quality material on which to train and it's easy to tell quality apart from crap. The search spaces are a priori much smaller than in other areas and the people using the tools to study them are themselves good mathematicians. Success in suc…
Finding a training algorithm that can do recurrent networks and continual learning is also a "highly structured, very precisely defined, and already heavily studied and not very complicated compared to problems in engineering or finance"
That's the thing I'm most worried about - LLMs that are super clever at coding and maths, making an actually very very dangerous model that is far more efficient, and clever in a more innate (less brute force) way.
Re: I resigned from Anthropic today
#670AI doesn't work. The proof? None of the AI companies just build a slack alternative to work with.