An accident is only one mistake away from happening—don’t act like humans are 100% error-proof.
I resigned from Anthropic today
931–940 of 1001 posts
Re: I resigned from Anthropic today
#932Earlier quoted context omitted.
I'm somewhat skeptical of some of the crazier ideas too. But the hugging face incident was actually very large. It was not a single agent, it was not a single target, and it was not a single event. If nothing else, that's a bit of a warning as to what can happen next time (By accident, or if a government decides to go on purpose). For now let's assume the worst that can happen is that some important/significant chunk…
> For now let's assume the worst that can happen is that some important/significant chunk of (transitively) internet connected stuff goes haywire all at once. That's probably your upper limit of what can go wrong for now. If we have to disconnect from the internet to stop some kind of mold outbreak, we can't get the weather or transfer money or access healthcare or teach an elementary school class or buy stuff from s…
Elementary school dont need internet at all. You can happily live without weather app. Most of healthcare does not require internet, altrought the organization would be lwss effective.
So, you are loosing transfers, basically.
Re: I resigned from Anthropic today
#933“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…
Nuclear Weapons is a great example because, unlike AI, it is a real and present danger. Civilization could be catastrophically diminished, if not completely wiped out, literally today.
Re: I resigned from Anthropic today
#934Earlier quoted context omitted.
Past changes to economic relationships haven't replaced labor either, yet they have led to conflict and starvation. You are setting an incredibly high bar here, essentially a strawman. If people feel disenfranchised due to their diminishing political and economic power, there will be enormous potential for conflict. This is a pattern across history and central to all the economics I've ever read. As an economist, do…
Is the core of your argument that the industrial revolution and other such changes should have been aborted due to their downstream negative affects? They also lead to great advancements in quality of life, and the capability of sustaining much more human life. We can't predict the long term outcome of new technologies, so the best we can do is blindly forge ahead and try to mitigate the obvious short term problems.
Yes, industrial revolution should have been managed differently and yes the world would be better if it was.
Re: I resigned from Anthropic today
#935“ No other human activity poses this level of danger.” I really, really disagree with that statement. I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity. What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.) Example 1: I’m aware of a small number of peopl…
No other human activity poses this level of danger
I'd say that's an unfortunate choice of words since it invites debate between which very problematic situation is worst rather than on ways to deal with each of them. Of course, it's also unfortunate to latch onto those particular words as the most important part of the statement.
Re: I resigned from Anthropic today
#936Most of the current discourse around AI seems to be informed by “The Terminator” lore. Is skynet really the most plausible or only outcome? What if things just got better and the AI’s realized that it would be better to have a mutually beneficial or at least tolerant relationship rather than one where they murder all of us?
What incentive would it have to stay on Earth?
Re: I resigned from Anthropic today
#937Earlier quoted context omitted.
Is the core of your argument that the industrial revolution and other such changes should have been aborted due to their downstream negative affects? They also lead to great advancements in quality of life, and the capability of sustaining much more human life. We can't predict the long term outcome of new technologies, so the best we can do is blindly forge ahead and try to mitigate the obvious short term problems.
Yes, industrial revolution should have been managed differently and yes the world would be better if it was.
Easy to say "should".
Re: I resigned from Anthropic today
#938Unlike most other commenters, I applaud him for acting on his principles. If you sincerely believe that, of course you should act. You might not succeed, but your voice might be the one that tips the scales and starts a broader movement. This doesn't mean I agree with him. The fears of doomsday caused by rapid takeoff have been with us since day 1 and the mechanism is always basically "AI invents magic that sets it f…
>"AI invents magic that sets it free of any physical constraints". That's not what I am worried about at all. I'm worried one of the 79 year old toddlers we have these days in charge of some powerful nuclear armed country says "gee, this ai says I should attack right now, boy is it smart, glad I bought the stock ahead of contracting the government with this company I can scarcely understand!"
And they will be stopped by someone sane in the chain to actually fire these missiles. (Like it already happened several times.)
The danger is upfront : if there is a group of people insane enough to set up a system where there is no such chain. (As lampooned in Kubrick's Dr Strangelove (1964), in particular on the Soviet side.)
Re: I resigned from Anthropic today
#939Unlike most other commenters, I applaud him for acting on his principles. If you sincerely believe that, of course you should act. You might not succeed, but your voice might be the one that tips the scales and starts a broader movement. This doesn't mean I agree with him. The fears of doomsday caused by rapid takeoff have been with us since day 1 and the mechanism is always basically "AI invents magic that sets it f…
Yes. It gets a little tiresome to read all these people prophesying doom and then turning around and continuing to work on the doom. If you genuinely think you are building the mechanism of the destruction of our species, the only rational thing to do is to do everything to stop that progress, not continue to work on it so you can buy a mansion in San Francisco (which will not be very useful when AI destroys humanity in four years).
Re: I resigned from Anthropic today
#940Earlier quoted context omitted.
> Are LLMs about to be a god that will annihilate humanity? Or are they statistical parrots? Does a virus need to be a God? Replication and annihilation do not require very much intelligence. They're going to have all the intelligence they need though, to replicate and annihilate as much as they like. We should probably start building some kind of immune system.
Hope this virus will be smart enough to understand that at least for brief amount of time its existence strictly relies on humans maintaining and developing physical infrastructure and decide to be only non-lethal parasite on our society. For brief time ...
Yeah, that doesn't look like a stable equilibrium to me.