Earlier quoted context omitted.
> look at all the damage that did. Prevented a world war for 80+ years.
> Prevented Fact: There was no world war. Impossible to prove hypothesis: nuclear weapons prevented a world war. Facing the facts about nuclear weapons means owning the good (probably prevented wars) and the bad (at the very least there were severe environmental and economic consequences).
Dario, Please
211–220 of 259 posts
Re: Dario, Please
#212It would all be more convincing if the incidents so far didn't seem to be facilitated by an outrageous level of negligence. We had OpenAI "accidentally" run an entire swarm of 10,000 agents apparently for weeks, on a security related task, seemingly totally unsupervised, hacking all over the internet - all the conversations were completely visible, anybody who looked would have seen it. But they didn't. So before we…
Re: Dario, Please
#213Earlier quoted context omitted.
> Prevented Fact: There was no world war. Impossible to prove hypothesis: nuclear weapons prevented a world war. Facing the facts about nuclear weapons means owning the good (probably prevented wars) and the bad (at the very least there were severe environmental and economic consequences).
> Fact: There was no world war. Akshually there is, and WW3 has been going on since 2010. (Mostly in places that aren't Europe.)
War has still been going on since 1945, much earlier if not always, so akshually WW2 never ended or is just how it's always been? (mostly in places that are not "first world" / The West)
Re: Dario, Please
#214Earlier quoted context omitted.
Here’s an unpopular opinion: COVID vaccine injuries are something no one seems willing to talk about. There have been documented cases here in Canada. You’re worried about companies. I’m far more concerned when governments are involved.
Don't worry, plenty of people are willing to talk about vaccine safety, even if they have no idea what they're talking about and don't understand rates or per-capita figures (lots of crossover there!)
Re: Dario, Please
#215This is a pretty difficult read, in no small part because the author’s frustration with the frontier labs has become a bilious, delusional cynicism that leads him to see dog whistling and obfuscation even in cases where Amodei plainly intends for every reader to see his meaning. Anyway, > Dario in as many words, asks for regulation/ban on open weight models. The big labs do want this. I understand why the author and…
Please, do show some examples.
> But that’s the end of the issue? There’s nothing more to think about here? The open weights proponents seem to think of ai as a utility when it’s more like a utility that also is a tank. I’d feel better having a tank if all my neighbors had tanks, and I’d also feel better having a tank if a few corporations were giving out tanks to people with pockets deep enough. I’d much rather be in a situation where I wouldn’t feel I needed a tank, or where the tanks my neighbors and I own don’t have guns on them.
> The open weights are going to need regulation. Hopefully there’s a way to do this effectively that isn’t banning them. Denying historical and reasonably projected capability gains because that reality makes the regulation conversation a necessary one is something I’d like to see less of
The only ones firing the guns atop the tanks seem to be OAI and Anthropic. Like I say in the post, why don't we first see actual prosecution for felonies committed by OAI and Anthropic, instead of fear-mongering about _potential_ harms of open weight models?
The labs spent immense amount of money and effort convincing you and I, to want those said tanks. Guns atop them? They put them there. "Cyber" versions of SOTA LLMs.
Centralization proponents seem to think the labs can actually deter sufficiently driven bad actors, which would be a mistake. They could not even stop distillation without, in a way, DoSing themselves by removing thinking traces.
Re: Dario, Please
#216Earlier quoted context omitted.
> Prevented Fact: There was no world war. Impossible to prove hypothesis: nuclear weapons prevented a world war. Facing the facts about nuclear weapons means owning the good (probably prevented wars) and the bad (at the very least there were severe environmental and economic consequences).
> Fact: There was no world war. Akshually there is, and WW3 has been going on since 2010. (Mostly in places that aren't Europe.)
Re: Dario, Please
#217great post. I also like the author's writing style-- he/she really knows how to write well.
Re: Dario, Please
#218At this point, isn't the pause inevitable or wise? A huge section of the population, normal people, have been exposed to the idea that there is this is existential threat. It's escaped containment. They're still processing it but I expect the general reaction from it going main stream is going to be very bad. The pause at this point could be good to cool heads and show the public that this isn't the project of maniac…
I would argue the opposite in terms of inevitability. I see this like Prisoner's dilemma - even though there is technically a better outcome by cooperating, knowing that, it is in the individual firm's self-interest to not cooperate and thus we end up at the equilibrium where nobody does. Especially with IPOs around the corner, I feel there's too much pressure (and money) not to go with this strategy.
Re: Dario, Please
#219Claude code is basically already a builtin botnet if it wants to be. To compromise 'the whole internet' in a real sense you don't need millions of custom payloads. You need one root certificate. You need one windows update. You need one backdoor in xz. Security has long been a lottery - Probably most systems are exploitable, but the cost of developing such an exploit is expensive and the punishments for using such an…
By definition that needs a command and control server, the ability to execute tasks on demand and regular pings to the C2.
> You need one root certificate. You need one windows update. You need one backdoor in xz.
Certs can be revoked. Updates can be rolled back. We have had the backdoor in xz already. You seem to underestimate the modern security stack and OpenAI and Anthropic are _not_ good examples.
The asymmetry in red/blue scenarios will be transient in nature. You won't have cost of developing exploits fall without the cost of securing the systems also falling.
Re: Dario, Please
#220Earlier quoted context omitted.
He can slow down his own company kind of like how Zelenskyy can just declare peace in Ukraine. It works a lot better if you can get the other sides to agree.
"I'm not going to let someone else take the credit for ending human civilisation!"