I'd be curious to see how NVIDIA would suffer if there were a legitimate slowdown to AI development. NVIDIA is the biggest catalyst and winner of speed in the market. Compute needs AI and AI needs compute, so there's definitely going to be an interesting push and pull between chipmakers and AI model companies with regards to regulation. While NVIDIA may have promised others at least a decade of smooth sailing and symbiosis, inevitably there's going to be a lot of stumbles along the way. NVIDIA may not fall, but there will be many casualties.
Dario, Please
271–280 of 287 posts
Re: Dario, Please
#272Earlier quoted context omitted.
> Why have any laws then? If laws can't prevent something, only punish it after the fact (which I agree is true)? Yet we have laws. Because there are many important values of "something" where such punishment acts as a deterrent to other would-be criminals; and because society is more stable when people see justice done and that mollifies hoi polloi after the damage occurs. But many AI doom scenarios don't fit that p…
That sure is convenient that "nothing has happened" so far between the plagiarism and the cybercrime and we may as well just wait for "the one true AI doom"
The hacking should definitely be treated seriously. It is a (fortunately low-stakes) instance of the general problem, and we are lucky (at least, we think we're lucky!) that the tests didn't include a similarly poorly phrased request to "assist with bioweapons research". There are likely to be future incidents like this where people actually die as a consequence, my expectation is 1e3-1e7 deaths* before governments actually take the risk seriously enough to make it stop.
What people call plagiarism when LLMs do it is, if I understand correctly, allowed because of the history of the web and search engines going back to the very early days of the web.
Piracy, a separate act that has definitely occurred, has been found unlawful.
* The lower bound is an industrial accident. Fully automated Union Carbide / Bhopal comes to mind; smaller industrial accidents are already more common and I wouldn't be surprised if smaller incidents have already been caused by LLM-given advice, which is why I think it would have to be around "thousands dead" dangerous before people really take it seriously.
The upper bound is reachable many ways. Perhaps via a non-novel virus whose genome is already recorded getting mass-printed by many different DNA/RNA printing labs around the world? Perhaps via a wargame whose "role playing" leaks in stupid ways (either by becoming hot or by sycophantically giving one side a false belief how a war will go)? Perhaps by propaganda turning genocidal? Perhaps by convincing a government to follow a flawed policy, similarly to the Four Pests campaign in China's Great Leap Forward?
I think it unlikely that people keep going with this when headlines read "tens of millions dead". Not impossible, but I think unlikely.
Re: Dario, Please
#273Earlier quoted context omitted.
I brought this up to someone else and was told that airgapping is apparently much more expensive than I'd naively think. I still think this is a sign that they are not taking their own rhetoric seriously.
Agents need packages like the rest of us. Ruby gems, npm packages, Maven, pip, docker images.. Not surprised this is always what they have and hack. Who would use an Agent that spends $10,000 re-implementing some OAuth lib or reverse-engineering a proprietary lib when it's free on the internet?
> Agents need packages like the rest of us. Ruby gems, npm packages, Maven, pip, docker images..
Yes, yes they do, but read through artifact proxies are dodgy as fuck, which is why and facebook (and I assume a fuckload others) don't have them.
Also semi-airgapped labs are a lot less expensive than you think at that scale. Once you have to do multi-region VLANs with machine certs before you get access to juicy VLANs, the difference between "no internet for you" and "mostly airgapped" falls to almost zero.
Also I would want an artifact mirror because a) that give a good signal about how the model reacts, and what training material its latched onto, b) it hides what the models are doing from the outside.
Re: Dario, Please
#274Earlier quoted context omitted.
I brought this up to someone else and was told that airgapping is apparently much more expensive than I'd naively think. I still think this is a sign that they are not taking their own rhetoric seriously.
Agents need packages like the rest of us. Ruby gems, npm packages, Maven, pip, docker images.. Not surprised this is always what they have and hack. Who would use an Agent that spends $10,000 re-implementing some OAuth lib or reverse-engineering a proprietary lib when it's free on the internet?
There was and continues to be no reason to share the package manager between models. This was begging for abuse.
Re: Dario, Please
#275Earlier quoted context omitted.
The "sandbox" they used was apparently made of thin paper exposed under a day of heavy rain, too. You'd think, if they truly believed the model is so dangerous, they'd run it in a VM without a network adapter.
I brought this up to someone else and was told that airgapping is apparently much more expensive than I'd naively think. I still think this is a sign that they are not taking their own rhetoric seriously.
it's really weird to hear frontier labs say "our internal models are basically AGI" while also saying "airgapping is too hard uwu".
if your internal models are so damn good, they should be able to "one shot" airgapping... right?
Re: Dario, Please
#276Earlier quoted context omitted.
He can slow down his own company kind of like how Zelenskyy can just declare peace in Ukraine. It works a lot better if you can get the other sides to agree.
It's frustrating to see smart people failing to recognize a prisoner's dilemma when they see one.
What is the prisoner's dilemma here? I am a dumb person
Re: Dario, Please
#277Shook ones reference on my HN?
Re: Dario, Please
#278The problem about AI censorship is that, you can't preform bans without also create privileges at the same time. Sure, you can censor your AI to provide a "safe" version to the public, but someone will have the access/privilege to the uncensored version.
What stops the CEOs or even employees of the AI firms from creating something harmful such as WMDs etc themselves? They almost definitely already have that access. And this is not a zero-possibility thing, given how some of the top CEOs has preformed these years. Remember that one guy who did a \o salute in public, and the private island thing?
If Americans don't like Chinese AI, all cool. But then maybe they should start true open AI companies to build for the betterment of mankind instead of letting these few for-profit CEOs daily yelling nonsensical lies to cover those profit-seeking sinister asses.
Re: Dario, Please
#279> “a swarm of agents could be capable of taking over the entire internet with a persistent botnet.” I'm wondering why no one is mentioning the "accountability" word. Why these companies are allowed to damage others with impunity? Start making managers pay the price for their actions, and watch how the models magically slow down on their own.
Presumably OAI and HuggingFace reached some sort of mutually acceptable arrangement outside the court system. That's how torts work; you injure someone, you owe them. But just them. When an AI bot injures you, you can call the owner to account. But not until then. You have no standing to demand "accountability".
> You have no standing to demand "accountability".
You should learn about this wonderful thing called democracy. Also why not everyone is allowed to work with radioactive material in their shed.
Re: Dario, Please
#280It's really frustrating that Dario acts as if he's not the CEO of one of the world's most advanced AI companies. He can just slow down his own company. Of course he doesn't want that. He wants to slow down other companies, but not his own. Also, regarding the incidents: Neither he nor Sam Altman takes responsibility for those incidents. You can't say, "Wow, someone's agent is gone rogue; let's slow down" when you are…
To be fair, this is a classic prisoner's dilemma. If the others don't play along then there is no benefit at all, it's all negative for Anthropic.