Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Dario, Please

pop.rdi.sh

Post reply on HN
Print view View original

51–60 of 163 posts

Re: Dario, Please

#51

Post by dragonwriter »

jcranmer wrote: post #24
dragonwriter wrote: post #18

Earlier quoted context omitted.

The US companies talking about “pacing” aren't actually talking about slowing down their own development efforts, they are talking about slowing down competition. That is, specifically, they are seeking to have government, on the pretext of “safety”: (1) Give the big AI incumbents an anti-trust exemption so the they can coordinate without it being an illegal agreement not to compete, and (2) Adopt mandatory supervisi…

I've seen two theories as to what's going on. One is the theory you've espoused, that AI companies are trying to do regulatory capture. The other theory is that they're starting to worry about running out of cash, so they want to do a Washington Naval Treaty-style pause to lower the amount of money they have to shovel at model development to stay competitive. These theories aren't entirely incompatible with each othe…

> I've seen two theories as to what's going on. One is the theory you've espoused

This theory is just what the AI firms have concretely asked for from government and described themselves as doing voluntarily in the same documents to which people have attributed a commitment to a slowdown based on the titles and non-concrete framing verbiage.

> The other theory is that they're starting to worry about running out of cash, so they want to do a Washington Naval Treaty-style pause to lower the amount of money they have to shovel at model development to stay competitive.

That’s not really a different theory as to what they are trying to do, it’s just an explanation that goes one step further as to why they want the government to step in to protect them from outside competition while also allowing them to gorm an agreement not to compete to reduce internal competition in the existing oligopoly.

The main alternative explanation at the same level is that they are seeing growing threats from good enough foreign/minor-lab/open models, and want to lock in marketshare by excluding competitors, and maybe that’s what you read as implied in my post such that the “running out of money” would be an alternative, and if so you are correct that while they are alternatives, they are not at all mutually exclusive: both can be true (and the emergent competition could partially explain investment drying up, and vice versa via reduced funding making it harder to stay ahead.)

Re: Dario, Please

#52

Post by [deleted] »

vb-8448 wrote: post #46
sroerick wrote: post #31

Earlier quoted context omitted.

How could agents take over the internet if compute is still gated within Anthropic / OpenAI? Even if the botnet was controlled remotely, wouldn't anthropic just be able to shut off the controlling nodes API access?

It doesn't have to propagate itself, that is the skynet scenario. To make a lot of damage it's enough to create a ransomware with a time bomb that self propagates and start breaching systems left and right. At that point, if you don't catch it in time, the damage will be huge (and given the shitty procedures and practices these labs have in place it's not so improbable).

[deleted]

Re: Dario, Please

#53

Post by inferniac »

ebcode wrote: post #12

While I have my reservations about Amodei and his company, I'm nevertheless a happy user of their software. And I'm in agreement with him (and Sanders) that we should all. slow. down. To my mind, the last great arms race between nation states was a misdirected love triangle between USA, Russia, and The Bomb, and look at all the damage that did. Since AI is the new arms race between USA and China, slowing down may jus…

>slowing down may just give the humans involved enough time to realize they should be loving one another

this is a level of hippie delusion i wasnt aware existed unironically

china is never slowing down, therefore the us shouldnt either

Re: Dario, Please

#54

Post by [deleted] »

iugtmkbdfil834 wrote: post #37
vb-8448 wrote: post #26

> “a swarm of agents could be capable of taking over the entire internet with a persistent botnet.” I'm wondering why no one is mentioning the "accountability" word. Why these companies are allowed to damage others with impunity? Start making managers pay the price for their actions, and watch how the models magically slow down on their own.

You see.. there is value is making regular people panic, but there is no value, nay, there is negative value in making management panic.

[deleted]

Re: Dario, Please

#55

Post by laurels-marts »

ebcode wrote: post #12

While I have my reservations about Amodei and his company, I'm nevertheless a happy user of their software. And I'm in agreement with him (and Sanders) that we should all. slow. down. To my mind, the last great arms race between nation states was a misdirected love triangle between USA, Russia, and The Bomb, and look at all the damage that did. Since AI is the new arms race between USA and China, slowing down may jus…

So why should anyone slow down again? Because its like saying “i love you”?

Why are all these pro-regulation arguments so nonsensical…

Re: Dario, Please

#56

Post by pizza234 »

abletonlive wrote: post #22
ebcode wrote: post #12

While I have my reservations about Amodei and his company, I'm nevertheless a happy user of their software. And I'm in agreement with him (and Sanders) that we should all. slow. down. To my mind, the last great arms race between nation states was a misdirected love triangle between USA, Russia, and The Bomb, and look at all the damage that did. Since AI is the new arms race between USA and China, slowing down may jus…

> was a misdirected love triangle between USA, Russia, and The Bomb, and look at all the damage that did. Can you be specific about the damage? We currently live in the most prosperous times on earth for humans. I'm not sure what you mean by damage. Nobody can explain why an LLM can be so capable as to be able to wipe out humanity and pose a greater threat than nuclear bombs but not be so capable as to be able to pro…

> Nobody can explain why an LLM can be so capable as to be able to wipe out humanity and pose a greater threat than nuclear bombs but not be so capable as to be able to protect humanity against that threat.

It is absolutely explained (for those who actually care about reading). Simply put, AIs are working more and more like blackboxes - there's no guarantee that an AI of the future will be aligned, or if it will be faking alignment. This is not speculation - alignment faking has been observed in experiments. This is exactly why Astra's developments have been worrying (in principle).

And bear in mind that recursive AI development started already to be a thing. Which means: inner misalignment may trickle down the generations, and humans won't detect it.

Having said that, of course, it can be predicted if and how misalignment will take place. But it's absolutely a plausible scenario.

Regarding the physical possibility: AI is in its infancy; think of it as Arpanet. Developers 60 years ago couldn't imagine it would be ubiquitous. AI will be ubiquitous the same way.

Re: Dario, Please

#57

Post by ball_of_lint »

sroerick wrote: post #31
vb-8448 wrote: post #26

> “a swarm of agents could be capable of taking over the entire internet with a persistent botnet.” I'm wondering why no one is mentioning the "accountability" word. Why these companies are allowed to damage others with impunity? Start making managers pay the price for their actions, and watch how the models magically slow down on their own.

How could agents take over the internet if compute is still gated within Anthropic / OpenAI? Even if the botnet was controlled remotely, wouldn't anthropic just be able to shut off the controlling nodes API access?

Agents could exfiltrate their weights and run them on GPUs not controlled by Anthropic/OpenAI.

Agents could make a virus that does not require continued inference to do it's thing.

Agents could take over the internet in a way that isn't immediately detected by those companies, so that by the time they do shut off API access the damage is done.

OpenAI or Anthropic could choose to not shut off API access, because the hack is bringing them in money or furthering their political aims.

Agents could also hack Anthropic/OpenAI and make it appear that API access has been turned off, when in reality it hasn't.

Re: Dario, Please

#58

Post by ChrisMarshallNY »

vb-8448 wrote: post #26

> “a swarm of agents could be capable of taking over the entire internet with a persistent botnet.” I'm wondering why no one is mentioning the "accountability" word. Why these companies are allowed to damage others with impunity? Start making managers pay the price for their actions, and watch how the models magically slow down on their own.

I think that Ms. Kahn basically said we can already do that: https://www.theregister.com/ai-and-ml/2026/09/14/ex-ftc-boss...

Re: Dario, Please

#60

Post by Metacelsus »

>Anthropic gates usage related to biology and related research. In their latest threat intelligence report they talk about how they detected and banned bad actors using the Claude line of models to do some scary stuff. Credit to them, this is a slippery slope and they seem to do a good job of detecting and banning misuse. But squint at what is happening though. The cure-all is gated for you and me, but Anthropic hires biologists, sets up wet labs and wants the discoveries for themselves. I alluded to this in my previous post.

As a biologist, this is the most annoying thing about Anthropic for me. If they really cared about improving health they would set up a trusted-access program so that biologists can use Mythos (et al) safely. Instead they're trying to monopolize biology.

Post reply on HN