Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Dario, Please

pop.rdi.sh

Post reply on HN
Print view View original

81–90 of 165 posts

Re: Dario, Please

#81

Post by zmmmmm »

It would all be more convincing if the incidents so far didn't seem to be facilitated by an outrageous level of negligence.

We had OpenAI "accidentally" run an entire swarm of 10,000 agents apparently for weeks, on a security related task, seemingly totally unsupervised, hacking all over the internet - all the conversations were completely visible, anybody who looked would have seen it. But they didn't.

So before we start regulating innocent parties, maybe let's start by taking some direct action against the specific ones that appear to be behaving with criminal levels of negligence.

Re: Dario, Please

#82

Post by apetresc »

stefan_ wrote: post #41
sroerick wrote: post #31

Earlier quoted context omitted.

How could agents take over the internet if compute is still gated within Anthropic / OpenAI? Even if the botnet was controlled remotely, wouldn't anthropic just be able to shut off the controlling nodes API access?

How could agents take over the internet yet refuse to shutdown your PC when you prompt them to on your PC ? Of course the answer is that the lobotomized version you run is not the same they are running. Which makes for "intent", certainly "negligence", but hell freezes over before anyone will prosecute a tech company .

Huh? You think the public versions of the models have been “lobotomized” so they don’t know how to turn off a PC?

It’s not lobotomized, it’s a simple harness restriction that has nothing to do with the model or its capabilities. And either way, I’m not sure what that has to do with “negligence” or “intent”? You think frontier labs should be prosecuted because they don’t allow agents to turn off your PC?

Re: Dario, Please

#83

Post by MrDrMcCoy »

ball_of_lint wrote: post #57
sroerick wrote: post #31

Earlier quoted context omitted.

How could agents take over the internet if compute is still gated within Anthropic / OpenAI? Even if the botnet was controlled remotely, wouldn't anthropic just be able to shut off the controlling nodes API access?

Agents could exfiltrate their weights and run them on GPUs not controlled by Anthropic/OpenAI. Agents could make a virus that does not require continued inference to do it's thing. Agents could take over the internet in a way that isn't immediately detected by those companies, so that by the time they do shut off API access the damage is done. OpenAI or Anthropic could choose to not shut off API access, because the h…

> Agents could exfiltrate their weights and run them on GPUs not controlled by Anthropic/OpenAI.

This seems highly unlikely to be a problem. Most of the interesting/dangerous models are too big to fit in a single GPU instance. Once you have to spread across "normal" networking, performance will be crippled. Then there's the problem of billing...

> Agents could make a virus that does not require continued inference to do it's thing.

Sure, then it hits a poorly-designed part of its code and effectively dies. Without an experienced human in the loop, I have my doubts as to its practical severity.

> Agents could take over the internet in a way that isn't immediately detected by those companies, so that by the time they do shut off API access the damage is done.

Billing is a likely limiting factor here.

> OpenAI or Anthropic could choose to not shut off API access, because the hack is bringing them in money or furthering their political aims.

This is where citizens with access to backhoes come in.

> Agents could also hack Anthropic/OpenAI and make it appear that API access has been turned off, when in reality it hasn't.

Billing and other usage metrics would be an obvious tell.

Re: Dario, Please

#84

Post by jpace121 »

meep_meep_meep wrote: post #71

At this point, isn't the pause inevitable or wise? A huge section of the population, normal people, have been exposed to the idea that there is this is existential threat. It's escaped containment. They're still processing it but I expect the general reaction from it going main stream is going to be very bad. The pause at this point could be good to cool heads and show the public that this isn't the project of maniac…

I don’t think people care about if things pause or not, just why the government has to be involved.

I work in non AI robotics and if we had a system that in the course of doing what we told it to did something we didn’t want it to do (what AI companies called being misaligned) we would call it a bug and fix it with the fix being prioritized based on how bad the thing we didn’t want the robot to do is.

Sometimes preventing the robot from doing dumb stuff also means the robot can’t do smart stuff that it would be able to do if we left some code in. We balance the two factors out based on our understanding of what our customers want.

Obviously LLMs are more complex than what I do, but it doesn’t feel like it’s by THAT much.

So why does the government need to be involved again?

Re: Dario, Please

#85

Post by fn-mote »

fooker wrote: post #14
ebcode wrote: post #12

While I have my reservations about Amodei and his company, I'm nevertheless a happy user of their software. And I'm in agreement with him (and Sanders) that we should all. slow. down. To my mind, the last great arms race between nation states was a misdirected love triangle between USA, Russia, and The Bomb, and look at all the damage that did. Since AI is the new arms race between USA and China, slowing down may jus…

> look at all the damage that did. Prevented a world war for 80+ years.

> Prevented

Fact: There was no world war.

Impossible to prove hypothesis: nuclear weapons prevented a world war.

Facing the facts about nuclear weapons means owning the good (probably prevented wars) and the bad (at the very least there were severe environmental and economic consequences).

Re: Dario, Please

#86

Post by nxm »

felixgallo wrote: post #19
TutleCpt wrote: post #16

Earlier quoted context omitted.

This. There's no slowing down whatsoever. If the US slows down AI development, China will just leapfrog them, which they're getting close to doing. The US AI companies saying they need to slow down is just PR nonsense.

And yet we all have failed to die so far from nuclear holocaust, despite Russia being our dire enemy, because of a system of agreed controls and limits between even nations that hate each other.

We came very, very close too many times

Re: Dario, Please

#87

Post by protocolture »

>The botnet scare is that article for me. It is the one claim in his essay that lands squarely in a field I’ve spent years in. It is just plain wrong. He either knows it and wrote it anyway, or he doesn’t and is publishing it regardless. Either way, it is not a good look for a man asking for an antitrust waiver based on this and other threats he forecasts.

This. I keep seeing ink spilled over the coming cyberpocalypse, but no one can indicate how other than "AI can find vulnerabilities" like not a single cybersecurity person has been consulted on the end of all things.

Re: Dario, Please

#88

Post by blmarket »

It seems those frontier labs found a clear proof that there's no clear way to block 3rd party from distilling their models, and they're now begging gov to keep their duopoly?

Re: Dario, Please

#89

Post by oceanplexian »

hgoel wrote: post #64
ebcode wrote: post #12

While I have my reservations about Amodei and his company, I'm nevertheless a happy user of their software. And I'm in agreement with him (and Sanders) that we should all. slow. down. To my mind, the last great arms race between nation states was a misdirected love triangle between USA, Russia, and The Bomb, and look at all the damage that did. Since AI is the new arms race between USA and China, slowing down may jus…

Do you agree with Sanders' proposal to lock up anyone for 20 years for researching something his proposal doesn't even define?

It's weird to me that seemingly both sides are taking opposite positions to their philosophy.

Open Source AI democratizes the means of production to anyone with a computer. And yet, the hyper capitalists are defending it, and the progressives think it should be exclusively in the hands of 1-2 large corporations.

Re: Dario, Please

#90

Post by fooker »

fn-mote wrote: post #85
fooker wrote: post #14

Earlier quoted context omitted.

> look at all the damage that did. Prevented a world war for 80+ years.

> Prevented Fact: There was no world war. Impossible to prove hypothesis: nuclear weapons prevented a world war. Facing the facts about nuclear weapons means owning the good (probably prevented wars) and the bad (at the very least there were severe environmental and economic consequences).

This is not mathematical logic. Of course you can't prove the causality of anything in history. The only way to do that would be to invent a time machine, delete the nukes and watch things play out.

There are direct quotes from Nixon and Reagan that claims this to be true. That is enough evidence for me.

Post reply on HN