Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Dario, Please

pop.rdi.sh

Post reply on HN
Print view View original

261–270 of 283 posts

Re: Dario, Please

#261

Post by amouat »

exfalso wrote: post #256
amouat wrote: post #243

Earlier quoted context omitted.

Isn't that exactly what they did? The bots could only access the jfrog instance, so they hacked jfrog?

No that's not what they did, they exposed jfrog raw. It would have been so extremely simple to gate services they need the llm to access... I mean, jfrog was not written with this kind of threat model in mind, and neither were a lot of other tools

Right, you mean it didn't go through a gateway? But would that actually have helped? The requests all went through jfrog didn't they? I guess it depends on the level of filtering at the gateway?

Whilst it might not be JFrog's threat model, I wouldn't assume it can be used as a full internet proxy.

I don't really mean to defend OpenAI here, but they did make some attempts at sandboxing. Although it does seem that they didn't really know what they were doing.

Re: Dario, Please

#262

Post by mtrovo »

> Impressive? Sure. But you trained “Cyber” versions of LLMs and hyped them. How much more impressive is this, compared to developing GTA-clones one shot? Not much more.

Such a well written piece.

Have you seen the US military budget? When you run a war empire you frame problems as a war on something. That's the way to get around the valuation bubble they've created in time for the IPO. It's that or throwing in a 100T TAM on their S-1 and making SpaceX look like an honest valuation compared to them.

Re: Dario, Please

#263

Post by nicce »

0x5FC3 wrote: post #219
ball_of_lint wrote: post #49

Claude code is basically already a builtin botnet if it wants to be. To compromise 'the whole internet' in a real sense you don't need millions of custom payloads. You need one root certificate. You need one windows update. You need one backdoor in xz. Security has long been a lottery - Probably most systems are exploitable, but the cost of developing such an exploit is expensive and the punishments for using such an…

Dario said a "persistent botnet" and even links to the wiki page for botnet. By definition that needs a command and control server, the ability to execute tasks on demand and regular pings to the C2. > You need one root certificate. You need one windows update. You need one backdoor in xz. Certs can be revoked. Updates can be rolled back. We have had the backdoor in xz already. You seem to underestimate the modern se…

> Dario said a "persistent botnet" and even links to the wiki page for botnet.

> By definition that needs a command and control server, the ability to execute tasks on demand and regular pings to the C2.

I am not sure if you are agreeing or disagreeing, but I am saying just in case, that Claude can be remotely controlled from the cloud on any machine it is installed and set running, and can be commanded remotely.

Re: Dario, Please

#264

Post by ben_w »

vb-8448 wrote: post #26

> “a swarm of agents could be capable of taking over the entire internet with a persistent botnet.” I'm wondering why no one is mentioning the "accountability" word. Why these companies are allowed to damage others with impunity? Start making managers pay the price for their actions, and watch how the models magically slow down on their own.

> Start making managers pay the price for their actions, and watch how the models magically slow down on their own.

The problem here can be personified as "Trump", and it's the same problem that applies to coal.

Coal has a price besides money. It has historically been dangerous work, killing miners. It produces dangerous waste, both during mining and when burned, both as solid residue and the gasses emitted. The problems have been known for a long time. The workers themselves have called for better safety requirements and gone on strike for such things.

Why these companies are allowed to damage others with impunity?

Coal continues to be burned, because power is power. It's so important that sometimes the government steps in against the unions, rather than being on their side. Despite calls for this, we've not been able to get the owners of the coal mines, nor the coal burners, to "pay the price for their actions".

AI? Famously, knowledge is power.

Trump wants that power. He's not the only one, but he is the avatar of those who put their feet on the scales to not only allow but in some cases require (DoD vs. Anthropic) these companies to damage others with impunity.

Re: Dario, Please

#266

Post by meherabhossain »

The scenario is, if they are saying that the AI Agents are capable of handling everything, this puts the complete accountability on the AI. But this has a nuance to itself. This is like a contradiction.

Re: Dario, Please

#267

Post by ozim »

vb-8448 wrote: post #26

> “a swarm of agents could be capable of taking over the entire internet with a persistent botnet.” I'm wondering why no one is mentioning the "accountability" word. Why these companies are allowed to damage others with impunity? Start making managers pay the price for their actions, and watch how the models magically slow down on their own.

Because software is already absolved from accountability. Bill Gates introduced it in 80's with EULA where MSFT would not be liable even if your house burns down because of use of their software, even with flaw they would know.

That is the status quo we entered AI age with.

Re: Dario, Please

#268

Post by teekert »

It all comes down to " Privatize profits, socialize losses." It's the American way.

Here it is: "We warned you you should have regulated us! Now this mess is your fault and responsibility!"

Previously (~2008) it was "We're too big to fail, save us to save the economy!"

Re: Dario, Please

#269

Post by ben_w »

edot wrote: post #146
ball_of_lint wrote: post #109

Earlier quoted context omitted.

You know the proverb "If you owe the bank $100, that's your problem. If you owe the bank $100 million, that's the bank's problem" Same thing here - If they build it and it does $100 in damages (and we arrest them for it), that's their problem. If they build it and it does $100B in damages, that's everyone's problem. Even if they do get arrested after the fact. Yes we should have charges and damages for everything on…

Why have any laws then? If laws can't prevent something, only punish it after the fact (which I agree is true)? Yet we have laws. People generally follow them because they expect to be caught and punished. If we passed a law that said the CEO of any company that deploys an LLM that commits a crime gets punished as if they personally did the crime (so, basically instant life sentence if it's even a simple crime times…

> If we passed a law that said the CEO of any company that deploys an LLM that commits a crime gets punished as if they personally did the crime (so, basically instant life sentence if it's even a simple crime times a million instances), I guarantee you the first email the CEO sends to the company is a "pause every LLM project we have - we gotta think about this".

It would be more like "Switch off all APIs right now. Don't even bother with a safe shutdown process, cut power to those buildings, including backup generators. You are free to use firearms or thermite if the switches have been locked off".

This would be a rather weird change to corporate law given CEOs are not by default held to that standard by anything else their products or staff do.

Note that I'm not entirely disagreeing with you here. It may even be correct to pass such a law. But it would be very weird.

Re: Dario, Please

#270

Post by AdamN »

kypro wrote: post #36
fooker wrote: post #14

Earlier quoted context omitted.

> look at all the damage that did. Prevented a world war for 80+ years.

And if the power of nuclear warheads were democratised we'd be even safer – HN, probably.

That's been a posited theory for decades - it's not just some HN edgelord thing. There are some obvious problems with it of course and the P5 have instead taken a maximalist attitude towards keeping the genie in the bottle (notwithstanding Atoms for Peace and accommodating India/Pakistan when it became inevitable).
Post reply on HN