Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Dario, Please

pop.rdi.sh

Post reply on HN
Print view View original

321–330 of 335 posts

Re: Dario, Please

#321

Post by zahlman »

Tanjreeve wrote: post #235
zahlman wrote: post #178

Earlier quoted context omitted.

> Why have any laws then? If laws can't prevent something, only punish it after the fact (which I agree is true)? Yet we have laws. Because there are many important values of "something" where such punishment acts as a deterrent to other would-be criminals; and because society is more stable when people see justice done and that mollifies hoi polloi after the damage occurs. But many AI doom scenarios don't fit that p…

That sure is convenient that "nothing has happened" so far between the plagiarism and the cybercrime and we may as well just wait for "the one true AI doom"

I claim nothing of the sort. I explicitly spoke in support of having laws, and of course I think they should be enforced too. The context is https://news.ycombinator.com/item?id=49705589 :

> Yes we should have charges and damages for everything on https://www.felonybench.com/, but that doesn't address the core issue of this being possible at all.

Re: Dario, Please

#322

Post by icedchai »

tosapple wrote: post #135
icedchai wrote: post #130

Earlier quoted context omitted.

While that would be bad if they broke all of TLS, it would not let agents "take over the internet." What would that even mean? Pumping out even more slop? Also, AI providers are literally getting a stream of traffic with every prompt and every response. How can they not know what's being worked on? They are more likely to use that an excuse to ban open models where they can't know what's being worked on.

two of these things elected a guy who handled the snowden leaks to field their own questions, what's stopping the next one from signalling in morse through a debian package mirror to putin or xi?

Why are they bothering to signal through a debian package mirror instead of just sending a message directly?

Re: Dario, Please

#323

Post by Quibblingeek »

dmitrygr wrote: post #248
ball_of_lint wrote: post #109

Earlier quoted context omitted.

You know the proverb "If you owe the bank $100, that's your problem. If you owe the bank $100 million, that's the bank's problem" Same thing here - If they build it and it does $100 in damages (and we arrest them for it), that's their problem. If they build it and it does $100B in damages, that's everyone's problem. Even if they do get arrested after the fact. Yes we should have charges and damages for everything on…

That’s because arrest is a dumb solution. Restitution. Make them fix what they broke. By hand. 80 hours a week from now until death of natural causes.

No. If you institute a culture that supports and encourages your subordinates to commit criminal acts in the furtherance of a private enterprise, the enterprise and its representatives should be held legally accountable.

A lot of problems with America could be fixed if the Department of Justice actually grew a pair and prosecuted people for committing criminal acts.

The fact that Les Wexner is not rotting in prison for facilitating Epstein and his friends exploits speaks volumes to the state of this country.

If corporations are legally people, they should be held to the same standard as natural persons.

As it stands, the “restitution” that is required is so insignificant that it’s treated as an additional tax. If there is no deterrence, there is no behavioral modification. If there is no behavioral modification, the law becomes unenforceable.

Re: Dario, Please

#324

Post by marcus_holmes »

rickdeckard wrote: post #258
zahlman wrote: post #178

Earlier quoted context omitted.

> Why have any laws then? If laws can't prevent something, only punish it after the fact (which I agree is true)? Yet we have laws. Because there are many important values of "something" where such punishment acts as a deterrent to other would-be criminals; and because society is more stable when people see justice done and that mollifies hoi polloi after the damage occurs. But many AI doom scenarios don't fit that p…

> The first time "something" happens (at least if you buy the argument) would be bad enough for legal punishment not to matter. One could argue that the "HuggingFace incident" is already "something" which should be investigated and punished very heavily. It's kinda ridiculous to build a machine that attacks a competitor of yours and the CEO gets to write blog-posts on how fascinating this machine is

This. I don't understand why OpenAI aren't being prosecuted for that attack.

Even if you allow that the LLM can't be held responsible, the sandboxes were clearly not up to the task and the whole incident was mismanaged. There are people who made those decisions and should be held accountable for them.

Re: Dario, Please

#325

Post by includenotfound »

zahlman wrote: post #315
includenotfound wrote: post #310

Earlier quoted context omitted.

The best part is, they are totally able to one shot airgapping. You are too. Go ask your local agent to set up a pre-configured Linux VM with whatever stuff you want on it, then ask it to airgap it allowing only X, Y, Z services. It will one shot it. You know what's better? They already do this per (paid) user - your ChatGPT subscription comes with a Linux VM that you can even legitimately SSH into, just ask your age…

Anything that's accomplished simply by running software does not qualify as "airgapping" in my view. The entire premise is that real-world software is buggy and the LLM is much better at locating and exploiting those bugs than you are at preventing them. But you can only connect to Wifi if you have Wifi hardware, and RF signals are contained by Faraday cages. Ethernet is still a thing for local connections.

> The entire premise is that real-world software is buggy and the LLM is much better at locating and exploiting those bugs than you are at preventing them

If that was the premise, why run LLMs in a lesser sandbox than a VM? Clearly it's not.

Re: Dario, Please

#326

Post by includenotfound »

zahlman wrote: post #314
includenotfound wrote: post #311

Earlier quoted context omitted.

ChatGPT paid subscriptions already give you a VM [0]. OpenAI couldn't spare a few VMs for the super ultra mega dangerous evals where they asked the agents to specifically go hack stuff? [0] https://news.ycombinator.com/item?id=49718530

The issue as I understand it is that they didn't expect Artifactory to be vulnerable in the way it was, or else were completely not paying attention. But I'm calling for something stronger than a VM here because we shouldn't rely on the VM being bulletproof just like we shouldn't rely on Artifactory being bulletproof. The access should be controlled at the hardware level. Like, networking on internal LANs only, and t…

That's just unwarranted at this stage, given model capability and resource constraints.

Furthermore, VMs are isolation at the hardware level, particularly through hypervisors. I'd say given current LLM capabilities, it's a reasonable containment.

Re: Dario, Please

#327

Post by danny_codes »

teekert wrote: post #268

It all comes down to " Privatize profits, socialize losses." It's the American way. Here it is: "We warned you you should have regulated us! Now this mess is your fault and responsibility!" Previously (~2008) it was "We're too big to fail, save us to save the economy!"

100%. American capitalism is broken. So far that illness has given us Trump. So bad, but apparently not a catastrophe (yet). I wonder what comes next (assuming we don't fix it)..

Re: Dario, Please

#328

Post by danny_codes »

threecheese wrote: post #318

For persistence, I think the most glaring gap is the inference - if these rogue agents need to call Claude's API, then they aren't persistent - Anthropic can turn them off. A memory resident program needs disk to be persistent, but above all it needs CPU; without it there are only bits. And so the implication is the swarm has access to enough local compute to perform its own inference, using a large enough model to p…

> It's that we can never allow this technology to advance outside of trusted labs.

Impossible. In 10 years you'll be able to train today's models for ~$10M from Moore's law and training innovation. In 20 years it'll be ~$200k. The genie won't go back in the bottle.

Re: Dario, Please

#329

Post by myaccountonhn »

neom wrote: post #246
myaccountonhn wrote: post #241

Earlier quoted context omitted.

It's not really about who, it's about having institutions with good governance. I don't believe in technocracy/theocracy.

Sure, but that's what I'm asking. What institution, with what governance? Someone ultimately has to decide what these systems will and won't do, and there isn't an obvious democratically legitimate institution with global jurisdiction over AI as far as I'm aware?

Today no legitimate institution exists, so I think open model are the best we have.

As to how a company like Anthropic should act if they want to be the ones to govern it, I think it's a great question but I can't answer that in a HN thread. Fundamentally they'd need to change their governance structure to be less tied to US and private interests, and address many of the ethical concerns raised by scholars/commentators/politicians like Cory Doctorow, Marietje Schaake, and Carissa Véliz to start with. That's just the tip of the iceberg.

Re: Dario, Please

#330

Post by account42 »

ball_of_lint wrote: post #109
vb-8448 wrote: post #26

> “a swarm of agents could be capable of taking over the entire internet with a persistent botnet.” I'm wondering why no one is mentioning the "accountability" word. Why these companies are allowed to damage others with impunity? Start making managers pay the price for their actions, and watch how the models magically slow down on their own.

You know the proverb "If you owe the bank $100, that's your problem. If you owe the bank $100 million, that's the bank's problem" Same thing here - If they build it and it does $100 in damages (and we arrest them for it), that's their problem. If they build it and it does $100B in damages, that's everyone's problem. Even if they do get arrested after the fact. Yes we should have charges and damages for everything on…

That's why you punish people doing $100B in damages (or even something that comes close to doing that much damage) harshly enough that no one even thinks of doing that.
Post reply on HN