Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Dario, Please

pop.rdi.sh

Post reply on HN
Print view View original

201–210 of 235 posts

Re: Dario, Please

#201

Post by augment_me »

ozozozd wrote: post #192
augment_me wrote: post #144

Earlier quoted context omitted.

Well the difference is that in the software realm, you can attach infinite loops of delegation and ownership at close to no cost, which makes this impossible to investigate conclusively. Let's say that OpenAI used a shell company that hosts server where an agent spun up another agent on instructions from another agent which was corrupted by bit errors from the inference framework which caused some major hack to happe…

I am sorry but that’s pure fantasy. When you say infinite loops, you mean infinite indirection, but obviously no such thing exists, because computer systems, just like other physical things, exist in physical space, not on the astral plane. Whoever had agency to start the domino effect carries the liability. Doesn’t matter if the model is open source or if you brewed it home. And if you weaponize OpenAI’s models thro…

Pure fantasy, yet you see this happen, literally every day, with the hacks, with content theft, etc, no repercussions for anyone.

Which judge will go down the rabbit hole of figuring this out? How will they do it? Will they have people tracing logs over 5000$? And if they do get there, in your fictional world, at some point, after 1 year of investigations and back-and-forth, it's okay, the technology is beyond what it was before, it's not relevant anymore, new technologies and new methods.

Re: Dario, Please

#202

Post by Refreeze5224 »

soundworlds wrote: post #150
turtletontine wrote: post #139

Earlier quoted context omitted.

If by “we” you mean “experts and professionals outside the US”, I think the short answer is they’re not talking to you here. They’re trying to secure regulatory capture in the US, and monopolize their hold on US corporations. To the extent they’re thinking of other “markets”, I would guess the assume the outsized influence of US corporations will have the much of the rest of the world using their products.

Yeah fair point. I believe he truly has humanity's best interests at heart.. And yet he keeps speaking as though everyone still sees the US as the shining beacon of the world. That's simply not true anymore, and his messaging will continue to be undermined until he treats other countries as equals (whether that's China, Australia, or anywhere else).

> I believe he truly has humanity's best interests at heart

How can you possibly believe this? What else would someone in his position say? No billionaire has humanity's interest at heart, so they try to hide that fact by building libraries (Carnegie) or creating foundations named after themselves.

All of his words and none of his actions support his caring at all about humanity, and you choose to believe his words? The man has an IPO coming up (probably one of the biggest ever!), he could not be in a less trustworthy position.

Re: Dario, Please

#203

Post by r3trohack3r »

jiggawatts wrote: post #133
hobom wrote: post #98

Earlier quoted context omitted.

They have such a trusted access program. "Life Sciences Verification Program: The LSVP is designed so that life sciences professionals can use Claude Mythos 5.1 with safeguards designed for professional research and development activities (while all other safeguards remain in place). In partnership with the US government, we have enrolled our first participants, and we plan to expand access to this program to the bro…

These "access gates" and export controls are going to look hilariously quaint in a few years. It reminds me of the export controls on PlayStation 2 consoles because it was deemed that 6 gigaflops was a "dangerous" amount of computer power, and it couldn't be allowed to fall into the hands of opposing militaries: https://www.latimes.com/archives/la-xpm-2000-apr-17-fi-20482... Now the phone in my pocket does 2,500 giga…

To be fair, my understanding of these arguments is that they’re about deltas and not about raw numbers.

I don’t agree with them, but I don’t think it was the raw compute power as much as it was maintaining the _delta_ in compute power.

Re: Dario, Please

#205

Post by verdverm »

includenotfound wrote: post #160
zmmmmm wrote: post #81

It would all be more convincing if the incidents so far didn't seem to be facilitated by an outrageous level of negligence. We had OpenAI "accidentally" run an entire swarm of 10,000 agents apparently for weeks, on a security related task, seemingly totally unsupervised, hacking all over the internet - all the conversations were completely visible, anybody who looked would have seen it. But they didn't. So before we…

The "sandbox" they used was apparently made of thin paper exposed under a day of heavy rain, too. You'd think, if they truly believed the model is so dangerous, they'd run it in a VM without a network adapter.

> You'd think, if they truly believed the model is so dangerous...

They would have been watching what it does, especially when running it on ExploitGym of all benchmarks... that is criminal worthy neglegence

Re: Dario, Please

#206

Post by rendang »

tosapple wrote: post #135
icedchai wrote: post #130

Earlier quoted context omitted.

While that would be bad if they broke all of TLS, it would not let agents "take over the internet." What would that even mean? Pumping out even more slop? Also, AI providers are literally getting a stream of traffic with every prompt and every response. How can they not know what's being worked on? They are more likely to use that an excuse to ban open models where they can't know what's being worked on.

two of these things elected a guy who handled the snowden leaks to field their own questions, what's stopping the next one from signalling in morse through a debian package mirror to putin or xi?

They elected who? What does this refer to?

Re: Dario, Please

#207

Post by slowerturnip »

This is a pretty difficult read, in no small part because the author’s frustration with the frontier labs has become a bilious, delusional cynicism that leads him to see dog whistling and obfuscation even in cases where Amodei plainly intends for every reader to see his meaning.

Anyway,

> Dario in as many words, asks for regulation/ban on open weight models.

The big labs do want this. I understand why the author and many others want open models protected. The economic and political power the labs will have if they succeed, ladder pull competitors, and avoid being nationalized (or even if they don’t avoid that) is a disturbing prospect.

But that’s the end of the issue? There’s nothing more to think about here? The open weights proponents seem to think of ai as a utility when it’s more like a utility that also is a tank. I’d feel better having a tank if all my neighbors had tanks, and I’d also feel better having a tank if a few corporations were giving out tanks to people with pockets deep enough. I’d much rather be in a situation where I wouldn’t feel I needed a tank, or where the tanks my neighbors and I own don’t have guns on them.

The open weights are going to need regulation. Hopefully there’s a way to do this effectively that isn’t banning them. Denying historical and reasonably projected capability gains because that reality makes the regulation conversation a necessary one is something I’d like to see less of

Re: Dario, Please

#208

Post by verdverm »

jiggawatts wrote: post #133
hobom wrote: post #98

Earlier quoted context omitted.

They have such a trusted access program. "Life Sciences Verification Program: The LSVP is designed so that life sciences professionals can use Claude Mythos 5.1 with safeguards designed for professional research and development activities (while all other safeguards remain in place). In partnership with the US government, we have enrolled our first participants, and we plan to expand access to this program to the bro…

These "access gates" and export controls are going to look hilariously quaint in a few years. It reminds me of the export controls on PlayStation 2 consoles because it was deemed that 6 gigaflops was a "dangerous" amount of computer power, and it couldn't be allowed to fall into the hands of opposing militaries: https://www.latimes.com/archives/la-xpm-2000-apr-17-fi-20482... Now the phone in my pocket does 2,500 giga…

I'm mildly sad you didn't use 2,500 jiggaflops

Re: Dario, Please

#209

Post by bigyabai »

slowerturnip wrote: post #207

This is a pretty difficult read, in no small part because the author’s frustration with the frontier labs has become a bilious, delusional cynicism that leads him to see dog whistling and obfuscation even in cases where Amodei plainly intends for every reader to see his meaning. Anyway, > Dario in as many words, asks for regulation/ban on open weight models. The big labs do want this. I understand why the author and…

> The open weights proponents seem to think of ai as a utility when it’s more like a utility that also is a tank

You could say the same about normal computers. Where are our regulations on Kali Linux, to prevent people from bruteforcing weak WPA passwords?

The single most-pressing concern with AI is that it can accelerate the process of hacking things. This is a preexisting problem that is inherent to software and needs proper addressing. Even if we regulate open weights tomorrow, people still have uncensored GLM-5 finetunes doing whatever they want on their own hardware. The "what if" of capable open models is here today, there are no guardrails.

Re: Dario, Please

#210

Post by verdverm »

DLA wrote: post #78
dmoose wrote: post #20

If we can get everyone to slow down we can hemorrhage less money going into our ipo.

This! $1.6+ TRILLION in infra spending from the big frontier labs. The earnings needed to drive a reasonable ROI to recoup that investment is simply not going to happen in a time frame where the numbers make sense. The other insanity in all this the smartest computer scientists in the world are asking Congress to regulate them. Come. On. Really? Do we remember “The internet is not a truck, it’s a series of tubes…” Wh…

> Non-democratic counties (hey there China) will not abide by any agreement that constrains their advantage.

The main counterexample to this was nuclear weapons. Atom bombs have not been used to kill since the US did so. However today, the two largest nuclear powers have no legal agreement on arms control because Donald the Trump declined Russia's offer for an extension to the existing agreement. Now other countries are looking at Ukraine, Iran (attacked for wanting nukes) vs NKorea (not attacked because they have them), and Donald's own musings about the US nuclear umbrella being a bad idea (France is going to build more nukes now too)... and we now face nuclear proliferation again on a global scale, with tech that is nearly 100 years old now.

Post reply on HN