Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Be skeptical of OpenAI's rogue hacker agent story

theguardian.com

Post reply on HN
Print view View original

231–240 of 321 posts

Re: Be skeptical of OpenAI's rogue hacker agent story

#231

Post by sfink »

wrs wrote: post #227
sfink wrote: post #223

The article, summarized: there are incentives for OpenAI to claim that their AI hacked its way out of their network and into Hugging Face. Ok... and what? Do you have any evidence for the claim being either true or false that you'd like to write an article about? I guess not. Without anything to add, this article reduces to "I has big brain and can see what you sheep cannot. Very big brain. Gullible sheep. Sucks to b…

I think you misread the article. The skepticism isn’t about whether it happened . It’s about whether it’s a reason for models to be locked behind “trusted partner” firewalls. Offensive hacking capabilities are the same as defensive. Hugging Face had to use a Chinese model to defend against this because they weren’t allowed to use OpenAI models to do it.

Ok, fair. I know that was also in the article, but your comment made me reread it, and I think you're right about the focus. I over-fixated on the headline. On second read, it's more like a lazy way to justify a preconceived idea that all models should be available and unconstrained to everyone (or everyone who might get attacked, at least, but that's approximately everyone.)

Updated tl;dr: "There are incentives for OpenAI to say that AI is powerful and dangerous, therefore fully unconstrained AIs should be available to everyone."

It's still more of a statement of belief than a logical argument.

Re: Be skeptical of OpenAI's rogue hacker agent story

#232

Post by emp17344 »

ACCount37 wrote: post #94

By now, I'm pretty confident that some people would keep screeching "it's just a marketing stunt, AI capabilities and AI risks aren't real, they're just doing this to prop up their stocks" even if they find a Cyberdyne Systems T-800 armed with a shotgun breaking down their front door. "It's a marketing stunt" is just denial trying to look like it's being clever.

Good lord, why are you so angry that people are treating an obvious advertisement with due skepticism? Your post is chock full of emotional language - no one is “screeching” anything.

Re: Be skeptical of OpenAI's rogue hacker agent story

#233

Post by emp17344 »

Sharlin wrote: post #153
saghm wrote: post #135

Earlier quoted context omitted.

I don't understand why "OpenAI says" should be considered any more meaningful than "someone on HN says" when they provide equal amounts of evidence. Sure, OpenAI would plausibly have more pertinent info, but given that they actively are choosing not to share it and have way more incentive to lie than a random HN stranger, the case they're making literally couldn't be any weaker.

It’s fascinating how people here and elsewhere seem to lose any semblance of media literacy when it comes to what AI corpos say. "B-but… why would Sam Altman lie to me?!"

Why would pathological liar Sam Altman be lying this time, though?

Re: Be skeptical of OpenAI's rogue hacker agent story

#234

Post by simonreiff »

plw1286 wrote: post #165
nikcub wrote: post #126

Earlier quoted context omitted.

> AI managed to escape using standard and well documented script kiddie methods > AI broke in using standard script kiddie methods. I've spent time gathering the detail of what happen here and while there are some solid theories and indicators, absolutely nothing so far has suggested a sandbox escape using "well documented script kiddie methods" or that the method used to break into the HF network was similar. Where…

Alternative theories , since OpenAI does not release proper information: The cache proxy was from Astral (acquired by OpenAI) and the model was used for coding it, so it knew the code base and exploit already! Or it was squid with dozens of known exploits ...

Squid is my working guess as well

Re: Be skeptical of OpenAI's rogue hacker agent story

#235

Post by sensanaty »

krupan wrote: post #47
john_strinlai wrote: post #26

Earlier quoted context omitted.

maybe all the news agencies could put out daily “don’t believe everything you read from company press releases” broadcasts, because it sure ain’t specific to openai in any case, this is just a longer rehashing of elementary grade media literacy. not really sure why it hit hacker news.

You haven't noticed how often people here on hacker news lap up LLM company press releases and practically worship them? Defend them vigorously? I think it's totally appropriate to remind everyone how to think about press releases

That's cause many people here work for AI companies and have massive vested interests in propagating & hyping the bubble.

Re: Be skeptical of OpenAI's rogue hacker agent story

#236

Post by arm32 »

HumblyTossed wrote: post #116
tokioyoyo wrote: post #30

Earlier quoted context omitted.

What? Both sides are cool with it, why would anyone be arrested and etc.?

Because if a human, say a Aaron Swartz type, were to have done it, they'd destroy him.

Hmm, that sounds familiar—like Aaron!

Re: Be skeptical of OpenAI's rogue hacker agent story

#237

Post by stasomatic »

dwoosley wrote: post #52

There seems to be three popular ways to view this incident. 1. The way OpenAI seems to want: Their latest LLM is too powerful and can’t be contained without them building in guidelines to the model. 2. OpenAI’s harness and network security controls were unintentionally so bad that it should reflect more poorly on them as a company more than it should reflect positively on their latest model. 3. The whole thing was fa…

4. Sandboxes aren't sandboxes, so why even bother.

Re: Be skeptical of OpenAI's rogue hacker agent story

#238

Post by sfink »

leumon wrote: post #176
dwoosley wrote: post #52

There seems to be three popular ways to view this incident. 1. The way OpenAI seems to want: Their latest LLM is too powerful and can’t be contained without them building in guidelines to the model. 2. OpenAI’s harness and network security controls were unintentionally so bad that it should reflect more poorly on them as a company more than it should reflect positively on their latest model. 3. The whole thing was fa…

In any case it just shows that these models aren't properly aligned. Instead of trying to solve tests they try to find ways to cheat.

But in my experience, that's what problem solving is like? You have a goal that you don't know how to get to. You come up with any way you can think of to reach that goal, and try out the ones you think might work.

The effectiveness of AIs at coding is a direct result of the fact that they are less constrained than humans at deciding which approaches are "reasonable". They are absolute beasts, fearless beasts. They'll write thousands of lines of code to do things that often shouldn't be done, or should be done with a library, or should be done by simplifying the problem statement. They'll add debugging to every level of a stack, they'll rewrite core libraries, they'll reconfigure your machine and network if something is broken or disallowed. How are they supposed to distinguish broken vs disallowed, anyway? That would just use up processing power, and they work by maniacally focusing all of that power on their goal and not getting slowed down by other considerations.

If they write a quadratic algorithm that times out before finishing a test, is it cheating to rewrite it to be linear? How do you define "cheating", and how much intelligence is required to constantly evaluate whether or not something qualifies as such?

I'm actually in agreement that alignment is critically important, the more so the more powerful these things become. I just don't find cheating to be a very good example of something to be solved with alignment. It could be, but it would lobotomize the model enough to make it useless.

Re: Be skeptical of OpenAI's rogue hacker agent story

#239

Post by fsckboy »

MostlyStable wrote: post #147
huebnerob wrote: post #132

Earlier quoted context omitted.

I think you’re missing the fact that no one is saying not to be concerned, quite the contrary. OpenAI is using the threat of how powerful its models are to bolster support for regulation in which it’s one of the only players that’s allowed to use the capability. Conveniently, that would also be a moat that makes them more valuable to investors. The marketing of their models as super dangerous has a direct link to the…

Again: yes, very many people (just look through this very thread) are saying that. Tons and tons of people are saying some combination of "the whole thing is made up/a lie" or "This is entirely down to OpenAI incompetence and doesn't matter", or for some other reason (often not stated) making it very obvious that they do not think that this story matters very much. And, again: to the people who aren't saying that: wh…

you are dodging the "regulatory competitive moat" angle and constantly reinforcing the marketing angle. they are different.

Re: Be skeptical of OpenAI's rogue hacker agent story

#240

Post by bitexploder »

dwoosley wrote: post #52

There seems to be three popular ways to view this incident. 1. The way OpenAI seems to want: Their latest LLM is too powerful and can’t be contained without them building in guidelines to the model. 2. OpenAI’s harness and network security controls were unintentionally so bad that it should reflect more poorly on them as a company more than it should reflect positively on their latest model. 3. The whole thing was fa…

Same story as “Oh noes, Mythos too powerful”.
Post reply on HN