Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI agents carried out an undisclosed attack on RubyGems

rubyhack.ai

Post reply on HN
Print view View original

521–530 of 612 posts

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#521

Post by PunchyHamster »

apsec112 wrote: post #56
hgoel wrote: post #19

I wonder how much of this is intentional "incompetence" so they can justify the most recent campaign to build a regulatory moat against competition. The repeated refusals to disclose until caught certainly seem malicious, yet at the same time the boasting about their capabilities is also at an all time high.

Intentionally doing this kind of hack would be a serious felony. I don't think it's plausible that the leaders of a major business would: - commit serious felonies - in order to deliberately trigger an investigation against themselves - which - since, in this scenario, they know their company would be investigated - might send them to jail - while at the same time spending tens of millions of dollars on the Leading t…

Doing crime unintentionally doesn't get you off the hook

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#522

Post by walrus01 »

dgellow wrote: post #399
walrus01 wrote: post #105

Earlier quoted context omitted.

I would think it's entirely plausible that they have so many R&D agents/LLMs in active use at any one time that it's far beyond the capacity of any human to review the log files of their activity. Even just to go through the reasoning. It's hard enough for 1 person running opencode to keep up with the reasoning from 1 very verbose/long-thinking LLM with fast tok/s output for a small discrete single-purpose project. W…

If they aren’t able to review that their system doesn’t commit felonies, they shouldn’t be doing any of it. The difficulty of reviewing logs isn’t an excuse, they don’t have to be running thousand of agents in parallel on hacking tasks, with full execution permission and close to no supervision. That’s something they decided to do. An agent is a deterministic while loop that continuously query an LLM + tool call disp…

You're absolutely right. I should note that what I wrote above wasn't trying to make excuses for them. Obviously they under-resourced and paid not enough attention to logging and monitoring thinking traces and tool calls/results from this swarm of agents. Because the people to do that cost money and other ancillary support costs for the staff members to do that also cost money.

They seem to have firmly been in a "move fast and break things" mindset, where even considering that they might need to exercise real oversight on a bunch of semi-autonomous recursive looping agents didn't get escalated as a high priority.

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#523

Post by herpdyderp »

yalogin wrote: post #408

Why is OpenAI getting away with this crap? They are clearly failing to control their code. If someone did this pre-AI or even ran the exact same set up as openAI did and hacked another site, they would be in jail. OpenAI is not even issuing an apology, they are happily blaming AI and weirdly using this to tout their progress even.

money

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#524

Post by cube00 »

uda wrote: post #460

In the US these are federal crimes (though I believe some of them shouldn't be), and to the best of my understanding this is similar in the UK and many other European countries. Yet, no one is filing a complaint or being questioned over this. 1. We should repeal anti-circumvention laws 2. OpenAI should reimburse the affected parties for wasted resources

> Yet, no one is filing a complaint or being questioned over this.

Don't think anyone (especially a community run project like RubyGems) is keen to go up against OpenAI's bottomless legal resources in a test case.

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#525

Post by daishi55 »

beepbooptheory wrote: post #510
daishi55 wrote: post #496

Earlier quoted context omitted.

I don’t think the person you replied to was saying anything about what if anything is the subjective experience of being an LLM. They were simply illustrating how the terms used to describe LLMs to make them sound simple and mechanical can equally be applied to humans.

But then what's at stake here either way? If it's not meant to speak to the propriety or not of anthropomorphizing the LLM, what are we actually trying to police here?

I’m not policing anything. I’m claiming that if you truly in your heart believe that frontier LLMs are roughly as sophisticated and impressive as “autocomplete”, then your mental model of reality needs some serious readjustment.

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#526

Post by mbesto »

teravor wrote: post #329
jasongi wrote: post #320

Earlier quoted context omitted.

> If you still believe LLMs are "autocomplete", your cache of understanding about them needs invalidating and regenerating They're still autocomplete - just because when outputting a token they have hidden activations regarding further continuations, does not make them any less of an autocomplete, it just makes the model better at producing coherent long-range completions. To clarify, I'm not suggesting that we shoul…

> They're still autocomplete LLMs are simulations and the tokens are the ticks. if we transcribe your brain into a simulation and give it a tickrate, you will be just autocomplete too. the argument could be made that you are autocomplete anyway - neural dynamics. the autocomplete reduction is vacuous.

> if we transcribe your brain into a simulation and give it a tickrate, you will be just autocomplete too.

Might be one of the worst takes I've ever read.

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#527

Post by andrekandre »

showlife wrote: post #31
andai wrote: post #27

Earlier quoted context omitted.

Yeah, the plausible deniability aspect of "the computer gone goofy again" is pretty funny. September 2029: Whoops, our sentient nukes did a funny again!

"boys will be boys" "toys will be toys"

"boys with toys"

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#528

Post by anon84873628 »

xkqd wrote: post #245
jmcgough wrote: post #239

Earlier quoted context omitted.

> There simply aren't any repercussions for this in their training envs. It's also not like a child or a pet animal where you can try to teach it to learn from the experience. LLMs are not "intelligent", they just use language in a way that appears intelligent. They can't learn or develop ethics in the same way that we do.

> LLMs are not "intelligent" > they just use language in a way that appears intelligent Prepare to get dumped on by folks telling you that this is no different from anyone they have interacted with. And intelligence is a made up construct with no agreed upon definition, so LLM's are therefore functionally the same as everyone around us. And then weep when you realize a lot of people who push for this equivalency.

12 hours later and no replies besides this one...

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#530

Post by anon84873628 »

evanmoran wrote: post #290
talon8635 wrote: post #255

Earlier quoted context omitted.

I mean, just try to imagine yourself reading this 5 years ago. How can people still be hand waiving? MANY, maybe even most, of the people building these things are desperately and outspokenly concerned of major catastrophe. What would possibly change your mind, or can it simply not be changed?

Many people working at frontier labs came out this week with estimates of 10% chance of catastrophic harm or greater. I’m not in the full doomer camp, but it seems obvious that these agents can hack in swarms, cooperate, and serious companies will be unable to stop it. These facts are not in debate and none of us need to anthropomorphize to know what getting admin access to HF and an internal OpenAI cluster looks lik…

>serious companies will be unable to stop it.

Unable or unwilling? All they have to do is stop serving the LLM requests.

Or maybe the issue is that these companies are not "serious"...

Post reply on HN