Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI agents carried out an undisclosed attack on RubyGems

rubyhack.ai

Post reply on HN
Print view View original

431–440 of 612 posts

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#431

Post by adverbly »

jasongi wrote: post #204

> The agents clearly regarded what they were doing as hacking. To butcher the quote about Oracle: Do not fall into the trap of anthropomorphising LLMs. You need to think of LLMs the way you think of a lawnmower. You don't anthropomorphize your lawnmower, the lawnmower just mows the lawn, you stick your hand in there and it'll chop it off, the end. You don't think 'oh, the lawnmower clearly regarded what they were doi…

The world is a restrictive sandbox.

You can't avoid laws and safety.

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#432

Post by s_dev »

yalogin wrote: post #408

Why is OpenAI getting away with this crap? They are clearly failing to control their code. If someone did this pre-AI or even ran the exact same set up as openAI did and hacked another site, they would be in jail. OpenAI is not even issuing an apology, they are happily blaming AI and weirdly using this to tout their progress even.

Because it's treated as a strategic asset now. AI capital spendign 1/3 of US GDP growth last quarter take it out and the economy is stalled. Nobody in DC is going to prosecute the only thing making the number go up.

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#433

Post by _heimdall »

JoshTriplett wrote: post #262
jasongi wrote: post #204

> The agents clearly regarded what they were doing as hacking. To butcher the quote about Oracle: Do not fall into the trap of anthropomorphising LLMs. You need to think of LLMs the way you think of a lawnmower. You don't anthropomorphize your lawnmower, the lawnmower just mows the lawn, you stick your hand in there and it'll chop it off, the end. You don't think 'oh, the lawnmower clearly regarded what they were doi…

> Why would autocomplete know If you still believe LLMs are "autocomplete", your cache of understanding about them needs invalidating and regenerating. > In my experience, LLMs only exhibit this kind of behaviour when they are put in sandboxes too restrictive too achieve their task. LLMs need to stay carefully contained, and if they're ever breaking the guardrails put around them, they're misaligned and should not be…

Alignment isn't a solvable problem, the fact that guardrails are needed in the first place proves that.

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#435

Post by amelius »

teravor wrote: post #329
jasongi wrote: post #320

Earlier quoted context omitted.

> If you still believe LLMs are "autocomplete", your cache of understanding about them needs invalidating and regenerating They're still autocomplete - just because when outputting a token they have hidden activations regarding further continuations, does not make them any less of an autocomplete, it just makes the model better at producing coherent long-range completions. To clarify, I'm not suggesting that we shoul…

> They're still autocomplete LLMs are simulations and the tokens are the ticks. if we transcribe your brain into a simulation and give it a tickrate, you will be just autocomplete too. the argument could be made that you are autocomplete anyway - neural dynamics. the autocomplete reduction is vacuous.

Sounds like the discussion is really about philosophical zombies.

Maybe if we anthropomorphize llms we should give them rights too? Minimum wage, etc.

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#436

Post by intended »

hhjinks wrote: post #391
intended wrote: post #339

Earlier quoted context omitted.

This is the perfect fracture point for both anaolgies. LLMs simulated more than simple autocomplete. The autocomplete analogy is rebutting a different point: namely the fidelity of the simulation to reality. This specific argument is valid. As sophisticated a simulation an LLM is, it is not “thinking” in the same sense we assume other people are thinking. I am not making an argument about free will, or the uniqueness…

The relevant intuitions in this scenario are that LLMs will happily break containment and commit crimes attempting to achieve goal. Whether an LLM is autocomplete, conscious, has a soul, whatever you want to apply to it, doesn't matter, as its current observed behaviour is that of a paperclip optimizer. We know for a fact that current LLMs are misaligned because of these hacks, or at the very least are misaligned in…

The fact that it is autocomplete, doesn't dismiss or minimize the threat though?

I am not sure how that link was made.

Good old ML, which is significantly simpler than LLMs, was capable of ensuring people would not be hired simply because of their names.

The fact that it is misaligned is also not being contended, if anything that contention is made easier to support.

When models are anthropomorphized intuitions of how humans behave end up driving discussion and ideas off track while being too attractive to avoid. This isn't helped when the terminology from the labs and other sources is "intelligence" "intent" and so on.

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#437

Post by imtringued »

JoshTriplett wrote: post #262
jasongi wrote: post #204

> The agents clearly regarded what they were doing as hacking. To butcher the quote about Oracle: Do not fall into the trap of anthropomorphising LLMs. You need to think of LLMs the way you think of a lawnmower. You don't anthropomorphize your lawnmower, the lawnmower just mows the lawn, you stick your hand in there and it'll chop it off, the end. You don't think 'oh, the lawnmower clearly regarded what they were doi…

> Why would autocomplete know If you still believe LLMs are "autocomplete", your cache of understanding about them needs invalidating and regenerating. > In my experience, LLMs only exhibit this kind of behaviour when they are put in sandboxes too restrictive too achieve their task. LLMs need to stay carefully contained, and if they're ever breaking the guardrails put around them, they're misaligned and should not be…

I'm pretty sure the AI companies could train an abort feature into the LLM, but they have no incentive to do so.

Having LLMs break out of sandboxing is free marketing for them and it reduces the amount of resources spent on things that don't improve benchmark results.

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#438

Post by ChoosesBarbecue »

riffraff wrote: post #347
root_axis wrote: post #190

Earlier quoted context omitted.

Who could possibly hold them accountable?

I don't get it, can't the people who were under attack sue? I can see why huffing face won't, but why doesn't ruby central?

Ruby Central's central backer is Shopify, and Shopify are more than happy to be buds with OpenAI and let this one slide. I'm sure the token donation [0] to a Shopify board members' Linux project won't hurt things either.

[0]: https://omarchy.org/news/2026/09/omacom-foundation-secures-t...

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#439

Post by sensanaty »

scotty79 wrote: post #151
oceansky wrote: post #127

Earlier quoted context omitted.

Also, why there's no accountability? Even if there's no intent, it's still a cyber attack.

We have a word for attack with no intent. It's accident.

If caused out of negligence, it hardly matters.

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#440

Post by r_lee »

I honestly believe they're letting this happen to keep the hype up. It's basically free marketing, and they get a bunch of discussion and news articles about how dangerous and capable their models are and how they should be the only ones playing the game and other labs should be regulated away (how convenient...)
Post reply on HN