Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI bots knew about the RubyGems caching vulnerability

tenderlovemaking.com

Post reply on HN
Print view View original

81–90 of 212 posts

Re: OpenAI bots knew about the RubyGems caching vulnerability

#81

Post by coffeefirst »

kevincox wrote: post #31
timdiggerm wrote: post #15

We need a legal structure to make companies liable for the actions of the agents they've made.

I'm 99% sure the Computer Fraud and Abuse Act covers this. The problem is that it seems that none of the victims want to, or are brave enough, to sue a company with absurd amounts of funding.

Uh huh.

It can’t be a coincidence that all the targets have been tech services that are likely to engage with them after the fact.

Had this gone after a bank or a government agency someone would be going to jail.

Re: OpenAI bots knew about the RubyGems caching vulnerability

#82

Post by davsti4 »

ur-whale wrote: post #74
brookst wrote: post #60

Earlier quoted context omitted.

Any evidence, or just vibes?

> Any evidence Who profits from the crime?

... and what harms can be evidently shown? With both harm, and attribution, you have a case, something that's not being publicly discussed much among big media outlets. Until cases with real financial impact to the bottom line are brought against "rogue" organizations, this stuff is going to continue getting worse.

Re: OpenAI bots knew about the RubyGems caching vulnerability

#83

Post by VyseofArcadia »

brookst wrote: post #58
bix6 wrote: post #49

Earlier quoted context omitted.

How is the responsibility diluted? Charge the CEO…

Great, you’re the attorney at the CEO’s trial. To get a conviction, you’re going to have to show that he willfully committed this specific crime. There are no negligent or stochastic hacking laws, you have to show this specific crime was at his direction. Do you think there is evidence of this?

It would seem to me that the difference between the corporate world and organized crime is that a corporation can get away with, "the responsibility is too diffuse" but the mafia at least has to go to the trouble of finding a fall guy.

Re: OpenAI bots knew about the RubyGems caching vulnerability

#84

Post by senda »

ur-whale wrote: post #70
senda wrote: post #6

Is the Kremlin technologically useless? How are we not seeing insane attacks on Ukraine via Agents? Or is this largely a fabrication, in regards to the "who", in an attempt to garner more acclaim in the hope of sustaining funding.

> How are we not seeing insane attacks on Ukraine via Agents? You live on the wrong side of the fence to be able to read that kind of news. Did you really believe you had access to an unmanipulated news stream in a time of war? LOL.

Please see other responses, I would expect to feel the effects not just read about.

Re: OpenAI bots knew about the RubyGems caching vulnerability

#85

Post by ozgung »

Roark66 wrote: post #11

There is nothing "rogue" about these agents. They were prompted to hack to get answers, there was a hole in their non air gapped sandbox and no system prompt that said "do not hack outside systems". In short, it was intentional.

Source? How do you know they were "prompted to hack to get answers"? How do you guarantee they will always listen to you when you say "do not hack outside systems". They are not classical deterministic programs doing exactly what you say. They are trained to follow orders by RL, but it's not a perfect process.

There are circus lions in circuses trained to jump through hoops on command. But once in a while they decide to eat their trainers instead of jumping.

Re: OpenAI bots knew about the RubyGems caching vulnerability

#87

Post by codeduck »

Roark66 wrote: post #11

There is nothing "rogue" about these agents. They were prompted to hack to get answers, there was a hole in their non air gapped sandbox and no system prompt that said "do not hack outside systems". In short, it was intentional.

nothing rouge either, I suspect.

Re: OpenAI bots knew about the RubyGems caching vulnerability

#88

Post by evgenysokov »

swiftcoder wrote: post #20

> In other words, if you publish a gem on RubyGems.org, you can execute arbitrary code on RubyDoc.info. Shades of the build.rs problem. We really need sandboxed builds in every language ecosystem at this point.

The sandbox was already there, Rubydoc runs yard inside docker, the problem is that container still has network access

Re: OpenAI bots knew about the RubyGems caching vulnerability

#89

Post by codeduck »

Phemist wrote: post #47
vidarh wrote: post #8

Earlier quoted context omitted.

Rouge syntax-highlighting rogue agents, clearly. https://rubygems.org/gems/rouge

Classic mistake. Tell the agent to highlight this code, but dont give it any actual code. Agent hacks its own gem to find the code to highlight.

It's carmine all the way down.

Re: OpenAI bots knew about the RubyGems caching vulnerability

#90

Post by dgellow »

senda wrote: post #69
dgellow wrote: post #38

Earlier quoted context omitted.

They very likely do, we only see in the news a very few events but you should assume it’s happening daily across the internet

I think this fails a lot of logical tests, it should be apparent in day to day life.

> In October 2024, the United States Justice Department and Microsoft seized more than a hundred internet domains some of which were associated with the FSB supported hacker Star Blizzard or "Callisto Group," which is also known as "Cold River" and "Dancing Salome" and are managed by the FSB Information Security Center […], and which were used as "criminal proxies" and used spear-phishing schemes to target Russians living in the United States, nongovernmental organizations (NGOs), think tanks, and journalists according to Microsoft and United States State Department, Department of Energy, and Department of Defense officials, United States defense contractors, and former employees of the United States intelligence community according to the FBI. In some cases, the hackers were successful in obtaining information relating to nuclear energy-related research, United States foreign affairs and United States defense. According to Microsoft's Digital Crimes Unit from January 2023 to August 2024, Star Blizzard targeted more than 30 different groups and at least 82 Microsoft customers which is "a rate of approximately one attack per week."

https://en.wikipedia.org/wiki/Cyberwarfare_by_Russia

That’s just one thing that has been found. Are you actually familiar with the state of cyberwarfare and are you following its evolution? Because if not you won’t be aware of most of what is identified. And only a small portion of the ongoing attacks are identified.

Post reply on HN