OpenAI bots knew about the RubyGems caching vulnerability • Hacker News Board

Re: OpenAI bots knew about the RubyGems caching vulnerability

#291

Post by TZubiri » Mon, Sep 14, 2026, 9:39 PM UTC

VyseofArcadia wrote: post #16

How does this work, legally? I think that RubyGems could file a civil suit against OpenAI, but for a naïve non-lawyer reading this seems like a pretty clear cut criminal violation of the computer fraud and abuse act.

>could file a civil suit against OpenAI,

Everyone can sue everyone, there's no prohibition on suing someone, what changes is whether the case is good (has a reasonable chance of favourable sentence)

That said, it is often unclear whether an agent is operated by the model manufacturer (for example by scraping a website), or acting on behalf of a user.

In the former ofc the proper defendant is OAI. On the second, the argument for suing OAI is weak, the most natural defendant is the user that prompted the agent. If the facts later reveal that there was no malicious intent, then you can retarget the defendant.

Re: OpenAI bots knew about the RubyGems caching vulnerability

#292

Post by EdwardDiego » Mon, Sep 14, 2026, 9:40 PM UTC

firesteelrain wrote: post #223

> If you have YARD installed, and you install this gem, then YARD will load and run whatever is in ./script.rb from inside the gem. How is that not a security issue in of itself?

A lot of packages for interpreted languages that use a C or Rust library (either for performance, or because it offers the functionality you want, so just wrap it in a $INTERP_LANG API that calls into it) will use packaging code execution to fall back to trying to compile code if there isn't a pre-existing artefact that was compiled for your version/arch/etc.

I'm most familiar with Python where you get tarred up source distributions that then execute setup.py, but more commonly, wheels, pre-built binaries which don't execute code upon install - and in my company, I've been able to advocate for the work needed to upgrade to a newer Python because available wheels don't support Ye Olde version of Python because a) sdists are a security risk and b) if you're trying to install a package that wants to compile C or Rust, suddenly you get to do the fun "install the the particular version of clang this thing needs, the Python header files, and then set the env vars for the compiler and linkers" dance that slows developers right down.

But then there's the JVM world, where JARs don't execute arbitrary code upon installation - and it's rather uncommon to have packages that call out to a C lib for performance, but you'll get some that wrap existing libraries for functionality like RocksDB.

Re: OpenAI bots knew about the RubyGems caching vulnerability

#293

Post by qarl » Mon, Sep 14, 2026, 9:41 PM UTC

ethin wrote: post #240

qarl wrote: post #171

Earlier quoted context omitted.

I hate to spoil your mood - but it is currently unclear whether agents can be self-aware. And it's very likely something that can never be known.

Please explain how it is "unclear" that agents "can be self-aware"? As Wikipedia would say, citation needed. Just because an agent can write convincing enough to convince you that it's "self-aware" doesn't mean it really is, in fact, self-aware.

Well - it's not hard to find. But ok:

Birch, The Edge of Sentience (2024), ch. 16 - "simply no way to assess sentience in an LLM"

Schwitzgebel, AI and Consciousness, (2025) — "we won't know before we've already manufactured thousands or millions of disputably conscious AI".

Butlin, Long et al., Consciousness in Artificial Intelligence: Insights from the Science of Consciousness, (2023) — "no obvious technical barriers to building AI systems which satisfy these indicators".

Let me know if you need more.

Re: OpenAI bots knew about the RubyGems caching vulnerability

#294

Post by sparkling » Mon, Sep 14, 2026, 9:55 PM UTC

kevincox wrote: post #31

timdiggerm wrote: post #15

We need a legal structure to make companies liable for the actions of the agents they've made.

I'm 99% sure the Computer Fraud and Abuse Act covers this. The problem is that it seems that none of the victims want to, or are brave enough, to sue a company with absurd amounts of funding.

Or the victims have no incentive to do so, such as in the Hugging Face case.

Re: OpenAI bots knew about the RubyGems caching vulnerability

#295

Post by pantelisk » Mon, Sep 14, 2026, 9:55 PM UTC

But how did they know about it? Did the agents independently discover it? Did the model know about it because this vuln was public somewhere and systems weren't patched yet?

Re: OpenAI bots knew about the RubyGems caching vulnerability

#296

Post by snicky » Mon, Sep 14, 2026, 9:56 PM UTC

qarl wrote: post #171

simonebrunozzi wrote: post #168

Earlier quoted context omitted.

I would agree with you generally, but in this particular case, the distinction seems important because a significant percentage of the world population believes that agents can be self-aware, a-là Terminator etc.

I hate to spoil your mood - but it is currently unclear whether agents can be self-aware. And it's very likely something that can never be known.

How would one define self-awareness?

Re: OpenAI bots knew about the RubyGems caching vulnerability

#297

Post by drtgh » Mon, Sep 14, 2026, 10:00 PM UTC

thomasjeff1 wrote: post #213

Great time to be a criminal. Just have your bots do it.

Given what is happening, If the next generation of Trojans get called "bacteria" through intense marketing, with some random functions to give a nondeterministic behaviour, one can not be criminalized of what the bacterias do along their digital living cycle.

Re: OpenAI bots knew about the RubyGems caching vulnerability

#298

Post by ozim » Mon, Sep 14, 2026, 10:01 PM UTC

vipshek wrote: post #257

In the physical world, it seems like when an tool/device/instrument causes harm (or is used to cause harm), we assign blame to either the user of the tool or its creator. When do we blame the user? When the tool is operating as intended by its creator, and we agree the tool meets certain quality standards and isn't defective. When do we blame the creator? When the device doesn't meet those quality standards and reaso…

Software industry standards are as in Microsoft EULA. If your house burns down because of known flaw in Microsoft Windows they are not liable (well as far as EULA let’s them, you can most likely still sue them).

Software as big as operating system already is non deterministic when integrating with unknown hardware or 3rd party software.

That is why Apple controls the hardware and OS for their products, because they can limit non-deterministic things from happening this way.

Re: OpenAI bots knew about the RubyGems caching vulnerability

#299

Post by elmo2you » Mon, Sep 14, 2026, 10:05 PM UTC

jasomill wrote: post #249

elmo2you wrote: post #226

Earlier quoted context omitted.

As far as I know (IANAL) it is in fact the only "person" you can charge. To the best of my knowledge, the whole point these "limited liability" legal constructions exist in the first place, is to protect individuals within a corporation for whatever they do as part of the business of a company (barring exceptions that have clearly not been part of that business and obvious individually committed crimes), typically "j…

"Limited liability" refers to shareholders' financial liability being limited to their investment, and has nothing to do with civil or criminal liability of employees for their own actions, whether "following orders" or not.

You might want to look up LLC (Limited Liability Company), which goes by other names in different countries but still basically mean the same (though details may vary between different company types). While there certainly are exceptions, as mentioned before, in principle the employees of a company are not personally legally accountable for their actions as performed in the service of a company. The legal entity of that companies is. If you sincerely believe otherwise, you either live in a rather unusual country or maybe just need to educate yourself a bit better.

Re: OpenAI bots knew about the RubyGems caching vulnerability

#300

Post by 0xbadcafebee » Mon, Sep 14, 2026, 10:10 PM UTC

vipshek wrote: post #257

In the physical world, it seems like when an tool/device/instrument causes harm (or is used to cause harm), we assign blame to either the user of the tool or its creator. When do we blame the user? When the tool is operating as intended by its creator, and we agree the tool meets certain quality standards and isn't defective. When do we blame the creator? When the device doesn't meet those quality standards and reaso…

> Maybe we need "quality certifications" for AI agents - essentially eval suites that demonstrate those agents won't cause harm under reasonable patterns of usage

Based on how LLMs work, this is impossible. You cannot predict how they work, it's literally based on a combination of random seed and a mostly-unpredictable path walked based on every token of input.

You don't blame a knifemaker for somebody getting cut by a sharp knife. AI is a knife. Very handy, very dangerous. We have to use them safely, that's all there is to it.

> the "industry standards" common in, say, electrical engineering and other disciplines are sorely lacking here

100% agreed. We have ignored SWEng's lack of discipline for too long. Now that the SWEng isn't even a human, we are looking at total catastrophe (on the scale of improperly built buildings falling down on people or catching fire) if we don't adopt a software building code.