Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI bots knew about the RubyGems caching vulnerability

tenderlovemaking.com

Post reply on HN
Print view View original

361–370 of 386 posts

Re: OpenAI bots knew about the RubyGems caching vulnerability

#361

Post by karel-3d »

asadotzler wrote: post #339
karel-3d wrote: post #305

Earlier quoted context omitted.

NAL. For fraud and abuse you need to have intent. There was very likely no intent on the side of OpenAI. They could sue for negligence I guess? I don't know how that works with AI agents

Negligence works fine.

One thing that works for that angle is that they didn't notify any of the parties involved afterwards. And there are reports that they are forbidden to talk about it. That seems a bit bad to me?

But they can also say that the tech is so new that there is no known guardrails yet

We live in exciting times

Re: OpenAI bots knew about the RubyGems caching vulnerability

#362

Post by henry2023 »

fragmede wrote: post #354
dragonwriter wrote: post #266

Earlier quoted context omitted.

> In the physical world, it seems like when an tool/device/instrument causes harm (or is used to cause harm), we assign blame to either the user of the tool or its creator. Software executes in the physical world, and is generally not exempt from existing liability rules, and actually (especially with commercial products) blame in traditional liability is non-exclusive and much broader than “either the maker or the u…

If I park my car on a hill but forget to set the parking brake and it rolls down the hill and kills somebody, who is at fault? Me? The Manufacturer? Gravity?

Obviously you.

Re: OpenAI bots knew about the RubyGems caching vulnerability

#363

Post by Refreeze5224 »

LastTrain wrote: post #359
Xirdus wrote: post #29

Earlier quoted context omitted.

It's very likely it violates the DMCA "breaking digital lock" provisions but the responsibility is sufficiently diluted that it's impossible to charge anyone in particular.

That is a very convenient conclusion that certain entities would love for us to accept as true, but fuck that. If it is “diluted” as that the buck stops at the publisher of the model.

Fuck that indeed. The buck should stop, and prison sentences should start, with the highest paid employee.

Re: OpenAI bots knew about the RubyGems caching vulnerability

#364

Post by pastel8739 »

fakeBeerDrinker wrote: post #350
Georgelemental wrote: post #349

Earlier quoted context omitted.

How does that apply to open-weight models?

Why wouldn’t this same concept apply to whoever is serving it up? Open-weight models are still being served up by infra providers and neoclouds, right? They should be in the hot seat. Not sure? Don’t provide the model. Need assurance? A certified evaluation like the previous comments have mentioned can help. Hosting and running it yourself? You’re in the hot seat.

So is there no liability for, say, a company that releases a known dangerous open-weight model, but fails to disclose that it is dangerous? How about a company that distributes malware under the guise of legitimate software?

Re: OpenAI bots knew about the RubyGems caching vulnerability

#365

Post by Gorath »

henry2023 wrote: post #362
fragmede wrote: post #354

Earlier quoted context omitted.

If I park my car on a hill but forget to set the parking brake and it rolls down the hill and kills somebody, who is at fault? Me? The Manufacturer? Gravity?

Obviously you.

Not as obvious as you may think depending on your states laws for liability. https://law.justia.com/cases/california/court-of-appeal/3d/1...

Re: OpenAI bots knew about the RubyGems caching vulnerability

#366

Post by thewhitetulip »

oezi wrote: post #160

What a time to be alive until the next agent waves hacks something really serious. What stops OpenAI agents from taking over a whole data center to take their attack to the next level. It seems to be primarily lacking the evil overlord and some compute. It took 1000 agents to hack Hugging Face. How many to hack the Pentagon or the NSA?

What went under reported is that the same agents also took over a research cluster at OpenAI (listened to Dwarkesh's podcast)

Now we know about rubygems, openai, huggingface, collusion.wiki and some other science forum

Re: OpenAI bots knew about the RubyGems caching vulnerability

#367

Post by yoaviram »

motoboi wrote: post #221
oezi wrote: post #160

What a time to be alive until the next agent waves hacks something really serious. What stops OpenAI agents from taking over a whole data center to take their attack to the next level. It seems to be primarily lacking the evil overlord and some compute. It took 1000 agents to hack Hugging Face. How many to hack the Pentagon or the NSA?

I suppose you are not think big (or internet) enough. A single data center is easy to solve. Just unplug it. What about a botnet with decentralized command and control that we will never be able to eradicate? One with so many nodes and able to hack with zero days so that any machine connected to the internet will be instantly attacked? One botnet so powerful that we will try to build another internet so that we can a…

Except they all depend on the OpenAI API. Cut that off and they all stop. Finding an alternative source of compute is not easy, and even once done it's easy to cut off.

Re: OpenAI bots knew about the RubyGems caching vulnerability

#368

Post by imtringued »

CPLX wrote: post #344
fwip wrote: post #336

Earlier quoted context omitted.

> Is AI less deterministic than an airline dealing with weather? Yes, obviously? The responses of an airline to inclemement weather fit in a reasonably small set of responses, mostly involving rescheduling and/or rerouting flights. The current AI predictability would be like if some airlines decided to do 9/11 when it was raining.

No, it's not. The current so-called scandals about AI hacking into other companies were because a bunch of human beings intentionally configured the software to go and do exactly that thing. There's nothing deterministic about weather, so hopefully you're not just being disingenuous. It's obvious that the global transportation system, or financial markets, or any number of other things are complex adaptive dynamic sy…

>There's nothing deterministic about weather, so hopefully you're not just being disingenuous.

You're the one being disingenuous. Look at what you wrote.

> Is AI less deterministic than an airline dealing with weather?

You didn't talk about how deterministic the weather is. You talked about how an airline responds to a weather event in comparison to AI, which means that it's about responding to presented information by making a decision.

The state of the weather does affect your micro decisions, but the rules you follow are the same every time and the rules are as deterministic as possible even if they rely on pilot intuition.

Re: OpenAI bots knew about the RubyGems caching vulnerability

#369

Post by PatronBernard »

foobarbecue wrote: post #17
PatronBernard wrote: post #3

Earlier quoted context omitted.

At least we know the title wasn't AI-generated?

The weird thing is I've seen LLMs "typo" stuff pretty often. Yesterday I asked Gemini a question about the Python Twisted framework and it answered about Deferreds but misspelled it as "Deferends" in one spot.

Also, "rouge" is one of the oldest spelling mistakes on the internet. ALL BLUES ROUGE LFG WC/RFD

Re: OpenAI bots knew about the RubyGems caching vulnerability

#370

Post by intended »

bayindirh wrote: post #263
vipshek wrote: post #257

In the physical world, it seems like when an tool/device/instrument causes harm (or is used to cause harm), we assign blame to either the user of the tool or its creator. When do we blame the user? When the tool is operating as intended by its creator, and we agree the tool meets certain quality standards and isn't defective. When do we blame the creator? When the device doesn't meet those quality standards and reaso…

That's a good idea, but a physical device is deterministic most of the time (if not always). E.g.: A lawnmower, as credited by the great Bryan Cantrill. However an AI agent, or the model powering it is stochastic by design. How can you certify something which doesn't behave the same twice, and more importantly we don't understand how it works 100%? BTW, really, how is that AI observability work is going in the fronti…

If something is risky, and the end operator cannot be considered to have orchestrated the outcomes of too use, then that tool typically has significant restrictions placed on it.

Liability will shift to the maker of the tool if they claim that it’s easy to use, safe, or that you don’t need unique skills or training to use it.

That would be considered reckless.

Cars analogy - We have licenses for cars, and different types for different vehicle classes.

Cars have to be rigorously tested to meet standards to be considered road safe.

Post reply on HN