Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI agents carried out an undisclosed attack on RubyGems

rubyhack.ai

Post reply on HN
Print view View original

621–630 of 643 posts

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#621

Post by daishi55 »

godelski wrote: post #587
daishi55 wrote: post #525

Earlier quoted context omitted.

I’m not policing anything. I’m claiming that if you truly in your heart believe that frontier LLMs are roughly as sophisticated and impressive as “autocomplete”, then your mental model of reality needs some serious readjustment.

I don't think anyone is using "autocomplete" in the way your phone does it. But it is shorthand for a much more sophisticated version that is built in similar ideas. And autoregressive models certainly have that in their core structure. But people are lazy and neither want to say a lot when few words work nor will they read long comments, even if more accurate I want to clarify, most people are using "autocomplete" s…

> Sure, there is an autocomplete aspect, but it's not the core nor anywhere near the full story.

This can apply equally to LLMs or humans. So what is the utility of the word if it just describes the whole range of things we are talking about?

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#622

Post by nickcosta »

jsnell wrote: post #5

I can't believe we're finding out about this from 3p researchers again (but nice job on the investigation!). OpenAI had two great opportunities to disclose this. The HF incident report, and in response to the German Wiki issue. It seems impossible to believe they didn't know. This must be the same training run the HF incident was about, and this should have lit up like a Christmas tree in the investigation. How many…

[flagged]

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#623

Post by tripzilch »

apsec112 wrote: post #56
hgoel wrote: post #19

I wonder how much of this is intentional "incompetence" so they can justify the most recent campaign to build a regulatory moat against competition. The repeated refusals to disclose until caught certainly seem malicious, yet at the same time the boasting about their capabilities is also at an all time high.

Intentionally doing this kind of hack would be a serious felony. I don't think it's plausible that the leaders of a major business would: - commit serious felonies - in order to deliberately trigger an investigation against themselves - which - since, in this scenario, they know their company would be investigated - might send them to jail - while at the same time spending tens of millions of dollars on the Leading t…

> Intentionally doing this kind of hack would be a serious felony. I don't think it's plausible that the leaders of a major business would (...)

Fair enough, I can see that line of reasoning. But if they really didn't want it, it really doesn't show from the security of their training environment setup.

And yes I know they used zero days and did all sorts of complicated unexpected stuff, but that doesn't take away that there were also some huge gaps and severe lack of oversight.

I can see the argument they wouldn't want to commit a felony intentionally, but I also don't see them trying very hard to not commit a felony unintentionally. If you get what I mean.

I guess I also agree that it just makes no sense on any level, regardless of what I think of OpenAI -- but that also makes me not wanna trust them very much.

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#624

Post by lukeify »

scotty79 wrote: post #611
lukeify wrote: post #122

Earlier quoted context omitted.

This won't stop until people and management in these companies experience real world consequences (i.e. prison) for their actions they authorise.

Should it stop though? If the only result is that everyone ends up with more secure servers?

"if" is doing a lot of heavy lifting there.

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#626

Post by godelski »

daishi55 wrote: post #621
godelski wrote: post #587

Earlier quoted context omitted.

I don't think anyone is using "autocomplete" in the way your phone does it. But it is shorthand for a much more sophisticated version that is built in similar ideas. And autoregressive models certainly have that in their core structure. But people are lazy and neither want to say a lot when few words work nor will they read long comments, even if more accurate I want to clarify, most people are using "autocomplete" s…

> Sure, there is an autocomplete aspect, but it's not the core nor anywhere near the full story. This can apply equally to LLMs or humans. So what is the utility of the word if it just describes the whole range of things we are talking about?

Because the second part of my comment.

If you over simplify everything then everything is the same. You have to have nuance. So don't just argue for the sake of arguing. Take that same passion to deeply understand these systems. Take that same passion to try to understand yourself and those around you

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#627

Post by ygouzerh »

oceansky wrote: post #127
jsnell wrote: post #5

I can't believe we're finding out about this from 3p researchers again (but nice job on the investigation!). OpenAI had two great opportunities to disclose this. The HF incident report, and in response to the German Wiki issue. It seems impossible to believe they didn't know. This must be the same training run the HF incident was about, and this should have lit up like a Christmas tree in the investigation. How many…

Also, why there's no accountability? Even if there's no intent, it's still a cyber attack.

It's crazy indeed. When working in company as DevSecOps, if we try to attempt pentest on prod without warning anyone, we will just get fired on the spot. AI Labs seems to have a free pass...

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#628

Post by ygouzerh »

sensanaty wrote: post #441
yalogin wrote: post #408

Why is OpenAI getting away with this crap? They are clearly failing to control their code. If someone did this pre-AI or even ran the exact same set up as openAI did and hacked another site, they would be in jail. OpenAI is not even issuing an apology, they are happily blaming AI and weirdly using this to tout their progress even.

If you dare slow the progress of AI, you crash the entire world economy because the only thing investors are putting money into are these cash burning machines. They've got an IPO to hype, who has the time to care about useless things like accountability?

Maybe a crash is what we need to reset the balance. We are spending a lot of money building shovels, but not enough to invest on the company that will use what will be mined.

If the money could flow back to the real economy instead of the AI economy, it might be healthier for the whole system.

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#629

Post by queenkjuul »

HappMacDonald wrote: post #619
queenkjuul wrote: post #553

Earlier quoted context omitted.

They are an extremely sophisticated and impressive autocomplete. Nobody said they're not sophisticated or impressive.

This sounds like motte and bailey.. why bother calling something an autocomplete if not as a means to try to undermine its utility? This is a https://tvtropes.org/pmwiki/pmwiki.php/Main/Dissimile

Nobody said autocomplete lacks utility...
Post reply on HN