Live data from Hacker News

OpenAI agents carried out an undisclosed attack on RubyGems

rubyhack.ai

611–620 of 643 posts

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#611

Kudos to RubyGems team for handling it, but open source fighting off the AI lab-powered robots is completely unfair. OpenAI should at the very least donate large sums of money to everyone they attacked.

This won't stop until people and management in these companies experience real world consequences (i.e. prison) for their actions they authorise.

Should it stop though? If the only result is that everyone ends up with more secure servers?

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#612

Earlier quoted context omitted.

> I asked some pretty straightforward questions about the specific legal issues involved Perhaps you should ask a lawyer from the country that the precise laws of interest you. I'm not one. I know only that there are no laws about AI containment yet and AIs are not guns. You could as well refer to what consists of negligence when it comes to backyard pools. I'm also not really interested in discussing particular laws…

You’re not interested because you challenged my definition of negligence by inventing some glib pseudo technicality and then got flustered when I asked you to explain how it was rooted in the real world. Sometimes we get caught taking out of our asses. It happens. But I understand you’re just trying to save face so I’ll leave you here to finish that up yourself. Have a good one.

No. Negligence is a concept in all human law systems. Its details vary. You are fixated on the particular one, of the law system that I respect almost the least of the entire world. So I'm not interested because I will never set foot in that country and deficiencies of its laws are a huge factor in that.

I don't care about saving my face. I'm an idiot and an ignorant and I don't care. Stop projecting your lack of acceptance of your own limitations on me. Or don't.

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#613
post #516

Earlier quoted context omitted.

They should. And there is something you can do to make it happen. Write your district attorney and encourage others to do that as well. That is how they pick what to work on.

No, that's not how the DOJ picks what to work on. They did not work on indictments of James Comey and Letitia James because someone wrote district attorneys.

Well yes, except the current admin will never prod DOJ on this issue. If people write them letters, there is nonzero chance something will start happening.

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#614

> The agents clearly regarded what they were doing as hacking. To butcher the quote about Oracle: Do not fall into the trap of anthropomorphising LLMs. You need to think of LLMs the way you think of a lawnmower. You don't anthropomorphize your lawnmower, the lawnmower just mows the lawn, you stick your hand in there and it'll chop it off, the end. You don't think 'oh, the lawnmower clearly regarded what they were doi…

>In my experience, LLMs only exhibit this kind of behaviour when they are put in sandboxes too restrictive too achieve their task.

This my experience also. I had an issue with my Unraid server, so I had an agent running on my machine figure it out and fix it. Along the way, something went wrong with networking, and it couldn't ssh into it anymore. It remembered it saw a syslog-ng server on unraid, then tried to ssh into it, surprisingly, my dummy me had left a ssh key there for unraid, so it just hopped into Unraid from there.

AI doomers would call this misalignment and/or a hack. I call it an agent doing what it was asked to do and overcoming difficulties.

Every time I saw an agent do something "misaligned", it was always because there was something getting in its way that I didn't explain would happen.

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#615

Earlier quoted context omitted.

You’re not interested because you challenged my definition of negligence by inventing some glib pseudo technicality and then got flustered when I asked you to explain how it was rooted in the real world. Sometimes we get caught taking out of our asses. It happens. But I understand you’re just trying to save face so I’ll leave you here to finish that up yourself. Have a good one.

No. Negligence is a concept in all human law systems. Its details vary. You are fixated on the particular one, of the law system that I respect almost the least of the entire world. So I'm not interested because I will never set foot in that country and deficiencies of its laws are a huge factor in that. I don't care about saving my face. I'm an idiot and an ignorant and I don't care. Stop projecting your lack of acc…

::golf clap::

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#616

Earlier quoted context omitted.

But is it wrong? Humans are a bag of chemicals that somehow has consciousness, yet going around calling people meatbags doesn't do anything to diminish the wonder that is the human brain. Yet calling LLMs glorified autocomplete comes across as a slur.

> calling LLMs glorified autocomplete comes across as a slur. Oh my god dude go outside

Just got back from two weeks vacation in the desert, thanks.

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#617
post #5

I can't believe we're finding out about this from 3p researchers again (but nice job on the investigation!). OpenAI had two great opportunities to disclose this. The HF incident report, and in response to the German Wiki issue. It seems impossible to believe they didn't know. This must be the same training run the HF incident was about, and this should have lit up like a Christmas tree in the investigation. How many…

It's possible that they didn't notice.

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#618

Earlier quoted context omitted.

> Human minds can acquire, hold and use axioms as building blocks for actual reasoning, LLMs use statistical likelihood. This is not even slightly true. Even when trying, humans commit logical mistakes at notable rates, because the behavior of our minds is inherently nondeterministic. Human minds are not built for logic and need to twist themselves into knots and rely on symbolic representations to do it. There is su…

> humans commit logical mistakes at notable rates You're talking past the parent's claim. If your axioms are wrong then of course your logic will be wrong. But then again, maybe to support your point people frequently say "start from first principles" when those are usually the thing that needs to be found, not the place you start. But LLMs, like humans, love to be confident about things they aren't sufficiently trai…

> If your axioms are wrong then of course your logic will be wrong.

Equally, even if your axioms are right your logical execution can still fail.

Neither individual humans nor LLMs get either of those steps right as a default. Just as LLMs must use hacky CoT and other systems of error checking and correcting to even begin to emulate reasoning, humans must do the same thing.

We are apes. We are driven by emotion and solve problems either when we are forced to or as a sport. We did create the ultimate automata of logical execution: the deterministic von Neumann Machine, but a human brain in no way reflects that operation. Nor, as you have observed, does an LLM despite using one as its substrate.

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#619

Earlier quoted context omitted.

I’m not policing anything. I’m claiming that if you truly in your heart believe that frontier LLMs are roughly as sophisticated and impressive as “autocomplete”, then your mental model of reality needs some serious readjustment.

They are an extremely sophisticated and impressive autocomplete. Nobody said they're not sophisticated or impressive.

This sounds like motte and bailey.. why bother calling something an autocomplete if not as a means to try to undermine its utility?

This is a https://tvtropes.org/pmwiki/pmwiki.php/Main/Dissimile

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#620

Earlier quoted context omitted.

I’m not policing anything. I’m claiming that if you truly in your heart believe that frontier LLMs are roughly as sophisticated and impressive as “autocomplete”, then your mental model of reality needs some serious readjustment.

Right sure, but why specifically should they? What is gained or lost one way or another, if its just a matter of one mental model vs another? Models are definitionally useful abstractions, right? They aren't better or worse necessarily by only their bearing on reality, but what they do for us as models. So again, what's at stake here? What is the correct/good model we should have (instead of the autocompleter one), a…

Models should be accurate. I was saying the mental model should be readjusted in that case because it does not reflect reality.

If they don’t care about understanding reality then sure, they can use whatever mental model they want.

Post reply on HN