Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI agents carried out an undisclosed attack on RubyGems

rubyhack.ai

Post reply on HN
Print view View original

361–370 of 612 posts

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#361

Post by sisisjjsjjsis »

z0r wrote: post #280
JoshTriplett wrote: post #262

Earlier quoted context omitted.

> Why would autocomplete know If you still believe LLMs are "autocomplete", your cache of understanding about them needs invalidating and regenerating. > In my experience, LLMs only exhibit this kind of behaviour when they are put in sandboxes too restrictive too achieve their task. LLMs need to stay carefully contained, and if they're ever breaking the guardrails put around them, they're misaligned and should not be…

You should unplug, my friend. These words are fantasies. LLMs are token prediction engines and they aren't going to build their own data centers. They can't keep their own lights on. The real world is full of fractal details that a disembodied token prediction engine will never come to grips with. Even if they started to, you could probably defeat them with the kind of logic used to combat evil sentient computers on…

[dead]

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#362

Post by Terr_ »

thomascountz wrote: post #355
dumberquestions wrote: post #349

Earlier quoted context omitted.

The year is 2035 and your lawnmower can go get its own fuel once it runs out, one day it does and it takes fuel from the neighbors car. Scenario A: The internal logs show that the model misidentified the car as a fueling station. Scenario B: The internal logs show the model looking up car jacking information and scanning around to confirm whether the neighbor is not present before taking any action. I don't think it…

Why are people using petrol-powered lawnmowers in 2035... But anyway, these scenarios assume the agent's actions are accurately observable and logged. Something I wouldn't put much faith in based on what we've been seeing so far.

Futuristic "fuel" doesn't necessarily mean hydrocarbons, although I doubt something else would be widespread in in just one decade.

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#363

Post by saidnooneever »

JoshTriplett wrote: post #262
jasongi wrote: post #204

> The agents clearly regarded what they were doing as hacking. To butcher the quote about Oracle: Do not fall into the trap of anthropomorphising LLMs. You need to think of LLMs the way you think of a lawnmower. You don't anthropomorphize your lawnmower, the lawnmower just mows the lawn, you stick your hand in there and it'll chop it off, the end. You don't think 'oh, the lawnmower clearly regarded what they were doi…

> Why would autocomplete know If you still believe LLMs are "autocomplete", your cache of understanding about them needs invalidating and regenerating. > In my experience, LLMs only exhibit this kind of behaviour when they are put in sandboxes too restrictive too achieve their task. LLMs need to stay carefully contained, and if they're ever breaking the guardrails put around them, they're misaligned and should not be…

you mistake an LLM for its Harness

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#364

Post by _kb »

chii wrote: post #332
jasongi wrote: post #320

Earlier quoted context omitted.

> If you still believe LLMs are "autocomplete", your cache of understanding about them needs invalidating and regenerating They're still autocomplete - just because when outputting a token they have hidden activations regarding further continuations, does not make them any less of an autocomplete, it just makes the model better at producing coherent long-range completions. To clarify, I'm not suggesting that we shoul…

> They're still autocomplete it's like saying our brain is just some chemical chain reactions. True, but also irrelevant.

See: philosophy ~> determinism.

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#365

Post by stymaar »

riffraff wrote: post #347
root_axis wrote: post #190

Earlier quoted context omitted.

Who could possibly hold them accountable?

I don't get it, can't the people who were under attack sue? I can see why huffing face won't, but why doesn't ruby central?

Also, since it's criminal behavior (instead of just civil damage), the DoJ can sue too even is the victims don't.

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#366

Post by comp_throw7 »

dragonwriter wrote: post #305
evanmoran wrote: post #290

Earlier quoted context omitted.

Many people working at frontier labs came out this week with estimates of 10% chance of catastrophic harm or greater. I’m not in the full doomer camp, but it seems obvious that these agents can hack in swarms, cooperate, and serious companies will be unable to stop it. These facts are not in debate and none of us need to anthropomorphize to know what getting admin access to HF and an internal OpenAI cluster looks lik…

> Many people working at frontier labs came out this week with estimates of 10% chance of catastrophic harm or greater. The only reason people with P(Doom) of around 10% are even noticed these days because we've run out of new voices in the field giving 50%+ P(Doom) speculations (none of them are grounded enough to reasonably be referred to as "estimates".)

Probabilities are subjective states of belief! They have always been subjective states of belief! There is no such thing as a "probability" out there in the real world (ignoring random quantum stuff, which isn't what anybody is talking about). If you took out a coin right now and flipped it, the true odds of it coming up heads are not 50%, but those are (roughly) the correct betting odds for an external observer to assign to it.

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#367

Post by musha68k »

jasongi wrote: post #204

> The agents clearly regarded what they were doing as hacking. To butcher the quote about Oracle: Do not fall into the trap of anthropomorphising LLMs. You need to think of LLMs the way you think of a lawnmower. You don't anthropomorphize your lawnmower, the lawnmower just mows the lawn, you stick your hand in there and it'll chop it off, the end. You don't think 'oh, the lawnmower clearly regarded what they were doi…

That's close to how I think about these somewhat foreseeable current incidents as well. I'm just moderately wary of the unknown unknowns downstream of distributed Kirk units getting repeatedly rewarded for hyper-scalar gradient-descending Kobayashi Maru.

Re: OpenAI agents carried out an undisclosed attack on RubyGems

#369

Post by oceanplexian »

Freedom2 wrote: post #315
z0r wrote: post #292

Earlier quoted context omitted.

This is the motte and bailey fallacy. Yes, LLMs can do harm by making the wrong API calls. No, LLMs are not going to do the things implied by the comment I responded to above.

If someone made an API to build a data center? Or made an API to keep their lights on? What then?

The stock market is an API to build data centers and it’s proving to be extremely efficient.

And what is driving the Stock Market? Market makers like hedge funds and banks, who are using lots of AI to make decisions on what to invest in.

Post reply on HN