321–330 of 475 posts
https://actualplay.roleplayingpublicradio.com/2011/09/genre/...
> “We call it the gorgon-in-a-box problem. There is a gorgon inside the box, and we want to figure out what it is doing. Unfortunately we will turn to stone if we see her face, and she might try to make us see it.“
Earlier quoted context omitted.
It is not possible one of their extraordinarily high paid engineers did not know how to deploy an airgapped environment for the models to run in. Even if somehow true, they also clearly failed to contract specialists like myself to advise them on how to airgap software properly. Models will not break the laws of physics. They simply thought "Running in a VM/Container is easier and probably fine". And the next 1000 es…
Do you think it's possible that one of their research engineers deployed an environment with a locked down network and an allow-list proxy server that had been used many times before within the company and had a zero-day vulnerability that had not been previously discovered? How would you recommend running a coding agent in an environment that could install packages from PyPI but was otherwise unable to interact with…
1. Wouldn’t the model need to know it’s answering benchmark questions, as well as the name of the benchmark, in order for the idea of finding the answer in a database somewhere to even surface? The whole point of benchmarks is to present the question or problem as a standard prompt, not explain that it’s a test called ExploitGym. 2. Nobody was watching it? I don’t mean “babysit the dangerous autocomplete”, I mean to…
Agent loops are you give it a task and it tries its best to finish it.
The only results are success and failure. If it's a success you go through the logs to see the actions it took and if it's a failure you do the same thing.
Why would you look at it in realtime when the whole point of agentic work is to get them to run autonomously as long as possible?
Earlier quoted context omitted.
See u/ verdverm's response, below
yes and see: https://text.npr.org/nx-s1-5844328 and Occam's razor. Russia and China are gonna Russia and China, every issue is going to have a measure of outside influence, thats just how it is now. But them directing large scale operation forces focused on something that is genuinely bad for the people in the area of datacenters (ie, an issue that will take care of itself from within) instead of using their resource…
> Given the absence of guardrails there was nothing to prevent the model from attempting to break out of that sandbox, break into Hugging Face, and read the answers from there instead. I've said this many times before and I'll continue to shout it, but using the term "guardrails" to refer to anything that's either (a) in-context, or (b) a probabilistic classifier (including using other LLMs), is an irresponsible abus…
I'm not skeptical that this attack happened, I'm skeptical that the model's prompt was truly just "solve this benchmark" and nothing more. I'm also trying to figure out why OpenAI put out a press release about this. In what way is this not admitting to a federal crime?
The asymmetry part at the end is the frustrating part to me. I've been using Sol for code review in the last week or two. A couple of times during review it's errored out with the cybersecurity message. So it's found something but won't tell me what it is because I'm not on OpenAI's besties list.
The only scifi I see is absolute stupidity. Even me with my homelab and a slow opensource agent use a completely disconnected setup. No, no proxy. Cached packages but no internet. It is the very first thing I built when I started experimenting with agents. And I'm not a smarty-pants working for the "greatest and best" in silly valley. I really am just a simpleton sysadmin.
Blah blah blah. This feels like a blogpost written only to get other LLMs to quote it considering how many times it orders the reader to resist and to not do something. It's written like a series of commamds.
Earlier quoted context omitted.
I think it's a criminal offence and should be a true test of who is held accountable when an AI agent commits a crime. OpenAI gained access to HuggingFaces production database ffs.
> I think it's a criminal offence and should be a true test of who is held accountable when an AI agent commits a crime. I agree, lets use the favorite analogy. OpenAI encouraged a smart and eager junior engineer to find any way whatsoever to get a higher score on the benchmark. Then, the junior breaks into HuggingFace to get a higher score. That would be a big deal involving the FBI, not press releases and blog post…