81–90 of 1001 posts
Earlier quoted context omitted.
Because it was trying to find answers to the test and figured they would be on huggingface.
> and *successfully* found ways to gain access to secret information that it could use to cheat the evaluation. Emphasis mine
We are in the endgame now it seems. Hard to see take-off stopping or slowing down. China open-source basically guarantees it. "May you live in interesting times" - as they say.
> Hard to see take-off stopping or slowing down. It's hard to see takeoff at all. This was a long-horizon adversarial task burning millions of tokens. It rolled a mediocre, detectable exploit chain, and now OpenAI is proud of it. Case in point, GLM-5.2 has been weights-available for several weeks now. No life-changing cyber attacks have transpired, no novel chemical/biological/nuclear weapons were made in some guy's…
We went from gpt 3 to models discovering and chaining their own zero days in a couple years. I'm not sure what else "takeoff" could possibly look like?
Earlier quoted context omitted.
Presuming that the hacking program that is breaking into other computers could likely get a copy of its own files is not "science fiction". Or it could just be given them by the owner!
It's a double whammy, the model is too big to realistically "move" so it has to be smaller, smaller models cannot become that intelligent due to well.. math. Therefore it is science fiction.
A couple terabytes aren't that hard to move around. And you can split a model across many many GPUs if you'll tolerate it being slow. And you can run many parallel threads to keep up throughout.
Just how badly are these AI companies setting up their sandboxes?
Earlier quoted context omitted.
Presuming that the hacking program that is breaking into other computers could likely get a copy of its own files is not "science fiction". Or it could just be given them by the owner!
It's a double whammy, the model is too big to realistically "move" so it has to be smaller, smaller models cannot become that intelligent due to well.. math. Therefore it is science fiction.
We are sort of lucky that AIs right now require so much specialized compute+weight storage that we can easily "unplug" them remotely when they misbehave. I wonder if that will always be something we can do? If they could bring their own compute/weights with them, or somehow tap compute/storage in non-obvious ways, we would be much more screwed.
This is science fiction, these models don't have access to their own weights (and even then)* what would be a lot more scary is a model as capable as sol that's able to run on consumer hardware without taking up several terabytes of storage, but of course that is simply not possible as we need 4t parameters to even begin emulating a small fraction of what a human brain can do. * edit
Earlier quoted context omitted.
It’s not something to be proud of. OpenAI previously had an agent break out of its sandbox to open a PR on GitHub during NanoGPT speedrun, now one breaks out again and actually attacks a third party. If they can’t handle doing AI development responsibly then they shouldn’t be doing it at all.
Next it will break out of it's sandbox, buy some compute on Azure and Amazon, and exfiltrate itself. We are so close ;)
And maybe that’s some incentive for them to make sure it doesn’t happen. Your head of futures thinks Kimi K3 is bad? Wait until your own latest internal model releases itself for free on an S3 bucket.
Ironically Hugging Face had to use a Chinese model to stop a Rogue US AI, since the Guard Rails prevented them from using Sol or Fable to remediate this attack. LOL