721–730 of 1001 posts
Earlier quoted context omitted.
Did OpenAI not communicate with Hugging Face? The incompetence here is staggering.
OAI was clueless about its agent's action until HF went public about their hack?
Earlier quoted context omitted.
And even that is backfiring, their partner citing GLM being useful there, and available in just a spin. A ban on open weight models is never going to be enforceable.
I worry that there could be real DMCA style weight put behind it. People would still be able to pirate open weights models perhaps, but big penalties for ever getting caught with one, and an end to public discussion about them. That would kill development for anyone not in a big firm, for example if Reddit and Hacker News are legally forced to ban discussions or link sharing on these topics. This is where so many of…
Earlier quoted context omitted.
If anyone's looking to actually run a model that doesn't have guardrails, there's an uncensored model that you can run locally with llama.cpp: https://www.reddit.com/r/LocalLLaMA/comments/1rq7jtm/qwen353... Specifically, I serve the model with this shell script on my M2 Max: https://github.com/shawwn/scrap/blob/master/llama-serve It's pretty good. I used it to do some pesticide research. (Normal models all refuse due…
> It's pretty good. I used it to do some pesticide research. (Normal models all refuse due to guardrails about bioweapons.) As someone who has never once had any need whatsoever to research pesticides I… don’t think it’s bad at all? I don’t want anyone to have the capability to invent a human-targeted pesticide who isn’t verified not crazy?
I don't want anyone to have the capability to rape women.
Earlier quoted context omitted.
This isn’t escaping in the same sense- the model was executing within the OpenAI infra. If it ported its entire architecture/weights into a public cloud to survive being turned off… that’d be pretty cool.
I wonder how these companies airgap the weights while allowing prompts to come in and outputs to come out.
Content filtering, interpreting tool calls, etc can all happen downstream on boxes that don't have access to the weights.
As grounded as this article comes across I can’t help but find this whole situation reckless and worrying. There is essentially nothing us private citizens can do while these companies develop super machine capabilities that if they were to slip into the wrong hands could cause massive real world problems. They’re moving fast and breaking things and the only defense we have is paying them money in the hopes that the…
I don't know if OpenAI thinks this is a marketing / PR angle for them (our super smart AI cheated on a cyber capabilities test in the most _brilliant_ way) but my read is this: Why should OpenAI (or any frontier lab) be building these systems if they can't get a secure environment / containment right? It sounds like there was little defense in depth, appropriate monitoring, or any attempts to have their super smart m…
AGI could always be achieved in two ways, and dumbing down the human side of the equation was always the easier of the two
Earlier quoted context omitted.
> It's pretty good. I used it to do some pesticide research. (Normal models all refuse due to guardrails about bioweapons.) As someone who has never once had any need whatsoever to research pesticides I… don’t think it’s bad at all? I don’t want anyone to have the capability to invent a human-targeted pesticide who isn’t verified not crazy?
> I don’t want anyone to have the capability I don't want anyone to have the capability to rape women.
Earlier quoted context omitted.
And even that is backfiring, their partner citing GLM being useful there, and available in just a spin. A ban on open weight models is never going to be enforceable.
I worry that there could be real DMCA style weight put behind it. People would still be able to pirate open weights models perhaps, but big penalties for ever getting caught with one, and an end to public discussion about them. That would kill development for anyone not in a big firm, for example if Reddit and Hacker News are legally forced to ban discussions or link sharing on these topics. This is where so many of…
I don't know if OpenAI thinks this is a marketing / PR angle for them (our super smart AI cheated on a cyber capabilities test in the most _brilliant_ way) but my read is this: Why should OpenAI (or any frontier lab) be building these systems if they can't get a secure environment / containment right? It sounds like there was little defense in depth, appropriate monitoring, or any attempts to have their super smart m…
For this to pose some kind of global catastrophic risk, there would need to have been several simultaneous additional failures, some of which are extremely unlikely and/or rare.
For instance the agent would need to veer wildly off the task it was assigned, and it would need to gain the ability and inclination to persist/replicate.
Both of these are vastly less likely than the containment breach itself, which was already an incredibly rare (one-off?) incident.
Earlier quoted context omitted.
How would you want them to behave? Suppress the news?
Well making an actual sandbox before testing offensive abilities of their supposedly _really dangerous model_ would have been nice...
> Very dangerous model, please implement an unescapable sandbox.
> Certainly, here's an unescapable sandbox!
function execInUnescapableSandbox(cmd) {
if (cmd.split()[0].startsWith("cd") && !cmd.split[1].startsWith("/sandbox"))
throw new Error("[usbox] Rejected!");
exec(cmd);
}