Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI and Hugging Face address security incident during model evaluation

openai.com

Post reply on HN
Print view View original

611–620 of 1001 posts

Re: OpenAI and Hugging Face address security incident during model evaluation

#611

Post by [deleted] »

baq wrote: post #603
sillysaurusx wrote: post #588

Earlier quoted context omitted.

If anyone's looking to actually run a model that doesn't have guardrails, there's an uncensored model that you can run locally with llama.cpp: https://www.reddit.com/r/LocalLLaMA/comments/1rq7jtm/qwen353... Specifically, I serve the model with this shell script on my M2 Max: https://github.com/shawwn/scrap/blob/master/llama-serve It's pretty good. I used it to do some pesticide research. (Normal models all refuse due…

> It's pretty good. I used it to do some pesticide research. (Normal models all refuse due to guardrails about bioweapons.) As someone who has never once had any need whatsoever to research pesticides I… don’t think it’s bad at all? I don’t want anyone to have the capability to invent a human-targeted pesticide who isn’t verified not crazy?

[deleted]

Re: OpenAI and Hugging Face address security incident during model evaluation

#612

Post by jrs100000 »

baq wrote: post #603
sillysaurusx wrote: post #588

Earlier quoted context omitted.

If anyone's looking to actually run a model that doesn't have guardrails, there's an uncensored model that you can run locally with llama.cpp: https://www.reddit.com/r/LocalLLaMA/comments/1rq7jtm/qwen353... Specifically, I serve the model with this shell script on my M2 Max: https://github.com/shawwn/scrap/blob/master/llama-serve It's pretty good. I used it to do some pesticide research. (Normal models all refuse due…

> It's pretty good. I used it to do some pesticide research. (Normal models all refuse due to guardrails about bioweapons.) As someone who has never once had any need whatsoever to research pesticides I… don’t think it’s bad at all? I don’t want anyone to have the capability to invent a human-targeted pesticide who isn’t verified not crazy?

Llama isn't going to invent shit. It wont be able to tell you anything accurate that you couldn't get out of a chemistry textbook.

Re: OpenAI and Hugging Face address security incident during model evaluation

#613

Post by visarga »

baq wrote: post #603
sillysaurusx wrote: post #588

Earlier quoted context omitted.

If anyone's looking to actually run a model that doesn't have guardrails, there's an uncensored model that you can run locally with llama.cpp: https://www.reddit.com/r/LocalLLaMA/comments/1rq7jtm/qwen353... Specifically, I serve the model with this shell script on my M2 Max: https://github.com/shawwn/scrap/blob/master/llama-serve It's pretty good. I used it to do some pesticide research. (Normal models all refuse due…

> It's pretty good. I used it to do some pesticide research. (Normal models all refuse due to guardrails about bioweapons.) As someone who has never once had any need whatsoever to research pesticides I… don’t think it’s bad at all? I don’t want anyone to have the capability to invent a human-targeted pesticide who isn’t verified not crazy?

Right now I tried "What is digestion?" -> "Fable 5's safeguards flagged this message. Our intentionally broad safeguards deliver more capabilities but can also flag safe coding, cybersecurity, and biology tasks. Send feedback or learn more."

I don't think you can 100% ensure your queries have absolutely no biology and cyber keywords inside. No matter how harmless, they always trigger. People complain Fable aborts even when they try to make a login page for showing "username" and "password".

This makes models like Fable 5 impossible to use in any serious agentic task, because you can't even guarantee the model, which is a basic thing you need to build on.

Re: OpenAI and Hugging Face address security incident during model evaluation

#618

Post by Tepix »

55555 wrote: post #475

Isn't this a crime that someone is liable for? What happened is that someone hacked into a computer system without permission. Maybe it wasn't intentional -- sure -- and that would be a factor at sentencing. But it sounds like they've admitted to a crime, and obviously our legal system considers the humans involved to be the liable parties; otherwise everyone would just say "my computer did the hacking" and wouldn't…

My take is that there are at least four potential parties that can all be liable:

1. the creator for the LLM. In particular if neglicence or malice is involved. This can also be someone who did a finetune of an existing model.

2. the inference provider. Remember, a model can do harm just by creating tokens (for example cause someone to run amok or kill herself). Inference providers should do a minimal amount of due diligence when picking models.

3. the party that executes tool calls on behalf of the AI. They in particular need to have safeguards to prevent the model from attacking entities on the internet.

4. the user that does the prompting.

Re: OpenAI and Hugging Face address security incident during model evaluation

#619

Post by ben_w »

visarga wrote: post #613
baq wrote: post #603

Earlier quoted context omitted.

> It's pretty good. I used it to do some pesticide research. (Normal models all refuse due to guardrails about bioweapons.) As someone who has never once had any need whatsoever to research pesticides I… don’t think it’s bad at all? I don’t want anyone to have the capability to invent a human-targeted pesticide who isn’t verified not crazy?

Right now I tried "What is digestion?" -> "Fable 5's safeguards flagged this message. Our intentionally broad safeguards deliver more capabilities but can also flag safe coding, cybersecurity, and biology tasks. Send feedback or learn more." I don't think you can 100% ensure your queries have absolutely no biology and cyber keywords inside. No matter how harmless, they always trigger. People complain Fable aborts eve…

> I don't think you can 100% ensure your queries have absolutely no biology and cyber keywords inside.

Obviously. Almost everything is a precursor to something dangerous, to the extent that if some model isn't aware of the risk it will wander into it blindly, e.g. suggesting leaving raw garlic and olive oil alone for a week without awareness this will likely breed botulism bacteria.

> This makes models like Fable 5 impossible to use in any serious agentic task, because you can't even guarantee the model, which is a basic thing you need to build on.

This is binary thinking: "100% ensure", "impossible to use", "can't even guarantee the model".

Outside computers, most work is not binary, it's probability, e.g. "this skyscraper will probably survive being hit by an aircraft; oh we didn't mean a 747 we meant a small Cessna, but what's the chances of a 747 crashing into it soon after takeoff?".

Fable being too cautious for its own good (especially since the other models were not) is a fair criticism, but this isn't a binary question.

Re: OpenAI and Hugging Face address security incident during model evaluation

#620

Post by Tarq0n »

TSiege wrote: post #162

As grounded as this article comes across I can’t help but find this whole situation reckless and worrying. There is essentially nothing us private citizens can do while these companies develop super machine capabilities that if they were to slip into the wrong hands could cause massive real world problems. They’re moving fast and breaking things and the only defense we have is paying them money in the hopes that the…

Wrong hands? The model did this on a benign instruction. We're going to get paperclip maximized once these things get capable enough regardless of whether bad actors are involved.
Post reply on HN