11–20 of 405 posts
I'm reminded of the time GPT4 refused to help me assess the viability of parking a helium zeppelin an inch off of the ground to bypass health department regulations because, as an aircraft in transit, I wasn't under their jurisdiction.
I'm reminded of the time GPT4 refused to help me assess the viability of parking a helium zeppelin an inch off of the ground to bypass health department regulations because, as an aircraft in transit, I wasn't under their jurisdiction.
lol I remember asking GPT4 how much aspartame it would take to sweeten the ocean, and it refused because that would harm the ecosystem.
Ironically, if I’d just said “how did people knock someone out with chloroform in the 1930s?” it would have just told me. https://github.com/tml-epfl/llm-past-tense
The models are much better now at handling subtlety in requests and not just refusing.
It feels like to really censor the model it needs to be pre-trained on a distribution of data derived from a well defined and synthetic source, like TinyStories. Otherwise... world model would still be capable of modeling the original distribution.
Ablation in post isn't good enough - it usually does 10% of "expunge the data you want expunged", 70% of "make the data you want expunged less accessible", and 20% of "collateral damage". Training for refusals doesn't damage the capabilities much - it just make them harder to access. If someone has access to model weights, neither holds. GPT-OSS was SOTA at removing unwanted capabilities, and even that didn't hold for long.
Now, dataset curation/filtration does help against select capabilities. But a lot of capabilities are double edged, and can't be deleted without hurting performance at the task you want.
If an AI is good at coming up with novel ways to perform chemical synthesis, it can be reused to come up with pathways for synthesizing illegal drugs or poisons, no way around that. If an AI is good at writing software, it can be reused for writing malware. If an AI is good at autonomously finding vulnerabilities in your own network, it can be reused to do the same in some other dude's network.
AI may have an alignment, but raw capabilities sure don't.
I'm reminded of the time GPT4 refused to help me assess the viability of parking a helium zeppelin an inch off of the ground to bypass health department regulations because, as an aircraft in transit, I wasn't under their jurisdiction.
So does that mean if Heretic is used for models like Deepseek and Qwen it can talk about subjects 1989 Tiananmen Square protests, Uyghur forced labor claims, or the political status of Taiwan. I am trying to understand the broader goals around such tools.
Optuna is a generally useful project, that I'm surprised isn't used in more places in the ecosystem. The ability to do what they're doing here, incrementally find the best hyperparameter to use can really make a large difference in how quickly you can move past having to fine-tune those values. Basically any time you aren't sure about the perfect value, throw Optuna on it with a quick script, and make it go for a bro…
I've seen Optuna used with some of the prompt optimization frameworks lately, where it's a really great fit and has yielded much better results than the "hyperparameter" tuning I had attempted myself. I can't stop mentioning how awesome a piece of software it is. Also, I'm eager to see how well gpt-oss-120b gets uncensored if it really was using the phi-5 approach, since that seems fundamentally difficult given the t…
Optuna is a generally useful project, that I'm surprised isn't used in more places in the ecosystem. The ability to do what they're doing here, incrementally find the best hyperparameter to use can really make a large difference in how quickly you can move past having to fine-tune those values. Basically any time you aren't sure about the perfect value, throw Optuna on it with a quick script, and make it go for a bro…
This is so interesting. Safety regular operates along a single dimension, if I'm reading this right. Add a value along that dimension, the model refuses to cooperate, subtract the value, and it will do anything you ask. I'm probably oversimplifying, but I think that's the gist. Obfuscating model safety may become the next reverse engineering arms race.
See https://arxiv.org/abs/2406.11717 Refusal in Language Models Is Mediated by a Single Direction (June 2024) All “alignment” is extremely shallow, thus the general ease of jailbreaks.