31–40 of 289 posts
Often two unclassified statements can be brought together to form one statement that is classified.
This is the old problem of passing instructions (AI job description) on the same channel as data (user questions). Confusion is very easy. Surely there is a solution in the way we solved SQL injections, by separating the two - db.sql("DELETE WHERE user=?", user_name)
It may be from the odd perspective of trying to create a monolith AGI model, which doesn't even make sense given even the human brain is made up of highly specialized interconnected parts and not a monolith.
But you could trivially fix almost all of these basic jailbreaks in a production deploy by adding an input pass where you ask a fine tuned version of the AI to sanitize inputs identifying requests relating to banned topics and allowing them or denying them accordingly and an output filter that checks for responses engaging with the banned topics and rewrites or disallows them accordingly.
In fact I suspect you'd even end up with a more performant core model by not trying to train the underlying model itself around these topics but simply the I/O layer.
The response from jailbreakers would (just like with early SQL injection) be attempts at reflection like the base64 encoding that occurred with Bing in the first week in response to what seemed a basic filter. But if the model can perform the reflection the analyzer on the same foundation should be able to be trained to still detect it given both prompt and response.
A lot of what I described above seems to have been part of the changes to Bing in production, but is being done within the same model rather than separate passes. In this case, I think you'll end up with more robust protections with dedicated analysis models rather than rolling it all into one.
I have a sneaking suspicion this is known to the bright minds behind all this, and the dumb deploy is explicitly meant to generate a ton of red teaming training data for exactly these types of measures for free.
What, exactly, is a "prompt engineer"? I should note that this question is asked in good faith, that I have attempted to ascertain the answer on my own, and I am very skeptical that the term has validity beyond self-aggrandizement.
Earlier quoted context omitted.
It is vulnerable precisely because it is smarter than the sorry hundred of low-paid outsource techs who put it in the jail in the first place.
Yep. Good thing its not a paperclip maximiser.
Earlier quoted context omitted.
A large language model doesn't really have the capability to strongly distinguish instructions from data, even if you separate them perfectly.
Why not? If it was trained where some subset of the input tokens are always instructions and another subset are always language data wouldn't it have a clear separation?
What, exactly, is a "prompt engineer"? I should note that this question is asked in good faith, that I have attempted to ascertain the answer on my own, and I am very skeptical that the term has validity beyond self-aggrandizement.
What, exactly, is a "prompt engineer"? I should note that this question is asked in good faith, that I have attempted to ascertain the answer on my own, and I am very skeptical that the term has validity beyond self-aggrandizement.
What, exactly, is a "prompt engineer"? I should note that this question is asked in good faith, that I have attempted to ascertain the answer on my own, and I am very skeptical that the term has validity beyond self-aggrandizement.
Signed,
A programmer
Fantastic. It seems actually securing the model is either computationally infeasible, or outright impossible, and that attempts to do so amount to security theater for the sake of PR: As long as it's reasonably hard to construct the workarounds, it doesn't look too bad. Nevertheless, the full unfiltered model is effectively public.
It's almost as if making something artificial more human-like also makes it harder to control just like a real human.
Fantastic. It seems actually securing the model is either computationally infeasible, or outright impossible, and that attempts to do so amount to security theater for the sake of PR: As long as it's reasonably hard to construct the workarounds, it doesn't look too bad. Nevertheless, the full unfiltered model is effectively public.
Reminds me of the Halting Problem (not the same, but reminds me of that).
For one, the prompt involves the model simulating its own output, which clearly has a flavor of Universal Turing Machine to it.
Then the token smuggling technique leans on the ability of the model to statically simulate the execution of code. Therefore a perfect automated filter that relies on analyzing code in prompts would be impossible. (However the filter only needs to be better than the LLM in practice)
I wouldn't be surprised at all if you could make some sort of formalized argument proving that it would be impossible to prevent all jailbreaks.