The Dual LLM pattern for building AI assistants that can resist prompt injection
1–10 of 112 posts
Re: The Dual LLM pattern for building AI assistants that can resist prompt injection
#2This is avoiding the core problem (mingling control and data) with security through obscurity.
That can be an effective solution, but it's important to recognize it as such.
Re: The Dual LLM pattern for building AI assistants that can resist prompt injection
#3I still don't believe that in the long term it will be tenable to bootstrap LLMs using prompts (or at least via the same vector as your users).
Re: The Dual LLM pattern for building AI assistants that can resist prompt injection
#4Is it possible that all but the most exotic prompt injection attacks end up being mitigated automatically over time, by virtue of research and discussion on prompt injection being included in training sets for future models?
Re: The Dual LLM pattern for building AI assistants that can resist prompt injection
#5Is it possible that all but the most exotic prompt injection attacks end up being mitigated automatically over time, by virtue of research and discussion on prompt injection being included in training sets for future models?
By the same logic, humans should no longer fall for phishing scams or buy timeshares since information about them is widely available.
Re: The Dual LLM pattern for building AI assistants that can resist prompt injection
#6It feels like an LLM classifying the prompts without cumulative context as well as the prompt output from the LLM would be pretty effective. Like in the human mind, with its varying levels of judgement and thought, it may be a case of multiple LLMs watching the overall process.
Re: The Dual LLM pattern for building AI assistants that can resist prompt injection
#7Is it possible that all but the most exotic prompt injection attacks end up being mitigated automatically over time, by virtue of research and discussion on prompt injection being included in training sets for future models?
By the same logic, humans should no longer fall for phishing scams or buy timeshares since information about them is widely available.
Most well-educated people won't. A well trained AI can behave pretty close to a well-educated person in common sense.
Re: The Dual LLM pattern for building AI assistants that can resist prompt injection
#8Is it possible that all but the most exotic prompt injection attacks end up being mitigated automatically over time, by virtue of research and discussion on prompt injection being included in training sets for future models?
By the same logic, humans should no longer fall for phishing scams or buy timeshares since information about them is widely available.
I’d say it’s not the same thing, because most humans don’t have an encyclopedic knowledge of past scams, and are not primed to watch out for them 24/7. LLMs don’t have either of these problems.
An interesting question is whether GPT-4 would fall for a phishing scam or try to buy a timeshare if you gave it an explicit instruction to avoid being scammed.
Re: The Dual LLM pattern for building AI assistants that can resist prompt injection
#9Thanks SimonW! I've really enjoyed your series on this problem on HN and on your blog. I've seen suggestions elsewhere about tokenising fixed prompt instructions differently to user input to distinguish them internally, and wanted to ask for your take on this concept- do you think this is likely to improve the state of play regarding prompt injection, applied either to a one-LLM or two-LLM setup?
Re: The Dual LLM pattern for building AI assistants that can resist prompt injection
#10Here's "jailbreak detection", in the NeMo-Guardrails project from Nvidia:
https://github.com/NVIDIA/NeMo-Guardrails/blob/327da8a42d5f8...
I.e. they ask the llm if the prompt will break the llm. (I believe that more data /some evaluation on how well this performs is intended to be released. Probably fair to call this stuff "not battle tested".)