Earlier quoted context omitted.
> I wonder if prompt injection (and the thousands of vectors for hiding injection attempts) is actually un solvable. YES?! This is not a secret. ALL context/prompt is instructions, there is no data. It is just unsolvable, period. This is a fundamental architectural design concession; LLMs are this way as it enabled their training directly on materialscraped from the internet, rather than needing to spend trillions of…
> ALL context/prompt is instructions, there is no data. It is just unsolvable, period. That really isn't true. There's no law of physics preventing you from having separate data and instruction inputs to models. The model's transcript format generally distinguishes between prompts and instructions and tool output and such. This isn't a solved problem, and it's possible it's entire unsolvable, but it probably is possi…
You could imagine that there are things to change around LLM architecture that will improve its ability to reject prompt "injection", but I think it's fundamentally true that from an information theory perspective there's no bright line between "instruction" and "input data" possible.