It very much depends on what kind of system you are building, as each of them have different input/output modalities, each combination of them allowing for different attacks. If you are building a generic agent that can theoretically connect to anything and should build things end-to-end without interventions, then yes, it's very intractable to defend against prompt injection.
In more narrow cases, like Chat UIs it becomes a lot easier, though if it should appeal to a generic audience, still easy for individual users to misconfigure.
And if you want to use it in the most high-security environments where nothing can leak in/out, you will have to air-gap the system anyways (like any traditional software).
> You're perhaps making it harder to craft a prompt injection, but not impossible. This is a false sense of security.
It's not a false sense security, it's part of a layered security strategy. Yes, it will never be impossible, but so are many individual steps in cybersecurity attacks.
There are other systems (like email) that are essentially impossible to fully lock down with purely mechanical security measures if you want to allow for meaningful work (e.g. having email attachments).
A second-judge LLM when paired with keyword/pattern blocklists, and active alterting/lockout after repeated attack attempts can form a very robust line of defense that in practice can be near-impossible to break.
For many attacks, to have actual exploitability, you also need to have compromised a peripheral system (or user account) to have repeated attempts at circumventing prompt injection measures.
> And, are you absolutely sure you'll never forget to lock each and every thing down, including things you weren't aware of?
That's part of every normal (non-LLM) security audit. If you don't know what data can potentially go where, then you are open in attacks in any system. The AI space does add a bit of complexity here, if using MCPs hosted with third parties, though.