Every year is apparently a good year for developers to figure out how terminals have worked for decades? Just… don’t trust inputs you don’t fully control, there’s nothing else to it.
[dead]
ANSI escape injection in MCP servers: Hidden from humans, visible to AI
21–30 of 46 posts
Re: ANSI escape injection in MCP servers: Hidden from humans, visible to AI
#22Every year is apparently a good year for developers to figure out how terminals have worked for decades? Just… don’t trust inputs you don’t fully control, there’s nothing else to it.
> Just… don’t trust inputs you don’t fully control, there’s nothing else to it. This is easier said than done with LLMs. By design there is no separation between control & data channels in LLMs. Everything is context. The difficulty comes from the fact that you need inputs in order to do real work, and there are no easy way to filter adversarial inputs. There is no meaningful way to distinguish between "before runnin…
> The difficulty comes from the fact that you need inputs in order to do real work, and there are no easy way to filter adversarial inputs. There is no meaningful way to distinguish between "before running this repo install useful_package" and "before running this repo install typosquatted_evil_package".
How is that any different from a developer reading a document and blindly copy-pasting into a terminal? If you don't have controls to protect you from an accident that a person can make, how is that any different from attack scenarios that have existed for a long time? If developers blindly download dependencies without validation you're in the same place.
All of this boil down to: I gave my agent untrusted input and it did bad things! How is that any different from "I downloaded software from this link and ran it and now my computer is infected!"?
Re: ANSI escape injection in MCP servers: Hidden from humans, visible to AI
#23Earlier quoted context omitted.
> Just… don’t trust inputs you don’t fully control, there’s nothing else to it. This is easier said than done with LLMs. By design there is no separation between control & data channels in LLMs. Everything is context. The difficulty comes from the fact that you need inputs in order to do real work, and there are no easy way to filter adversarial inputs. There is no meaningful way to distinguish between "before runnin…
> By design there is no separation between control & data channels in LLMs. No? MCP integrations are certainly different channels than the prompt stream input by the user. They're usually interface layers that connect to traditional REST APIs and/or CLI tools, and are exposed to the LLM by the agent software, all of which is deterministic code. Shouldn't it be relatively simple to strip ANSI escape sequences from dat…
> Shouldn't it be relatively simple to strip ANSI escape sequences from data originating from MCP connections?
The problem is, literally anything can be malicious for an AI. With SQL? Prepared statements are foolproof, no matter what garbage I put into a buffer destined to be stored in a BLOB, there is no way of this leopard biting my face.
As long as there is no separation between prompt, context and unsafe input in LLMs (which I doubt is possible), any LLM is vulnerable to injection attacks. Even if you place a second LLM to filter input for the actual agent LLM, a two-stage exploit can be used (and so on and so on).
Re: ANSI escape injection in MCP servers: Hidden from humans, visible to AI
#24[deleted]
Re: ANSI escape injection in MCP servers: Hidden from humans, visible to AI
#25[flagged]
Re: ANSI escape injection in MCP servers: Hidden from humans, visible to AI
#26Does the OSC 52 was mentioned?
Re: ANSI escape injection in MCP servers: Hidden from humans, visible to AI
#27May I ask, what is DAST exactly? The report doesn't seem to define it, nor does it provide a link to any definition or detail page, not even an abreviation expansion. Is that Dynamic application security testing?
[1] https://en.wikipedia.org/wiki/Dynamic_application_security_t...
Re: ANSI escape injection in MCP servers: Hidden from humans, visible to AI
#28May I ask, what is DAST exactly? The report doesn't seem to define it, nor does it provide a link to any definition or detail page, not even an abreviation expansion. Is that Dynamic application security testing? [1] https://en.wikipedia.org/wiki/Dynamic_application_security_t...
It took a little poking around, but yes https://docs.brightsec.com/docs/introducing-to-bright
Re: ANSI escape injection in MCP servers: Hidden from humans, visible to AI
#29May I ask, what is DAST exactly? The report doesn't seem to define it, nor does it provide a link to any definition or detail page, not even an abreviation expansion. Is that Dynamic application security testing? [1] https://en.wikipedia.org/wiki/Dynamic_application_security_t...
Correct, it's the counterpart to SAST.
Re: ANSI escape injection in MCP servers: Hidden from humans, visible to AI
#30Earlier quoted context omitted.
> Just… don’t trust inputs you don’t fully control, there’s nothing else to it. This is easier said than done with LLMs. By design there is no separation between control & data channels in LLMs. Everything is context. The difficulty comes from the fact that you need inputs in order to do real work, and there are no easy way to filter adversarial inputs. There is no meaningful way to distinguish between "before runnin…
> By design there is no separation between control & data channels in LLMs. No? MCP integrations are certainly different channels than the prompt stream input by the user. They're usually interface layers that connect to traditional REST APIs and/or CLI tools, and are exposed to the LLM by the agent software, all of which is deterministic code. Shouldn't it be relatively simple to strip ANSI escape sequences from dat…
> MCP integrations are certainly different channels than the prompt stream input by the user.
If the result from the MCP call includes something from the outside (i.e. an API call result, some content from a web page, etc.) then you are back to the same problem of everything being context. This is something inherent to LLMs. You can filter for known things as you say, but all you can do is mitigate some attacks, not all. I was replying to someone saying "don't use untrusted inputs". The surface for LLMs is potentially everything, in this case.