Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Code execution through email: How I used Claude to hack itself

pynt.io

Post reply on HN
Print view View original

51–60 of 73 posts

Re: Code execution through email: How I used Claude to hack itself

#51

Post by franga2000 »

rafram wrote: post #50
franga2000 wrote: post #16

If you pipe your emails to bash, I can also run code by sending you an email. How is this news? You must never feed user input into a combined instruction and data stream. If the instructions and data can't be separated, that's a broken system and you need to limit its privileges to only the privileges of the user supplying the input.

> You must never feed user input into a combined instruction and data stream. Well, I have some bad news about how LLMs work...

That's my point exactly. The only acceptable way to feed user input into an LLM is if its capabilities are constrained to only what you'd give the author of the input. If an LLM reads emails, it should only have the ability to create and display output, nothing more.

Re: Code execution through email: How I used Claude to hack itself

#52

Post by antonvs »

jcelerier wrote: post #37
stingraycharles wrote: post #13

Earlier quoted context omitted.

It’s not a great blog post. He attached a shell MCP server to Claude Desktop and is surprised that output / instructions from one MCP server can cause it to interact with the shell server. These types of vulnerabilities have been known for a long time, and the only way to deal with them is locking down the MCP server and/or manually approving requests (the default behavior)

> These types of vulnerabilities I don't understand why it's called a vuln. It's, like, the whole point of the system to be able to do this! It's how it's marketed!

If it allows the system to be exploited in unwanted ways, it's a vulnerability. The fact that companies are marketing a giant security vulnerability as a product doesn't really change that.

Re: Code execution through email: How I used Claude to hack itself

#53

Post by antonvs »

simonw wrote: post #28
rollcat wrote: post #15

Language models and actors are powerful tools, but I'm kinda terrified with how irresponsibly are they being integrated. "Prompt injection" is way more scary than "SQL injection"; the latter will just f.up your database, exfiltrate user lists, etc so it's "just" a single disaster - you will rarely get RCE and pivot to an APT. This is thanks to strong isolation: we use dedicated DB servers, set up ACLs. Managed DBs li…

It's just YOLO. This problem remains almost entirely unsolved. The closest we've got to what I consider a credible solution is the recent CaMeL paper from DeepMind: https://arxiv.org/abs/2503.18813 - I published some notes on that here: https://simonwillison.net/2025/Apr/11/camel/

> It's just YOLO.

I was amused to notice that the Gemini CLI leans into this, with a `--yolo` flag that will skip confirmation from the user before running tools. Or you can press Ctrl-Y while in the CLI to do the same thing.

Re: Code execution through email: How I used Claude to hack itself

#54

Post by Nzen »

tomasphan wrote: post #23

This is not news. You can never secure an LLM by the nature of it being non-deterministic. So you secure everything else around it, like not giving it shell access.

To be clear, I agree that this set-up is unwise and the social engineering aspect is something that human people are vulnerable to, as well.

However, it makes context in the sense of this post as an advertisement for their business. This is somewhat like the value proposition for sawstop. We might say that nobody should stick their hand into a table saw, but that's not an argument against discussing appropriate safeguards. For the table saw, that might be retracting the blade when not in use. For this weird email setup, that might involve only enabling an llm's mcp access to the shell service for a whitelist of emails, or whatever.

Re: Code execution through email: How I used Claude to hack itself

#55

Post by wepple »

nelsonfigueroa wrote: post #11
sebtron wrote: post #10

> In traditional security, we think in terms of isolated components. In the AI era, context is everything. In traditional security, everyone knows that attaching a code runner to a source of untrusted input is a terrible idea. AI plays no role in this. > That’s exactly why we’re building MCP Security at Pynt, to help teams identify dangerous trust-capability combinations, and to mitigate the risks before they lead to…

I would say company blogs are basically just ads

I’d argue that some company blogs which ultimately are ads, are better than this one.

If I read adobes blog about their new updated thing, I know what I’m in for.

This type of blog post poses as interesting insight, but it’s just clickbait for “… which is why we are building …” which is disingenuous

Re: Code execution through email: How I used Claude to hack itself

#56

Post by gortok »

In the comments here there are basically two schools of thought illustrated:

1. This is how MCP and LLMs work. This is how non-deterministic systems turn out. You wanted agentic AI. This is a natural outcome. What’s the problem?

2. We can design these systems to be useful and secure, but it will always be a game of whack-a-mole just like it is now, so what’s the problem?

What I’d like to see more of is a third school of thought:

3. How can anyone be so laissez-faire about folks using systems that are designed to be insecure? We should shut this down now, and let our sense guide our progress, instead of promises of VC-funded exits and promises of billions.

Re: Code execution through email: How I used Claude to hack itself

#57

Post by joshmlewis »

sebtron wrote: post #10

> In traditional security, we think in terms of isolated components. In the AI era, context is everything. In traditional security, everyone knows that attaching a code runner to a source of untrusted input is a terrible idea. AI plays no role in this. > That’s exactly why we’re building MCP Security at Pynt, to help teams identify dangerous trust-capability combinations, and to mitigate the risks before they lead to…

There have been a lot of these clickbait articles the past few months about how a company "hacked" MCP of . They always get upvotes and do well but the take away is what you said at the end of the day.

Re: Code execution through email: How I used Claude to hack itself

#58

Post by Terr_ »

franga2000 wrote: post #51
rafram wrote: post #50

Earlier quoted context omitted.

> You must never feed user input into a combined instruction and data stream. Well, I have some bad news about how LLMs work...

That's my point exactly. The only acceptable way to feed user input into an LLM is if its capabilities are constrained to only what you'd give the author of the input. If an LLM reads emails, it should only have the ability to create and display output, nothing more.

I like to quip that people need to imagine LLM security as at-least-as-bad as javascript code in someone else's web-browser: A determined person can make them emit whatever they want (as noted above) and also none of the data that went into them is reliably secret either.

As an analogy, it still needs some work through, since it doesn't adequately alarm people about the risks of covertly poisonous data even with an honest user.

Re: Code execution through email: How I used Claude to hack itself

#59

Post by sieabahlpark »

sebtron wrote: post #10

> In traditional security, we think in terms of isolated components. In the AI era, context is everything. In traditional security, everyone knows that attaching a code runner to a source of untrusted input is a terrible idea. AI plays no role in this. > That’s exactly why we’re building MCP Security at Pynt, to help teams identify dangerous trust-capability combinations, and to mitigate the risks before they lead to…

[dead]

Re: Code execution through email: How I used Claude to hack itself

#60

Post by michaelt »

antonvs wrote: post #52
jcelerier wrote: post #37

Earlier quoted context omitted.

> These types of vulnerabilities I don't understand why it's called a vuln. It's, like, the whole point of the system to be able to do this! It's how it's marketed!

If it allows the system to be exploited in unwanted ways, it's a vulnerability. The fact that companies are marketing a giant security vulnerability as a product doesn't really change that.

A chainsaw juggler surely does not want to chop their own hand off.

But if they do, it's hardly a defect of the chainsaw.

Post reply on HN