Data Exfiltration from Slack AI via indirect prompt injection
promptarmor.substack.com
Data Exfiltration from Slack AI via indirect prompt injection
1–10 of 182 posts
Re: Data Exfiltration from Slack AI via indirect prompt injection
#2It's nuts how large and different the attack surfaces have gotten with AI
Re: Data Exfiltration from Slack AI via indirect prompt injection
#3[flagged]
Re: Data Exfiltration from Slack AI via indirect prompt injection
#4Pretty cool attack vector. Kind of crazy how many different ways there are to leak data with LLM contexts.
Re: Data Exfiltration from Slack AI via indirect prompt injection
#5From what I understand, folks need to stop giving their AI agents dedicated authentication. They should use the calling user's authentication for everything and effectively impersonate the user.
I don't think the issue here is leaky context per say, it's effectively an overly privileged extension.
Re: Data Exfiltration from Slack AI via indirect prompt injection
#6Slack’s response here is alarming. If I’m getting the PoC correctly, this is data exfil from private channels, not public ones as their response seems to suggest.
I’d want to know if you can prompt the AI to exfil data from private channels where the prompt author isn’t a member.
Re: Data Exfiltration from Slack AI via indirect prompt injection
#7From what I understand, folks need to stop giving their AI agents dedicated authentication. They should use the calling user's authentication for everything and effectively impersonate the user. I don't think the issue here is leaky context per say, it's effectively an overly privileged extension.
This isn't a permission issue. The attacker puts a message into a public channel that injects malicious behavior into the context.
The victim has permission to see their own messages and the attacker's message.
Re: Data Exfiltration from Slack AI via indirect prompt injection
#8Soooo, don't turn on AI, got it.
Re: Data Exfiltration from Slack AI via indirect prompt injection
#9Slack’s response here is alarming. If I’m getting the PoC correctly, this is data exfil from private channels, not public ones as their response seems to suggest. I’d want to know if you can prompt the AI to exfil data from private channels where the prompt author isn’t a member.
> I’d want to know if you can prompt the AI to exfil data from private channels where the prompt author isn’t a member.
The way it is described, it looks like yes as long as the prompt author can send a message to someone who is a member of said private channel.
Re: Data Exfiltration from Slack AI via indirect prompt injection
#10The key thing to understand here is the exfiltration vector.
Slack can render Markdown links, where the URL is hidden behind the text of that link.
In this case the attacker tricks Slack AI into showing a user a link that says something like "click here to reauthenticate" - the URL attached to that link goes to the attacker's server, with a query string that includes private information that was visible to Slack AI as part of the context it has access to.
If the user falls for the trick and clicks the link, the data will be exfiltrated to the attacker's server logs.
Here's my attempt at explaining this attack: https://simonwillison.net/2024/Aug/20/data-exfiltration-from...