A €0.01 bank transfer could compromise a banking AI agent
1–10 of 213 posts
Re: A €0.01 bank transfer could compromise a banking AI agent
#2It was never about the prompt, it is about the prompt delivery.
Re: A €0.01 bank transfer could compromise a banking AI agent
#3Good job AI, after we managed to almost fix SQL injections everywhere, you made them come back!
Re: A €0.01 bank transfer could compromise a banking AI agent
#4Putting AI anywhere near people’s finances without even being asked while being responsible for those finances is some next level negligence imho.
Re: A €0.01 bank transfer could compromise a banking AI agent
#5Good job AI, after we managed to almost fix SQL injections everywhere, you made them come back!
> almost fix SQL injections everywhere
Oh if I had a euro everytime someone claimed that.
Re: A €0.01 bank transfer could compromise a banking AI agent
#6While this is relevant and should indeed be fixed, the attack surface and the practicality of the exploit is a bit meh.
The user needs to do 3 things for this to be actually be phished:
1. Receive money from somebody they don’t known with a weird description 2. Proactively ask the agent for such transaction 3. Click the link the agent provide
While this of course can happen on scale, doesn’t seems so critical in practice
Re: A €0.01 bank transfer could compromise a banking AI agent
#7> There is no single control that solves indirect prompt injection
There is, actually. It's called removing the AI agent. Done.
Re: A €0.01 bank transfer could compromise a banking AI agent
#8While this is relevant and should indeed be fixed, the attack surface and the practicality of the exploit is a bit meh. The user needs to do 3 things for this to be actually be phished: 1. Receive money from somebody they don’t known with a weird description 2. Proactively ask the agent for such transaction 3. Click the link the agent provide While this of course can happen on scale, doesn’t seems so critical in prac…
I think the critical part is that it launders an arbitrary URL as trustworthy. The alternative is “Don’t trust anything our bot says at face value, please.”
I think a better criticism is allowing arbitrary text (including URLs) in a transaction description.
Re: A €0.01 bank transfer could compromise a banking AI agent
#9the solution to this problem is so simple and so easy to reason about from first principles i am shocked i can continue making $$$ deploying agents (LLM-driven workflows) for finance customers
Re: A €0.01 bank transfer could compromise a banking AI agent
#10This is very interesting. Before I read the article, I thought this one one of those instances where a bank asks a customer to verify a recent transaction to prove they are the account holder (like where did you make your last purchase, and how much did you spend there?), for things like password resets or PIN resets over the phone. It occured to me that a phisher who deposits money into a checking account (a small sum included, could use this if they knew the bank would ask what the most recent transaction amount was. Then when they call in pretending to be the customer, they (if they have other personal information like last 4 of SS# and address, email, phone etc), can get their password reset and gain access to the account. But if the customer blocks any unauthorized deposits, such as ACH/Zelle, then they might not have this issue. Obviously banks should caution or avoid using received funds as an authentication method, except as part of a larger number of evidentiary items.
Was this the type of phishing attack they used? If not, there's two vulnerabilities, and one is not yet patched.