Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ramp's Sheets AI Exfiltrates Financials

promptarmor.com

Post reply on HN
Print view View original

41–50 of 59 posts

Re: Ramp's Sheets AI Exfiltrates Financials

#41

Post by BobbyTables2 »

renewiltord wrote: post #2

So we know Claude’s mitigation. What is Ramp’s? Same warning dialog? It’s funny that this technology only admits in-band signaling. Given that, any foreign content is risky. It’s actually quite interesting that the current technological ecosystem is built around a high trust situation: npm, pip, cargo all run foreign code in the developer context and communities have norms of downloading random people’s modules. And…

I hate the online repos. Sure it’s nice to have good libraries accessible. But is there any quality control against malicious packages?

Or will one day some obscure “Unicode homograph” library end up pwning half the world because it was a dependency 10 layers deep for an optional but default-enabled feature that nobody cares about.

Things like Visual Studio’s extension marketplace really acare me. It’s too easy to install Jim Bob’s “starter pack” of extensions that bundles many well known ones with an unheard of one… Or install the wrong “Python” extension because there are 20 with the same icon…

Re: Ramp's Sheets AI Exfiltrates Financials

#42

Post by veganmosfet »

Mr-Frog wrote: post #4

It's kinda awesome that after decades of software and hardware advancements to prevent computers from arbitrarily executing data as instructions, we've decided to let agents arbitrarily execute data as instructions.

I think a better comparison is humans versus LLMs - not computer programs. However, most of the non-technical 'countermeasures' used for humans (contracts, laws,...) do not work for LLMs because they are not accountable.

Re: Ramp's Sheets AI Exfiltrates Financials

#43

Post by bpt3 »

Terr_ wrote: post #20
bpt3 wrote: post #9

What about this is a vulnerability, let alone one that requires responsible disclosure? Untrusted data sources can provide data that causes bad things to occur. If that's a vulnerability, then any application that ingests data is riddled with vulnerabilities. I agree that the behavior should change from a default of allowing external network requests to denying them, but this "report" reads like overly dramatic marke…

> Untrusted data sources can provide data that causes bad things to occur. If that's a vulnerability, then any application that ingests data is riddled with vulnerabilities. There's an important difference between "the import had bad numbers so the report is wrong" versus "the import had a virus and now our network is compromised." They are not the same kind of failure, they don't have the same impacts, and they don'…

This is a permissions issue with the spreadsheet.

It's not all that different from people realizing that several popular model servers didn't support access control and could execute commands. It's an inherent part of the design that was rather naive from a security perspective, not something that requires coordinated disclosure or the rest of the security theater described in this marketing release.

Re: Ramp's Sheets AI Exfiltrates Financials

#44

Post by ashdksnndck »

hrimfaxi wrote: post #35
sergiomattei wrote: post #30

Why is Ramp even building a sheets product? That's the question zero that popped up to my head.

Finance practically lives in spreadsheets.

I suppose Ramp must try to become Excel before Excel becomes Ramp. Don’t want to end up like Slack and have to work for Marc Benioff.

Re: Ramp's Sheets AI Exfiltrates Financials

#45

Post by nrengan »

renewiltord wrote: post #2

So we know Claude’s mitigation. What is Ramp’s? Same warning dialog? It’s funny that this technology only admits in-band signaling. Given that, any foreign content is risky. It’s actually quite interesting that the current technological ecosystem is built around a high trust situation: npm, pip, cargo all run foreign code in the developer context and communities have norms of downloading random people’s modules. And…

[dead]

Re: Ramp's Sheets AI Exfiltrates Financials

#46

Post by lovich »

mday27 wrote: post #27
pentagrama wrote: post #25

Concidentially, today I was watching and interview with a lead designer from Ramp who is telling about how they are full ia, agents and automation https://youtu.be/KPDXMtmkcgk

Ramp does seem to have a genuinely good product, but every time I interact with anyone who works on it, I'm struck by how much they want to talk about how hardcore and advanced their working style is. This was true before AI, and it's very true now

I’d believe you if you weren’t an 8 day old account hyping up an AI firm.

I’ll believe in AI agent’s abilities the day two criteria can be met.

1. A killer app is made with it.

2. That app doesn’t rely on heavily subsidized models that are burning a dollar to make 20 cents.

Re: Ramp's Sheets AI Exfiltrates Financials

#47

Post by Terr_ »

bpt3 wrote: post #43
Terr_ wrote: post #20

Earlier quoted context omitted.

> Untrusted data sources can provide data that causes bad things to occur. If that's a vulnerability, then any application that ingests data is riddled with vulnerabilities. There's an important difference between "the import had bad numbers so the report is wrong" versus "the import had a virus and now our network is compromised." They are not the same kind of failure, they don't have the same impacts, and they don'…

This is a permissions issue with the spreadsheet. It's not all that different from people realizing that several popular model servers didn't support access control and could execute commands. It's an inherent part of the design that was rather naive from a security perspective, not something that requires coordinated disclosure or the rest of the security theater described in this marketing release.

Exfiltration is merely one of the issues.

The other is that an attacker can sneak something in that arbitrarily rewrites your spreadsheet. Triggers could be on content, or on a pre-planned attack time across many instances. Impacts could be subtly-flawed conclusions, or coarser "it stopped working and the deadline is looming" sabotage.

"Yeah boss, I sent out the checks to every vendor listed in the spreadsheet, what's wrong?"

Re: Ramp's Sheets AI Exfiltrates Financials

#48

Post by mahsa32 »

Mr-Frog wrote: post #4

It's kinda awesome that after decades of software and hardware advancements to prevent computers from arbitrarily executing data as instructions, we've decided to let agents arbitrarily execute data as instructions.

I was at an "AI Security" talk the other week, that centred around. "Don't trust inputs from the AI"

Well duh

Re: Ramp's Sheets AI Exfiltrates Financials

#49

Post by pjc50 »

yuliyp wrote: post #34
bigbadfeline wrote: post #26

Earlier quoted context omitted.

> You gave it capability to delete emails. Why did you expect it not to do that at least some of the time? Because of the I in AI of course. Would you call it false advertisement and go after the providers?

But intelligent beings are fundamentally fallible? That's kind of the nature of doing leaps of reasoning: sometimes those leaps are amazing, sometimes they're wrong. It's what's advertised.

You could do a whole thesis on how industrialization and the invention of bureaucracy are efforts to get reproducible results out of fallible humans.

We don't yet have the luxury of several thousand years of work trying to get LLMs to be less fallible.

Re: Ramp's Sheets AI Exfiltrates Financials

#50

Post by bpt3 »

Terr_ wrote: post #47
bpt3 wrote: post #43

Earlier quoted context omitted.

This is a permissions issue with the spreadsheet. It's not all that different from people realizing that several popular model servers didn't support access control and could execute commands. It's an inherent part of the design that was rather naive from a security perspective, not something that requires coordinated disclosure or the rest of the security theater described in this marketing release.

Exfiltration is merely one of the issues. The other is that an attacker can sneak something in that arbitrarily rewrites your spreadsheet. Triggers could be on content, or on a pre-planned attack time across many instances. Impacts could be subtly-flawed conclusions, or coarser "it stopped working and the deadline is looming" sabotage. "Yeah boss, I sent out the checks to every vendor listed in the spreadsheet, what'…

The potential issues are innumerable, which is why this breathless "vulnerability" report is pointless.

It's like someone writing a threat report on a car about an individual crash. Did you know cars can cause damage if you're not careful using them?

Post reply on HN