Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files

alexschapiro.com

Post reply on HN
Print view View original

141–150 of 301 posts

Re: Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files

#141

Post by dghlsakjg »

observationist wrote: post #2

I think this class of problems can be protected against. It's become clear that the first and most important and most valuable agent, or team of agents, to build is the one that responsibly and diligently lays out the opsec framework for whatever other system you're trying to automate. A meta-security AI framework, cursor for opsec, would be the best, most valuable general purpose AI tool any company could build, imo…

In this case, AI was a red herring.

This was just plain terrible web security.

Re: Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files

#142

Post by manbash »

culanuchachamim wrote: post #131

-The Filevine team was responsive, professional, and took the findings seriously throughout the disclosure process. They acknowledged the severity, worked to remediate the issues, allowed responsible disclosure, and maintained clear communication. This is another great example of how organizations should handle security disclosures. In the same tenure I think that a professional etical hacker or a curious fellow that…

How else can you take responsibility if you don't make it public? You can't have integrity if you hide away your faults.

Re: Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files

#143

Post by dghlsakjg »

marginalx wrote: post #75
venturecruelty wrote: post #62

Earlier quoted context omitted.

And jail time for executives who are responsible for data leaks.

Are you saying executives cannot make mistakes ever (ask because you didn't qualify your statement)?

The key word in that is "responsible".

The legal world has plenty of ways for determining if you are legally responsible for the outcome of an event. Right now the standard is civil punishments for provable negligence.

It sounds like GP is proposing a framework where we tighten up the definition of negligence, and add criminal penalties in addition to civil ones.

Re: Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files

#144

Post by londons_explore »

icyfox wrote: post #12

I'm always a bit surprised how long it can take to triage and fix these pretty glaring security vulnerabilities. October 27, 2025 disclosure and November 4, 2025 email confirmation seems like a long time to have their entire client file system exposed. Sure the actual bug ended up being (what I imagine to be) a Is the issue that people aren't checking their security@ email addresses? People are on holiday? These emai…

The security@ inbox has so much junk these days with someone reporting that if you paste alert('hacked') into devtools then it makes the website hacked!

I reckon only 1% of reports are valid.

LLM's can now make a plausible looking exploit report ('there is a use after free bug in your server side implementation of X library which allows shell access to your server if you time these two API calls correctly'), but the LLM has made the whole thing up. That can easily waste hours of an experts time for a total falsehood.

I can completely see why some companies decide it'll be an office-hours-only task to go through all the reports every day.

Re: Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files

#145

Post by j45 »

dogman144 wrote: post #114
j45 wrote: post #8

Earlier quoted context omitted.

It's a little hilarious. First, as an organization, do all this cybersecurity theatre, and then create an MCP/LLM wormhole that bypasses it all. All because non-technical folks wave their hands about AI and not understanding the most fundamental reality about LLM software being fundamentally so different than all the software before it that it becomes an unavoidable black hole. I'm also a little pleased I used two sp…

Assuming a 101 security program past the quality bar, there are a number of reason why this can still happen at companies. Summarized as - security is about risk acceptance, not removal. There’s massive business pressure to risk accept AI. Risk acceptance usually means some sort of supplemental control that’s not the ideal but manages. There are very little of these with AI tools however - small vendors, they’re not…

You missed risk creation vs reward creation.

And then folks can gasp and faint like goats and pretend they didn’t know.

It reminds me of the time I met an IT manager who dint have an IT background. Outsourced hilarity ensued through sales people who were also non-technical.

Re: Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files

#146

Post by theoldgreybeard »

The bigwigs at my company want to build out a document management suite. After talking to VP of technology about requirements I ask about security as well as what the regulatory requirements are and all I get is a blank stare.

I used to think developers had to be supremely incompetent to end up with vulnerabilities like this.

But now I understand it’s not the developers who are incompetent…

Re: Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files

#147

Post by hughes »

sys32768 wrote: post #19

I work for a finance firm and everyone is wondering why we can store reams of client data with SaaS Company X, but not upload a trust document or tax return to AI SaaS Company Y. My argument is we're in the Wild West with AI and this stuff is being built so fast with so many evolving tools that corners are being cut even when they don't realize it. This article demonstrates that, but it does sort of beg the question…

While the FileVine service is indeed a Legal AI tool, I don't see the connection between this particular blunder and AI itself. It sure seems like any company with an inexperienced development team and thoughtless security posture could build a system with the same issues.

Specifically, it does not appear that AI is invoked in any way at the search endpoint - it is clearly piping results from some Box API.

Re: Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files

#149

Post by chickensong »

canopi wrote: post #3

The first thing that comes to my mind is SOC2 HIPAA and the whole security theater. I am one of the engineers that had to suffer through countless screenshots and forms to get these because they show that you are compliant and safe. While the real impactful things are ignored

You have to start somewhere though. Security theater sucks, and it's not like compliance is a silver bullet, but at least it's something. Having been through implementing standards compliance, it did help the company in some areas. Was it perfect? Definitely not. Was it driven by financial goals? Absolutely. It did tighten up some weak spots though.

If the options mainly consist of "trust me bro" vs "we can demonstrate that we put in some effort", the latter seems more preferable, even if it's not perfect.

Re: Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files

#150

Post by habosa »

They took a month to fix this? That’s beyond inexcusable. I can’t imagine how any customer could justify working with them going forward.

Also … shows you what a SOC 2 audit is worth: https://www.filevine.com/news/filevine-proves-industry-leade...

Even the most basic pentest would have caught this.

Post reply on HN