Live data from Hacker News

Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files

alexschapiro.com

181–190 of 301 posts

Re: Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files

#181
post #12

I'm always a bit surprised how long it can take to triage and fix these pretty glaring security vulnerabilities. October 27, 2025 disclosure and November 4, 2025 email confirmation seems like a long time to have their entire client file system exposed. Sure the actual bug ended up being (what I imagine to be) a Is the issue that people aren't checking their security@ email addresses? People are on holiday? These emai…

I'm a bit conflicted about what responsible disclosure should be, but in many cases it seems like these conditions hold: 1) the hack is straightforward to do; 2) it can do a lot of damage (get PII or other confidential info in most cases); 3) downtime of the service wouldn't hurt anyone, especially if we compare it to the risk of the damage. But, instead of insisting on the immediate shutting down of the affected ser…

Don't do this.

I understand you think you are doing the right thing but be aware that by shutting down a medical communication services there's a non-trivial chance someone will die because of slower test results.

Your responsibility is responsible disclosure.

Their responsibility is how to handle it. Don't try to decide that for them.

Re: Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files

#184

"(after looking through minified code, which SUCKS to do)" Would there be a "pretty printer" or some other "unminifier" for this task If not, then is minification effectively a form of obfuscation

https://webcrack.netlify.app/ was made exactly for this, it's a really useful tool

Re: Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files

#185

Earlier quoted context omitted.

I'm a bit conflicted about what responsible disclosure should be, but in many cases it seems like these conditions hold: 1) the hack is straightforward to do; 2) it can do a lot of damage (get PII or other confidential info in most cases); 3) downtime of the service wouldn't hurt anyone, especially if we compare it to the risk of the damage. But, instead of insisting on the immediate shutting down of the affected ser…

> I think in the future I'll anonymously contact companies with way more strict deadlines if their customers (or others) are in serious risk. I'll lose the ability to brag with my real name, but I can live with it. What you're describing is likely a crime. The sad reality is most businesses don't view protection of customers' data as a sacred duty, but simply another of the innumerable risks to be managed in the cour…

Precisely - they view security as just one part of many of their business, instead of viewing it as one of the most important parts. They've insured themselves against a breach, so it's not a big deal for them. But it should be.

The more casualties, the more media attention -> the more likely they, and others in their field, will take security more seriously in the future.

If we let them do nothing for a month, they'll eventually fix it, but in the mean time malicious hackers may gain access to the PII. They might not make it public, but sell that PII via black markets. The company may not get the negative publicity it deserves and likely won't learn to fix their systems in time and to adopt adequate security measures. The sale of the PII and the breach itself might become public knowledge months after the fact, while the company has had a chance to grow in the meantime, and make more security mistakes that may be exploited later on.

And yes, I know it may be a crime - that's why I said I'd report it anonymously from now on. But if the company sits on their asses for a month, shouldn't that count as a crime, as well? The current definition of responsible disclosure gives companies too much leeway, in my opinion.

If I knew I operated a service that was trivial to exploit and was hosting people's PII, I'd shut it down until I fixed it. People won't die if I make everything in my power to provide the test results (in my example of medical labs) to doctors and patients via other means, such as via paper or phone. And if people do die, it would be devastating, of course, but it would mean society has put too much trust into a single system without making sure it's not vulnerable to the most basic of attacks. So it would happen sooner or later, anyway. Although I can't imagine someone dying because their doctor had to make a phone call to the lab instead of typing in a URL.

The same argument about people dying due to the disruption of the medical communications system could be made about too-big-to-fail companies that are entrenched into society because a lot of pension funds have invested in them. If the company goes under, the innocent people dependent on the pension fund's finances would suffer. While they would suffer, which would be awful, of course, would the alternative be to not let such companies go bankrupt? Or would it be better for such funds to not rely so much on one specific company in the first place? That is to say, in both cases (security or stocks in general) the reality is that currently people are too dependent on a few singular entities, while they shouldn't be. That has to change, and the change has to begin somewhere.

Re: Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files

#186

So, 1) a public service, 2) with no authentication, 3) and no encryption? (http only??), 4) sent every single response with a token, 5) giving full admin access to every client's legal documents . This is like a law firm with an open back door, open back window, and all the confidential legal papers sprawled out on the floor. Imagine the potential impact. You're a single mother, fighting for custody of your kids. You…

http-only makes it also very easy to sniff for LE if they decide to. This allows them to get knowledge about cases. Like, they could be scanning it with their own AI tool for all we know. In a free country with proper LE, this would neither be legal nor happening. But I am not sure the USA is remaining one, given the leader is a convicted felon with very dubious moral standards.

The problem here however is that they get away with their sloppiness as long as the security researcher who found this is a whitehat, and the regular news won't pick it up. Once regular media pick this news up (and the local ones should), their name is tarnished and they may regret their sloppiness. Which is a good way to ensure they won't make the same mistake. After all, money talks.

Re: Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files

#187
post #164

Earlier quoted context omitted.

Maybe I have just been lucky, but I have not had the displeasure of working with people either tha incompetent or willfully ignorant yet.

Oh, I should have been more careful in my formulation: There are organisations that are generally competent, and there are places that are less competent. It's not all that uncommon for the whole organisation to be generally incompetent. The saddest places (for me) are those where almost every individual you talk to seems generally competent, but judging by their output the company might as well be stuffed by idiots.…

> suss out how competent people and the organisation are.

how does one do this, without first having the job and being embedded in there? From the outside, it's near impossible to see these details imho.

Re: Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files

#188

I'm less and less sure that when a billion-dollar company screws up this bad, the right thing to do is privately disclose it and let them fix it. This kind of thing just allows companies to go on taking people's money without facing the consequences of their mistakes.

Does a disclosure like this absolve them of any responsibility? They still violated whatever user privacy act.

What are you going to do, sue them? The place is literally teeming with lawyers.

Re: Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files

#189
post #187
post #164

Earlier quoted context omitted.

Oh, I should have been more careful in my formulation: There are organisations that are generally competent, and there are places that are less competent. It's not all that uncommon for the whole organisation to be generally incompetent. The saddest places (for me) are those where almost every individual you talk to seems generally competent, but judging by their output the company might as well be stuffed by idiots.…

> suss out how competent people and the organisation are. how does one do this, without first having the job and being embedded in there? From the outside, it's near impossible to see these details imho.

[dead]

Re: Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files

#190
post #27

Earlier quoted context omitted.

This. There is so much spam from random people about meaningless issues in our docs. AI has made the problem worse. Determining the meaningful from the meaningless is a full time job.

This is where “managed” bug bounty programs like BugCrowd or HackerOne deliver value: only telling you when there is something real. It can be a full time job to separate the wheat from the chaff. It’s made worse by the incentive of the reporters to make everything sound like a P1 hair-on-fire issue.

[dead]
Post reply on HN