Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files

alexschapiro.com

Post reply on HN
Print view View original

71–80 of 301 posts

Re: Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files

#71

Post by jfindper »

Despyte wrote: post #44
jfindper wrote: post #31

Earlier quoted context omitted.

Every other headline and conversation having ai is super annoying. But also, its super annoying to sift through people saying "the word critical was used, this is obviously ai!". not to mention it really fucking sucks when you're the person who wrote something and people start chanting "ai slop! ai slop!". like, how am i going to prove is not AI? I can't wait until ai gets good enough that no one can tell the differe…

Cultural acceptance of conversation with AI should've come because of actual AI that are indistinguishable from humans, being forced to swallow recognizable if not blatant LLM slop and turn a blind eye feels unfair

the original comment in this chain is not blatant llm slop.

Re: Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files

#73

Post by corry »

"Companies often have a demo environment that is open" - huh?

And... Margolis allowed this open demo environment to connect to their ENTIRE Box drive of millions of super sensitive documents?

HUH???!

Before you get to the terrible security practices of the vendor, you have to place a massive amount of blame on the IT team of Margolis for allowing the above.

No amount of AI hype excuses that kind of professional misjudgement.

Re: Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files

#74

Post by a_victorp »

fallinditch wrote: post #63

> ... after looking through minified code, which SUCKS to do ... AI tends to be good at un-minifying code.

Legit question: when working on finding security issues, are there any guidelines on what you can send to LLMs/AI?

Re: Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files

#75

Post by marginalx »

venturecruelty wrote: post #62
croes wrote: post #13

Earlier quoted context omitted.

> I think this class of problems can be protected against. Of course, it’s called proper software development

And jail time for executives who are responsible for data leaks.

Are you saying executives cannot make mistakes ever (ask because you didn't qualify your statement)?

Re: Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files

#76

Post by pr337h4m »

sys32768 wrote: post #19

I work for a finance firm and everyone is wondering why we can store reams of client data with SaaS Company X, but not upload a trust document or tax return to AI SaaS Company Y. My argument is we're in the Wild West with AI and this stuff is being built so fast with so many evolving tools that corners are being cut even when they don't realize it. This article demonstrates that, but it does sort of beg the question…

FWIW this company was founded in 2014 and appears to have added LLM-powered features relatively recently: https://www.reuters.com/legal/transactional/legal-tech-compa...

Re: Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files

#77

Post by whstl »

Barathkanna wrote: post #36
icyfox wrote: post #12

I'm always a bit surprised how long it can take to triage and fix these pretty glaring security vulnerabilities. October 27, 2025 disclosure and November 4, 2025 email confirmation seems like a long time to have their entire client file system exposed. Sure the actual bug ended up being (what I imagine to be) a Is the issue that people aren't checking their security@ email addresses? People are on holiday? These emai…

A lot of the time it’s less “nobody checked the security inbox” and more “the one person who understands that part of the system is juggling twelve other fires.” Security fixes are often a one-hour patch wrapped in two weeks of internal routing, approvals, and “who even owns this code?” archaeology. Holiday schedules and spam filters don’t help, but organizational entropy is usually the real culprit.

I've once had a whole sector of a fintech go down because one DevOps person ignored daily warning emails for three months that an API key was about to expire and needed reset.

And of course nobody remembered the setup, and logging was only accessible by the same person, so figuring out also took weeks.

Re: Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files

#78

Post by etamponi »

I don't disagree with the sentiment. But let's also be honest. There is a lot of improvement to be made in security software, in terms of ease of use and overcomplicating things.

I worked at Google and then at Meta. Man, the amount of "nonsense" of the ACL system was insane. I write nonsense in quotes because for sure from a security point of view it all made a lot of sense. But there is exactly zero chance that such a system can be used in a less technical company. It took me 4 years to understand how it worked...

So I'll take this as another data point to create a startup that simplifies security... Seems a lot more complicated than AI

Re: Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files

#79

Post by whstl »

canopi wrote: post #27
ipdashc wrote: post #21

Earlier quoted context omitted.

security@ emails do get a lot of spam. It doesn't get talked about very much unless you're monitoring one yourself, but there's a fairly constant stream of people begging for bug bounty money for things like the Secure flag not being set on a cookie. That said, in my experience this spam is still a few emails a day at the most, I don't think there's any excuse for not immediately patching something like that. I guess…

This. There is so much spam from random people about meaningless issues in our docs. AI has made the problem worse. Determining the meaningful from the meaningless is a full time job.

Half of the emails I used to get in a previous company were pointless issues, some coming from a honey pot.

The other half was people demanding payment.

Re: Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files

#80

Post by LoganDark »

Aurornis wrote: post #59
simonw wrote: post #11

Earlier quoted context omitted.

That comment didn't read like AI generated content to me. It made useful points and explained them well. I would not expect even the best of the current batch of LLMs to produce an argument that coherent. This sentence in particular seems outside of what an LLM that was fed the linked article might produce: > What's wild is that nothing here is exotic: subdomain enumeration, unauthenticated API, over-privileged token…

The users' comment history does read like generic LLM output. Look at the first lines of different comments: > Interesting point about Cranelift! I've been following its development for a while, and it seems like there's always something new popping up. > Interesting point about the color analysis! It kinda reminds me of how album art used to be such a significant part of music culture. > Interesting point about the…

> I suspect most of these cases aren't bots, they're users who put their thoughts, possibly in another language, into an LLM and ask it to form the comment for them. They like the text they see so they copy and paste it into HN.

Yes, if this is LLM then it definitely wouldn't be zero-shot. I'm still on the fence myself as I've seen similar writing patterns with Asperger's (specifically what used to be called Asperger's; not general autism spectrum) but those comments don't appear to show any of the other tells to me, so I'm not particularly confident one way or the other.

Post reply on HN