Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files

alexschapiro.com

Post reply on HN
Print view View original

51–60 of 301 posts

Re: Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files

#51

Post by bfxbjuf »

icyfox wrote: post #12

I'm always a bit surprised how long it can take to triage and fix these pretty glaring security vulnerabilities. October 27, 2025 disclosure and November 4, 2025 email confirmation seems like a long time to have their entire client file system exposed. Sure the actual bug ended up being (what I imagine to be) a Is the issue that people aren't checking their security@ email addresses? People are on holiday? These emai…

Well we have 600 people in the global response center I work at. And the priority issue count is currently 26000. That means its serious enough that its been assigned to some one. There are tens of thousands of unassigned issues cuz the traige teams are swamped. People dont realize as systems get more complex issues increase. They never decrease. And the chimp troupes response has always been a Story - we can handle it.

Re: Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files

#52

Post by throwaway290 »

Barathkanna wrote: post #36
icyfox wrote: post #12

I'm always a bit surprised how long it can take to triage and fix these pretty glaring security vulnerabilities. October 27, 2025 disclosure and November 4, 2025 email confirmation seems like a long time to have their entire client file system exposed. Sure the actual bug ended up being (what I imagine to be) a Is the issue that people aren't checking their security@ email addresses? People are on holiday? These emai…

A lot of the time it’s less “nobody checked the security inbox” and more “the one person who understands that part of the system is juggling twelve other fires.” Security fixes are often a one-hour patch wrapped in two weeks of internal routing, approvals, and “who even owns this code?” archaeology. Holiday schedules and spam filters don’t help, but organizational entropy is usually the real culprit.

It's not about fixing it, it's about acknowledging it exists

Re: Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files

#53

Post by Aurornis »

Barathkanna wrote: post #36
icyfox wrote: post #12

I'm always a bit surprised how long it can take to triage and fix these pretty glaring security vulnerabilities. October 27, 2025 disclosure and November 4, 2025 email confirmation seems like a long time to have their entire client file system exposed. Sure the actual bug ended up being (what I imagine to be) a Is the issue that people aren't checking their security@ email addresses? People are on holiday? These emai…

A lot of the time it’s less “nobody checked the security inbox” and more “the one person who understands that part of the system is juggling twelve other fires.” Security fixes are often a one-hour patch wrapped in two weeks of internal routing, approvals, and “who even owns this code?” archaeology. Holiday schedules and spam filters don’t help, but organizational entropy is usually the real culprit.

> A lot of the time it’s less “nobody checked the security inbox” and more “the one person who understands that part of the system is juggling twelve other fires.”

At my past employers it was "The VP of such-and-such said we need to ship this feature as our top priority, no exceptions"

Re: Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files

#55

Post by Aurornis »

stavros wrote: post #38
gwbas1c wrote: post #35

Earlier quoted context omitted.

Not every organization prioritizes being able to ship a code change at the drop of a hat. This often requires organizational dedication to heavy automated testing a CI, which small companies often aren't set up to do.

I can't believe that any company takes a month to ship something. Even if they don't have CI, surely they'd prefer to break the app (maybe even completely) than risk all their legal documents exfiltrated.

> I can't believe that any company takes a month to ship something.

Outside of startups and big tech, it's not uncommon to have release cycles that are months long. Especially common if there is any legal or regulatory involvement.

Re: Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files

#56

Post by samdoesnothing »

simonw wrote: post #11
electric_muse wrote: post #5

[flagged]

That comment didn't read like AI generated content to me. It made useful points and explained them well. I would not expect even the best of the current batch of LLMs to produce an argument that coherent. This sentence in particular seems outside of what an LLM that was fed the linked article might produce: > What's wild is that nothing here is exotic: subdomain enumeration, unauthenticated API, over-privileged token…

It's probably a list of bullet points or disjointed sentences fed to the LLM to clean up. Might be a non-English speaker using it to become fluent. I won't criticize it, but it's clearly LLM generated content.

Re: Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files

#57

Post by valbaca »

Given the absurd amount startups I see lately that have the words "healthcare" and "AI", I'm actually incredibly concerned that in just a couple of months we're going to have an multiple, enormous HIPAA-data disasters

Just search "healthcare" in https://news.ycombinator.com/item?id=46108941

Re: Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files

#58

Post by Conasg »

electric_muse wrote: post #5

[flagged]

Yeah, you have a point... the comment - and their other comments, on average - seem to fit quite a specific pattern. It's hard to really draw a line between policing style and actually recognising AI-written content, though.

Re: Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files

#59

Post by Aurornis »

simonw wrote: post #11
electric_muse wrote: post #5

[flagged]

That comment didn't read like AI generated content to me. It made useful points and explained them well. I would not expect even the best of the current batch of LLMs to produce an argument that coherent. This sentence in particular seems outside of what an LLM that was fed the linked article might produce: > What's wild is that nothing here is exotic: subdomain enumeration, unauthenticated API, over-privileged token…

The users' comment history does read like generic LLM output. Look at the first lines of different comments:

> Interesting point about Cranelift! I've been following its development for a while, and it seems like there's always something new popping up.

> Interesting point about the color analysis! It kinda reminds me of how album art used to be such a significant part of music culture.

> Interesting point about the ESP32 and music playback! I've been tinkering with similar projects, and it’s wild how much potential these little devices have.

> We used to own tools that made us productive. Now we rent tools that make someone else profitable. Subscriptions are not about recurring value but recurring billing

> Meshtastic is interesting because it's basically "LoRa-first networking" instead of "internet with some radios attached." Most consumer radios are still stuck in the mental model of walkie-talkies, while Meshtastic treats RF as an IP-like transport layer you can script, automate, and extend. That flips the stack:

> This is the collision between two cultures that were never meant to share the same data: "move fast and duct-tape APIs together" startup engineering, and "if this leaks we ruin people's lives" legal/medical confidentiality.

The repeated prefixes (Interesting point about!) and the classic it's-this-not-that LLM pattern are definitely triggering my LLM suspicions.

I suspect most of these cases aren't bots, they're users who put their thoughts, possibly in another language, into an LLM and ask it to form the comment for them. They like the text they see so they copy and paste it into HN.

Re: Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files

#60

Post by mattfrommars »

This might be off topic since we are in topic of AI tool and on HackerNews.

I've been pondering a long time how does one build a startup company in domain they are not familiar with but ... Just have this urge to 'crave a pie' in this space. For the longest time, I had this dream of starting or building a 'AI Legal Tech Company' -- big issue is, I don't work in legal space at all. I did some cold reach on lawfirm related forums which did not take any traction.

I later searched around and came across the term, 'case management software'. From what I know, this is what Cilo fundamentally is and make millions if not billion.

This was close to two years or 1.5 years ago and since then, I stopped thinking about it because of this understanding or belief I have, "how can I do a startup in legal when I don't work in this domain" But when I look around, I have seen people who start companies in totally unrelated industry. From starting a 'dental tech's company to, if I'm not mistaken, the founder of hugging face doesn't seem to have PHD in AI/ML and yet founded HuggingFace.

Given all said, how does one start a company in unrelated domain? Say I want to start another case management system or attempt to clone FileVine, do I first read up what case management software is or do I cold reach to potential lawfirm who would partner up to built a SAAS from scratch? Other school of thought goes like, "find customer before you have a product to validate what you want to build", how does this realistically work?

Apologies for the scattered thoughts...

Post reply on HN