Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files

alexschapiro.com

Post reply on HN
Print view View original

241–250 of 301 posts

Re: Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files

#241

Post by tryauuum »

londons_explore wrote: post #144
icyfox wrote: post #12

I'm always a bit surprised how long it can take to triage and fix these pretty glaring security vulnerabilities. October 27, 2025 disclosure and November 4, 2025 email confirmation seems like a long time to have their entire client file system exposed. Sure the actual bug ended up being (what I imagine to be) a Is the issue that people aren't checking their security@ email addresses? People are on holiday? These emai…

The security@ inbox has so much junk these days with someone reporting that if you paste alert('hacked') into devtools then it makes the website hacked! I reckon only 1% of reports are valid. LLM's can now make a plausible looking exploit report ('there is a use after free bug in your server side implementation of X library which allows shell access to your server if you time these two API calls correctly'), but the…

My favorite was "we can trigger your website to initiate a connection to the server we control". They were running their own mail servers and were creating a new accounts on our website. Of course someone needs to initiate a TCP connection to deliver an email message!

Of course this could be a real vulnerability if it would disclose the real server IP behind cloudflare. This was not the case, we were sending via AWS email gateway

Re: Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files

#242

Post by input_sh »

dr_dshiv wrote: post #238
input_sh wrote: post #231

Earlier quoted context omitted.

Prediction: it won't. You can't fit every security consideration into the context window.

90% of human devs are not aware of every security consideration.

90% of human devs can fit more than 3-5 files into their short-term memory.

They also know not to, say, temporarily disable auth to be able to look at the changes they've made on a page hidden behind auth, which is what I observed Gemini 3 Pro doing just yesterday.

Re: Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files

#243

Post by anshumankmr »

0xbadcafebee wrote: post #173

So, 1) a public service, 2) with no authentication, 3) and no encryption? (http only??), 4) sent every single response with a token, 5) giving full admin access to every client's legal documents . This is like a law firm with an open back door, open back window, and all the confidential legal papers sprawled out on the floor. Imagine the potential impact. You're a single mother, fighting for custody of your kids. You…

Can't even make the basic auth of the password is password123.

Re: Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files

#244

Post by Bootvis »

Bootvis wrote: post #34
canopi wrote: post #27

Earlier quoted context omitted.

This. There is so much spam from random people about meaningless issues in our docs. AI has made the problem worse. Determining the meaningful from the meaningless is a full time job.

Use AI for that :)

Not kidding, I bet llm’s are excellent at triaging these reports. Humans, in a corporate setting, are apparently not.

Re: Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files

#246

Post by aduwah »

dr_dshiv wrote: post #230
blitzar wrote: post #222

Earlier quoted context omitted.

humans used open s3 buckets stuffed with text files of usernames, passwords, addresses, credit card numbers etc long before vibe coding was a thing.

Prediction: Vibe coding systems will be better at security in 2 years than 90% of devs.

This will age badly

Re: Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files

#248

Post by dr_dshiv »

aduwah wrote: post #246
dr_dshiv wrote: post #230

Earlier quoted context omitted.

Prediction: Vibe coding systems will be better at security in 2 years than 90% of devs.

This will age badly

That’s why we make concrete measurable predictions.

Re: Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files

#249

Post by dr_dshiv »

input_sh wrote: post #242
dr_dshiv wrote: post #238

Earlier quoted context omitted.

90% of human devs are not aware of every security consideration.

90% of human devs can fit more than 3-5 files into their short-term memory. They also know not to, say, temporarily disable auth to be able to look at the changes they've made on a page hidden behind auth, which is what I observed Gemini 3 Pro doing just yesterday.

Ok, and that’s your prediction for 2 years from now? It’d be quite remarkable if humans had a bigger short term memory than LLMs in 2 years. Or that the kind of dumb security mistakes LLMs make today don’t trigger major, rapid improvements.

Re: Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files

#250

Post by Cthulhu_ »

dr_dshiv wrote: post #230
blitzar wrote: post #222

Earlier quoted context omitted.

humans used open s3 buckets stuffed with text files of usernames, passwords, addresses, credit card numbers etc long before vibe coding was a thing.

Prediction: Vibe coding systems will be better at security in 2 years than 90% of devs.

It'll play a role in both securing and security research I'm sure, but I'm not confident it'll be better.

But also, you'd need to have some metrics - how good are developers at security already? What if the bar is on the floor and LLM code generators are already better?

Post reply on HN