I'm less and less sure that when a billion-dollar company screws up this bad, the right thing to do is privately disclose it and let them fix it. This kind of thing just allows companies to go on taking people's money without facing the consequences of their mistakes.
Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files
171–180 of 301 posts
Re: Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files
#172The bigwigs at my company want to build out a document management suite. After talking to VP of technology about requirements I ask about security as well as what the regulatory requirements are and all I get is a blank stare. I used to think developers had to be supremely incompetent to end up with vulnerabilities like this. But now I understand it’s not the developers who are incompetent…
Re: Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files
#173Imagine the potential impact. You're a single mother, fighting for custody of your kids. Your lawyer has some documentation of something that happened to you, that wasn't your fault, but would look bad if brought up in court. Suddenly you receive a phone call - it's a mysterious voice, demanding $10,000 or they will send the documents to the opposition. Neither of them knows each other; someone just found a trove of documents in an open back door and wanted to make a quick buck.
This is exactly what a software building code would address (if we had one!). Just like you can't open a new storefront in a new building without it being inspected, you should not be able to process millions of sensitive files without having your software's building inspected. The safety and privacy of all of us shouldn't be optional.
Re: Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files
#174I'm less and less sure that when a billion-dollar company screws up this bad, the right thing to do is privately disclose it and let them fix it. This kind of thing just allows companies to go on taking people's money without facing the consequences of their mistakes.
Does a disclosure like this absolve them of any responsibility? They still violated whatever user privacy act.
Re: Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files
#175Earlier quoted context omitted.
Does a disclosure like this absolve them of any responsibility? They still violated whatever user privacy act.
It does. Most privacy laws are based on time-from-discovery. If they immediately sprung into action at the moment they were informed and remediated the issue, they're in compliance.
Re: Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files
#176[flagged]
We detached this subthread from https://news.ycombinator.com/item?id=46137863 and marked it off topic.
Re: Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files
#177It's so great that they allowed him to publish a technical blog post. I once discovered a big vulnerability in a listed consumer tech company -- exposing users' private messages and also allowing to impersonate any user. The company didn't allow me to write a public blogpost.
Re: Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files
#178Earlier quoted context omitted.
It does. Most privacy laws are based on time-from-discovery. If they immediately sprung into action at the moment they were informed and remediated the issue, they're in compliance.
So is that true if they find out when the public does too? It seems that disclosing it privately has some upside (protecting the users) and no downside.
Re: Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files
#179I've worked in several "agentic" roles this year alone (I'm very poachable lol) and otherwise well structured engineering orgs have lost their goddamn minds with move fast and break things because they're worried that OpenAI/Google/Meta/Amazon/Anthropic will release the tool they're working on tomorrow literally all of them are like this
Re: Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files
#180Earlier quoted context omitted.
security@ emails do get a lot of spam. It doesn't get talked about very much unless you're monitoring one yourself, but there's a fairly constant stream of people begging for bug bounty money for things like the Secure flag not being set on a cookie. That said, in my experience this spam is still a few emails a day at the most, I don't think there's any excuse for not immediately patching something like that. I guess…
This. There is so much spam from random people about meaningless issues in our docs. AI has made the problem worse. Determining the meaningful from the meaningless is a full time job.