Stanford, USC, Duke study shows AI can read contracts better than lawyers • Hacker News Board

Re: Stanford, USC, Duke study shows AI can read contracts better than lawyers

#41

Post by tantalor » Mon, Feb 26, 2018, 5:11 PM UTC

dragonwriter wrote: post #18

The AI was clearly faster, which is no surprise. But since the "correct" answers for the test were assigned by one group of humans, and the correct answers in real practice are also determined by (a different group of, compared to either those running the test or those taking the test) humans, I'm not sure that result that the AI was more correct can be in any sense meaningful.

That's exactly how every human-vs-AI evaluation is measured. For example, if you have an AI detect whether an image contains a cat, your first step is to have a person manually check every image. Of course it is possible for the test/evaluation labels to be incorrect. Nothing about this case is special, at least nothing you have pointed out.

Re: Stanford, USC, Duke study shows AI can read contracts better than lawyers

#42

Post by jasonmaydie » Mon, Feb 26, 2018, 5:15 PM UTC

I find it weird that whenever AI comes up, it's not to help but to replace.

Re: Stanford, USC, Duke study shows AI can read contracts better than lawyers

#43

Post by lykahb » Mon, Feb 26, 2018, 5:22 PM UTC

With the increasing use of AI in law, there will be an incentive to make the texts more tractable for AI. Perhaps this may lead to the changes in the law language, eliminating the remaining ambiguities.

Re: Stanford, USC, Duke study shows AI can read contracts better than lawyers

#44

Post by dbcurtis » Mon, Feb 26, 2018, 5:24 PM UTC

dghf wrote: post #31

dbcurtis wrote: post #24

Earlier quoted context omitted.

cat contract.txt | grep -i "indemnify" | wc There's your MVP right there. Seriously, over time I've come to the conclusion that a lot of the most egregious things put in contracts are around open-ended liabilities and indemnifications. It seems like it would be straight-forward to create a tool that simply scanned for which party has liabilities, and what limits there are on those liabilities. It is not everything, b…

> cat contract.txt | grep -i "indemnify" | wc I hate to be that guy, but you have a Useless Use of Cat[0] there. [0] http://porkmail.org/era/unix/award.html#cat

Sure. grep could take the file. I knew that when I wrote the above line. We could have a nice debate about readability and maintainability, compositional design over compact expression, etc. But that's kind of over-kill for a joke, eh?

Re: Stanford, USC, Duke study shows AI can read contracts better than lawyers

#45

Post by MBCook » Mon, Feb 26, 2018, 5:25 PM UTC

Could this lead to a ‘spreadsheet moment’ for lawyers? Something that does the rote work that lawyers used to have to do by hand?

Re: Stanford, USC, Duke study shows AI can read contracts better than lawyers

#46

Post by matthewmcg » Mon, Feb 26, 2018, 5:26 PM UTC

We are evaluating similar software for use during due diligence review in investment and M&A transactions at my firm. We have found that the software is good for identifying when contracts have certain troublesome clauses such as restrictions on assignments in an asset sale. These tools certainly save time and make document review faster and more accurate.

The biggest issue we have found is that this type of review does not identify when contracts are missing key terms. For that, we still need someone with experience in the relevant field thinking about the business context of the agreement and the potential risks.

Re: Stanford, USC, Duke study shows AI can read contracts better than lawyers

#47

Post by mv4 » Mon, Feb 26, 2018, 5:30 PM UTC

I've always felt that a well-written contract should "compile" without errors and warnings.

Instead, in dealing with even simple business contracts, I routinely see things that contradict each other, or are open to interpretation.

There needs to be a good validation process for contracts.

Re: Stanford, USC, Duke study shows AI can read contracts better than lawyers

#48

Post by DanHulton » Mon, Feb 26, 2018, 5:32 PM UTC

frgtpsswrdlame wrote: post #3

I have to be honest, I'm a bit suspicious considering the study was conducted by the company which developed the AI and that the academic auditors are all law professors and not AI professors.

I'm not surprised at all. I just joined a company that has a similar product, and everyone we demo it for is amazed at how accurate and quick it is. AI and law is a huge untapped market that's just starting to get explored.

Consider - law language is very frequently very similar, to the point where that's where the phrase "boilerplate" was adopted from for coding. Is it that surprising that AI would be very, very applicable to this kind of problem?

Re: Stanford, USC, Duke study shows AI can read contracts better than lawyers

#49

Post by mv4 » Mon, Feb 26, 2018, 5:34 PM UTC

mrleiter wrote: post #9

> "I strongly believe that law students and junior lawyers need to understand these AI tools, and other technologies, that will help make them better lawyers and shape future legal practice," Buell told Mashable in an email. I am a law student and that is not what is going through my head. What I see is a tougher job market, justifiably. Understanding, and more importantly, developing legal tech will help me make to…

I suspect that learning even the basic programming principles (e.g. if/then/else, loops, OR/XOR) may be more helpful to a lawyer than understanding "AI tools".

Re: Stanford, USC, Duke study shows AI can read contracts better than lawyers

#50

Post by IncRnd » Mon, Feb 26, 2018, 5:34 PM UTC

What about false positives? False negatives? What about when different legal language is used? How does punctuation affect this? What about differing laws per jurisdiction?

The article is great for an instant hit of wonder about how great AIs are, but an attorney still needs to review the output of the AI.