"This technology will never fully replace a human lawyer..."
Why not?
Stanford, USC, Duke study shows AI can read contracts better than lawyers
51–60 of 72 posts
Re: Stanford, USC, Duke study shows AI can read contracts better than lawyers
#52> "I strongly believe that law students and junior lawyers need to understand these AI tools, and other technologies, that will help make them better lawyers and shape future legal practice," Buell told Mashable in an email. I am a law student and that is not what is going through my head. What I see is a tougher job market, justifiably. Understanding, and more importantly, developing legal tech will help me make to…
Couple of thoughts. My wife was a law firm junior associate in the banking law group of a major firm during the S&L crisis. She spent a lot of time on the due diligence work involved in merging sick S&L's into each other. I suspect that if faced with a similar Augean stable, having auto-summarization and auto-triage would be a hugely beneficial. True, need for fewer junior associates since they are looking only at ex…
And then you end up writing text in a way that fools auto summarizing AI. War against AI starts in law.
Re: Stanford, USC, Duke study shows AI can read contracts better than lawyers
#53Earlier quoted context omitted.
Couple of thoughts. My wife was a law firm junior associate in the banking law group of a major firm during the S&L crisis. She spent a lot of time on the due diligence work involved in merging sick S&L's into each other. I suspect that if faced with a similar Augean stable, having auto-summarization and auto-triage would be a hugely beneficial. True, need for fewer junior associates since they are looking only at ex…
And then you end up writing text in a way that fools auto summarizing AI. War against AI starts in law.
In a way, the way contracts and legal documents are written now are already an antagonistic optimization against humans natural language ability. A typical contract is pretty difficult for a human to parse without experience. Imagine how much more tricky and complex legalese can get if lawyers start battling machines!
Re: Stanford, USC, Duke study shows AI can read contracts better than lawyers
#54Earlier quoted context omitted.
Couple of thoughts. My wife was a law firm junior associate in the banking law group of a major firm during the S&L crisis. She spent a lot of time on the due diligence work involved in merging sick S&L's into each other. I suspect that if faced with a similar Augean stable, having auto-summarization and auto-triage would be a hugely beneficial. True, need for fewer junior associates since they are looking only at ex…
And then you end up writing text in a way that fools auto summarizing AI. War against AI starts in law.
This. So much this. One big problem with AI/ML as is currently eating the world is that it's just a really fancy averaging engine. It memorizes (incredibly, beautifully, superhumanly) but it doesn't really actually understand. There's a whole spate of "lets trick the AI" in image processing, I have to believe that there are easy ways to do the same with text.
That said, one counterargument could be that obfuscating an AI will lead to more confusing contracts which could actually end up making them harder to enforce. So perhaps in this case there's a counter-force.
Re: Stanford, USC, Duke study shows AI can read contracts better than lawyers
#55I'm gonna go full populist here and say that instead of making AI really smart so that they can read a contract, we should make AI as smart as an average human and discard the contracts it can't understand.
Re: Stanford, USC, Duke study shows AI can read contracts better than lawyers
#56The AI was clearly faster, which is no surprise. But since the "correct" answers for the test were assigned by one group of humans, and the correct answers in real practice are also determined by (a different group of, compared to either those running the test or those taking the test) humans, I'm not sure that result that the AI was more correct can be in any sense meaningful.
That's exactly how every human-vs-AI evaluation is measured. For example, if you have an AI detect whether an image contains a cat, your first step is to have a person manually check every image. Of course it is possible for the test/evaluation labels to be incorrect. Nothing about this case is special, at least nothing you have pointed out.
> That's exactly how every human-vs-AI evaluation is measured.
No, it's not. For instance, there are comparison on games with fixed definite rules.
And even if it was, the fact that a methodological weakness is universal in a field doesn't make it not a weakness, it just makes the field problematic as a whole.
> For example, if you have an AI detect whether an image contains a cat, your first step is to have a person manually check every image.
Sure, but the thing is if you are comparing humans to AI at finding cars in pictures, you are usually testing against lay humans with no special expertise, so the assumption on which usefulness of the experiment rests is “an academic expert evaluating pictures for the presence of cats approximates perfect accuracy near enough that it is unlikely that the score of either lay humans or AI against that expert significantly misstates their accuracy.”
With the legal case, the assumption is something like: “an academic expert evaluating the legal effect that a court would find in a set of contracts approximates perfect accuracy near enough that it is unlikely that the score of either experienced lawyers practicing in the the field or AI against that panel significantly misstates their accuracy.”
The former is, of course, not certain, but I think most people would accept it to be more likely than not to be true.
The latter is less believable (unless, for example, the expert used as the oracle is actually the Supreme Court of the jurisdiction whose law is to be applied in evaluating the contracts.)
Re: Stanford, USC, Duke study shows AI can read contracts better than lawyers
#57Earlier quoted context omitted.
And then you end up writing text in a way that fools auto summarizing AI. War against AI starts in law.
This. So much this. One big problem with AI/ML as is currently eating the world is that it's just a really fancy averaging engine. It memorizes (incredibly, beautifully, superhumanly) but it doesn't really actually understand. There's a whole spate of "lets trick the AI" in image processing, I have to believe that there are easy ways to do the same with text. That said, one counterargument could be that obfuscating a…
>There's a whole spate of "lets trick the AI" in image processing
Check out this "let's trick the human" in image processing:
Re: Stanford, USC, Duke study shows AI can read contracts better than lawyers
#58The AI was clearly faster, which is no surprise. But since the "correct" answers for the test were assigned by one group of humans, and the correct answers in real practice are also determined by (a different group of, compared to either those running the test or those taking the test) humans, I'm not sure that result that the AI was more correct can be in any sense meaningful.
You would assume that the test/answer creation involved a lot more eyes and time to double-check everything. Hell, they might even have used an AI to make sure that they didn't miss anything.
Re: Stanford, USC, Duke study shows AI can read contracts better than lawyers
#59I'm gonna go full populist here and say that instead of making AI really smart so that they can read a contract, we should make AI as smart as an average human and discard the contracts it can't understand.
I'd be happy if as a first pass the AIs went through and cleaned up our mess of legal frameworks so they made logical sense to a being of any intelligence.
Re: Stanford, USC, Duke study shows AI can read contracts better than lawyers
#60"This technology will never fully replace a human lawyer..." Why not?
Because it benefits the company to say so in PR statements.
Also it's going to be hard to _completely_ replace human lawyers; you'll need general intelligence.