Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An AI lawyer was set to argue in court – real lawyers shut it down

npr.org

Post reply on HN
Print view View original

541–550 of 708 posts

Re: An AI lawyer was set to argue in court – real lawyers shut it down

#541

Post by thinking4real »

ufmace wrote: post #506
jchw wrote: post #478

Earlier quoted context omitted.

Pretty sure the whole reason why DoNotPay actually exists is because defending against parking tickets didn't actually require a strong defense. The tickets were flawed automation, and their formulaic nature justified and equally formulaic response, or something to that effect. Whether the LLM was actually going to output answers directly, or just be used to drive a behavior tree or something like that, is a question…

Thinking about how the problem would "elegantly solve itself" seems to illustrate the issue. Someone using it in an actual courtroom would make a boneheadedly dumb argument or refer to a nonexistent precedent or something. Then maybe the judge gets upset and gives them the harshest punishment or contempt of court or they just lose the case. They may or may not ever get a chance to fix it. A failure mode of jail time…

“Then maybe the judge gets upset and gives them the harshest punishment or contempt of court”

That sounds like a horrible judge, maybe this AI can be used to sniff them out and get rid of them.

Re: An AI lawyer was set to argue in court – real lawyers shut it down

#542

Post by intrasight »

danShumway wrote: post #503
jchw wrote: post #478

Earlier quoted context omitted.

Pretty sure the whole reason why DoNotPay actually exists is because defending against parking tickets didn't actually require a strong defense. The tickets were flawed automation, and their formulaic nature justified and equally formulaic response, or something to that effect. Whether the LLM was actually going to output answers directly, or just be used to drive a behavior tree or something like that, is a question…

> Does anyone else have a better explanation for why there was such a visceral response? I can't speak for lawyers in general or what everyone's motivations would be, but my initial reaction was that it seemed like a somewhat unethical experiment. I assume the client would have agreed or represented themselves, but even there -- legal advice is tricky because it's advice -- it feels unethical to tell a person to rely…

From what I've read recently, the legal profession is the one most at risk of adverse financial effects from AI. Not the court appearances nor the specialized work. But the run-of-the-mill boilerplate legal writing that is the bread and butter profit center of most first. You bet they are threatened and will push back.

Now the question is this. If an AI is doing something illegal like practicing law, how does one sanction an AI?

Edit: found this:

https://jolt.richmond.edu/is-your-artificial-intelligence-gu...

"A person is presumed to be practicing law when engaging in any of the following conduct on behalf of another"

Every state seems to use the word "person" in their rules.

An AI is not a person, and therefore can't be sanctioned for practicing law - my take anyway.

If non-persons can be prosecuted for illegally practicing law, then those non-persons must have the right to get a license. IMHO.

Re: An AI lawyer was set to argue in court – real lawyers shut it down

#543

Post by g_p »

jchw wrote: post #478
lolinder wrote: post #246

This company is either run by someone who doesn't understand the tech or is willfully fraudulent. ChatGPT and company are far from good enough to be entrusted with law. Having interacted extensively with modern LLMs, I absolutely know something like this would happen: > Defendant (as dictated by AI): The Supreme Court ruled in Johnson v. Smith in 1978... > Judge: There was no case Johnson v. Smith in 1978. LLMs hallu…

Pretty sure the whole reason why DoNotPay actually exists is because defending against parking tickets didn't actually require a strong defense. The tickets were flawed automation, and their formulaic nature justified and equally formulaic response, or something to that effect. Whether the LLM was actually going to output answers directly, or just be used to drive a behavior tree or something like that, is a question…

Browder (founder) appeared to also acknowledge that it was not fit for purpose as well [0].

If something that's providing input to a formal legal process (which, let's not forget, means false or inaccurate statements have real and potentially prejudicial repercussions), "makes facts up and exaggerates", then there seems to be no reason they should be talking about taking this anywhere near a courthouse.

This feels a lot like "move fast and break things" being applied - where the people silly enough to use this tool and say whatever it came up with would end up with more serious legal issues. It seems like that only stopped when the founder himself was the one facing the serious legal issues - 'good enough for thee, but not for me'...

I think what many are overlooking is that bad inputs to the legal system can jeopardise someone's position in future irretrievably, with little or no recourse (due to his class action/arbitration waiver). Once someone starts down the road of legal action, there's real consequences if you get it wrong - not only through exposure, but also through prejudicing your own position and making it impossible to take a different route, having previously argued something.

[0] https://twitter.com/SemaforComms/status/1618306993902743555

Re: An AI lawyer was set to argue in court – real lawyers shut it down

#544

Post by freedomben »

thombat wrote: post #505
freedomben wrote: post #498

Earlier quoted context omitted.

> If the AI cites a non-existent precedent, there's literally no way to incorporate that error back into the AI because there's no factual underlying model against which to check the AI's output--unless you had an actual lawyer checking it, in which case, what's the point of the AI? IANAL, but I would bet the level of effort to fact check an AI's output would be orders of magnitude lower than researching and building…

But when appearing in court you're in real-time: you can't take 5 minutes to validate the AI output before passing it on. You can do that for your opening statements but once faced with the judge's rulings or cross-examination you'll be in the weeds.

Yeah that's fair, although if it was AI-assisted lawyer then presumably you'd have done the research ahead of time. But, for spontaneous stuff, you're totally right. My original statement was thinking about it as a "prep time" exercise, but spontaneous stuff would appear in court. Although, the human lawyer (who should still be simiarly prepared for court) would be there to handle those, possibly with some quick assistance.

Re: An AI lawyer was set to argue in court – real lawyers shut it down

#545

Post by intrasight »

ethanbond wrote: post #261
lolinder wrote: post #246

This company is either run by someone who doesn't understand the tech or is willfully fraudulent. ChatGPT and company are far from good enough to be entrusted with law. Having interacted extensively with modern LLMs, I absolutely know something like this would happen: > Defendant (as dictated by AI): The Supreme Court ruled in Johnson v. Smith in 1978... > Judge: There was no case Johnson v. Smith in 1978. LLMs hallu…

DoNotPay seems to know very well what they’re doing. It really doesn’t strike me as true that law requires absolute precision. There are many adjacent (both near and far) arguments that can work in law for any given case, since the interpreter is a human. You just need no silly mistakes that shatter credibility, but that’s very different from “get one thing wrong and the system doesn’t work at all or works in wildly…

>DoNotPay seems to know very well what they’re doing.

And even when they lose the win. Look at the publicity they are getting.

Re: An AI lawyer was set to argue in court – real lawyers shut it down

#546

Post by danShumway »

Forgeties79 wrote: post #532
danShumway wrote: post #503

Earlier quoted context omitted.

> Does anyone else have a better explanation for why there was such a visceral response? I can't speak for lawyers in general or what everyone's motivations would be, but my initial reaction was that it seemed like a somewhat unethical experiment. I assume the client would have agreed or represented themselves, but even there -- legal advice is tricky because it's advice -- it feels unethical to tell a person to rely…

Yeah I feel like you're right on the money on re: the ethics of using someone who is in legal trouble who will have to live with the results. It's not as sexy but they should just build a fake case (or just use an already settled one if possible) and play out the scenario. No reason it wouldn't be just as effective as a "real" case.

I'd have no objections at all to them setting up a fake test case with a real judge or real prosecutors and doing controlled experiments where there's no actual legal risk and where everyone knows it's not a real court case. You're right that it wouldn't be as attention-grabbing, but I suspect it would be a lot more useful for actually determining the AI's capabilities, with basically zero of the ethical downsides. I'd be fully in support of an experiment like that.

Run it multiple times with multiple defendants, set up a control group that's receiving remote advice from actual lawyers, mask which group is which to the judges, then ask the judge(s) at the end to rank the cases and see which defendants did best.

That would be a lot more work, but it would also be much higher quality data than what they were trying to do.

Re: An AI lawyer was set to argue in court – real lawyers shut it down

#547

Post by abfan1127 »

jchw wrote: post #478
lolinder wrote: post #246

This company is either run by someone who doesn't understand the tech or is willfully fraudulent. ChatGPT and company are far from good enough to be entrusted with law. Having interacted extensively with modern LLMs, I absolutely know something like this would happen: > Defendant (as dictated by AI): The Supreme Court ruled in Johnson v. Smith in 1978... > Judge: There was no case Johnson v. Smith in 1978. LLMs hallu…

Pretty sure the whole reason why DoNotPay actually exists is because defending against parking tickets didn't actually require a strong defense. The tickets were flawed automation, and their formulaic nature justified and equally formulaic response, or something to that effect. Whether the LLM was actually going to output answers directly, or just be used to drive a behavior tree or something like that, is a question…

Protectionism. Its why they shout for regulation. Keep the others out, force consumers to use your product, make lots of money.

Re: An AI lawyer was set to argue in court – real lawyers shut it down

#548

Post by freitzkriesler »

Lawyer Bar Associations trying their best to starve off the inevitable. If only ChatGPT was trained with case law and law school texts. Then when they sue, the ChatGPT model can defend itself. I'm affected by this too, but watching lawyers be rendered obsolete makes me very excited.

Re: An AI lawyer was set to argue in court – real lawyers shut it down

#549

Post by freedomben »

gamblor956 wrote: post #517
freedomben wrote: post #498

Earlier quoted context omitted.

> If the AI cites a non-existent precedent, there's literally no way to incorporate that error back into the AI because there's no factual underlying model against which to check the AI's output--unless you had an actual lawyer checking it, in which case, what's the point of the AI? IANAL, but I would bet the level of effort to fact check an AI's output would be orders of magnitude lower than researching and building…

You would lose that bet. Fact-checking nonsensical output would take a lot longer than researching a single body of law, which you can generally do by just looking up a recent case on the matter. You don't need to check every cite; that will have been done for you by the lawyers and judges involved in that case. But checking every cite in an AI's output: many of those citations won't exist, and for the ones that do,…

> You don't need to check every cite; that will have been done for you by the lawyers and judges involved in that case.

Why would this be different with an AI assistant to help you? It's not a binary "do or do not". Just because you have an assistant doesn't mean you don't do anything. Kind of like driver assist can handle some of the load vs full self-driving.

> But checking every cite in an AI's output: many of those citations won't exist, and for the ones that do, you'll need to closely read all of them to confirm that they say what the AI claims they say, or are even within the ballpark of what the AI claims they say.

But you'd have to do this anyway if you did all the research yourself. At least the AI assistant can help give you some good leads so you don't have to start from scratch. A lazy lawyer could skip some verifying, but a good lawyer would still benefit from an AI assistant as was my original bet, just like they would benefit from interns or paralegals, etc. And all those interns and paralegals could still be there, helping verify facts.

Re: An AI lawyer was set to argue in court – real lawyers shut it down

#550

Post by [deleted] »

cwkoss wrote: post #527
marmetio wrote: post #509

Earlier quoted context omitted.

The specific scenario doesn't matter. It's illegal to represent someone else in court if you're not a lawyer. There are a lot of things that you can't get a second chance at if your lawyer messes up that suing them can't fix. Lawyers and judges also negotiate, which a machine can't do because nobody feels an obligation to cut them some slack. Also now you're tainting case law with machine-generated garbage. Everythin…

If you can sell a book that helps teach someone how to represent themselves, why can't you sell a person access to a robot that helps teach them how to represent themselves? Why is the robot not "speech"?

[deleted]
Post reply on HN