Honestly I think this guy was super clever. It was abundantly clear to anyone thinking about this that there was no way this ploy would work. But he got pretty big on Twitter, is getting all of this press, and has now built up awareness of his startup which is doing a far saner and less ambitious task incredible publicity which otherwise he would've had a hard time getting.
An AI lawyer was set to argue in court – real lawyers shut it down
501–510 of 708 posts
Re: An AI lawyer was set to argue in court – real lawyers shut it down
#502I don't like that they're testing this out live. Do what you'd have to do if this were say a medical device: hire a retired judge or two and set up double-blind fake trials with AI or human representation. Prove it works, then try it with real people.
That was my first thought too. Prove it quietly and brag about it later.
Re: An AI lawyer was set to argue in court – real lawyers shut it down
#503This company is either run by someone who doesn't understand the tech or is willfully fraudulent. ChatGPT and company are far from good enough to be entrusted with law. Having interacted extensively with modern LLMs, I absolutely know something like this would happen: > Defendant (as dictated by AI): The Supreme Court ruled in Johnson v. Smith in 1978... > Judge: There was no case Johnson v. Smith in 1978. LLMs hallu…
Pretty sure the whole reason why DoNotPay actually exists is because defending against parking tickets didn't actually require a strong defense. The tickets were flawed automation, and their formulaic nature justified and equally formulaic response, or something to that effect. Whether the LLM was actually going to output answers directly, or just be used to drive a behavior tree or something like that, is a question…
> Does anyone else have a better explanation for why there was such a visceral response?
I can't speak for lawyers in general or what everyone's motivations would be, but my initial reaction was that it seemed like a somewhat unethical experiment. I assume the client would have agreed or represented themselves, but even there -- legal advice is tricky because it's advice -- it feels unethical to tell a person to rely on something that is very likely going to give them sub-par legal representation.
Sneaking it into a courtroom without the judge's knowledge feels a lot like a PR stunt, and one that might encourage further legal malpractice in the future.
I assume there are other factors at play, I assume many lawyers felt insulted or threatened, but ignoring that, it's not an experiment I personally would have lauded even as a non-lawyer who wishes the legal industry was, well... less of an industry. The goal of automating parts of the legal industry and improving access to representation is a good goal that I agree with. And maybe there are ways where AI can help with that, sure. I'm optimistic, I guess. But this feels to me like a startup company taking advantage of someone who's in legal trouble for a publicity stunt, not like an ethically run experiment with controls and with efforts made to mitigate harm.
Details have been scarce, so maybe there were other safety measures put in place; I could be wrong. But my understanding was that this was planned to be secret representation where the judge didn't know. And I can't think of any faster way to get into trouble with a judge then pulling something like that. Even if the AI was brilliant, it apparently wasn't brilliant enough to counsel its own developers that running experiments on judges is a bad legal strategy.
Re: An AI lawyer was set to argue in court – real lawyers shut it down
#504Earlier quoted context omitted.
Is this using chatGPT or gpt3 though? There's a big difference, a fine-tuned llm can do leaps and bounds better than chatGPT. I'm thinking a good SaaS might just be train localized llm's for every city, state, county law and partition the lm based on where it can seek info, then just use it as one big search engine, and of course work in citations, etc.
Take a look at https://www.legalquestions.help/ which gets the above issues wrong at times enough to not rely on it. Maybe their training was bad or insufficient (and I fully expect that sometime in the future this not to be the case).
If you're paying by the hour, and you go to the lawyer with all the data the ai gives you, they can have a paralegal fact check it, and get back to you, but we're at the early stages, things only get better from here on out. Creating some sort of fact algorithm to go w/ gpt3 seems like the next big thing to me. If you can hold it accountable to only give facts, except when an opinion or 'idea' is sought after, which is more ethereal, then you can get some amazing things. Law and even medicine diagnosis will probably be way easier for it than coding, even though it's pretty remarkable on that front already.
Re: An AI lawyer was set to argue in court – real lawyers shut it down
#505Earlier quoted context omitted.
It had to be shot down harshly because there are some premises to a courtroom proceeding that aren't met by an AI as we currently have. One of those is that the lawyer arguing a case is properly credentialed and has been admitted to the bar, and is a professional subject to malpractice standards, who can be held responsible for their performance. An AI spitting out statistically likely responses can't be considered a…
> If the AI cites a non-existent precedent, there's literally no way to incorporate that error back into the AI because there's no factual underlying model against which to check the AI's output--unless you had an actual lawyer checking it, in which case, what's the point of the AI? IANAL, but I would bet the level of effort to fact check an AI's output would be orders of magnitude lower than researching and building…
But when appearing in court you're in real-time: you can't take 5 minutes to validate the AI output before passing it on. You can do that for your opening statements but once faced with the judge's rulings or cross-examination you'll be in the weeds.
Re: An AI lawyer was set to argue in court – real lawyers shut it down
#506This company is either run by someone who doesn't understand the tech or is willfully fraudulent. ChatGPT and company are far from good enough to be entrusted with law. Having interacted extensively with modern LLMs, I absolutely know something like this would happen: > Defendant (as dictated by AI): The Supreme Court ruled in Johnson v. Smith in 1978... > Judge: There was no case Johnson v. Smith in 1978. LLMs hallu…
Pretty sure the whole reason why DoNotPay actually exists is because defending against parking tickets didn't actually require a strong defense. The tickets were flawed automation, and their formulaic nature justified and equally formulaic response, or something to that effect. Whether the LLM was actually going to output answers directly, or just be used to drive a behavior tree or something like that, is a question…
Thinking about how the problem would "elegantly solve itself" seems to illustrate the issue.
Someone using it in an actual courtroom would make a boneheadedly dumb argument or refer to a nonexistent precedent or something. Then maybe the judge gets upset and gives them the harshest punishment or contempt of court or they just lose the case. They may or may not ever get a chance to fix it.
A failure mode of jail time and/or massive fines for your customers doesn't sound all that elegant to me. This isn't a thing to show people cat pictures, I don't think move fast and break things is a good strategy.
Not to say that there aren't some entrenched possibly corrupt and self-serving interests here. But that doesn't mean they don't have a point.
Re: An AI lawyer was set to argue in court – real lawyers shut it down
#507Earlier quoted context omitted.
> The issue I think is that it's pulling from too many sources. There are plenty of sources that are pretty machine readable that will give it good answers. There's a lot of training that can be eked out from the legal databases that already exist that could make it a lot better. If it takes in too much information from too many sources, it tends to get garbled. No, this is a common misunderstanding about the way the…
If it is never pulling from a source, then why is it able to provide citations? check the chat bot on you.com: https://you.com/search?q=list%20top%205%20best%20selling%20c...
Asking "Can you cite some legal precedence for lemon law cases?" gives an answer containing
"In California, for example, the California Supreme Court in the case of Lemon v. Kurtzman (1941) held that a vehicle which did not meet the manufacturer's express warranty was a "lemon" and the manufacturer was liable for damages."
I dont think that case exist, there is a first amendment case Lemon v. Kurtzman, 403 U.S. 602 (1971) though.
I can't find any reference to Kurtzman or 1941 in any of the references. I think the answer is that the AI generating the text, and the code supplying the references are distinct and do not interact.
Re: An AI lawyer was set to argue in court – real lawyers shut it down
#508Earlier quoted context omitted.
> The issue I think is that it's pulling from too many sources. There are plenty of sources that are pretty machine readable that will give it good answers. There's a lot of training that can be eked out from the legal databases that already exist that could make it a lot better. If it takes in too much information from too many sources, it tends to get garbled. No, this is a common misunderstanding about the way the…
If it is never pulling from a source, then why is it able to provide citations? check the chat bot on you.com: https://you.com/search?q=list%20top%205%20best%20selling%20c...
You.com is hugged to death right now, but from what I can see it's a different kind of chatbot. It looks closer to Google's featured snippets than it is to ChatGPT.
That kind of chatbot has different limitations that would make it unsuitable to be an unsupervised legal advice generator.
Re: An AI lawyer was set to argue in court – real lawyers shut it down
#509This company is either run by someone who doesn't understand the tech or is willfully fraudulent. ChatGPT and company are far from good enough to be entrusted with law. Having interacted extensively with modern LLMs, I absolutely know something like this would happen: > Defendant (as dictated by AI): The Supreme Court ruled in Johnson v. Smith in 1978... > Judge: There was no case Johnson v. Smith in 1978. LLMs hallu…
Pretty sure the whole reason why DoNotPay actually exists is because defending against parking tickets didn't actually require a strong defense. The tickets were flawed automation, and their formulaic nature justified and equally formulaic response, or something to that effect. Whether the LLM was actually going to output answers directly, or just be used to drive a behavior tree or something like that, is a question…
The specific scenario doesn't matter. It's illegal to represent someone else in court if you're not a lawyer. There are a lot of things that you can't get a second chance at if your lawyer messes up that suing them can't fix. Lawyers and judges also negotiate, which a machine can't do because nobody feels an obligation to cut them some slack. Also now you're tainting case law with machine-generated garbage. Everything about the justice system assumes humans in the loop. You can't bolt on this one thing without denying people justice.
Re: An AI lawyer was set to argue in court – real lawyers shut it down
#510Earlier quoted context omitted.
What's interesting is that sometimes it does a great job at something like telling you the holdings of a case, but then other times it gives you a completely incorrect response. If you ask it for things like "the X factor test from Johnson v. Smith" sometimes it will dutifully report the correct test in bullets, but other times will say the completely wrong thing. The issue I think is that it's pulling from too many…
> The issue I think is that it's pulling from too many sources. There are plenty of sources that are pretty machine readable that will give it good answers. There's a lot of training that can be eked out from the legal databases that already exist that could make it a lot better. If it takes in too much information from too many sources, it tends to get garbled. No, this is a common misunderstanding about the way the…
What if we prompt the LLM to generate a response with citations, and then we have program which looks up the citations in a citation database to validate their correctness? Responses with hallucinated citations are thrown away, and the LLM is asked to try again. Then, we could retrieve the text of the cited article, and get another LLM to opine on whether the article text supports the point it is being cited in favour of. I think a lot of these problems with LLMs could be worked around with a few more moving parts.