An AI lawyer was set to argue in court – real lawyers shut it down • Hacker News Board

Re: An AI lawyer was set to argue in court – real lawyers shut it down

#371

Post by bloak » Thu, Jan 26, 2023, 4:36 PM UTC

parsimo2010 wrote: post #130

dools wrote: post #123

This is so fucking dumb. As if you need a lawyer to contest a parking ticket in the first place. The last time I got a parking ticket I had photos and documentary evidence that I should not have been liable. After I lodged my intent to contest the fine, the council sent me a letter saying how they win 97% of cases and I should just pay up now to avoid the risk. I called bullshit and turned up on my court date. There…

The issue here (as with many disruptive tech companies), is the regulatory system. It is illegal in most states to give legal advice if you are not licensed to practice law. If DoNotPay isn't licensed to practice law in California, they can't do this. And unless they have a plan to either get licensed in many states, or somehow change the law, then their business model sucks. It sounds to me that they haven't actuall…

> It is illegal in most states to give legal advice if you are not licensed to practice law.

Presumably that's in the USA, where all sorts of things require a licence. But what's the definition of "legal advice", I wonder? Can an unlicensed person dodge the law just by saying "this is not legal advice" while advising someone how to draft a contract or what to say in court and charging for that advice?

If you want an example of advice that may or may not be "legal advice" think about how to fill in a tax return, how to apply for a government grant, how to apply for or challenge planning permission, how to deal with a difficult employer/employee/neighbour/tenant/landlord, how to apply for a patent, how to deal with various kinds of government inspector, ... That's all specialist stuff for which you might want professional advice but not necessarily from a "lawyer" (depending on what that word means in your part of the world).

Re: An AI lawyer was set to argue in court – real lawyers shut it down

#372

Post by gnicholas » Thu, Jan 26, 2023, 4:37 PM UTC

lolinder wrote: post #246

This company is either run by someone who doesn't understand the tech or is willfully fraudulent. ChatGPT and company are far from good enough to be entrusted with law. Having interacted extensively with modern LLMs, I absolutely know something like this would happen: > Defendant (as dictated by AI): The Supreme Court ruled in Johnson v. Smith in 1978... > Judge: There was no case Johnson v. Smith in 1978. LLMs hallu…

Exactly. I asked it for books on Hong Kong history and it spit out five complete fabrications. The titles were plausible and authors were real people, but none of them had written the books listed.

Re: An AI lawyer was set to argue in court – real lawyers shut it down

#373

Post by weatherlite » Thu, Jan 26, 2023, 4:38 PM UTC

lolinder wrote: post #246

This company is either run by someone who doesn't understand the tech or is willfully fraudulent. ChatGPT and company are far from good enough to be entrusted with law. Having interacted extensively with modern LLMs, I absolutely know something like this would happen: > Defendant (as dictated by AI): The Supreme Court ruled in Johnson v. Smith in 1978... > Judge: There was no case Johnson v. Smith in 1978. LLMs hallu…

They will pretty soon be able to fact check everything they say when they gain real time internet connectivity. But for now yeah you're right. A year-two from now this won't be true anymore.

Re: An AI lawyer was set to argue in court – real lawyers shut it down

#374

Post by gnicholas » Thu, Jan 26, 2023, 4:39 PM UTC

Nifty3929 wrote: post #361

hwillis wrote: post #290

Earlier quoted context omitted.

> One wonders why we have not developed something explicit like mathematical notations for legal stuff. 1. Laws are written or at least voted on by representatives, and they don't vote for things that they don't think they understand. Also, they're pretty regularly swapped out and often totally bonkers. Especially at the state level. 2. Things change. Look at how the right of search and seizure is applied to digital…

A recent US president said about a large bill "we have to pass it just to see what's in it."

I'm unaware of a President saying that. Sounds more like the Nancy Pelosi (then House Minority Leader, subsequently Speaker of the House), talking about Obamacare: https://www.snopes.com/fact-check/pelosi-healthcare-pass-the...

Re: An AI lawyer was set to argue in court – real lawyers shut it down

#375

Post by phpisthebest » Thu, Jan 26, 2023, 4:41 PM UTC

lolinder wrote: post #246

This company is either run by someone who doesn't understand the tech or is willfully fraudulent. ChatGPT and company are far from good enough to be entrusted with law. Having interacted extensively with modern LLMs, I absolutely know something like this would happen: > Defendant (as dictated by AI): The Supreme Court ruled in Johnson v. Smith in 1978... > Judge: There was no case Johnson v. Smith in 1978. LLMs hallu…

>>The legal profession is perhaps the closest one to computer programming

There is a ton of people using ChatGPT for programming.... so much so that I wonder if we will have a crisis in skills as people forgo how to write code.

sysadmin circles have tons people celebrating how they will not have to learn powershell now as an example

Re: An AI lawyer was set to argue in court – real lawyers shut it down

#376

Post by ethanbond » Thu, Jan 26, 2023, 4:41 PM UTC

notahacker wrote: post #370

ethanbond wrote: post #338

Earlier quoted context omitted.

Sure if your objective is to minimize your own personal exposure. If your goal is to push toward a world where poor folks aren’t coerced into not contesting their tickets because they can’t afford to go to court or get representation, then maybe it is a good option.

I prefer a world in which people pay a small fine or make their own excuses rather than pay the fine money to a VC-backed startup for access to an LLM to mount a legally unsound defence that ends up getting them into a lot more trouble, yes. If your goal is to ensure poor folks are pushed towards paying an utterly unaccountable service further fees to escalate legal cases they don't have the knowledge to win so the L…

Yes yes, Andreessen bad and assumed negative outcomes bad, therefore my solution (nil) is just fine.

Re: An AI lawyer was set to argue in court – real lawyers shut it down

#377

Post by lolinder » Thu, Jan 26, 2023, 4:42 PM UTC

winReInstall wrote: post #363

lolinder wrote: post #246

This company is either run by someone who doesn't understand the tech or is willfully fraudulent. ChatGPT and company are far from good enough to be entrusted with law. Having interacted extensively with modern LLMs, I absolutely know something like this would happen: > Defendant (as dictated by AI): The Supreme Court ruled in Johnson v. Smith in 1978... > Judge: There was no case Johnson v. Smith in 1978. LLMs hallu…

Eh, what if it was trained on all the previous cases ever to have existed? I think it could be pretty good, as long as it detects novelty as a to flag and confirm error case.

You could layer another system on top of the LLM generations that attempts to look up cases referenced and discards the response if they don't exist, but that only solves that particular failure mode.

There are other kinds of failures that will be much harder to detect: arguments that sound right but are logically flawed, lost context due to inability to read body language and tone of voice, and lack of a coherent strategy, to name a few.

All of these things could theoretically be solved individually, but each would require new systems to be added which have their own new failure modes. At our current technological level the problem is intractable, even for seemingly simple cases like this one. A defendant is better off defending themselves with their own preparation than they are relying on modern AI in the heat of the moment.

Re: An AI lawyer was set to argue in court – real lawyers shut it down

#378

Post by rafaelero » Thu, Jan 26, 2023, 4:43 PM UTC

dogleash wrote: post #291

jillesvangurp wrote: post #181

There is going to be a lot of this happening. With lawyers, doctors, journalists, all kinds of expensive experts and consultants are going to face some competition from tools like this used by their customers to reduce their dependence on expensive experts or at least to get a second opinion; or even a first opinion. Whether that's misguided or not is not the question. The only question is how good/valuable the AI ad…

>There is going to be a lot of this happening. With lawyers, doctors, journalists, all kinds of expensive experts and consultants are going to face some competition from tools like this used by their customers to reduce their dependence on expensive experts or at least to get a second opinion; or even a first opinion. What are you on about? This has been ongoing for decades. You're talking down to hypothetical doctor…

Google's AI can actually pass the medical bar test and offer diagnosis almost as accurate as clinicians'. That seems very, very different from a search engine.

Re: An AI lawyer was set to argue in court – real lawyers shut it down

#379

Post by phpisthebest » Thu, Jan 26, 2023, 4:43 PM UTC

rhino369 wrote: post #333

pintxo wrote: post #268

Absolute precision? Have you read any law lately? Legal texts are (to me) surprisingly imprecise. One wonders why we have not developed something explicit like mathematical notations for legal stuff. I mean, comma/and/or separated lists in full text? Not even parenthesis? That’s not precision.

>One wonders why we have not developed something explicit like mathematical notations for legal stuff. Because you have to apply the law to fact and facts lack mathematical precision. "No vehicles in the park" would require someone to categorize everything in the world into vehicle or !vehicle. Does a wheel chair count? It's easier to lay out the principle and let judges determine edge cases as they play out.

Even you example is more precise that most laws.

For example Indiana recently changed their Turn signal law from "Must put in on 300 feet before a lane change" to "Must signal in a reasonable time"

WTF is a reasonable time... well what ever time the cop, judge, or prosecutors says it is

Re: An AI lawyer was set to argue in court – real lawyers shut it down

#380

Post by Phlarp » Thu, Jan 26, 2023, 4:44 PM UTC

weatherlite wrote: post #373

lolinder wrote: post #246

This company is either run by someone who doesn't understand the tech or is willfully fraudulent. ChatGPT and company are far from good enough to be entrusted with law. Having interacted extensively with modern LLMs, I absolutely know something like this would happen: > Defendant (as dictated by AI): The Supreme Court ruled in Johnson v. Smith in 1978... > Judge: There was no case Johnson v. Smith in 1978. LLMs hallu…

They will pretty soon be able to fact check everything they say when they gain real time internet connectivity. But for now yeah you're right. A year-two from now this won't be true anymore.

AI hallucinations are going to be the new database query injection. Saying that real time internet connected fact checking will solve that is every bit as naive as thinking the invention of higher level database abstractions like an ORM will solve trivially injectable code.

We can't even make live fact checking work with humans at the wheel. Legacy code bases are so prolific and terrible we're staring down the barrel of a second major industry crises for parsing dates past 2037, but sure LLM's are totally going to get implemented securely and updated regularly unlike all the other software in the world.