Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Two US lawyers fined for submitting fake court citations from ChatGPT

theguardian.com

Post reply on HN
Print view View original

151–160 of 170 posts

Re: Two US lawyers fined for submitting fake court citations from ChatGPT

#151

Post by OliverJones »

And every time this happens from now on, the fine should be doubled. When the fine gets too high for the lazy lawyer to pay, they may instead relinquish their bar admission (lose their license to practice law) for five years.

Lawyers admitted to the bar aren't innocents. They're "officers of the court", meaning the legal system relies on them to avoid BS in their citation of other cases and of statutes and laws. It's never been acceptable for a lawyer to sign off on something without reading it and making sure it's right.

What if registered professional civil engineers started asking generative AI for sewer-system or bridge designs and then didn't inspect them before approving them and handing them off to the guys with dozers and cement trucks? Not only would people fall into rivers, but the rivers would be polluted.

The real world consequences are similarly serious when lawyers make stuff up, or let their interns or word-processing software make stuff up.

Re: Two US lawyers fined for submitting fake court citations from ChatGPT

#152

Post by xen2xen1 »

ssnistfajen wrote: post #124
xen2xen1 wrote: post #115

Earlier quoted context omitted.

I've worked with legal software professionally a few times, and kept thinking software won't replace lawyers, it'll replace paralegals. Not all, but a great deal of grunt work could be replaced with AI and good forms. Now if we could get them not to lie?

There still needs to be a human manually proofreading them, even if it's as trivial as looking up whether the cited cases actually exist. Same goes for code outputs from currently available AI tools: some still needs to debug them to make sure it works, and fix any troubles that spring up during deployment and production. Of course after some number of iterations, AI may gain the ability to self-check and self-correc…

Maybe. But I keep thinking it's not going to replace the single paralegal that a single lawyer has, but it might reduce the 9 paralegals to 6, and then 4, and then..

Re: Two US lawyers fined for submitting fake court citations from ChatGPT

#153

Post by qayxc »

damnesian wrote: post #3

Everyone letting ChatGPT do their jobs, and the people training ChatGPT are too lazy to do the work themselves, they actually use ChatGPT to train ChatGPT... what could possibly go wrong in this scenario?

Related: https://arxiv.org/abs/2305.17493v2

Basically, if this paper is right, 2021 was the last year usable training data could be pulled from the internet. It could be all downhill from hereon out, as the amount of AI generated content will soon overtake actual human generated and -curated data on the internet.

So the party could be over soon ¯\_(ツ)_/¯

Re: Two US lawyers fined for submitting fake court citations from ChatGPT

#154

Post by dwmcqueen »

jcranmer wrote: post #69
jeroenhd wrote: post #46

LegalEagle did a video on this case two weeks ago, digging into it in some more depth: https://youtu.be/oqSYljRYDEM They lied to a judge, doubled down, showed further incompetence by admitting to not knowing how the legal reference they supposedly quoted was even structured, then lied to a judge again trying to get more time to get their story straight, and all it cost them was $5000? And even then they have the guts…

> and all it cost them was $5000? They also have to write written apologies to their client, and all of the judges who were mentioned in their fake cases, with proof filed in the docket. And they are the brunt of a very well-publicized lawsuit. The judge is effectively saying that all that publicity is punishment enough; he doesn't need to do anymore. He doesn't need to refer them to the bar for disbarment--the bar a…

What is even better is this attorney - who relied on ChatGPT for research - when called to task to provide the legal opinions....couldn't even get the affidavit right.

"The April 25 affidavit was filed in response to the Orders of April 11 and 12, 2023, but is sworn to before a Notary Public on the 25th of January 2023. Mr. LoDuca should be prepared to address this anomaly on June 8, 2023 and bring with him a wet-ink signed copy of the affidavit with the notary’s signature and stamp."

Re: Two US lawyers fined for submitting fake court citations from ChatGPT

#155

Post by worrycue »

xen2xen1 wrote: post #115
droopyEyelids wrote: post #98

Earlier quoted context omitted.

Everyone is piling onto you here, but another aspect is that many lawyers already don't do their own research, they use paralegals to do it for them. With that information I think it's less mind-blowing, they were simply assuming ChatGPT could do the work of a paralegal

I've worked with legal software professionally a few times, and kept thinking software won't replace lawyers, it'll replace paralegals. Not all, but a great deal of grunt work could be replaced with AI and good forms. Now if we could get them not to lie?

> Now if we could get them not to lie?

This reminds me. Whatever happened to IBM’s Watson? It’s not a LLM right? But it seem similar enough function wise.

Re: Two US lawyers fined for submitting fake court citations from ChatGPT

#156

Post by asdfsadfk »

jcranmer wrote: post #69
jeroenhd wrote: post #46

LegalEagle did a video on this case two weeks ago, digging into it in some more depth: https://youtu.be/oqSYljRYDEM They lied to a judge, doubled down, showed further incompetence by admitting to not knowing how the legal reference they supposedly quoted was even structured, then lied to a judge again trying to get more time to get their story straight, and all it cost them was $5000? And even then they have the guts…

> and all it cost them was $5000? They also have to write written apologies to their client, and all of the judges who were mentioned in their fake cases, with proof filed in the docket. And they are the brunt of a very well-publicized lawsuit. The judge is effectively saying that all that publicity is punishment enough; he doesn't need to do anymore. He doesn't need to refer them to the bar for disbarment--the bar a…

> The judge is effectively saying that all that publicity is punishment enough; he doesn't need to do anymore. He doesn't need to refer them to the bar for disbarment--the bar already knows about it. (The judge has no power to disbar them himself, he can only refer to the bar for disbarment.)

a reference from the judge and the bar just knowing about it are very different.

Re: Two US lawyers fined for submitting fake court citations from ChatGPT

#157

Post by karim79 »

It would seem that in the future, people will be less inclined to proof-read their work, or even read it at all prior to submission.

I'd wager a guess that this will only get worse over time. LLMs will do more stuff for people - from helping school children to get through their homework, to higher learning assignments and all the way up to the "real" jobs. Along the way people might just place far too much trust in these tools. I'm pretty sure that's something we can all agree on.

What I do find striking here though, is the incredible laziness of the lawyers. Not even checking whether or not the cases actually happened? That's just bad lawyering. The AI actually helped the system to call them out on their bad lawyering.

That's not "bad things happening because AI" but just plain stupidity.

Re: Two US lawyers fined for submitting fake court citations from ChatGPT

#158

Post by tpxl »

jstarfish wrote: post #141
droopyEyelids wrote: post #97

Earlier quoted context omitted.

Are we all using the same definition of "good faith"? That just means to deal honestly and sincerely without any intent to defraud, deceive, or harm the other party. I don't see any proof that shows they intended to deceive rather than that they were clueless.

This right here is what people are overlooking in their contempt. The attorney used it as a research tool, and even foolishly thought to use it to validate that it wasn't making shit up. It's a really embarrassing mistake that's only obvious in the eyes of people familiar with the technology. There is nothing about this that suggests they intended to deceive, only intent to be a lazy idiot.

When your job consists of not being 'a lazy idiot', being one is nothing short of bad faith behaviour.

Re: Two US lawyers fined for submitting fake court citations from ChatGPT

#159

Post by Ensorceled »

mindcrime wrote: post #57
Ensorceled wrote: post #10

Earlier quoted context omitted.

They doubled down on their original mistake rather than admit what they did, even when it was quite clear the judge and opposing council was on to them. They should be disbarred.

They should be disbarred. I just can't agree. To my way of thinking there are very few (if any) things that should be "career ending" the first time they happen. And the main guy involved here, as I recall, has a long career with what has been a spotless record until now. I just can't see the justification for ending that career over one fuck-up. But, again... the punishment does seem lighter than I expected. I was t…

An officer of the court notarized that they had reviewed the imaginary citations and that those imaginary citations were material to the case. This is a gross breach of ethics and, given how easy it is to look up cases and verify them, grossly incompetent and lazy.

These people should not be practicing law.

Re: Two US lawyers fined for submitting fake court citations from ChatGPT

#160

Post by sh34r »

jlarocco wrote: post #119

I wonder if this case was a setup for a lawsuit against OpenAI? Given the marketing and the way Microsoft and Google were tripping over themselves to "catch up", the impression in the media and to the general public is that chat bots are a magic silver bullet for everything. Even a lot of tech people are unclear about exactly how these things work. The case seems pretty clear: "They said it was artificial intelligenc…

“You even said ChatGPT passed the bar exam!”

As entertaining as this is to ponder, I don’t think they’re that clever. I don’t think they’d do so much damage to their reputation for such a flimsy case. They’re just incompetent.

Post reply on HN