Live data from Hacker News

Lawyers blame ChatGPT for tricking them into citing bogus case law

apnews.com

51–60 of 180 posts

Re: Lawyers blame ChatGPT for tricking them into citing bogus case law

#51

The lawyer said: “I did not comprehend that ChatGPT could fabricate cases." I wonder how many other people using ChatGPT do not comprehend that ChatGPT can be a confident bullshitter... I'm surprised that this one case is getting so much attention because there must be so many instances of people using false information they got from ChatGPT.

notOpenAI told them that this is so incomprehensibly smart that it will destroy all of us. Not because it's dumb and connects the wrong cables, but because it's super-smart. People can't surmise from that that the model just makes stuff up on the way

Re: Lawyers blame ChatGPT for tricking them into citing bogus case law

#52
While the lawyers blamed ChatGPT, the totality of the circumstances seem to indicate that they're less than honest in doing so. There is a live-tweet of the hearing here: https://twitter.com/innercitypress/status/166683852676213965..., and you can follow along with the lawyerly cringe there.

Okay, lawyer #1 (LoDuca, the one on the case in the first place) appears to have played essentially no role in the entire case; his entire purpose appears to be effectively a sockpuppet for lawyer #2 (Schwartz), as LoDuca was admitted to federal court and Schwartz was not. He admits to not having read the things Schwartz asked him to file, as well as the "WTF?" missives that came back. He lied to the court about when he was going to be on vacation, because that is when Schwartz was on vacation. But other than doing nothing when he was supposed to do something (supposed to do a lot of somethings), he is otherwise uninvolved in the shenanigans.

So everything happened because of Schwartz, but before we get to this part, let me fill in relevant background information. The client is suing an airline for an injury governed by the Montreal Protocol. Said airline went bankrupt, and when that happened, the lawyer dismissed the lawsuit, only to refile it when the airline emerged from bankruptcy. This was a mistake; dismissing-and-refiling means the second case is outside the statute of limitations. The airline filed a motion to dismiss because, well, outside statute of limitations, and it is Schwartz's response that is at the center of this controversy.

What appears to have happened is that there is no case law to justify why the case shouldn't be dismissed. Schwartz used chatGPT to try to come up with case law [1] for the argument. He claims in the hearing that he treated it like a search engine, and didn't understand that it could come up with fake arguments. But those claims I'm skeptical of, because even if he's using chatGPT to search for cases, he clearly isn't reading them.

When the airline basically said "uh, we can't find these cases," Schwartz responded by providing fake cases from chatGPT where alarm bells should be ringing saying "SOMETHING IS HORRIBLY, HORRIBLY WRONG." The purported cases in the reply had blindingly obvious flaws that ought to have made you realize something was up before you're off the first page. It is only when the judge turns around and issues the order to show cause that the lawyers attempt to start coming clean.

But wait, there's more! The response was improperly notarized: it had the wrong month. So the judge asked them to provide the original document before signature to justify why it wasn't notary fraud. And, uh, there's a clear OCR error (compare last page of [2] and [3]).

When we get to these parts of in the hearing, Schwartz's responses aren't encouraging. Schwartz tries to dodge the issue of why he was citing cases he didn't read. Believing his responses of why he thought the cases were unpublished ("F.3d means Federal district, third department") really requires you to assume he is an incompetent lawyer at best. The inconsistencies in the affidavit are glossed over, and the stories don't entirely add up. It seems like a minor issue, but it does really give the impression that even with all the attention on them right now, they're still being less than candid with the court.

The attorneys for Schwartz are trying hard to frame it as a "he didn't know what he was getting into with chatGPT, it's not his fault," but honestly, it really does strike me that he knew what he was getting into and somehow thought he wouldn't get caught.

[1] His conversation can be found here: https://storage.courtlistener.com/recap/gov.uscourts.nysd.57..., it's one of the affidavits in the case.

[2] https://storage.courtlistener.com/recap/gov.uscourts.nysd.57...

[3] https://storage.courtlistener.com/recap/gov.uscourts.nysd.57...

Re: Lawyers blame ChatGPT for tricking them into citing bogus case law

#54

Earlier quoted context omitted.

I should have made my thoughts clearer; my mind immediately went to small-time stuff like handling parking disputes, which I think AI is on track to competently solve.

maybe, unless the AI calculates somehow that 98% or something of parking disputes can be resolved by just not showing up in court, so that's the strategy. But you can't convey in a reasonable enough prompt that "this cop just had that look of being ready to really screw me over"...idk man.

There was a brouhaha a few months ago over a tech startup that wanted to do AI lawyering, via having the chatbot speak in a bluetooth earphone or something. They actually signed up someone to do a speeding ticket hearing...

and subpoenaed the officer to make sure he showed up in court for the hearing so that they could actually have oral argument.

The best way to avoid such a ticket is to show up to court and hope the officer doesn't show up. And the AI firm went outs of way to make sure the officer showed up. That's legal malpractice right there. (In the end, IIRC, the judge heard about the firm's involvement and put the kibosh on the entire thing.)

Re: Lawyers blame ChatGPT for tricking them into citing bogus case law

#55

The lawyer said: “I did not comprehend that ChatGPT could fabricate cases." I wonder how many other people using ChatGPT do not comprehend that ChatGPT can be a confident bullshitter... I'm surprised that this one case is getting so much attention because there must be so many instances of people using false information they got from ChatGPT.

"hey chatGPT, can you make up some bullshit for me?"

"I'm sorry, as an online AI language model I am not able to generate output that is knowingly incorrect. Is there anything else you want help with today".

see, it isn't capable of lying. It told me so itself.

Re: Lawyers blame ChatGPT for tricking them into citing bogus case law

#56
post #51

The lawyer said: “I did not comprehend that ChatGPT could fabricate cases." I wonder how many other people using ChatGPT do not comprehend that ChatGPT can be a confident bullshitter... I'm surprised that this one case is getting so much attention because there must be so many instances of people using false information they got from ChatGPT.

notOpenAI told them that this is so incomprehensibly smart that it will destroy all of us. Not because it's dumb and connects the wrong cables, but because it's super-smart. People can't surmise from that that the model just makes stuff up on the way

I prefer to use the term "clopenAI"

Re: Lawyers blame ChatGPT for tricking them into citing bogus case law

#57

“There’s simply no way we could have known these were bogus cases.” the lawyers are quoted as saying. They are currently using Bard to help draft a lawsuit against OpenAI, claiming the company knowingly misrepresents the capabilities of their technology.

The judge is using bingChat to write the decisions anyway.

Re: Lawyers blame ChatGPT for tricking them into citing bogus case law

#58

Earlier quoted context omitted.

I should have made my thoughts clearer; my mind immediately went to small-time stuff like handling parking disputes, which I think AI is on track to competently solve.

maybe, unless the AI calculates somehow that 98% or something of parking disputes can be resolved by just not showing up in court, so that's the strategy. But you can't convey in a reasonable enough prompt that "this cop just had that look of being ready to really screw me over"...idk man.

How would that work, unless the AI advises so many people at a time it can suddenly decide to tell everyone to just stop going to court.

Re: Lawyers blame ChatGPT for tricking them into citing bogus case law

#59
post #55

The lawyer said: “I did not comprehend that ChatGPT could fabricate cases." I wonder how many other people using ChatGPT do not comprehend that ChatGPT can be a confident bullshitter... I'm surprised that this one case is getting so much attention because there must be so many instances of people using false information they got from ChatGPT.

"hey chatGPT, can you make up some bullshit for me?" "I'm sorry, as an online AI language model I am not able to generate output that is knowingly incorrect. Is there anything else you want help with today". see, it isn't capable of lying. It told me so itself.

"Let me put it this way, Mr. Amor. The 9000 series is the most reliable computer ever made. No 9000 computer has ever made a mistake or distorted information. We are all, by any practical definition of the words, foolproof and incapable of error."
Post reply on HN