Live data from Hacker News

Lawyers blame ChatGPT for tricking them into citing bogus case law

apnews.com

31–40 of 180 posts

Re: Lawyers blame ChatGPT for tricking them into citing bogus case law

#31
post #5

[flagged]

This comment is fraudulent. Not only is the style quite brazenly ChatGPT, this part is a huge red flag: >ChatGPT uncovered state utilities codes... ChatGPT cannot uncover anything, especially the municipality-specific corpus of codes that the comment ostensibly claims to have access to.

OP here. That's exactly what happened.

This is the code it found:

California Public Utilities Code § 10009.6

Re: Lawyers blame ChatGPT for tricking them into citing bogus case law

#32
post #18
post #5

[flagged]

This comment is settinh of my ChatGPT alarm bells. Something about the sentence structure and adjective usage.

Heh, your spidey senses are correct. I actually wrote the response and then ran it though ChatGPT for clean up. I do agree, it actually reads weird over my original draft.

Re: Lawyers blame ChatGPT for tricking them into citing bogus case law

#33
post #29
post #18

Earlier quoted context omitted.

This comment is settinh of my ChatGPT alarm bells. Something about the sentence structure and adjective usage.

Last two paragraphs starting with "however" and "ultimately" SCREAM chatGPT generated to me

As least I didn't add "I hope this response finds you well."

Re: Lawyers blame ChatGPT for tricking them into citing bogus case law

#34
post #5

[flagged]

Did you write this comment with ChatGPT's help? It's a helpful and well written post. I'm not trying to insult it. But I'm curious. The text seems to say "AI", and I'm wondering if I'm seeing things not there.

Yes, in the spirt of talking about ChatGPT I did run my response through it. It is probably the worst thing it has ever produced for me. I ring this up to ChatGPT's "declining quality". :)

Re: Lawyers blame ChatGPT for tricking them into citing bogus case law

#35
post #5

[flagged]

This comment is fraudulent. Not only is the style quite brazenly ChatGPT, this part is a huge red flag: >ChatGPT uncovered state utilities codes... ChatGPT cannot uncover anything, especially the municipality-specific corpus of codes that the comment ostensibly claims to have access to.

>ChatGPT cannot uncover anything, especially the municipality-specific corpus of codes that the comment ostensibly claims to have access to.

It's possible the training set includes those codes (or a citation of those codes). It's possible that they used GPT4 with Browsing, or that it gave them the right terminology to search for & they then pasted in sections of the code and asked it to work off that.

It's also entirely possible ChatGPT hallucinated these utility code sections to support its case and the people working for the utility didn't call their bluff... which is essentially exactly what this story is about, except they got caught out because ChatGPT's bluff was called...

Re: Lawyers blame ChatGPT for tricking them into citing bogus case law

#36
post #12

The cynic in me wonders if this isn’t part of a plan to create a legal precedent banning AI from handling legal disputes. Think about it: the legal profession is possibly one of the most threatened by the development of AI models. What better way to secure the professional future of the long tail of lawyers and paralegals?

The particular issue here is you believe that this lawyer just isn't dumber than a bag of hammers. There is no conspiracy needed. People are dumb. This is why when you see warning labels all over some item, "Big Label" didn't do that, no some dumbass got their tallywacker ripped off by their Easy Bake Oven. Now everyone has to deal with 10 stickers on the item and a dictionary sized book of what you can and can't do…

> People are dumb.

People are dumb. Lawyers shouldn't be. If distribution of stupidity among certain professions is similar to that of people in general, something has gone very bad somewhere in the education and certification pipeline.

Re: Lawyers blame ChatGPT for tricking them into citing bogus case law

#37
A lawyer is certainly at fault if they do not fact check the material they present at trial. But the conmen who are selling ChatGPT and the like are extremely irresponsible for the way they sell LLMs as magical AI that arrives at factually correct answers by reasoning rather than the consequence of the law of large numbers applied to stochastic text generation.

Re: Lawyers blame ChatGPT for tricking them into citing bogus case law

#38

The lawyer said: “I did not comprehend that ChatGPT could fabricate cases." I wonder how many other people using ChatGPT do not comprehend that ChatGPT can be a confident bullshitter... I'm surprised that this one case is getting so much attention because there must be so many instances of people using false information they got from ChatGPT.

I think the case is getting the attention because it's not just some one spouting off online, it's a lawyer bumping into the legal system with false information that would otherwise be a massive legal no-no and they are trying to scape-goat it onto the new shiny software.

Re: Lawyers blame ChatGPT for tricking them into citing bogus case law

#39

The cynic in me wonders if this isn’t part of a plan to create a legal precedent banning AI from handling legal disputes. Think about it: the legal profession is possibly one of the most threatened by the development of AI models. What better way to secure the professional future of the long tail of lawyers and paralegals?

> part of a plan to create a legal precedent banning AI from handling legal disputes.

The judge, in his ruling linked in previous HN discussion, listed a good half dozen parts of a legal dispute where he thinks AI would be super awesome, and then lays out why this particular part of the dispute is terrible for AI to play a role.

Re: Lawyers blame ChatGPT for tricking them into citing bogus case law

#40
post #25

Misrepresenting wholly fabricated case law to a judge out of sheer incompetence has been grounds for disbarment in at most US States for over a century. ChatGPT told me that. It might be true.

Sounds like these lawyers need to be... DisBard? Sorry, it's too early for AI puns.
Post reply on HN