Live data from Hacker News

Lawyers blame ChatGPT for tricking them into citing bogus case law

apnews.com

171–180 of 180 posts

Re: Lawyers blame ChatGPT for tricking them into citing bogus case law

#171
post #161

Earlier quoted context omitted.

But it wasn't dishonesty it was total obliviousness, maybe negligence.

A negligent civil engineer that failed to verify construction details provided by GPT, which later led to a bridge collapse, would lose their license. A negligent doctor that failed to vet a treatment protocol to a patient that later caused them harm would lose their license. Negligence is a fair, reasonable, and valid reason for someone to be determined unqualified to practice their profession in the future.

> A negligent doctor that failed to vet a treatment protocol to a patient that later caused them harm would lose their license

No, they wouldn’t

Re: Lawyers blame ChatGPT for tricking them into citing bogus case law

#173
post #143

Earlier quoted context omitted.

This is what I don't get. Not only have they not said otherwise, but they put it right up front in a pretty easy to understand brief message before you start using it. I guess lawyers just click agree without reading too.

People are accustomed to ignoring the fine print as legal CYA with no real-world relevance. This is also why the product warnings that "The State of California considers this to cause cancer" are a joke and not a useful message.

It’s not even fine print, there is a big Limitations section almost at the top of the chatgpt landing page with this as first bullet:

”ChatGPT sometimes writes plausible-sounding but incorrect or nonsensical answers.”

It does admittedly sound innocent, it should be written with a more scary tone on to how big extent it bullshits, not just “sometimes”. Still doesn’t excuse the incompetence of the lawyer in question.

Re: Lawyers blame ChatGPT for tricking them into citing bogus case law

#175
post #148
post #109

Earlier quoted context omitted.

oh ok! so if he used gpt-4, still might have been illegal but the output would have been perfect. good to know

Your reading comprehension seems to be as good as that of the lawyer here. You asked what this says about the claim that GPT-4 was good enough to pass the bar. I didn't say anything about GPT-4's quality or the legality here, only that we cannot assume this was the output of GPT-4 without evidence it was given that more people overwhelmingly use the default 3.5-turbo.

[deleted]

Re: Lawyers blame ChatGPT for tricking them into citing bogus case law

#176

Earlier quoted context omitted.

Bingo. With or without ChatGPT this is easily malpractice and potentially even fraud. My (cynical) guess is not only are they lazy and sloppy, they likely (fraudulently) over billed their clients in terms of representing billable hours spent drafting this lawsuit. They're almost certainly billing their clients (or in the case of contingency, keeping the cut they normally would) as though humans (with high hourly rate…

>They're almost certainly billing their clients If they sent that bill in an envelope with a stamp and placed it in the mail...sounds very familiar. Maybe they should ask ChatGPT what are the possible outcomes of me using the answers you provide

Hahahaha I almost included a reference to “The Firm”.

But seriously it would be wire and/or mail fraud.

Re: Lawyers blame ChatGPT for tricking them into citing bogus case law

#177

Earlier quoted context omitted.

Bingo. With or without ChatGPT this is easily malpractice and potentially even fraud. My (cynical) guess is not only are they lazy and sloppy, they likely (fraudulently) over billed their clients in terms of representing billable hours spent drafting this lawsuit. They're almost certainly billing their clients (or in the case of contingency, keeping the cut they normally would) as though humans (with high hourly rate…

> With or without ChatGPT this is easily malpractice and potentially even fraud. AI has been used and will continue to grow in use as a way to launder discrimination and fraud. AI will never face a penalty from the justice system, so why not blame everything on it?

I’m not sure what you’re trying to say here but I pretty clearly place all of the blame on the attorneys - going so far as to describe an equivalent scenario where they just make stuff up.

Re: Lawyers blame ChatGPT for tricking them into citing bogus case law

#178

I'm so confused. Why do the lawyers not simply check to see if the references are real or not? How hard is it to look through an LLM's output and do a quick search to see if any of the laws or cases mentioned in it are in fact unsubstantiated?

Because people aren't used to the idea that computers will make things up. I don't see why anyone is surprised by what the lawyer did here and for sure there is a lot more of this going on than is getting into the public.

People spend their whole lives thinking of machines as logical and correct, but lacking in creativity. Suddenly generative AI comes along and totally upends this. If you're an HN reader and see in depth discussions of LLMs every day then of course it's obvious that these things can make up citations, but if you primary experience of the tech industry is search engines that have never made up fake results ever, then why would you even imagine it's possible?

Re: Lawyers blame ChatGPT for tricking them into citing bogus case law

#179
post #165

Earlier quoted context omitted.

ChatGPT isn’t just an LLM. It’s literally not. There’s a web server, interfaces, plugins, etc. LLMs are this super powerful thing (like a motor) and people are getting to play around with it before it’s fully harnessed. There’s this strange phenomenon where because it’s not totally harnessed, people just rip on it. I don’t know if they think it makes them sound smart, but it sure doesn’t to me. It’s like seeing a mot…

I think you've got hold of the wrong end of your analogy. If you just hold it the right way you will be illuminated and we will all agree how relevant it is. If someone announces to the public, they have a revolutionary new motor that is going to transform personal transportation and solve climate change... ...but it is presented with a chat interface - the motor itself is directed through typing text and it demonstr…

Judging them for having a chat interface is the exact type of thing that I think is unintelligent. It would be like… judging an extremely popular paper for their choice of font. We’re past that.

Re: Lawyers blame ChatGPT for tricking them into citing bogus case law

#180
post #165

Earlier quoted context omitted.

I think you've got hold of the wrong end of your analogy. If you just hold it the right way you will be illuminated and we will all agree how relevant it is. If someone announces to the public, they have a revolutionary new motor that is going to transform personal transportation and solve climate change... ...but it is presented with a chat interface - the motor itself is directed through typing text and it demonstr…

Judging them for having a chat interface is the exact type of thing that I think is unintelligent. It would be like… judging an extremely popular paper for their choice of font. We’re past that.

Who is past it? Is AI past it? "It" = processing the world in terms of correlations?
Post reply on HN