Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

B.C. lawyer reprimanded for citing fake cases invented by ChatGPT

cbc.ca

Post reply on HN
Print view View original

11–20 of 38 posts

Re: B.C. lawyer reprimanded for citing fake cases invented by ChatGPT

#11

Post by bigstrat2003 »

rcbdev wrote: post #9

Despite what some in the tech space might think, I'm not sure I can lay the blame 100 % on the lawyer. It's true that this is definitely malpractice and they should have known better, it's also true that big tech for years overplayed their hand with the marketing of "AI" to laypeople and management, which has created completely unreasonable expectations. As long as OpenAI has a profit motive to make people believe Ch…

Right. I think it's fair to ask whether a reasonable, non-technical person would even know that ChatGPT will hallucinate answers. That is widely known in the tech world (and even still you have people relying on it for work, which blows my mind), but it takes time for that knowledge to get out to the general public. That goes double when vendors are putting out advertising which misleads people (as you pointed out).

Re: B.C. lawyer reprimanded for citing fake cases invented by ChatGPT

#12

Post by quantified »

DoreenMichele wrote: post #4

"I did not intend to generate or refer to fictitious cases in this matter. That is clearly wrong and not something I would knowingly do," Ke wrote in her deposition. "I never had any intention to rely upon any fictitious authorities or to mislead the court." So beware what ChatGPT tells you. It might not be real and this can be a big problem in some situations.

Assuming you believe the lawyer. The fact that ChatGPT hallucinates citations has been very well publicized and this lawyer would need to have been under the proverbial rock to not know this. My money's on their simply not expecting to get caught.

Re: B.C. lawyer reprimanded for citing fake cases invented by ChatGPT

#13

Post by quantified »

rcbdev wrote: post #9

Despite what some in the tech space might think, I'm not sure I can lay the blame 100 % on the lawyer. It's true that this is definitely malpractice and they should have known better, it's also true that big tech for years overplayed their hand with the marketing of "AI" to laypeople and management, which has created completely unreasonable expectations. As long as OpenAI has a profit motive to make people believe Ch…

The legal peril for using ChatGPT has been well publicized already.

Re: B.C. lawyer reprimanded for citing fake cases invented by ChatGPT

#14

Post by DoreenMichele »

Lio wrote: post #8
DoreenMichele wrote: post #4

"I did not intend to generate or refer to fictitious cases in this matter. That is clearly wrong and not something I would knowingly do," Ke wrote in her deposition. "I never had any intention to rely upon any fictitious authorities or to mislead the court." So beware what ChatGPT tells you. It might not be real and this can be a big problem in some situations.

"That is clearly wrong and not something I would knowingly do," Such a bullshit excuse. If you are using ChatGPT to do the donkey work for you then you are knowingly using it to cut corners. It's like turning up in court and saying "well I asked this postman Cliff at the bar I go to his opinion on the case...". She should loose here licence for pulling a stunt like that.

Law isn't my strong point. I have had two Law classes -- environmental law -- and made A minuses in spite of bothering to actually study. So this isn't really an area I feel genuinely qualified to suggest a course of action.

But I don't think I have ever used ChatGPT and was blissfully ignorant for some months as to what that hodgepodge of letters even stood for and I'm inclined to be generally appalled by the whole thing.

And I'm sure people simultaneously don't actually care about my opinion but will nonetheless not hesitate to hand me my head for saying anything.

But if you are more qualified than me and feel strongly about this, perhaps you should do something about it -- something more than being cranky about it on the internet.

Re: B.C. lawyer reprimanded for citing fake cases invented by ChatGPT

#16

Post by DoreenMichele »

quantified wrote: post #12
DoreenMichele wrote: post #4

"I did not intend to generate or refer to fictitious cases in this matter. That is clearly wrong and not something I would knowingly do," Ke wrote in her deposition. "I never had any intention to rely upon any fictitious authorities or to mislead the court." So beware what ChatGPT tells you. It might not be real and this can be a big problem in some situations.

Assuming you believe the lawyer. The fact that ChatGPT hallucinates citations has been very well publicized and this lawyer would need to have been under the proverbial rock to not know this. My money's on their simply not expecting to get caught.

I wasn't actually suggesting that I believe the lawyer, fwiw. Just that if you use ChatGPT, "caveat emptor" so to speak.

Re: B.C. lawyer reprimanded for citing fake cases invented by ChatGPT

#18

Post by mdp2021 »

bigstrat2003 wrote: post #11
rcbdev wrote: post #9

Despite what some in the tech space might think, I'm not sure I can lay the blame 100 % on the lawyer. It's true that this is definitely malpractice and they should have known better, it's also true that big tech for years overplayed their hand with the marketing of "AI" to laypeople and management, which has created completely unreasonable expectations. As long as OpenAI has a profit motive to make people believe Ch…

Right. I think it's fair to ask whether a reasonable, non-technical person would even know that ChatGPT will hallucinate answers. That is widely known in the tech world (and even still you have people relying on it for work, which blows my mind), but it takes time for that knowledge to get out to the general public. That goes double when vendors are putting out advertising which misleads people (as you pointed out).

The last time I tried a LLM based chatbot, for testing purposes, was yesterday, and I had fully clear signals of dealing with a moron.

First you have a matter of due diligence when using a tool - especially if you are past the age of twelve, especially if you are a professional -, due diligence that will inform you of the boundaries,

but then you also have to have clean senses and be able to see things for what they are as they patently show to be such. It is like having empowered masses of people that in real "darwinian", selective contexts would die before being impactful.

Re: B.C. lawyer reprimanded for citing fake cases invented by ChatGPT

#19

Post by mdp2021 »

ifdefdebug wrote: post #17

They have to realize that the language model did it's job just fine. It produced a convincing piece of well-formed language that fits the context window. If they wanted knowledge instead, they should have used a knowledge model.

> convincing piece of well-formed language that fits the context window

I hope you do realize that "artificial conman" is hardly a goal.

Re: B.C. lawyer reprimanded for citing fake cases invented by ChatGPT

#20

Post by miningape »

mdp2021 wrote: post #19
ifdefdebug wrote: post #17

They have to realize that the language model did it's job just fine. It produced a convincing piece of well-formed language that fits the context window. If they wanted knowledge instead, they should have used a knowledge model.

> convincing piece of well-formed language that fits the context window I hope you do realize that "artificial conman" is hardly a goal.

I hope you do realise this is what it was trained to do, not be correct, not be inventive/creative, but to make text that _sounds_ good.
Post reply on HN