Live data from Hacker News

Lawyer cited 6 fake cases made up by ChatGPT; judge calls it “unprecedented”

arstechnica.com

191–200 of 212 posts

Re: Lawyer cited 6 fake cases made up by ChatGPT; judge calls it “unprecedented”

#191
post #24

Wait, this is handled as "ChatGPT made up these cases" and not as "Layers deliberately used ChatGPT to fabricate stuff"? Is anyone really believing a lawyer is that stupid? I know, adssume good intentions and all, but in this case, really?

There are plenty of cases of otherwise intelligent people trusting GPT output too much. For example the professor who asked ChatGPT if it wrote the homework students handed in, and took the "yes" at face value.

Re: Lawyer cited 6 fake cases made up by ChatGPT; judge calls it “unprecedented”

#192
post #95

Earlier quoted context omitted.

It seems that you don't know what you don't know, really. There's no way to definitively know what properties ChatGPT has. It does seem to reason to some extent and it does often say that some information isn't known/there's no data. And it almost obnoxiously often tells you that it's simplifying a complex and multifaceted situation.

Its a model that takes an input and spits out the most likely output given its training. "There's no way to definitively know what properties ChatGPT has." - yes there is: ask it how the war in Ukraine is progressing or some other time based thing. It stops in 2021. It is a really useful tool but it isn't sentient.

I don't agree. It's like saying humans are just animated meat. You're massively simplifying the situation.

Re: Lawyer cited 6 fake cases made up by ChatGPT; judge calls it “unprecedented”

#194

Earlier quoted context omitted.

It's not a person, it's a machine. And it's one that will still produce hallucinations that embarrassingly prove that it has no notion of intelligence, and do so confidently. That it does so less than it's sibling is entirely irrelevant.

Nope, wrong. The number of errors and the magnitude of each is very relevant.

Black box hallucination engines do what they do https://news.ycombinator.com/item?id=36134249

Re: Lawyer cited 6 fake cases made up by ChatGPT; judge calls it “unprecedented”

#195
post #164

Earlier quoted context omitted.

GPT does nothing but predict the next word (token, to be more exact). If you ask it for hints, it's going to come up with some hints that sound like they could be plausible in context of the original riddle. If you then ask it what the answer was, it's going to try to take all the earlier hints into account, because that's now the text it's trying to complete. There's no entity sitting on the other side of the screen…

> You should expect similar results as if it was bunch of people passing a single piece of paper to each other, each writing a single word on it, not communicating with each other in any other way and trying their best to come up with coherent result. I'm pretty sure GPT-4's context window is bigger than '1 token'.

Yes, obviously...? A piece of paper holds more than a single word as well. That piece of paper is the context in this analogy.

It still generates one token at a time and can't backtrack afterwards.

Re: Lawyer cited 6 fake cases made up by ChatGPT; judge calls it “unprecedented”

#196

Earlier quoted context omitted.

A very intelligent person can simultaneously be very stupid, and stupid behavior can be increased with tight deadlines or a high workload.

It was his first try with the tech. He should have popped the cases into West Law and at least skimmed the briefs, he is at a big firm with that subscription. I'm not a lawyer and I at least have a clue for the legal research shortcuts. This guy must have been drinking and or having an affair to have pit so little effort into his first try using a new tech. I'm guessing there is a risk of humorous legal parody gettin…

He might not have thought it was a shortcut though.

If he thinks it is like querying a database and had never heard of hallucinations then this could just be an honest mistake. Especially if he thinks AI would be smarter than a database.

My first thought that he was a mess in general but we really don't have enough information. Like the other guy saying he cheated in life, it is pretty absurd to infer that.

Re: Lawyer cited 6 fake cases made up by ChatGPT; judge calls it “unprecedented”

#197

Sigh, I'm getting very sick of hearing about how "ChatGPT" makes stuff up. Yes, 3.5 made a lot of stuff up, 4.0 still does, but it's much rarer. I wish people would mention this, it's all treated as the same thing. It's like talking about how unreliable these "Airplanes" are when they are talking about prop planes, even though jets are out.

Every post that is even remotely critical of ChatGPT has someone posting this exact sentiment like clockwork. It's probably just mindless fanboys, but I'm genuinely starting to wonder if it's some kind of astroturfed ad campaign for OpenAI.

Re: Lawyer cited 6 fake cases made up by ChatGPT; judge calls it “unprecedented”

#198
post #24

Wait, this is handled as "ChatGPT made up these cases" and not as "Layers deliberately used ChatGPT to fabricate stuff"? Is anyone really believing a lawyer is that stupid? I know, adssume good intentions and all, but in this case, really?

I believe some lawyers are that stupid. I also believe lawyers who are stupid enough not to verify ChatGPT's responses should be treated as if they willfully lied to the court. "Oops, I didn't know" is a good defence when you're caught accidentally walking on the grass, not when you're in court.

Are there traditional American puns with "lawyer" vs. "liar"?

Re: Lawyer cited 6 fake cases made up by ChatGPT; judge calls it “unprecedented”

#199
post #90

Earlier quoted context omitted.

I've had a few moments with ChatGPT that are great anecdotes similar to your own: - Asked it to generate a MadLib for me to play that was no more than a paragraph long. It produced something that was several paragraphs wrong. I told it "no. That's X paragraphs. I asked for one that is only 1 paragraph long" and it would respond "I'm sorry for the misunderstanding. Let me try again" and then would make the same mistak…

Did you really expect ChatGPT to be a full blown dungeon master?

Eventually, sure. But the point is that ChatGPT claims to be. But then does nothing more tell a story and then say “You win!”

Re: Lawyer cited 6 fake cases made up by ChatGPT; judge calls it “unprecedented”

#200
post #182

Earlier quoted context omitted.

The lawyer didn't use ChatGPT to purposefully fabricate cases, he just relied on GPT and assumed it wasn't lying.

How do you know this?

Context clues! Logic? Like, why would the lawyer intentionally have ChatGPT make up bogus cases? Did you read them? They don't make any sense at all and were not likely to trick anyone. He's clearly just being lazy. My suggestion is, unlike the ChatGPT attorney, you use your head and apply some thought to the situation. Imagine what you can now do with this power!
Post reply on HN