Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Lawyer cited 6 fake cases made up by ChatGPT; judge calls it “unprecedented”

arstechnica.com

Post reply on HN
Print view View original

161–170 of 212 posts

Re: Lawyer cited 6 fake cases made up by ChatGPT; judge calls it “unprecedented”

#161

Post by Freedom2 »

cj wrote: post #149
BugsJustFindMe wrote: post #106

Earlier quoted context omitted.

I need to know what version of ChatGPT you were using, because this is a critical piece of information that everyone just blatantly ignores, and I can only imagine that it's out of ignorance of the significance of the difference. This is what happened when I asked ChatGPT 4... ME Give me hints without outright telling me the answer to the riddle: "What is always hungry, needs to be fed, and makes your hands red?" Cha…

If I were still able to edit my original comment, I would add a note at the bottom that says to take the experience as a casual person downloading an AI app after hearing about it on the news. Such as a lawyer who’s not particularly tech savvy. The main point is it’s irresponsible to trust LLM output for any critical/important purpose because it’s not perfect. But too many first time users think it is perfect and tru…

Then why did you specifically claim that you asked ChatGPT? These distinctions, while small, are important. Not really the type of mistake to make for a tech-savvy forum.

Re: Lawyer cited 6 fake cases made up by ChatGPT; judge calls it “unprecedented”

#162

Post by uguuo_o »

I thought I had just seen a post yesterday [0], so I immediately thought how can there be 2 such lawyers? Competition on who would get caught first? It seems to be referring to the same case.

[0] https://news.ycombinator.com/item?id=36116652

Re: Lawyer cited 6 fake cases made up by ChatGPT; judge calls it “unprecedented”

#163

Post by inopinatus »

dylan604 wrote: post #111
inopinatus wrote: post #103

Earlier quoted context omitted.

Grinding against the safety rails on a mountain hairpin is not indicative of a competent driver.

no, but that driver might just be that much more successful if they get away with it. in fact, while not a mountain hairpin as your example, there was a recent race car driver that did a similar thing by intentionally using the wall as a push back to allow a maneuver that allowed for success. so a clever comment attempting to prove a point is not always indicative of a proven point ;-)

This amounts to openly advocating for dangerous driving, which is exactly the reductio ad absurdum in rejecting the advice to lawyers above.

That maneuver by Ross Chastain? It is banned.

Re: Lawyer cited 6 fake cases made up by ChatGPT; judge calls it “unprecedented”

#164

Post by dmd »

seba_dos1 wrote: post #118
cj wrote: post #19

I asked ChatGPT to tell me a riddle. It was “What is always hungry, needs to be fed, and makes your hands red?” (Or something like that) I asked for a hint about 5 times and it kept giving more legitimate sounding hints. Finally I gave up and asked for the answer to the riddle, and it spit out a random fruit which made no sense as the answer to the riddle. I then repeated the riddle and asked ChatGPT what the answer…

GPT does nothing but predict the next word (token, to be more exact). If you ask it for hints, it's going to come up with some hints that sound like they could be plausible in context of the original riddle. If you then ask it what the answer was, it's going to try to take all the earlier hints into account, because that's now the text it's trying to complete. There's no entity sitting on the other side of the screen…

> You should expect similar results as if it was bunch of people passing a single piece of paper to each other, each writing a single word on it, not communicating with each other in any other way and trying their best to come up with coherent result.

I'm pretty sure GPT-4's context window is bigger than '1 token'.

Re: Lawyer cited 6 fake cases made up by ChatGPT; judge calls it “unprecedented”

#165

Post by Buttons840 »

cj wrote: post #19

I asked ChatGPT to tell me a riddle. It was “What is always hungry, needs to be fed, and makes your hands red?” (Or something like that) I asked for a hint about 5 times and it kept giving more legitimate sounding hints. Finally I gave up and asked for the answer to the riddle, and it spit out a random fruit which made no sense as the answer to the riddle. I then repeated the riddle and asked ChatGPT what the answer…

> Playing “20 questions” with ChatGPT is another great way to expose its limitations. It knows the game and tries to play, but is terrible at asking questions to narrow down possible answers.

GPT4 did well enough for me. The questions below are asked by GPT4, and the yes/no answers are my responses to GPT4. I think it did quite well at 20 questions.

Is it a person? No. Is it a place? No. Is it a tangible object? Yes. Is the object man-made? Yes. Is the object typically found indoors? Yes. Is it an electronic device? No. Can it be held in one hand? Yes. Is it used primarily in the kitchen? No. Is it related to a specific hobby or activity? Yes. Is it used in a sport or physical activity? No. Is it used in creating or making something? No. Is it associated with a board game or a card game? No. Is the object related to music? Yes. Is it a musical instrument? No. Is it used to control or manipulate sound? Yes. Is it typically used in the process of recording music? Yes. Is it a type of microphone? No. Is it used to alter or modify the sound of a musical instrument during the recording process? No. Is it related to the control or adjustment of volume or sound levels? No. Is the object used to store or convey recorded music? Yes. Is it a vinyl record? No. Is it a compact disc (CD)? Yes. (22 questions total.)

Re: Lawyer cited 6 fake cases made up by ChatGPT; judge calls it “unprecedented”

#166

Post by knifie_spoonie »

glenngillen wrote: post #23
ftxbro wrote: post #2

> "Lawyer: ChatGPT said the cases were real"

Oh boy, and this one! > The plaintiff's lawyer continued to insist that the cases were real. LoDuca filed an affidavit on April 25 in which he swore to the authenticity of the fake cases

> LoDuca filed an affidavit on April 25 in which he swore to the authenticity of the fake cases

From the document another commenter linked above, it seems that affidavit is also dodgy:

"The April 25 affidavit was filled in response to the Orders of April 11 and 12, 2023, but is sworn to before a Notary Public on the 25th of January 2023."

Re: Lawyer cited 6 fake cases made up by ChatGPT; judge calls it “unprecedented”

#167

Post by more_corn »

Every citation created by chat gpt will be hallucinated. It knows what they look like. It doesn’t know what they are. It doesn’t actually “know” anything. It is a statistical engine for generating the next reasonable looking word.

Re: Lawyer cited 6 fake cases made up by ChatGPT; judge calls it “unprecedented”

#168

Post by cj »

Freedom2 wrote: post #161
cj wrote: post #149

Earlier quoted context omitted.

If I were still able to edit my original comment, I would add a note at the bottom that says to take the experience as a casual person downloading an AI app after hearing about it on the news. Such as a lawyer who’s not particularly tech savvy. The main point is it’s irresponsible to trust LLM output for any critical/important purpose because it’s not perfect. But too many first time users think it is perfect and tru…

Then why did you specifically claim that you asked ChatGPT? These distinctions, while small, are important. Not really the type of mistake to make for a tech-savvy forum.

The iOS app that I had this interaction on was ChatOn.

https://apps.apple.com/us/app/chaton-ai-chat-bot-assistant/i...

The subtitle on the Apple App Store is "Powered by ChatGPT & GPT-4".

If you're to believe the app's advertising, it is powered by ChatGPT GPT-4.

Re: Lawyer cited 6 fake cases made up by ChatGPT; judge calls it “unprecedented”

#169

Post by hospitalhusband »

danielmarkbruce wrote: post #132
hospitalhusband wrote: post #125

Earlier quoted context omitted.

> But was it GPT4 I keep seeing this cop-out, which ignores that it's fundamentally the same architecture, and has the same flaws. More wallpaper to hide the cracks better makes it an even worse tool for these use cases because all it does is fool more people into thinking it has capabilities that it fundamentally doesn't.

This is nonsense. It's not a cop-out to say "use the latest, most capable model before complaining". Anyone remotely close to this field knows model size matters, amount of training data matters, quality of training data matters, and several other variables matter. Even if someone knows zero about it, just using 3.5 v 4 is enough to see they are two different things. Like a lizard v a human.

It's still fundamentally the same, hallucinates just the same, and anthropomorphizes itself as a confident, knowledgeable, intelligent being just the same. A newer, better, faster, more capable car still isn't an airplane, even if it go fast enough to spend several seconds in the air.

Re: Lawyer cited 6 fake cases made up by ChatGPT; judge calls it “unprecedented”

#170

Post by danielmarkbruce »

hospitalhusband wrote: post #169
danielmarkbruce wrote: post #132

Earlier quoted context omitted.

This is nonsense. It's not a cop-out to say "use the latest, most capable model before complaining". Anyone remotely close to this field knows model size matters, amount of training data matters, quality of training data matters, and several other variables matter. Even if someone knows zero about it, just using 3.5 v 4 is enough to see they are two different things. Like a lizard v a human.

It's still fundamentally the same, hallucinates just the same, and anthropomorphizes itself as a confident, knowledgeable, intelligent being just the same. A newer, better, faster, more capable car still isn't an airplane, even if it go fast enough to spend several seconds in the air.

Sure, and 40 year olds have the same capabilities as 4 year olds, because "same architecture" or "fundamentally the same". And putting random weights inside the GPT-4 model architecture should behave "fundamentally the same" as the trained GPT-4 weights, because it's "same architecture". Forget this "training" stuff.
Post reply on HN