Live data from Hacker News

GPT-5 outperforms federal judges in legal reasoning experiment

papers.ssrn.com

151–160 of 254 posts

Re: GPT-5 outperforms federal judges in legal reasoning experiment

#151

The fact that the most elite judges in the land, those of the Supreme Court, disagree so extremely and so routinely really says a lot about the farcical nature of the judicial system. Ideally, these people would be selected for their ice-cold and unbiased skills in interpreting the law, and the judgments would be unanimous so frequently that a dissent would be shocking news. Law is complicated, especially the require…

At least you can't buy ChatGPT a nice RV or expensive vacations.

Re: GPT-5 outperforms federal judges in legal reasoning experiment

#152
post #71

Earlier quoted context omitted.

Yeah, I'm reminded of the various child porn cases where the "perpetrator" is a stupid teenager who took nude pics of themselves and sent them to their boy/girlfriend. Many of those cases have been struck down by judges because the letter of the law creates a non-sequitur where the teenager is somehow a felon child predator who solely preyed on themselves, and sending them to jail and forcing them to sign up for a se…

This is one of the roles of justice, but it is also one of the reasons why wealthy people are convicted less often. While it often delivered as a narrative of wealth corrupting the system, the reality is that usually what they are buying is the justice that we all should have. So yes, a judge can let a stupid teenager off on charges of child porn selfies. but without the resources, they are more likely be told by a p…

Whole "democracy" thing is legal framework that wealthy and powerful people built to make safe wealth transfer down the generations possible while giving away as little as possible to average joe.

In a countries without this legal framework its usually free for all fight every time ruling power changes. Not good for preserving capital.

So wealthy having more rights is system working as intended. Not inherently bad thing either as alternative system is whoever best with AK47 having more rights.

Re: GPT-5 outperforms federal judges in legal reasoning experiment

#153

On page 13 you'll see _why_ the judges don't apply the letter of the law - they're seeking to do justice to the victims _in spite of_ the law. "there is another possible explanation: the human judges seek to do justice. The materials include a gruesome description of the injuries the plaintiff sustained in the automobile accident. The court in the earlier proceeding found that she was entitled to [details] a total of…

As damning as these comments are, this comment kinda scared because it reminds me of the times when judges decide against applying empathy against society's most marginalized.

Hopefully as these models get better, we get to a place where judges are pressured to apply empathy more justly.

Re: GPT-5 outperforms federal judges in legal reasoning experiment

#154

Earlier quoted context omitted.

It's probably rather difficult and risky to enact vigilante justice against someone who's in prison. I think the problem is with places where they don't have life sentences at all, but rather let murderers back out into society after some time. I don't know if vigilante justice is a problem there in reality, but at least I can see it as a possibility: someone might still be angry that you murdered their relative afte…

The reference to vigilante justice may be about killing a suspect before they're imprisoned or even tried, such as when a mob storms the local jail. The theory is, if people believe only death can bring justice, and the state doesn't have the death penalty, then the vigilantes will take matters into their own hands. Ergo, the state should have the death penalty. Having recently done an in-depth review of arguments fo…

I see; this makes more sense. It's a little hard to imagine these days though, but ages ago, mobs storming the local jail and hanging a suspect wasn't that uncommon.

Re: GPT-5 outperforms federal judges in legal reasoning experiment

#155
post #152
post #71

Earlier quoted context omitted.

This is one of the roles of justice, but it is also one of the reasons why wealthy people are convicted less often. While it often delivered as a narrative of wealth corrupting the system, the reality is that usually what they are buying is the justice that we all should have. So yes, a judge can let a stupid teenager off on charges of child porn selfies. but without the resources, they are more likely be told by a p…

Whole "democracy" thing is legal framework that wealthy and powerful people built to make safe wealth transfer down the generations possible while giving away as little as possible to average joe. In a countries without this legal framework its usually free for all fight every time ruling power changes. Not good for preserving capital. So wealthy having more rights is system working as intended. Not inherently bad th…

>"So wealthy having more rights is system working as intended. Not inherently bad thing either"

Sorry but I do not feel this way. "Not inherently bad thing either" - I think it is maddening and has to be fixed no matter what. You know, wealthy generally do not really do bad in dictatorial regimes either.

Re: GPT-5 outperforms federal judges in legal reasoning experiment

#156
I'd be more interested in whether it outperforms public defenders for indigent defendants. Human public defenders are notoriously overloaded and can't spend the time needed on every case to research and present a robust defense. Perhaps an LLM could.

Re: GPT-5 outperforms federal judges in legal reasoning experiment

#157
Setting aside all the flaws in the premise, and whatever flaws occurred in the study itself, the basic notion of " outperforms federal judges" comes as no surprise; a rusty length of rebar is probably better at applying the law than most federal judges.

Re: GPT-5 outperforms federal judges in legal reasoning experiment

#158

IANAL, but this seems like an odd test to me. Judges do what their name implies - make judgment calls. I find it re-assuring that judges get different answers under different scenarios, because it means they are listening and making judgment calls. If LLMs give only one answer, no matter what nuances are at play, that sounds like they are failing to judge and instead are diminishing the thought process down to black-…

You have a lot more faith in judges not being biased than I do. I’m about to say something that really makes me throw up a little in my mouth because it harkens back to the forced banal DEI training I had to suffer through in 2020 at BigTech [1]…

But judges have all sorts of biases both conscious and unconscious. Where little Jacob will get in trouble for mischief and little Jerome will do the same thing and Jacob is just “a kid being a kid”. But little Jerome is “a thug in training who we need to protect society from”.

[1] yes I’m well aware that biases exist. Not only did my still living parents grow up in the Jim Crow South. We had a house built in an infamous what was a “sundown town” as recently as 1990.

We have seen how quickly the BS corporate concern was just marketing when it was convenient.

Re: GPT-5 outperforms federal judges in legal reasoning experiment

#159

IANAL, but this seems like an odd test to me. Judges do what their name implies - make judgment calls. I find it re-assuring that judges get different answers under different scenarios, because it means they are listening and making judgment calls. If LLMs give only one answer, no matter what nuances are at play, that sounds like they are failing to judge and instead are diminishing the thought process down to black-…

Yeah, I'm reminded of the various child porn cases where the "perpetrator" is a stupid teenager who took nude pics of themselves and sent them to their boy/girlfriend. Many of those cases have been struck down by judges because the letter of the law creates a non-sequitur where the teenager is somehow a felon child predator who solely preyed on themselves, and sending them to jail and forcing them to sign up for a se…

While some cases have been struck down, about 1/4 of people on the sex offender registry were minors at the time of the offense, 14 is the age at which it is most likely to happen, and this exact scenario accounts for a significant fraction of cases.

Common sense does not always get to show up.

Re: GPT-5 outperforms federal judges in legal reasoning experiment

#160
post #25

IANAL, but this seems like an odd test to me. Judges do what their name implies - make judgment calls. I find it re-assuring that judges get different answers under different scenarios, because it means they are listening and making judgment calls. If LLMs give only one answer, no matter what nuances are at play, that sounds like they are failing to judge and instead are diminishing the thought process down to black-…

The main job of a judicial system is to appear just to people. As long as people think it's just -- everyone is happy. But if it's strictly by the law, but people consider it's unjust -- revolutions happen. In both cases, lawmakers must adapt the law to reflect what people think is "just". That's why there are jury duty in some countries -- to involve people to the ruling, so they see it's just .

No revolution only happens when the law is unjust to people who are in their same tribe…
Post reply on HN