Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-5 outperforms federal judges in legal reasoning experiment

papers.ssrn.com

Post reply on HN
Print view View original

211–220 of 254 posts

Re: GPT-5 outperforms federal judges in legal reasoning experiment

#211

Post by Eddy_Viscosity2 »

scottLobster wrote: post #19
codingdave wrote: post #2

IANAL, but this seems like an odd test to me. Judges do what their name implies - make judgment calls. I find it re-assuring that judges get different answers under different scenarios, because it means they are listening and making judgment calls. If LLMs give only one answer, no matter what nuances are at play, that sounds like they are failing to judge and instead are diminishing the thought process down to black-…

Yeah, I'm reminded of the various child porn cases where the "perpetrator" is a stupid teenager who took nude pics of themselves and sent them to their boy/girlfriend. Many of those cases have been struck down by judges because the letter of the law creates a non-sequitur where the teenager is somehow a felon child predator who solely preyed on themselves, and sending them to jail and forcing them to sign up for a se…

I've often wondered what the prosecutor was thinking when they bring a case like this to trial in the first place.

Re: GPT-5 outperforms federal judges in legal reasoning experiment

#212

Post by FpUser »

SXX wrote: post #183
FpUser wrote: post #155

Earlier quoted context omitted.

>"So wealthy having more rights is system working as intended. Not inherently bad thing either" Sorry but I do not feel this way. "Not inherently bad thing either" - I think it is maddening and has to be fixed no matter what. You know, wealthy generally do not really do bad in dictatorial regimes either.

> "You know, wealthy generally do not really do bad in dictatorial regimes either." Until they found dead with unexpected heart attack, their car blow up or they fall out of the window. In dictatorship vast majority of wealthy people no more than managers of dictators property. Usually with literal golden cages that impossible to sell and transfer. Once person fall out of favor or stop being useful all their "wealth"…

>"Until they found dead with unexpected heart attack"

Or "hang" themselves in jail cell. Cut the crap please. Do you actually have stats to how many of the wealthy percentage wise fell out of the window?

Re: GPT-5 outperforms federal judges in legal reasoning experiment

#213

Post by DannyBee »

bawolff wrote: post #87
codingdave wrote: post #2

IANAL, but this seems like an odd test to me. Judges do what their name implies - make judgment calls. I find it re-assuring that judges get different answers under different scenarios, because it means they are listening and making judgment calls. If LLMs give only one answer, no matter what nuances are at play, that sounds like they are failing to judge and instead are diminishing the thought process down to black-…

> Judges do what their name implies - make judgment calls. I find it re-assuring that judges get different answers under different scenarios, because it means they are listening and making judgment calls. I disagree - law should be the same for everyone. Yes sometimes crimes have mitigating curcumstances and those should be taken into account. However that seems like a separate question of what is and is not illegal.

This view seems to miss the goal of the justice system in the first place. The goals are societal. Any consistency is a means and not an end. (IE being consistent at all is simply one thing that helps achieve some of the societal goals. It is not a goal itself. A totally consistent system that did not achieve the societal goals would be pointless)

Re: GPT-5 outperforms federal judges in legal reasoning experiment

#215

Post by DannyBee »

qgin wrote: post #34

It seems that a lot of people would rather accept a relatively high risk of unfair judgement from a human than accept any nonzero risk of unfair judgement from a computer, even if the risk is smaller with the computer.

“Fair” is a complex moral question that llms are not qualified to answer, since they have no morals or empathy, and aren’t answering here.

Instead they are being “consistent” and the humans are not. Consistency has no moral component and llms are at least theoretically well suited to being consistent (model temperature choices aside)

Fairness and consistency are two different things, and you definitely want your justice system to target fairness above consistency.

Re: GPT-5 outperforms federal judges in legal reasoning experiment

#216

Post by DannyBee »

The number of people in this thread confusing fairness with consistency, and who seem to think llms achieve the former when they achieve the latter seems a bit high. However the fact that some people believe justice systems should actually prize consistency over fairness is … frightening

Re: GPT-5 outperforms federal judges in legal reasoning experiment

#217

Post by Nursie »

fluidcruft wrote: post #210
Nursie wrote: post #209

Earlier quoted context omitted.

Well for a start, it assumes good faith on the part of the participants, rather than the default assumption of bad faith and corruption you'd like to project. There are also built-in controls in the form of reviews and appeals. And more generally, humans are squishy and imprecise, trying to apply precise, inflexible, code-like law to immensely analog situations is not a recipe for good outcomes.

Why should anyone assume good faith about or trust any part of the government? It is well known, studied and documented that justice is not uniform in practice. Yes, ideally it is not, but we do not live in some abstract utopia where judges are not corrupt. It's so odd that people would say that it is a feature that judges are inconsistent. Juries are one thing, but judges driving their own agendas independent of law…

> Why should anyone assume good faith about or trust any part of the government?

Because we’re not all paranoid libertarians? I dunno man, going through life with that attitude seems like a recipe for unhappiness and frustration.

It’s not about judges being inconsistent. It’s about providing good judgements, which are appropriate to the specific circumstances.

Re: GPT-5 outperforms federal judges in legal reasoning experiment

#218

Post by krapp »

Nursie wrote: post #217
fluidcruft wrote: post #210

Earlier quoted context omitted.

Why should anyone assume good faith about or trust any part of the government? It is well known, studied and documented that justice is not uniform in practice. Yes, ideally it is not, but we do not live in some abstract utopia where judges are not corrupt. It's so odd that people would say that it is a feature that judges are inconsistent. Juries are one thing, but judges driving their own agendas independent of law…

> Why should anyone assume good faith about or trust any part of the government? Because we’re not all paranoid libertarians? I dunno man, going through life with that attitude seems like a recipe for unhappiness and frustration. It’s not about judges being inconsistent. It’s about providing good judgements, which are appropriate to the specific circumstances.

>going through life with that attitude seems like a recipe for unhappiness and frustration.

Going through life in blissful ignorance of the incompetence and malice driving the systems of violence and control around you is even worse. Unhappiness and frustration are necessary prerequisites to any improvements to quality of life. Terminally happy people are just grazing cattle, fit only for the slaughterhouse.

Re: GPT-5 outperforms federal judges in legal reasoning experiment

#219

Post by Nursie »

krapp wrote: post #218
Nursie wrote: post #217

Earlier quoted context omitted.

> Why should anyone assume good faith about or trust any part of the government? Because we’re not all paranoid libertarians? I dunno man, going through life with that attitude seems like a recipe for unhappiness and frustration. It’s not about judges being inconsistent. It’s about providing good judgements, which are appropriate to the specific circumstances.

>going through life with that attitude seems like a recipe for unhappiness and frustration. Going through life in blissful ignorance of the incompetence and malice driving the systems of violence and control around you is even worse. Unhappiness and frustration are necessary prerequisites to any improvements to quality of life. Terminally happy people are just grazing cattle, fit only for the slaughterhouse.

> Going through life in blissful ignorance of the incompetence and malice driving the systems of violence and control around you is even worse.

I’m not ignorant of that, I disagree that it matches reality.

And see that’s what I’m talking about. There’s no reasoned view of the world here just unthinking, unfocused vitriol.

> Happy people are just grazing cattle, fit only for the slaughterhouse.

Yet here I live in a stable democracy with a historically unprecedented standard of living. It’s not perfect, but the idea that judges should not use judgement and compassion in the application of the law just seems nuts. It’s a human system for humans, not some branch of mathematics. :shrug:

Re: GPT-5 outperforms federal judges in legal reasoning experiment

#220

Post by krapp »

Nursie wrote: post #219
krapp wrote: post #218

Earlier quoted context omitted.

>going through life with that attitude seems like a recipe for unhappiness and frustration. Going through life in blissful ignorance of the incompetence and malice driving the systems of violence and control around you is even worse. Unhappiness and frustration are necessary prerequisites to any improvements to quality of life. Terminally happy people are just grazing cattle, fit only for the slaughterhouse.

> Going through life in blissful ignorance of the incompetence and malice driving the systems of violence and control around you is even worse. I’m not ignorant of that, I disagree that it matches reality. And see that’s what I’m talking about. There’s no reasoned view of the world here just unthinking, unfocused vitriol. > Happy people are just grazing cattle, fit only for the slaughterhouse. Yet here I live in a st…

The reason you live in a stable democracy with a historically unprecedented standard of living is that generations of unhappy and frustrated people made it so. It certainly isn't thanks to the people satisfied with the status quo, who maintained faith in either the virtue of the Church or the Crown. Progress depends on unreasonable people doing unreasonable things like killing monarchs and nailing theses to church doors.

>It’s not perfect, but the idea that judges should not use judgement and compassion in the application of the law just seems nuts.

I agree with you. They should. Absolutism in terms of the law reduces to fascism, and even the "code is law" crowd discover religion as soon as they realize code can have loopholes just as laws can. But we shouldn't assume by default that the courts will act fairly, because they won't, they will act in their own interests as all power structures do, and fairly only when fairness isn't a threat to those interests.

For the same reason we shouldn't assume software created by humans and controlled by the those very same power structures would be any better.

Post reply on HN