Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI outperforms law professors in Stanford Law study

law.stanford.edu

Post reply on HN
Print view View original

131–140 of 384 posts

Re: AI outperforms law professors in Stanford Law study

#131

Post by toss1 »

b40d-48b2-979e wrote: post #122
Merad wrote: post #118

Earlier quoted context omitted.

> Some things seem harmless, i.e. drafting a will Absolutely not harmless if you're the executor of an estate forced to deal with a screwed up AI will. I just handler my dad's estate this spring. It's a frustrating and confusing process even with the simplest of estates.

Most people don't have anything that could even be called an "estate".

Ummm, not quite.

An "estate" is a legal term for property, assets, and liabilities a person leaves behind upon their death. A family member is a top practitioner in the field of estate planning and resolution, and some of the messiest estates they have handled are pro-bono cases of exactly the type of people you would put in italicized "most people": poor, not really able to upkeep a house they inherited from a relative which hadn't had title properly transferred on a previous death because they didn't have money for an attny, now can't get a loan to fix the roof...

Yeah, if you are homeless, carless, and have only the clothes on your back and a shopping cart of stuff, you don't have an estate. Everyone in the middle class in the US has an estate. Much of the time it passes automatically to their spouse on death, but it's still an estate.

And if you are concerned about where it goes, get a GOOD attny. There are many bad ones hanging out their shingle as "Trust & Estate" attnys, and some of the next messiest cases are fixing problems made by those not-so-good attnys.

And NO, AI is not good enough.

Re: AI outperforms law professors in Stanford Law study

#132

Post by grogenaut »

_heimdall wrote: post #79
causal wrote: post #7

As a software engineer I have some intuition for what the risks are of letting agents do some tasks vs others. I don't have a similar intuition calibrated for what could go wrong when asking AI to draft a legal document. Some things seem harmless, i.e. drafting a will, but I don't really know- our legal system is notoriously rife with footguns.

I wouldn't consider drafting a will to be harmless. If its done poorly the next of kin could have to deal with a huge headache and potentially months or years of probate proceedings.

I had a very well crafted will from my parents, one of whom was a very good lawyer hiring other good lawyers. It was still a pain in the ass for many of the reasons they were trying to make it easy for us.

One thing I learned, just bite the bullet and re-write the whole fucking will instead of making riders.

Piecing the will together from riders was terrible. Al the clauses fell away everyone got older. The final will could have been 8 pretty clear pages.

The other part that is hard is just knowing all of the things that happen with assets and a passing. Luckily we had another lawyer and financial folks to advise us. It was still a lot and not that easy to find details. This was pre-ai that would have helped walk through his shit.

Re: AI outperforms law professors in Stanford Law study

#133

Post by jatora »

zuzululu wrote: post #48
jatora wrote: post #34

Earlier quoted context omitted.

definitely not needed if you're in the middle-man slime trades (law)

what do you think software devs do all day

create value and utility without purposely gatekeeping and hamstringing society

Re: AI outperforms law professors in Stanford Law study

#134

Post by conception »

causal wrote: post #7

As a software engineer I have some intuition for what the risks are of letting agents do some tasks vs others. I don't have a similar intuition calibrated for what could go wrong when asking AI to draft a legal document. Some things seem harmless, i.e. drafting a will, but I don't really know- our legal system is notoriously rife with footguns.

This is why I can’t see how college grads are going to survive the AI apocalypse. domain experts driving LLMs are super powerful because they can spot where they make mistakes. Juniors don’t have that insight and the LLMs then cost them productivity.

Re: AI outperforms law professors in Stanford Law study

#136

Post by vessenes »

signatoremo wrote: post #69
34981t wrote: post #2

He is basically an AI professor for law. This study just confirms his existence: https://juliannyarko.com/ Stanford and its donors of course want to replace anyone but its administrators, so they cheer on such anti-intellectual nonsense.

This is the state of HN. Created new account. Accused without evidence. Emotional clickbait.

I vibe coded hn10k earlier this year. You could choose to see pages with comments only started by 1k+, 10k+ or 100k+ karma contributors. I'm too lazy to keep it up, but I found 1k and 10k both to be better experiences than "vanilla".

Re: AI outperforms law professors in Stanford Law study

#137

Post by apparent »

galaxyLogic wrote: post #105

I'm going to need some legal help for my startup. But I can't pay much. So I figured I will ask AI all relevant questions, as well as forms filled etc. Perhaps even create a patent-application for me. THEN I find a human lawyer and give AI's answers to them and say "Can you find any errors in this? Can you improve it?" . That way I think my legal bills should be smaller because the AI has already done most of the wor…

I think that within a few years, most lawyers will expect that clients will have run contracts through an LLM prior to sending them to outside counsel. Emails will be along the lines of:

Please see attached contract we received from [counterparty]. ChatGPT says blah, blah and blah should be revised. What do you think? Is there anything else that we should change?

Re: AI outperforms law professors in Stanford Law study

#139

Post by lern_too_spel »

OhioMan2943 wrote: post #76

Library outperforms student... more news at 9

This was an open book test. The real problem with this study is that winning the most head-to-head preference tests is not the right metric. It doesn't much matter if two answers are right, and one is written a little better than the other. It matters quite a lot if one answer is right and another is wrong.

The authors point out that this other metric was computed in prior work and incorrectly dismiss it as being not as good as winning percentage in head to head competitions. The cited prior work shows that the models fare poorly on that metric. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=5166938

Re: AI outperforms law professors in Stanford Law study

#140

Post by galaxyLogic »

apparent wrote: post #137
galaxyLogic wrote: post #105

I'm going to need some legal help for my startup. But I can't pay much. So I figured I will ask AI all relevant questions, as well as forms filled etc. Perhaps even create a patent-application for me. THEN I find a human lawyer and give AI's answers to them and say "Can you find any errors in this? Can you improve it?" . That way I think my legal bills should be smaller because the AI has already done most of the wor…

I think that within a few years, most lawyers will expect that clients will have run contracts through an LLM prior to sending them to outside counsel. Emails will be along the lines of: Please see attached contract we received from [counterparty]. ChatGPT says blah, blah and blah should be revised. What do you think? Is there anything else that we should change?

Right. That will reduce workload for the lawyers. But will their fees then go down? I'm kinda worried that if I don't give them the LLM produced legal docs for review they will just use the LLM themselves and then charge me for the work the LLM did :-)

It's bit like with doctors, you'll want a second opinion, if you can afford it.

Post reply on HN