Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT Takes the Bar Exam

github.com

Post reply on HN
Print view View original

111–120 of 147 posts

Re: GPT Takes the Bar Exam

#111

Post by vasilipupkin »

flanked-evergl wrote: post #106
mrweasel wrote: post #66

One of the few things I can currently see GPT do well is helping find legal precedence. Rather than having five law students roam through old cases, it would make sense to feed 100+ years of legal rulings into the model and then query the AI for any previous rulings similar to your own, with the desired outcome. That would allow lawyers to state question like: In cases like this, which evidence or arguments caused th…

> One of the few things I can currently see GPT do well is helping find legal precedence. GPT is really not good at finding anything, and actually it can't find anything. It is not designed to do that. It is designed to make things up. The difference may be subtle, but it is quite important. GPT makes up APIs, citations, tools, papers and rules. It makes up anything really. What it makes up is incredibly plausible, a…

I keep seeing comments along these lines. It doesn’t “make things up”. It outputs text under the objective function of maximizing probability of the token being outputted as “making sense” on some criteria. So, actually, it is biased towards not making things up. Occasionally, it hallucinates. I suspect we’ll see subsequent versions hallucinate less and less.

Re: GPT Takes the Bar Exam

#112

Post by Sander_Marechal »

vasco wrote: post #16

Potentially in ~100 years you could have two AIs sort out the cases. If you can't afford a really good AI to defend you, you can use the public-defender-AI which is trained on the same dataset as the prosecutor-AI. Only involve humans on appeal. It sounds dystopian but I can't see why not to do it, in most simpler cases it's a waste of a human to repeat the same argument for the hundredth time. There's already things…

> for example EU flight compensation (companies like https://www.airhelp.com/en-int/)

I really don't like companies like that. EU flight compensation is really easy. Airlines have a simple webform that you fill out, and you get your money 1-2 weeks later. Companies like AirHelp make it sound like a really hard and difficult process, then take a big cut of your compensation as payment. And the SEO the crap out of it, drowning out the real claim webforms from the airlines themselves.

Re: GPT Takes the Bar Exam

#113

Post by PartiallyTyped »

spiderfarmer wrote: post #23
PartiallyTyped wrote: post #11

Earlier quoted context omitted.

Project to the next 10 years given the trends.

Like with Full Self Driving, the low hanging fruit has been picked and now we'll see that it takes considerably longer than thought to level up.

Which is why I pointed 10 years ahead, and not next 3 or 5 years. I have made similar comments before regarding low hanging fruits, and GANs, and then diffusion became a thing.

Re: GPT Takes the Bar Exam

#114

Post by intelVISA »

vasco wrote: post #16

Potentially in ~100 years you could have two AIs sort out the cases. If you can't afford a really good AI to defend you, you can use the public-defender-AI which is trained on the same dataset as the prosecutor-AI. Only involve humans on appeal. It sounds dystopian but I can't see why not to do it, in most simpler cases it's a waste of a human to repeat the same argument for the hundredth time. There's already things…

GPT-3 is great at producing nonsense, so in that sense it'd be a good lawyer.

Re: GPT Takes the Bar Exam

#115

Post by willyt »

mrweasel wrote: post #66

One of the few things I can currently see GPT do well is helping find legal precedence. Rather than having five law students roam through old cases, it would make sense to feed 100+ years of legal rulings into the model and then query the AI for any previous rulings similar to your own, with the desired outcome. That would allow lawyers to state question like: In cases like this, which evidence or arguments caused th…

But then how do the humans coming up through the system gain the breadth of knowledge and experience required to carry out the more complicated senoir roles.

Re: GPT Takes the Bar Exam

#116

Post by Uristqwerty »

vasco wrote: post #16

Potentially in ~100 years you could have two AIs sort out the cases. If you can't afford a really good AI to defend you, you can use the public-defender-AI which is trained on the same dataset as the prosecutor-AI. Only involve humans on appeal. It sounds dystopian but I can't see why not to do it, in most simpler cases it's a waste of a human to repeat the same argument for the hundredth time. There's already things…

Lawyers don't exist to decide who wins a case. It's all about making the other side show their work, prove that they have valid evidence, it was collected legally, etc. An AI might be able to find flaws in the work of a sloppy human who wasn't expecting opposition, but will it know when to stop grasping at increasingly-inane straws? If it develops a reputation for subtly misrepresenting certain laws when pressed to find some answer, after all the easy options have run out, wouldn't it be kicked out of the courtroom for wasting the legal system's time, unless you have a team of programmers on standby to keep writing output filters to censor it from repeating its mistakes? Human lawyers have to fix their behaviour or lose their licenses when caught repeating mistakes, malice, or misleading practices, I think, and current AI paradigms have immense trouble learning from small datasets.

Re: GPT Takes the Bar Exam

#117

Post by taink »

BlueTemplar wrote: post #94
teekert wrote: post #43

Earlier quoted context omitted.

Maybe good to realize that judge/jury system is not how it's done everywhere. For example, here in the Netherlands, there are just lawyers and a judge. There is much less emotional swaying of peers (for better or for worse) the focus is on objectivity according to the law. No saying it's better or worse, but maybe more suited to AI assistants? That said, I'm sure ChatGPT can make emotional appeals.

Sure, but until computers get citizenship (and maybe even then), it will be illegal to use non-algorithmic programs for decision-making, because it runs afoul of the laws that mandate that you must be able to explain how the program helped you to came to a decision (tracing the algorithm step by step) in language that your average person can understand.

Maybe the program could detail its decision process if it is sufficiently advanced

Re: GPT Takes the Bar Exam

#118

Post by bryankaplan »

vasilipupkin wrote: post #111
flanked-evergl wrote: post #106

Earlier quoted context omitted.

> One of the few things I can currently see GPT do well is helping find legal precedence. GPT is really not good at finding anything, and actually it can't find anything. It is not designed to do that. It is designed to make things up. The difference may be subtle, but it is quite important. GPT makes up APIs, citations, tools, papers and rules. It makes up anything really. What it makes up is incredibly plausible, a…

I keep seeing comments along these lines. It doesn’t “make things up”. It outputs text under the objective function of maximizing probability of the token being outputted as “making sense” on some criteria. So, actually, it is biased towards not making things up. Occasionally, it hallucinates. I suspect we’ll see subsequent versions hallucinate less and less.

So what you're saying is it makes things up.

Re: GPT Takes the Bar Exam

#119

Post by lm28469 »

sirsinsalot wrote: post #83
lm28469 wrote: post #60

Earlier quoted context omitted.

> AI can .... drive a car Yeah no. Everything in your list should be prefixed by "kinda", and "kinda" doesn't really cut it in most of these situations. They certainly are good tools but it's very far from what it promised, even decades ago. > still be in denial AI bros are in denial since 1950. With ifs and buts you can write compelling arguments for anything, in the meantime reality is a thing and virtually none of…

My mother can "kinda" drive.

And that's exactly why she's not the taxi driver of everyone on earth

Re: GPT Takes the Bar Exam

#120

Post by imtringued »

ilaksh wrote: post #54
lm28469 wrote: post #46

Earlier quoted context omitted.

It's been at least 70 years since AI is supposed to replace everything a human can do. It's still mostly shit at basically everything besides playing chess and go, I don't think 100 years is unrealistic, especially given the other very important issues we face and will face as humanity

AI can play chess, Go, Jeopardy, StarCraft, Minecraft, Atari games, write essays, summarize text, answer almost any question, drive a car, do above average on a SAT test, fool many people into wondering on Twitter if there were real people typing answers, write chapters of books, act as a dungeon master, write code to spec, win programming competitions, act as a therapist, fold proteins, provide medical diagnosis, vi…

Ok then get rid of all humans, they aren't needed anymore.
Post reply on HN