Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT Takes the Bar Exam

github.com

Post reply on HN
Print view View original

61–70 of 147 posts

Re: GPT Takes the Bar Exam

#61

Post by ilaksh »

lui8906 wrote: post #39
hilbertseries wrote: post #30

Earlier quoted context omitted.

I don’t really know to what extent AI is going to take over. But crypto is a Terri comparison, it didn’t really solve any problems. There’s nothing you can do with a blockchain that’s you couldn’t already do before. The potential with Chatgpt if it can be refined is wild.

- I can send someone 1 million USDC for 0.000001 US cent with confirmation in Couple examples that have come up recently for me personally. I can agree that there is something comparable (i.e. international wire) but the cost and speed is in a different ballpark and international wires are not available to some parts of the world.

Which chain and service is that?

Re: GPT Takes the Bar Exam

#62

Post by mrjin »

lm28469 wrote: post #46
ranting-moth wrote: post #38

Earlier quoted context omitted.

I think ~100 years is an overestimate by an order of magnitude. I know that people like to bash GPT-3 and ChatGPT for not being perfect. But people like to bash everything, especially things they don't understand. Technology tends to grow much closer to an exponential speed than linear. What ChatGPT can do today is truly, utterly, astonishing. That's the new baseline.

It's been at least 70 years since AI is supposed to replace everything a human can do. It's still mostly shit at basically everything besides playing chess and go, I don't think 100 years is unrealistic, especially given the other very important issues we face and will face as humanity

My gut still tells me you might be too optimistic about it. Sometimes it's just like a function f(x)=1/x. f(x) gets closer and closer to 0 when x grows, but it can never reach it in a finite scope.

Re: GPT Takes the Bar Exam

#63

Post by BoxOfRain »

teekert wrote: post #43
lordnacho wrote: post #28

Earlier quoted context omitted.

The AI is not a person in the sense that a person is a person. You see, when a person makes a judgement, you judge the judgement based on personal qualities: did they look like they were paying attention during the case, did they say things that were well reasoned, do they have the right experience? If we put an AI in the role of the judge/attorneys, we will come up against "ah but the robot didn't do this thing that…

Maybe good to realize that judge/jury system is not how it's done everywhere. For example, here in the Netherlands, there are just lawyers and a judge. There is much less emotional swaying of peers (for better or for worse) the focus is on objectivity according to the law. No saying it's better or worse, but maybe more suited to AI assistants? That said, I'm sure ChatGPT can make emotional appeals.

I quite like jury trials personally, they're a good mechanism as an authentic people's check on officialdom in my opinion. A similar concept could also serve as a good check on AI by humans I think!

Re: GPT Takes the Bar Exam

#64

Post by Karellen »

lordnacho wrote: post #28
vasco wrote: post #16

Potentially in ~100 years you could have two AIs sort out the cases. If you can't afford a really good AI to defend you, you can use the public-defender-AI which is trained on the same dataset as the prosecutor-AI. Only involve humans on appeal. It sounds dystopian but I can't see why not to do it, in most simpler cases it's a waste of a human to repeat the same argument for the hundredth time. There's already things…

The AI is not a person in the sense that a person is a person. You see, when a person makes a judgement, you judge the judgement based on personal qualities: did they look like they were paying attention during the case, did they say things that were well reasoned, do they have the right experience? If we put an AI in the role of the judge/attorneys, we will come up against "ah but the robot didn't do this thing that…

Ah, but if we take the lessons of the Shitty Technology Adoption Curve, we just need to start by applying it against^Wwith people "you can abuse with impunity (prisoners, kids, migrants, etc) and then work their way up the privilege gradient."

https://pluralistic.net/2022/08/21/great-taylors-ghost/

So, we start by using AI to help decide immigration claims or parole reviews, which frees up real lawyers to deal with the terrible backlog of more important cases. And once it's in place there, and the results have stood long enough without complaints (from anyone who matters) that society has effectively legitimised them, we can start using the AIs to start "helping" more with other cases.

...now I have to go shower.

Re: GPT Takes the Bar Exam

#65

Post by mrjin »

spiderfarmer wrote: post #23
PartiallyTyped wrote: post #11

Earlier quoted context omitted.

Project to the next 10 years given the trends.

Like with Full Self Driving, the low hanging fruit has been picked and now we'll see that it takes considerably longer than thought to level up.

FSD was lie. This isn't any better.

Re: GPT Takes the Bar Exam

#66

Post by mrweasel »

One of the few things I can currently see GPT do well is helping find legal precedence. Rather than having five law students roam through old cases, it would make sense to feed 100+ years of legal rulings into the model and then query the AI for any previous rulings similar to your own, with the desired outcome. That would allow lawyers to state question like: In cases like this, which evidence or arguments caused the ruling to favor the accused.

You still need a human to make the case, present the arguments and adapt every to the current situation, but there's no reason to have human search through thousands of cases and write summaries, not when an AI can do it in an instance.

Re: GPT Takes the Bar Exam

#67

Post by ben_w »

lm28469 wrote: post #58
ben_w wrote: post #53

Earlier quoted context omitted.

> potential for good or bad ? Last time they tried to use AI for police/justice it ended up exacerbating existing biases Both. OpenAI are trying to be politically neutral, but it's not hard to break the rails, and someone else can and will replicate the work without those protections anyway. > What can you do with an AI you can't with humans in a court of law ? Me personally? Afford more than a fairly small case. > W…

> That's actually a great idea. The estimated cost of each query is cents, so we might very well be able to retry everyone on each update, for less than the cost of keeping them imprisoned. You're thinking like a developer and you're absolutely blind when it comes to the lives and families potentially destroyed by something like that. And that's exactly why technocrats should rarely be in charge "ah really sorry dude…

> "ah really sorry dude, we released you last year but now you're on death row, wait for the next update, if it comes before you're fried on the chair you might get a lesser sentence. The last dude forgot to uncomment a branch in a switch/case and you were flagged incorrectly. Sorry for the inconvenience xoxo"

That's the exact opposite of the scenario you previously described, as you were previously saying people could get better defences not getting better prosecutions.

I'm not legally trained so this might just be Hollywood logic, but doesn't re-prosecuting someone require a court case with new evidence?

Anyway…

> That's a regulation issues, not an AI issue.

No, it's a "you're hiring at contractor rates" problem.

Just as Stable Diffusion is giving a hard time to professional artists who earn at contractor rates by producing "it might not be amazing but it's certainly not bad" art for a dozen images per cent of electricity, ChatGPT is giving "it might BS me 5-20% of the time depending on the subject, but it's still better than I know" responses on basically everything.

It can't do every job at an expert level yet, but it's better than almost everyone's baseline for the skills they are non-expert at. (No idea how long it will take to get to superhuman even just for law, and I was over-optimistic about self-driving car AI, but we shall see).

Re: GPT Takes the Bar Exam

#68

Post by ilaksh »

mrjin wrote: post #62
lm28469 wrote: post #46

Earlier quoted context omitted.

It's been at least 70 years since AI is supposed to replace everything a human can do. It's still mostly shit at basically everything besides playing chess and go, I don't think 100 years is unrealistic, especially given the other very important issues we face and will face as humanity

My gut still tells me you might be too optimistic about it. Sometimes it's just like a function f(x)=1/x. f(x) gets closer and closer to 0 when x grows, but it can never reach it in a finite scope.

It's especially like that for people who have a dualistic worldview.

"Substance dualism is a philosophical position compatible with most theologies which claim that immortal souls occupy an independent realm of existence distinct from that of the physical world."

Re: GPT Takes the Bar Exam

#69

Post by xiphias2 »

ben_w wrote: post #49
xiphias2 wrote: post #17

Earlier quoted context omitted.

The success rate is not what’s impressive in itself. It’s the speed of improving on the test. We can expect it to get to 90% soon, and that will have real world impact (replacing lawyers for easy advice questions).

> We can expect it to get to 90% soon I won't be surprised , but that's less than "can expect", and I disagree that this is straightforward to forecast… unless you're currently playing with another similar model that hasn't been published and which can do this. As the saying goes, "forecasting is hard, especially if it's about the future". AI progress has always been this weird combination of two sides, one saying fo…

Generally when AI gets good on a specific task, it doesn't stop.

What's more common though is that even though it gets good on that task, it may not translate to practical real world application.

The best example I can think of is object detection vs self driving: lot of people though that the improvements in object detection on images will easily translate to great self driving, and here we are, still with cars not stopping when a car is blinking in front of it.

Re: GPT Takes the Bar Exam

#70

Post by alfiedotwtf »

vasco wrote: post #16

Potentially in ~100 years you could have two AIs sort out the cases. If you can't afford a really good AI to defend you, you can use the public-defender-AI which is trained on the same dataset as the prosecutor-AI. Only involve humans on appeal. It sounds dystopian but I can't see why not to do it, in most simpler cases it's a waste of a human to repeat the same argument for the hundredth time. There's already things…

> Potentially in ~100 years you could have two AIs sort out the cases.

Marty: Within two hours of his arrest, Martin McFly Jr was tried, convicted, and sentenced to 15 years in the state penitentiary.

Marty: Within two hours?

Doc: The justice system works swiftly now that they've abolished lawyers.

Marty: This is heavy

Post reply on HN