Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT Takes the Bar Exam

github.com

Post reply on HN
Print view View original

11–20 of 147 posts

Re: GPT Takes the Bar Exam

#12

Post by fullstackchris »

smrtinsert wrote: post #6

The next ten years will be very interesting.

If you're alluding to the notion "AI will take ALL the jobs!", well, here I am, still building websites for well established companies that still don't (in 2023!!!) have a web presence. Hell, half the worlds banking system still runs on COBOL!

Leading edge tech development is rapid, sure. But tech understanding, adoption, and implementation is painfully slow. Humans gon' be humans.

It's like the same thing every 5 years now, last time it was crypto "end of inequality!" "bank the poor!" "same global currency!" sorry to be skeptical, but I didn't see the quantum leap with crypto, and I don't see the quantum leap with GPT - what are essentially fancy NLP models

Re: GPT Takes the Bar Exam

#13

Post by fullstackchris »

PartiallyTyped wrote: post #11
fullstackchris wrote: post #8

Earlier quoted context omitted.

Did you look at the results? All models that were tested failed significantly (10%+) to acheive the passing range.

Project to the next 10 years given the trends.

So I'm going to put a laptop on the stand when I go to trial? Give me a break.

Re: GPT Takes the Bar Exam

#15

Post by PartiallyTyped »

fullstackchris wrote: post #13
PartiallyTyped wrote: post #11

Earlier quoted context omitted.

Project to the next 10 years given the trends.

So I'm going to put a laptop on the stand when I go to trial? Give me a break.

Nowhere near what I said.

Companies have hordes of lawyers scouring through contracts, e.g. think associates and paralegals (which are not technically lawyers). Expect that a non trivial portion will get axed.

Re: GPT Takes the Bar Exam

#16

Post by vasco »

Potentially in ~100 years you could have two AIs sort out the cases. If you can't afford a really good AI to defend you, you can use the public-defender-AI which is trained on the same dataset as the prosecutor-AI. Only involve humans on appeal. It sounds dystopian but I can't see why not to do it, in most simpler cases it's a waste of a human to repeat the same argument for the hundredth time.

There's already things like this without AI to sort out for example EU flight compensation (companies like https://www.airhelp.com/en-int/), parking tickets (https://www.appwinit.com/) and probably more that I don't know about. All of these are complete waste of time to actually have humans deal with, and there's many more small crimes that could work the same way, also to make prosecutor offices more efficient and less biased hopefully.

Re: GPT Takes the Bar Exam

#17

Post by xiphias2 »

quanticle wrote: post #5
quanticle wrote: post #3

It's a bit odd to see the supplementary data repository linked, rather than the actual paper [1] (HN discussion [2]). [1]: https://arxiv.org/abs/2212.14402 [2]: https://news.ycombinator.com/item?id=34216239

I would also note that the paper only covers the multiple choice portion of a bar exam, the Uniform Bar Examination (UBE) that has been adopted by most, but not all states. The UBE consists of a multiple choice portion (the Multistate Bar Exam, or MBE), an essay portion and a scenario-based performance test. GPT-3.5 gets a 50% success rate on a practice version of the MBE. It's impressive, but I wouldn't go so far as…

The success rate is not what’s impressive in itself. It’s the speed of improving on the test. We can expect it to get to 90% soon, and that will have real world impact (replacing lawyers for easy advice questions).

Re: GPT Takes the Bar Exam

#18

Post by aflag »

fullstackchris wrote: post #13
PartiallyTyped wrote: post #11

Earlier quoted context omitted.

Project to the next 10 years given the trends.

So I'm going to put a laptop on the stand when I go to trial? Give me a break.

If it could build a case better than a human, you would. But currently it's nowhere close to that

Re: GPT Takes the Bar Exam

#20

Post by heywhatupboys »

vasco wrote: post #16

Potentially in ~100 years you could have two AIs sort out the cases. If you can't afford a really good AI to defend you, you can use the public-defender-AI which is trained on the same dataset as the prosecutor-AI. Only involve humans on appeal. It sounds dystopian but I can't see why not to do it, in most simpler cases it's a waste of a human to repeat the same argument for the hundredth time. There's already things…

While HN often have over the top, "computers will solve everything" takes, your comment is why I keep coming here. What a great fantasy, which at least is a good discussion.

We see this already now i high frequency trading, where companies essentially battle trading bots against each other

Post reply on HN