Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT Takes the Bar Exam

github.com

Post reply on HN
Print view View original

141–147 of 147 posts

Re: GPT Takes the Bar Exam

#141

Post by mminer237 »

quanticle wrote: post #5
quanticle wrote: post #3

It's a bit odd to see the supplementary data repository linked, rather than the actual paper [1] (HN discussion [2]). [1]: https://arxiv.org/abs/2212.14402 [2]: https://news.ycombinator.com/item?id=34216239

I would also note that the paper only covers the multiple choice portion of a bar exam, the Uniform Bar Examination (UBE) that has been adopted by most, but not all states. The UBE consists of a multiple choice portion (the Multistate Bar Exam, or MBE), an essay portion and a scenario-based performance test. GPT-3.5 gets a 50% success rate on a practice version of the MBE. It's impressive, but I wouldn't go so far as…

Yeah, I tried it out with the MEE[1] it did not come close to a "50%" equivalent there.

[1]: https://news.ycombinator.com/item?id=34320270

Re: GPT Takes the Bar Exam

#142

Post by naveen99 »

ilaksh wrote: post #54
lm28469 wrote: post #46

Earlier quoted context omitted.

It's been at least 70 years since AI is supposed to replace everything a human can do. It's still mostly shit at basically everything besides playing chess and go, I don't think 100 years is unrealistic, especially given the other very important issues we face and will face as humanity

AI can play chess, Go, Jeopardy, StarCraft, Minecraft, Atari games, write essays, summarize text, answer almost any question, drive a car, do above average on a SAT test, fool many people into wondering on Twitter if there were real people typing answers, write chapters of books, act as a dungeon master, write code to spec, win programming competitions, act as a therapist, fold proteins, provide medical diagnosis, vi…

AI can’t play ping pong, soccer, basketball, fold clothes, clean up a mess. Also, it’s not allowed to accumulate or allocate capital, has no legal standing.

Also, there is entirely too much hype on gpt and it’s unwieldy nature and not enough on bert and actually useful tools.

Re: GPT Takes the Bar Exam

#143

Post by NoPicklez »

naveen99 wrote: post #142
ilaksh wrote: post #54

Earlier quoted context omitted.

AI can play chess, Go, Jeopardy, StarCraft, Minecraft, Atari games, write essays, summarize text, answer almost any question, drive a car, do above average on a SAT test, fool many people into wondering on Twitter if there were real people typing answers, write chapters of books, act as a dungeon master, write code to spec, win programming competitions, act as a therapist, fold proteins, provide medical diagnosis, vi…

AI can’t play ping pong, soccer, basketball, fold clothes, clean up a mess. Also, it’s not allowed to accumulate or allocate capital, has no legal standing. Also, there is entirely too much hype on gpt and it’s unwieldy nature and not enough on bert and actually useful tools.

I asked GPT and it gave multiple examples of where AI has played ping pong.

Re: GPT Takes the Bar Exam

#144

Post by NoPicklez »

fullstackchris wrote: post #12
smrtinsert wrote: post #6

The next ten years will be very interesting.

If you're alluding to the notion "AI will take ALL the jobs!", well, here I am, still building websites for well established companies that still don't (in 2023!!!) have a web presence. Hell, half the worlds banking system still runs on COBOL! Leading edge tech development is rapid, sure. But tech understanding, adoption, and implementation is painfully slow. Humans gon' be humans. It's like the same thing every 5 ye…

The problem is looking at these technologies in absolutes and thinking "this is the end", "massive change", "game changer" etc.

If these AI technologies help to do even 50% of low skilled tasks, that's still huge.

Re: GPT Takes the Bar Exam

#145

Post by NoPicklez »

carapace wrote: post #127

I think the machines should do the scut work to free up humans to do the really important human stuff, like parenting, teaching, and, yeah, lawyering and judging too. Another angle: if there is some legal task that's simple enough for machines to do reliably then (almost by definition) that task will turn out to be pointless bureaucratic busywork, eh? Last but not least, who gets to keep lawyer-bot's pay? - - - - edi…

This is what I have been saying.

If these tools can do even 50% of mundane, rudimentary tasks, that's still huge in of itself and allows people to do more important work.

Think of moving from manual book research, to Google. Rather than having to go to the library and find a particular book regarding something I want to find, I can just search Google and trawl through websites. Now with GPT I potentially don't need to trawl through websites, I can ask the AI and use my judgement as to the validity of the output, I could then use that answer to inform when I fact check etc.

It's not terrible, but it's also not the best thing since sliced bread.

Re: GPT Takes the Bar Exam

#146

Post by spiderfarmer »

aflag wrote: post #134
spiderfarmer wrote: post #25

Earlier quoted context omitted.

I'd still trust a lawyer that used ten different laptops better than just one laptop.

Would you trust more an intern consulting 10 lawyers about your case instead of any single lawyer? Assuming a world where AI has surprassed human ability to make a case, having a human component would just make it worse.

If it makes a case that is too difficult for a human to comprehend I wouldn't make that case in front of a judge.

Re: GPT Takes the Bar Exam

#147

Post by aflag »

spiderfarmer wrote: post #146
aflag wrote: post #134

Earlier quoted context omitted.

Would you trust more an intern consulting 10 lawyers about your case instead of any single lawyer? Assuming a world where AI has surprassed human ability to make a case, having a human component would just make it worse.

If it makes a case that is too difficult for a human to comprehend I wouldn't make that case in front of a judge.

Not a matter of comprehension, not of what's written anyway. But comprehension of how what is in there is the hardest case that can be made (among many possibilities), which the other side will have the most difficulty in making a case against. Also, a better lawyer knows which causes are lost and which ones are worth fighting for so that the client has the lowest sentence.

If a bad lawyer sees the case from 10 good ones, he might pick and choose and make ultimately a worse case, or take one of the good ones and just repeat what they said. Which is, in the best case scenario, just as good as having that single lawyer.

Post reply on HN