GPT Takes the Bar Exam

It's a bit odd to see the supplementary data repository linked, rather than the actual paper [1] (HN discussion [2]).

[1]: https://arxiv.org/abs/2212.14402

[2]: https://news.ycombinator.com/item?id=34216239

I would also note that the paper only covers the multiple choice portion of a bar exam, the Uniform Bar Examination (UBE) that has been adopted by most, but not all states. The UBE consists of a multiple choice portion (the Multistate Bar Exam, or MBE), an essay portion and a scenario-based performance test. GPT-3.5 gets a 50% success rate on a practice version of the MBE. It's impressive, but I wouldn't go so far as to say that it's imminent that AIs will threaten lawyers.

RIP lawyers...

And doctors, writers and programmers...

I'd be more interested in seeing the minimum training for a human + chatGPT to pass the bar.