GPT-4 could pass bar exam, AI researchers say • Hacker News Board

Re: GPT-4 could pass bar exam, AI researchers say

#161

Post by w1nst0nsm1th » Wed, Jan 04, 2023, 11:39 AM UTC

ben_w wrote: post #157

w1nst0nsm1th wrote: post #156

Earlier quoted context omitted.

I asked it what was the xbrl taxinomy tag on us gaap for change in executive management of a SEC registered company in SEC filling and the answer doesn't fit compared to the whole xbrl taxinomy published on the SEC website. It also answered me 2 different kind of SEC form for it. It also gave me the correct url for the us gaap xbrl taxinomy on sec website. That being said, both xbrl.org and the SEC document for us ga…

It absolutely does indeed hallucinate[0] on occasion. Despite how remarkable and useful it already is, don't make the mistake of putting it unsupervised in charge of anything, as it's going to mess up at least as often as a self driving car. [0] or whatever we want to call the behaviour; also seen it called BSing (because it doesn't really know what truth is) and "mansplaining as a service"

It seems accurate for domains where it has large dataset (major programming langages like python, html, ...) to build model from.

xbrl is probably not the case as it is a very specialized domain, that is business reporting in standardized electronic format, according to specific local accounting standard, for example US GAAP in the us.

Only banks, (possibly) investment funds and accounting department in publicly traded companies, and financial regulation organisations (at least in the US) have invested that field.

This explains may be that.

Re: GPT-4 could pass bar exam, AI researchers say

#162

Post by _nalply » Wed, Jan 04, 2023, 4:23 PM UTC

Workaccount2 wrote: post #144

Aperocky wrote: post #53

Earlier quoted context omitted.

GPT has no reasoning ability, it has billions of parameters that make it pretend it has it, purely going off of previously digested material. As long as it comes across some reasoning process that have not been seen before in the training wordset, which can be as easy as a middle school math question, it fails. Because it has no ability to extrapolate logic. If it manages to pass Bar test, that says more about the Ba…

Until someone can point out the difference between neuronal reasoning and silicon reasoning, I remain completely agnostic about the underlying mechanics of whatever model. Gun to my head where I had to put money down, I would put it on "Brains are not nearly as special as we (they?) think they are." No fairy dust or supernatural beings required, brains are just another AI model (and likely not even a particularly gre…

Human brains helped humans survive for a long time.

An AI that survives that long surely has to be great. Probably you meant that human brains are not made for the world of today.

Re: GPT-4 could pass bar exam, AI researchers say

#163

Post by sebzim4500 » Wed, Jan 04, 2023, 9:36 PM UTC

petilon wrote: post #80

sebzim4500 wrote: post #66

Earlier quoted context omitted.

>As long as it comes across some reasoning process that have not been seen before in the training wordset, which can be as easy as a middle school math question Is this true even if you tell it to show its working? In my experience that drastically improves its ability to do math problems.

It can't solve quadratic equations.

Prompt: solve x^2 + 11x + 30 = 0. Show your working

Answer: To solve this equation, we can use the quadratic formula:

x = (-b +/- sqrt(b^2 - 4ac)) / (2a)

Plugging in the values for a, b, and c, we get:

x = (-11 +/- sqrt(11^2 - 4130)) / (2*1)

x = (-11 +/- sqrt(121 - 120)) / 2

x = (-11 +/- sqrt(1)) / 2

x = (-11 +/- 1) / 2

x = -5 or x = -6

So the solutions to the equation are x = -5 and x = -6.

Re: GPT-4 could pass bar exam, AI researchers say

#164

Post by dragonwriter » Wed, Jan 04, 2023, 9:52 PM UTC

> By passing this exam, lawyers are admitted to the bar of a U.S. state.

No, they aren't.

Meeting certain preparatory requirements (the details vary but in most US jurisdiction an accredited/approved law school program or, in some, what amounts to an apprenticeship with a licensed practitioner of certain duration and standards is required) and then passing the bar exam allows this.

The difference is important, the bar exam is not seen, standing alone, aa adequate proof of readiness.

Re: GPT-4 could pass bar exam, AI researchers say

#165

Post by Iwan-Zotow » Thu, Jan 05, 2023, 11:55 PM UTC

criddell wrote: post #124

Iwan-Zotow wrote: post #96

So, how new knowledge would be created? GPT has no reasoning capability. So, as time goes on, information massive(s) will be filled with GPT-X made up answers. It means GPT-X+1 will be trained on GPT-X generated data. So, without reasoning, how this thing will work in perspective?

I wouldn't assume that future versions are going to work the same way past versions did.

MAybe, maybe not.

Problem is with data/content creation. If all new data are created with GPT-3, how it will help GPT-4?

No new original content -> no new model

Re: GPT-4 could pass bar exam, AI researchers say

#166

Post by [deleted] » Fri, Jan 06, 2023, 3:42 PM UTC

booleandilemma wrote: post #133

xyzelement wrote: post #42

Earlier quoted context omitted.

Bar exam is not a test of sentience but of the ability to recall, interpret, and apply the law. Because law is an entirely textual thing, I would expect GPT to be exceedingly well suited for it. I've said for a long time that most doctors and lawyers are just databases with quick and imperfect retrieval.

And so as AI advances, the goal posts for what counts as intelligence are moved yet again.

[deleted]

Re: GPT-4 could pass bar exam, AI researchers say

#167

Post by andsoitis » Fri, Jan 06, 2023, 3:47 PM UTC

booleandilemma wrote: post #133

xyzelement wrote: post #42

Earlier quoted context omitted.

Bar exam is not a test of sentience but of the ability to recall, interpret, and apply the law. Because law is an entirely textual thing, I would expect GPT to be exceedingly well suited for it. I've said for a long time that most doctors and lawyers are just databases with quick and imperfect retrieval.

And so as AI advances, the goal posts for what counts as intelligence are moved yet again.

Maybe a useful way to think about it is that we don't know for sure what intelligence is nor how the gradient of intelligence expresses itself. For instance, does a human grow in intelligence as it ages (a baby doesn't "know" stuff, but has the capacity for learning and then applying in new situations as its experience grows).

I interpret your statement as implying that ChatGPT is somewhere on the spectrum of intelligence, yes?