Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT Takes the Bar Exam

github.com

Post reply on HN
Print view View original

101–110 of 147 posts

Re: GPT Takes the Bar Exam

#101

Post by antihero »

candiodari wrote: post #79
newaccount74 wrote: post #59

Earlier quoted context omitted.

Aside from games, AI can do all these things only with human assistance. It's like saying "Excel can do your taxes". Sure, it can, if you put the correct numbers and formulas into the cells first.

You forget that a lot of these are moving the goal posts. AI beat the top players at all those games, but new rules were introduced until AI researchers mostly abandoned those efforts. For instance, the starcraft AI found a really good build order that let it quickly build "Stalkers", a versatile but relatively weak unit. I will note that that build order had several factors that humans found very surprising (consist…

Not doubting you but do you have some articles for these vast incredible use cases? Seems fascinating.

Re: GPT Takes the Bar Exam

#102

Post by espadrine »

impossiblefork wrote: post #76
fullstackchris wrote: post #8

Earlier quoted context omitted.

Did you look at the results? All models that were tested failed significantly (10%+) to acheive the passing range.

10% is actually small though, also, the latest program tested is GPT3. GPT3.5 was what really impressed people and was the point when people saw that these models could be capable.

> also, the latest program tested is GPT3. GPT3.5 was what really impressed people

The latest program shown in this chart[0] is text-davinci-003, which is part of GPT-3.5[1].

[0]: https://github.com/mjbommar/gpt-takes-the-bar-exam#progressi...

[1]: https://beta.openai.com/docs/model-index-for-researchers/mod...

Re: GPT Takes the Bar Exam

#104

Post by heywhatupboys »

blitzar wrote: post #31
heywhatupboys wrote: post #20

Earlier quoted context omitted.

While HN often have over the top, "computers will solve everything" takes, your comment is why I keep coming here. What a great fantasy, which at least is a good discussion. We see this already now i high frequency trading, where companies essentially battle trading bots against each other

> companies essentially battle trading bots against each other two computers playing heads or tails is about the extent of it

surely they are making the traders money though. The smart thing about the free markets is that companies are generally not pursuing things that are not of benefit to them

Re: GPT Takes the Bar Exam

#105

Post by dmurray »

candiodari wrote: post #79
newaccount74 wrote: post #59

Earlier quoted context omitted.

Aside from games, AI can do all these things only with human assistance. It's like saying "Excel can do your taxes". Sure, it can, if you put the correct numbers and formulas into the cells first.

You forget that a lot of these are moving the goal posts. AI beat the top players at all those games, but new rules were introduced until AI researchers mostly abandoned those efforts. For instance, the starcraft AI found a really good build order that let it quickly build "Stalkers", a versatile but relatively weak unit. I will note that that build order had several factors that humans found very surprising (consist…

> Due to cheating, in chess championships you now are "only allowed to play the moves any of the top 3 AIs would play twice in a match, after the first 5 moves".

This is such utter nonsense that I assume the rest of your post is also closer to being false than being true.

Re: GPT Takes the Bar Exam

#106

Post by flanked-evergl »

mrweasel wrote: post #66

One of the few things I can currently see GPT do well is helping find legal precedence. Rather than having five law students roam through old cases, it would make sense to feed 100+ years of legal rulings into the model and then query the AI for any previous rulings similar to your own, with the desired outcome. That would allow lawyers to state question like: In cases like this, which evidence or arguments caused th…

> One of the few things I can currently see GPT do well is helping find legal precedence.

GPT is really not good at finding anything, and actually it can't find anything. It is not designed to do that. It is designed to make things up.

The difference may be subtle, but it is quite important. GPT makes up APIs, citations, tools, papers and rules. It makes up anything really. What it makes up is incredibly plausible, as it was trained to be as plausible as possible, to the point of being correct most of the time, but it still is just making things up.

If you use it with some different expectation you will be disappointed.

Re: GPT Takes the Bar Exam

#107

Post by ranting-moth »

ctrwu2843 wrote: post #96
ranting-moth wrote: post #86

Earlier quoted context omitted.

It can be tricky to grasp exponential growth. Water trickles into a big pool. The flow was slow to start with but flow rate increases exponentially so that the amount of water in the pool doubles every year. Now, it started flowing 100 years ago when the pool was empty. It's now half full. When will it be full? Remember it's been flowing for 100 years and it's only half full.

Useful analogy but what makes you think what we're dealing with is exponential ? Human intelligence isn't growing exponentially and moores law is said to be breaking down.

https://www.kurzweilai.net/the-law-of-accelerating-returns

Re: GPT Takes the Bar Exam

#108

Post by xtracto »

qikInNdOutReply wrote: post #19

I cant wait for GPT to be added to a DAO. Imagine the dao setting objectives and the chat gpt generating the communication. Wage negotiations with a machine..

DAO as in decentralized autonomous organization? As a non USian, im not grokking that.

Re: GPT Takes the Bar Exam

#109

Post by BlueTemplar »

ben_w wrote: post #57
fullstackchris wrote: post #12

Earlier quoted context omitted.

If you're alluding to the notion "AI will take ALL the jobs!", well, here I am, still building websites for well established companies that still don't (in 2023!!!) have a web presence. Hell, half the worlds banking system still runs on COBOL! Leading edge tech development is rapid, sure. But tech understanding, adoption, and implementation is painfully slow. Humans gon' be humans. It's like the same thing every 5 ye…

> If you're alluding to the notion "AI will take ALL the jobs!", well, here I am, still building websites for well established companies that still don't (in 2023!!!) have a web presence. Hell, half the worlds banking system still runs on COBOL! One thing that I have noticed in the just-over-a-decade since Word Lens first demonstrated real time augmented reality translations on a smart phone (IIRC they were later bou…

    *The future is already here – it's just not evenly distributed.*  

    The Economist, December 4, 2003
William Gibson

Re: GPT Takes the Bar Exam

#110

Post by xtracto »

spiderfarmer wrote: post #23
PartiallyTyped wrote: post #11

Earlier quoted context omitted.

Project to the next 10 years given the trends.

Like with Full Self Driving, the low hanging fruit has been picked and now we'll see that it takes considerably longer than thought to level up.

This is important and relevant. I was into AI already in the late 1990s when symbolic AI was all the rage, and we had just passed the first AU winter. Technology like "Creatures" was again amazing people , NN also sparked new interest, but things did not progress as much as we had hoped in the last 30 years.

GPT is great, as they say, it's a great bullshit generator. It's the right direction , but it's just that.

Post reply on HN