Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4 gets a B on my quantum computing final exam

scottaaronson.blog

Post reply on HN
Print view View original

11–20 of 261 posts

Re: GPT-4 gets a B on my quantum computing final exam

#11

Post by ben_w »

cypress66 wrote: post #8
pubby wrote: post #6

When I was growing up in the 2000s, it was required to learn a foreign language in school. I took Spanish but dropped the class after a year, thinking that when I grew up, computers would be able to translate text far better humans could. Would you believe it, transformers were invented ten years later. I wonder if it's worth learning anything anymore.

Machine translation isn't super human yet. But yes, it's probably only a few years away.

It might not be super human in the sense of beating the C2 standard, but it sure knows more German than I do.

And Spanish, and Greek, and Esperanto, and… well, everything.

But I've been really trying with that German.

Re: GPT-4 gets a B on my quantum computing final exam

#12

Post by pclmulqdq »

An LLM being able to pass your graduate-level exam almost certainly means that your exam is bad.

Lots of lazy professors do things like this (true/false and multiple choice answers, very soft questions, etc.), and the presence of GPTs should help them understand that this is not sufficient for evaluating someone's knowledge of a highly technical topic.

Re: GPT-4 gets a B on my quantum computing final exam

#13

Post by jakeinspace »

hgsgm wrote: post #9
kaesar14 wrote: post #5

This is pretty incredible. I guess the thing that we have to keep in mind is that we really can't compete with an AI that understands natural language this well and also has the entirety of the internet as its reference materials. Maybe we need to accept that or rework our definition of what makes humans special

More that most undergrad colleges only teach and test regurgitation because the profs don't care about education.

I took the graduate version of this course, and thankfully did better on this exam than GPT-4. It was a fairly rigorous course, and getting an A required about 20 hours per week of work.

Re: GPT-4 gets a B on my quantum computing final exam

#14

Post by CrazyStat »

hgsgm wrote: post #7
TheCycoONE wrote: post #4

Earlier quoted context omitted.

Multiple choice and true false questions were fairly common in my undergraduate experience, particularly in first year courses. Often they were Scantron forms for maximum laziness.

I never saw a scantron at my college. I thought UTAustin was an elite school.

University prestige is primarily based on research productivity, which doesn't necessarily correlate positively with quality of teaching.

Re: GPT-4 gets a B on my quantum computing final exam

#16

Post by kaesar14 »

hgsgm wrote: post #9
kaesar14 wrote: post #5

This is pretty incredible. I guess the thing that we have to keep in mind is that we really can't compete with an AI that understands natural language this well and also has the entirety of the internet as its reference materials. Maybe we need to accept that or rework our definition of what makes humans special

More that most undergrad colleges only teach and test regurgitation because the profs don't care about education.

Fwiw Scott Aaronson is maybe the most well respected researcher in this field. I don't think this is a case of some shitty professor asking his students to regurgitate useless information

Re: GPT-4 gets a B on my quantum computing final exam

#17

Post by timy2shoes »

hgsgm wrote: post #7
TheCycoONE wrote: post #4

Earlier quoted context omitted.

Multiple choice and true false questions were fairly common in my undergraduate experience, particularly in first year courses. Often they were Scantron forms for maximum laziness.

I never saw a scantron at my college. I thought UTAustin was an elite school.

The enrollment at UT Austin is ~50K. No way you can grade that many tests by hand.

Source: I went to UT for grad school and was a TA.

Re: GPT-4 gets a B on my quantum computing final exam

#18

Post by MengerSponge »

pubby wrote: post #6

When I was growing up in the 2000s, it was required to learn a foreign language in school. I took Spanish but dropped the class after a year, thinking that when I grew up, computers would be able to translate text far better humans could. Would you believe it, transformers were invented ten years later. I wonder if it's worth learning anything anymore.

If you don't want to leave home? No. You don't need to learn anything anymore. It does help, though, to learn the basic abilities of the most commonly played heroes and a few standard item/ability builds.

But if you want to travel? Go to Mexico City or Madrid? Automated translation is better than nothing, but it's not at all the same as communicating fluently with someone.

Re: GPT-4 gets a B on my quantum computing final exam

#19

Post by opdahl »

kaesar14 wrote: post #5

This is pretty incredible. I guess the thing that we have to keep in mind is that we really can't compete with an AI that understands natural language this well and also has the entirety of the internet as its reference materials. Maybe we need to accept that or rework our definition of what makes humans special

What this technology has shown is that is basically superhuman in the _amount_ of «expert level knowledge» it has. Not completely sure if we have said that what makes human special is the amount knowledge an individual can acquire. What we still see lacking, and I would wager will still pertain if we keep just training on the internet, is that it is not good at reasoning to generate _new_ knowledge. It is quite able to compare etc, or give examples but you can clearly see that its just in a way regurgitating some information with a twist.

Hopefully we can find a way to use this technology to enhance the parts that do make us human, whatever your definition of that is.

Re: GPT-4 gets a B on my quantum computing final exam

#20

Post by kaesar14isazero »

kaesar14 wrote: post #5

This is pretty incredible. I guess the thing that we have to keep in mind is that we really can't compete with an AI that understands natural language this well and also has the entirety of the internet as its reference materials. Maybe we need to accept that or rework our definition of what makes humans special

[dead]
Post reply on HN