GPT-4 gets a B on my quantum computing final exam
scottaaronson.blog
GPT-4 gets a B on my quantum computing final exam
1–10 of 261 posts
Re: GPT-4 gets a B on my quantum computing final exam
#2True/false questions on an exam seem a bit lazy, is that common?
Re: GPT-4 gets a B on my quantum computing final exam
#3True/false questions on an exam seem a bit lazy, is that common?
In my country, yes. In fact because the True/False were there, or to circle the correct answer, I was in this way given a hint of what the correct answer is. Had I needed to think of it, I wouldn't have managed to graduate high school.
Full disclose, I graduated with something close to a D, but in actuality we are graded from 2(F) to 6(A) and my HS diploma says 3.97. And it is largely fictitious, in reality I was closer to 3 and less.
Re: GPT-4 gets a B on my quantum computing final exam
#4True/false questions on an exam seem a bit lazy, is that common?
Multiple choice and true false questions were fairly common in my undergraduate experience, particularly in first year courses. Often they were Scantron forms for maximum laziness.
Re: GPT-4 gets a B on my quantum computing final exam
#5This is pretty incredible. I guess the thing that we have to keep in mind is that we really can't compete with an AI that understands natural language this well and also has the entirety of the internet as its reference materials. Maybe we need to accept that or rework our definition of what makes humans special
Re: GPT-4 gets a B on my quantum computing final exam
#6When I was growing up in the 2000s, it was required to learn a foreign language in school. I took Spanish but dropped the class after a year, thinking that when I grew up, computers would be able to translate text far better humans could.
Would you believe it, transformers were invented ten years later.
I wonder if it's worth learning anything anymore.
Re: GPT-4 gets a B on my quantum computing final exam
#7True/false questions on an exam seem a bit lazy, is that common?
Multiple choice and true false questions were fairly common in my undergraduate experience, particularly in first year courses. Often they were Scantron forms for maximum laziness.
I never saw a scantron at my college.
I thought UTAustin was an elite school.
Re: GPT-4 gets a B on my quantum computing final exam
#8When I was growing up in the 2000s, it was required to learn a foreign language in school. I took Spanish but dropped the class after a year, thinking that when I grew up, computers would be able to translate text far better humans could. Would you believe it, transformers were invented ten years later. I wonder if it's worth learning anything anymore.
Machine translation isn't super human yet. But yes, it's probably only a few years away.
Re: GPT-4 gets a B on my quantum computing final exam
#9This is pretty incredible. I guess the thing that we have to keep in mind is that we really can't compete with an AI that understands natural language this well and also has the entirety of the internet as its reference materials. Maybe we need to accept that or rework our definition of what makes humans special
More that most undergrad colleges only teach and test regurgitation because the profs don't care about education.
Re: GPT-4 gets a B on my quantum computing final exam
#10It seems they just passed the questions in, which is totally fine but I'd just add that this gives you pretty much a lower bound of performance. I think it's worse if you try and make it output very small responses, and does a lot better if it can output a lot (chain of thought). I'm continually astonished at how well gpt4 can handle problems with literally no additional work, and how much you can improve it with some small help.
Edit I asked question 1m but said it was an expert in QM and was there to explain the answer to test questions followed by answering them and it gets it right with a page or so of proof (please don't attack my use of the word proof, I'm a lay man - this also explains my failure to fix the numbering, latex editing on a phone is not my forte). I've not checked it, but it gets the right answer.
https://figshare.com/articles/preprint/GPT4_answers_a_QM_que...