Earlier quoted context omitted.
Humans are going to want to do the worst parts of the job?
Humans used to go in coal mines.
They still do. But that job hasn't gotten worse, it's always been the same level of suck.
GPT-4 gets a B on my quantum computing final exam
81–90 of 261 posts
Re: GPT-4 gets a B on my quantum computing final exam
#82This is pretty incredible. I guess the thing that we have to keep in mind is that we really can't compete with an AI that understands natural language this well and also has the entirety of the internet as its reference materials. Maybe we need to accept that or rework our definition of what makes humans special
What this technology has shown is that is basically superhuman in the _amount_ of «expert level knowledge» it has. Not completely sure if we have said that what makes human special is the amount knowledge an individual can acquire. What we still see lacking, and I would wager will still pertain if we keep just training on the internet, is that it is not good at reasoning to generate _new_ knowledge. It is quite able…
quite, augment, not replace
Re: GPT-4 gets a B on my quantum computing final exam
#83This is obviously very cool, but at this point — who knows what I’ll say in a year — my concern with these LLMs is that they’re in the uncanny valley. Here’s one passing a very difficult test. Amazing! Now, rely on it to build a nuclear doohickey for a power station or a multi-billion dollar device for CERN or anything really and, well, no. So humans still have to check the output, and now we’re in that situation whe…
>No negativity towards AI here. It’s amazing and it’ll change the future. But we need to be careful on the way. Yeah, I suspect a lot of fields will have a similar trajectory to how AI has impacted radiology. It might catch the tumor in 99.9999% of cases, better than any human doctor. But missing a malignant tumor 0.0001% of the time is unacceptable, because it spikes the hospital's malpractice costs. So every single…
Even with the 1 in 10000 false negative rate, I bet someone is doing the cost calculation of risk vs how many hours it would take for a doctor to check 10000 scans. Doctors themselves are not perfect so they may even have a higher error rate.
Re: GPT-4 gets a B on my quantum computing final exam
#84Earlier quoted context omitted.
The enrollment at UT Austin is ~50K. No way you can grade that many tests by hand. Source: I went to UT for grad school and was a TA.
Well, if a uni with 5k enrolled can hand grade tests, then obviously UT Austin could, if they had 10x faculty of the first uni.
The more you spend on faculty the less you can spend on sports.
Re: GPT-4 gets a B on my quantum computing final exam
#85Earlier quoted context omitted.
It’s impressive and surprising. Imagine going back to the year 2021 and telling the good people of HN that in 2023 AI would be as advanced like it is now. Literally no one would have believed you in 2021 if you did.
The knee-jerk contrarianism seems to be in vogue right now. Reminds me of some colleagues in 2007 lamenting about the iPhone, "It's not that great; it doesn't even have copy & paste!" Fast forward a few years, and no blackberries in sight.
Oh God, when the iPhone came out people were raging "But it doesn't have a keyboard!"
It's so hard for people, including myself, to look at the future potential of a technology. My attitude is to simply keep an open mind nowadays instead of holding very strong opinions about the trajectory of a given technology.
Re: GPT-4 gets a B on my quantum computing final exam
#86This is obviously very cool, but at this point — who knows what I’ll say in a year — my concern with these LLMs is that they’re in the uncanny valley. Here’s one passing a very difficult test. Amazing! Now, rely on it to build a nuclear doohickey for a power station or a multi-billion dollar device for CERN or anything really and, well, no. So humans still have to check the output, and now we’re in that situation whe…
>No negativity towards AI here. It’s amazing and it’ll change the future. But we need to be careful on the way. Yeah, I suspect a lot of fields will have a similar trajectory to how AI has impacted radiology. It might catch the tumor in 99.9999% of cases, better than any human doctor. But missing a malignant tumor 0.0001% of the time is unacceptable, because it spikes the hospital's malpractice costs. So every single…
And the class action lawsuit will be orchestrated by an AI
Re: GPT-4 gets a B on my quantum computing final exam
#87When I was growing up in the 2000s, it was required to learn a foreign language in school. I took Spanish but dropped the class after a year, thinking that when I grew up, computers would be able to translate text far better humans could. Would you believe it, transformers were invented ten years later. I wonder if it's worth learning anything anymore.
Cars exist but we still run and walk.
Re: GPT-4 gets a B on my quantum computing final exam
#88It's impressive on the surface... But give me Google and I too can pass some extremely challenging tests that I actually know nothing about.
If someone told you I passed the "bar" exam, you would be impressed. If they then said, he had google and lots of time. You wouldn't be that impressed anymore.
The impressive thing here is that AI can read and answer questions... it's not overly impressive that it can use information found online and reconstruct that a bit.
Re: GPT-4 gets a B on my quantum computing final exam
#89This is obviously very cool, but at this point — who knows what I’ll say in a year — my concern with these LLMs is that they’re in the uncanny valley. Here’s one passing a very difficult test. Amazing! Now, rely on it to build a nuclear doohickey for a power station or a multi-billion dollar device for CERN or anything really and, well, no. So humans still have to check the output, and now we’re in that situation whe…
> So humans still have to check the output, and now we’re in that situation where humans driving a Tesla on autopilot who are supposed to be 100% aware of the road aren’t, because they get lazy and doze off, and now the car crashes and whoops. The only question is whether, on average, AI-augmented code suggestions reduce overall bug rates. Human devs are already dozing off with crashes and whoopsies at high rates so…
I don't think it's as simple as a mere bug rate comparison. When a human makes a whoopsie, we blame that human. Maybe it's a bit uncomfortable, but it it works to prevent them from cooperating with attempts to inject whoopsies into their code.
If we create a culture of tolerating fatal AI hallucinations as long as they happen at a rate that's below the human caused fatality baseline, we create an opportunity for bad actors to use the "nobody is to blame" window as a plausible deniability weapon (so long as they don't use it too often).
Re: GPT-4 gets a B on my quantum computing final exam
#90Earlier quoted context omitted.
I agree with the point you're making here, but it’s also funny that the description of someone passing a test but not being able to do much without a lot of human supervision is… exactly the description of a human college graduate.
Can anyone answer the chance that example tests of these questions were in its training set? And it's just regurgitating the answers someone else wrote? As I imagine it's a very high chance given how much uni lecturers recycle exam questions. When I was at uni you could just get the last 5 years worth of questions from the library for almost any subject and guess what the questions were probably going to be. Often th…
I don’t know anything about quantum computing. I successfully answered a few true/false questions by pasting them into Google.
The exact questions aren’t in the top results, but the answers are.