Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT scores 80% on SAT reading/writing with collective chain of thought

old.reddit.com

Post reply on HN
Print view View original

61–70 of 126 posts

Re: ChatGPT scores 80% on SAT reading/writing with collective chain of thought

#61

Post by shaunsingh0207 »

thaumasiotes wrote: post #58
shaunsingh wrote: post #22

Earlier quoted context omitted.

What I find more concerning is the 100% scores that ChatGPT has been able to achieve on AP CSA and APUSH, among others. It makes sense to teach 3rd graders 2+2, but if college board is creating "college level courses" with problems that can be easily solved through whats still a fairly new technology, we need to reevaluate what we're teaching in schools. So many AP courses are just memorization and identifying formul…

> if college board is creating "college level courses" with problems that can be easily solved through whats still a fairly new technology, we need to reevaluate what we're teaching in schools. I don't get it. AP calculus is problems that can be easily solved with what is now very old technology. The point of an AP test isn't that the problems are so difficult.

These specific subjects shouldn't be solely about memorization, rather they should teach kids how to link that information together to solve problems (or in the case of history, avoid them in the future).

Thats the issue. The point of an AP test shouldn't be that the problems are difficult. But currently thats what it's all about. They're just trying to test that kids are capable of surviving a "rigorous academic curriculum".

[1] https://apcentral.collegeboard.org/about-ap/ap-a-glance

Re: ChatGPT scores 80% on SAT reading/writing with collective chain of thought

#62

Post by shaunsingh0207 »

fdr wrote: post #27
shaunsingh wrote: post #22

Earlier quoted context omitted.

What I find more concerning is the 100% scores that ChatGPT has been able to achieve on AP CSA and APUSH, among others. It makes sense to teach 3rd graders 2+2, but if college board is creating "college level courses" with problems that can be easily solved through whats still a fairly new technology, we need to reevaluate what we're teaching in schools. So many AP courses are just memorization and identifying formul…

The AP CSA pass surprises me (I've never taken it myself). I had this thought because I tried to use chatgpt for work programming stuff and it got stuff wrong very frequently. So frequently, and to a great enough degree, that I had to keep re-circumscribing what kind of problems were worth my time to throw at it, and now it's a tiny cross-section.

Thats what surprises me as well. If I give chatGPT anything that requires actual thought or reasoning put into it, it just produces gibberish.

It's great for getting a blueprint to start from. I recently had to write a gtk+rust client, and I asked chatGPT to give me some starter code which largely worked.

All the chatGPT hype feels the same to me as the copilot hype from a while back. The people who got the most excited about it are the same people who its going to replace.

Re: ChatGPT scores 80% on SAT reading/writing with collective chain of thought

#64

Post by loehnsberg »

I have been testing it extensibly in the past weeks. It is a great tool for writing. I had it solve a data science exam that I give to my students. It passed with flying banners. It even got close in the parts that involved calculations and math. My take is that it can safely help you rewrite, rephrase, and even come up with new content based on the context you provide. However, the latter is risky if you don‘t know or check the facts, as it it generates content based on what it deems plausible. Plausible does not mean correct. Just have it dig out a list of references to support a thesis and you will see, it just makes stuff up. I am sure that it will get better, but it is dangerous, as long as I cannot be sure that it somehow let’s me know what it does not know.

Re: ChatGPT scores 80% on SAT reading/writing with collective chain of thought

#66

Post by trompetenaccoun »

Kiro wrote: post #65
hmate9 wrote: post #63

It’s interesting how HN likes to hate on GitHub copilot but loves chatgpt.

HN seems to hate both, and technology in general.

Being critical doesn't necessarily equal hate. It's not surprising a forum for coders would have people criticize and not just rave about tech like people who do not understand how it works and to whom it's basically magic.

Re: ChatGPT scores 80% on SAT reading/writing with collective chain of thought

#67

Post by ekianjo »

shreyshnaccount wrote: post #45
sublinear wrote: post #4

I think this is evidence against the usefulness of standardized tests rather than support for ChatGPT

There is a leap in this logic- going from the fact that SAT is solvable by a LLM to the conclusion that the SAT is useless. Questions about addition can be solved using calculators, does that mean that teaching addition is useless? Also there is an underlying assumption that memorization is a useless skill, which I'd argue against, as in today's world most innovation is done by identifying patterns between discipline…

not so much memorization as decomposition of numerous examples into logic based knowledge. As a simple example, knowing volumes of words is not as useful as knowing etymology since etymology brings you a deeper structure.

Re: ChatGPT scores 80% on SAT reading/writing with collective chain of thought

#68

Post by mrtksn »

This is like Coca-Cola making the Covid-19 tests positive.

Cola making the test positive doesn't mean that the test is fake or that the Cola has virus and ChatGPT scoring something on SAT doesn't mean it's equivalent to a person scoring the same and should study in Stanford.

These tests are designed to pick people to limited number of spots and doesn't say much more than the persons' preparedness for the test.

Re: ChatGPT scores 80% on SAT reading/writing with collective chain of thought

#69

Post by ekianjo »

fifilura wrote: post #36
shaunsingh wrote: post #22

Earlier quoted context omitted.

What I find more concerning is the 100% scores that ChatGPT has been able to achieve on AP CSA and APUSH, among others. It makes sense to teach 3rd graders 2+2, but if college board is creating "college level courses" with problems that can be easily solved through whats still a fairly new technology, we need to reevaluate what we're teaching in schools. So many AP courses are just memorization and identifying formul…

There is one argument against this though that is somehow brewing in my brain. The end goal is to internalize the subject so that you can extrapolate your own insights. "This happened in WWII so if these conditions starts to appear it risks to happen again". Just to take history as an example. All other subjects have similar examples. And to reach that kind of insight you need to start filling the brain with informat…

> This happened in WWII so if these conditions starts to appear it risks to happen again". Just to take history as an example. All other subjects have similar examples

History is a particular bad case to drive such example since there are thousands of factors that are always different and we have very poor data beyond a few centuries so theres that. With History you are much more likely to end up with confirmation bias than actual understanding.

Re: ChatGPT scores 80% on SAT reading/writing with collective chain of thought

#70

Post by aflag »

29athrowaway wrote: post #15
dieselgate wrote: post #7

Maybe ChatGPT should host an AMA

ChatGPT is an AMA, just that the responses are not shared. Anyone can host an AMA on behalf of ChatGPT by just serving as a proxy for the questions and answers.

That’s true about any person you talk with, to be fair.
Post reply on HN