11–18 of 18 posts
Earlier quoted context omitted.
GEOS (2015) scores 49% on SAT problems and it is in geometry https://www.semanticscholar.org/paper/Solving-Geometry-Probl... You were good at guessing!
SAT problems are multiple choice, with 5 options. So 50% is barely twice random guessing (1/5). See how far randomly guessing an integer 1-1000 gets you with OP's word problems with freeform responses.
Earlier quoted context omitted.
> It does not know anything. I would argue that it "knows" an awful lot, but it can't actually reason with it. However impressive GPT3 type models are, I am not particularly convinced that they're much more than glorified hashtables. If the hash table is large enough, it can produce lot of answers to a lot of questions, or approximately imitate a lot of stuff it's seen before. Whether it can actually combine "knowled…
Re: "Glorified Hastable" There is a 1-1 correspondence between data compression and generative models. GPT-2 is a highly effective loseless data compression tool: https://bellard.org/textsynth/sms.html Always wondered why this insight is not taught as much, especially in the context of things like dimensionality reduction...
> Richard, Jerry, and Robert are going to share 60 cherries. If Robert has 30 cherries, and has 10 more than Richard, how many more cherries does Robert have than Jerry? > answer: > Robert has 30 + 10 = 40 cherries. > If there are 60 cherries to be shared, then Richard and Jerry will have 60 - 40 = 20 cherries each. > Robert has 40 - 20 = 20 more cherries than Jerry. Um, the answer is "correct" but isn't the actual r…
"and" here is kind of inexact, it implies a sum, so something else. "If Robert has 30 cherries, 10 more than Richard" would be better.
> Richard, Jerry, and Robert are going to share 60 cherries. If Robert has 30 cherries, and has 10 more than Richard, how many more cherries does Robert have than Jerry? > answer: > Robert has 30 + 10 = 40 cherries. > If there are 60 cherries to be shared, then Richard and Jerry will have 60 - 40 = 20 cherries each. > Robert has 40 - 20 = 20 more cherries than Jerry. Um, the answer is "correct" but isn't the actual r…
> Richard, Jerry, and Robert are going to share 60 cherries. If Robert has 30 cherries, and has 10 more than Richard, how many more cherries does Robert have than Jerry? "and" here is kind of inexact, it implies a sum, so something else. "If Robert has 30 cherries, 10 more than Richard" would be better.
richard + jerry + robert = 60
robert = 30
robert = richard + 10
[0]: https://www.kaggle.com/c/the-allen-ai-science-challenge/over...
Earlier quoted context omitted.
GEOS (2015) scores 49% on SAT problems and it is in geometry https://www.semanticscholar.org/paper/Solving-Geometry-Probl... You were good at guessing!
SAT problems are multiple choice, with 5 options. So 50% is barely twice random guessing (1/5). See how far randomly guessing an integer 1-1000 gets you with OP's word problems with freeform responses.
“Richard, Jerry, and Robert are going to share 60 cherries. If Robert has 30 cherries, and has 10 more than Richard, how many more cherries does Robert have than Jerry?”
A rudimentary model will likely already know the answer is between 0-60.
Knowing that the answer involves addition and subtraction narrows it down to maybe 8 answers.
While SAT problems have only 4 answers, there’s usually one trick/trap answer, which I think might be be difficult for a model to not accidentally guess. The analogy I can think of is sometimes it’s better to cover up the answers first and work out a solution, to not get biased by any particular answer choice.