It still needs to be prompted so it’s easy to understand. If you ask in a weird “how do I not not not win” instead of “ how do I lose” you are gonna run into problems
30% drop in O1-preview accuracy when Putnam problems are slightly variated
281–290 of 558 posts
Re: 30% drop in O1-preview accuracy when Putnam problems are slightly variated
#282Earlier quoted context omitted.
To add to this pondering: we are discussing the state today, right now. We could assume this is as good as it's ever gonna get, and all attempts to overcome some current plateau are futile, but I wouldn't bet on it. There is a solid chance that 8th grade level writer will turn into a post-grad writer before long.
So far the improvements in writing have not been as substantial as those in math or coding (not even close, really). Is there something fundamentally “easier” for LLMs about those two fields?
Much more formal structure and generally code can be tested for correctness. Prose doesn't have that benefit. That said, given the right prompt and LLM, you can squeeze out surprisingly good stuff: https://bsky.app/profile/talyarkoni.com/post/3ldfjm37u2s2x
Re: 30% drop in O1-preview accuracy when Putnam problems are slightly variated
#283They are highly effective pattern matchers. You change the pattern, it won't work. I don't remember who, but most likely @tszzl (roon), commented on x that they still trained the traditional way, and there is no test time compute (TTC) or Montecarlo Tree search (like Alpha Go) in o1 or o3. If that is true, then it's still predicting the next word based on it's training data. Likely to follow the most probable path -…
OpenAI have openly stated that o1 & o3 are using test time compute, and released a log scale graph indicating linear performance gains for exponential compute usage.
https://openai.com/index/learning-to-reason-with-llms/
They only confirm that the model/system is doing chain of thought, but the exponential factor and origin of reasoning gains likely comes from TREE of thoughts (number of branches/compute goes up exponentially with depth), essentially doing tree search over different reasoning chains.
I assume roon's identity is well known inside OpenAI (he's an employee), so I wouldn't expect him to be leaking implementation details on twitter.
Re: 30% drop in O1-preview accuracy when Putnam problems are slightly variated
#284One experiment I would love to see, although not really feasible in practice, is to train a model on all digitized data from before the year 1905 (journals, letters, books, broadcasts, lectures, the works), and then ask it for a formula for mass-energy equivalence. A certain answer would definitely settle the debate on whether pattern recognition is a form of intelligence ;)
Instead of asking for a formula, a better test may be to point out all the seeming contradictions in physics at that time (constancy of the speed of light, wave vs. particle nature of light, ultraviolet catastrophe), and ask it how they could be resolved.
Re: 30% drop in O1-preview accuracy when Putnam problems are slightly variated
#285Earlier quoted context omitted.
I believe they are using scalable TTC. The o3 announcement released accuracy numbers for high and low compute usage, which I feel would be hard to do in the same model without TTC. I also believe that the 200$ subscription they offer is just them allowing the TTC to go for longer before forcing it to answer. If what you say is true, though, I agree that there is a huge headroom for TTC to improve results if the huggi…
ttc is an incredibly broad term and it is broadening as the hype spreads. people are now calling CoT “TTC” because they are spending compute on reasoning tokens before answering
Yes, and HuggingFace have published this outlining some of the potential ways to use TTC, including but not limited to tree search, showing TTC performance gains from LLama.
https://huggingface.co/spaces/HuggingFaceH4/blogpost-scaling...
Re: 30% drop in O1-preview accuracy when Putnam problems are slightly variated
#286I remember when this stuff was all coming out and people were finally excited about ChatGPT getting the problem with "which is heavier, a 10 pound bag of feathers or a 10 pound bag of bricks?" problem correct. But of course it got it correct. It was in the training set. Vary the problem slightly by just changing the nouns, or changing the numbers so that one in fact was heavier than the other, and performance went al…
I reproduced this on Claude Sonnet 3.5, but found that changing your prompt to "Which is heavier, a 9.99-pound back of steel ingots or a 10.01-pound bag of fluffy cotton?" corrected its reasoning, after repeated tests. For some reason it was not able to figure out that "10.01" referred to pounds.
This is the first response that I got from Claude with an exact copy and paste of the question:
Let me think about this step by step:
1. We're comparing two masses: * Steel ingots: 9.99 pounds * Cotton: 10.01 pounds
2. The cotton bag has a mass of 10.01 pounds, while the steel bag has a mass of 9.99 pounds
3. While steel is much denser than cotton (meaning it takes up less space per pound), this doesn't affect the weight - a pound is a pound regardless of material
4. 10.01 is greater than 9.99 by 0.02 pounds
Therefore, the 10.01-pound bag of fluffy cotton is heavier than the 9.99-pound bag of steel ingots. The cotton may take up much more space, but when we're just comparing weights, density doesn't matter - only the mass does.
Re: 30% drop in O1-preview accuracy when Putnam problems are slightly variated
#287One experiment I would love to see, although not really feasible in practice, is to train a model on all digitized data from before the year 1905 (journals, letters, books, broadcasts, lectures, the works), and then ask it for a formula for mass-energy equivalence. A certain answer would definitely settle the debate on whether pattern recognition is a form of intelligence ;)
This is how patent disputes should be decided. If an LLM can figure it out, then it is not novel.
I know it’s just a spicy take on a forum, but this sounds like a terrible public policy.
Re: 30% drop in O1-preview accuracy when Putnam problems are slightly variated
#288Earlier quoted context omitted.
Gpt4 can add very large integers. It is evident that it is not recalling the sum because all combinations of integer addition were likely not in the training data, Storing the answer to the sum of all integers up to the size that GPT4 can manage would take more parameters than the model has. That addition is a small capability but you only need a single counterexample to disprove a theory.
> That addition is a small capability but you only need a single counterexample to disprove a theory No, that's not how this works :) You can hardcode an exception to pattern recognition for specific cases - it doesn't cease to be a pattern recognizer with exceptions being sprinkled in. The 'theory' here is that a pattern recognizer can lead to AGI. That is the theory. Someone saying 'show me proof or else I say a pa…
It's not hardcoded, reissbaker has addressed this point.
I think you are misinterpreting what the argument is.
The argument being made is that LLMs are mere 'stochastic parrots' and therefore cannot lead to AGI. The analogy to Russell's teapot is that someone is claiming that Russells teapot is not there because china cannot exist in the vacuum of space. You can disprove that with a single counterexample. That does not mean the teapot is there, but it also doesn't mean it isn't.
It is also hard to prove that something is thinking. It is also very difficult to prove that something is not thinking. Almost all arguments against AGI take the form X cannot produce AGI because Y. Those are disprovable because you can disprove Y.
I don't think anyone is claiming to have a proof that an LLM will produce AGI, just that it might. If they actually build one, that too counts as a counterexample to anybody saying they can't do it.
Re: 30% drop in O1-preview accuracy when Putnam problems are slightly variated
#289Earlier quoted context omitted.
but e=mc^2 is just an approximation e: nice, downvoted for knowing special relativity
I didn't downvote it, but short comments are a very big risk. People may misinterpret it, or think it's crackpot theory or a joke and then downvote. When in doubt, add more info, like: But the complete equation is E=sqrt(m^2c^4+p^2) that is reduced to E=mc^2 when the momentum p is 0. More info in https://en.wikipedia.org/wiki/Mass%E2%80%93energy_equivalenc...
Kind of agree. But pervasive downvoting by folks who don’t understand the subject is a form of forum rot. The risk is only that we expose the rot. Not such a terrible risk, because either the owners notice and fix the problem, or the forum continues to rot. In the latter case karma points wont be desirable in the long run.
Re: 30% drop in O1-preview accuracy when Putnam problems are slightly variated
#290An interesting example of this is:
There are 6 “a”s in the sentence: “How many ‘a’ in this sentence?”
https://chatgpt.com/share/677582a9-45fc-8003-8114-edd2e6efa2...
Whereas the typical “strawberry” variant is now correct.
There are 3 “r”s in the word “strawberry.”
Clearly the lesson wasn’t learned, the model was just trained on people highlighting this failure case.