841–850 of 1001 posts
Buried under the drama is the fact that OpenAI is claiming that an internal model they’ve been training for less than two weeks is more than twice as capable in mathematics as Astra, which was only made public a week ago. Even if this improvement is limited to mathematics, that is an astounding feat.
Not only that, but it used 10k agents coherently over 88 hours to come up with the proof. This is a significant advance.
Earlier quoted context omitted.
Not a counterpoint per se, but I burned $50k recently on a much more modest math problem (result already known, just thought I had a sketch of a more interesting proof), and the LLM thought it had proved it within those bounds but had instead subtly fucked up the Lean definition. Take from that what you will. Not to mention, it's still very much up in the air whether the model derived the answer of its own accord or…
But the researchers also did their research using essentially the same models, so that isn’t a counterpoint to AI models being at the far frontier…
Is this the one that was allegedly based on someone else's actual work & prompts? https://news.ycombinator.com/item?id=49605915 https://bsky.app/profile/quantian.bsky.social/post/3muyhwbcd... https://cims.nyu.edu/~tristanb/statement.pdf
This really need to be a top-level story on HN..
I asked whether the model had been trained on, or had access to, our sessions in Codex, into which we had been putting all our drafts for the whole of this project. I was told the model did not look up user data. I asked again, about training, and I did not get an answer.
This whole episode is more horrific than "AI is eating math". We now have a clear and economically damaging (or at least career damaging) example of the "training on customer tokens" problem.
We can't ignore this problem any longer.
Earlier quoted context omitted.
How can you possible say this sort of thing in context of what looks like a millenium prize being solved. I swear there's nobody blinder than those who won't see.
You have to look at the incentives
How can they "not rule out" that Tristan and Levent's data was used for training?
Earlier quoted context omitted.
Hm. Looks like its possible everyone behaved terribly here unfortunately. :/ I remained impressed by ChatGPT however!
Everyone?? No, most definitely OpenAI. But they have learnt their lesson, next time they won't reach out to who they stole it from, they will publish first.
Earlier quoted context omitted.
Recursive self improvement of their upcoming IPO value maybe. They are fluffy PR pieces otherwise.
How can you possible say this sort of thing in context of what looks like a millenium prize being solved. I swear there's nobody blinder than those who won't see.
Earlier quoted context omitted.
Not a counterpoint per se, but I burned $50k recently on a much more modest math problem (result already known, just thought I had a sketch of a more interesting proof), and the LLM thought it had proved it within those bounds but had instead subtly fucked up the Lean definition. Take from that what you will. Not to mention, it's still very much up in the air whether the model derived the answer of its own accord or…
But the researchers also did their research using essentially the same models, so that isn’t a counterpoint to AI models being at the far frontier…
>> isn't a counterpoint
Where do we disagree?