I didn't know the sofa problem had been resolved. Link for anyone else: https://arxiv.org/abs/2411.19826
Discussion at the time of publication: https://news.ycombinator.com/item?id=42300382
Still not peer-reviewed
Mathematical exploration and discovery at scale
111–120 of 135 posts
Re: Mathematical exploration and discovery at scale
#112Hopefully this will finally stop the continuing claims[1] that LLMs can only solve problems they have seen before! If you listen carefully to the people who build LLMs it is clear that post-training RL forces them to develop a world-model that goes well beyond a "fancy Markov chain" that some seem to believe. Next step is building similar capabilities on top of models like Genie 3[2] [1] eg https://news.ycombinator.c…
That's not what "world-model" means: see https://en.wiktionary.org/wiki/world_model . Your [2] is equivocating in an attempt to misrepresent the state-of-the-art. Genie 3 is technically impressive, don't get me wrong, but it's strictly inferior to procedural generation techniques from the 20th century, physics simulation techniques from the 20th century, and PlayStation 2-era graphics engines. (Have you seen the char…
Inferior in what sense? Genie 3 is addressing a fundamentally different problem to a physics sim or procgen: building a good-enough (and broad-enough) model of the real world to train agents that act in the real world. Sims are insufficient for that purpose, hence the "sim2real" gap that has stymied robotics development for years.
Re: Mathematical exploration and discovery at scale
#113Earlier quoted context omitted.
all mathematicians and scientists work with a feedback loop. that's what the scientific method is.
Not one that amounts to a literal, pre-supplied objective function that's run on a computer to evaluate their outputs.
I don't know, it feels exactly like how I work
Re: Mathematical exploration and discovery at scale
#114Earlier quoted context omitted.
> it's also true that typically mathematicians fix a logic or universe before getting down to work This is true for programmers, it's not true for mathematicians. You can say programming is a subset of mathematics but mathematics is more than programming so proof search does not exhaust all the activities of a mathematician but it does exhaust all the activities of a programmer.
> it's not true for mathematicians It actually is true for mathematicians. Most of us work in ZFC. Some people choose to work in constructive logic. Some people choose to only use countable choice. But we always know what logic we're working in and which theorems we can invoke. E.g. if you choose not to accept the axiom of choice you also have a sense of which results depend on Zorn's lemma and have to work around th…
You seem to have a very limited understanding of mathematics. You're welcome to stick to it but I'd recommend learning what mathematics is actually about before thinking you can reduce it all to proof search in a single & fixed finite axiomatic system.
Re: Mathematical exploration and discovery at scale
#115> AlphaEvolve did not perform equally well across different areas of mathematics. When testing the tool on analytic number theory problems, such as that of designing sieve weights for elementary approximations to the prime number theorem, it struggled to take advantage of the number theoretic structure in the problem, even when given suitable expert hints (although such hints have proven useful for other problems). T…
Of course people don't say it, but there are many cases where reported algorithmic improvements are attributable to poor baseline tuning or shoddy statistical treatment. Tao is exhibiting a lot more epistemic humility than most researchers who probably have stronger incentives to market their work and publish.
Re: Mathematical exploration and discovery at scale
#116There is a very funny and instructive story in Section 44.2 of the paper, which I quote: Raymond Smullyan has written several books (e.g. [265]) of wonderful logic puzzles, where the protagonist has to ask questions from some number of guards, who have to tell the truth or lie according to some clever rules. This is a perfect example of a problem that one could solve with our setup: AE has to generate a code that sen…
Holy crap, this should be higher. One AI figured out it could cheat by exploiting the other AI's with a prompt injection attack!
This is reminiscent of that time agents "cheated" on coding benchmarks where the solution was leaked in the git log: https://news.ycombinator.com/item?id=45214670 -- Except that was somewhat accidental. I mean, nobody expects to be given a problem to solve with a solution right there if you looked, and indeed, the LLMs seemed to stumble upon this.
This is downright diabolical because it's an intentional prompt injection attack.
Re: Mathematical exploration and discovery at scale
#117Earlier quoted context omitted.
> it's not true for mathematicians It actually is true for mathematicians. Most of us work in ZFC. Some people choose to work in constructive logic. Some people choose to only use countable choice. But we always know what logic we're working in and which theorems we can invoke. E.g. if you choose not to accept the axiom of choice you also have a sense of which results depend on Zorn's lemma and have to work around th…
You seem to have a very limited understanding of mathematics. You're welcome to stick to it but I'd recommend learning what mathematics is actually about before thinking you can reduce it all to proof search in a single & fixed finite axiomatic system.
> You seem to have a very limited understanding of mathematics. You're welcome to stick to it but I'd recommend learning what mathematics is actually about
You'd think I'd know what mathematics is about since I have a Ph.D. in it...
> fixed finite axiomatic system
ZFC is not a finite axiomatic system. nowhere did we specify that axiomatic systems have to be finite
> single...fixed
As I said above, there are people who study different logics. However the topic of conversation is individual problems. An individual problem does not switch logics mid stream and anyone working on a problem works in the context of a logic.
Re: Mathematical exploration and discovery at scale
#118Earlier quoted context omitted.
You seem to have a very limited understanding of mathematics. You're welcome to stick to it but I'd recommend learning what mathematics is actually about before thinking you can reduce it all to proof search in a single & fixed finite axiomatic system.
> You seem to have a very limited understanding of mathematics. You're welcome to stick to it but I'd recommend learning what mathematics is actually about You'd think I'd know what mathematics is about since I have a Ph.D. in it... > fixed finite axiomatic system ZFC is not a finite axiomatic system. nowhere did we specify that axiomatic systems have to be finite > single...fixed As I said above, there are people wh…
You don't need a degree to understand mathematics but plenty of people w/ degrees think they understand what it's all about. Seems a bit odd you have a degree in mathematics but lack the basic understanding of what counts for mathematical activity. I don't think you should be proud of that but then again I don't know what they teach in PhD departments these days so maybe that's what they told you you needed to think to get the degree.
Re: Mathematical exploration and discovery at scale
#119Earlier quoted context omitted.
> You seem to have a very limited understanding of mathematics. You're welcome to stick to it but I'd recommend learning what mathematics is actually about You'd think I'd know what mathematics is about since I have a Ph.D. in it... > fixed finite axiomatic system ZFC is not a finite axiomatic system. nowhere did we specify that axiomatic systems have to be finite > single...fixed As I said above, there are people wh…
You don't need a degree to understand mathematics but plenty of people w/ degrees think they understand what it's all about. Seems a bit odd you have a degree in mathematics but lack the basic understanding of what counts for mathematical activity. I don't think you should be proud of that but then again I don't know what they teach in PhD departments these days so maybe that's what they told you you needed to think…
I see. It sounds like you're coming at this from an outsider perspective as someone who hasn't formally studied math and doesn't know what research level mathematicians are taught.
There's nothing wrong with taking an interest in a subject as an amateur. I thoroughly support it. But knowing what is taught in PhD departments and what mathematicians do is a precondition for understanding whether what AlphaEvolve is doing is similar to it.
If I wanted to know whether some AI tool was doing what humans do for, say, song writing, the first thing I'd do is figure out how song writing is taught, how professionals think about song writing, etc. And this is true regardless of the fact that I enjoy, support and often prefer outsider musicians.
Re: Mathematical exploration and discovery at scale
#120Earlier quoted context omitted.
You don't need a degree to understand mathematics but plenty of people w/ degrees think they understand what it's all about. Seems a bit odd you have a degree in mathematics but lack the basic understanding of what counts for mathematical activity. I don't think you should be proud of that but then again I don't know what they teach in PhD departments these days so maybe that's what they told you you needed to think…
I see. It sounds like you're coming at this from an outsider perspective as someone who hasn't formally studied math and doesn't know what research level mathematicians are taught. There's nothing wrong with taking an interest in a subject as an amateur. I thoroughly support it. But knowing what is taught in PhD departments and what mathematicians do is a precondition for understanding whether what AlphaEvolve is doi…
I think you're making too many assumptions. Something they should have taught you while you were getting your degree but again, I'm not sure what they teach these days b/c it's been a while since last time I was listening to a graduate lecture on mathematics in a classroom.