To the “LLMs just interpolate their training data” crowd: Ayer, and in a different way early Wittgenstein, held that mathematical truths don’t report new facts about the world. Proofs unfold what is already implicit in axioms, definitions, symbols, and rules. I think that idea is deeply fascinating, AND have no problem that we still credit mathematicians with discoveries. So either “recombining existing material” isn…
I'm not sure how feasible this is, but I love the thought experiment of limiting a training set to a certain time period, then seeing how much hinting it takes for the model to discover things we already know. E.g. training on physics knowledge prior to 1915, then attempting to get from classical mechanics to general relativity.
An OpenAI model has disproved a central conjecture in discrete geometry
311–320 of 1001 posts
Re: An OpenAI model has disproved a central conjecture in discrete geometry
#312Earlier quoted context omitted.
> I think that idea is deeply fascinating, AND have no problem that we still credit mathematicians with discoveries. Most discoveries are indeed implied from axioms, but every now and then, new mathematics is (for lack of a better word) "created"—and you have people like Descartes, Newton, Leibniz, Gauss, Euler, Ramanujan, Galois, etc. that treat math more like an art than a science. For example, many belive that to…
Creation is done by humans who have been trained on the data of their life experiences. Nothing new is being created, just changing forms. A scientist has to extract the "Creation" from an abstract dimension using the tools of "human knowledge". The creativity is often selecting the best set of tools or recombining tools to access the platonic space. For instance a "telescope" is not a new creation, it is recombinati…
[dead]
Re: An OpenAI model has disproved a central conjecture in discrete geometry
#313The proof brings unexpected, sophisticated ideas from algebraic number theory to bear on an elementary geometric question. The more I read about these achievements the more I get a feeling that a lot of the power of these models comes from having prior knowledge on every possible field and having zero problems transferring to new domains. To me the potential beauty of this is that these tools might help us break thro…
Yep. The thing is people (maybe because of our limited scope) just focus on the depth and not the breadth. Because this is a general purpose model - it also has PhD+ knowledge in Physics, Biology, History, etc.
I think we still don't really comprehend how much can be achieved by a single "mind" that has internalized so much knowledge from so many areas.
Re: An OpenAI model has disproved a central conjecture in discrete geometry
#314To the “LLMs just interpolate their training data” crowd: Ayer, and in a different way early Wittgenstein, held that mathematical truths don’t report new facts about the world. Proofs unfold what is already implicit in axioms, definitions, symbols, and rules. I think that idea is deeply fascinating, AND have no problem that we still credit mathematicians with discoveries. So either “recombining existing material” isn…
As others have pointed out, both can be true:
* LLMs do just interpolate their training data, BUT-
* That can still yield useful "discoveries" in certain fields, absent the discovery of new mechanics that exist outside said training data
In the case of mathematics, LLMs are essentially just brute-forcing the glorified calculators they run on with pseudo-random data regurgitated along probabilities; in that regard, mathematics is a perfect field for them to be wielded against in solving problems!
As for organic chemistry, or biology, or any of the numerous fields where brand new discoveries continue happening and where mathematics alone does not guarantee predicted results (again, because we do not know what we do not know), LLMs are far less useful for new discoveries so much as eliminating potential combinations of existing data or surfacing overlooked ones for study. These aren't "new" discoveries so much as data humans missed for one reason or another - quack scientists, buried papers, or just sheer data volume overwhelming a limited populace of expertise.
For further evidence that math alone (and thus LLMs) don't produce guaranteed results for an experiment, go talk to physicists. They've been mathematically proving stuff for decades that they cannot demonstrably and repeatedly prove physically, and it's a real problem for continued advancement of the field.
Re: An OpenAI model has disproved a central conjecture in discrete geometry
#315Earlier quoted context omitted.
"new kind of math" Well I think the point is there is no "new kind of math". There's just types of math we've discovered and what we haven't. No new math is created, just found.
The map is not the territory.
I don't know what you're even trying to argue here.
We're not comparing math to reality (though there's a strong argument to be made that reality has a structure that is mathematical in nature - structural realism didn't die a scientific philosophy just because someone came up with a pithy saying), we're talking about if math is discovered or invented.
Most mathematicians would argue both - math is a language, we have created operations, axioms are proposed based on human creativity, etc., but the actual laws, patterns, etc. are discovered. Pi is going to be pi no matter if you're a human or someone else - we might represent it differently with some other number system or whatever, but that's a matter of representation, not mathematical truth.
Re: An OpenAI model has disproved a central conjecture in discrete geometry
#316Earlier quoted context omitted.
I'd hope most functional adults understand that the Fields Medal and basically every other annual "prize" out there is awarded to both "recombinant" innovations and "new-dimensional thinking" innovations. Humans aren't going to come up with "new-dimensional" innovations in every field, every single year. I'd say yes, LLMs "just" recombine things. I still don't think if you trained an LLM with every pre-Newton/Liebniz…
To keep my usual rant short: I think you’re assuming a categorical distinction between those two types of innovations that just doesn’t exist. Calculus certainly required some fundamental paradigm shifts, but there’s a reason that they didn’t have to make up many words wholesale to explain it! Also we shouldn’t be thinking about what LLMs are good at, but rather what any computer ever might be good at. LLMs are alrea…
The fundamental paradigm shift is the categorical distinction. And what would constitute many new words for you? It introduced a bunch of concepts and terms which we take for granted today, including "derivative", "integral", "infinitesimal", "limit" and even "function", the latter two not a new words, but what does it matter? – the associated meanings were new.
Re: An OpenAI model has disproved a central conjecture in discrete geometry
#317Few questions that the blog did not answer, if anyone knows that'll be great:
- Does anyone know if this was a 1 minute of inference or 1 month?
- How many times did the model say it was done disproving before it was found out that the model was wrong/hallucinating?
- One of the graphs say - the model produced the right answer almost half the times at the peak compute??? did i understand that right? what does peak compute mean here?
Re: An OpenAI model has disproved a central conjecture in discrete geometry
#318Earlier quoted context omitted.
What's your basis for assuming LLM is capable of doing this? I honestly don't know personally either way. Based on my limited understanding of how LLMs work, I don't see them be making the next great song or next great book and based on that reasoning I'm betting that it probably wont be able to do whatever next "Descartes, Newton, Leibnitz, Gauss, Euler, Ramanujan, Galois" are going to do. Of course AI as a wider fi…
"I don't see them be making the next great song" Meanwhile, songs are hitting number one on some charts on Spotify that people think are humans and are actually AI. And Spotify has to start labelling them as such. One AI "band" had an entire album of hits. Also - music is a subjective. Mathematics isn't. And in this case, an LLM discovered a new way to reason about a conjecture. I don't know how much proof is needed…
>> Meanwhile, songs are hitting number one on some charts on Spotify that people think are humans and are actually AI. And Spotify has to start labelling them as such. One AI "band" had an entire album of hits.
There is quite some questions around that. Music is subjective and obviously different people have different taste, but I wouldn't call any of them to be actual good music / real hits.
>> LLM discovered a new way to reason about a conjecture
I wasn't questioning LLMs ability to prove things. Parent threads were talking about building new kind of maths , or approaching it in a creative/artistic way. Thats' what I was referring to.
I can't speak for maths of hard science as I'm not trained in that, but the creativity aspect in code is definitely lacking when it comes to LLMs. May not matter down the line.
Re: An OpenAI model has disproved a central conjecture in discrete geometry
#319To the “LLMs just interpolate their training data” crowd: Ayer, and in a different way early Wittgenstein, held that mathematical truths don’t report new facts about the world. Proofs unfold what is already implicit in axioms, definitions, symbols, and rules. I think that idea is deeply fascinating, AND have no problem that we still credit mathematicians with discoveries. So either “recombining existing material” isn…
"LLMs just interpolate their training data" Cracks me up. What exactly do we think that human brains do?
[deleted]
Re: An OpenAI model has disproved a central conjecture in discrete geometry
#320Earlier quoted context omitted.
How are you going to train a frontier level llm with no references to post 1700 mathematics?
"frontier level" is doing a lot of work there, but the idea would be to only feed it earlier sources. There are people working on this. e.g. https://github.com/haykgrigo3/TimeCapsuleLLM
The problem is the amount of data with that cutoff is really minuscule to produce anything powerful. You might be able to generate a lot of 1700s sounding data, you’d have to be careful not to introduce newer concepts or ways of thinking in that synthetic data though. A lot of modern texts talk about rates of change and the like in ways that are probably influenced by preexisting knowledge of calculus.