Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An OpenAI model has disproved a central conjecture in discrete geometry

openai.com

Post reply on HN
Print view View original

351–360 of 1001 posts

Re: An OpenAI model has disproved a central conjecture in discrete geometry

#351

Post by __s »

paulddraper wrote: post #150
m-hodges wrote: post #40

To the “LLMs just interpolate their training data” crowd: Ayer, and in a different way early Wittgenstein, held that mathematical truths don’t report new facts about the world. Proofs unfold what is already implicit in axioms, definitions, symbols, and rules. I think that idea is deeply fascinating, AND have no problem that we still credit mathematicians with discoveries. So either “recombining existing material” isn…

"LLMs just interpolate their training data" Cracks me up. What exactly do we think that human brains do?

Creativity is hard. Pretty much needs a fuzzer process to generate new strings, mostly nonsense, & pick up when that nonsense happens to be correct

Re: An OpenAI model has disproved a central conjecture in discrete geometry

#352

Post by nextaccountic »

symfrog wrote: post #260
bbor wrote: post #246

Earlier quoted context omitted.

To keep my usual rant short: I think you’re assuming a categorical distinction between those two types of innovations that just doesn’t exist. Calculus certainly required some fundamental paradigm shifts, but there’s a reason that they didn’t have to make up many words wholesale to explain it! Also we shouldn’t be thinking about what LLMs are good at, but rather what any computer ever might be good at. LLMs are alrea…

We have had LLMs for much longer than 3 years.

Fine, 8 years? That's not a long time

Re: An OpenAI model has disproved a central conjecture in discrete geometry

#353

Post by beering »

ykl wrote: post #334
m-hodges wrote: post #40

To the “LLMs just interpolate their training data” crowd: Ayer, and in a different way early Wittgenstein, held that mathematical truths don’t report new facts about the world. Proofs unfold what is already implicit in axioms, definitions, symbols, and rules. I think that idea is deeply fascinating, AND have no problem that we still credit mathematicians with discoveries. So either “recombining existing material” isn…

I like to think of it as: Imagine every bit of human knowledge as a discrete point within some large high dimensional space of knowledge. You can draw a big convex hull around every single point of human knowledge in a space. A LLM, being trained within this convex hull, can interpolate between any set of existing discrete points in this hull to arrive at a point which is new, but still inside of the hull. Then there…

A lot of the space outside of the convex hull is just untried things. You can brute-force trying random things and checking the result and eventually learn something new. With a better heuristic, you can make better guesses and learn new things much more efficiently. There’s no reason to believe that kind of guess-and-check is outside of the reach of LLMs, or that most of our new discoveries are not found the same way.

Re: An OpenAI model has disproved a central conjecture in discrete geometry

#354

Post by black_knight »

dvt wrote: post #61
m-hodges wrote: post #40

To the “LLMs just interpolate their training data” crowd: Ayer, and in a different way early Wittgenstein, held that mathematical truths don’t report new facts about the world. Proofs unfold what is already implicit in axioms, definitions, symbols, and rules. I think that idea is deeply fascinating, AND have no problem that we still credit mathematicians with discoveries. So either “recombining existing material” isn…

> I think that idea is deeply fascinating, AND have no problem that we still credit mathematicians with discoveries. Most discoveries are indeed implied from axioms, but every now and then, new mathematics is (for lack of a better word) "created"—and you have people like Descartes, Newton, Leibniz, Gauss, Euler, Ramanujan, Galois, etc. that treat math more like an art than a science. For example, many belive that to…

It could be that RH is independent of current mathematical axiom systems. We might even prove that it is some day. But that means we are free to give it different truth values depending on the circumstances!

This is also true for established theorems! We can can imagine mathematical universes (toposes) where every (total) function on the reals is continuous! Even though it is an established theorems that there are discontinuous functions! We just need to replace a few axioms (chuck out law of the excluded middle, and throw in some continuity axioms).

Re: An OpenAI model has disproved a central conjecture in discrete geometry

#355

Post by astrange »

jmmcd wrote: post #339
stego-tech wrote: post #314

Earlier quoted context omitted.

As others have pointed out, both can be true: * LLMs do just interpolate their training data, BUT- * That can still yield useful "discoveries" in certain fields, absent the discovery of new mechanics that exist outside said training data In the case of mathematics, LLMs are essentially just brute-forcing the glorified calculators they run on with pseudo-random data regurgitated along probabilities; in that regard, ma…

> LLMs do just interpolate their training data "interpolate" has a technical meaning - in this meaning, LLMs almost never interpolate. It also has a very vague everyday meaning - in this meaning, LLMs do interpolate, but so do humans.

An LLM in a harness with any tools (even a calculator) doesn't just interpolate because it can reach states out of its own distribution.

Re: An OpenAI model has disproved a central conjecture in discrete geometry

#357

Post by wavemode »

danielmarkbruce wrote: post #293
symfrog wrote: post #260

Earlier quoted context omitted.

We have had LLMs for much longer than 3 years.

No, we haven't, for any reasonable definition of L.

OpenAI themselves must not have a "reasonable definition of L", then. Their own papers and press releases refer to GPT-2 (from 2019) as a "large language model".

https://openai.com/index/better-language-models/

Re: An OpenAI model has disproved a central conjecture in discrete geometry

#358

Post by 3abiton »

stego-tech wrote: post #314
m-hodges wrote: post #40

To the “LLMs just interpolate their training data” crowd: Ayer, and in a different way early Wittgenstein, held that mathematical truths don’t report new facts about the world. Proofs unfold what is already implicit in axioms, definitions, symbols, and rules. I think that idea is deeply fascinating, AND have no problem that we still credit mathematicians with discoveries. So either “recombining existing material” isn…

As others have pointed out, both can be true: * LLMs do just interpolate their training data, BUT- * That can still yield useful "discoveries" in certain fields, absent the discovery of new mechanics that exist outside said training data In the case of mathematics, LLMs are essentially just brute-forcing the glorified calculators they run on with pseudo-random data regurgitated along probabilities; in that regard, ma…

> * That can still yield useful "discoveries" in certain fields, absent the discovery of new mechanics that exist outside said training data

One can argue, new knowledge is just restructured data.

I think the main concerns about LLMs is the inherent "generative" aspects leading to hallucinations as a biproduct, because that's what produces the noi. Joint Embedding approaches are rather an interesting alternative that try to overcome this, but that's still in research phase.

Re: An OpenAI model has disproved a central conjecture in discrete geometry

#359

Post by adi_kurian »

Nevermark wrote: post #302
symfrog wrote: post #260

Earlier quoted context omitted.

We have had LLMs for much longer than 3 years.

I took humans thousands of years, then hundreds of years, to come to terms with very basic concepts about numbers. Its amazing to me when people talk about recombining things, or following up on things as somehow lesser work. People can't separate the perspective they were given when they learned the concepts, that those who developed the concepts didn't have because they didn't exist. Simple things are hard, or ever…

Anyone in the arts is aware that creativity is not the new, it is the repackaging of what already exists into something that is itself new.

Re: An OpenAI model has disproved a central conjecture in discrete geometry

#360

Post by black_knight »

kenjackson wrote: post #187
dvt wrote: post #61

Earlier quoted context omitted.

> I think that idea is deeply fascinating, AND have no problem that we still credit mathematicians with discoveries. Most discoveries are indeed implied from axioms, but every now and then, new mathematics is (for lack of a better word) "created"—and you have people like Descartes, Newton, Leibniz, Gauss, Euler, Ramanujan, Galois, etc. that treat math more like an art than a science. For example, many belive that to…

"new kind of math" Well I think the point is there is no "new kind of math". There's just types of math we've discovered and what we haven't. No new math is created, just found.

Where does this mathematics exist before we discover it?

I know of no realm where mathematical objects live except human minds.

No, it seems clear to me that mathematics is a creation of our minds.

Post reply on HN