Live data from Hacker News

Tao: Open math problems being non-renewably mined by AI

mathstodon.xyz

311–320 of 354 posts

Re: Tao: Open math problems being non-renewably mined by AI

#311
post #277

Earlier quoted context omitted.

Could you tell the difference between a grandmaster and stockfish if playing them online? If not, why would you care which one you are playing against?

Of course you can. Stockfish plays very different compared to a human, and it never ever blunders or makes mistakes.

I’ve never played against a grandmaster, but I have a feeling that he/she would play very different compared to me and would never make mistakes I could notice. Though admittedly I’m not very good at chess.

Re: Tao: Open math problems being non-renewably mined by AI

#312
post #60
post #52

I'm with @nilesh on this one, and not exactly sure how merely the existence of a solution precludes the advancement of human knowledge. If a problem is "solved" (say, symbolically verified) without any insights gained, it doesn't seem very interesting to the profession. Navier-Stokes is a bit different (because there's a prize attached, so "scooping" matters), but almost all interesting problems don't have any prizes…

Mathematicians will be less likely to work on a problem if there is a solution - even an incomprehensible one.

Yes, but presumably they'll work on another problem instead, because they're mathematicians who enjoy doing mathematics.

Is there value lost in them working on problems that don't have solutions instead of problems that do?

Re: Tao: Open math problems being non-renewably mined by AI

#313

There's nothing that AI won't be able to mine and accomplish (aside from being literally human), it's only a matter of hardware and scale at this point. Generalized problem solving is a factor of search efficiency over the problem space. The actual software part is all figured out, the only open questions are how to do things efficiently and what the trade-offs are from a hardware perspective, but if hardware paradig…

> There's nothing that AI won't be able to mine and accomplish (aside from being literally human), it's only a matter of hardware and scale at this point.

Sure there is: problems that require knowledge that simply doesn't exist yet. Until "AI" turns into general purpose robots that can develop new tools to explore the world, it is, in fact, pretty damned limited in what it can do without human help. The world is vast. Math is small.

Biology is replete with examples. Computers "solve" protein folding [1], and midwits immediately leap to conclusions that drug development will also quickly fall. But we literally have no idea how most of biology works, and simply getting to the starting line for drug development problems is often 95% of the battle. Come talk to me when you've done a million experiments to find the fundamental knowledge that unlocks the pathway(s) we didn't know about that makes a drug discovery program possible in the first place [2].

I am not pessimistic about humans running out of challenges. We'll just declare one class of problems "done" [3], and move on to the next frontier, as we always have. The problem with AI doomers is that they lack imagination that extends beyond computers, or perhaps more accurately, are so sophomoric in their thinking that they skip over the hard parts of any problem they don't fully understand. This stuff reminds me of the endless smartypants whinging about the end of human intelligence when chess machines started beating grandmasters. Chess was never really that great a measurement of human intellectual capacity, and we found new things to do with our big monkey brains.

[1] They did not solve protein folding, except in the minds of people who don't fully understand the problem.

[2] ...and invented new machinery to make the experiments possible in the first place.

[3] ...and we'll likely be wrong about that.

Re: Tao: Open math problems being non-renewably mined by AI

#314
post #311

Earlier quoted context omitted.

Of course you can. Stockfish plays very different compared to a human, and it never ever blunders or makes mistakes.

I’ve never played against a grandmaster, but I have a feeling that he/she would play very different compared to me and would never make mistakes I could notice. Though admittedly I’m not very good at chess.

I've played both. The GM plays tremendously differently than Stockfish.

Engines - specifically heuristically-driven ones like Stockfish - don't play like a strong GM. They play engine-perfect chess, which isn't how a GM plays with any consistency.

I'm only a decent amateur (1550 USCF) but when I lose to a titled player it's largely explainable in human terms how it happened.

Re: Tao: Open math problems being non-renewably mined by AI

#315
post #269

Tao's central point seems to be : "In short, the indiscriminate use of powerful solution-extraction tools can achieve the immediate short-term goal of solving problems at hand, but at the cost of sustaining the ecosystem for the next wave of progress, or in understanding the progress already obtained. " I am no mathematician, may have misunderstood his point and would be delighted to receive any corrections.

How is this any different from people in any field that are impacted by AI and lose the utility of their skills and endeavors over the past decades? Are we saying that we're running out of problems to solve because of AI and hence it should be stopped? I am not underestimating the importance of the collective knowledge of the mathematics community and the role of mathematics as the enablers of other sciences, but opp…

His point is twofold: that the process of solving the problems leads to more than just solving the problem in front of you but other interesting things (he has an example of going on a hike to a waterfall and all the other things you might spot over in the distance or nearby on the way, which you’d miss if you were able to jump straight there), and also lots of the simpler open problems are ones early researchers learn on (this is akin to the “if we automate junior engineers how does anyone learn to be a senior?”).

Re: Tao: Open math problems being non-renewably mined by AI

#316

Earlier quoted context omitted.

Why was there a prize attached to this problem then? What does humanity get out of this being proved?

The millennium problems is something done by a single institute to motivate progress on known open problems: https://en.wikipedia.org/wiki/Millennium_Prize_Problems

Yes, but why?

Re: Tao: Open math problems being non-renewably mined by AI

#317
In short, after after training AI on an extraordinarily amount of human cognitive output, we are now facing the possibility that our ability to train by working on hard problems will be slowly stripped away at least in some domains.

It’s like someone offers to build mag lev gym weights. It’s very cool that I can now lift the 500 pound weight with a finger. But what will I do when there’s no power and 500 pounds to lift?

Of course, cognition isn’t a single outcome problem like weight lifting. But we build cognition not wholly unlike how we build muscle: one needs resistance. Otherwise I’m not at all confident we “learn” in any depth.

Re: Tao: Open math problems being non-renewably mined by AI

#318

Earlier quoted context omitted.

I don't think we can confidently say, yet, that LLMs can't discover those connections. We just haven't explored them yet, because 99.99% of the prestige is locked up in proving hard results, which also happen to be easier to assess objectively (thanks to automated proof checkers), and so that's where all the effort has thus far been exerted.

LLMs can discover anything. It is just a matter of creating the right goal function and teaching it the right heuristics. Right now, humans are required because humans know what humans want, and LLMs are not good at predicting what humans want to the point where they can safely and autonomously run off on their own to solve problems we didn't know we had, or to define the problems we have that we're not good at defin…

Debatably in your scenario the last one will or is already falling with robots that will do that. I think there is a real philosophical question of whether AI will be eventually capable of everything we are. I think the question of what makes us unique as humans needs to be asked. I am inclined to think that there is always something that will separate us as human beings from digital robots. I just believe the language and reasoning used for the last 200 years is no longer sufficient. In 25-50 years I believe we will have a clearer picture. Your view essentially falls into a nihilistic framework if I am understanding it correctly.

Re: Tao: Open math problems being non-renewably mined by AI

#319
post #20

Earlier quoted context omitted.

Also Andrew Wiles working in secret for 7 years out of fear of someone scooping him.

I've always found the story of A. Wiles sad and frustrating. He worked in secret for 7 years. He submitted a (incorrect) proof at year 4 or so. Reviewers found a problem, but he decided kept all secret for many years after. He didn't even proof the last theorem of Fermat directly, he proved some conjeture that someone else before him, proved that it implied Fermat last theorem... I found this behavior against healthy…

> He didn't even proof the last theorem of Fermat directly, he proved some conjeture that someone else before him, proved that it implied Fermat last theorem...

That's how maths works yes...

Re: Tao: Open math problems being non-renewably mined by AI

#320

Earlier quoted context omitted.

My computer contains the prime factorization of probably several dozen (if not more) large integers, and I refuse to share them with anyone! (Because they are my private RSA keys)

Why are you still using RSA?

It is easier to trust what you can understand.
Post reply on HN