Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What sort of maths are LLMs good at?

gowers.wordpress.com

Post reply on HN
Print view View original

51–60 of 189 posts

Re: What sort of maths are LLMs good at?

#51

Post by fn-mote »

root-parent wrote: post #43
criley2 wrote: post #30

Earlier quoted context omitted.

I totally agree - designing a competent AI agent with a fully customized harness to successfully pull off this task is a much more challenging engineering effort than merely creating an ordinary computer program. Had OP made chatgpt write an ordinary program instead, they likely would have succeeded in their task.

You said "Had OP made ChatGPT write an ordinary program" but that assumes enough structure exists to specify that program. If so that may be the right architecture, but also demonstrates why the "agentic AI" does not automatically solve the original open ended task. You have converted a fuzzy task into a conventional software engineering problem, and then relying on conventional software for the reliability :-)

> converted a fuzzy task into a conventional software engineering problem, and then relying on conventional software for the reliability

What can I say? This is how I get results from AI. It also gives me context & tools to fight the AI when I have to.

Re: What sort of maths are LLMs good at?

#52

Post by [deleted] »

suddenlybananas wrote: post #36
criley2 wrote: post #26

Earlier quoted context omitted.

What a sloppy reply. You've hijacked a thread on mathematics first to complain that your incompetent attempt to use ChatGPT to find a job failed, but it seems now that this was a ruse to instead begin arguments unrelated to the article at all where you just spam arxiv links you've never read to "prove" that AI is a scam. This comes across, frankly, as either Dunning-Kruger (classic illusory superiority), or potential…

Why are you so upset that someone is criticising LLMs that you call them schizophrenic? (I'd recommend refreshing your memory with this https://news.ycombinator.com/newsguidelines.html )

[deleted]

Re: What sort of maths are LLMs good at?

#53

Post by steinwinde »

For a list of AI accomplishments in mathematics see https://mathoverflow.net/questions/502120/examples-for-the-u... - or a candidate list here: https://aimath.robertj1.com/ . Many have observed an affinity of AI to the search for counterexamples - or examples. Looking at afore lists, something much more sociological crosses my mind: There is a hunt for answering prominent, clearly stated problems. I'm not a mathematician, but is this mostly what progress in mathematics is about? How about stating worthwhile problems in the first place? What about theory building? Am I right saying this is equally important, but none of those utilizing AI for mathematics seem to be interested in such?

Re: What sort of maths are LLMs good at?

#54

Post by root-parent »

fragmede wrote: post #34
tcp_handshaker wrote: post #4

Earlier quoted context omitted.

>>A good sign that LLMs have reached human level for a much wider class >> of problems will be if they start proving theorems using methods that, like much of the very best human mathematics, are new and surprising but that with hindsight come to seem beautiful and natural. I must be taking crazy pills and the AGI surely will pass me by... But TODAY, middle August 2026...And in the context of testing and evaluating t…

> TASK REQUEST: Clear, not too long not too short prompt Why do you think there's such a thing as too long for an LLM prompt? You'll run into context window limits at some point, but the more verbose you are with what you ask of it, the better the results will be.

>> t the more verbose you are with what you ask of it, the better the results will be.

Trivially falsifiable:

"Context Length Alone Hurts LLM Performance Despite Perfect Retrieval"

https://aclanthology.org/2025.findings-emnlp.1264/

"Large Language Models Can Be Easily Distracted by Irrelevant Context"

https://arxiv.org/abs/2302.00093

Re: What sort of maths are LLMs good at?

#55

Post by groundzeros2015 »

tcp_handshaker wrote: post #4
n4r9 wrote: post #2

A thoughtful and measured post, as usual from Gowers. The final note is neat and worth pasting out here in full: > A good sign that LLMs have reached human level for a much wider class of problems will be if they start proving theorems using methods that, like much of the very best human mathematics, are new and surprising but that with hindsight come to seem beautiful and natural. They should also be methods that ar…

>>A good sign that LLMs have reached human level for a much wider class >> of problems will be if they start proving theorems using methods that, like much of the very best human mathematics, are new and surprising but that with hindsight come to seem beautiful and natural. I must be taking crazy pills and the AGI surely will pass me by... But TODAY, middle August 2026...And in the context of testing and evaluating t…

This is an off topic rant unrelated to mathematical ability which is a closed problem often with complete logical information.

Re: What sort of maths are LLMs good at?

#56

Post by groundzeros2015 »

scronkfinkle wrote: post #8

> A good sign that LLMs have reached human level for a much wider class of problems will be if they start proving theorems using methods that, like much of the very best human mathematics, are new and surprising but that with hindsight come to seem beautiful and natural. They should also be methods that are difficult to stumble on by accident. It is hard to say precisely what would count as such a proof, but I think…

That’s not the criteria outlined in the quote you just used.

Re: What sort of maths are LLMs good at?

#57

Post by kqr »

dataviz1000 wrote: post #9

If you want to peek inside how a model solves a math problem have a look at some data visualizations I made solving basic multiplication.[0] I wanted to demonstrate capacity (how well it does a thing) instead of capability (which things it does, like drawing a pelican on a bicycle with SVG or solving a Rubik's Cube). To understand how LLMs solve math, look at the simplest case of multiplication. I deconstructed and c…

You probably know it, but Boyd did not at all suggest speed beats quality. If anything, his realisation was the opposite: the US aircraft had a more open canopy and thus improved quality of observation, and that was what won despite the superior power and turning performance of the Soviet aircraft.

Re: What sort of maths are LLMs good at?

#58

Post by fn-mote »

svantana wrote: post #29
scronkfinkle wrote: post #8

> A good sign that LLMs have reached human level for a much wider class of problems will be if they start proving theorems using methods that, like much of the very best human mathematics, are new and surprising but that with hindsight come to seem beautiful and natural. They should also be methods that are difficult to stumble on by accident. It is hard to say precisely what would count as such a proof, but I think…

I would be extremely surprised if something as elegant, terse, and useful as the Fourier Transform had been missed by human mathematicians up until now. All expressible theorems are enumerable, after all (if we limit ourselves to a finite alphabet). It seems likely that any new theorems are long, highly complex and esoteric, regardless of human or machine origin.

1. The computer is going to struggle to recognize elegance. I’m not sure it’s relevant at this point (but who knows).

2. The statement about proofs is just way wrong. It doesn’t sound like you are familiar enough with them.

This isn’t exactly what you implied, but witness the very short disproof of the Jacobean Conjecture.

Re: What sort of maths are LLMs good at?

#59

Post by [deleted] »

vhantz wrote: post #45
tcp_handshaker wrote: post #4

Earlier quoted context omitted.

>>A good sign that LLMs have reached human level for a much wider class >> of problems will be if they start proving theorems using methods that, like much of the very best human mathematics, are new and surprising but that with hindsight come to seem beautiful and natural. I must be taking crazy pills and the AGI surely will pass me by... But TODAY, middle August 2026...And in the context of testing and evaluating t…

Stop wasting your time and use actual code for most of what you give an LLM to do. Make them write the code even. Anything that can be verified mechanically should be code. Only use LLMs to fill in the gaps where things are fuzzy. Don't fall for the idea that those harnesses are general purpose, make your own fit to your task with the guards and verification steps you need. Make the LLM create the harness even. There…

[deleted]

Re: What sort of maths are LLMs good at?

#60

Post by root-parent »

vhantz wrote: post #45
tcp_handshaker wrote: post #4

Earlier quoted context omitted.

>>A good sign that LLMs have reached human level for a much wider class >> of problems will be if they start proving theorems using methods that, like much of the very best human mathematics, are new and surprising but that with hindsight come to seem beautiful and natural. I must be taking crazy pills and the AGI surely will pass me by... But TODAY, middle August 2026...And in the context of testing and evaluating t…

Stop wasting your time and use actual code for most of what you give an LLM to do. Make them write the code even. Anything that can be verified mechanically should be code. Only use LLMs to fill in the gaps where things are fuzzy. Don't fall for the idea that those harnesses are general purpose, make your own fit to your task with the guards and verification steps you need. Make the LLM create the harness even. There…

The irony about your comment is that, this is probably the most likely opinion and most consensual around many technological practitioners.

But the mathematicians here in this thread, are having a hard time with these clearly dumb models, doing so well in proving theorems in their domains :-)

Post reply on HN