Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What sort of maths are LLMs good at?

gowers.wordpress.com

Post reply on HN
Print view View original

11–20 of 189 posts

Re: What sort of maths are LLMs good at?

#11

Post by xnorswap »

tcp_handshaker wrote: post #7
m348e912 wrote: post #6

Earlier quoted context omitted.

>TASK REQUEST: Clear, not too long not too short prompt, for LLMs to go out and research freelance consulting gigs for one specific IT domain, and in one specific country in Europe, including maybe opportunities driven from temp agencies based in geographically close countries. As a human, not an LLM, I could interpret "including maybe opportunities driven from temp agencies based in geographically close countries" a…

I am describing the contents of the prompt that was not the prompt. The prompt was very clear to the model that some freelance opportunities in country A, the only one in consideration could be available via agencies in country B and C. And it was a clear prompt. So what happen is a prompt said for example, find freelance opportunities in Ireland but keep in mind some of these might be available via temp agencies in…

I found it very difficult to parse your description, "Clear, not too long not too short prompt, for LLMs to go out and research freelance consulting gigs for one specific IT domain, and in one specific country in Europe, including maybe opportunities driven from temp agencies based in geographically close countries."

( I was trying to quote a single sentence and then realised it ran on for the whole paragraph. )

Given how difficult I found that to follow, are you sure your prompt is actually "Clear, not too long not too short"? We now only have your word for it. I too had assumed that was a prompt given to an LLM to further prompt agents.

Re: What sort of maths are LLMs good at?

#12

Post by coldtea »

someguyiguess wrote: post #10
tcp_handshaker wrote: post #7

Earlier quoted context omitted.

I am describing the contents of the prompt that was not the prompt. The prompt was very clear to the model that some freelance opportunities in country A, the only one in consideration could be available via agencies in country B and C. And it was a clear prompt. So what happen is a prompt said for example, find freelance opportunities in Ireland but keep in mind some of these might be available via temp agencies in…

Based on the rest of your writing I’m going to assume that the prompt was the problem.

He was perfectly clear in both cases.

If a human misunderstood this, they'd be a dumb human.

Re: What sort of maths are LLMs good at?

#13

Post by laszlojamf »

h_mirin wrote: post #5

This is really an argument about test-time scaling, even though the post never uses the term. These days "test-time scaling" mostly means letting the model talk to itself for longer, but the first genuinely surprising results came from plain sampling. Google's AlphaCode generated millions of candidate programs and filtered them down to a handful of submissions, which beat the average human programmer in 2022, before…

for somebody who's out of the loop: what's the fuss over the ABC conjecture?

Re: What sort of maths are LLMs good at?

#14

Post by geon »

tcp_handshaker wrote: post #7
m348e912 wrote: post #6

Earlier quoted context omitted.

>TASK REQUEST: Clear, not too long not too short prompt, for LLMs to go out and research freelance consulting gigs for one specific IT domain, and in one specific country in Europe, including maybe opportunities driven from temp agencies based in geographically close countries. As a human, not an LLM, I could interpret "including maybe opportunities driven from temp agencies based in geographically close countries" a…

I am describing the contents of the prompt that was not the prompt. The prompt was very clear to the model that some freelance opportunities in country A, the only one in consideration could be available via agencies in country B and C. And it was a clear prompt. So what happen is a prompt said for example, find freelance opportunities in Ireland but keep in mind some of these might be available via temp agencies in…

Would the llm work better if it was given the job ad and asked where the job was located?

It seems to me that such simplified tasks tend work better. The rest of the loop is just scraping websites, which doesn’t really have a reason to rely on ai agents.

Re: What sort of maths are LLMs good at?

#15

Post by geon »

xnorswap wrote: post #11
tcp_handshaker wrote: post #7

Earlier quoted context omitted.

I am describing the contents of the prompt that was not the prompt. The prompt was very clear to the model that some freelance opportunities in country A, the only one in consideration could be available via agencies in country B and C. And it was a clear prompt. So what happen is a prompt said for example, find freelance opportunities in Ireland but keep in mind some of these might be available via temp agencies in…

I found it very difficult to parse your description, "Clear, not too long not too short prompt, for LLMs to go out and research freelance consulting gigs for one specific IT domain, and in one specific country in Europe, including maybe opportunities driven from temp agencies based in geographically close countries." ( I was trying to quote a single sentence and then realised it ran on for the whole paragraph. ) Give…

That was not the prompt.

Re: What sort of maths are LLMs good at?

#17

Post by criley2 »

tcp_handshaker wrote: post #4
n4r9 wrote: post #2

A thoughtful and measured post, as usual from Gowers. The final note is neat and worth pasting out here in full: > A good sign that LLMs have reached human level for a much wider class of problems will be if they start proving theorems using methods that, like much of the very best human mathematics, are new and surprising but that with hindsight come to seem beautiful and natural. They should also be methods that ar…

>>A good sign that LLMs have reached human level for a much wider class >> of problems will be if they start proving theorems using methods that, like much of the very best human mathematics, are new and surprising but that with hindsight come to seem beautiful and natural. I must be taking crazy pills and the AGI surely will pass me by... But TODAY, middle August 2026...And in the context of testing and evaluating t…

[flagged]

Re: What sort of maths are LLMs good at?

#18

Post by tcp_handshaker »

someguyiguess wrote: post #10
tcp_handshaker wrote: post #7

Earlier quoted context omitted.

I am describing the contents of the prompt that was not the prompt. The prompt was very clear to the model that some freelance opportunities in country A, the only one in consideration could be available via agencies in country B and C. And it was a clear prompt. So what happen is a prompt said for example, find freelance opportunities in Ireland but keep in mind some of these might be available via temp agencies in…

Based on the rest of your writing I’m going to assume that the prompt was the problem.

Keep deluding yourself, unless you work for an LLM provider...

"Frontier LLMs Still Struggle with Simple Reasoning Tasks" https://arxiv.org/abs/2507.07313

"General365: Benchmarking General Reasoning in Large Language Models Across Diverse and Challenging Tasks" https://arxiv.org/abs/2604.11778

"...General365, a benchmark specifically designed to assess general reasoning in LLMs. By restricting background knowledge to a K-12 level, General365 explicitly decouples reasoning from specialized expertise. The benchmark comprises 365 seed problems and 1,095 variant problems across eight categories, ensuring both high difficulty and diversity. Evaluations across 26 leading LLMs reveal that even the top-performing model achieves only 62.8% accuracy, in stark contrast to the near-perfect performances of LLMs in math and physics benchmarks..."

Re: What sort of maths are LLMs good at?

#19

Post by tcp_handshaker »

xnorswap wrote: post #11
tcp_handshaker wrote: post #7

Earlier quoted context omitted.

I am describing the contents of the prompt that was not the prompt. The prompt was very clear to the model that some freelance opportunities in country A, the only one in consideration could be available via agencies in country B and C. And it was a clear prompt. So what happen is a prompt said for example, find freelance opportunities in Ireland but keep in mind some of these might be available via temp agencies in…

I found it very difficult to parse your description, "Clear, not too long not too short prompt, for LLMs to go out and research freelance consulting gigs for one specific IT domain, and in one specific country in Europe, including maybe opportunities driven from temp agencies based in geographically close countries." ( I was trying to quote a single sentence and then realised it ran on for the whole paragraph. ) Give…

You can easily test this yourself with the SOTA models....or read the corroborating literature...

"General365: Benchmarking General Reasoning in Large Language Models Across Diverse and Challenging Tasks" https://arxiv.org/abs/2604.11778

"...General365, a benchmark specifically designed to assess general reasoning in LLMs. By restricting background knowledge to a K-12 level, General365 explicitly decouples reasoning from specialized expertise. The benchmark comprises 365 seed problems and 1,095 variant problems across eight categories, ensuring both high difficulty and diversity. Evaluations across 26 leading LLMs reveal that even the top-performing model achieves only 62.8% accuracy, in stark contrast to the near-perfect performances of LLMs in math and physics benchmarks..."

Re: What sort of maths are LLMs good at?

#20

Post by CodeCompost »

tcp_handshaker wrote: post #4
n4r9 wrote: post #2

A thoughtful and measured post, as usual from Gowers. The final note is neat and worth pasting out here in full: > A good sign that LLMs have reached human level for a much wider class of problems will be if they start proving theorems using methods that, like much of the very best human mathematics, are new and surprising but that with hindsight come to seem beautiful and natural. They should also be methods that ar…

>>A good sign that LLMs have reached human level for a much wider class >> of problems will be if they start proving theorems using methods that, like much of the very best human mathematics, are new and surprising but that with hindsight come to seem beautiful and natural. I must be taking crazy pills and the AGI surely will pass me by... But TODAY, middle August 2026...And in the context of testing and evaluating t…

LLMs have no sense of geography. They measure distances between parts of words, not distances between parts of world.
Post reply on HN