Why Large Language Models Fail at Tabular Prediction

Post by antonvs » Wed, Aug 05, 2026, 10:17 AM UTC

_joel wrote: post #6

The first thing I'd do if working with an LLM on tabular data is to ask what the best tool would be to work with that data and build up a proper harness to work with the data sensibly. Rawdogging LLM isn't the tool for forecasting like this, as they found.

> Rawdogging LLM isn't the tool for forecasting like this, as they found.

You seem to have misunderstood the paper. They didn’t find that LLMs weren’t the tool for this, that was essentially assumed as a well-known premise. The paper investigates why, specifically, that might be the case, by evaluating 5 hypotheses. What they found was a connection to dimensionality.

Post by antonvs » Wed, Aug 05, 2026, 10:28 AM UTC

gowld wrote: post #17

scott_s wrote: post #11

Earlier quoted context omitted.

I find that an odd take. The paper claims to establish what causes the problem: dimensionality. They are clear in that they don't understand why. But this sort of work is what needs to be done to eventually solve the problem.

Solve what problem? My hammer can't drive screws. Is that a problem to solve?

The problem it helps solve is our lack of understanding of how the models work.

The way we got these models in the first place is by people researching to try to understand how the relevant systems worked. There’s a great deal we don’t know about how LLMs work. This paper is exploring one aspect of the limitations of LLMs, which helps to understand them better - their capabilities and limitations, and the underlying causes.

This kind of work is exactly what’s needed to develop better models - not just LLMs.

Re: Why Large Language Models Fail at Tabular Prediction

Re: Why Large Language Models Fail at Tabular Prediction