Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Which table format do LLMs understand best?

improvingagents.com

Post reply on HN
Print view View original

31–40 of 115 posts

Re: Which table format do LLMs understand best?

#31

Post by mingtianzhang »

The current OCR approach typically relies on a Vision-Language Model (VLM) to convert a table into a JSON structure. However, a table inherently has a 2D spatial structure, while Large Language Models (LLMs) are optimized for processing 1D sequential text. This creates a fundamental mismatch between the data representation and the model’s input format.

Most existing pipelines address this by preprocessing the table into a linearized 1D string before passing it to the LLM — a question-agnostic step that may lose structural information.

Instead, one could retain the original table form and, when a question is asked, feed both the question and the original table (as an image) directly into the VLM. This approach allows the model to reason over the data in its native 2D domain, providing a more natural and potentially more accurate solution.

Re: Which table format do LLMs understand best?

#32

Post by fnordpiglet »

mritchie712 wrote: post #16
Sharlin wrote: post #5

> where accuracy is paramount > accuracy: 60% Not to mention that the least poorly performing format is probably the stupidest way to encode tabular data, beating even XML. But I guess that’s the new normal because we’re trying to shoehorn conversational AI models to every use case rather than, say, training finetunes that are better at particular tasks. (Yes, of course you can’t train finetunes when the model is a p…

they used GPT-4.1 nano, results would be quite different with sonnet or gpt5.

I was looking for the frontier curve where they tested their benchmark across different models since this sort of behavior is highly parameter, architecture, training, and fine tuning sensitive. It’s a practically useful question so I was really disappointed when a) they didn’t publish their code so you could test yourself, b) they didn’t do even a cursory examination of other models and sizes.

Re: Which table format do LLMs understand best?

#34

Post by fragmede »

mingtianzhang wrote: post #31

The current OCR approach typically relies on a Vision-Language Model (VLM) to convert a table into a JSON structure. However, a table inherently has a 2D spatial structure, while Large Language Models (LLMs) are optimized for processing 1D sequential text. This creates a fundamental mismatch between the data representation and the model’s input format. Most existing pipelines address this by preprocessing the table i…

Yeah, I wonder how PNG would fare in this contest.

Re: Which table format do LLMs understand best?

#36

Post by zaidf »

sega_sai wrote: post #15

Bizarre conclusions when on average all the formats perform poorly with average accuracy of 50%. Sure 60% is better than 40% but they are both unusable if you actually care about numbers...

I've been stunned by how many smart people talk so casually about LLMs becoming better at math. Do they just forget that a calculator that is wrong 1% of the time is a de facto calculator that doesn't work and should not be used?

Re: Which table format do LLMs understand best?

#38

Post by grey-area »

dcre wrote: post #35

Only testing GPT-4.1-nano makes this basically useless. Most people are almost certainly using GPT-5 mini or better. This very poor analysis is like an LLM literacy test for readers.

Please go away and do the work for us and let us know what anmazing accuracy you got with whatever version you think is better.

Anything below 100% is actually pretty useless when it comes to stats.

Re: Which table format do LLMs understand best?

#39

Post by xnx »

zaidf wrote: post #36
sega_sai wrote: post #15

Bizarre conclusions when on average all the formats perform poorly with average accuracy of 50%. Sure 60% is better than 40% but they are both unusable if you actually care about numbers...

I've been stunned by how many smart people talk so casually about LLMs becoming better at math. Do they just forget that a calculator that is wrong 1% of the time is a de facto calculator that doesn't work and should not be used?

> I've been stunned by how many smart people talk so casually about LLMs becoming better at math

Could they be referring to this?

"Advanced version of Gemini with Deep Think officially achieves gold-medal standard at the International Mathematical Olympiad" https://deepmind.google/discover/blog/advanced-version-of-ge...

Re: Which table format do LLMs understand best?

#40

Post by westoncb »

zaidf wrote: post #36
sega_sai wrote: post #15

Bizarre conclusions when on average all the formats perform poorly with average accuracy of 50%. Sure 60% is better than 40% but they are both unusable if you actually care about numbers...

I've been stunned by how many smart people talk so casually about LLMs becoming better at math. Do they just forget that a calculator that is wrong 1% of the time is a de facto calculator that doesn't work and should not be used?

Doing math is not the same as calculating. LLMs can be very useful in doing math; for calculating they are the wrong tool (and even there they can be very useful, but you ask them to use calculating tools, not to do the calculations themselves—both Claude and ChatGPT are set up to do this).

If you're curious, check out how mathematicians like Robert Ghrist or Terence Tao are using LLMs for math research, both have written about it online repeatedly (along with an increasing number of other researchers).

Apart from assisting with research, their ability on e.g. math olympiad problems is periodically measured and objectively rapidly improving, so this isn't just a matter of opinion.

Post reply on HN