Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Which table format do LLMs understand best?

improvingagents.com

Post reply on HN
Print view View original

41–50 of 115 posts

Re: Which table format do LLMs understand best?

#41

Post by bwestergard »

jitl wrote: post #19
brap wrote: post #10

I wonder how this compares to a more agentic approach where the LLM composes SQL queries to answer the questions, for example.

Yeah I mean for many real world scale datasets you don’t want to blow the whole context window on a massive markdown file. Instead you can provide a tool that presents the data as a SQLite database. In my testing Claude code seems very capable of answering questions via SQLite queries or even `head` and `grep` on CSV files.

But the result from the SQL query is going to be... a table. So at some point, tables need to go into context, and we need to know how well LLMs can incorporate those tables.

Re: Which table format do LLMs understand best?

#42

Post by magicalhippo »

zaidf wrote: post #36
sega_sai wrote: post #15

Bizarre conclusions when on average all the formats perform poorly with average accuracy of 50%. Sure 60% is better than 40% but they are both unusable if you actually care about numbers...

I've been stunned by how many smart people talk so casually about LLMs becoming better at math. Do they just forget that a calculator that is wrong 1% of the time is a de facto calculator that doesn't work and should not be used?

The best math lecturers I had at university sucked at mental calculations. Some almost screwed up 2+2 on the blackboard.

Yes LLMs suck at calculating stuff. However they can manipulate equations and such, and sometimes impressively so.

Re: Which table format do LLMs understand best?

#43

Post by simonw »

grey-area wrote: post #38
dcre wrote: post #35

Only testing GPT-4.1-nano makes this basically useless. Most people are almost certainly using GPT-5 mini or better. This very poor analysis is like an LLM literacy test for readers.

Please go away and do the work for us and let us know what anmazing accuracy you got with whatever version you think is better. Anything below 100% is actually pretty useless when it comes to stats.

If you want 100% accuracy from these kinds of tasks with LLMs you can get it today, but you need to provide the LLM with the ability to run Python code and tell it to use something like Pandas.

You can confirm it's doing the right thing by reviewing the code it wrote.

Re: Which table format do LLMs understand best?

#45

Post by dcre »

grey-area wrote: post #38
dcre wrote: post #35

Only testing GPT-4.1-nano makes this basically useless. Most people are almost certainly using GPT-5 mini or better. This very poor analysis is like an LLM literacy test for readers.

Please go away and do the work for us and let us know what anmazing accuracy you got with whatever version you think is better. Anything below 100% is actually pretty useless when it comes to stats.

Simon is right about using code execution, but many tables one might look at outside of formal data work are small enough for LLMs to be very reliable at, so this format question is practically relevant. I wish they had tested better models.

Re: Which table format do LLMs understand best?

#46

Post by thom »

brap wrote: post #10

I wonder how this compares to a more agentic approach where the LLM composes SQL queries to answer the questions, for example.

Well, ironically you then have the issue of how to present your database schema (including important things like the values in some categorical fields) to the LLM and in what format, so you never really escape this issue.

Re: Which table format do LLMs understand best?

#47

Post by crazygringo »

zaidf wrote: post #36
sega_sai wrote: post #15

Bizarre conclusions when on average all the formats perform poorly with average accuracy of 50%. Sure 60% is better than 40% but they are both unusable if you actually care about numbers...

I've been stunned by how many smart people talk so casually about LLMs becoming better at math. Do they just forget that a calculator that is wrong 1% of the time is a de facto calculator that doesn't work and should not be used?

You realize that when typing into a calculator, you probably hit a wrong key more than 1% of the time? Which is why you always type important calculations twice?

I've been stunned by how many smart people talk so casually about how because LLMs aren't perfect, they therefore have no value. Do they just forget that nothing in the world is perfect, and the values of things are measured in degrees?

Re: Which table format do LLMs understand best?

#48

Post by mattcollins »

veryrealsid wrote: post #26

I'm surprised by the accuracy, in practice, I feel like I generally have a lot better results

I'm the person who ran the test.

The context I used in the test was pretty large. You'll see much better (near 100%) accuracy if you're using smaller amounts of context.

[I chose the context size so that the LLM would be scoring in the ballpark of 50% accuracy (with variation between formats) to maximise the discriminative power of the test.]

Re: Which table format do LLMs understand best?

#49

Post by gpt5 »

Sharlin wrote: post #5

> where accuracy is paramount > accuracy: 60% Not to mention that the least poorly performing format is probably the stupidest way to encode tabular data, beating even XML. But I guess that’s the new normal because we’re trying to shoehorn conversational AI models to every use case rather than, say, training finetunes that are better at particular tasks. (Yes, of course you can’t train finetunes when the model is a p…

Isn't the best performing (markdown tables) and the worst (pipe delimited tables) basically the same format?

Re: Which table format do LLMs understand best?

#50

Post by Ciantic »

This is a bit silly way to use LLMs to process tabular data. In reality, you'd ask it to write functions and execute them. First you'd ask it to create a type definition from the table, then ask it to create functions to process the data.

"Write a function to find years of experience by name? Return just the number, e.g. '12'."

It works much better, and it can single-shot many of the processing requirements just from type definitions it can infer from the data.

This way it's easier to stick to tabular formats that have easy reading libraries, like with TypeScript/JavaScript JSON, and with Python, maybe CSV...

Post reply on HN