Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Which table format do LLMs understand best?

improvingagents.com

Post reply on HN
Print view View original

71–80 of 115 posts

Re: Which table format do LLMs understand best?

#71

Post by naren87 »

Great benchmark! It highlights an important but often downstream problem. In real-world pipelines, the bigger issue comes before this: extracting tables from PDFs or scans without breaking their layout. Once the structure is lost (merged headers, nested cells, footnotes, etc.), no data format can fully recover it.

Check out LLMWhisperer from Unstract —> it preserves table and layout fidelity when converting documents for LLM use. You can try it on complex PDFs or forms here: https://pg.llmwhisperer.unstract.com (no signup needed)

Layout preservation upstream often improves downstream accuracy more than choosing between CSV, JSON, or Markdown. Find more details here: https://unstract.com/llmwhisperer/

Re: Which table format do LLMs understand best?

#73

Post by noosphr »

I've found that xml is surprisingly good for llms when it comes to table extraction in production. I only found out when I send the raw xml storage format to benchmark again various flavours of everything else. XML turns out to the best format for tables that have more than three levels of nesting.

Re: Which table format do LLMs understand best?

#74

Post by rovr138 »

mattcollins wrote: post #51
Sharlin wrote: post #5

> where accuracy is paramount > accuracy: 60% Not to mention that the least poorly performing format is probably the stupidest way to encode tabular data, beating even XML. But I guess that’s the new normal because we’re trying to shoehorn conversational AI models to every use case rather than, say, training finetunes that are better at particular tasks. (Yes, of course you can’t train finetunes when the model is a p…

I'm the person who ran the test. To explain the 60% a bit more... With small amounts of input data, the accuracy is near 100%. As you increase the size of the input data, the accuracy gradually decreases. For this test, I intentionally chose an input data set large enough that the LLM would score in the region of 50% accuracy (with variation between formats) in order to maximise the discriminative power of the test.

> As you increase the size of the input data, the accuracy gradually decreases.

Interesting.

On your section "Limitations and Areas for Further Study",

What I'd be curious on future work would be,

    - changing the order of the data on each table type
    - changing the order of the questions
I'm curious to know if what it fails is the same, if it changes depending on the location, if it's a bias.

Is it always a specific question? Is it always a specific value? Is it always question #x (or around question #x?). Does it tend towards x or y on types of questions?

Good idea

Re: Which table format do LLMs understand best?

#75

Post by svachalek »

simonw wrote: post #57
gpt5 wrote: post #49

Earlier quoted context omitted.

Isn't the best performing (markdown tables) and the worst (pipe delimited tables) basically the same format?

The best performing isn't markdown tables, it's markdown key/value pairs: ## Record 1 ``` id: 1 name: Charlie A0 age: 56 city: New York department: Operations salary: 67896 years_experience: 7 project_count: 1 ``` Which makes sense to me because the problem with formats like CSV and regular markdown tables is that it is too easy for the model to mistakenly associate a value in a row with the wrong header. Explicit ke…

Yeah that was my intuition as well. I think the KV-Markdown format gains additional advantage over JSON and YAML in the special syntax for headers helping to break up records.

Re: Which table format do LLMs understand best?

#76

Post by svachalek »

ggm wrote: post #2

I find this extremely surprising. I would have expected dict structures to have higher semantic context associated with them.

They did. The KV-Markdown is essentially a dict with ``` wrapper, and INI which is similar scored very high as well. The worst performers were index-based rows like CSV or Markdown tables. JSON is in the middle with high context and more syntactic noise and less clear record labels.

The odd ones to me are HTML which uses th and td to make indexed-based rows but did better than JSON somehow, and XML which is like JSON with even more syntactic noise placing better than INI. If I had to guess I'd say because vast amounts of the web were in the training set.

Re: Which table format do LLMs understand best?

#77

Post by wagslane »

johnfn wrote: post #68

The article has interesting data. But it’s frustrating to read AI generated text like this: > Performance Optimization: Reducing processing overhead while maintaining accuracy What on earth does it mean that this “optimized performance”? This is nonsensical content. Performance wasn’t even measured, accuracy was. You can tell this was AI generated because “ Reducing processing overhead while maintaining accuracy” wou…

I think they may be referring to token usage, which is mentioned in the article. fewer tokens = higher performance

Re: Which table format do LLMs understand best?

#78

Post by phyzome »

mritchie712 wrote: post #16
Sharlin wrote: post #5

> where accuracy is paramount > accuracy: 60% Not to mention that the least poorly performing format is probably the stupidest way to encode tabular data, beating even XML. But I guess that’s the new normal because we’re trying to shoehorn conversational AI models to every use case rather than, say, training finetunes that are better at particular tasks. (Yes, of course you can’t train finetunes when the model is a p…

they used GPT-4.1 nano, results would be quite different with sonnet or gpt5.

trust me bro, the next model bro, it's just way better bro

Re: Which table format do LLMs understand best?

#79

Post by jcheng »

xnx wrote: post #70
jcheng wrote: post #56

I was curious enough to have Codex create a similar benchmark: https://github.com/jcheng5/table-formats With 1000 rows and 100 samples and markdown-kv, I got these scores: - gpt-4.1-nano: 52% - gpt-4.1-mini: 72% - gpt-4.1: 93% - gpt-5: 100% I was so surprised by gpt-5 getting 100% that I ran it again with 1000 samples. It got 999 correct, and one wrong. To reproduce it yourself, clone the repo, add a .env file with O…

Cool tool. I tried a few different things to get to work with google/gemini-2.5-pro, but couldn't figure it out.

    uv add google-genai
    uv run scripts/run_benchmarks.py --models google/gemini-2.5-pro --formats markdown_kv --limit 100
And add GOOGLE_API_KEY= to a file called .env in the repo root.

Unfortunately I started getting "quota exceeded" almost immediately, but it did give 6/6 correct answers before it crapped out.

Re: Which table format do LLMs understand best?

#80

Post by typpilol »

phyzome wrote: post #78
mritchie712 wrote: post #16

Earlier quoted context omitted.

they used GPT-4.1 nano, results would be quite different with sonnet or gpt5.

trust me bro, the next model bro, it's just way better bro

To be fair nano was an absolute crap model when it came out
Post reply on HN