Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Which table format do LLMs understand best?

improvingagents.com

Post reply on HN
Print view View original

61–70 of 115 posts

Re: Which table format do LLMs understand best?

#61

Post by ysleepy »

mattcollins wrote: post #51
Sharlin wrote: post #5

> where accuracy is paramount > accuracy: 60% Not to mention that the least poorly performing format is probably the stupidest way to encode tabular data, beating even XML. But I guess that’s the new normal because we’re trying to shoehorn conversational AI models to every use case rather than, say, training finetunes that are better at particular tasks. (Yes, of course you can’t train finetunes when the model is a p…

I'm the person who ran the test. To explain the 60% a bit more... With small amounts of input data, the accuracy is near 100%. As you increase the size of the input data, the accuracy gradually decreases. For this test, I intentionally chose an input data set large enough that the LLM would score in the region of 50% accuracy (with variation between formats) in order to maximise the discriminative power of the test.

Wouldn't it be more useful to measure the number of rows the model can process while still hitting 100% accuracy?

Re: Which table format do LLMs understand best?

#62

Post by cwmoore »

simonw wrote: post #57
gpt5 wrote: post #49

Earlier quoted context omitted.

Isn't the best performing (markdown tables) and the worst (pipe delimited tables) basically the same format?

The best performing isn't markdown tables, it's markdown key/value pairs: ## Record 1 ``` id: 1 name: Charlie A0 age: 56 city: New York department: Operations salary: 67896 years_experience: 7 project_count: 1 ``` Which makes sense to me because the problem with formats like CSV and regular markdown tables is that it is too easy for the model to mistakenly associate a value in a row with the wrong header. Explicit ke…

I was surprised that XML (56%), with closing tags, wasn’t as good as YAMl/KV(60%), though line breaks perform the same kind of grouping function.

Then I realized from the table that XML used about 50% more tokens (~75K vs ~50K) for similar accuracy, and for the first time felt a kind of sympathy for the LLM…

Re: Which table format do LLMs understand best?

#63

Post by olliem36 »

We ended up making middleware for LLM 'tools/functions' that take common data/table formats like CSV, Excel and JSON.

The tool uses an LLM to write code to parse the data and conduct the analysis to return back to the LLM. Otherwise, we found pumping raw table data into a LLM is just not reliable, even if you go to the effort to conduct analysis on smaller chunks and merge the results.

Re: Which table format do LLMs understand best?

#65

Post by faxmeyourcode »

Curious how text-aligned tabular formats work for LLMs considering humans probably find them more readable than other formats

                                                                 System Sales(a)  
                                           Number of Units         (in Millions)  
         ──────────────────────────────────────────────────────────────────────── 
          KFC Division                              31,981    $           34,452  
          Taco Bell Division                         8,757                17,193  
          Pizza Hut Division                        20,225                13,108  
          Habit Burger & Grill Division                383                   713  
          YUM                                       61,346    $           65,466
I'm seeing pretty good success with extracting data out of 10-Qs which are formatted like this by default using the `edgartools` library's default `filing.text()` method.

Re: Which table format do LLMs understand best?

#66

Post by efitz »

coeneedell wrote: post #28
efitz wrote: post #12

Earlier quoted context omitted.

This was exactly my thought. Rather than feed the table directly to the LLM, build agents that extract the data and have the LLM act on the extracted data items. Then it’s a preference issue. The author didn’t see much more than 60% accuracy which is not very useful for many (most?) real world tasks.

“Agents that extract the data” Are we really reinventing data frame readers to have an LLM in the critical path?

Reinventing? No. Using? Yes, for a lot of good reasons.

LLMs are expensive. Spending tokens to do something in bulk that is well suited to existing tools and algorithms, is wasteful and slow. And the main reason is that, using LLMs, the original author indicated only a 60% success rate for the task. Why spend many times more time and money and energy just to use an LLM on a well-understood preparatory task that it sucks at, when you can get much better results more inexpensively with off-the-shelf tools, and feed their results to the LLM for its unique value.

Re: Which table format do LLMs understand best?

#68

Post by johnfn »

The article has interesting data. But it’s frustrating to read AI generated text like this:

> Performance Optimization: Reducing processing overhead while maintaining accuracy

What on earth does it mean that this “optimized performance”? This is nonsensical content. Performance wasn’t even measured, accuracy was. You can tell this was AI generated because “ Reducing processing overhead while maintaining accuracy” would likely be true for a perf optimization, but it has no meaning whatsoever in the context of the article.

This really throws into question whether I can take the rest of the article and data seriously.

Re: Which table format do LLMs understand best?

#70

Post by xnx »

jcheng wrote: post #56

I was curious enough to have Codex create a similar benchmark: https://github.com/jcheng5/table-formats With 1000 rows and 100 samples and markdown-kv, I got these scores: - gpt-4.1-nano: 52% - gpt-4.1-mini: 72% - gpt-4.1: 93% - gpt-5: 100% I was so surprised by gpt-5 getting 100% that I ran it again with 1000 samples. It got 999 correct, and one wrong. To reproduce it yourself, clone the repo, add a .env file with O…

Cool tool. I tried a few different things to get to work with google/gemini-2.5-pro, but couldn't figure it out.
Post reply on HN