Live data from Hacker News

The design philosophy of Great Tables

posit-dev.github.io

51–60 of 94 posts

Re: The design philosophy of Great Tables

#51
post #9

Great tables has done some really nice work on python/jupyter tables. It looks like they are almost building a "grammar of tables" similar to a grammar of graphics. More projects should write about their philosophy and aims like this. I have built a different table library for jupyter called buckaroo. My approach has been different. Buckaroo aims to allow you to interactively cycle through different formats and post-…

Thanks for your work on Buckaroo! Jupyter print() and IPython display() have limitations given their dead static output and feels like printf debugging of yore, which I know Buckaroo was written to address.

What are your thoughts on Visidata's hotkeys and controls? I used Visidata in the past and always wondered why it couldn't be added into Jupyter (eventually) for dataframe explorations.

>It looks like they are almost building a "grammar of tables" similar to a grammar of graphics.

Agreed that Great Tables seems to be taking annother crack at formalizing a "grammar of tables", and I welcome this approach too given the power of tabular formats and wider adoption of the dataframe concept via the R/pandas/Arrows/polars ecosystem, although I believe the term was initially referred to in the 90s[1] from the statistical S language.

[1] https://towardsdatascience.com/preventing-the-death-of-the-d...

Re: The design philosophy of Great Tables

#52

The example they show of a Great Table is, to my taste, way too busy. Here is my unsolicited opinion: The top and bottom horizontal rules on the Title appear to be superfluous, and I dislike how it is aligned with the first column (row labels) rather than the second. I feel like a little space to breath at the bottom, along with a bold font would add visual hierarchy w/o the clutter. The row label backgrounds are far…

You might want to read Edward Tufte's Beautiful Evidence.[1] He discusses stuff like what you brought up about readability and distracting from the message / point of the data. If you've seen sparklines, [2] Tufte coined the term. Whenever I do a UI review I end up paging through it just to see if there's something we're not thinking about, and its an interesting book to just open to a random page and read. Plus he h…

I'm a big fan of Tufte and he certainly informs a lot of my opinions on making tables and figures :)

Re: The design philosophy of Great Tables

#53
post #51
post #9

Great tables has done some really nice work on python/jupyter tables. It looks like they are almost building a "grammar of tables" similar to a grammar of graphics. More projects should write about their philosophy and aims like this. I have built a different table library for jupyter called buckaroo. My approach has been different. Buckaroo aims to allow you to interactively cycle through different formats and post-…

Thanks for your work on Buckaroo! Jupyter print() and IPython display() have limitations given their dead static output and feels like printf debugging of yore, which I know Buckaroo was written to address. What are your thoughts on Visidata's hotkeys and controls? I used Visidata in the past and always wondered why it couldn't be added into Jupyter (eventually) for dataframe explorations. > It looks like they are al…

Buckaroo started as a lowcode UI with an accompanying table. The low code UI lets you click on columns and perform actions (drop, fillNA, groupby). The dataframe is then modified, AND python code to perform the same action is emitted. Controlling the lowcode UI through keyboard shortcuts should be fairly straightforward.

The other feature I have played with in this area is auto-cleaning. Auto-cleaning looks at individual columns and emits cleaning commands to the low-code UI. Different cleaning strategies can be implemented and toggled through.

Buckaroo takes the view that being opinionated is good, so long as you can toggle through opinions to get the right combination of cleaning, display, or post-processing that you are looking for quickly. All of the features of buckaroo are also built to be easily extendable by users.

This feature saw very little use, so I haven't developed it much (I had to disable it after some refactorings). The lowcode UI is demonstrated at the end of the youtube video linked above.

Re: The design philosophy of Great Tables

#54
The historical background about tabular displays of quantitative information is very interesting. I imagine it must have been fun think deeply about this problem.

Unfortunately, the API design in the example is just not very good:

    (
       GT(simple_table, rowname_col='Name')
      .tab_header(title='Names, Addresses, and Characteristics of Remote Correspondents')
      .tab_stubhead(label=md('*Name*'))
      ...
    )
I'm uncertain if it's trying to mimic something in another language like R (or some grammar of graphics thing or D3.js.) Hopefully, it's not trying to mimic the look of long, chained `pandas.DataFrame` operations (because it misses the point of why those look the way it does.)

Of course, for ad hoc, in-a-notebook, cut-and-paste/written-from-scratch use, the API design doesn't really matter that match. Usually, users will readily memorise the required incantations then fiddle with the result until they get what they want or they give up.

It's probably the case that for most tools that produce visual outputs, a majority of users are creating things in this style. (There are, e.g., millions of casual Matplotlib users out there.) But programmatic use is not too far off. Tools that produce visual outputs (even those as formally rigidly at display tables,) are often subject to consistency requirements, which directly implies programmatic use.

So, when I discover that my colleagues and I have six tables across three notebooks that need a consistent look, and I decide to interact with this tool programmatically, am I expected to write…?

    def standard_table(source, /, rowname_col, header_title, stubhead_label, weight_columns):
      return (
        GT(source, rowname_col=rowname_col)
        .tab_header(title=header_title)
        .tab_stubhead(label=md(f"*{stubhead_label}*"))
        .fmt_integer(columns=weight_columns, pattern="{x} lbs")
        ...
      )

    standard_table(simple_table, rowname_col='Name', header_title='Names, Addresses, and Characteristics of Remote Correspondents', stubhead_label='Name', weight_columns='Weight')
Or maybe…?

    def format_table(weight_columns):
      return (
        tbl
        .tab_stubhead(label=md(f"*{tbl.stubhead.label}*")) # what if not present?
        .fmt_integer(columns=weight_columns, pattern="{x} lbs")
        ...
      )

    format_table(
      GT(simple_table, rowname_col='Name')
      .tab_header(title='Names, Addresses, and Characteristics of Remote Correspondents')
      .tab_stubhead(label='Name')
      ...
    )
Or maybe…?

     class StandardTable(GT):
       def tab_stubhead(self, *a, **kw):
         # inspect.signature.bind(...) # ...
         return super().tab_stubhead(*a, **kw)

    StandardTable(...)
These aren't great options. The API design is just not very good.

Re: The design philosophy of Great Tables

#56
I overall like the approach for complex scenarios but their example is not the best one. The original version is much more readable and their final version adds mostly noise.

Also, in documents all images and tables should have descriptive captions. So their header with title and subtitle would be redundant.

Re: The design philosophy of Great Tables

#57
This a good article with some fascinating history.

More recent history involves the production of CALS tables https://en.wikipedia.org/wiki/CALS_Table_Model. The company Datalogics https://en.wikipedia.org/wiki/Datalogics was heavily involved in the CALS table initiative. Datalogics staff was part of the ISO committee forming SGML, and trained many people on SGML, including DoD staff and their contractors involved with documentation.

I was involved with the team that produced an editor for SGML-based documents. It had as one of its features the ability to specify the formatting of an element based on the SGML context of that element. This was before XSLT and its kin.

Alumni of Datalogics helped Microsoft learn about XML ("No, you can't arbitrarily switch case on XML element tags").

Also TeX practitioners have pretty well-formed opinions about how tables should be formatted.

Odd side-note: I learned that the documentation for a fighter airplane of the time, if printed out, would weigh more than the aircraft and would fill a football-field sized collection of filing cabinets.

And as much as many today don't like XML, coming from the SGML world it is a boon.

Re: The design philosophy of Great Tables

#58
I Use SAS reports in my job pretty heavily I'm keen to find alternatives - this looks pretty promising.

One thing in particular I'm interested in but could not see an example for is if this will let you insert "break lines" i.e. for displaying sub totals and similar.

For example based on the demo, which shows names and addresses from census data it might be nice to be able to break at each change in postcode and display some summary data like a count of people found at that postcode or an average age (based on the DOB) living at that postcode or similar.

Otherwise conditional formatting is another pain point either using rules i.e. if value in column B is greater than a specified threshold make the entire row bold. Or automatically creating a color gradient to highlight the cells ala Excel.

For bonus points management types like things like red and green traffic lights (or down/up arrows) you can display next to kpi data in a table It's a gimmick but wins you points.

Re: The design philosophy of Great Tables

#60

There's also a book on the subject: https://en.wikipedia.org/wiki/The_History_of_Mathematical_Ta... Interesting aside: AI models trained on spreadsheets need "good tables" such as column names, headers, etc. to understand context. Like Fortap: https://arxiv.org/abs/2109.07323

Thanks for sharing the book info! I really need to find a copy of that somewhere :)

Me too. It‘s listed on eBay for $145 which is a lot of money.
Post reply on HN