Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Tidy Viewer – a cross-platform CSV pretty printer for viewer enjoyment

github.com

Post reply on HN
Print view View original

101–110 of 141 posts

Re: Show HN: Tidy Viewer – a cross-platform CSV pretty printer for viewer enjoyment

#101

Post by flusteredBias »

thamer wrote: post #91

Some (most?) tools that output data in columns and fit each one to the largest value in that column need to scan the whole file as a first pass just to start displaying data. Not only is it the case with this tool, but from what I'm reading in main.rs it looks like it's also loading the whole file in memory. I was going to say that scanning the file was a deal-breaker, but if true this is much more resource-intensive…

> these design choices seem to limit its use to relatively small files

1. As a rule-of-thumb, I have been working on functionality before optimization. That said, `tv` is really fast. It is completely false that `tv` only works for relatively small files. I just pushed a 624MB file to `tv`. It ran in 2.8 seconds. With `column` it takes 5.0 seconds. Now, I would love help from programmers smarter than me. I am sure there are a lot of optimization gains to be had in `tv`. I just wanted to make sure potential users are not misled. `tv` is performant.

> Some (most?) tools that output data in columns and fit each one to the largest value in that column need to scan the whole file as a first pass just to start displaying data.

> Not only is it the case with this tool, but from what I'm reading in main.rs it looks like it's also loading the whole file in memory.

2. `tv` reads once, but parse partly. This means that it reads the full file only to grab the number of rows. It only parses(take) the first n rows.

https://github.com/alexhallam/tv/blob/b548f0d19f64438d53f732...

https://github.com/alexhallam/tv/blob/b548f0d19f64438d53f732...

Re: Show HN: Tidy Viewer – a cross-platform CSV pretty printer for viewer enjoyment

#103

Post by qorrect »

sixothree wrote: post #75
claimred wrote: post #71

Wanted to mention that Windows PowerShell supports pretty CSV printing out of the box, like so Import-Csv .\Levels.csv | Format-Table Count Level elevation Level name Name Object type Unique ID ----- --------------- ---------- ---- ----------- --------- 1 -600.0000000 Store -0,600 - Store Level dc611fed-1783-d759-053a-b19848c51491 1 2850.0000000 Store +2,850 - Store Level c59f2ae4-0e94-6ea0-bd82-8306971e628c 1 3350.0…

There's also ConsoleGridView. https://devblogs.microsoft.com/powershell/introducing-consol...

Dang looks sick, wonder if I get it on *nix.

Re: Show HN: Tidy Viewer – a cross-platform CSV pretty printer for viewer enjoyment

#104

Post by c3534l »

brap wrote: post #30
wly_cdgr wrote: post #18

I thought this article was about a new way of understanding actual televisions

Same. Read this as some profound observation on what TVs actually are. Even thought to myself "huh, I guess OP does has a point..."

What is CSV other than a matrix of data? Its matrices all the way down.

Re: Show HN: Tidy Viewer – a cross-platform CSV pretty printer for viewer enjoyment

#106

Post by TAForObvReasons »

flusteredBias wrote: post #101
thamer wrote: post #91

Some (most?) tools that output data in columns and fit each one to the largest value in that column need to scan the whole file as a first pass just to start displaying data. Not only is it the case with this tool, but from what I'm reading in main.rs it looks like it's also loading the whole file in memory. I was going to say that scanning the file was a deal-breaker, but if true this is much more resource-intensive…

> these design choices seem to limit its use to relatively small files 1. As a rule-of-thumb, I have been working on functionality before optimization. That said, `tv` is really fast. It is completely false that `tv` only works for relatively small files. I just pushed a 624MB file to `tv`. It ran in 2.8 seconds. With `column` it takes 5.0 seconds. Now, I would love help from programmers smarter than me. I am sure th…

If the goal is to calculate the correct column width, you have to do one pass through the data before writing the first row.

If the file can be read multiple times (not a UNIX stream), you can just read the file twice.

If the file is a stream, instead of retaining the entire dataset in memory, you can write to a temporary file and re-parse it after calculating the widths.

Re: Show HN: Tidy Viewer – a cross-platform CSV pretty printer for viewer enjoyment

#107

Post by flusteredBias »

baggiponte wrote: post #100
flusteredBias wrote: post #45

Earlier quoted context omitted.

I love xsv! I mention in the readme that command line data manipulation tools are great compliments to tv. https://github.com/alexhallam/tv#tools-to-pair-with-tv

that's exactly the comment I was looking for! xsv is super powerful and I think you might both draw inspiration from one another. I read above that tv reads everything into memory: maybe you can exploit some xsv tricks to avoid that. I feel tv looks great to visualise the outcome at the end of a pipeline, perhaps with xsv. I am no Ruby expert either, but this can become a cool Homebrew binary: people on macOS will us…

I will add some Homebrew installation instructions. That is now an open issue. I want this tool to be highly accessible. Again, xsv is the best. I like the idea of small utilities that specialize in a specific task.

From the Unix philosophy:

> Make each program do one thing well. To do a new job, build afresh rather than complicate old programs by adding new "features".

Re: Show HN: Tidy Viewer – a cross-platform CSV pretty printer for viewer enjoyment

#108

Post by flusteredBias »

TAForObvReasons wrote: post #106
flusteredBias wrote: post #101

Earlier quoted context omitted.

> these design choices seem to limit its use to relatively small files 1. As a rule-of-thumb, I have been working on functionality before optimization. That said, `tv` is really fast. It is completely false that `tv` only works for relatively small files. I just pushed a 624MB file to `tv`. It ran in 2.8 seconds. With `column` it takes 5.0 seconds. Now, I would love help from programmers smarter than me. I am sure th…

If the goal is to calculate the correct column width, you have to do one pass through the data before writing the first row. If the file can be read multiple times (not a UNIX stream), you can just read the file twice. If the file is a stream, instead of retaining the entire dataset in memory, you can write to a temporary file and re-parse it after calculating the widths.

The correct column width is calculated from the first n rows not the full file.

A stream does not work for tv because a stream does not know how many rows are in the file a priori. Displaying the dimensions of the file is a priority for `tv`. I am very happy with that trade-off. I would rather know the dimensions of a file than have a file stream of unknown dimensions.

Re: Show HN: Tidy Viewer – a cross-platform CSV pretty printer for viewer enjoyment

#109

Post by saulpw »

flusteredBias wrote: post #33
rlue wrote: post #14

Why not visidata? https://www.visidata.org/ https://www.youtube.com/watch?v=N1CBDTgGtOU (It does much, much more than pretty printing, but no reason you can't use it for that.)

I love visidata! But when I want to just glance at a csv file I reach for tv (I used to use `column` which is more of a tv competitor than visidata). This is for a couple reasons. 1. tv gives a quick summary of the count of rows and columns 2. tv shows all columns at the bottom that don't fit in the terminal. With vd I have to scroll on wide data. 3. tv guides the eye to missing data better with NA highlights 4. tv h…

Hey there, VisiData author here. Nice work with tv! I'm sure it's more useful than VisiData for certain use cases. I just want to clear some things up since there are a few misconceptions here (which will happen if you don't use VisiData a lot):

1. In VisiData, The number of rows is always shown in the lower right, and you can see the number of columns with either Ctrl+G or a list of the columns with Shift+C. Or Shift+I for the list of columns with summary statistics (mode/distinct/errors/etc). This is an extra keystroke, but the amount of data you can get with that keystroke more than justifies it.

5. VisiData will instantly open and show any file it can, and continue to load the rest until it's done or you press Ctrl+C (or quit). Everything in VisiData is lazily evaluated, so it's not actually doing any more work than tv when you view the first page of rows, and then you can see the next few pages of rows with only one keystroke (PgDn, as opposed to having to edit a command and rerun it). Fewer keypresses ftw!

A lot of people think VisiData is a TUI spreadsheet, but vd is not a "spreadsheet" in the classic sense, as it's not cell-based. Its primary use-case is exploring and wrangling tabular data. It just turns out that this is what a lot of people are doing with their spreadsheets, but they have to bend over backwards to get Excel/whatever to play nice with their data's structure. By the same token, if you try to do little single-cell formulas in VisiData, it's going to be quite difficult.

For people who like static binaries and only need to view a few rows of CSV files, or produce part of a larger report in a pipeline, tv could be a better fit than VisiData, especially if it continues to be maintained. I'm always excited to see new data tools in the terminal space!

Re: Show HN: Tidy Viewer – a cross-platform CSV pretty printer for viewer enjoyment

#110

Post by flusteredBias »

eli wrote: post #98
fiddlerwoaroof wrote: post #96

Earlier quoted context omitted.

Or: head -n5 data/a.csv | tv

Unless your csv has embedded line breaks

Can you give an example of what you mean? If it breaks tv then I would like to add it to the automated tests and see if we can work on it.
Post reply on HN