Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tad, a tabular data viewer

tadviewer.com

Post reply on HN
Print view View original

71–80 of 113 posts

Re: Tad, a tabular data viewer

#71

Post by burntsushi »

TheAceOfHearts wrote: post #45

Related to this, if you're looking for a CLI tool for handling CSVs, xsv [0] is looking promising. Based on prior experiences with CSV, one of the big problems I've seen has been figuring out what text encoding is being used. This problem appears to be more prominent with people outside of the US. It looks like Tad is using fast-csv, which I don't think will properly handle different file encodings. Life would be so…

You basically have two approaches:

1. Write your CSV parser with an assumption that the data is ASCII compatible (this means it works with either UTF-8 or Latin-1 out of the box, possibly modulo non-ASCII meta characters). To support additional encodings---such as UTF-16---either the CSV library or the caller must transcode first.

2. Write your CSV parser such that it can work on multiple different encodings. For example, this means looking for `\x2C\x00` when parsing UTF-16LE data instead of just `,`. This introduces implementation complexity, and you'll be unlikely to support the full gamut of encodings that other tools support whose job it is to do that sort of thing.

(2) is kind of weird but probably quite a bit faster than (1), although I can imagine it being useful in very niche circumstances. e.g., "I have a boat load of UTF-16 encoded CSV data and transcoding it to UTF-8 to use this CSV parser isn't worth my time because ______." I can't actually fill in that blank, so solutions in (1) tend to be the way to go.

Now... If you're building a full on CSV tabular viewer, then I might understand why it should handle encoding for you automatically, but when it comes down to it, the viewer is still going to need to choose between (1) and (2). Unless they want to hand roll their own CSV library, I imagine they're just going to pick (1), and when possible, transcode the data first. In that case, it shouldn't really matter whether their underlying CSV parser supports alternative encodings or not.

Re: Tad, a tabular data viewer

#72

Post by bunderbunder »

baldfat wrote: post #49
Macuyiko wrote: post #36

Nice, some initial remarks: - Sad to see it's an Electron app, but I don't want to start this flamewar all over again - Mid-sized CSV file opens fairly quick - If you want to build this into a full-fledged app, be prepared to handle a lot of CSV edge cases, also think about supporting AVRO, Parquet files - Regarding filtering: I can filter on CONTAINS and '=' but not on NOT-CONTAINS or ' ' I think? This is annoying t…

> Sad to see it's an Electron app, but I don't want to start this flamewar all over again Then stop mentioning it. Just don't use anything electron and don't mention it. I will use my RStudio and VS Code.

I don't want to start a flamewar, either, but I'm glad that the top poster mentioned that it's an electron app. Having that knowledge ahead of time allows me to manage my expectations.

Re: Tad, a tabular data viewer

#73

Post by antonycourtney »

johnlbevan2 wrote: post #59
misterdata wrote: post #46

Just noticed the installer changes the file association for CSV's so they all open with Tad, without asking me first. Please do not do this. Otherwise, nice app. How does it scale to larger CSV files? I have been working on something similar ( https://warp.one ) for Mac, which streams CSV files (apparently you use a SQLite database behind the scenes as cache?)

Upvote for "don't change default file handler without asking".

Really sorry about this! I do specify Tad as a handler for CSV files, but becoming the default handler was not my intent at all; I will look into what's causing this.

Re: Tad, a tabular data viewer

#74

Post by Asooka »

aargh_aargh wrote: post #61

Dies immediately when launched on Debian Jessie: Error: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /tmp/.org.chromium.Chromium.ibqKoR) $ strings /usr/lib/x86_64-linux-gnu/libstdc++.so.6 | grep GLIBCXX_ ... GLIBCXX_3.4.19 GLIBCXX_3.4.20 GLIBCXX_DEBUG_MESSAGE_LENGTH

I think you should be able to rebuild the electron npm module from source to have a compatible runtime.

Re: Tad, a tabular data viewer

#76

Post by leandot »

antonycourtney wrote: post #69
Macuyiko wrote: post #36

Nice, some initial remarks: - Sad to see it's an Electron app, but I don't want to start this flamewar all over again - Mid-sized CSV file opens fairly quick - If you want to build this into a full-fledged app, be prepared to handle a lot of CSV edge cases, also think about supporting AVRO, Parquet files - Regarding filtering: I can filter on CONTAINS and '=' but not on NOT-CONTAINS or ' ' I think? This is annoying t…

All great feedback, thanks! Some quick replies: - The omission of ' ' from the operator list was just an embarrassing mistake on my part. #facepalm #WILLFIX - Also agree that a drop-down list of checkboxes (ideally a searchable one!) with an 'in' operator for low cardinality columns would be great. But the UI for that is a bit involved, and probably also need to provide some user control over when to do this since ga…

Good job, curious about the performance when doing multiple AND, OR clauses on 1m+ rows? You'd need compound indices or?

Re: Tad, a tabular data viewer

#77

Post by leandot »

Macuyiko wrote: post #36

Nice, some initial remarks: - Sad to see it's an Electron app, but I don't want to start this flamewar all over again - Mid-sized CSV file opens fairly quick - If you want to build this into a full-fledged app, be prepared to handle a lot of CSV edge cases, also think about supporting AVRO, Parquet files - Regarding filtering: I can filter on CONTAINS and '=' but not on NOT-CONTAINS or ' ' I think? This is annoying t…

Genuinely curious, what can be used instead of Electron for cross-platform desktop app?

Re: Tad, a tabular data viewer

#78

Post by MrMattWright »

baldfat wrote: post #52
MrMattWright wrote: post #28

This is absolutely terrific! I often work with csv or tsv and large data, and I just want to quickly look at it without loading excel or exporting to google sheets. https://atom.io/packages/tablr Tablr for Atom is good too. One feature request. I very often get data in JSON format (then convert to csv), it would be amazing to also support JSON. Thanks for sharing.

OpenRefine http://openrefine.org/ for years has been my go to tool when looking at csv or tsv. I really love this project. > OpenRefine (formerly Google Refine) is a powerful tool for working with messy data: cleaning it; transforming it from one format into another; and extending it with web services and external data.

Oh thanks for reminding me, I did look at that a while ago, it's absolutely great. The transforming steps are amazing. Time to have another look at it. Exploratory.io is similar (nice transformation steps using dplyr) but for me it was just a bit too close to actually writing the R code.

Re: Tad, a tabular data viewer

#79

Post by sillysaurus3 »

mhuffman wrote: post #70
baldfat wrote: post #49

Earlier quoted context omitted.

> Sad to see it's an Electron app, but I don't want to start this flamewar all over again Then stop mentioning it. Just don't use anything electron and don't mention it. I will use my RStudio and VS Code.

I think it is ok to express displeasure at the state of something without just silently doing something else and hoping that the entire ecosystem catches up to you. For example, I take every opportunity (like right now, for example) to shit on node.js and npm for that very reason. There is so much evangelism on one side, it is nice to hear the opposing view points.

Ok, so what are some legit criticisms of node.js and npm? I'm no fanboy, but every time I try to ask this, the answers are along the lines of "if you install packages in a different order, you get different results." Which is fair, but everybody installs using `npm i`, so that's mostly a moot point.

Is there any substance here?

I think it's fair to talk about this, as long as we go out of our way not to reduce it to flamewar territory. The app is built on Electron, after all.

Maybe avoid mentioning memory usage, since that point has been done to death.

Re: Tad, a tabular data viewer

#80

Post by wereHamster »

leandot wrote: post #77
Macuyiko wrote: post #36

Nice, some initial remarks: - Sad to see it's an Electron app, but I don't want to start this flamewar all over again - Mid-sized CSV file opens fairly quick - If you want to build this into a full-fledged app, be prepared to handle a lot of CSV edge cases, also think about supporting AVRO, Parquet files - Regarding filtering: I can filter on CONTAINS and '=' but not on NOT-CONTAINS or ' ' I think? This is annoying t…

Genuinely curious, what can be used instead of Electron for cross-platform desktop app?

Qt is an option. Supported on Windows, Mac, and Linux.
Post reply on HN