Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tad, a tabular data viewer

tadviewer.com

Post reply on HN
Print view View original

41–50 of 113 posts

Re: Tad, a tabular data viewer

#41

Post by mrmondo »

Macuyiko wrote: post #36

Nice, some initial remarks: - Sad to see it's an Electron app, but I don't want to start this flamewar all over again - Mid-sized CSV file opens fairly quick - If you want to build this into a full-fledged app, be prepared to handle a lot of CSV edge cases, also think about supporting AVRO, Parquet files - Regarding filtering: I can filter on CONTAINS and '=' but not on NOT-CONTAINS or ' ' I think? This is annoying t…

Good summary, thanks for the tldr;

+1 for the comment about Electronic/Javascript, also don't want to start a flame war heh.

exploratory.io looks really neat, it's a shame about the pricing and I'm not quite sure if it's a native app once you purchase or if it's an in-browser online-only app, because if it's online only 1) I'd be worried about uploading data to a 3rd party and 2) The internet in Australia is _terrible_ so uploading / downloading larger data sets might be painful.

Re: Tad, a tabular data viewer

#42

Post by camkego »

elcritch wrote: post #23
antonycourtney wrote: post #9

Earlier quoted context omitted.

I invested some effort to keep it performant even with fairly large CSV files, including a custom port of some C++ code for fast CSV import. My current favorite example is the Met Museum's 228MB 450k row collection data set; takes about 12 sec. to open in Tad on my 2013 MacBook Pro. Definitely not lag free (and hard to achieve that without going to some serious column store data warehouse like Amazon Redshift), but s…

Thanks for putting this out there! There are some projects out there using memory mapped files to do fast CSV parsing. Could be a nice way to speed up the memory loading and scroll it in real time. Can't find the link to the library I saw it used in, but it might be an interesting venue to consider. Another library that does it seems to be astropy fast ascii IO module [1]. [1]: http://docs.astropy.org/en/stable/io/as…

Try benchmarking OS read() calls vs. either sequential or random reads using memory mapping, whenever I do this OS read() calls end up being quite a bit faster.

Re: Tad, a tabular data viewer

#43

Post by Pamar »

mrmondo wrote: post #41
Macuyiko wrote: post #36

Nice, some initial remarks: - Sad to see it's an Electron app, but I don't want to start this flamewar all over again - Mid-sized CSV file opens fairly quick - If you want to build this into a full-fledged app, be prepared to handle a lot of CSV edge cases, also think about supporting AVRO, Parquet files - Regarding filtering: I can filter on CONTAINS and '=' but not on NOT-CONTAINS or ' ' I think? This is annoying t…

Good summary, thanks for the tldr; +1 for the comment about Electronic/Javascript, also don't want to start a flame war heh. exploratory.io looks really neat, it's a shame about the pricing and I'm not quite sure if it's a native app once you purchase or if it's an in-browser online-only app, because if it's online only 1) I'd be worried about uploading data to a 3rd party and 2) The internet in Australia is _terribl…

From their FAQ (https://exploratory.io/faq)

- I have very sensitive data. Is my data safe?

Any data you import into Exploratory Desktop always stays on your PC and never leave your PC unless you explicitly publish (share) it to Exploratory Cloud (exploratory.io). If you decided to publish the data to Exploratory Cloud for sharing or scheduling, you can share it in a private way so that so that only you and others you have invited can view it. The data is also stored in encrypted. Please take a look at our Privacy Policy for more details.

- Where exactly my data is stored after importing?

All the data you import into Exploratory Desktop is saved as a binary form (R's Rdata format) inside your repository, which is located under '/.exploratory' on your PC.

Re: Tad, a tabular data viewer

#44

Post by tconley »

Macuyiko wrote: post #36

Nice, some initial remarks: - Sad to see it's an Electron app, but I don't want to start this flamewar all over again - Mid-sized CSV file opens fairly quick - If you want to build this into a full-fledged app, be prepared to handle a lot of CSV edge cases, also think about supporting AVRO, Parquet files - Regarding filtering: I can filter on CONTAINS and '=' but not on NOT-CONTAINS or ' ' I think? This is annoying t…

exploratory.io looking promising. Thanks for the links and since it's an electron app then there's hope for a web version :)

Re: Tad, a tabular data viewer

#45

Post by TheAceOfHearts »

Related to this, if you're looking for a CLI tool for handling CSVs, xsv [0] is looking promising.

Based on prior experiences with CSV, one of the big problems I've seen has been figuring out what text encoding is being used. This problem appears to be more prominent with people outside of the US. It looks like Tad is using fast-csv, which I don't think will properly handle different file encodings. Life would be so much simpler if everyone just used UTF-8.

[0] https://github.com/BurntSushi/xsv

Re: Tad, a tabular data viewer

#46

Post by misterdata »

Just noticed the installer changes the file association for CSV's so they all open with Tad, without asking me first. Please do not do this.

Otherwise, nice app. How does it scale to larger CSV files? I have been working on something similar (https://warp.one) for Mac, which streams CSV files (apparently you use a SQLite database behind the scenes as cache?)

Re: Tad, a tabular data viewer

#48

Post by naushit »

MrMattWright wrote: post #28

This is absolutely terrific! I often work with csv or tsv and large data, and I just want to quickly look at it without loading excel or exporting to google sheets. https://atom.io/packages/tablr Tablr for Atom is good too. One feature request. I very often get data in JSON format (then convert to csv), it would be amazing to also support JSON. Thanks for sharing.

+1 Instead of "can I import X", please allow import preprocessor to import any kind of data. People can write customer pre-processor script. ( sometime for sanity check ).

Re: Tad, a tabular data viewer

#49

Post by baldfat »

Macuyiko wrote: post #36

Nice, some initial remarks: - Sad to see it's an Electron app, but I don't want to start this flamewar all over again - Mid-sized CSV file opens fairly quick - If you want to build this into a full-fledged app, be prepared to handle a lot of CSV edge cases, also think about supporting AVRO, Parquet files - Regarding filtering: I can filter on CONTAINS and '=' but not on NOT-CONTAINS or ' ' I think? This is annoying t…

> Sad to see it's an Electron app, but I don't want to start this flamewar all over again

Then stop mentioning it. Just don't use anything electron and don't mention it. I will use my RStudio and VS Code.

Re: Tad, a tabular data viewer

#50

Post by baldfat »

rabidrat wrote: post #25

If you prefer working from the terminal, you should check out VisiData ( https://github.com/saulpw/visidata ). It's a curses tabular data tool that can start browsing terabyte .csv files immediately, with few dependencies and no clicking.

Been using this daily. Great tool. I almost forgot the awk commands :) BUT I still love awk!
Post reply on HN