If you prefer working from the terminal, you should check out VisiData ( https://github.com/saulpw/visidata ). It's a curses tabular data tool that can start browsing terabyte .csv files immediately, with few dependencies and no clicking.
Tad, a tabular data viewer
81–90 of 113 posts
Re: Tad, a tabular data viewer
#82Earlier quoted context omitted.
> Sad to see it's an Electron app, but I don't want to start this flamewar all over again Then stop mentioning it. Just don't use anything electron and don't mention it. I will use my RStudio and VS Code.
I think it is ok to express displeasure at the state of something without just silently doing something else and hoping that the entire ecosystem catches up to you. For example, I take every opportunity (like right now, for example) to shit on node.js and npm for that very reason. There is so much evangelism on one side, it is nice to hear the opposing view points.
It is, but it's not okay to bring up a common flame war topic and then say "but I don't want to start a flame war". If you don't want to start one, just don't start one. Starting it, and then saying you didn't want to, is disingenuous at best.
Re: Tad, a tabular data viewer
#83Nice, some initial remarks: - Sad to see it's an Electron app, but I don't want to start this flamewar all over again - Mid-sized CSV file opens fairly quick - If you want to build this into a full-fledged app, be prepared to handle a lot of CSV edge cases, also think about supporting AVRO, Parquet files - Regarding filtering: I can filter on CONTAINS and '=' but not on NOT-CONTAINS or ' ' I think? This is annoying t…
I believe my http://easymorph.com would also count as competition. It loads CSV files and queries databases, filters, pivots/unpivots, cleanses, aggregates, merges. Free edition. Windows only (yeah, Mac owners, I know).
Re: Tad, a tabular data viewer
#84Earlier quoted context omitted.
I think it is ok to express displeasure at the state of something without just silently doing something else and hoping that the entire ecosystem catches up to you. For example, I take every opportunity (like right now, for example) to shit on node.js and npm for that very reason. There is so much evangelism on one side, it is nice to hear the opposing view points.
Ok, so what are some legit criticisms of node.js and npm? I'm no fanboy, but every time I try to ask this, the answers are along the lines of "if you install packages in a different order, you get different results." Which is fair, but everybody installs using `npm i`, so that's mostly a moot point. Is there any substance here? I think it's fair to talk about this, as long as we go out of our way not to reduce it to…
All the JS-the-language criticisms apply, those are probably the biggest at this point. The classic "node.js is cancer" is still somewhat relevant though I never thought it made its point well enough on how forced-asynchronous is at least as bad as the forced OOP and checked exceptions we put up with in e.g. Java again. Callback hell was long a common complaint, until people found Promises / the waterfall structure, but then the complaint is debugging hell. The new await stuff in node 8 sounds nice at least.
For a long time npm didn't get stuff over https by default, I think it does for a year now? The only other criticisms I can think of are just shoddy engineering that turns into drama (I remember seeing something about their colorized output would slow everything down quite a lot) or more taste-level things like how seemingly trivial concepts like leftpad are core libraries (that go through multiple versions because it couldn't have just been done right the first time) and the drama involved when something everyone depends on has an issue.
Disclaimer, I haven't done production node since ~2013. It was actually mostly enjoyable, and I wouldn't mind doing it again, despite on the taste-level I think the whole node ecosystem is just worse than many other options (when you have a choice) for many specific problems. But life is a continuous lesson in Worse is Better.
Re: Tad, a tabular data viewer
#85Earlier quoted context omitted.
I believe my http://easymorph.com would also count as competition. It loads CSV files and queries databases, filters, pivots/unpivots, cleanses, aggregates, merges. Free edition. Windows only (yeah, Mac owners, I know).
Looks fantastic. I've been complaining for years that there wasn't any program like this, and even started many times working on it. Thumbs up!
Re: Tad, a tabular data viewer
#86Nice, some initial remarks: - Sad to see it's an Electron app, but I don't want to start this flamewar all over again - Mid-sized CSV file opens fairly quick - If you want to build this into a full-fledged app, be prepared to handle a lot of CSV edge cases, also think about supporting AVRO, Parquet files - Regarding filtering: I can filter on CONTAINS and '=' but not on NOT-CONTAINS or ' ' I think? This is annoying t…
All great feedback, thanks! Some quick replies: - The omission of ' ' from the operator list was just an embarrassing mistake on my part. #facepalm #WILLFIX - Also agree that a drop-down list of checkboxes (ideally a searchable one!) with an 'in' operator for low cardinality columns would be great. But the UI for that is a bit involved, and probably also need to provide some user control over when to do this since ga…
It's a modified dataTable(so search is builtin. I didn't see search in Tad yet), with selectize for categorical variable filters(which is a searchable dropdown list)
Another good to have feature is to show column index. We often need to manipulate the columns in code, a column index is helpful.
Based on column index, you can also select a subset of columns faster with numeric input -- over 100 columns are normal, using checkbox to select is too cumbersome.
Re: Tad, a tabular data viewer
#87Earlier quoted context omitted.
I think it is ok to express displeasure at the state of something without just silently doing something else and hoping that the entire ecosystem catches up to you. For example, I take every opportunity (like right now, for example) to shit on node.js and npm for that very reason. There is so much evangelism on one side, it is nice to hear the opposing view points.
Ok, so what are some legit criticisms of node.js and npm? I'm no fanboy, but every time I try to ask this, the answers are along the lines of "if you install packages in a different order, you get different results." Which is fair, but everybody installs using `npm i`, so that's mostly a moot point. Is there any substance here? I think it's fair to talk about this, as long as we go out of our way not to reduce it to…
I am glad you asked! You know how everyone runs to circle-jerk the praise "use the right tool for the job"? Well, Javascript is NEVER the right answer when the "job" is business logic or systems development or almost anything else. But here we are.
Do you know why people think Javascript is fucking cancer? Because it was developed to animate visitor counters and help with animated menu and resize issues that css wasn't up to at the time. It doesn't have strong types and rigid structure because it was created to be a language that assists your browser in little ways. So a little more history is in order.
Along comes Microsoft and creates XMLHttpRequest which could offer a "real-time" feel to some websites, and it was awesome! This was the first horseman of the apocalypse.
Then Google blew everyone's fucking minds with google maps (inc. google earth) and gmail. And was basically single-handedly responsible for the Web 2.0 (single-page app) explosion...and the second horse of the apocalypse.
Javascript grew and metastasized in every way imaginable, because everyone felt that single-page apps were exactly what they needed for their new toilet-sharing app idea, and so it went.
Then one dark day Node.js emerged from is leathery egg and blew everyone's fucking mind again with what could be done when you get a genius to abuse the javascript engine for entertainment. And the worst pale horse of the apocalypse rode forth.
For, you see, now webpage developers who know two days worth of shitty Javascript could now call themselves "full-stack developers". And oh my fucking god did they develop! Every afterbirth of an idea was haphazardly put together into a short script and posted to blow everyones fucking minds with how awesome their new 1,000-line sentence capitalization script was.
Now with this torrent of javascript libraries from all the new developers that didn't know they were developers but totally were, came a problem. How could we get all of the diarrhea into everyone's mouths as efficiently as possible? I mean how are you going to write your single-page todo-list webapp without the 300 supporting javascript libraries?
So npm came to the rescue! People saw how shitty the dependency hell that RPM was and said, "How can we make this worse in every possible way?" And the last terrible horse of the apocalypse was born.
Now all the pieces were there for people that don't know what they are doing to do those things on a large scale, with nothing remotely close to a reasonable development/debugging environment, and inject themselves as dependencies everywhere in the matrix. And then, of course, capriciously abandon the idea a week later.
So rather than list all the valid criticism of the ways Javascript is being abused, just ask yourself "If I should use the right tool for the job, when is Javascript it?"
I will tell you now that if "the job" is web-server, database, command line tool, yet another web-application framework (maybe), anything that needs reasonable or reliable error and failure handling, anything that looks like or acts like an desktop app, or anything that doesn't look or act like an desktop app (except for visitor counters and animated menus), or anything that any living human will ever have to support or debug ... then the answer is no.
Re: Tad, a tabular data viewer
#88Nice, some initial remarks: - Sad to see it's an Electron app, but I don't want to start this flamewar all over again - Mid-sized CSV file opens fairly quick - If you want to build this into a full-fledged app, be prepared to handle a lot of CSV edge cases, also think about supporting AVRO, Parquet files - Regarding filtering: I can filter on CONTAINS and '=' but not on NOT-CONTAINS or ' ' I think? This is annoying t…
Genuinely curious, what can be used instead of Electron for cross-platform desktop app?
Re: Tad, a tabular data viewer
#89Earlier quoted context omitted.
OpenRefine http://openrefine.org/ for years has been my go to tool when looking at csv or tsv. I really love this project. > OpenRefine (formerly Google Refine) is a powerful tool for working with messy data: cleaning it; transforming it from one format into another; and extending it with web services and external data.
Oh thanks for reminding me, I did look at that a while ago, it's absolutely great. The transforming steps are amazing. Time to have another look at it. Exploratory.io is similar (nice transformation steps using dplyr) but for me it was just a bit too close to actually writing the R code.
Re: Tad, a tabular data viewer
#90Feature Suggestions (top level comment to hold various suggestions, so upvoting can be used per-suggestion to bubble the best to the top)
- a very important use case for my team in support of business users who are comfortable in Excel which cannot manage large (>1M rows) files. Tad is useful even with 10M+ rows, a user can import, review datafile and filter/pivot to desired subset. The next step for them would be exporting and creating formatted tables or charts for analysis/reporting documents.