There is also the (impossible to google) https://github.com/harelba/q that lets you SQL over CSV and is unix pipeline friendly.
Miller is like sed, awk, cut, join, and sort for name-indexed data such as CSV
11–20 of 81 posts
Re: Miller is like sed, awk, cut, join, and sort for name-indexed data such as CSV
#12This is lovely. Portable C, awesome.
Re: Miller is like sed, awk, cut, join, and sort for name-indexed data such as CSV
#13I sometimes dumb down CSV to what sqlite understands and then just import it there. Gonna have a look at this, but whether it's SQL or Ruby/Python/Perl structures, I generally prefer to spend the least amount possible within actual CSV. Even more so if it contains hierarchical data.
Re: Miller is like sed, awk, cut, join, and sort for name-indexed data such as CSV
#14This is nice! It'd be great to download a .deb package for it.
good idea. I also have a request for a homebrew formula, which looks simple enough to do.
Re: Miller is like sed, awk, cut, join, and sort for name-indexed data such as CSV
#15I have my own little toolkit of csv parsing tools that do most of these (using python's csv module.) I suspect most people who do deal with bulk CSV files all the time do as well. I'd be curious about how gracefully it handles large CSV files. The lack of quoting support kills it for my use case however. I wrote my own tools, starting with csv_cut because cut(1) didn't do quoting.
quoting is key but i've not yet stumbled for lack of it. but it's very much worth doing.
miller handles large files well. it's fast enough, and doesn't hold entire file contents unless it has to (e.g. sort). so gigabytes of input data are fine.
Re: Miller is like sed, awk, cut, join, and sort for name-indexed data such as CSV
#16This is lovely. Portable C, awesome.
He says he used GNU C99, how is that portable C?
Re: Miller is like sed, awk, cut, join, and sort for name-indexed data such as CSV
#17Re: Miller is like sed, awk, cut, join, and sort for name-indexed data such as CSV
#18To build: install flex and ctags, and -Wno-error=unused-but-set-variable
Re: Miller is like sed, awk, cut, join, and sort for name-indexed data such as CSV
#19Can anyone suggest something similar for Windows?
Re: Miller is like sed, awk, cut, join, and sort for name-indexed data such as CSV
#20This is lovely. Portable C, awesome.
Not totally portable. They have built some ctags option dependencies that I'm going to have to futz with in order to get the makefile to run.
12:58 shephard:c shephard$ make all
ctags -R .
ctags: illegal option -- R
usage: ctags [-BFadtuwvx] [-f tagsfile] file ...
make: *** [tags] Error 1
Edit2 - Now, it seems to be breaking on lemon not being explicitly called out in the path during a make:
lemon filter_dsl_parse.y
make[1]: lemon: No such file or directory
make[1]: *** [filter_dsl_parse.c] Error 1
make: *** [dsls] Error 2
13:07 shephard:c shephard$
]