Miller: Like Awk, sed, cut, join, and sort for CSV, TSV, and tabular JSON
1–10 of 109 posts
Re: Miller: Like Awk, sed, cut, join, and sort for CSV, TSV, and tabular JSON
#2Re: Miller: Like Awk, sed, cut, join, and sort for CSV, TSV, and tabular JSON
#3Re: Miller: Like Awk, sed, cut, join, and sort for CSV, TSV, and tabular JSON
#4 $ cat example.csv
color,shape,flag,index
yellow,triangle,1,11
red,square,1,15
red,circle,1,16
red,square,0,48
purple,triangle,0,51
red,square,0,77
# pretty printing
$ column -ts',' example.csv
color shape flag index
yellow triangle 1 11
red square 1 15
red circle 1 16
red square 0 48
purple triangle 0 51
red square 0 77
# sorting with skipped headers is a mess.
$ (head -n 1 example.csv && tail -n +2 example.csv | sort -r -k4 -t',') | column -ts','
color shape flag index
red square 0 77
purple triangle 0 51
red square 0 48
red circle 1 16
red square 1 15
yellow triangle 1 11Re: Miller: Like Awk, sed, cut, join, and sort for CSV, TSV, and tabular JSON
#5This is nice. I use `column` for pretty printing CSV/TSV but it fixes two tiny gaps in `sort` (skipping header lines) and `jq` (parsing CSV input. `jq` supports `@csv` for output conversion but not input). $ cat example.csv color,shape,flag,index yellow,triangle,1,11 red,square,1,15 red,circle,1,16 red,square,0,48 purple,triangle,0,51 red,square,0,77 # pretty printing $ column -ts',' example.csv color shape flag inde…
I like these command line tools, but I think they can cripple someone actually learning programming language. For example, here is a short program that does your last example:
Re: Miller: Like Awk, sed, cut, join, and sort for CSV, TSV, and tabular JSON
#6But seems like it cannot handle a simple use case: CSV without header.
$ mlr --csv head -n 20 pp-2002.csv
mlr: unacceptable empty CSV key at file "pp-2002.csv" line 1.
You have to explicitly pass it (FYI `implicit-csv-header` is terrible arg name)
$ mlr --csv --implicit-csv-header head -n 20 pp-2002.csv
While `head` obliges rightly
$ head -n 20 pp-2002.csv
Re: Miller: Like Awk, sed, cut, join, and sort for CSV, TSV, and tabular JSON
#7This is nice. I use `column` for pretty printing CSV/TSV but it fixes two tiny gaps in `sort` (skipping header lines) and `jq` (parsing CSV input. `jq` supports `@csv` for output conversion but not input). $ cat example.csv color,shape,flag,index yellow,triangle,1,11 red,square,1,15 red,circle,1,16 red,square,0,48 purple,triangle,0,51 red,square,0,77 # pretty printing $ column -ts',' example.csv color shape flag inde…
(sed -u '1q' ; sort -r -k4 -t',') Re: Miller: Like Awk, sed, cut, join, and sort for CSV, TSV, and tabular JSON
#8This is nice. I use `column` for pretty printing CSV/TSV but it fixes two tiny gaps in `sort` (skipping header lines) and `jq` (parsing CSV input. `jq` supports `@csv` for output conversion but not input). $ cat example.csv color,shape,flag,index yellow,triangle,1,11 red,square,1,15 red,circle,1,16 red,square,0,48 purple,triangle,0,51 red,square,0,77 # pretty printing $ column -ts',' example.csv color shape flag inde…
> sorting with skipped headers is a mess I like these command line tools, but I think they can cripple someone actually learning programming language. For example, here is a short program that does your last example: https://go.dev/play/p/9bASZ97lLWv
I like programming languages, but I think they can cripple someone actually learning Unix!
At the end of the day, you should just use whatever tools make you the most productive most quickly.
Re: Miller: Like Awk, sed, cut, join, and sort for CSV, TSV, and tabular JSON
#9BUT, leaks memory like crazy.
Despite documentation stating the verbs are fully streaming.
> Fully streaming verbs > These don't retain any state from one record to the next. They are memory-friendly, and they don't wait for end of input to produce their output.
https://miller.readthedocs.io/en/6.7.0/streaming-and-memory/...
Re: Miller: Like Awk, sed, cut, join, and sort for CSV, TSV, and tabular JSON
#10Get-Content .\example.csv | ConvertFrom-Csv | Where-Object -Property color -eq red