11–20 of 156 posts
[1] http://csvkit.readthedocs.org/en/0.8.0/ [2] https://code.google.com/p/csvfix/ [3] https://unix.stackexchange.com/questions/7425/is-there-a-rob...
tac [file] | head -n 1
(Strange, OS X doesn't seem to have tac, but Cygwin does...)
caveat: delimiter-based commands are not quote-aware. For example, this is a CSV line with two fields: foo,"bar,baz" However, the tools will treat it as 3 columns: $ echo 'foo,"bar,baz"' | awk -F, '{print NF}' 3
Is there any workaround?
In my current workflow, I generate JSON from the excel files and use the really awesome JQ command (http://stedolan.github.io/jq/) to process
You know that thing about not making HTML with regexs? Same rule applies to CSV, TSV, and XLSX. All these can be created, manipulated and read using Python, which is probably already on your system.
>> If we don't want new file we can redirect the output to same file which will overwrite original file You need to be a little careful with that. If you do: uniq -u movies.csv > movies.csv The shell will first open movies.csv for writing (the redirect part) then launch the uniq command connecting stdout to the now emptied movies.csv. Of course when uniq opens movies.csv for consumption, it'll already be empty. There…
Thank you for inputs, how about this? uniq -u movies.csv > temp.csv temp.csv > movie.csv rm temp.csv
uniq -u movies.csv > /tmp/temp.csv && mv /temp/temp.csv movies.csv
1) I use this before using R or Python and ONLY do this when this is something I consistently need to be done all the time. Makes my R scripts shorter.
2) Somethings just need something simple to be fixed and these commands are just great.
Learn awk and sed and your tools just go much larger in munging data.
caveat: delimiter-based commands are not quote-aware. For example, this is a CSV line with two fields: foo,"bar,baz" However, the tools will treat it as 3 columns: $ echo 'foo,"bar,baz"' | awk -F, '{print NF}' 3
Is there any workaround?
Certain people might miss the point of why to use command line. 1) I use this before using R or Python and ONLY do this when this is something I consistently need to be done all the time. Makes my R scripts shorter. 2) Somethings just need something simple to be fixed and these commands are just great. Learn awk and sed and your tools just go much larger in munging data.