21–30 of 73 posts
Can somebody explain the advantage of doing it on the command line vs in Python or R? What would a practical use case look like?
Why people use Linux in place of *nix ? Even worst, most of the tools (cat, grep, awk) are Unix commands, redeveloped by the GNU project in most of the GNULinux distros.
Oh well, I guess a lot of people just think all Unix-like systems are called “Linux” now. Perhaps it’s become like the word “Kleenex”.
Hi, (I wrote the article). A few people commented noting that I included "Data Science" in the title, but the content doesn't include any statistics or machine learning which is closer to the core definition of 'data science'. I still think the title is appropriate since any kind of low-fidelity data science task you do on some had-hoc data (log files, heaps of text, web pages) is going to start with setting up a pro…
For anyone who is interested in going a little deeper into data science, I’d also recommend the “Introduction to Data Science with R” series by David Langer:
Useless use of cat detected! Rememeber, nearly all cases where you have: cat file | some_command and its args ... you can rewrite it as: and in some cases, such as this one, you can move the filename to the arglist as in: some_command and its args ... file — Randal L. Schwartz ( http://porkmail.org/era/unix/award.html#cat )
I actually prefer useless cat because when you're prototyping a pipeline it's very awkward to use non-useless cat. You'll probably start off with something like this to observe the content of the file:
cat something.txt
cat something.txt | grep stuff
grep stuff cat something.txt | grep stuff | sed 's/"//g'
cat something.txt | grep -v negmatch | grep stuff
Useless use of cat detected! Rememeber, nearly all cases where you have: cat file | some_command and its args ... you can rewrite it as: and in some cases, such as this one, you can move the filename to the arglist as in: some_command and its args ... file — Randal L. Schwartz ( http://porkmail.org/era/unix/award.html#cat )
hah, I knew someone would point that out (which is why I talked about it in the article). I actually prefer useless cat because when you're prototyping a pipeline it's very awkward to use non-useless cat. You'll probably start off with something like this to observe the content of the file: cat something.txt Using this doesn't work in bash: Then, continuing with useless cat to build on it you do cat something.txt | g…
Can be the starting command. When you no longer need the head there, just get rid of “head |”.
Although I agree that the pointing out of “useless cat” is usually not particularly useful or constructive.
Earlier quoted context omitted.
hah, I knew someone would point that out (which is why I talked about it in the article). I actually prefer useless cat because when you're prototyping a pipeline it's very awkward to use non-useless cat. You'll probably start off with something like this to observe the content of the file: cat something.txt Using this doesn't work in bash: Then, continuing with useless cat to build on it you do cat something.txt | g…
If you’re working with a lot of data you probably want to pipe it into head anyway, initially, so Can be the starting command. When you no longer need the head there, just get rid of “head |”. Although I agree that the pointing out of “useless cat” is usually not particularly useful or constructive.
Why people use Linux in place of *nix ? Even worst, most of the tools (cat, grep, awk) are Unix commands, redeveloped by the GNU project in most of the GNULinux distros.
Hi, (I wrote the article). A few people commented noting that I included "Data Science" in the title, but the content doesn't include any statistics or machine learning which is closer to the core definition of 'data science'. I still think the title is appropriate since any kind of low-fidelity data science task you do on some had-hoc data (log files, heaps of text, web pages) is going to start with setting up a pro…
awk -F, '$2 == "F" {$0=(($1-32)*5/9)",C"} {print}'