An introduction to data processing on the Linux command line
blog.robertelder.org
An introduction to data processing on the Linux command line
1–10 of 73 posts
Re: An introduction to data processing on the Linux command line
#2Command Line tools are powerful beasts (e.g. awk) and they were always central to data preprocessing. But do we need to call it now a data science?
Re: An introduction to data processing on the Linux command line
#3Command Line tools are powerful beasts (e.g. awk) and they were always central to data preprocessing. But do we need to call it now a data science?
Yeah this article is about processing text data and not any form of statistics, modeling, etc. I'm guessing they added "data science" because it's in vogue? In any case, the provided title does not reflect the article.
Re: An introduction to data processing on the Linux command line
#4This is still useful information for data scientist who end up on Linux.
Re: An introduction to data processing on the Linux command line
#5Huh, so it turns out that I've been a 'data scientist' for over 20 years. Who knew?
Re: An introduction to data processing on the Linux command line
#6Good intro to data processing.
tsort and comm were news to me.
Re: An introduction to data processing on the Linux command line
#7This is great for anyone allergic to learning linux cli. One bit tha that erks me is the usage of 'one simple command' example on the awk section. Presumably, if i am not familiar with all these commands, I am not going to view the F->C conversion as simple.
Its simple to me bc i have strong familiarity but I find many tech tutorials occasionally forget newbies are...new.
Re: An introduction to data processing on the Linux command line
#8If you're interested in this space, a great resource can be found at https://www.datascienceatthecommandline.com/ (a free guide to go along with an orielly book)
Re: An introduction to data processing on the Linux command line
#9Command Line tools are powerful beasts (e.g. awk) and they were always central to data preprocessing. But do we need to call it now a data science?
Yeah this article is about processing text data and not any form of statistics, modeling, etc. I'm guessing they added "data science" because it's in vogue? In any case, the provided title does not reflect the article.
NLU. It relates to extracting intelligence from human language. Most of which comes in the form of text.
Re: An introduction to data processing on the Linux command line
#10Hi, (I wrote the article). A few people commented noting that I included "Data Science" in the title, but the content doesn't include any statistics or machine learning which is closer to the core definition of 'data science'. I still think the title is appropriate since any kind of low-fidelity data science task you do on some had-hoc data (log files, heaps of text, web pages) is going to start with setting up a processing pipeline that involves these commands. I could have re-named it "An intro to text processing" or "An intro to data processing", but then the people who need to see this content won't associate the title with something they're interested in, so they never benefit from it. The list of commands was chosen specifically with the question "What Linux commands would someone answering data science/business intelligence questions use?" in mind. These commands are also among the list of ones that are usually already installed on every system.