Live data from Hacker News

Ask HN: What journals and blogs should I be reading to become a data scientist?

news.ycombinator.com

31–40 of 46 posts

Re: Ask HN: What journals and blogs should I be reading to become a data scientist?

#31
I'd recommend Hadley Wickam's papers: http://vita.had.co.nz/

He is the prolific author of many R packages, which are more like little languages than libraries. His papers are both philosophical and practical, and informed by writing a huge amount of code.

The first one on that page is really good, and along with another paper of his got me explicitly thinking of organize my data in R using the relational model (a thing people with computer science backgrounds will know well).

It made me realize that R is actually a better SQL. It's a language for tables, or an algebra of tables.

Re: Ask HN: What journals and blogs should I be reading to become a data scientist?

#32
post #30
post #9

Seen on twitter today: https://twitter.com/jeremyjarvis/status/428848527226437632 "A data scientist is a statistician who lives in San Francisco."

I like that. Even worse would be "an actuary who lives in SF."

What would that be?

Am I being dense?

Re: Ask HN: What journals and blogs should I be reading to become a data scientist?

#33
post #10

You know, I absolutely see where the poster is coming from, and the suggestions look helpful so far, but the question might as well read: What journals and blogs should I be reading to become a Cardiothoracic Surgeon? (though hopefully nobody bleeds out on a table when someone misconstrues statistical data) We've lived through an amazing time where one could learn by doing, and talented people have been able to compe…

Just a bit of a counterpoint (taken from a comment on the Data Tau site):

"Data kiddies like me are coming. I just ran multiple passes of the Broyden–Fletcher–Goldfarb–Shanno algorithm with a 100-layer neural network on a tfidf-vectorized dataset. I have no clue what that all exactly means, all I know is that it took under an hour and it gives a higher (top 10%) AUC score. Kaggler amateurs are beating the academics by brute force or smarter use of the many tools that are currently freely available. Show a regular Python dev some examples and library docs and she can compete in ML competitions. I was getting good results with LibSVM before I even understood how SVM's work on the surface. Feed the correct input format and some parameters and you are good to go. Random Forests can be applied to nearly anything and get you 75%+ accuracy. Maybe I am just a engineer looking for pragmatic and practical use of techniques from ML and data science. Hard data scientists will be the statisticians, the algorithmic theory experts, the experimental physicists. It takes me 7 years to understand a complex mathematical paper. It takes me 7 minutes to train a model and predict a 1 million test set with Vowpal Wabbit."

The point is that a Data Scientist is really a person who is a blend of statistician and software engineer. Sure, there are brilliant people who will invent new ML algorithms, but you don't need to invent that stuff to be of tremendous value to a business who has data that they aren't currently getting much value out of. Just as a software engineer at a small business doesn't need to write a database, she just needs to be able to implement one somebody else wrote to add tremendous value.

Re: Ask HN: What journals and blogs should I be reading to become a data scientist?

#34
post #32
post #30

Earlier quoted context omitted.

I like that. Even worse would be "an actuary who lives in SF."

What would that be? Am I being dense?

I'm guessing it's saying that the new 'data scientist' is more an actuary than a statistician. Just a bunch of predictive models trying to enhance a bottom line more than anything else.

Re: Ask HN: What journals and blogs should I be reading to become a data scientist?

#35
post #9

Seen on twitter today: https://twitter.com/jeremyjarvis/status/428848527226437632 "A data scientist is a statistician who lives in San Francisco."

BigDataBorat's classic: "Data Science is statistics on a Mac."

Re: Ask HN: What journals and blogs should I be reading to become a data scientist?

#37
post #5

Becoming a data scientist isn't a matter of reading journals and blogs. You can get a sense of the field and what is required by reading those sites but becoming a data scientist is years of hard work. You need to develop serious skills in at least 4 of the following disciplines. Statistical analysis RDMS query development NoSQL databases Machine learning Natural Language Processing Web crawling and data harvesting t…

Well that’s just, like, your opinion, man. I’m not a “data scientist” (or statistician, for that matter), but of the (excellent) data scientists that I know, the only specific skill they really have in common is statistical analysis. I’d say the truth is probably closer to “statistical analysis + ability to do independent research + computational chops using whatever their tools of choice may be"

As a data scientist, I have to agree with his opinion.

Usually you have a team where each person is "specialized" in a few of those categories.

You can call a data scientist a statistician, but I don't think you can necessarily call a statistician a data scientist.

The truth is, you need only a shallow understanding of machine learning and stats to be a data scientist. But you also need the know-how to collect data - this ends up being the much bigger issue to tackle in my environment. (For what it's worth, you need to have a strong understanding of how data points relate to one another, how accurate they might be, why they might not be accurate, and you also need to be constantly thinking about the long term vision for your data.)

Re: Ask HN: What journals and blogs should I be reading to become a data scientist?

#38
post #33
post #10

You know, I absolutely see where the poster is coming from, and the suggestions look helpful so far, but the question might as well read: What journals and blogs should I be reading to become a Cardiothoracic Surgeon? (though hopefully nobody bleeds out on a table when someone misconstrues statistical data) We've lived through an amazing time where one could learn by doing, and talented people have been able to compe…

Just a bit of a counterpoint (taken from a comment on the Data Tau site): "Data kiddies like me are coming. I just ran multiple passes of the Broyden–Fletcher–Goldfarb–Shanno algorithm with a 100-layer neural network on a tfidf-vectorized dataset. I have no clue what that all exactly means, all I know is that it took under an hour and it gives a higher (top 10%) AUC score. Kaggler amateurs are beating the academics b…

I absolutely agree, and in that sense I'm sort of living the dream already. But even skilled people don't always know what they don't know, and that can show though somewhat more easily in this field.

Re: Ask HN: What journals and blogs should I be reading to become a data scientist?

#39
post #33
post #10

You know, I absolutely see where the poster is coming from, and the suggestions look helpful so far, but the question might as well read: What journals and blogs should I be reading to become a Cardiothoracic Surgeon? (though hopefully nobody bleeds out on a table when someone misconstrues statistical data) We've lived through an amazing time where one could learn by doing, and talented people have been able to compe…

Just a bit of a counterpoint (taken from a comment on the Data Tau site): "Data kiddies like me are coming. I just ran multiple passes of the Broyden–Fletcher–Goldfarb–Shanno algorithm with a 100-layer neural network on a tfidf-vectorized dataset. I have no clue what that all exactly means, all I know is that it took under an hour and it gives a higher (top 10%) AUC score. Kaggler amateurs are beating the academics b…

Sure, most anyone can throw some data into an SVM and get a result out, maybe even a good one. The problem comes when someone like this has to answer questions beyond a simple 90% accuracy rate. What does the computed separation direction tell me? Could I improve accuracy by using some a priori information like how often one class occurs in relation to the other? What 10% of the population am I failing on? Is it an important part? Is there some easy way I could do better? Is my data so high dimensional that I'm getting some trivial separation and not anything driven by the data itself?

And what happens when this person gets a new data set and they are suddenly getting garbage out of some standard SVM? Is it just a matter of the data not being well-separated using a linear model but throwing some simple kernel at the SVM will do the trick?

Even something as simple as taking a mean can fall apart when you are dealing with data which doesn't live in a Euclidean space, let alone something like PCA or SVM which also make assumptions of linearity.

The point is, it isn't just about being able to invent new methods. Things like SVM make assumptions about your data and applying them in cases when these assumptions don't hold can give completely worthless information, even if it looks good on the surface. Using something you don't understand, even if it is at a (much) more basic level than someone with a PhD in statistics, is just asking for trouble.

Re: Ask HN: What journals and blogs should I be reading to become a data scientist?

#40
Grab this set: http://shop.oreilly.com/category/get/data-science-kit.do for Data Science, and maybe this set aswell: http://shop.oreilly.com/category/get/machine-learning-kit.do if you're into Machine Learning.

Both from O'Reilly (with some Packt mixed in). Excellent content.

Post reply on HN