Show HN: An Open-Source Data Science Curriculum • Hacker News Board

Re: Show HN: An Open-Source Data Science Curriculum

#21

Post by hadley » Thu, Jan 16, 2014, 6:01 PM UTC

hootener wrote: post #13

I do a lot of statistical work, but wouldn't call myself a data scientist. To that end: > I geared the original curriculum toward Python tools and resources, so I've explicitly marked when resources use other tools to teach conceptual material (like R) Why did you choose Python over R? Personal preference, a bent toward Python in the online courses you found, or is Python generally considered the de facto language ch…

Python is definitely not the de facto choice. Python and R both have strengths and weaknesses, and relative use depends quite a bit on community. A few recent surveys (http://blog.revolutionanalytics.com/2014/01/in-data-scientis..., http://www.kdnuggets.com/2013/10/rexer-analytics-2013-data-m..., http://blog.revolutionanalytics.com/2013/09/top-languages-fo...) show strong growth for both R and python, and I expect that will continue in the future. There's no reason to limit yourself to one language, and knowing both R and python can only help. That said, you do need to be carefully about spreading yourself too thin, and you want to make sure you're an expert in at least one data analysis environment.

Re: Show HN: An Open-Source Data Science Curriculum

#22

Post by Ajoo » Thu, Jan 16, 2014, 6:01 PM UTC

A resource I've found invaluable and that I can't find listed is videolectures.net

Particularly http://videolectures.net/pascal/ has plenty of lectures and tutorials from their summer schools on very relevant topics for machine learning.

Re: Show HN: An Open-Source Data Science Curriculum

#23

Post by Beliavsky » Thu, Jan 16, 2014, 6:16 PM UTC

I see many books on Python listed. It's a good language for data analysis and scientific computing, especially with scipy, but there are alternatives, of course. I like Fortran 95, which is available in gcc as gfortran. A relevant book for data scientists would be "Developing Statistical Software in Fortran 95" (2005).

Re: Show HN: An Open-Source Data Science Curriculum

#24

Post by hdevalence » Thu, Jan 16, 2014, 6:37 PM UTC

Beliavsky wrote: post #23

I see many books on Python listed. It's a good language for data analysis and scientific computing, especially with scipy, but there are alternatives, of course. I like Fortran 95, which is available in gcc as gfortran. A relevant book for data scientists would be "Developing Statistical Software in Fortran 95" (2005).

Out of curiosity, why do you prefer Fortran?

Note: preemptive clarity: this isn't a language flamewar thing, I'm genuinely curious.

Re: Show HN: An Open-Source Data Science Curriculum

#25

Post by Beliavsky » Thu, Jan 16, 2014, 6:48 PM UTC

hdevalence wrote: post #24

Beliavsky wrote: post #23

I see many books on Python listed. It's a good language for data analysis and scientific computing, especially with scipy, but there are alternatives, of course. I like Fortran 95, which is available in gcc as gfortran. A relevant book for data scientists would be "Developing Statistical Software in Fortran 95" (2005).

Out of curiosity, why do you prefer Fortran? Note: preemptive clarity: this isn't a language flamewar thing, I'm genuinely curious.

Arrays in Fortran 90+ are a powerful feature -- there are whole array operations and operations on array slices, as in Matlab and Python with numpy. It's easy to allocate multidimensional arrays. Compilers are good at optimizing code -- if it's easiest to do something with loops you can go ahead and not worry about vectorizing the code, as you might with R or Matlab. There is a lot of statistics code in Fortran, so it's good to have at least a reading knowledge of it.

Re: Show HN: An Open-Source Data Science Curriculum

#26

Post by pyoung » Thu, Jan 16, 2014, 6:49 PM UTC

hadley wrote: post #21

hootener wrote: post #13

I do a lot of statistical work, but wouldn't call myself a data scientist. To that end: > I geared the original curriculum toward Python tools and resources, so I've explicitly marked when resources use other tools to teach conceptual material (like R) Why did you choose Python over R? Personal preference, a bent toward Python in the online courses you found, or is Python generally considered the de facto language ch…

Python is definitely not the de facto choice. Python and R both have strengths and weaknesses, and relative use depends quite a bit on community. A few recent surveys ( http://blog.revolutionanalytics.com/2014/01/in-data-scientis... , http://www.kdnuggets.com/2013/10/rexer-analytics-2013-data-m... , http://blog.revolutionanalytics.com/2013/09/top-languages-fo... ) show strong growth for both R and python, and I expec…

And lets not forget SAS, Matlab/Octave, Julia, STATA, SPSS, etc... There are a ton of choices out there. While not currently in vogue, there are a lot of SAS/STATA/SPSS jobs out there. Generally it is bigger, more established companies that use these software/languages, but if your goal is to get into a more stats focused position, these languages can be a good choice to learn.

Re: Show HN: An Open-Source Data Science Curriculum

#27

Post by SkyMarshal » Thu, Jan 16, 2014, 7:14 PM UTC

blakerson wrote: post #14

I'm biased as a non-technical analyst with an academic history, but I'm concerned that the curriculum doesn't meet the most basic needs. Quoting the section 'An Academic Shortfall': "Academic credentials are important but not necessary for high-quality data science. The core aptitudes – curiosity, intellectual agility, statistical fluency, research stamina, scientific rigor, skeptical nature – that distinguish the be…

The best advice I've both heard and passed on, when asked about hiring "data scientists", is you want someone who can look at raw data, massage it, and develop their own original opinions and insights about it, preferably derived from a deep understanding of the nuances of statistics and probability and the messy real world. It's the judgement and insight you're primarily looking for. So, hire the best mathematician, statistician, probabilist, economist, or heck biologist, epidemiologist, etc. you can find, and then teach them the vocational tools - hadoop, etc.

Re: Show HN: An Open-Source Data Science Curriculum

#28

Post by clarecorthell » Thu, Jan 16, 2014, 7:45 PM UTC

Beliavsky wrote: post #25

hdevalence wrote: post #24

Earlier quoted context omitted.

Out of curiosity, why do you prefer Fortran? Note: preemptive clarity: this isn't a language flamewar thing, I'm genuinely curious.

Arrays in Fortran 90+ are a powerful feature -- there are whole array operations and operations on array slices, as in Matlab and Python with numpy. It's easy to allocate multidimensional arrays. Compilers are good at optimizing code -- if it's easiest to do something with loops you can go ahead and not worry about vectorizing the code, as you might with R or Matlab. There is a lot of statistics code in Fortran, so i…

This is why I'm so in love with pandas (pandas.pydata.org) -- Wes McKinney did the world a favor creating a library with powerful, manipulable multidimensional data structures

Re: Show HN: An Open-Source Data Science Curriculum

#29

Post by clarecorthell » Thu, Jan 16, 2014, 7:55 PM UTC

hootener wrote: post #13

I do a lot of statistical work, but wouldn't call myself a data scientist. To that end: > I geared the original curriculum toward Python tools and resources, so I've explicitly marked when resources use other tools to teach conceptual material (like R) Why did you choose Python over R? Personal preference, a bent toward Python in the online courses you found, or is Python generally considered the de facto language ch…

Working with a technology that is more highly documented is better when you're starting out. Especially when you're teaching yourself and StackOverflow is your TA. Python is very appropriate for someone who's new to data work.

Otherwise, asking what technology to use is like asking what mode of transportation to use to get to a destination -- it's not the point. The important part is that you arrive. Some days walking over the mountain is the least sensible method, other days high seas make taking the boat around it impossible. The tool that gets the job done is the best tool.

Re: Show HN: An Open-Source Data Science Curriculum

#30

Post by the_watcher » Thu, Jan 16, 2014, 7:55 PM UTC

It would be nice to have a clear list of assumed capabilities (for example, I am familiar with basic programming and have built a few websites, but I haven't taken a math class since senior year of high school, and don't think I remember enough about Calculus to do anything that assumes knowledge of it). Just a simple list of what level math, stats, and programming fluency this starts from would be great.