Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

R Passes SAS, but Python Leaves Them Both Behind

r4stats.com

Post reply on HN
Print view View original

51–60 of 71 posts

Re: R Passes SAS, but Python Leaves Them Both Behind

#51

Post by nl »

thearn4 wrote: post #32
fnord123 wrote: post #13

Earlier quoted context omitted.

If you work in a team mixing data engineers and data scientists then python has been a superior choice for a decade as the different team members are all using the same language to build the platform and to use it. We've had a lot of success weaning people of R and MATLAB in finance (so replace "data scientist" with "quant"). Of course if you work in a field that doesn't have the libraries and can't build them in hou…

Off-topic, but what is a "data engineer"?

In my team, "data engineer" refers to someone who deals with the ETL process, but also some feature engineering (ie, designing and extracting features. There's a lot of cross over here with the work Data Scientists do, but often the tools are different: for example, in the Python world the data engineer will use Pycharms while the data scientist will use Jupyter.

Re: R Passes SAS, but Python Leaves Them Both Behind

#53

Post by nl »

cwyers wrote: post #23
dredmorbius wrote: post #22

Earlier quoted context omitted.

"Microsoft was a difficult search since it appears in data science ads that mention other Microsoft products such as Windows or SQL Server. To eliminate such over-counting, I treated Microsoft different from the rest by including product names such as Azure Machine Learning and Microsoft Cognitive Toolkit. So there’s a good chance I went from over-emphasizing Microsoft to under-emphasizing it with only 157 jobs." Rea…

Okay, but what does that mean? Microsoft Cognitive Toolkit is like Tensorflow, which is included as an item on the list. Azure Machine Learning is something you can script with R or Python, and allows you to create APIs for predictive use on Azure. What good does lumping those together do? And what does it tell us that some things that are basically an R/Python library are less popular than R or Python themselves? It…

This is pretty much what data science in the real world is like.

Define the question you are interested in (in this case, a somewhat reasonable attempt to compare R/Python/SAS) and the put other things in blobs with a note that says this is what this blob is. Enjoy.

Re: R Passes SAS, but Python Leaves Them Both Behind

#54

Post by est »

rm999 wrote: post #27
fnord123 wrote: post #13

Earlier quoted context omitted.

If you work in a team mixing data engineers and data scientists then python has been a superior choice for a decade as the different team members are all using the same language to build the platform and to use it. We've had a lot of success weaning people of R and MATLAB in finance (so replace "data scientist" with "quant"). Of course if you work in a field that doesn't have the libraries and can't build them in hou…

This is a great point. Python has been a solid data engineer/pipeline language for years now. It makes it easy for data scientists and data engineers (or people who wear both hats) to work in the same environment and codebase.

The only area of un-touched by Python is web frontends and mobile apps.

Hope Web assembly would change that.

Re: R Passes SAS, but Python Leaves Them Both Behind

#55

Post by makeset »

CloudYeller wrote: post #24
rm999 wrote: post #9

I'm glad to see the quick convergence on Python. I've evaluated Python every ~2 years since 2006 for "data science" tasks (machine learning, statistics, data munging, and visualization). I'd argue that Python only properly covered this full data science stack 1.5-2 years ago. R covered this stack adequately probably around 2011-2012. Matlab had this before 2005. What makes Python a superior language to Matlab and R i…

Another reason why I'm glad python is winning (and a fun read): http://arrgh.tim-smith.us/

It's immature to rail on R for things like how its syntax doesn't look like that other language you already know, but yes, it's extremely quirky. And its unchecked flexibility, while great for experts providing DSLs and such, enables non-programmers to produce garbage implementations. The vast majority of R's myriad community packages (R's biggest strength!) seem to be written by a grad student who read half a tutorial.

Re: R Passes SAS, but Python Leaves Them Both Behind

#56

Post by blauditore »

R_haterade wrote: post #45
blauditore wrote: post #14

I once worked on a piece of software communicating with a SAS instance (doing live decision management), but everything about it seemed sketchy. No one using it really liked it, and its internals always appeared like a blackbox to me. Also, either it is terribly engineered or the devs working with it were just bad - we wanted a JSON-based REST API, but they said that it's "not possible with SAS", so we fell back to b…

Anecdotal, but our SAS decision support environment went down Friday. I can report back after the post-mortem if you like.

Yeah, would surely be interesting.

Re: R Passes SAS, but Python Leaves Them Both Behind

#57

Post by dredmorbius »

nl wrote: post #53
cwyers wrote: post #23

Earlier quoted context omitted.

Okay, but what does that mean? Microsoft Cognitive Toolkit is like Tensorflow, which is included as an item on the list. Azure Machine Learning is something you can script with R or Python, and allows you to create APIs for predictive use on Azure. What good does lumping those together do? And what does it tell us that some things that are basically an R/Python library are less popular than R or Python themselves? It…

This is pretty much what data science in the real world is like. Define the question you are interested in (in this case, a somewhat reasonable attempt to compare R/Python/SAS) and the put other things in blobs with a note that says this is what this blob is. Enjoy .

Bingo. Thanks. I was losing (patience|interest).

Re: R Passes SAS, but Python Leaves Them Both Behind

#58

Post by roystonvassey »

rm999 wrote: post #9

I'm glad to see the quick convergence on Python. I've evaluated Python every ~2 years since 2006 for "data science" tasks (machine learning, statistics, data munging, and visualization). I'd argue that Python only properly covered this full data science stack 1.5-2 years ago. R covered this stack adequately probably around 2011-2012. Matlab had this before 2005. What makes Python a superior language to Matlab and R i…

+ The versatility of Python. It is one of the "funnest" languages to work with because I'm always surprised by the number and variety of packages available to do all kinds of tasks.

Re: R Passes SAS, but Python Leaves Them Both Behind

#59

Post by fnord123 »

nerdponx wrote: post #35
thearn4 wrote: post #32

Earlier quoted context omitted.

Off-topic, but what is a "data engineer"?

I'm sure someone can come along with a better description, but it's kinda like devops/sysadmin but specifically for data storage and access. You could also look it up...

A snappier version which gives the flavour: Data scientists use {tensor flow, scikit-learn, parquet, hadoop, spark, scrapers, etc}. Data engineers write {tensor flow, scikit-learn, parquet, hadoop, spark, scrapers, etc}.

Re: R Passes SAS, but Python Leaves Them Both Behind

#60

Post by fnord123 »

nerdponx wrote: post #11
rm999 wrote: post #9

I'm glad to see the quick convergence on Python. I've evaluated Python every ~2 years since 2006 for "data science" tasks (machine learning, statistics, data munging, and visualization). I'd argue that Python only properly covered this full data science stack 1.5-2 years ago. R covered this stack adequately probably around 2011-2012. Matlab had this before 2005. What makes Python a superior language to Matlab and R i…

How do you feel about Julia? Data analysis in Python has plenty of warts.

http://danluu.com/julialang/
Post reply on HN