Is “data scientist” the new “programmer”? • Hacker News Board

Re: Is “data scientist” the new “programmer”?

#91

Post by squirrelicus » Wed, Sep 19, 2018, 1:57 AM UTC

skybrian wrote: post #69

thidr0 wrote: post #41

Something about this article strikes me as a thinly-veiled complaint about poorly designed object-oriented systems. Take, for example this comment by the author: >Even if the money were half of what today’s coder gets paid it might still be a better job because one is spared the tedium of looking at millions of lines of Java that do almost nothing! What all those millions of lines of code are is abstractions, decoupl…

All of today's software is built on millions of lines of code. This isn't much of a problem when it's hidden away behind a good abstraction. Being written in another language forces the API to be documented well enough so you don't need to go deeper: "native code", "kernel code", "part of the browser". Crappy million-line Java apps are generally crappy not due to raw line count but rather due to leaky abstractions an…

Don't forget that one spring scheduled job that goes rogue and wrecks your sanity constantly.

/mylife

Re: Is “data scientist” the new “programmer”?

#92

Post by gnulinux » Wed, Sep 19, 2018, 2:04 AM UTC

booleandilemma wrote: post #33

Sure, we’ve stolen the term “engineer” for long enough, let’s bother the scientists now.

Why is software engineering not a valid engineering? I worked on both software and hardware engineering and general principles seem to be the same. You deal with complexity and simplify it by making abstractions. You make calculations to make sure your project is feasible. It's not like EE and Aerospace engineering are literally the same field but there are some principles shared in those fields, and with software engineering. Am I missing something?

Re: Is “data scientist” the new “programmer”?

#93

Post by gnulinux » Wed, Sep 19, 2018, 2:09 AM UTC

aogl wrote: post #62

In my company I am a software engineer and my colleague is a data scientist, our current project that we work together on does a lot of NLU and NLP type work (think bots) and our skillsets often don't overlap and are both equally valuable to the projects success. That is, I tend to write the infrastructure and platform code that ties everything together and deal with all the software engineering type work, while my d…

My experience is very similar to this as a "software engineer" in a company who has 50% 50% split software engineers and data scientists.

Re: Is “data scientist” the new “programmer”?

#94

Post by gnulinux » Wed, Sep 19, 2018, 2:13 AM UTC

itronitron wrote: post #81

only if "software engineer" is the new "system administrator"

This doesn't make any sense. My job title is "software engineer" I never do any system administration. I produce code in python, javascript, C and SQL; never do any sort of administration. Sure, I occasionally deal with linux since our servers are linux and so some knowledge of it is useful, and I use unix tools pretty extensively (in OSX) since I prefer to write code this way. All the "software engineer"s I know have similar experience to mine with varying languages, so please suggest some evidence.

Re: Is “data scientist” the new “programmer”?

#95

Post by 110285591136 » Wed, Sep 19, 2018, 2:15 AM UTC

EdwardDiego wrote: post #42

danso wrote: post #36

This is such a bizarre post. The reason why people use a language like R is because it is easy to learn and use (and install, via RStudio) for data analysis without having to be a well-trained programmer. I can’t recall ever hearing from anyone who has relied on R, doing so because it was computationally efficient. The point of the language is convenience — particularly with how easy it is to create attractive graphi…

The fact that R has such buy-in despite being a rather awful programming language (a friend of mine worked on the next Lisp-like version of R under Ross Ihaka, and the next version is based on the fact that current R is a bit awful) is precisely because it offers such convenience to non-programmers. In my sister company, they have data scientists, and data engineers. The data scientists write their algorithms in the…

> ... despite being a rather awful programming language (...) it offers such convenience to non-programmers

I've heard people say similar things about MATLAB - that it's a poorly designed language, but many that people (mostly non-CS folk) use it out of convenience.

Can someone with experience using R explain what makes it so appealing to non-programmers? It seems like these two factors, "poorly designed" and "easy to use", should be at odds with each other.

Re: Is “data scientist” the new “programmer”?

#96

Post by nsriv » Wed, Sep 19, 2018, 2:16 AM UTC

EdwardDiego wrote: post #42

danso wrote: post #36

This is such a bizarre post. The reason why people use a language like R is because it is easy to learn and use (and install, via RStudio) for data analysis without having to be a well-trained programmer. I can’t recall ever hearing from anyone who has relied on R, doing so because it was computationally efficient. The point of the language is convenience — particularly with how easy it is to create attractive graphi…

The fact that R has such buy-in despite being a rather awful programming language (a friend of mine worked on the next Lisp-like version of R under Ross Ihaka, and the next version is based on the fact that current R is a bit awful) is precisely because it offers such convenience to non-programmers. In my sister company, they have data scientists, and data engineers. The data scientists write their algorithms in the…

R is cumbersome, but then you look at Matlab and Stata and try to explain that to social science graduate students that just managed to grok LaTex, and R starts looking like a streamlined vision of the future.

Re: Is “data scientist” the new “programmer”?

#97

Post by pizzazzaro » Wed, Sep 19, 2018, 2:19 AM UTC

danso wrote: post #36

This is such a bizarre post. The reason why people use a language like R is because it is easy to learn and use (and install, via RStudio) for data analysis without having to be a well-trained programmer. I can’t recall ever hearing from anyone who has relied on R, doing so because it was computationally efficient. The point of the language is convenience — particularly with how easy it is to create attractive graphi…

I think the post is referring to some idea of "glamour" or the lucrative nature of a rapidly emerging field.

Meanwhile, both demand that the employee spend all day telling a computer what to do.

Re: Is “data scientist” the new “programmer”?

#98

Post by anoncoward111 » Wed, Sep 19, 2018, 2:22 AM UTC

sixdimensional wrote: post #80

This post states that a data scientist uses compact languages such as SQL and R. Genuine question - do people really believe that being able to write and understand complex SQL makes you a data scientist? I ask because, I've been writing some of the nastiest, most difficult looking SQL around for probably at least 15 years. And yet, I would NOT call myself a data scientist because I know and can work with data and us…

youre being too hard on yourself and you should go apply for the big bucks. most scientists barely deserve the title

Re: Is “data scientist” the new “programmer”?

#99

Post by nicwilson » Wed, Sep 19, 2018, 2:30 AM UTC

110285591136 wrote: post #95

EdwardDiego wrote: post #42

Earlier quoted context omitted.

The fact that R has such buy-in despite being a rather awful programming language (a friend of mine worked on the next Lisp-like version of R under Ross Ihaka, and the next version is based on the fact that current R is a bit awful) is precisely because it offers such convenience to non-programmers. In my sister company, they have data scientists, and data engineers. The data scientists write their algorithms in the…

> ... despite being a rather awful programming language (...) it offers such convenience to non-programmers I've heard people say similar things about MATLAB - that it's a poorly designed language, but many that people (mostly non-CS folk) use it out of convenience. Can someone with experience using R explain what makes it so appealing to non-programmers? It seems like these two factors, "poorly designed" and "easy t…

(Havn't used much R, Have used MATLAB)

Its all about availability of libraries. I did control systems in undergrad, and despite being a shitty language, being able to describe and manipulate dynamic systems (ODEs) was very useful. Doing numerical integration by hand for the nonlinear systems was horrible, though better than simulink (which is about as much fun as using LabVIEW or sculling H2SO4).

Re: Is “data scientist” the new “programmer”?

#100

Post by jhbadger » Wed, Sep 19, 2018, 2:32 AM UTC

110285591136 wrote: post #95

EdwardDiego wrote: post #42

Earlier quoted context omitted.

The fact that R has such buy-in despite being a rather awful programming language (a friend of mine worked on the next Lisp-like version of R under Ross Ihaka, and the next version is based on the fact that current R is a bit awful) is precisely because it offers such convenience to non-programmers. In my sister company, they have data scientists, and data engineers. The data scientists write their algorithms in the…

> ... despite being a rather awful programming language (...) it offers such convenience to non-programmers I've heard people say similar things about MATLAB - that it's a poorly designed language, but many that people (mostly non-CS folk) use it out of convenience. Can someone with experience using R explain what makes it so appealing to non-programmers? It seems like these two factors, "poorly designed" and "easy t…

What makes R appealing is basically all statistical methods are available in it -- and it is often the initial implementation language of new methods in statistics. Often an R program involves very little programming as such other than to read in data, run some existing statistical methods on it and print or plot the results. I'm not a particular fan of the language itself (I kind of wish XLispStat hadn't died), but every time I feel like checking out Python or Julia I find things I need that haven't been implemented yet in those languages that are in R.