Is “data scientist” the new “programmer”? • Hacker News Board

Re: Is “data scientist” the new “programmer”?

#201

Post by mywittyname » Wed, Sep 19, 2018, 2:24 PM UTC

thisisit wrote: post #172

vinayms wrote: post #127

The take down on abstraction and software engineers (by using Java as an example) is similar to saying "back in the day to find a prime number we would simply use a sieve, but today it is a tedium, what with all the pi's and e's and thetas that get in the way, and what are geometry and polynomials doing here, and what in the god's name is this i , I just want to count the prime numbers which are nice round whole numb…

> who otherwise had never dealt with statistics before. And why is statistics required? Let's face it most companies who need "Data Scientists" are looking for regular BI guys with fancy terms. Most of the problems are solvable using out of box functionalities in python/keras etc. Sure there are places and problems which require hard mathematics and stats but those are few and far between.

This is true in my experience. Data Scientist seem to run the gamut from "knows SQL" to "has a Ph.D in Behavioral Psych and spent four years getting scientific results published in peer-reviewed journals".

The company that I work for has changed role titles from Data Analyst to Data Scientist specifically because people who know SQL, but don't program, won't apply to/accept jobs without that title.

Re: Is “data scientist” the new “programmer”?

#202

Post by thousandautumns » Wed, Sep 19, 2018, 2:26 PM UTC

natalyarostova wrote: post #22

closeparen wrote: post #6

Earlier quoted context omitted.

Hah! At my company a decent proportion of engineers spend their lives scrambling to productionalize and operate the Lovecraftian concoctions of R and Python that our data scientists cook up on their laptops.

As a data scientists I view other data scientists who need engineers to productionize their code with contempt. Perhaps (probably?) their data pipelines and systems are sufficiently more complicated than mine, but I'd feel embarrassed if I couldn't write production Python code

Just what the data science needed more of: gatekeeping.

Data science is an extremely broad, vague buzzword encompassing a variety of jobs and skillsets, most of which have existed for decades under different names. You do work that involves putting models into production in Python, congrats. The insistence that all data scientists must also do so is silly, especially considering that there are surely many skills used by many data scientists that you are incapable of.

Re: Is “data scientist” the new “programmer”?

#203

Post by Jtsummers » Wed, Sep 19, 2018, 2:27 PM UTC

thousandautumns wrote: post #196

Jtsummers wrote: post #86

Deming, from Out of the Crisis (1986): People with master's degrees in statistical theory accept jobs in industry and government to work with computers. It is a vicious cycle. Statisticians do not know what statistical work is, and are satisfied to work with computers. People that hire statisticians likewise have no knowledge about statistical work, and somehow suppose that computers are the answer. Statisticians and…

I'm not sure I follow. Specifically, what is meant by, "Statisticians do not know what statistical work is, and are satisfied to work with computers."?

What he meant was that they (being fresh out of school) don't know what statistical work for business is. They can crunch numbers, but they don't actually know what questions they need to be answering for their employers. And their employers don't really know what's needed either, just that they need a statistician. So they end up doing number crunching on the computers, which is all fine and good, but of relatively low value. Additionally, and many who have programmed can attest to this, computers offer an emotional satisfaction when you work with them: Oh, cool, I finished this neat Travis CI integration so now my workflow is more automated. But I spent two weeks doing that and what value have I added to the business? (Not that automation is bad, but people get distracted by the side problems and not the core problem.)

Of course things have changed, you don't have to invent your own statistical packages anymore. But see some comments elsewhere in this post: People are saying that the data scientists are the ones that know process automation better than anyone else in their offices. The ones who best understand docker and continuous integration. This leads to a question: Why are they so good at that? Is it because it's solving real problems for them and letting them be more effective? Or are they like every "data scientist" in the last couple offices I worked in: They have no real work to do because no one knows what's expected of them, so they solve interesting (to them) problems rather than business problems.

I'm not trying to knock the whole field, but it's a trend we've seen play out before. Smart companies and smart people figure out that they need X. Or they discover a technique or process that works well (see devops). They do it, they create a position called Xian or X Scientist or X Analyst. Now everyone wants to be like the successful guys and start imitating, without comprehending the value or purpose of the work or process. Lots of people take on the new title, schools offer courses in the techniques they use, but with a poor emphasis (due to time or their own lack of understanding) on the business case for it.

The current trend with data scientist is no different. There's positive value when it's understood, and negative value when it's not (best worst case: just an extra body being fed but doing no harm to the business other than the cost of their salary).

Re: Is “data scientist” the new “programmer”?

#204

Post by dr_zoidberg » Wed, Sep 19, 2018, 2:28 PM UTC

thisisit wrote: post #172

vinayms wrote: post #127

The take down on abstraction and software engineers (by using Java as an example) is similar to saying "back in the day to find a prime number we would simply use a sieve, but today it is a tedium, what with all the pi's and e's and thetas that get in the way, and what are geometry and polynomials doing here, and what in the god's name is this i , I just want to count the prime numbers which are nice round whole numb…

> who otherwise had never dealt with statistics before. And why is statistics required? Let's face it most companies who need "Data Scientists" are looking for regular BI guys with fancy terms. Most of the problems are solvable using out of box functionalities in python/keras etc. Sure there are places and problems which require hard mathematics and stats but those are few and far between.

Because data science and machine learning are applied statistics, and if you don't understand how it works under the hood (and not necesarily a very deep understanding, sometimes just a broad understanding is enough) you will have trouble adjusting things, debugging edge cases, or simply not know why something works and something else doesn't.

(edited for slightly better clarity)

Re: Is “data scientist” the new “programmer”?

#205

Post by CodeSheikh » Wed, Sep 19, 2018, 2:32 PM UTC

In the absence of marketeers (think bootcamps) and recruiters the alternative or correct title would have been: 'Is “statistician” the new “programmer”?'

Re: Is “data scientist” the new “programmer”?

#206

Post by casper345 » Wed, Sep 19, 2018, 2:34 PM UTC

> How did applications get so bloated and therefore boring to look at?

I love reading code at this level. 1. You get to see into the mind of the programmer and learn new techniques. 2. Boring?! 3. A good text editor makes it attractive to look at for hours :)

Re: Is “data scientist” the new “programmer”?

#207

Post by smt88 » Wed, Sep 19, 2018, 2:35 PM UTC

CodeSheikh wrote: post #205

In the absence of marketeers (think bootcamps) and recruiters the alternative or correct title would have been: 'Is “statistician” the new “programmer”?'

A lot of data scientists are doing very light statistics, though. Many of them don't have formal degrees in stats at all. There's a huge range of ability represented by that job title, ans "statistician" doesn't capture the lower end.

Re: Is “data scientist” the new “programmer”?

#208

Post by wiz21c » Wed, Sep 19, 2018, 2:45 PM UTC

philipodonnell wrote: post #194

wiz21c wrote: post #162

Earlier quoted context omitted.

I've been hearing the same complaint between Cobol and Java for years : it was simpler before, more efficient, etc. Of course it was, but you were tied to one system (no application server), security was login/pw, database had no constraint, typing systems were ultra limited, everybody has its own way of writing batches (no Spring), business code was mixed with tons of technical code (no JPA). Now, sure, if you glue…

> But all of that exist just because we have digitalized all of the processes, data collection, etc. And the rise of data scientists will continue only if there are more stuff put in the databases, thanks to you plain, regular, normal programmers... IMHO there's far to little attention paid to how data might be valuable in an economic sense when storage strategies are being designed by database designers. I recently…

>> The preconception that you have to be maximally efficient with storage has led to huge quantities of valuable data being lost.

I 100% agree. Many programmers are trained as if memory, CPU are finite resources. Although it's true, in many cases, that reality may be safely ignored, opening tons of opportunities (to store data, to develop faster because you don't optimize, etc.).

At a time where my phone has gigabytes of memory, I'm always surprised that some people ask me to put a limit on a text field. I understand the technicalities behind the question, but from a conceptual point of view, that's often pointless.

Re: Is “data scientist” the new “programmer”?

#209

Post by jeffreyrogers » Wed, Sep 19, 2018, 2:46 PM UTC

There is a good comment on the original article by a user named LauraConrad. I'm excerpting it so HN readers will see it:

> I was a “Programmer” in the ’70’s, and I keep thinking how much of what my early programs did would be done by a spreadsheet now (or any time since the late ’80’s).

Re: Is “data scientist” the new “programmer”?

#210

Post by chosenbreed » Wed, Sep 19, 2018, 2:46 PM UTC

oldandtired wrote: post #164

vinayms wrote: post #127

The take down on abstraction and software engineers (by using Java as an example) is similar to saying "back in the day to find a prime number we would simply use a sieve, but today it is a tedium, what with all the pi's and e's and thetas that get in the way, and what are geometry and polynomials doing here, and what in the god's name is this i , I just want to count the prime numbers which are nice round whole numb…

A good programmer is someone who can communicate with the problem domain experts and provide an a solution to fit their problem. Someone who understands the limitations of the computing environment and can engineer a solution that is adequate for the problem space problem. Many who consider themselves programmers produce solution space solutions that just don't get to the core of the problem space problems. This is a…

> Many who consider themselves programmers produce solution space solutions that just don't get to the core of the problem space problems. This is a function of the simple fact that many programmers never have the opportunity to see what the problem space experts are doing or actually need. This is a real shortcoming in the education of programmers.

I can't argue with that. I would add that another factor is how the work is structured and presented to the programmers. In many places programmers largely disconnected from any users. The programmers are usually given a set of requirements by a third party who themselves derived it from someone other than a user/consumer. Thus, programmers may end producing lovely programs that don't actually address the needs of users/consumers.