Live data from Hacker News

Is “data scientist” the new “programmer”?

blogs.harvard.edu

61–70 of 246 posts

Re: Is “data scientist” the new “programmer”?

#61
post #41

Something about this article strikes me as a thinly-veiled complaint about poorly designed object-oriented systems. Take, for example this comment by the author: >Even if the money were half of what today’s coder gets paid it might still be a better job because one is spared the tedium of looking at millions of lines of Java that do almost nothing! What all those millions of lines of code are is abstractions, decoupl…

nit: is it "about poorly-designed, object-oriented systems"?

Re: Is “data scientist” the new “programmer”?

#62
In my company I am a software engineer and my colleague is a data scientist, our current project that we work together on does a lot of NLU and NLP type work (think bots) and our skillsets often don't overlap and are both equally valuable to the projects success. That is, I tend to write the infrastructure and platform code that ties everything together and deal with all the software engineering type work, while my data scientist feeds in trained models and the likes. Both are necessary to handle contractual requests/responses as per our scope design.

Re: Is “data scientist” the new “programmer”?

#63

I wonder why "data engineer" isn't one of the suggested terms. Scientists do not really program science, nor do programmers research programs, as their respective fields of expertise.

To be nitpicky, in the US, engineer means you graduated from an ABET accredited program in something like: Chemical engineering, mechanical engineering, civil engineering, electrical engineering, industrial engineering, computer engineering....etc. That is not to say programming isn't a difficult job that requires a lot of analytical and creative thinking similar to an engineer. The difference is in getting a degree…

I don't think your nitpick is correct. To be a licensed engineer within some fields, you do need ABET but there are many people who are "engineers" (their job title) but aren't required to be licensed.

Re: Is “data scientist” the new “programmer”?

#64
Well I am "lucky" enough that I can work on a legacy application written in OpenACS. It wasn't written in the 70's but it's definitely old and outdated. So this kind of narrative that everything was better back then, simply does not convince me at all. I might be wrong of course, but the author tells anecdotes which is not a real argument, so there is that.

Re: Is “data scientist” the new “programmer”?

#65

Earlier quoted context omitted.

SQL is not more expressive than Turing Complete languages, no.

They’re all Turing complete inc SQL with case and recursion. I meant density, 1 line of a code, a SQL window function with a filter clause would probably take a page of Java to achieve the same result.

Nope, Java has map and filter just fine. Eg

``` Words.stream() .map(word -> String.toUpperCase(word)) .filter(word -> word.startsWith('A')) ```

Re: Is “data scientist” the new “programmer”?

#66

Earlier quoted context omitted.

To be nitpicky, in the US, engineer means you graduated from an ABET accredited program in something like: Chemical engineering, mechanical engineering, civil engineering, electrical engineering, industrial engineering, computer engineering....etc. That is not to say programming isn't a difficult job that requires a lot of analytical and creative thinking similar to an engineer. The difference is in getting a degree…

A Computer Engineering degree does not require ABET accreditation - it didn't when I went to UCSD... the difference between Computer Science and Computer Engineering was just 4 classes. Exactly the same calc/physics classes between the two, which were engineering level.

There are plenty of engineering programs without ABET accreditation. If you get a degree from a non accredited program, you'll likely have a different title than engineers who graduated from an ABET school and will make less. This is because there has to be some standardization in what the people who build bridges learn in school. All companies are different, but my company won't even look at an engineer without ABET accreditation. Even a Ph.D. from China, which is just really dumb. It is reality though.

Re: Is “data scientist” the new “programmer”?

#67

Earlier quoted context omitted.

The fact that R has such buy-in despite being a rather awful programming language (a friend of mine worked on the next Lisp-like version of R under Ross Ihaka, and the next version is based on the fact that current R is a bit awful) is precisely because it offers such convenience to non-programmers. In my sister company, they have data scientists, and data engineers. The data scientists write their algorithms in the…

Base R is annoying, but IMO the tidyverse alone makes the language worthwhile.

IMO it's the exact opposite. "Il mondo e' bello perche' e' vario".

Re: Is “data scientist” the new “programmer”?

#68

Earlier quoted context omitted.

Hah! At my company a decent proportion of engineers spend their lives scrambling to productionalize and operate the Lovecraftian concoctions of R and Python that our data scientists cook up on their laptops.

As a data analyst who is pushing towards data science, do you have any resources or advice on how to steer away from these "Lovecraftian concoctions" you're speaking of?

https://ai.google/research/pubs/pub43146

http://martin.zinkevich.org/rules_of_ml/rules_of_ml.pdf

https://en.wikipedia.org/wiki/Functional_programming

Re: Is “data scientist” the new “programmer”?

#69
post #41

Something about this article strikes me as a thinly-veiled complaint about poorly designed object-oriented systems. Take, for example this comment by the author: >Even if the money were half of what today’s coder gets paid it might still be a better job because one is spared the tedium of looking at millions of lines of Java that do almost nothing! What all those millions of lines of code are is abstractions, decoupl…

All of today's software is built on millions of lines of code. This isn't much of a problem when it's hidden away behind a good abstraction. Being written in another language forces the API to be documented well enough so you don't need to go deeper: "native code", "kernel code", "part of the browser".

Crappy million-line Java apps are generally crappy not due to raw line count but rather due to leaky abstractions and badly designed APIs, so you do have to navigate through a lot of that code.

Re: Is “data scientist” the new “programmer”?

#70
post #9

The basic premise of the article is that systems programming=irrelevant bloat and abstraction while data reduction=definite purpose and utility People writing python notebooks to do data analysis are probably fairly comparable to the scientific computing programmers of the past, but I feel like this picture tends to dismiss the computer science side of systems programming:things like GUIs, network code, processes and…

Yeah, the author's take is myopic. What they call bloat, people from the 70s would call wondrous: ubiquitous networking with and without wires, beautiful graphical interfaces, encryption everywhere (and expanding), far more open systems than proprietary re-engineered ones, the list goes on and on.

The author is Philip Greenspun, who in the 1980s worked with the people that created all of the things you listed: https://en.wikipedia.org/wiki/Philip_Greenspun

There is nothing myopic about his perspective.

Post reply on HN