Is “data scientist” the new “programmer”? • Hacker News Board

Re: Is “data scientist” the new “programmer”?

#71

Post by darkhorn » Wed, Sep 19, 2018, 12:40 AM UTC

stochastic_monk wrote: post #10

closeparen wrote: post #6

Earlier quoted context omitted.

Hah! At my company a decent proportion of engineers spend their lives scrambling to productionalize and operate the Lovecraftian concoctions of R and Python that our data scientists cook up on their laptops.

Even the Stanford NLP Java implementations don’t always match the code they’re meant to be a translation of. I think R is a worse offender than Python.

In Python 0.1 + 0.2 is not equal to 0.3 because the result is 0.30000000000000004.

In R 0.1 + 0.2 is equal to 0.3.

Re: Is “data scientist” the new “programmer”?

#72

Post by mr_toad » Wed, Sep 19, 2018, 12:46 AM UTC

manfredo wrote: post #47

No, at least in my understanding data scientists specialize in the analysis of data rather than the development of software. You'd hire a data scientist to look for interesting patterns in data, or create machine learning models, and other data analysis tasks. These tasks may involve writing code, but it's usually specific to data analysis, often in R or Matlab or similar. A lot like how many people in the natural sc…

I think the point of the article was that it used to be more common back in the 60’s and 70’s for programmers to work on data problems. From basic stuff like census tabulation or designing file systems, to creating trigonometry or t-statistic tables, to AI.

There was less specialisation, less of a divorce between programmers and users.

There also seemed to be a conflation of computing and AI back then. Lisp was considered AI. And the early computing pioneers and theorists were strongly interested in AI, logic, and mathematics.

Re: Is “data scientist” the new “programmer”?

#73

Post by shoo » Wed, Sep 19, 2018, 12:49 AM UTC

closeparen wrote: post #6

sigfubar wrote: post #4

No. Data scientists exist at the mercy of programmers: without the tooling and the pipes, data science would not be a going concern.

Hah! At my company a decent proportion of engineers spend their lives scrambling to productionalize and operate the Lovecraftian concoctions of R and Python that our data scientists cook up on their laptops.

Ia! Ia! Prototype into production! Research Quality code! RPy2 Fhtagn!

Re: Is “data scientist” the new “programmer”?

#74

Post by detaro » Wed, Sep 19, 2018, 12:51 AM UTC

darkhorn wrote: post #71

stochastic_monk wrote: post #10

Earlier quoted context omitted.

Even the Stanford NLP Java implementations don’t always match the code they’re meant to be a translation of. I think R is a worse offender than Python.

In Python 0.1 + 0.2 is not equal to 0.3 because the result is 0.30000000000000004. In R 0.1 + 0.2 is equal to 0.3.

R:

    > 0.1+0.2 == 0.3
    [1] FALSE

?

Re: Is “data scientist” the new “programmer”?

#75

Post by analog31 » Wed, Sep 19, 2018, 12:54 AM UTC

dekhn wrote: post #63

3rdAccount wrote: post #37

Earlier quoted context omitted.

To be nitpicky, in the US, engineer means you graduated from an ABET accredited program in something like: Chemical engineering, mechanical engineering, civil engineering, electrical engineering, industrial engineering, computer engineering....etc. That is not to say programming isn't a difficult job that requires a lot of analytical and creative thinking similar to an engineer. The difference is in getting a degree…

I don't think your nitpick is correct. To be a licensed engineer within some fields, you do need ABET but there are many people who are "engineers" (their job title) but aren't required to be licensed.

Indeed, in the US, most engineers work under an "industrial exemption," for instance if they are employed within a company that makes a product, and not providing engineering services directly to the public. Most of the engineers at my workplace do not have licenses. On the other hand, our products get checked out by a certification lab, and the people who sign off on the test reports are in fact licensed. The work they do is phenomenally dull and bureaucratic.

Of course there are also fields where everybody pursues a license such as civil.

If I were to call myself an engineer, it would be in a field that doesn't have a discipline-specific license in my state. I don't know if it means that I don't need a license, or what. So far it has never been an issue.

Re: Is “data scientist” the new “programmer”?

#76

Post by geebee » Wed, Sep 19, 2018, 12:56 AM UTC

3rdAccount wrote: post #66

brandall10 wrote: post #40

Earlier quoted context omitted.

A Computer Engineering degree does not require ABET accreditation - it didn't when I went to UCSD... the difference between Computer Science and Computer Engineering was just 4 classes. Exactly the same calc/physics classes between the two, which were engineering level.

There are plenty of engineering programs without ABET accreditation. If you get a degree from a non accredited program, you'll likely have a different title than engineers who graduated from an ABET school and will make less. This is because there has to be some standardization in what the people who build bridges learn in school. All companies are different, but my company won't even look at an engineer without ABET…

Interesting. Here's a list of mid career pay by major. Fields that tend to be accredited and value accreditation are certainly present in high places on the list, but variants of mathematics are in comparable places. My guess is that "economics and mathematics" or "computer science and mathematics" are probably examples of fields that require a lot of math but aren't part of an ABET accredited degree. Doesn't look like there's much of a pattern here.

https://www.payscale.com/college-salary-report/majors-that-p...

Now, if you plan to work in structural or civil engineering or another field where accreditation is important, than yes, I would tend to agree that having an ABET accredited degree is important (as well as taking the PE exam for that field).

Fields where ABET accreditation isn't terribly important (computational finance, google, facebook, and so forth) often pay more than the fields where it is (mechanical, civil). My guess is that passing the PE for computer engineering wouldn't make a big difference in your compensation at a top tech company, I certainly don't know anyone at those companies who has bothered - though I certainly am willing to evidence to the contrary!

Re: Is “data scientist” the new “programmer”?

#77

Post by sampo » Wed, Sep 19, 2018, 12:58 AM UTC

darkhorn wrote: post #71

stochastic_monk wrote: post #10

Earlier quoted context omitted.

Even the Stanford NLP Java implementations don’t always match the code they’re meant to be a translation of. I think R is a worse offender than Python.

In Python 0.1 + 0.2 is not equal to 0.3 because the result is 0.30000000000000004. In R 0.1 + 0.2 is equal to 0.3.

Not on my R:

    > (0.1 + 0.2) == 0.3
    [1] FALSE

Re: Is “data scientist” the new “programmer”?

#78

Post by joker3 » Wed, Sep 19, 2018, 1:00 AM UTC

The output of a data scientist's work includes plenty of things that aren't code. Yes, the code that I write tends to be very short, but if it represented everything I had to do to get there it would be quite a bit longer.

Re: Is “data scientist” the new “programmer”?

#79

Post by closeparen » Wed, Sep 19, 2018, 1:02 AM UTC

natalyarostova wrote: post #22

closeparen wrote: post #6

Earlier quoted context omitted.

Hah! At my company a decent proportion of engineers spend their lives scrambling to productionalize and operate the Lovecraftian concoctions of R and Python that our data scientists cook up on their laptops.

As a data scientists I view other data scientists who need engineers to productionize their code with contempt. Perhaps (probably?) their data pipelines and systems are sufficiently more complicated than mine, but I'd feel embarrassed if I couldn't write production Python code

We don’t allow (new) Python code in production, which has something to do with it. Feature engineering is also a very different game with streaming systems and online datastores than with CSVs.

Re: Is “data scientist” the new “programmer”?

#80

Post by sixdimensional » Wed, Sep 19, 2018, 1:15 AM UTC

This post states that a data scientist uses compact languages such as SQL and R.

Genuine question - do people really believe that being able to write and understand complex SQL makes you a data scientist?

I ask because, I've been writing some of the nastiest, most difficult looking SQL around for probably at least 15 years. And yet, I would NOT call myself a data scientist because I know and can work with data and use SQL. It might make me a data engineer.

What would make me a scientist is the process, method and rigor I apply to data-driven research and in practice. It's not about what tool I use or how complicated that tool is.

I often get a whiff of imposter syndrome over this because, if being "great at SQL and R" is enough to get the big bucks as a data scientist, then I'm clearly doing it wrong. But, then again, maybe I'm being too literal thinking that a scientist means something different.