Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is “data scientist” the new “programmer”?

blogs.harvard.edu

Post reply on HN
Print view View original

221–230 of 246 posts

Re: Is “data scientist” the new “programmer”?

#221

Post by darkhorn »

j7ake wrote: post #146
darkhorn wrote: post #71

Earlier quoted context omitted.

In Python 0.1 + 0.2 is not equal to 0.3 because the result is 0.30000000000000004. In R 0.1 + 0.2 is equal to 0.3.

Try this: > print(.1+.2, digits=18) [1] 0.300000000000000044

In R 0.1 + 0.2 returns 0.3. In Python 0.1 + 0.2 returns 0.30000000000000004

Why do you compare point to point? I have never once in my Statistics education compared point to point. You always need to see the probability of the result if it is within 2 points.

But if you want to compare use all.equal(0.1 + 0.2, 0.3)

Re: Is “data scientist” the new “programmer”?

#222

Post by screye »

mlthoughts2018 wrote: post #104
dasil003 wrote: post #55

Earlier quoted context omitted.

If the data scientists know more about CI, unit testing, profiling, and caching than the engineers then they are better engineers and I'd wonder a bit about their math/stats chops and whether their role was just re-branded "data scientist" to keep up with trends.

It’s quite common to start out with a PhD / masters in math / stats, with deep specialization in fields like NLP, computer vision, MCMC sampling, and then to become an experienced expert in GPU computing, containerization, web service layers, etc., while working on implementations of ML models. This was true for me anyway. The main thing I do is deep learning for computer vision and image search, but I think it’s fai…

If you don't mind me asking, where do you work at? (or even a family of companies that have this type of roles)

What you describe sounds like where I want to be, and it would help to where I could go looking. (Deep Learning / Vision specialization with a math focused CS background but want to learn more core software skills going ahead)

Re: Is “data scientist” the new “programmer”?

#223

Post by Silhouette »

How did applications get so bloated and therefore boring to look at?

That's an easy one. Too many people who lacked sufficient experience to make their own informed judgements yet trusted consultants peddling soundbites instead of skilled and experienced developers who knew better.

If you have ever read a book or watched a talk by someone who advocated very short functions and minimal nesting, and you subsequently adjusted your personal programming style or corporate coding standards as a result, please do yourself a favour and go back and look at whether they offered any evidence -- anything at all -- or even just a reasonable argument that stands up to scrutiny -- to support their position.

The relatively plentiful resources in a lot of modern systems do remove one barrier that forced developers to do better, but I don't really believe that's a big factor. It's more that when you have an industry so focussed on young people, a lot of what happens is the one-eyed leading the blind, because too many people who have been around long enough to see the big picture get shipped off to management or other positions before they can pass on what they've learned widely enough to advance common practice.

Re: Is “data scientist” the new “programmer”?

#224

Post by learc83 »

3rdAccount wrote: post #56
vostok wrote: post #39

Earlier quoted context omitted.

> To be nitpicky, in the US, engineer means you graduated from an ABET accredited program in something like: Chemical engineering, mechanical engineering, civil engineering, electrical engineering, industrial engineering, computer engineering....etc. Do you happen to have a reference for this? At first glance, it seems to be incorrect rather than nitpicky. Anecdotally, I know plenty of people who do not have ABET acc…

Unless you have a professional engineers license, you can't testify in court as an engineer. To get a professional engineer's license, you must graduate from an ABET accredited program.

Since we're being nitpicky, neither of those claims are true.

>To get a professional engineer's license, you must graduate from an ABET accredited program.

In some states you can take the FE and PE exam with a related non-ABET accredited degree and work experience based, and in some states you can take the exams with no degree based on experience only. For example, NY let's you substitute work experience for a degree. There's a table where you can see how much experience you need based on your degree (or lack of one).

>Unless you have a professional engineers license, you can't testify in court as an engineer

Courts determine what credentials qualify someone to testify as an expert witness, not state licensing boards.

In some states, testifying in court as an engineer does qualify as practicing engineering without a license, and the state licensing board could fine you after the fact. However, in other states testifying in court doesn’t necessarily qualify as practicing engineering.

Re: Is “data scientist” the new “programmer”?

#225

Post by detaro »

darkhorn wrote: post #219
detaro wrote: post #74

Earlier quoted context omitted.

R: > 0.1+0.2 == 0.3 [1] FALSE ?

Why do you compare point to point, it is meaningless? The result for 0.1 + 0.2 shows 0.3. It doesn't show you wrong result. If you print it prints 0.3, nothing else. Why do you compare point to point? I have never once in my Statistics education compared point to point. You always need to see the probability of the result if it is within 2 points. But if you want to compare use all.equal(0.1 + 0.2, 0.3)

I thought that's what you mean by "equal". It didn't appear to me that you were merely talking about the default formatting of numbers printed on the REPL, which is utterly inconsequential to developing and porting applications.

Re: Is “data scientist” the new “programmer”?

#226

Post by rsynnott »

pjmlp wrote: post #176
gnulinux wrote: post #92

Earlier quoted context omitted.

Why is software engineering not a valid engineering? I worked on both software and hardware engineering and general principles seem to be the same. You deal with complexity and simplify it by making abstractions. You make calculations to make sure your project is feasible. It's not like EE and Aerospace engineering are literally the same field but there are some principles shared in those fields, and with software en…

Yes, while in many countries Software Engineering is actually a degree that needs accreditation and in some cases even a professional exam if you need to sign off projects as the responsible person, on US apparently you can call yourself software engineer after a three week bootcamp.

My degree was called computer science, and run by the school of maths rather than the school of engineering, but I’m eligible to apply to be accredited as a chartered engineer (the regulated professional title for engineers in this country), though people who graduated after 2012 aren’t, due to course changes. The whole thing is a bit of a mess.

Just to add to the confusion, my degree was a BA(Mod) rather than a BSc or BEng, for obscure historical reasons.

Re: Is “data scientist” the new “programmer”?

#227

Post by remram »

Why is Java, a byte-compiled statically-typed language, "bloated", while R it SQL, interpreted scripting languages, not? I find the point hard to follow. Execution of an R script will go through many more "layers", downloaded from many different sources, and implemented in different languages.

Re: Is “data scientist” the new “programmer”?

#228

Post by mlthoughts2018 »

screye wrote: post #222
mlthoughts2018 wrote: post #104

Earlier quoted context omitted.

It’s quite common to start out with a PhD / masters in math / stats, with deep specialization in fields like NLP, computer vision, MCMC sampling, and then to become an experienced expert in GPU computing, containerization, web service layers, etc., while working on implementations of ML models. This was true for me anyway. The main thing I do is deep learning for computer vision and image search, but I think it’s fai…

If you don't mind me asking, where do you work at? (or even a family of companies that have this type of roles) What you describe sounds like where I want to be, and it would help to where I could go looking. (Deep Learning / Vision specialization with a math focused CS background but want to learn more core software skills going ahead)

I work in a fairly mature startup that’s been around for over 10 years. It began primarily with an app, but shifted focus to other business areas. The image processing products are mostly related to offering information retrieval and search services for app users that have curated personal image collections.

I would say your description is accurate but only accidentally. The reason we have to learn more core engineering skills is that infrastructure will not take responsibility of bringing our solutions to production, and seek to limit the tools in our toolbox with policy.

It’s not fun when your everyday life is constant impedance mismatch against tools that infra will allow you to use. You can constantly see better / faster / safer / cheaper ways to solve problems, which have no downsides at all relative to the bad, slow, insecure, expensive ways infra currently makes everyone solve problems, but you just feel constantly sad that you are superficially prevented from the autonomy to select the efficient solution accordung to your creativity and skill.

Re: Is “data scientist” the new “programmer”?

#229

Post by quadyeast »

jiggunjer wrote: post #115
quadyeast wrote: post #61

Earlier quoted context omitted.

nit: is it "about poorly-designed, object-oriented systems"?

no.

It's "Something about poorly designed, object-oriented systems."

1. compound adjectives are hyphenated, except when the first world ends in 'ly.' 2. two adjectives, should have comma, similar to 'big, green systems.'

Re: Is “data scientist” the new “programmer”?

#230

Post by bloomer »

philipodonnell wrote: post #194
wiz21c wrote: post #162

Earlier quoted context omitted.

I've been hearing the same complaint between Cobol and Java for years : it was simpler before, more efficient, etc. Of course it was, but you were tied to one system (no application server), security was login/pw, database had no constraint, typing systems were ultra limited, everybody has its own way of writing batches (no Spring), business code was mixed with tons of technical code (no JPA). Now, sure, if you glue…

> But all of that exist just because we have digitalized all of the processes, data collection, etc. And the rise of data scientists will continue only if there are more stuff put in the databases, thanks to you plain, regular, normal programmers... IMHO there's far to little attention paid to how data might be valuable in an economic sense when storage strategies are being designed by database designers. I recently…

The net value of nearly all data is negative. It’s a liability not an asset. Unless you are actually selling data to customers or solving a specific problem that requires collection of specific data, it’s just wasting the company money. You would never hear a manufacturer say “ let’s stock pile aluminum in case we need it”. But that line of thinking comes up all the time with respect to data.
Post reply on HN