Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Data Science Is America’s Hottest Job

bloomberg.com

Post reply on HN
Print view View original

151–160 of 224 posts

Re: Data Science Is America’s Hottest Job

#151

Post by Nomentatus »

joker3 wrote: post #93
fnbr wrote: post #85

Earlier quoted context omitted.

I would certainly agree with you, but I don't think someone with a master's degree is necessarily an expert. What's important, IMO, is having practical experience in data science, which may or may not come with a graduate degree.

No, it takes a lot more than a master's degree. Even a PhD is only really entry-level. There's no substitute for practical experience and talking to someone very experienced. Shame there aren't a lot more very experienced statisticians and data scientists out there.

Shame there aren't more concert pianists out there... but given the leverage big data can give (a small percentage change in what a big organization does can still save or earn big bucks) this is a competitive position, just as playing concerts is only for the few.

You want a better employee than your competitors have. You'll take all the expertise you can get, plus the ability to invent new techniques as needed. Everybody wants the very best 'cause that last 1% of skill can still be worth a lot of money. Everybody wants someone who can hit a lot of long balls for their team, as it were. There might be a floor on the skill level you could make some use of, but there's no ceiling.

Re: Data Science Is America’s Hottest Job

#152

Post by mywittyname »

joker3 wrote: post #74
kitanata wrote: post #41

I recently quit a gig and spent a few months looking for a data science/machine learning gig. I was surprised just how gatekeeped these positions were. Everyone wanted a PhD or a masters degree. I have 15 years of experience in software. I’ve done everything to low level game programming and graphics programming to web development, to AI (but not as a specific position title). The gatekeeping in this field surprised…

Data science is like security: it's very easy to come up with something that looks really good but is horribly broken in subtle ways that you won't find until it's too late. I think there's a lot of value in having lots of people be literate at it, but part of that literacy is understanding that it's a hard problem and that it does need attention from dedicated specialists.

> it's very easy to come up with something that looks really good but is horribly broken in subtle ways that you won't find until it's too late.

My experience has been that this doesn't matter in a lot of cases. A product can be both, highly useful and subtly broken.

One of the biggest boons to my productivity was realizing when something is good enough and to move on. You can waste so much time tuning for precision and recall.

Re: Data Science Is America’s Hottest Job

#153

Post by clatan »

You'll also need a strong background in statistics if you wanna bring value to a company other than "machine learn all the things".

I'd even argue that - for data science - it's more important than any ammout of years doing engineering.

Re: Data Science Is America’s Hottest Job

#154

Post by monadmancer »

The White Walkers of the data science field are out the box enterprise solutions. These are enterprise software, data science consulting and ops solutions in a package. The corporate customer need not hold in house a data science team. The ambitions of the enterprise solutions (think DataRobot, H2o, etc) are to effectively bring one click production ready solutions that even C-level can participate in.

I see this is as the greatest threat to the demand of the "in house" data scientist.

If this turns out to be the case, I see the greatest demand for those who can write production grade code (i.e. software engineers) and those who are effectively trained data scientists. We see this job often called research scientist or research engineer.

Re: Data Science Is America’s Hottest Job

#155

Post by aje403 »

kitanata wrote: post #87
hellofunk wrote: post #83

Earlier quoted context omitted.

Of the skills necessary to succeed in data science, the ability to program is actually the least important. Perhaps you were over-emphasising your software skills at the expense of really demonstrating proper understanding of the science of data analysis. A data science position is not going to require your experience in low-level game programming, for example. In fact, being able to program at all is secondary, sinc…

This right here is a perfect example of why this position is gatekeeped so much. Having a PhD doesn’t automatically mean you think like a scientist of a mathematician, and having a bachelor’s degree and 15 years of experience writing code and then putting in the effort to read Elements of Statistical Learning, compeleting several online courses in machine learning and data science, studying probability, combinatorics…

Actually no, a lot of PhD's are a bit smarter than you, some PhD's are a lot smarter than you

Re: Data Science Is America’s Hottest Job

#156

Post by claytonjy »

joker3 wrote: post #74
kitanata wrote: post #41

I recently quit a gig and spent a few months looking for a data science/machine learning gig. I was surprised just how gatekeeped these positions were. Everyone wanted a PhD or a masters degree. I have 15 years of experience in software. I’ve done everything to low level game programming and graphics programming to web development, to AI (but not as a specific position title). The gatekeeping in this field surprised…

Data science is like security: it's very easy to come up with something that looks really good but is horribly broken in subtle ways that you won't find until it's too late. I think there's a lot of value in having lots of people be literate at it, but part of that literacy is understanding that it's a hard problem and that it does need attention from dedicated specialists.

Along the same line, I find the hardest thing to get good at for junior data-sciency people is how to tell if your model sucks. Too easy to trick yourself into thinking you have a good model.

I think this is an essential, foundational skill for modern data science, not unlike some knowledge of R or Python, but an order of magnitude or two harder to learn. A lot of the difficulty is in lack of good educational material. I also think this is significantly more important than a deep understanding of various modeling approaches; you can learn that as you go, so long as you know if what you have now is "good enough" or if you need a fancier approach.

Unfortunately you can't just throw k-fold CV around, average some fit metrics, and call it a day.

Re: Data Science Is America’s Hottest Job

#157

Post by abhishekjha »

sannee wrote: post #35
frockington wrote: post #31

Earlier quoted context omitted.

Do a linear regression, call it ML with AI and you'll be running your own team in a week

Just the other day, I heard someone talking about a "single layer neural network with no activation"...

What does that look like?

Re: Data Science Is America’s Hottest Job

#158

Post by achompas »

peatmoss wrote: post #115
brational wrote: post #96

Earlier quoted context omitted.

To be fair though, can't you say the same about the title "software engineer". Without further qualifiers such a title can run quite the gamut of skill-sets.

I see more segmentation in software roles: front-end, back-end, full-stack, embedded, devops, realtime systems are all labels that get you in roughly the right place. Beyond that, there are labels that refer to specific technologies that get you even closer: Rails, Elixir, Django, React, QT, C#/Winforms, Verilog. There really aren’t the equivalent shorthand filters in data science.

> There really aren’t the equivalent shorthand filters in data science.

There are, though, right? The catch-all "Data Scientist" title is being split into Data Engineer, Machine Learning Engineer, Data/BI Analyst, etc. Each of those, in my experience, have clear definitions.

Re: Data Science Is America’s Hottest Job

#159

Post by whataretensors »

kitanata wrote: post #41

I recently quit a gig and spent a few months looking for a data science/machine learning gig. I was surprised just how gatekeeped these positions were. Everyone wanted a PhD or a masters degree. I have 15 years of experience in software. I’ve done everything to low level game programming and graphics programming to web development, to AI (but not as a specific position title). The gatekeeping in this field surprised…

This is commonplace. I think it's due to the dilbert effect. If you can't judge candidates based on ability, you have to judge them based on accreditation. Larger organizations tend to have more layers of non-technical people evaluating technical candidates.

It's not particular to ML, though ML is worse than others. CS degrees are luckily becoming less needed for basic programming jobs.

Re: Data Science Is America’s Hottest Job

#160

Post by whataretensors »

mikert5671 wrote: post #67
cup-of-tea wrote: post #65

Earlier quoted context omitted.

You're right, it's not that hard. A lot of the gatekeeping is because the departments are run by nontechnical people who just don't understand your skill set. They also need to guard the bullshit and not let anyone realise it's just really shoddy programming at the end of the day.

this is 100% wrong

That's not a rebuttal. It's absolutely non-technical gatekeepers looking for certificates.

Graduates get mad because they realize they've wasted a bunch of money on something they can learn for free, so they reject the notion and make others go through the same abusive system.

Post reply on HN