Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Data Science Is America’s Hottest Job

bloomberg.com

Post reply on HN
Print view View original

81–90 of 224 posts

Re: Data Science Is America’s Hottest Job

#81

Post by nightski »

mikert5671 wrote: post #58
kitanata wrote: post #41

I recently quit a gig and spent a few months looking for a data science/machine learning gig. I was surprised just how gatekeeped these positions were. Everyone wanted a PhD or a masters degree. I have 15 years of experience in software. I’ve done everything to low level game programming and graphics programming to web development, to AI (but not as a specific position title). The gatekeeping in this field surprised…

I actually think that machine learning can be much harder than you are claiming here, and that your inexperience is showing when you make such claims. Sure a toy problem from some dataset in a book or pulled off the internet is easy. So is implementing a k-means algorithm. But go to a large corporation with 100 datasets, all with 100 fields each, each having significant seasonal bias (among other biases), and build s…

Often this comes down to the ability to communicate, work with people, and understanding real world business. In addition, often the more simple a learning method used the better. I actually find most businesses needs can be solved by straightforward analytical methods such as traditional statistics. Very few situations require actual machine learning and it is often misapplied.

In fact, having a PhD does not tell you a whole lot about these skills.

More advanced modern techniques such as deep neural networks, reinforcement learning, etc.. are all extremely proficient at certain niche problems but these do not come up nearly as often in a business context.

This is why I don't advertise myself as a machine learning engineer. Rather, I am a business consultant that knows when, and when not to utilize machine learning methods.

Re: Data Science Is America’s Hottest Job

#82

Post by lainga »

galieos_ghost wrote: post #76
kitanata wrote: post #41

I recently quit a gig and spent a few months looking for a data science/machine learning gig. I was surprised just how gatekeeped these positions were. Everyone wanted a PhD or a masters degree. I have 15 years of experience in software. I’ve done everything to low level game programming and graphics programming to web development, to AI (but not as a specific position title). The gatekeeping in this field surprised…

The Data Scientist "shortage" is another scam by the big tech companies to drive down wages. My boss had hundreds of applications for the position he was looking to fill and the majority were so well qualified that it was basically a coin flip between multiple PhDs with great resumes. Tech worker's arrogance reminds me of "made in the USA" factory workers who thought their jobs were safe. If people don't start pushin…

wouldn't a real or fake shortage of talent drive wages up, not down?

Re: Data Science Is America’s Hottest Job

#83

Post by hellofunk »

kitanata wrote: post #41

I recently quit a gig and spent a few months looking for a data science/machine learning gig. I was surprised just how gatekeeped these positions were. Everyone wanted a PhD or a masters degree. I have 15 years of experience in software. I’ve done everything to low level game programming and graphics programming to web development, to AI (but not as a specific position title). The gatekeeping in this field surprised…

Of the skills necessary to succeed in data science, the ability to program is actually the least important. Perhaps you were over-emphasising your software skills at the expense of really demonstrating proper understanding of the science of data analysis. A data science position is not going to require your experience in low-level game programming, for example. In fact, being able to program at all is secondary, since many can pick up some basic skills in languages like Octave, R or even Python, to support their mastery of the math. And it's also not so much math implementation but also a genuine scientist's eye on how to approach a problem. There are no cookie-cutter formulas you can throw at a non-trivial problem: you have to really understand the field and the art of analysis to do it well.

Re: Data Science Is America’s Hottest Job

#84

Post by ItsMe000001 »

lainga wrote: post #82
galieos_ghost wrote: post #76

Earlier quoted context omitted.

The Data Scientist "shortage" is another scam by the big tech companies to drive down wages. My boss had hundreds of applications for the position he was looking to fill and the majority were so well qualified that it was basically a coin flip between multiple PhDs with great resumes. Tech worker's arrogance reminds me of "made in the USA" factory workers who thought their jobs were safe. If people don't start pushin…

wouldn't a real or fake shortage of talent drive wages up, not down?

Yes, no.

Yes if it's real. No if it's fake.

This still ignores the question of if entry limits for jobs are set reasonably or not though, you can create a real shortage by limiting yourself needlessly. At which point those who think that it is needless will call it "fake", so which word you use also depends on opinion and interpretation of data :-)

Re: Data Science Is America’s Hottest Job

#85

Post by fnbr »

joker3 wrote: post #74
kitanata wrote: post #41

I recently quit a gig and spent a few months looking for a data science/machine learning gig. I was surprised just how gatekeeped these positions were. Everyone wanted a PhD or a masters degree. I have 15 years of experience in software. I’ve done everything to low level game programming and graphics programming to web development, to AI (but not as a specific position title). The gatekeeping in this field surprised…

Data science is like security: it's very easy to come up with something that looks really good but is horribly broken in subtle ways that you won't find until it's too late. I think there's a lot of value in having lots of people be literate at it, but part of that literacy is understanding that it's a hard problem and that it does need attention from dedicated specialists.

I would certainly agree with you, but I don't think someone with a master's degree is necessarily an expert. What's important, IMO, is having practical experience in data science, which may or may not come with a graduate degree.

Re: Data Science Is America’s Hottest Job

#86

Post by usrusr »

lainga wrote: post #82
galieos_ghost wrote: post #76

Earlier quoted context omitted.

The Data Scientist "shortage" is another scam by the big tech companies to drive down wages. My boss had hundreds of applications for the position he was looking to fill and the majority were so well qualified that it was basically a coin flip between multiple PhDs with great resumes. Tech worker's arrogance reminds me of "made in the USA" factory workers who thought their jobs were safe. If people don't start pushin…

wouldn't a real or fake shortage of talent drive wages up, not down?

A fake shortage drives wages down by increasing supply.

Also, you create a fake shortage by driving wages down: "we have so many positions that we cannot fill (at the price that we are willing to pay)"

Re: Data Science Is America’s Hottest Job

#87

Post by kitanata »

hellofunk wrote: post #83
kitanata wrote: post #41

I recently quit a gig and spent a few months looking for a data science/machine learning gig. I was surprised just how gatekeeped these positions were. Everyone wanted a PhD or a masters degree. I have 15 years of experience in software. I’ve done everything to low level game programming and graphics programming to web development, to AI (but not as a specific position title). The gatekeeping in this field surprised…

Of the skills necessary to succeed in data science, the ability to program is actually the least important. Perhaps you were over-emphasising your software skills at the expense of really demonstrating proper understanding of the science of data analysis. A data science position is not going to require your experience in low-level game programming, for example. In fact, being able to program at all is secondary, sinc…

This right here is a perfect example of why this position is gatekeeped so much. Having a PhD doesn’t automatically mean you think like a scientist of a mathematician, and having a bachelor’s degree and 15 years of experience writing code and then putting in the effort to read Elements of Statistical Learning, compeleting several online courses in machine learning and data science, studying probability, combinatorics, graph theory, does not mean you don’t have a scientific mindset. A scientific mindset can be learned. You are not special because you have a PhD and I don’t. The only difference between you and me is that I had the ability to learn for free what you paid for. But when you go “oh hah, he’s just a coder and only has a bachelors degree” and you won’t even call me to talk to me that means you are missing out, and you are gatekeeping. You are not special and you are not smarter than me. You’ve just read a book I haven’t and wrote a white paper. I can read that book too. I can write a paper too.

I can do everything you can do.

Re: Data Science Is America’s Hottest Job

#88

Post by ItsMe000001 »

tormeh wrote: post #77
Mc91 wrote: post #64

Earlier quoted context omitted.

I know a mid-level manager at a Fortune 100 company. Somewhere up the management chain, someone read an article about how data science is the hot new thing that your company has to be doing or it will have missed the boat. So word was sent down that their next project was to do data science. So now they are launching a data science project to analyze their business data, which this manager is overseeing. They're hiri…

That's unfair. "Not missing the boat" is a natural need for a company. Everyone with an MBA has studied companies that missed the boat. Throwing some money at various new fads isn't necessarily irrational.

Nevertheless, it does make one reconsider million dollar compensation packages if all those guys can come up with is to deploy randomness. Either they have foresight and a plan or they don't. Either way is fine, but only the first option supports the million dollar salaries for executives. If they are just like any other mortal... they didn't even do anything to identify new trends, they just followed what was in the major business magazines.

Re: Data Science Is America’s Hottest Job

#89

Post by [deleted] »

mikert5671 wrote: post #67
cup-of-tea wrote: post #65

Earlier quoted context omitted.

You're right, it's not that hard. A lot of the gatekeeping is because the departments are run by nontechnical people who just don't understand your skill set. They also need to guard the bullshit and not let anyone realise it's just really shoddy programming at the end of the day.

this is 100% wrong

[deleted]

Re: Data Science Is America’s Hottest Job

#90

Post by peatmoss »

CuriouslyC wrote: post #57
peatmoss wrote: post #21

Earlier quoted context omitted.

My goodness, are you me? I've been having exactly the same thoughts. Provided one finds a data science role that roughly aligns with your training / interests, the actual work is comically easy compared to what you go through in academia. A boot camp can easily teach someone to, say, estimate a linear model or run k-means. I dread the future when the industry decides the right way to put up barriers is by creating ev…

A boot camp probably won't tell the process to make sure a linear model is the right choice. It probably also won't tell you when you should and shouldn't use k-means. And in most cases you probably won't have very good answers as to the certainty of your models. Imagine you're hiring someone to build a house for you. Would you feel comfortable with someone who's just been drilled on how to use individual tools? I wo…

> A boot camp probably won't tell the process to make sure a linear model is the right choice.

Very true. On the other hand, it's a pleasant rarity when I see positions that appear to index more heavily on, "how well is this person able to conceptualize the problem and choose an appropriate method?" than "can this person do X?"

Lots of folks can do X; fewer can conceptualize a research question and choose the appropriate X; even fewer can carry out the X and communicate robustly what it means.

The latter two start to get into squishy territory, but also are where the value is. They also seem to get the least focus in advertising / recruiting / interviewing data scientists.

It reminds me of studying evaluation methods in planning. One that people are really familiar with (at least anecdotally) is cost-benefit analysis. Conceptually, it's very simple. The problem is that the costs and benefits that are hardest to measure are very often NOT measured. And they're very often the sorts of things that people find the most important. So, you end up with an answer that encodes a ratio of easily measured things rather than important things.

So too with data science. Easier to check whether someone can remember basic probability rules and carry out a linear regression than it is to diagnose whether someone can reason carefully about an amorphous business problem.

Post reply on HN