Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Data Science Is America’s Hottest Job

bloomberg.com

Post reply on HN
Print view View original

171–180 of 224 posts

Re: Data Science Is America’s Hottest Job

#171

Post by make3 »

PunchTornado wrote: post #52
make3 wrote: post #47

Earlier quoted context omitted.

In real life, problems are hard and require theoretical investigation to do well. That's the actual job! The theoretical investigation. Not the programming. The programming is really easy. which is part of why having 15 years in software doesn't have so much weight.

I would say the non-programming part, the theoretical investigation is really easy. Plus the majority of data science do data cleaning 80% of their time.

If you're a programmer by formation, then indeed you will be given a programming intensive, theoretically non-intensive task (or be found to be qualified for such a job). If you are more theoretically inclined, unsurprisingly, you end up with a more theoretically heavy job.

Re: Data Science Is America’s Hottest Job

#173

Post by snowwrestler »

stcredzero wrote: post #143
Puer wrote: post #134

Earlier quoted context omitted.

I mean, in my graduate ML classes the first "homework" assignment was always just a problem set of probability and distribution questions. They were simple questions, but you'd be surprised how many people dropped the class after that first assignment. There are a lot of people out there who have memorized how to implement k-means and PCA but would absolutely struggle to understand what they're actually doing or inte…

That's why PhDs are favored because with their experience having to write grants to get funding for their research they're exactly the type of people that can take a naive problem, work it to a result, and then sit in front of a board room of non-technical people and explain why their result was worth the money that was given to them. My wife has a PhD in comparative lit, but she now works in banking. Her PhD gave he…

I think training in literary analysis is a secret superpower for life. It gives people tools to make reliable inferences from text about motivations, assumptions, biases, etc., and to spot attempts to use phrasing to hide or obscure things. Great for reading emails, contracts, reports, etc.--and for editing your own writing for clarity (or not, if that's what you're going for...).

Re: Data Science Is America’s Hottest Job

#175

Post by HelloMcFly »

kitanata wrote: post #87
hellofunk wrote: post #83

Earlier quoted context omitted.

Of the skills necessary to succeed in data science, the ability to program is actually the least important. Perhaps you were over-emphasising your software skills at the expense of really demonstrating proper understanding of the science of data analysis. A data science position is not going to require your experience in low-level game programming, for example. In fact, being able to program at all is secondary, sinc…

This right here is a perfect example of why this position is gatekeeped so much. Having a PhD doesn’t automatically mean you think like a scientist of a mathematician, and having a bachelor’s degree and 15 years of experience writing code and then putting in the effort to read Elements of Statistical Learning, compeleting several online courses in machine learning and data science, studying probability, combinatorics…

This comment reads as full of hubris. It is easy to say you can do what someone else does, but the fact is you haven't done it, don't have the evidence to show that you can, and are exasperated that others don't see in you what you see in yourself. Having a PhD in a relevant field is an objective signal - not proof - of capability.

When these jobs are hot and candidates are plentiful, using signals to narrow the field to a group you can more rigorously interview is typically a more effective use of time than buying into everyone's self-belief. Candidly, I find most individuals from a programming background vastly underestimate the skillset required in this space. I know that is not an uncommon perception and you are likely being penalized for it, fairly or unfairly. I'll say this: anyone who refers to data science as "just linear algebra and calculus" would be immediately removed from any candidate pool I was managing.

Others have evidence of capability, you do not. Programming experience is not evidence enough to elevate you above candidates with more reliable and relevant credentials. A shelf full of books is not evidence either. You either need to find a version of this job created by people that don't really know what it is they want (hint: if the Data Science JD says "Excel" that's an indicator, it's not too uncommon) to create a work history, find a way create a portfolio that you can use as evidence (e.g., Kaggle competitions, hobbyist projects with available datasets), or network with others in the industry and academia such that they will vouch for you.

Re: Data Science Is America’s Hottest Job

#176

Post by volker48 »

stcredzero wrote: post #128
resolaibohp wrote: post #102

Earlier quoted context omitted.

When my company posted a data science job we received something like 300 applications on the first day. When looking at them most were people with no experience but a fresh degree or bootcamp grad plus all the other people like you who want to change careers from software with portfolios of data science projects. The field is absolutely saturated with people who want to be a data scientist but have no experience. Thi…

The field is absolutely saturated with people who want to be a data scientist but have no experience. This is where some of that gate keeping comes from. What are the data science "gotchas?" A lot of people can pick up basic programming in a weekend, but they wouldn't necessarily know what they don't know and might well get deeply mired in problems with concurrency or algorithmic complexity. It's such "gotchas" which…

The surface area of "gotchas" in data science is much larger than software engineering because you have all the "gotchas" from programming/software engineering AND all the new "gotchas" from data science.

For data science the big one is over fitting, which everyone talks about, but can happen in really insidious ways in production. You have to be very disciplined and careful with the data to prevent over fitting.

Another big one is productionizing data science, which in my opinion most data scientists don't have a ton of experience with.

The actual training of the models part of data science isn't that hard, its actually making it work with the crappy data that exists in the real world and putting it into production that are the really hard parts.

Re: Data Science Is America’s Hottest Job

#177

Post by volker48 »

resolaibohp wrote: post #102
kitanata wrote: post #41

I recently quit a gig and spent a few months looking for a data science/machine learning gig. I was surprised just how gatekeeped these positions were. Everyone wanted a PhD or a masters degree. I have 15 years of experience in software. I’ve done everything to low level game programming and graphics programming to web development, to AI (but not as a specific position title). The gatekeeping in this field surprised…

When my company posted a data science job we received something like 300 applications on the first day. When looking at them most were people with no experience but a fresh degree or bootcamp grad plus all the other people like you who want to change careers from software with portfolios of data science projects. The field is absolutely saturated with people who want to be a data scientist but have no experience. Thi…

I was recently back filling a data science position on my team and experienced the same thing. I was getting on average 5 new resumes a day and for the most part it was individuals who were currently in a data science MS program somewhere with no experience.

Re: Data Science Is America’s Hottest Job

#178

Post by vowelless »

kitanata wrote: post #41

I recently quit a gig and spent a few months looking for a data science/machine learning gig. I was surprised just how gatekeeped these positions were. Everyone wanted a PhD or a masters degree. I have 15 years of experience in software. I’ve done everything to low level game programming and graphics programming to web development, to AI (but not as a specific position title). The gatekeeping in this field surprised…

This is a bizarre comment. You have 15 years of software development experience and you are surprised this experience doesn't carry over to data science or ML directly?

Why should someone bring you in? Do you have a kaggle profile with your notebooks that people can look at? Have you replicated research papers ? Do you have any independent results that you can share, publish, or talk about?

Re: Data Science Is America’s Hottest Job

#179

Post by rifung »

kitanata wrote: post #87
hellofunk wrote: post #83

Earlier quoted context omitted.

Of the skills necessary to succeed in data science, the ability to program is actually the least important. Perhaps you were over-emphasising your software skills at the expense of really demonstrating proper understanding of the science of data analysis. A data science position is not going to require your experience in low-level game programming, for example. In fact, being able to program at all is secondary, sinc…

This right here is a perfect example of why this position is gatekeeped so much. Having a PhD doesn’t automatically mean you think like a scientist of a mathematician, and having a bachelor’s degree and 15 years of experience writing code and then putting in the effort to read Elements of Statistical Learning, compeleting several online courses in machine learning and data science, studying probability, combinatorics…

> I can do everything you can do.

Sorry but how would you even know?

I understand your frustrations well since I don't have a BS.

Still, as much as you seem to want to talk about how capable you are, you can't seem to understand the perspective of employers. Given what you've said about your history I suspect this is not due to a lack of intelligence but empathy.

Employers have to go through many candidates, each of which has some true capability but of which the employer can only see some signals. Signals have varying degrees of quality, and interviewing candidates costs time and money. That being the case, it is only natural that they try to use the strongest signals they have.

Nobody believes that there are not capable people who do not have an MS or PhD as you seem to be suggesting. The reality is that the proportion of people who have a BS and can do the job is much less than the number with MS or PhDs, and so it's one of the more effective filters they have at their limited disposal.

I'm sure companies are not happy about skipping great candidates like yourself, but they have not figured out a way to do so that is scalable and cost efficient. It's a difficult problem but maybe you can figure it out.

> You are not special because you have a PhD and I don’t. The only difference between you and me is that I had the ability to learn for free what you paid for.

How much do you even know about PhD programs? It seems like not much because PhD candidates, at least in the US, get paid. It's not much but they certainly are not paying for their education.

I apologize for this unsolicited advice but your lack of humility is frankly very off putting. You sound like a very hard working person. PhD programs are extremely difficult to both be admitted into and to finish. I'd expect that you would respect others like yourself who are very hard working.

Re: Data Science Is America’s Hottest Job

#180

Post by brookhaven_dude »

ims wrote: post #25
brookhaven_dude wrote: post #12

We have two closely related groups in our company. Mathematical Optimization and Data Science. I am in the optimization group, but when I query the data scientists, they seem to be doing similar things - simulation, network optimization, resource allocation and planning. I am not sure why management wants to call them data science.

I'd guess that historically the Mathematical Optimization department has comprised "legacy" operations research people (spreadsheet modeling and decision analysis, linear and nonlinear modeling with AMPL-type tools, SAS statistics, etc) and the Data Science group is newer and more software-ish. Am I right?

I guess so. They work with SAS and python, while we work with C++ and CPLEX on vi editor in a linux terminal.
Post reply on HN