Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Beware the data science pin factory: The power of the data science generalist

multithreaded.stitchfix.com

Post reply on HN
Print view View original

41–50 of 79 posts

Re: Beware the data science pin factory: The power of the data science generalist

#41

Post by [deleted] »

mlthoughts2018 wrote: post #36
gk1 wrote: post #33

Earlier quoted context omitted.

You'll have to do better than an ad hominem + "the opposite is true." Author is Chief Algorithms Officer at Stitchfix and former VP Data Science & Engineering at Netflix.

No, sorry. Argument from authority doesn’t mean the original article has a cogent point. There’s no burden on anyone to refute anything from this piece, as the piece itself has not met any basic requirement of presenting facts or evidence in the first place. It’s merely a matter of fact to point out this deficiency of the article. The premises of the article could still be accurate (though I think that is fleetingly…

[deleted]

Re: Beware the data science pin factory: The power of the data science generalist

#42

Post by apohn »

rchaud wrote: post #27
apohn wrote: post #19

Earlier quoted context omitted.

Have you tried paid services/consulting arms of software or cloud companies? Teams that bill customers at an hourly rate? They generally look for generalists who can help customers tackle problems at different levels of the stack. They aren't looking for PhDs in statistics. When I interview people who have your type of background, I tend to get confused by what exactly it is the person wants to do (Analyze Data? Buil…

> I tend to get confused by what exactly it is the person wants to do (Analyze Data? Build an Analytics Pipeline/Architecture? Write Software/Services? But this is the crux of the job-seeker's dilemma. If he/she is specific about their interests when speaking to an interviewer, they might get a response like "well, we're really looking for someone whose operational focus is [something else]". And if they're not super…

I don't think a person has to be super specific or say "I am interested in X and Y." What they do need is consistency in a resume so the reviewer/interviewer can evaluate what their primary and secondary focus areas are. Sometimes you have to leave stuff out. For example, Data Scientists typically don't deploy and maintain a Data Science stack unless it's a really small company. And a Data Science Infrastructure person at a bigger company probably isn't analyzing data unless they are just playing around to validate their stack.

But if you have a resume (or say this during an interview) that gives equal weight to the data analysis and the stack deployment, it's just confusing to the person reading it. Especially in Data Science, which already confusing from a skillset perspective. Lots of resumes look like the applicants just thought 10 things with minimal overlap were cool and decided to put them on their resume.

Even if you did work at a 5 person startup and had the unoffical title of "Data Scientist, Data Engineer, Data DevOps, DB Admin, and Chief Data Officer" I'd recommend you downplay some of those based on the jobs you are applying for. Figure out what is essential and what is +1

Re: Beware the data science pin factory: The power of the data science generalist

#43

Post by dchichkov »

mlthoughts2018 wrote: post #20

This article is terrible. You can’t make a case by putting a bunch of unsupported assertions into section-heading fonts and then just filling in paragraphs. This reads like a desperate business person wrote it, who wishes that one full-stack set of drives made sense and coexisted in a single person to make that labor cheaper and more commidity, despite the reality that it’s simply not true. The person who spent the t…

I somewhat agree with you. Someone who is spending time now studying jQuery and becoming proficient at developing web services would nessesarily not be able to keep up with the pace of deep learning. On the other hand, there are people that had managed to become relatively proficient at developing software a decade ago. And spend last decade at becoming proficient at deep learning.

Re: Beware the data science pin factory: The power of the data science generalist

#44

Post by mobjack »

mlthoughts2018 wrote: post #20

This article is terrible. You can’t make a case by putting a bunch of unsupported assertions into section-heading fonts and then just filling in paragraphs. This reads like a desperate business person wrote it, who wishes that one full-stack set of drives made sense and coexisted in a single person to make that labor cheaper and more commidity, despite the reality that it’s simply not true. The person who spent the t…

It is rare that deep learning performs better than simple analysis and statistics.

It is a lot more common that a data anomaly is caused by a bug in implementing a web framework.

Re: Beware the data science pin factory: The power of the data science generalist

#45

Post by mlthoughts2018 »

dchichkov wrote: post #43
mlthoughts2018 wrote: post #20

This article is terrible. You can’t make a case by putting a bunch of unsupported assertions into section-heading fonts and then just filling in paragraphs. This reads like a desperate business person wrote it, who wishes that one full-stack set of drives made sense and coexisted in a single person to make that labor cheaper and more commidity, despite the reality that it’s simply not true. The person who spent the t…

I somewhat agree with you. Someone who is spending time now studying jQuery and becoming proficient at developing web services would nessesarily not be able to keep up with the pace of deep learning. On the other hand, there are people that had managed to become relatively proficient at developing software a decade ago. And spend last decade at becoming proficient at deep learning.

You need more than a decade to become proficient with deep learning at the level of researchers solving novel business problems.

It takes at least a decade just to study the prerequisite materials in vector calculus, linear algebra, advanced statistics, classifier algorithms, convex and gradient-based optimization, matrix computations and numerical methods, and associated software engineering skills. That’s all just to get to “base camp” of deep learning.

On the flip side, it’s pretty low effort to just use plug-n-play network components from popular libraries and follow a few tutorials or open source projects.

That’s why there’s effectively zero employment demand for the skill of naive keras or pytorch lego building. It’s as easy as it is meaningless.

Given that you’d already have been spending a decade+ of your life on advanced math if you planned to work on deep learning to solve real problems, there’s a huge impedance mismatch with this idea that you’d somehow also magically just be happy ignoring that specialized skill and the time investment sunk into it to then instead be happy writing throw-away little Flask apps or optimizing routine ETL queries.

Re: Beware the data science pin factory: The power of the data science generalist

#46

Post by mlthoughts2018 »

mobjack wrote: post #44
mlthoughts2018 wrote: post #20

This article is terrible. You can’t make a case by putting a bunch of unsupported assertions into section-heading fonts and then just filling in paragraphs. This reads like a desperate business person wrote it, who wishes that one full-stack set of drives made sense and coexisted in a single person to make that labor cheaper and more commidity, despite the reality that it’s simply not true. The person who spent the t…

It is rare that deep learning performs better than simple analysis and statistics. It is a lot more common that a data anomaly is caused by a bug in implementing a web framework.

It depends on what you’re working on. For generic descriptive statistics, then I agree, and also that has nothing whatsoever to do with data science.

If you’re trying to do reverse image search or machine translation or creating custom embeddings unique to your business problem at hand, then deep learning is hands down better.

This bolsters my point as well. If you only hired “full stack” data scientists and you’re trusting them to correctly tell you if / how deep learning is applicable to a new problem, instead of hiring specialists who actually know how to systematically diagnose that situation, you’re setting yourself up to fail. You may already be too biased towards believing simpler things “should” do better, and you’ll take the full stack person’s inability to outperform with deep learning as if it is confirmatory evidence, when really all it is telling you is that you need a specialist.

Re: Beware the data science pin factory: The power of the data science generalist

#47

Post by lincpa »

I'm Financial Analyst, CPA, CIA, CTA, Statistician, Expert System Developer.

I independently developed a financial analysis expert system, with a strong ability to innovate and execute.

All my expertise is entirely self-taught.

My Project: https://github.com/linpengcheng/fa

My technology Blog: https://github.com/linpengcheng/PurefunctionPipelineDataflow

Re: Beware the data science pin factory: The power of the data science generalist

#48

Post by ummonk »

reilly3000 wrote: post #7

I really wish hiring managers read this. I am a data generalist, and have had no traction with obtaining even an interview for a data science job. I’ve setup a private JupyterHub where I run python ETL, interactive models, and dashboards. I deployed Metabase several times and have written hundreds of SQL queries. I’ve used Tableau with gigantic datasets. I built a front end serverless analytics pipeline from scratch…

Have you tried data engineering jobs rather than data scientist?

Re: Beware the data science pin factory: The power of the data science generalist

#49

Post by dchichkov »

mlthoughts2018 wrote: post #45
dchichkov wrote: post #43

Earlier quoted context omitted.

I somewhat agree with you. Someone who is spending time now studying jQuery and becoming proficient at developing web services would nessesarily not be able to keep up with the pace of deep learning. On the other hand, there are people that had managed to become relatively proficient at developing software a decade ago. And spend last decade at becoming proficient at deep learning.

You need more than a decade to become proficient with deep learning at the level of researchers solving novel business problems. It takes at least a decade just to study the prerequisite materials in vector calculus, linear algebra, advanced statistics, classifier algorithms, convex and gradient-based optimization, matrix computations and numerical methods, and associated software engineering skills. That’s all just…

My assumption was "starting from a post-graduate level in computer science, natural sciences or equivalent". By the way, I don't see how anyone could have more than a decade specifically in deep learning, considering that the field had started at around that time.

On a flip side, TensorFlow 2.0 and AutoML are coming ;). And generic RL agents that do not require reward hacking are also on the horizon. Who cares, if a researcher spend 10000 hours reading articles AND 10000 hours building products, if a more general algorithm obsoletes it all ;)

Re: Beware the data science pin factory: The power of the data science generalist

#50

Post by ebullientocelot »

reilly3000 wrote: post #7

I really wish hiring managers read this. I am a data generalist, and have had no traction with obtaining even an interview for a data science job. I’ve setup a private JupyterHub where I run python ETL, interactive models, and dashboards. I deployed Metabase several times and have written hundreds of SQL queries. I’ve used Tableau with gigantic datasets. I built a front end serverless analytics pipeline from scratch…

Please post some contact information. No idea what your geography is but I am very interested in chatting with you about some roles that I have that may be very well aligned to the skills and interests you are describing.
Post reply on HN