Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Beware the data science pin factory: The power of the data science generalist

multithreaded.stitchfix.com

Post reply on HN
Print view View original

61–70 of 79 posts

Re: Beware the data science pin factory: The power of the data science generalist

#61

Post by lincpa »

lincpa wrote: post #47

I'm Financial Analyst, CPA, CIA, CTA, Statistician, Expert System Developer. I independently developed a financial analysis expert system, with a strong ability to innovate and execute. All my expertise is entirely self-taught. My Project: https://github.com/linpengcheng/fa My technology Blog: https://github.com/linpengcheng/PurefunctionPipelineDataflow

@pirocks, Don't you think this is a case of data science generalist creating good products? What kind of psychology makes you give a downvote? Why delete your comments again!

Re: Beware the data science pin factory: The power of the data science generalist

#62

Post by _fol8 »

reilly3000 wrote: post #7

I really wish hiring managers read this. I am a data generalist, and have had no traction with obtaining even an interview for a data science job. I’ve setup a private JupyterHub where I run python ETL, interactive models, and dashboards. I deployed Metabase several times and have written hundreds of SQL queries. I’ve used Tableau with gigantic datasets. I built a front end serverless analytics pipeline from scratch…

It's not you, it's capitalism. There just aren't enough (well paying, decent) jobs to go around. I'm in the same boat. Welcome aboard matey.

Re: Beware the data science pin factory: The power of the data science generalist

#63

Post by mlthoughts2018 »

jonathankoren wrote: post #57
mlthoughts2018 wrote: post #45

Earlier quoted context omitted.

You need more than a decade to become proficient with deep learning at the level of researchers solving novel business problems. It takes at least a decade just to study the prerequisite materials in vector calculus, linear algebra, advanced statistics, classifier algorithms, convex and gradient-based optimization, matrix computations and numerical methods, and associated software engineering skills. That’s all just…

>You need more than a decade to become proficient with deep learning at the level of researchers solving novel business problems. No. Even these people haven’t been doing it for a decade.

Yes. They were doing 4 years of math intensive undergrad and 5+ years of math intensive post-grad work before there even was an advent of deep learning in 2008-2012.

Re: Beware the data science pin factory: The power of the data science generalist

#64

Post by mlthoughts2018 »

dchichkov wrote: post #49
mlthoughts2018 wrote: post #45

Earlier quoted context omitted.

You need more than a decade to become proficient with deep learning at the level of researchers solving novel business problems. It takes at least a decade just to study the prerequisite materials in vector calculus, linear algebra, advanced statistics, classifier algorithms, convex and gradient-based optimization, matrix computations and numerical methods, and associated software engineering skills. That’s all just…

My assumption was "starting from a post-graduate level in computer science, natural sciences or equivalent". By the way, I don't see how anyone could have more than a decade specifically in deep learning, considering that the field had started at around that time. On a flip side, TensorFlow 2.0 and AutoML are coming ;). And generic RL agents that do not require reward hacking are also on the horizon. Who cares, if a…

> “My assumption was "starting from a post-graduate level in computer science, natural sciences or equivalent".”

Yes, same for me. This builds in nearly a decade of preparatory work into the timeline... so it seems we agree.

> “On a flip side, TensorFlow 2.0 and AutoML are coming ;). And generic RL agents that do not require reward hacking are also on the horizon.”

I work professionally in deep learning for image processing. This quote reads like parody to me. I cannot imagine anyone familiar with the realities of AutoML or deep reinforcement learning talking this way. It’s like an excerpt from the script of Silicon Valley.

Re: Beware the data science pin factory: The power of the data science generalist

#65

Post by PointersAreFun »

reilly3000 wrote: post #7

I really wish hiring managers read this. I am a data generalist, and have had no traction with obtaining even an interview for a data science job. I’ve setup a private JupyterHub where I run python ETL, interactive models, and dashboards. I deployed Metabase several times and have written hundreds of SQL queries. I’ve used Tableau with gigantic datasets. I built a front end serverless analytics pipeline from scratch…

Why would anyone hire you? You want a scientist role without the qualifications. Go get a degree in Computer Science or Mathematics, then try again. It is your fault for majoring in music. Though you might find a Data Musician role, I don’t know?

Re: Beware the data science pin factory: The power of the data science generalist

#66

Post by [deleted] »

opportune wrote: post #23
reilly3000 wrote: post #7

I really wish hiring managers read this. I am a data generalist, and have had no traction with obtaining even an interview for a data science job. I’ve setup a private JupyterHub where I run python ETL, interactive models, and dashboards. I deployed Metabase several times and have written hundreds of SQL queries. I’ve used Tableau with gigantic datasets. I built a front end serverless analytics pipeline from scratch…

The one thing I see kind of missing is a math background or at least a project proving that that is in your skillset (recommendations sounds like it could fit this). There are a lot of people with a similar background to you and normally those are in "business intelligence/analytics" or "data engineering" where they are mostly writing sql queries and interacting with dashboards/OLAP cubes or setting up those dashboar…

[deleted]

Re: Beware the data science pin factory: The power of the data science generalist

#67

Post by School-Cotton »

PointersAreFun wrote: post #65
reilly3000 wrote: post #7

I really wish hiring managers read this. I am a data generalist, and have had no traction with obtaining even an interview for a data science job. I’ve setup a private JupyterHub where I run python ETL, interactive models, and dashboards. I deployed Metabase several times and have written hundreds of SQL queries. I’ve used Tableau with gigantic datasets. I built a front end serverless analytics pipeline from scratch…

Why would anyone hire you? You want a scientist role without the qualifications. Go get a degree in Computer Science or Mathematics, then try again. It is your fault for majoring in music. Though you might find a Data Musician role, I don’t know?

This would be unnecessarily harsh even if it were true.

It’s not, though. I know a data scientist who studied Arabic language. And software engineers who studied music.

Re: Beware the data science pin factory: The power of the data science generalist

#68

Post by navigatesol »

apohn wrote: post #19
reilly3000 wrote: post #7

I really wish hiring managers read this. I am a data generalist, and have had no traction with obtaining even an interview for a data science job. I’ve setup a private JupyterHub where I run python ETL, interactive models, and dashboards. I deployed Metabase several times and have written hundreds of SQL queries. I’ve used Tableau with gigantic datasets. I built a front end serverless analytics pipeline from scratch…

Have you tried paid services/consulting arms of software or cloud companies? Teams that bill customers at an hourly rate? They generally look for generalists who can help customers tackle problems at different levels of the stack. They aren't looking for PhDs in statistics. When I interview people who have your type of background, I tend to get confused by what exactly it is the person wants to do (Analyze Data? Buil…

>what exactly it is the person wants to do (Analyze Data? Build an Analytics Pipeline/Architecture? Write Software/Services? Be an Analytics IT person?).

But you realize that the vast majority of businesses in North America need someone to solve all of those problems. They aren't going to hire and cna't afford an experienced data team of specialists.

This is the point of the article. Getting the 80% is far more valuable than having some PhD optimizing the hell out of features. Silicon Valley tends to overthink things.

Re: Beware the data science pin factory: The power of the data science generalist

#69

Post by PointersAreFun »

School-Cotton wrote: post #67
PointersAreFun wrote: post #65

Earlier quoted context omitted.

Why would anyone hire you? You want a scientist role without the qualifications. Go get a degree in Computer Science or Mathematics, then try again. It is your fault for majoring in music. Though you might find a Data Musician role, I don’t know?

This would be unnecessarily harsh even if it were true. It’s not, though. I know a data scientist who studied Arabic language. And software engineers who studied music.

I feel bad for whoever employed them. People manage to get into jobs they are unqualified for all the time.

Music and Computer Science/Mathematics are two very different qualifications. Why would someone who spent their time in college partying, because that’s what Art/Music majors do, and avoiding STEM like the plague be employed as a Data Scientist or any scientist position? They don’t have the background or diligence required. They obviously went “no, I don’t like math or science, I’ll study music instead”.

I really wish people who went through bootcamps or read a tutorial or ten, would stop feeling qualified to do something they are most certainly not qualified to do.

Re: Beware the data science pin factory: The power of the data science generalist

#70

Post by apohn »

navigatesol wrote: post #68
apohn wrote: post #19

Earlier quoted context omitted.

Have you tried paid services/consulting arms of software or cloud companies? Teams that bill customers at an hourly rate? They generally look for generalists who can help customers tackle problems at different levels of the stack. They aren't looking for PhDs in statistics. When I interview people who have your type of background, I tend to get confused by what exactly it is the person wants to do (Analyze Data? Buil…

> what exactly it is the person wants to do (Analyze Data? Build an Analytics Pipeline/Architecture? Write Software/Services? Be an Analytics IT person?). But you realize that the vast majority of businesses in North America need someone to solve all of those problems. They aren't going to hire and cna't afford an experienced data team of specialists. This is the point of the article. Getting the 80% is far more valu…

>But you realize that the vast majority of businesses in North America need someone to solve all of those problems. They aren't going to hire and cna't afford an experienced data team of specialists.

The vast majority of business don't need Data Scientists. They need a BI person with SQL skills and some of the skills of a Database Admin. What most companies really need is a good set of Dashboards and clean data to feed it. This enables the business people to get the information/visibility they need an make decisions.

Also, most businesses should not be building analytic services and deploying them - they should be paying for a good product with a cloud or easy on-prem install and getting support from the company that sells the product. A few licenses of a good BI product are a lot cheaper than a Data Scientist.

Post reply on HN