Live data from Hacker News

Ask HN: Am I too late for the “Data Science” wave?

news.ycombinator.com

61–70 of 148 posts

Re: Ask HN: Am I too late for the “Data Science” wave?

#61
post #24

I work as a data scientist and have some perspective on this. There's no boat to miss, you'll probably be fine. Just keep a couple of things in mind - The fundamental skills that you need are mathematics and software engineering. Depending on your background it might take years of additional studying. - There is a big oversupply of people for the junior-mid level data science jobs. There are more people who want to g…

As a software engineer who has worked intimately with data scientists in Big Tech companies, this comment sums it up exactly.

One minor thing to add is that not only is the data science field overflowing with junior/mid candidates, but the entire computer science field as a whole.

Re: Ask HN: Am I too late for the “Data Science” wave?

#62
post #24

I work as a data scientist and have some perspective on this. There's no boat to miss, you'll probably be fine. Just keep a couple of things in mind - The fundamental skills that you need are mathematics and software engineering. Depending on your background it might take years of additional studying. - There is a big oversupply of people for the junior-mid level data science jobs. There are more people who want to g…

I don't exactly agree on why employers prefer PhDs even if not exactly in the field.

Having PhD shows that you have committed to one field hard enough and long enough to be granted the title even if it is PhD in philosophy.

Also, if you switch from another career try to figure out how you can use your past experience to differentiate yourself from other people.

For example, I have studied theoretical math and I have spent 20 years as software developer working on varied projects from embedded to algorithmic trading to backend to frontend applications.

I am currently switching to electronics and will use my knowledge to work on projects that require high level of both math and software engineering skills. Think in terms of realtime control systems, building reliable and performant stuff (reliability to differentiate from competition), etc.

Never think of moving as rebooting your career. Always start with mindset of building on top of what you know -- always move forward, not backward.

A little change in mindset goes a long way.

Re: Ask HN: Am I too late for the “Data Science” wave?

#63

Earlier quoted context omitted.

>apply data science in new fields I see what you did there. But seriously, if you have a background in agriculture (and don't hate it) and want to get into data science, aim for the intersection of the venn diagram between agriculture and data science. I understand agriculture is getting quite technical and data driven these days, and that can surely only become more the case in the future. Especially if vertical far…

The skill sets most needed for vertical farms probably have nothing to do with agriculture. If that is your goal then you should research or design fusion reactors instead.

It is hard imagine that all those millenia of knowledge about growing crops is going to be irrelevant, even if you manage to take soil, seasons and pests out of the equation.

Re: Ask HN: Am I too late for the “Data Science” wave?

#64
No, I don't think you are too late. The predictions that data science will be fully automated in the near future are, in my professional opinion, unrealistic.

I see the future of data science benefiting from better software design, e.g. transformative frameworks like pytorch and sklearn, which are powerful tools but hardly fully automated. We'll continue to need skilled workers who are current in the latest software stacks.

It also benefits from what I'll call the "lotto effect", where data scientists will occasionally multiply the bottom line by 10x or more. This is of course rare, but companies will continue to chase that fantasy and hire data scientists because it's too tempting to ignore.

My only advice would be lean heavily into the software side of things. There's too many data scientists who are novice programmers.

Re: Ask HN: Am I too late for the “Data Science” wave?

#65

1970- Hey Guys I want to start programming with the PDP-10.Did I miss the boat? 1980- Guys I want to learn about micro-controllers, is it too late? 1990- GUI programming 2000- Linux, Internet, you name it 2010 -. Javascript In 30 years time (at the very least) there will be still Data Science. So if you are really up to it, id does not matter if you should have started 5 years ago or now. If you suck at it or really…

But there's been Data Science jobs since at least the 50's. 50-60's: Operations Research 70's: Statistics 80's: KDD (knowledge discovery in databases) 90's: analytics (statistics again) 00's: Data science 10's: ML/AI 20's: ??? The tools and problems may have changed, but the core skills (statistics, some coding and data awareness) are identical.

This is accurate. The difference this time are the absurd levels of hype we have to deal with.

Re: Ask HN: Am I too late for the “Data Science” wave?

#66
My background: I am a tech architect/manager on IoT projects who designs and helps develop analytics systems. I do some basic data modeling in my role, but defer to experts to build a better model once we find some area of interest.

The actual skills of light programming (R, Python), data literacy/manipulation and some basic modeling (statistical/machine learning, Bayesian methods, time series, etc) are useful for many job roles, and I think will be considered basic skills for college graduates in the near future. This isn't new - operations research, particularly Six Sigma and quality control, have used statistics and some light programming to solve business problems for decades.

By itself, I don't see Data Science evolving the way promised by schools and boot camps. Most of the positions named "Data Scientist" (at non-tech companies) are really just senior business analysts; I work with a group of them at my company and 90% of their day to day is just extracting and analyzing various reports for other managers and directors. When an interesting (and potentially lucrative) business problem does come along, they usually outsource to a specialized analytics firm and the data scientist helps coordinate that project. (If you have a good data dictionary, a clear outcome in mind, and some basic knowledge of the field it's relatively easy to outsource the advanced work.)

st1x7 had the best advice below -- learn the basic methods and then apply them to your field. If you google "agriculture iot research papers" you'll find tons of examples of people using sensors for data collection and then analyzing the data to improve some process.

TL;DR I see Data Science melting into other roles, but the basic skills/data literacy are useful for almost everyone.

Re: Ask HN: Am I too late for the “Data Science” wave?

#67
post #24

I work as a data scientist and have some perspective on this. There's no boat to miss, you'll probably be fine. Just keep a couple of things in mind - The fundamental skills that you need are mathematics and software engineering. Depending on your background it might take years of additional studying. - There is a big oversupply of people for the junior-mid level data science jobs. There are more people who want to g…

The data science professionals my company has had on staff have been a joke. It’s similar to the wave of inexperienced developers hired in the late 1990s; employers didn’t have the skills to prove that the contractors didn’t know what they were doing, so a ton of money was poured in with big dreams.

Part of that is that they don’t understand what we do and weren’t trained. They other part is that the business was just told they needed data people.

What businesses need today are to understand what they’re interested in first, and then hire people with proven experience and knowledge to accomplish those goals.

Having a PhD in any applicable subject in addition to sufficient practical knowledge of data science, machine learning, and AI would be a substantial plus for a niche position in the industry, a non-profit, or even in finance.

Don’t give up.

Re: Ask HN: Am I too late for the “Data Science” wave?

#68
post #24

I work as a data scientist and have some perspective on this. There's no boat to miss, you'll probably be fine. Just keep a couple of things in mind - The fundamental skills that you need are mathematics and software engineering. Depending on your background it might take years of additional studying. - There is a big oversupply of people for the junior-mid level data science jobs. There are more people who want to g…

> - If you're already employed with your agriculture PhD, there must be a number of opportunities for you apply the techniques that you're currently learning wihout leaving the industry. That's probably the path that I would suggest - it would allow you to expand your skillset without taking big risks and you'll have more options in the future. Use the career capital that you already have and explore your options ins…

In general, domain knowledge is incredibly valuable and will generally trump someone who may have some more technical skills but doesn't know the field. I'd add that a lot of larger companies--heard someone from Shell talking about this a couple months ago--are training up people who have knowledge of the business with "citizen data scientist" skill sets.

Without knowing all the details of your situation, it seems at least a much lower risk path to acquire some data science skills--maybe your company will even pay for it--that you can pair with your existing domain knowledge.

Re: Ask HN: Am I too late for the “Data Science” wave?

#69
post #48

I've worked at productionising data science models for the past 4 years. I'm currently responsible for delivering technology platforms to ~180 data scientists. I find the data scientist label misleading. Roughly 70% of of the data scientists I've encountered are actually Excel analysts with little experience outside of a Windows desktop bar Facebook on a Mac. They're unable to use basic software engineering tools suc…

This is a deeply misleading (though somewhat accurate) comment. The reason it's misleading is because the 70% above (who may be called data scientists) are not actually data scientists, at best they are data analysts. In general, the core difference between data scientists and data analysts is that the former can code in at least one language (SQL doesn't count, unfortunately). However, because the term data science…

> are not actually data scientists, at best they are data analysts.

Unfortunately the phrase's usage has been corrupted by HR departments, and the BA types of job listings now outnumber the "real data scientist" listings.

Re: Ask HN: Am I too late for the “Data Science” wave?

#70
post #62
post #24

I work as a data scientist and have some perspective on this. There's no boat to miss, you'll probably be fine. Just keep a couple of things in mind - The fundamental skills that you need are mathematics and software engineering. Depending on your background it might take years of additional studying. - There is a big oversupply of people for the junior-mid level data science jobs. There are more people who want to g…

I don't exactly agree on why employers prefer PhDs even if not exactly in the field. Having PhD shows that you have committed to one field hard enough and long enough to be granted the title even if it is PhD in philosophy. Also, if you switch from another career try to figure out how you can use your past experience to differentiate yourself from other people. For example, I have studied theoretical math and I have…

I actually think that we agree on the PhD requirement.

> Having PhD shows that you have committed to one field hard enough and long enough to be granted the title even if it is PhD in philosophy.

Yes, exactly. That's what I meant by "strong initial filter".

Post reply on HN