Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Am I too late for the “Data Science” wave?

news.ycombinator.com

Post reply on HN
Print view View original

41–50 of 148 posts

Re: Ask HN: Am I too late for the “Data Science” wave?

#41

Post by te_chris »

vector_spaces wrote: post #23
st1x7 wrote: post #20

Earlier quoted context omitted.

I work as a data scientist and every statement in your post sounds incorrect to me. SQL is useful but it's far from the most important tool. It's certainly unlikely to land you a job. It definitely won't solve 97% of problems. You either have a very skewed perspective of what data science is or you spend a lot of time on linkedin/medium where bad advice like this is parroted a lot. Think about this - there are a bunc…

A surprisingly vast contingent of software developers do not know SQL, let alone know it very well. And as others have mentioned in this thread, data science looks differently at different companies. For many, the math and "ML-specific stuff" ends up being a very small part of the process. For them, data quality and data cleaning take up the overwhelming majority of hours in a given project, and SQL chops will take y…

I don’t think it’s that surprising. Most web dev is just doing mundane CRUD, and mostly through ORMs or other db abstractions. If you don’t practice something how can you be expected to be good at it?

Re: Ask HN: Am I too late for the “Data Science” wave?

#42

Post by cambalache »

1970- Hey Guys I want to start programming with the PDP-10.Did I miss the boat?

1980- Guys I want to learn about micro-controllers, is it too late?

1990- GUI programming

2000- Linux, Internet, you name it

2010 -. Javascript

In 30 years time (at the very least) there will be still Data Science. So if you are really up to it, id does not matter if you should have started 5 years ago or now. If you suck at it or really dont like it, it would make no difference either.

Re: Ask HN: Am I too late for the “Data Science” wave?

#43

Post by cambalache »

iamgopal wrote: post #37
enriquto wrote: post #35

I do not understand the concept of "being too late". If you are too late, what about the future generations? You can only be too late if the subject is already dead and not used anymore.

Being too late means, like in the field of agriculture, all fruits of innovation are already taken, rest is just hard mundane work.

An agriculture engineer, a food scientist and a botanist from 100 years in the future have just read this and they are still laughing.

Re: Ask HN: Am I too late for the “Data Science” wave?

#44

Post by imtringued »

hermitcrab wrote: post #21
tomhoward wrote: post #17

Have you explored opportunities to apply data science to agriculture? I'm interested in this space; I do some work with agricultural data acquisition hardware and software (e.g., soil moisture, environmental conditions, sap flow, plant/fruit growth monitoring), irrigation, fertiliser application) and I'm interested in ways this data could be used in predictive models, but I'm not at the stage of being able to focus o…

>apply data science in new fields I see what you did there. But seriously, if you have a background in agriculture (and don't hate it) and want to get into data science, aim for the intersection of the venn diagram between agriculture and data science. I understand agriculture is getting quite technical and data driven these days, and that can surely only become more the case in the future. Especially if vertical far…

The skill sets most needed for vertical farms probably have nothing to do with agriculture. If that is your goal then you should research or design fusion reactors instead.

Re: Ask HN: Am I too late for the “Data Science” wave?

#45

Post by avlewis »

tomhoward wrote: post #17

Have you explored opportunities to apply data science to agriculture? I'm interested in this space; I do some work with agricultural data acquisition hardware and software (e.g., soil moisture, environmental conditions, sap flow, plant/fruit growth monitoring), irrigation, fertiliser application) and I'm interested in ways this data could be used in predictive models, but I'm not at the stage of being able to focus o…

> Have you explored opportunities to apply data science to agriculture?

This is the most straightforward route imo. Just pick up some of the skills required to get going, which it sounds like they are based on the post, and just start tweaking with stuff in your field. They already have a great advantage of specialized knowledge about the subject they would be applying it to and Ag, from someone who grew up and worked on a row crop farm, seems very ripe for exploration through data science.

Re: Ask HN: Am I too late for the “Data Science” wave?

#46

Post by yilugurlu »

st1x7 wrote: post #24

I work as a data scientist and have some perspective on this. There's no boat to miss, you'll probably be fine. Just keep a couple of things in mind - The fundamental skills that you need are mathematics and software engineering. Depending on your background it might take years of additional studying. - There is a big oversupply of people for the junior-mid level data science jobs. There are more people who want to g…

> - If you're already employed with your agriculture PhD, there must be a number of opportunities for you apply the techniques that you're currently learning wihout leaving the industry. That's probably the path that I would suggest - it would allow you to expand your skillset without taking big risks and you'll have more options in the future. Use the career capital that you already have and explore your options instead of making a sharp turn in your career direction that might leave you disappointed.

This is real gold. If you have existing knowledge about some area than you can learn and apply those things, thus you'll get the traction that you want earlier, and it would be more rewarding in the end.

If there is a chance, I won't miss that opportunity. Good luck.

Re: Ask HN: Am I too late for the “Data Science” wave?

#47

Post by YeGoblynQueenne »

It's not too late, no, but can I ask- why are you intersted in a data scientist role? Is that the only, or best way to "get out of the lab" given your background?

Because I must confess that I don't see the immediate connection between an MSc and PhD in agriculture and a (assumingly generic) data scientist job. You should be perfectly capable of performing data science tasks in an agriculture context, but it seems to me you are asking for something different, a "pivot".

At the risk of sounding rude (for which I apologise in advance) are you asking simply whether there's still space on the current bandwagon? If so, I must advise you against it, because employment bandwagons are awful things to get on. Crodwed, badly paid, poorly understood, not that useful, scarcely productive- in short, short-term and not very fullfilling.

Is it just a matter of making lots and lots of money with the skills you clearly have and that you must have worked hard to acquire? Well then, there should be much, much better placements for you, outside the lab, in the sector you studied about.

My concern is that seeking to jump on the data science bandwagon right now will only flood industry with more and more semi-skilled, half-baked professionals, who don't really understand and don't really care to understand their subject matter, similar to what has happened with software development. The world is full of bad devs who are "passionate about javascript" or something like that and who are struggling to promote their personal brand because they have no other skills than the promotion of their personal brand. Don't allow yourself to fall that low.

Edit: in the interest of full discolosure, I'm a CS graduate with an MSc in data science and studying for a PhD in AI, but I'm not looking for data scientist jobs and am not interested in them, because I find them boring, unproductive and unfullfilling. I have actually worked as a (freelance) data scientist for a while.

Re: Ask HN: Am I too late for the “Data Science” wave?

#48

Post by qxmat »

I've worked at productionising data science models for the past 4 years. I'm currently responsible for delivering technology platforms to ~180 data scientists.

I find the data scientist label misleading.

Roughly 70% of of the data scientists I've encountered are actually Excel analysts with little experience outside of a Windows desktop bar Facebook on a Mac. They're unable to use basic software engineering tools such as git, vscode and python. Excel users and their managers are hostile to solutions that aren't excel-like. They will fist-fight you if you restrict them from downloading and exploring data on their computer. Few understand their compliance/legal obligations.

Another 20% are familiar with a wide range of tools - such as Matlab, R, Jupyter notebooks and various ML/AI toolsets. As developers they're unaware of the tech stack, short of "I installed ananconda and it doesn't work" but are happy to work in the cloud and learn new tech. They understand PII requirements and memory/cpu limits but don't always demonstrate the latter in practice. Nonetheless they produce the bulk of your analysis, having studied classification, and reasonably cost efficient if you pair with a SWE.

The final 10% have mastered containers, venvs, wheels, cloud sdks and how to configure their software in environment independent way. They require help to achieve production quality but are great self-starters. Given enough time and support they're able to quickly replicate this effort and teach others. As relative superstars they're in high demand which makes capacity planning difficult. This pushes up their premium.

IMO the best data scientists are 1 in 10. Because we're desperate for quality almost anyone can assume the title meaning the market open to new comers - you just need to be skilled in Excel (harder than it seems - most developers can learn a lot observing an analyst/consultant use Excel).

To answer your question: No - you're not too late. Just by posting here I expect you'll be in the top 30% - an asset in demand.

Re: Ask HN: Am I too late for the “Data Science” wave?

#50

Post by disgruntledphd2 »

qxmat wrote: post #48

I've worked at productionising data science models for the past 4 years. I'm currently responsible for delivering technology platforms to ~180 data scientists. I find the data scientist label misleading. Roughly 70% of of the data scientists I've encountered are actually Excel analysts with little experience outside of a Windows desktop bar Facebook on a Mac. They're unable to use basic software engineering tools suc…

This is a deeply misleading (though somewhat accurate) comment.

The reason it's misleading is because the 70% above (who may be called data scientists) are not actually data scientists, at best they are data analysts.

In general, the core difference between data scientists and data analysts is that the former can code in at least one language (SQL doesn't count, unfortunately).

However, because the term data science became so popular, everyone re-branded their analyst roles as data scientists leading to this concern.

Additionally, the post I'm replying to is pretty biased, as the OP talks about productionising models. While this is a major facet of DS work, it's not the whole thing. TBH, I can find people to productionise models a lot quicker than I can find people who can figure out what to model, and how to measure it.

Some of those people are most comfortable with Excel, and while I'd prefer they used a different tool, I can't argue with their output.

Also, the OP here is focused on deployment of Python ML models, which again is a subset of a very, very broad field.

That being said, i agree with most of the categorisations, except that the two critical attributes of good data scientists are a strong background in statistics and data common sense.

Data common sense is a weird attribute where when you look at the numbers and see if they are reasonable. For example, if you are running a mobile gaming company and see an ARPU of $5, something has either gone horribly wrong, or you're going to be a billionaire (assuming you have equity).

This attribute is actually not that common amongst DS people, so it tends to be the limiting factor, rather than ability with containers and deployment (which I do agree is very important).

Post reply on HN