No. Data Science is still a very immature field. Like DevOps 5ish years ago. Everyone wants it. Few people know what they could actually do with it. Lot's people people hire for it and then let those they hire figure it out.
Ask HN: Am I too late for the “Data Science” wave?
81–90 of 148 posts
Re: Ask HN: Am I too late for the “Data Science” wave?
#82Earlier quoted context omitted.
Your comment will forever be under rated due to how spot on it is. It's a fundamental issue in tech that people don't appreciate. While yes, some problems are universal and general tech can solve them without industry experience, I'm pretty sure most people agree that those problems are either solved or there's an army of devs already on it. These days it's not really enough to just be a dev or data sci. You need to…
The problem is that real world, physical business data is taken from what one can get, not what one would want. And usually "what the business was collecting for (unrelated purpose)." This means it has nearly infinite caveats and assumptions. A specifications doc or readout will never sufficiently express all of these. Especially if humans were involved in the data generated. Consequently, the most useful data produc…
>The problem is that real world, physical business data is taken from what one can get, not what one would want.
Yea, sorry, but part of your job in data sci is to collect the right data. Data doesn't magically exist and we are not stuck with what's out there. A data sci job is to figure this stuff out. Tech has a weird culture of not doing their job. Kind of like the Zip Recruiter ads. "Working as a hiring manager, hiring new people is the worst part of my job." Bitch, that IS your job. If you dont do that, what's the point in keeping you around? Bee keepers collect honey. Yea it's not exactly easy if you're not careful, but they dont bitch about it because they knew what they signed up for.
Data sci/analysis is about collecting and analyzing data, in not straightforward ways. Because if it were easy and didnt require any effort, why are they needed?
Re: Ask HN: Am I too late for the “Data Science” wave?
#83Earlier quoted context omitted.
The problem is that real world, physical business data is taken from what one can get, not what one would want. And usually "what the business was collecting for (unrelated purpose)." This means it has nearly infinite caveats and assumptions. A specifications doc or readout will never sufficiently express all of these. Especially if humans were involved in the data generated. Consequently, the most useful data produc…
>The problem is that real world, physical business data is taken from what one can get, not what one would want. Yea, sorry, but part of your job in data sci is to collect the right data. Data doesn't magically exist and we are not stuck with what's out there. A data sci job is to figure this stuff out. Tech has a weird culture of not doing their job. Kind of like the Zip Recruiter ads. "Working as a hiring manager,…
Have you collected data from and deployed products to a 2000+ store environment?
Re: Ask HN: Am I too late for the “Data Science” wave?
#84Earlier quoted context omitted.
> I found that for embedded C++ to be successful, you need to have some good (excellent sometimes was harmful, but that's a different topic) Why were excellent C++ skills sometimes harmful?
First of all, it leaves the team behind if you use features that are too advanced. Makes collaboration hard if you have wildly diverging skills (in both ways). Also, it did in our case take you away so far from the bare metal, that the code was elegant, but slow. It did not play well with our static allocators and allocated/deallocated way too much, especially temporaries. You might i.e. check out talks like [0]. [0]…
I agree. Another thing to consider is portability. You might have to port your software to obscure platforms with bad C++ compilers where compiler bugs are not uncommon at all, so it's better to stay clear of more advanced features. Also, you might have a legacy system which does not have modern C++ compilers, so you might have to restrict yourself to C++03. Another thing is code size, so templates should be used judiciously.
Re: Ask HN: Am I too late for the “Data Science” wave?
#85I work as a data scientist and have some perspective on this. There's no boat to miss, you'll probably be fine. Just keep a couple of things in mind - The fundamental skills that you need are mathematics and software engineering. Depending on your background it might take years of additional studying. - There is a big oversupply of people for the junior-mid level data science jobs. There are more people who want to g…
This reply should be made into billboard and placed in front of every "Data science bootcamp" out there.
Re: Ask HN: Am I too late for the “Data Science” wave?
#86I work as a data scientist and have some perspective on this. There's no boat to miss, you'll probably be fine. Just keep a couple of things in mind - The fundamental skills that you need are mathematics and software engineering. Depending on your background it might take years of additional studying. - There is a big oversupply of people for the junior-mid level data science jobs. There are more people who want to g…
The data science professionals my company has had on staff have been a joke. It’s similar to the wave of inexperienced developers hired in the late 1990s; employers didn’t have the skills to prove that the contractors didn’t know what they were doing, so a ton of money was poured in with big dreams. Part of that is that they don’t understand what we do and weren’t trained. They other part is that the business was jus…
> practical knowledge of data science
This is key. What do companies actually want when they hire a data scientist? Actionable products that make their business better.
What does it take to produce actionable products? A data strategy (collection, ingress, normalize, enrich, store, expose), a compute provisioning strategy, data engineering (pull from source system(s), land in target stores in a reliable, available, automated manner), data science, and data application (reporting, integration with target systems, app development).
Which of those components do they typically have? A data scientist. Because they just hired one.
Career-wise, the more unifaceted your skillset is, the more you're limited to employers that already have all the other pieces in place. Which effectively limits you to very large enterprise (~T100).
Start to be able to fill some of the other roles yourself, and you can compete for and succeed in smaller and more interesting opportunities.
Re: Ask HN: Am I too late for the “Data Science” wave?
#87I work as a data scientist and have some perspective on this. There's no boat to miss, you'll probably be fine. Just keep a couple of things in mind - The fundamental skills that you need are mathematics and software engineering. Depending on your background it might take years of additional studying. - There is a big oversupply of people for the junior-mid level data science jobs. There are more people who want to g…
> There is a big oversupply of people for the junior-mid level data science jobs. Honest question is there any field where that isn’t true right now? Seems like there are no junior jobs in any sector.
one factor here is that 'senior' doesn't really mean anything anymore other than 3+ years of experience.
Re: Ask HN: Am I too late for the “Data Science” wave?
#88Earlier quoted context omitted.
The problem is that real world, physical business data is taken from what one can get, not what one would want. And usually "what the business was collecting for (unrelated purpose)." This means it has nearly infinite caveats and assumptions. A specifications doc or readout will never sufficiently express all of these. Especially if humans were involved in the data generated. Consequently, the most useful data produc…
>The problem is that real world, physical business data is taken from what one can get, not what one would want. Yea, sorry, but part of your job in data sci is to collect the right data. Data doesn't magically exist and we are not stuck with what's out there. A data sci job is to figure this stuff out. Tech has a weird culture of not doing their job. Kind of like the Zip Recruiter ads. "Working as a hiring manager,…
You're both right. Sometimes we have to work with the data we have. Other times we have to create or buy the data we need.
Some companies aren't experienced at building infra to collect data and don't know how to do it. Or their environment is too complex or expensive to sample data from. The data scientist's job in such cases is to do their best with what exists, show success and make a business case for investing resources into data collection infrastructure.
In other cases, when the required sensors don't exist and the information is critical to decision making, you can either buy the data or work with with an engineering group or external vendor to integrate and build out the sensors needed. Need foot traffic data? You can buy from a data marketplace like https://datarade.ai, where there exist various vendors (like SafeGraph -- which was recently used in a COVID19 study published in Nature) aggregating foot traffic data from cell phones. There are datasets that can be used as inferential proxies (so called "alternative data") for the actual data one needs.
Need to collect in-store data? I was at the NRF conference (the world's largest retail tech conference) in NYC back in January and there were a boatload of vendors hawking different types of retail analytics sensors.
In certain small scale operations, you can even engage field operations and get the in-store retail staff to help collect data and upload manually. (you'll need a good relationship with the field supervisor of course)
Sometimes the data does exist but is inaccessible, say in the ERP or in some proprietary format -- then you have negotiate with certain business groups or with OEM vendors in order to get the data out.
It all boils down to whether the data has value that exceeds (by a margin) the cost of collecting them. If the answer is yes, there's often a way to do it (albeit sometimes imperfectly).
Is it part of the data scientist's job description to create or participate in creating data collection infrastructure? I guess this depends on the company but for many companies the answer is yes.
Re: Ask HN: Am I too late for the “Data Science” wave?
#89Yes, you are too late. But if you retrain and become a data critic, you are definitely ahead of the curve ;) The robot armies of AI experts and big data scientists are doing incredible harm to society. There is a real task for you there.
Hijacking the top comment because it's plain wrong. 1/ No there's no "incredible harm" to the society, not more than any technological revolution in the past and it's bringing more good than bad, like any technological revolution. 2/ You can perfectly do a data scientist formation (or preferably a ML formation), there are thousands of free courses on the net but if you get into a reknowned formation it's better, and…
1/ No there's no "incredible harm" to the society, not more than any technological revolution in the past and it's bringing more good than bad, like any technological revolution.
Incorrect at best, apathetic and myopic at worst:
https://www.technologyreview.com/2019/01/21/137783/algorithm...
https://www.propublica.org/article/machine-bias-risk-assessm...
Re: Ask HN: Am I too late for the “Data Science” wave?
#90Earlier quoted context omitted.
Hijacking the top comment because it's plain wrong. 1/ No there's no "incredible harm" to the society, not more than any technological revolution in the past and it's bringing more good than bad, like any technological revolution. 2/ You can perfectly do a data scientist formation (or preferably a ML formation), there are thousands of free courses on the net but if you get into a reknowned formation it's better, and…
1/ No there's no "incredible harm" to the society, not more than any technological revolution in the past and it's bringing more good than bad, like any technological revolution. Incorrect at best, apathetic and myopic at worst: https://www.technologyreview.com/2019/01/21/137783/algorithm... https://www.propublica.org/article/machine-bias-risk-assessm...
You think AIs are worst and more biased than some typical america judges ? Ill give you a big fat LOL.
Any atrocity and injustice has been done at all scales by humankind. Ai will not amplify that and actually i think that well used, it could help fight them
You are just choosing to see the wrong side of it because you think it makes you look woke.