Live data from Hacker News

Ask HN: As a data scientist, what should be in my toolkit in 2018?

news.ycombinator.com

81–90 of 177 posts

Re: Ask HN: As a data scientist, what should be in my toolkit in 2018?

#81
post #57

Earlier quoted context omitted.

good list. I would add tidyverse in R ecosystem to it

I would go as far as saying the tidyverse is an essential piece of working with R. Base R sans tidyverse is not a pleasant experience.

It's not that bad. It's inconsistent and clunky, but all of the tools are there (and tend to be faster than the tidyverse versions). Don't get me wrong, I love the tidyverse but R is very, very usable without it.

Re: Ask HN: As a data scientist, what should be in my toolkit in 2018?

#82

Earlier quoted context omitted.

I think data scientist, much like software engineer, is something you can call yourself without having any credentials whatsoever. It’s why technical interviews can be so brutal, unfortunately. There are a lot of frauds out there. Money attracts frauds. What’s the fizzbuzz test for data scientists anyway?

I'm a data engineer for a startup that's trying to hire its first data scientist. The range of candidates that apply with this title is massive. Defining our expectations has been challenging. My phone screen "fizzbuzz" is having them calculate a standard deviation from an array of data w/out with only basic operators (no numpy.std). Then explain why they choose population/sample and explain the difference. I studied…

I have to admit this scares me just a little bit. I'm a senior sysadmin who is trying to lateral transition into data science, but I'm no math whiz, I'm just good at pragmatic use of tech stacks and have a generally analytical mind. If you are a math undergrad how could I ever expect to know more math than you? Of course a standard deviation should be easy, but your comment on math just stuck out to me.

Re: Ask HN: As a data scientist, what should be in my toolkit in 2018?

#83
post #22

Mathematics. Which branch of math is domain dependent. Stats come up everywhere. Graphs do too. In addition to baseline math, you really need to understand the problem domain and goals of the analysis. Languages and libraries are just tools: knowing APIs doesn’t tell you at all how to solve a problem. They just give you things to throw at a problem. You need to know a few tools, but to be honest, they’re easy and you…

R is not present in your list, did you ever try it and what's your opinion about it?

R is good for one-off analysis problems, but is really bad for large distributed production systems. My team moved a large internal analysis application from R to Python because Python works well for both statisticians and software engineers.

Re: Ask HN: As a data scientist, what should be in my toolkit in 2018?

#84
post #62
post #22

Mathematics. Which branch of math is domain dependent. Stats come up everywhere. Graphs do too. In addition to baseline math, you really need to understand the problem domain and goals of the analysis. Languages and libraries are just tools: knowing APIs doesn’t tell you at all how to solve a problem. They just give you things to throw at a problem. You need to know a few tools, but to be honest, they’re easy and you…

Do people get careers as 'data scientists' without masters degrees? I'm heavily biases towards experts... but still would call it fair in the general case to call anyone doing data science without at least a masters degree or the equivalent mental toolkit more like a 'data quack'.

I'd call myself more of a "data plumber", with a double-BS in physics and computer science, but I'm considering returning for a MS so I'm more qualified to do interesting work.

Is there a consensus about what kind of Master's would be most useful data-sciency stuff? Computer science? Stats?

Re: Ask HN: As a data scientist, what should be in my toolkit in 2018?

#85
post #65
post #49

Earlier quoted context omitted.

I really like to get a degree in Mathematics but I simply don't have the time to throw at it (work, children, etc). What do you suggest I should do to have something on my resume? MOOC maybe?

Usually MOOC for resume don't help as everyone does them. The advice that I found useful for resume building is working on projects that you can catalog in a portfolio. With regards to gaining math skills, this upcoming MOOC from Microsoft on EdX looks promising[1]. [1] https://www.edx.org/course/essential-mathematics-for-artific...

So you suggest that I should learn from MOOC then go on and work on some projects so that I can prove I really know it.

Re: Ask HN: As a data scientist, what should be in my toolkit in 2018?

#86

Earlier quoted context omitted.

I think data scientist, much like software engineer, is something you can call yourself without having any credentials whatsoever. It’s why technical interviews can be so brutal, unfortunately. There are a lot of frauds out there. Money attracts frauds. What’s the fizzbuzz test for data scientists anyway?

I'm a data engineer for a startup that's trying to hire its first data scientist. The range of candidates that apply with this title is massive. Defining our expectations has been challenging. My phone screen "fizzbuzz" is having them calculate a standard deviation from an array of data w/out with only basic operators (no numpy.std). Then explain why they choose population/sample and explain the difference. I studied…

At least with fizzbuzz you are working through how to logically solve a problem. This is just regurgitating a formula. I don't see how this is helpful.

Re: Ask HN: As a data scientist, what should be in my toolkit in 2018?

#87

Earlier quoted context omitted.

I'm a data engineer for a startup that's trying to hire its first data scientist. The range of candidates that apply with this title is massive. Defining our expectations has been challenging. My phone screen "fizzbuzz" is having them calculate a standard deviation from an array of data w/out with only basic operators (no numpy.std). Then explain why they choose population/sample and explain the difference. I studied…

I have to admit this scares me just a little bit. I'm a senior sysadmin who is trying to lateral transition into data science, but I'm no math whiz, I'm just good at pragmatic use of tech stacks and have a generally analytical mind. If you are a math undergrad how could I ever expect to know more math than you? Of course a standard deviation should be easy, but your comment on math just stuck out to me.

Maybe consider being a data engineer or a systems engineer? There's a pretty big demand for people that can set up, maintain, and assist the data scientists with the more complex tech stacks out there. In a former job as a systems engineer, I set up Hadoop clusters and helped manage data going into and out of it. And if you do decide to continue learning to become a data scientist, you'll already have a solid footing on the tech they actually use.

Re: Ask HN: As a data scientist, what should be in my toolkit in 2018?

#88
post #80

I'm a scientist (PhD student in microbiolgy) that works with lots of data. My data is on the order of hundreds of gigabytes (genome collections and other sequencing data) or megabytes (flat files). I use the `tidyverse` from R[0] for everything people use `pandas` for. I think the syntax is soooo much more pleasant to use. It's declarative and because of pipes and "quosures" is highly readable. Combined with the powe…

Sometimes I think I'm the only one who isn't really a fan of the tidyverse. I've found it slower, more prone to dependency issues, more prone to silent errors, and less well documented than most R packages (ie most of what you find on CRAN).

You're not the only one. Though I've found there seems to be a bit of a cult surrounding the tidyverse, a mere hint of criticism usually results in outrage and attacking other tools/packages (by users, not the authors).

Re: Ask HN: As a data scientist, what should be in my toolkit in 2018?

#89

Earlier quoted context omitted.

I'm a data engineer for a startup that's trying to hire its first data scientist. The range of candidates that apply with this title is massive. Defining our expectations has been challenging. My phone screen "fizzbuzz" is having them calculate a standard deviation from an array of data w/out with only basic operators (no numpy.std). Then explain why they choose population/sample and explain the difference. I studied…

I have to admit this scares me just a little bit. I'm a senior sysadmin who is trying to lateral transition into data science, but I'm no math whiz, I'm just good at pragmatic use of tech stacks and have a generally analytical mind. If you are a math undergrad how could I ever expect to know more math than you? Of course a standard deviation should be easy, but your comment on math just stuck out to me.

I think it is a reasonable expectation to require a certain baseline of expertise, considering the first thing the poster admitted to was: "I'm a data engineer for a startup that's trying to hire its first data scientist."

Much like how the early hires at Twitter were not deeply experienced in high availability work -- segregating the architecture of a predominantly RoR code base to be resilient at scale, which lead to countless "fail whale" outages, before they eventually landed someone who helped them re-think their architecture to use RoR for what it's good at while introducing the JVM and other languages to handle other aspects of their workload.

Re: Ask HN: As a data scientist, what should be in my toolkit in 2018?

#90
post #62

Earlier quoted context omitted.

Do people get careers as 'data scientists' without masters degrees? I'm heavily biases towards experts... but still would call it fair in the general case to call anyone doing data science without at least a masters degree or the equivalent mental toolkit more like a 'data quack'.

I'd call myself more of a "data plumber", with a double-BS in physics and computer science, but I'm considering returning for a MS so I'm more qualified to do interesting work. Is there a consensus about what kind of Master's would be most useful data-sciency stuff? Computer science? Stats?

Lots of universities have Data Science Master's Programs, which may ultimately fall under the Computer Science, Mathematics or Stats department. So, it depends on the university.
Post reply on HN