Data science interview questions with answers

There are myriads of ways for two things to be unequal. In fact its fair to state that unless you are taking two sets of measurements of some fundamental physical constant or measuring artificially generated data, they are not going to be from the same exact distribution -- you cannot bathe in the same river twice is very true.

The way to address the problem you speak of is to think hard about the kind deviations from equality that would be most damaging to the application (were it to slip in undetected). Once you know what's the most damaging deviation, you can then select a test that is very powerful in detecting that specific type of deviation.

For example, you mention Kolmogorov(Smirnov) test. Those are daft blind at the tails of the distribution. So in case you need to catch deviations at the tail, then you can safely skip KS test, they are no good. On the other hand KS tests are very powerful around the median, if your application requires high resolution there, KS is the test you want.

Data Science jobs cover a wide gamut from theoretical math to applied machine learning to statistics to data engineering to analytics. You can generally tell which one they're aiming for from the job description and requirements. Some are basically looking for a Statistician while other looks for a CS Machine Learning Engineer. If the title is ML Engineer than that generally indicates it's a lot more focused on what you studied than stats.

I wish. I saw it asked by otherwise great teams run by very competent people. It's done in place of a leetcode algo question and those are even more divorced from day to day work. System design questions make more sense but I've noticed that you score better on them if you study the area beforehand but convincingly lie that you barely know it (and are just that clever and fast on your feet). Interviews are a shit show in general.

These are at best machine learning questions, not data science. ML is definitely a sub-category of data science, but if you see a job posting for data science, you're 100% not going to be doing machine learning. That would have been labeled as a machine learning job posting.

> Its very sensitive to fat tails and skew. That's the reason robust testing and estimation is a thing.

Yeah, I shouldn't have said "extremely". Its much more robust than is commonly perceived, but that does not make it "extremely" robust. Thank you for the correction. But please note that I specified equal sample sizes, so the fact that "there are general conditions under which these methods can provide poor control over the probability of a type I error" is not really a refutation—I already implied that the t-test is not generally (= in all cases) robust. I also mentioned non-parametric tests as a potential alternative. I do not wish to imply that the t-test is always the right choice. But I stand by the assertion that approximate knowledge of the distribution, such as obtained by inspecting a histogram, is perfectly adequate to choose a test. The main point remains — list what you know about your data, and pick a test that tells you what you want to know with a tolerable error level for your application.

You can spend an awful lot of time picking the "correct" statistical test (if there is such a thing; tradeoffs exist) with little gain. Worse, making a lot of decisions about what test to use after looking at the data leads to p-hacking, potentially leaving you with more bias than if you naively used a slightly-wrong test from the start.

To elaborate on the t-test discussion:

From your ref [1], "If sampling is from nonnormal distributions that are absolutely identical, so in particular the variances are equal, the probability of a Type I error will not exceed 0.05 by very much, assuming the method is applied with the desired probability of a Type I error set at α = 0.05. These two results have been known for some time and have been verified in various studies conducted in more recent years" (page 79). I think Wilcox makes this statement under the equal sample size caveat, but the text is unclear on this point. This does not guarantee robustness under nonnormality, but it does refute the idea that nonnormality is always fatal to the t-test.

Yes, a t-test can be affected by skew even with equal sample sizes. Whether this is a problem depends on the application. The example in [1] uses a lognormal distribution and ends up with actual α = 0.15 for n = 20, desired α = 0.05. Which isn't great, but is somewhat tolerable. Also, a lognormal distribution is strongly skew right and left-truncated, and this is easily noticeable on a plot and can be transformed to a normal distribution. So this does not refute the idea that a sort-of-normal distribution is fatal to the t-test.

You can look up the potential α level and β level errors from violating the assumptions of a particular test—they're tabulated. E.g., applying the t-test to a Pearson distribution produces a typical α level error of 0.005 at desired α = 0.05 [2]. That's definitely tolerable. This is why I stated that sort-of-normal is good enough for a t-test. It's also fairly straightforward to calculate the expected errors yourself for a particular situation, if reassurance is needed. If your statistics are intended to support a high-stakes decision this may be a good use of time. Working to validate your finding by other means is better, though.

I'm not sure what your ref [0] (the "Robustness of Standard Tests" book chapter) is arguing for or against from the abstract alone. I don't have a copy of that book. The abstract mentions most flavors of maximum likelihood ratio tests, which is too broad a set of topics to discuss effectively. The implication seems to be that null hypothesis statistical tests are bad and something else (Bayesian analysis? visual inspection?) is better, which I don't necessarily disagree with. If you could please clarify its contents and if it is worth tracking down, I would appreciate it.

[0] Wilcox, Robustness of Standard Tests

[1] Fundamentals of Modern Statistical Methods: Substantially Improving Power and Accuracy

[2] Posten, H.O. (1984). Robustness of the Two-Sample T-Test. In Robustness of Statistical Methods and Nonparametric Statistics, D. Rasch, and M.L. Tiku, eds. (Springer, Dordrecht), pp. 92–99.