Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Data science interview questions with answers

github.com

Post reply on HN
Print view View original

51–60 of 75 posts

Re: Data science interview questions with answers

#51

Post by srean »

jpeloquin wrote: post #49
srean wrote: post #28

Earlier quoted context omitted.

> The t-test is extremely robust to departures from normality given equal sample sizes That's over selling it. Its very sensitive to fat tails and skew. That's the reason robust testing and estimation is a thing. Wilcox's research would be a good near contemporary place to start [0][1]. [0] Wilcox, Robustness of Standard Tests https://onlinelibrary.wiley.com/doi/abs/10.1002/978111844511... (paywalled) Abstract: Conve…

> Its very sensitive to fat tails and skew. That's the reason robust testing and estimation is a thing. Yeah, I shouldn't have said "extremely". Its much more robust than is commonly perceived, but that does not make it "extremely" robust. Thank you for the correction. But please note that I specified equal sample sizes , so the fact that "there are general conditions under which these methods can provide poor contro…

No worries and I totally agree that t-test are surprisingly robust to some benign deviations from Gaussianity, a lot more than one would have thought. In my line of work I have had to watch out for fat tails (ridiculously common) and skew -- they can be potent t-test killers.

In the book Wilcox champions robust estimators and tests (a la Huber, Tukey) because efficiency of MLE is very brittle.

Re: Data science interview questions with answers

#52

Post by claudeganon »

minimaxir wrote: post #25

The weirdest thing about data science interviews (when I was actively interviewing) is SQL gotcha questions. Especially with window functions. Here's a long HN thread a few months ago about an annoying situation with an interviewer asserting uncommon syntax is common: https://news.ycombinator.com/item?id=23053981 Another related SQL gotcha I saw multiple times is finding the top n records of each group in a table. Wh…

These kinds of stories seem insane to me. I currently have to do some data science-y stuff in the more generalist consulting dev role I have. I used to do a lot of SQL stuff years ago, but have forgotten most of the syntax beyond the basics. That being said, all it took is some minor googling and reading of a few blog posts to solve some middling hard problems for my client.

What kind of companies make people jump through these nerd hoops? Do they actually have real work that needs doing or is this all just posturing by interviewers?

Re: Data science interview questions with answers

#53

Post by srean »

bonoboTP wrote: post #26
bobdosherman wrote: post #15

Linear regression does not require errors to be iid, normal, and homoscedastic for it to "work". One of the ways to separate candidates is to push on what (and how) assumptions can be weakened, what the consequences are for estimation and inference, and what sort of corrections can be incorporated for maintaining consistency, improving efficiency, correcting biases, etc. An entry level candidate may not have (nor nee…

Interesting and perhaps shows the cultural differences between ML and stats people. I took a machine learning course in my bachelor's and two more ML courses in my master's (CS). These weren't some "deep learning lite", mess-around-in-Keras courses, because DL wasn't even big back then. We covered lots of stuff, Bayesian linear regression, Gaussian processes, Gibbs sampling, Metropolis-Hastings, hierarchical Dirichle…

By tribe I am firmly in the machine learning camp but I have serious doubts that one can be a good hands-on data-science practitioner if one does not have a good foundation in statistics.

Re: Data science interview questions with answers

#54

Post by noodlenotes »

y42 wrote: post #32
ryndbfsrw wrote: post #27

I’ve worked in the field for 7 years now so not that long but long enough to build some heuristics. The best data scientists are just people who try to understand the ins and outs of business processes and look at problems with healthy suspicion and curiosity. The ability to explain the nuances of manifolds in SVMs is not something that comes into it outside these contrived interviews. I prefer to ask candidates how…

>> I prefer to ask candidates how they would approach solving a problem Word. Totally off topic, but: I work in the field of information technology since more than 20 years now, more or less. Not always the same focus, not always full time, but always IT related. I consider myself a good problem solver because of my self learning and analytical skills. I recently applied for a job as a BI developer. The interview con…

There's another possibility but only because you mentioned SQL specifically. People often use SQL as shorthand for "understand how to manipulate data," and if a new hire doesn't have this skill it can really set a data team back, so interviewers are touchy about SQL. It would be helpful if they clarified if it was the SQL syntax skills or the data manipulation skills they had a problem with. But generally I agree with you that companies should prioritize problem solving skills over technical minutiae.

Re: Data science interview questions with answers

#55

Post by BeetleB »

jpeloquin wrote: post #19
fractionalhare wrote: post #13

The quality and depth of answers here is pretty inconsistent. But this in particular is a pet peeve of mine: > Plot a histogram out of the sampled data. If you can fit the bell-shaped "normal" curve to the histogram, then the hypothesis that the underlying random variable follows the normal distribution can not be rejected. This is commonly taught in undergrad stats, but you shouldn't do this. I'm of the opinion that…

> You should strive to obtain a deep understanding of your data and its distribution, and you should be deeply skeptical if the sample you happen to have looks normal. Although normality testing is useless in many situations, the parent comment somewhat overstates the degree of caution required. In many contexts the exact distribution doesn't matter; sort-of-normal is good enough. For example, the t-test is used ubiq…

I'm guessing his concern is visually looking at a histogram and gauging its normality vs doing so with a q-q plot, which will likely show deviations from normality much better.

Re: Data science interview questions with answers

#56

Post by proc-iomem »

minimaxir wrote: post #25

The weirdest thing about data science interviews (when I was actively interviewing) is SQL gotcha questions. Especially with window functions. Here's a long HN thread a few months ago about an annoying situation with an interviewer asserting uncommon syntax is common: https://news.ycombinator.com/item?id=23053981 Another related SQL gotcha I saw multiple times is finding the top n records of each group in a table. Wh…

> Especially with window functions.

Why are companies so bad at hiring?

Re: Data science interview questions with answers

#57

Post by logarhythmic »

fractionalhare wrote: post #12
marcusabu wrote: post #9

Earlier quoted context omitted.

I think it's a bit far fetched to assume that recruiters tailor interview questions based on these Github repo's. For me as a junior data scientist it really useful to test my own knowledge and highlight areas which I need to study more.

Lots of tech and finance companies (particularly those with standardized interview processes) will blacklist questions if they're found online. Those companies will constantly check GitHub, GeeksForGeeks and Leetcode to see if their questions are listed there with solutions. This probably won't be the case for a question as basic as, "what is regression?" But for any intermediate to advanced interview question involv…

A bit off topic but how much of data science work requires this probability/statistics knowledge on the job? I've heard you basically need a PhD to do modelling and "real" data science

Re: Data science interview questions with answers

#58

Post by atty »

pw6hv wrote: post #10
atty wrote: post #5

Earlier quoted context omitted.

I read two answers from technical.md, and one of them was inconsistent with what was asked for... I would be very suspicious of using this repo for studying.

Could you elaborate on which one in particular? I've also read the first questions but haven't noticed anything like this.

Sorry, I should have put it in my original reply. If you search for where it asks for an implementation of the standard deviation, you'll see that it asks for an implementation including Bessel's correction (using N-1 instead of N for the normalization, used to correct for the one degree of freedom lost by the calculation of the mean). However, when you look at the answer, provided, you'll notice that it divides by N, and does not provide 'NaN' for the condition N=1, as was asked.

Re: Data science interview questions with answers

#59

Post by gerbler »

claudeganon wrote: post #52
minimaxir wrote: post #25

The weirdest thing about data science interviews (when I was actively interviewing) is SQL gotcha questions. Especially with window functions. Here's a long HN thread a few months ago about an annoying situation with an interviewer asserting uncommon syntax is common: https://news.ycombinator.com/item?id=23053981 Another related SQL gotcha I saw multiple times is finding the top n records of each group in a table. Wh…

These kinds of stories seem insane to me. I currently have to do some data science-y stuff in the more generalist consulting dev role I have. I used to do a lot of SQL stuff years ago, but have forgotten most of the syntax beyond the basics. That being said, all it took is some minor googling and reading of a few blog posts to solve some middling hard problems for my client. What kind of companies make people jump th…

I think an element of it is that you can have unambiguous questions with SQL, which taken to the extreme do fall into gotcha territory.

I was an interviewer for a business analyst & data science team recently and we needed to hire several analysts & data-scientists who had strong SQL skills because 90% of the data manipulation/processing used SQL. I was definitely aware that this limits who we hire, but we were very short-staffed and needed people so it was easiest for us to hire people who had a good understanding of SQL. That said, we did not care about syntax subtleties, just more if could you generally answer questions using SQL.

And for the data-scientist role there was a lot more than just SQL but it was a useful check.

Re: Data science interview questions with answers

#60

Post by tgv »

srean wrote: post #41
tgv wrote: post #38

Earlier quoted context omitted.

There's a test for normality. Well, I knew one, Kolmogorov, but Wikipedia already lists 8. What data scientist doesn't know of such a test? And how would you fit the data? There's not one, unique way to fit. And as you say, a small deviation can mean a lot: if you use L2 distance, the errors at the outer regions of a normal distribution are probably dwarfed by any deviation that occurs more towards the center.

There are myriads of ways for two things to be unequal. In fact its fair to state that unless you are taking two sets of measurements of some fundamental physical constant or measuring artificially generated data, they are not going to be from the same exact distribution -- you cannot bathe in the same river twice is very true. The way to address the problem you speak of is to think hard about the kind deviations fro…

I would have blindly gone for KS...
Post reply on HN