Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

My Favorite Engineering Interview Question

skife.org

Post reply on HN
Print view View original

1–10 of 131 posts

Re: My Favorite Engineering Interview Question

#2

Post by codeslinger »

Why have SSDs killed this question? Seems like you could tweak the amount of data on disk and/or the size of the values and/or the requests/second requirement and keep using it. Also, there are no 1TB SSDs available at this point, so you'd still have to assume spinning platters for this, no?

Re: My Favorite Engineering Interview Question

#4

Post by bialecki »

I like this question, but I'm wondering what a good interview looks like if someone is completely blind-sided by the question because they don't have a lot (or any) systems experience? I'm imagining an engineer who's experience ends with MySQL so knowledge of stuff like tcb is a definite no. Maybe that's not an issue, because they wouldn't be interviewing for this job if that was the case.

Re: My Favorite Engineering Interview Question

#5

Post by derwiki »

Advice for new questions: instead of contrived ones, pick a problem you've actually encountered in your job and ask the candidate to solve it. I've never had to implement a datastore like this -- so unless that's what you're actually doing, it doesn't seem particularly relevant. Another plus is that if you're doing interesting work, you have an almost endless supply of questions to choose from. Candidates also seem to respond better when you wrap up the question with "this is something I actually worked on a few weeks ago."

FWIW, Ning turned me down back in April 2008 so feel free to ignore my advice :)

Re: My Favorite Engineering Interview Question

#6

Post by drv »

codeslinger wrote: post #2

Why have SSDs killed this question? Seems like you could tweak the amount of data on disk and/or the size of the values and/or the requests/second requirement and keep using it. Also, there are no 1TB SSDs available at this point, so you'd still have to assume spinning platters for this, no?

I haven't thought about it deeply, but my first guess would be that the seek time of hard disks is what makes serving 5000 requests per second difficult; consumer-grade hard disks can perform something in the range of a few hundred (randomly distributed) IOPS at best due to seek time. SSDs make seeks (almost) free, so even one decent SSD should be able to service 5000 requests per second, assuming you can get a big enough one.

If multiple storage devices (hard disks or otherwise) are allowed, RAID (or just splitting the data across multiple disks) would ameliorate the seek problem somewhat, since seeks can then happen in parallel. I wouldn't see this problem as requiring a distributed solution, unless the bottleneck is the bus or storage controller rather than the single-disk seek performance.

Re: My Favorite Engineering Interview Question

#7

Post by barrkel »

I'm interested in why he's so against the "custom solution". Almost everything a DB will try and add in for key-value lookup is predicated on the idea that the requests aren't randomly distributed. The DB index will probably be based around b-trees, which will do a logarithmic-time search for the top few levels cached in RAM, but will fall over fairly miserably with multiple seeks as it has to page in leaf nodes and likely the nodes a level above those: I don't see it as realistic to expect a DB to do it all in a single seek.

On the other hand, if you have a perfect hash function for the keyspace into a hash addressing space (you'll want something a bit larger than 2^32), you could simply choose the device with the first few bits, then do a single seek into a (large) file on disk with the position indicated by the remaining bits of the hash addressing space, shifted over appropriately for the 128-byte stride of values. You don't even need a perfect hash: with a merely good hash function, you can cheaply read in more than 128 bytes from disk (OS will probably be reading in 4K anyhow), and use open addressing with linear probing.

You'll need enough devices to spread the load to get good latency for random seeks; probably a combination of mirroring and what amounts to sharding, using a portion of the key hash to select the device. But that's a relatively cheap numbers game.

But I'm not really seeing what a DB buys you here, in this specific scenario.

EDIT: Further investigation of tc and cdb (as linked in the article) suggests that tc (Tokyo Cabinet) may work in its hash form, but I'd rule out cdb for using two seeks, which I would expect to double the number of devices required.

Re: My Favorite Engineering Interview Question

#8

Post by jsankey »

An idea I've tried a bit in the past (and would like to try more, given the chance): ask a question that you don't know the answer to. This shifts the power balance of the interview - you don't automatically have the answer over the candidate. You also don't have your mind closed by your expected answer(s). Both of these make the resulting discussion more comparable to how you will work with a colleague, which is what you are trying to test the candidate for, after all.

Doing this requires more effort and guts from the interviewer, but it's still easier than being on the other side of the desk. Finding a supply of suitable questions is probably the trickiest part - you could try looking up some problems from competitions just prior to the interview, or you could perhaps take a real problem you are trying to solve at that point in time (abstracted as required).

Post reply on HN