Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Data Science Challenges at Instacart

tech.instacart.com

Post reply on HN
Print view View original

51–60 of 65 posts

Re: Data Science Challenges at Instacart

#51

Post by JasonCEC »

disgruntledphd2 wrote: post #40
minimaxir wrote: post #18

Earlier quoted context omitted.

There is definitely more technical detail and statistical process in that post, although I strongly question the use of a hiring test that intricate and time consuming for an internship .

That is very true. However, I'm all for these kinds of tests in general. I read the assignment (and after lol'ing at the specification of methods), found it reasonably interesting. I reckon it would take me an hour or two to complete (as long as I didn't get sucked in to exploring the data) :). Then again, I'm not looking for an internship.

You should consider applying for a full time position ;)

Re: Data Science Challenges at Instacart

#52

Post by ves »

sandGorgon wrote: post #34
jeremystan wrote: post #14

Earlier quoted context omitted.

We've worked hard to open source projects whenever we think they'll be useful broadly: https://www.instacart.com/opensource . There is definitely more of this we can (and I hope will) do in the future.

How do you guys run R in production? Just getting started with R based datascience and it has been a struggle to figure out how to build a production data science stack. Do you snapshot the computed models as RData and stream them to s3, etc

That's what my company does

Re: Data Science Challenges at Instacart

#53

Post by ves »

sandGorgon wrote: post #34
jeremystan wrote: post #14

Earlier quoted context omitted.

We've worked hard to open source projects whenever we think they'll be useful broadly: https://www.instacart.com/opensource . There is definitely more of this we can (and I hope will) do in the future.

How do you guys run R in production? Just getting started with R based datascience and it has been a struggle to figure out how to build a production data science stack. Do you snapshot the computed models as RData and stream them to s3, etc

That's what my company does

Re: Data Science Challenges at Instacart

#54

Post by CPLX »

yummyfajitas wrote: post #47
CPLX wrote: post #46

Earlier quoted context omitted.

Because science is an academic pursuit designed to create and test generalizable hypotheses and add to our collective knowledge, while the people in the article are trying to figure out how to optimize the act of underpaying someone to go grab some cans off a supermarket shelf and bring them to me. They're not scientists, they're engineers perhaps, or business analysts.

So a person doing basic biology research for Monsanto isn't a scientist? And whether or not Kantorovich qualifies as a scientist depends on whether he was working for the military or a university at the time he came up with linear programming? That's an interesting definition.

No, basic biology research is science of course.

It doesn't depend on where the person works (though sure that's a relevant sign) it matters what they're doing.

Analyzing business related data and optimizing KPI's isn't science. At best it's applied science, which we have names for, such as engineering or statistics or financial analysis.

Re: Data Science Challenges at Instacart

#55

Post by bmm6o »

jsprogrammer wrote: post #32
callmeed wrote: post #31

Earlier quoted context omitted.

See, this is the problem with the entire business model. Your friend has now put the onus of paying her living wage on the consumer. I don't want to hire a personal delivery person for $10/hour. Unless I've got a $100+ dinner bill, tipping $25 sounds crazy to me. Calling people "cheapskates" for not doing so is completely unfair–especially when these people are already paying a delivery fee. Do you honestly think fam…

The onus of a living wage is always on the consumer. Even if instacart were paying a living wage to the people who perform deliveries, they would need to reflect at least those wages and other expenses in whatever they charge the consumer (unless deliveries are subsidized by VC's or other interested parties).

> The onus of a living wage is always on the consumer

In the sense that the money has to come from somewhere, sure. But wages are paid by employers, and it's shitty to underpay your employees under the premise that the customer will make up the difference in tips. If a single customer failing to tip $20 pushes the worker under minimum wage for the day, then the system is broken and the worker is getting screwed, as usual. And make no mistake, this is how it was designed to work - if they actually cared about their workers, they would charge a reasonable delivery fee to the customer and give all of it to the delivery person.

Re: Data Science Challenges at Instacart

#56

Post by bmm6o »

yummyfajitas wrote: post #45
CPLX wrote: post #44

So we no longer use the word scientist to describe people who do science? What a shame, I think science is really neat and scientists deserve unique respect. As far as I can tell what these people do every day is called "business" or maybe "logistics"

What do you believe distinguishes "science" from "data science"? I.e., why can't "logistics" be a subset of "science"?

There's the old quip that if your discipline has "science" in the name, it probably isn't really a science. More seriously, scientists follow the scientific method, formulating hypotheses and designing experiments to test them. And that's a broad enough definition that it includes A/B testing, so it must apply to some of what they do. But science typically goes another step, generalizing observations and hypotheses into theories; I would be surprised if there was a lot of that going on at Instacart.

Which isn't a slight against them or the field in the least, it's just a debate about definitions.

Re: Data Science Challenges at Instacart

#57

Post by grayclhn »

JasonCEC wrote: post #50
asdfologist wrote: post #35

Earlier quoted context omitted.

So you had 100 interns and 99 of them were from Princeton??

That's now how percentages work....

I'm really curious what numerator and denominator you have in mind, then... are you somehow using fractional internship units? :)

(Ordinarily this would be off topic, but....)

Re: Data Science Challenges at Instacart

#58

Post by sandGorgon »

jeremystan wrote: post #48
sandGorgon wrote: post #34

Earlier quoted context omitted.

How do you guys run R in production? Just getting started with R based datascience and it has been a struggle to figure out how to build a production data science stack. Do you snapshot the computed models as RData and stream them to s3, etc

We use R in production in two ways: 1. For batch processes that run daily, hourly or minutely, where the models are rebuilt on every run, and outputs (often predictions) are written to a database 2. For computation of coefficients in large sparse regularized models, where the coefficients are written to a database and scoring is done in another language in real-time For situations where we want real-time predictions,…

@jeremy - that's helpful! Could you hint at the way you persist sparse models to the DB in R. Especially if you are changing your variables pretty frequently. do you use something like Postgres JSONB (which is funky in R).

Also about scoring in another language - is this really worthwhile for you ? I have often debated just throwing 128GB of RAM on an R machine and calling it a day. As I figure, your "real time" requirements are probably seconds or even minutes (similar to mine).

Re: Data Science Challenges at Instacart

#59

Post by sandGorgon »

ves wrote: post #52
sandGorgon wrote: post #34

Earlier quoted context omitted.

How do you guys run R in production? Just getting started with R based datascience and it has been a struggle to figure out how to build a production data science stack. Do you snapshot the computed models as RData and stream them to s3, etc

That's what my company does

could you talk a bit more about your production setup ? Any multi-threading problems ?
Post reply on HN