Data Science Challenges at Instacart • Hacker News Board

Re: Data Science Challenges at Instacart

#61

Post by Jtsummers » Thu, Feb 18, 2016, 5:18 PM UTC

bmm6o wrote: post #55

jsprogrammer wrote: post #32

Earlier quoted context omitted.

The onus of a living wage is always on the consumer. Even if instacart were paying a living wage to the people who perform deliveries, they would need to reflect at least those wages and other expenses in whatever they charge the consumer (unless deliveries are subsidized by VC's or other interested parties).

> The onus of a living wage is always on the consumer In the sense that the money has to come from somewhere, sure. But wages are paid by employers, and it's shitty to underpay your employees under the premise that the customer will make up the difference in tips. If a single customer failing to tip $20 pushes the worker under minimum wage for the day, then the system is broken and the worker is getting screwed, as u…

In the restaurant world, the employer would have to make up the difference and ensure their employees make minimum wage if tips aren't enough. While that's still not satisfactory (to me, at least), it at least guarantees that a slow shift, or a lousy-tipping, long-staying, large-group table won't push you down to $2.13/hour level income (though minimum wage doesn't get you much more).

Re: Data Science Challenges at Instacart

#62

Post by jeremystan » Thu, Feb 18, 2016, 5:23 PM UTC

sandGorgon wrote: post #58

jeremystan wrote: post #48

Earlier quoted context omitted.

We use R in production in two ways: 1. For batch processes that run daily, hourly or minutely, where the models are rebuilt on every run, and outputs (often predictions) are written to a database 2. For computation of coefficients in large sparse regularized models, where the coefficients are written to a database and scoring is done in another language in real-time For situations where we want real-time predictions,…

@jeremy - that's helpful! Could you hint at the way you persist sparse models to the DB in R. Especially if you are changing your variables pretty frequently. do you use something like Postgres JSONB (which is funky in R). Also about scoring in another language - is this really worthwhile for you ? I have often debated just throwing 128GB of RAM on an R machine and calling it a day. As I figure, your "real time" requ…

To persist sparse models to the DB, especially if you use L1 regularization (like Lasso) then many coefficients will be 0, and don't need to be stored or processed. Insofar as you store coefficients and features in a "tall" format (e.g., user, feature_key, feature_value), then space is conserved. Scoring can be done in DB with joins and group by, or in another language with similar operations.

Changing variables frequently can be versioned in the feature and model coefficients tables, but takes care.

I haven't used Postgres JSONB, but if you have problems with JSON in R check out the tidyjson package (I wrote when dealing with Mongo data previously).

Scoring in another language is best avoided if you can. But supporting R "real time" services will also come with many complications. Hence, we use Python when we really want that.

SparkR was completely unreliable when I first tested it over a year ago, but may have improved. Though the Spark Python API has some limitations compared to Scala, so I would guess the latest SparkR is even further behind, but we haven't tested it. Long term I'd love for that to be the answer to these questions.

Re: Data Science Challenges at Instacart

#63

Post by jsprogrammer » Thu, Feb 18, 2016, 7:22 PM UTC

bmm6o wrote: post #55

jsprogrammer wrote: post #32

Earlier quoted context omitted.

The onus of a living wage is always on the consumer. Even if instacart were paying a living wage to the people who perform deliveries, they would need to reflect at least those wages and other expenses in whatever they charge the consumer (unless deliveries are subsidized by VC's or other interested parties).

> The onus of a living wage is always on the consumer In the sense that the money has to come from somewhere, sure. But wages are paid by employers, and it's shitty to underpay your employees under the premise that the customer will make up the difference in tips. If a single customer failing to tip $20 pushes the worker under minimum wage for the day, then the system is broken and the worker is getting screwed, as u…

Of course, the pricing of resources should be complete and incorporate all costs. It is certainly unacceptable for tips to be required to make up a living wage.

However, no one should be under the delusion that the consumer is not responsible for those costs.

Re: Data Science Challenges at Instacart

#64

Post by bmm6o » Fri, Feb 19, 2016, 1:01 AM UTC

jsprogrammer wrote: post #63

bmm6o wrote: post #55

Earlier quoted context omitted.

> The onus of a living wage is always on the consumer In the sense that the money has to come from somewhere, sure. But wages are paid by employers, and it's shitty to underpay your employees under the premise that the customer will make up the difference in tips. If a single customer failing to tip $20 pushes the worker under minimum wage for the day, then the system is broken and the worker is getting screwed, as u…

Of course, the pricing of resources should be complete and incorporate all costs. It is certainly unacceptable for tips to be required to make up a living wage. However, no one should be under the delusion that the consumer is not responsible for those costs.

I'm not sure who in this scenario you think is delusional, or if you're conflating being delusion with being uninformed of the specific employment arrangement between the people you interact with and their employers. Excuse me, they aren't employers anymore, it's a contractor/"logistics match-making company" relationship now.

Re: Data Science Challenges at Instacart

#65

Post by jsprogrammer » Fri, Feb 19, 2016, 4:37 AM UTC

bmm6o wrote: post #64

jsprogrammer wrote: post #63

Earlier quoted context omitted.

Of course, the pricing of resources should be complete and incorporate all costs. It is certainly unacceptable for tips to be required to make up a living wage. However, no one should be under the delusion that the consumer is not responsible for those costs.

I'm not sure who in this scenario you think is delusional, or if you're conflating being delusion with being uninformed of the specific employment arrangement between the people you interact with and their employers. Excuse me, they aren't employers anymore, it's a contractor/"logistics match-making company" relationship now.

It is neither.

Certainly a business should follow the rules of its market, but I think that we agree that the expectation of a consumer is that the price they pay covers all expenses + profit.