I Solved a 7-Day Calculation Problem in a Weekend
1–10 of 23 posts
Re: I Solved a 7-Day Calculation Problem in a Weekend
#2I had a 7-day compute problem, 3 days to solve it, and no extra hardware. Here's what worked.
Re: I Solved a 7-Day Calculation Problem in a Weekend
#3It's odd that OP didn't seem to consider applying the nearest cached value for any given slider stop.
The Gaussian frequency was a cool idea, however.
Also, I would speculate that projected sales would likely be a continuous function in most cases, so I'm curious why they didn't try fitting a function based on initial results.
Re: I Solved a 7-Day Calculation Problem in a Weekend
#4It's odd that OP didn't seem to consider applying the nearest cached value for any given slider stop. The Gaussian frequency was a cool idea, however. Also, I would speculate that projected sales would likely be a continuous function in most cases, so I'm curious why they didn't try fitting a function based on initial results.
OP here,
Ah, good point. To be honest, interpolation didn't even cross my mind.
The model output wasn't just one number, it was a messy JSON with a 12-week forecast. Trying to average two of those felt like a whole other task, and with the deadline, my brain was just stuck on how to pick the right numbers to cache.
But yeah, it's a really great idea. Will definitely keep it in mind for the next demo.
Re: I Solved a 7-Day Calculation Problem in a Weekend
#5Tbh, I'm really not sure why something like this should take 15 seconds to compute. That's roughly a few trillion floating point ops for a problem that has been solvable for decades. I have trouble imagining any reasonable model for mapping price -> sales needing that much compute.
Also, fwiw, I really wouldn't expect clients slider clicks to follow a normal distribution. A normal distribution occurs when you have the sum of a large number of random variables with Finite (and bounded) expectation and variance; or alternatively, when you're modelling a process with known Expectation and Variance, but not any higher order moments. If anything, I'd expect human beings to play with the slider more around extremal points, like the start and end.
Re: I Solved a 7-Day Calculation Problem in a Weekend
#6What is the actual model that takes 15 seconds to compute?
If I understand the setting, you are estimating the demand curve for a given price... And there are only 40 such curves to compute.
Surely each curve is fit with only a few parameters, probably fewer than five. (I think for small price ranges the demand curve is usually approximated as something trivial like y=mx+b or y=a/(x+b)+c)
Why does evaluating the model at a particular price need to take milliseconds, yet alone 15 seconds?
Re: I Solved a 7-Day Calculation Problem in a Weekend
#7I have this problem all the time and I can usually run the calcs in a simple multithreaded process pool/queues. While each calc may still take 15s, I run a dozen or more at a time. This helps me refresh the cache in a reasonable amount of time, doesn’t really focus on improving calc speed of the underlying service which is obviously another potential opportunity
Re: I Solved a 7-Day Calculation Problem in a Weekend
#8So parallel precomputation wasn't an option?
Re: I Solved a 7-Day Calculation Problem in a Weekend
#9Unless I'm misunderstanding, seems like the approach taken here (normal distribution and Monte Carlo simulation) might achieve similar?
Re: I Solved a 7-Day Calculation Problem in a Weekend
#10Tbh, I'm really not sure why something like this should take 15 seconds to compute. That's roughly a few trillion floating point ops for a problem that has been solvable for decades. I have trouble imagining any reasonable model for mapping price -> sales needing that much compute. Also, fwiw, I really wouldn't expect clients slider clicks to follow a normal distribution. A normal distribution occurs when you have th…
Also very curious about what kind of model this is and how it could (so far as it sounds) take 100% of the hardware for 15 seconds per request.