[dead]
How we decreased GitLab repo backup times from 48 hours to 41 minutes
161–170 of 273 posts
Re: How we decreased GitLab repo backup times from 48 hours to 41 minutes
#162Earlier quoted context omitted.
This is not the precise mathematical definition of exponential, but rather the colloquial one, where it just means "a lot".
You shouldn't use a word that can carry a precise mathematical meaning in a sentence that literally uses mathematical notation in order to speak precisely and then expect readers not to interpret the word in the precise mathematical way.
I find the whole article rather poorly written. Most likely using an LLM.
Re: How we decreased GitLab repo backup times from 48 hours to 41 minutes
#163Earlier quoted context omitted.
Not often but occasionally I will chose the nlogn algorithm which obviously has no bugs over the O(n) algorithm with no obvious bugs. Less brittleness is worth paying a few percent. Especially if it unmuddies the waters enough for someone to spot other accidental (time) complexity.
Considerably more than a few percent, IMHO. :) But I also don't dabble in this area nearly enough to know whether there's years of tears and toil finding out repeatedly that O(n) is ~impossible to implement and verify :) | n | n log n | | 5 | 8.0472 | | 10 | 23.0259 | | 25 | 80.4719 | | 50 | 195.6012 | | 100 | 460.5170 |
This is magic thinking about how C, memory hierarchies, networking, and system calls work.
Re: How we decreased GitLab repo backup times from 48 hours to 41 minutes
#164Earlier quoted context omitted.
I made the “mistake” in an interview of equating two super-quadratic solutions in an interview. What I meant was what Dawson meant. It doesn’t matter because they’re both too ridiculous to even discuss.
They’re too ridiculous… unless a more optimal solution does not exist
Absolutely not.
If the cost of doing something goes above quadratic, you shouldn't do it at all. Because essentially every customer interaction costs you more than the one before. You will never be able to come up with ways to cover that cost faster than it ramps. You are digging a hole, filling it with cash and lighting it on fire.
If you can't do something well you should consider not doing it at all. If you can only do it badly with no hope of ever correcting it, you should outsource it.
Re: How we decreased GitLab repo backup times from 48 hours to 41 minutes
#165Earlier quoted context omitted.
They’re too ridiculous… unless a more optimal solution does not exist
Absolutely not. If the cost of doing something goes above quadratic, you shouldn't do it at all. Because essentially every customer interaction costs you more than the one before. You will never be able to come up with ways to cover that cost faster than it ramps. You are digging a hole, filling it with cash and lighting it on fire. If you can't do something well you should consider not doing it at all. If you can on…
Chess engines faced worse than quadratic scaling and came out the other side…
Software operates in a crazy number of different domains with wildly different constraints.
Re: How we decreased GitLab repo backup times from 48 hours to 41 minutes
#166Cool discovery but the article could have been about 1/10 as long and still communicated effectively. At least they didn't post it as a video, so it was easy to skim to the important details.
Yes. I read the whole article thinking that this must have been generated by LLM, because at least the style remembers it.
Glad I wasn’t the only one who thought this. The post is also missing one obvious thing that I expect in any technical post: code snippets. Let me see the code.
ChatGPT has ruined bullet points for the rest of us…
No offense but writing this blog post couldn’t take more than a few minutes, why spoil it with LLM? Shoot, use one to check grammar and recommend edits even.
Re: How we decreased GitLab repo backup times from 48 hours to 41 minutes
#167I miss a good tech blog post. Cheers for this
Re: How we decreased GitLab repo backup times from 48 hours to 41 minutes
#168[flagged]
We detached this subthread from https://news.ycombinator.com/item?id=44203252 and marked it off-topic.
Re: How we decreased GitLab repo backup times from 48 hours to 41 minutes
#169Earlier quoted context omitted.
They’re too ridiculous… unless a more optimal solution does not exist
Absolutely not. If the cost of doing something goes above quadratic, you shouldn't do it at all. Because essentially every customer interaction costs you more than the one before. You will never be able to come up with ways to cover that cost faster than it ramps. You are digging a hole, filling it with cash and lighting it on fire. If you can't do something well you should consider not doing it at all. If you can on…
All of modern Neural Network AI is based on GEMM which are O(n^2) algorithms. There are sub-cubic alternatives, but it's my understanding that the cache behavior of those variants mean they aren't practically faster when memory bound.
n is only rarely related to "customers". As long as n doesn't grow, the asymptotic complexity doesn't actually matter.
Re: How we decreased GitLab repo backup times from 48 hours to 41 minutes
#170Reminds me of the timeless advice I got for acing leet code interviews: “remember to use a hash map.” Tends to work out pretty well.