Earlier quoted context omitted.
If the requirement is to check uniqueness, what assumptions could possibly cause a bug? In this case, why does it matter if the uniqueness is tested with a nested for loop or with a map? There are many identical ways to check uniqueness, some being faster than others.
[flagged]
How we decreased GitLab repo backup times from 48 hours to 41 minutes
91–100 of 273 posts
Re: How we decreased GitLab repo backup times from 48 hours to 41 minutes
#92> What this means for GitLab customers — [a bunch of stuff about how customers can now back up more frequently and more robustly] Realtalk: They should rewrite this post's headline to be in a positive tense instead of leading with a negative word. I'm glad I read the post, because it is a cool and good fix, but I saw “Decreasing […] repo backup” and my first thought was that it was an announcement of some service dow…
I don’t think it’s unreasonable to expect interested people to read five words from the title. I know people don’t always do that, but complaining about it as if they did something wrong is ridiculous.
Re: How we decreased GitLab repo backup times from 48 hours to 41 minutes
#93Earlier quoted context omitted.
I disagree. Misuse of the word "exponential" is a major pet peeve of mine. It's a particular case of the much more common "use mathematically precise phrasing to sound careful/precise" that you often find in less than honest writing. Here they are actually using it to refer to growth functions (which is rare for this error) and being honest (which is also rare IMO) but it's still wrong. They should have written about…
Sloppy writing is up orders of magnitude lately.
Re: How we decreased GitLab repo backup times from 48 hours to 41 minutes
#9448 hours is a crazy amount of time to spend just to compress a git folder, it's only a couple GB. 41 minutes still seems like quite a long time. Why aren't they just snapshotting and archiving the full git repo? Does `git bundle` add something over frequent ZFS backups?
... is this really the way people "back up" git repos? I mean, it is git, so isn't there some way to mirror changes to the repo in another repo and just use ZFS / snapshots / backup software / etc to do that? It's a distributed version control system. Just make sure the version control information is ... distributed?
Re: How we decreased GitLab repo backup times from 48 hours to 41 minutes
#95Cool discovery but the article could have been about 1/10 as long and still communicated effectively. At least they didn't post it as a video, so it was easy to skim to the important details.
Re: How we decreased GitLab repo backup times from 48 hours to 41 minutes
#96Earlier quoted context omitted.
OP here. Feedback is always welcome, I did mean exponentially in the colloquial sense. I do see how it is confusing here, will change it.
Thank you. (I don't think that anyone should use "exponentially" that way: it is an art term with a specific and particular meaning, so find another word if you mean something else! Like misusing specific legal or sporting terms...)
Re: How we decreased GitLab repo backup times from 48 hours to 41 minutes
#97I'm confused why you wouldn't simply snapshot the block-level device if the protocol of the information on top is going to cause this much headache. Quiescing git operations for block level activity is probably not trivial, but it sounds like an easier problem to solve to me. This is the approach I've taken with SQLite in production environments. Turn on WAL and the problem gets even easier to solve. Customer configu…
Bingo. One of the worst problems is helping a client piece back together a corrupted repo when they are using snapshots. Check my profile to see how I know. :)
It's usually an OMG down scenario, and then you are adding in the "oh no, now the restore is corrupted."
It's fixable but it's definitely annoying.
Re: How we decreased GitLab repo backup times from 48 hours to 41 minutes
#98Re: How we decreased GitLab repo backup times from 48 hours to 41 minutes
#99Re: How we decreased GitLab repo backup times from 48 hours to 41 minutes
#100Earlier quoted context omitted.
The second law is that O(n * log n) is for practical intents and purposes O(n).
To be clear though, that isn't his second law, at least as of two months ago, according to https://bsky.app/profile/randomascii.bsky.social/post/3lk4c6...
Also, the wise statement that 'memory is fairly cheap compared to CPU for scaling'. It's insane to see how often folks would rather manually open and scan a 'static-on-deploy' 20-100MB Json file for each request vs just parsing it into structures in memory (where, for most cases, the in memory usage is a fraction of the json itself) and just caching the parsed structure for the length of the application.