We Put Half a Million Files in One Git Repository, Here’s What We Learned
1–10 of 270 posts
Re: We Put Half a Million Files in One Git Repository, Here’s What We Learned
#2>just under 60 million lines of code in 2022.
I'm always surprised at the fact that almost every product has more lines of code than the entire Linux repo. The scale of these products is astounding.
Re: We Put Half a Million Files in One Git Repository, Here’s What We Learned
#3>just under 60 million lines of code in 2022. I'm always surprised at the fact that almost every product has more lines of code than the entire Linux repo. The scale of these products is astounding.
From the rest of the article, it sounds like a big chunk of these lines are from generated files. What I don't understand is why they're checking in generated files into Git.
Re: We Put Half a Million Files in One Git Repository, Here’s What We Learned
#4>just under 60 million lines of code in 2022. I'm always surprised at the fact that almost every product has more lines of code than the entire Linux repo. The scale of these products is astounding.
As well as generated files the sibling mentioned I wonder how much is due to vendoring.
Re: We Put Half a Million Files in One Git Repository, Here’s What We Learned
#5>just under 60 million lines of code in 2022. I'm always surprised at the fact that almost every product has more lines of code than the entire Linux repo. The scale of these products is astounding.
From the rest of the article, it sounds like a big chunk of these lines are from generated files. What I don't understand is why they're checking in generated files into Git.
Yes, the article glosses over that a little bit, and it's an unusual decision.
Re: We Put Half a Million Files in One Git Repository, Here’s What We Learned
#6Pretty interesting.... I don't know that much about git, but still fun to read. I guess the main takeaway is don't put all your eggs in one basket? Although it kinda seems like they are going to stick with the monorepo, ("Here’s how we solve them at Canva")
Also, I looked up .xlf files and I still don't understand. It's xml, that part makes sense, but it's basically a config file? To tell what process to read which files?
Also, I've heard of Canva, but had no idea they were this big/ubiquitous/whatever.... and learning about pseudo localization is interesting too. And the graph for lines of code looks pretty exponential, maybe it's common up to a point, but if it continues at that rate, it will be infinite by about 2026 (okay, I just made that number up, but you get the idea)
Re: We Put Half a Million Files in One Git Repository, Here’s What We Learned
#7>just under 60 million lines of code in 2022. I'm always surprised at the fact that almost every product has more lines of code than the entire Linux repo. The scale of these products is astounding.
From the rest of the article, it sounds like a big chunk of these lines are from generated files. What I don't understand is why they're checking in generated files into Git.
As a counter-point, package managers generate “lock files” that are designed to be tracked in VC.
For these translation files, I’d imagine there may be occasional work to modify them even after they are initially generated.
Re: We Put Half a Million Files in One Git Repository, Here’s What We Learned
#8>just under 60 million lines of code in 2022. I'm always surprised at the fact that almost every product has more lines of code than the entire Linux repo. The scale of these products is astounding.
From the rest of the article, it sounds like a big chunk of these lines are from generated files. What I don't understand is why they're checking in generated files into Git.
Reading the article they are not generated files, but files that are never touched by developers. Translators will work with those files. I expect that for translators they have a different sparse checkout that only fetches .xlf files for their target languages.
Re: We Put Half a Million Files in One Git Repository, Here’s What We Learned
#9>just under 60 million lines of code in 2022. I'm always surprised at the fact that almost every product has more lines of code than the entire Linux repo. The scale of these products is astounding.
From the rest of the article, it sounds like a big chunk of these lines are from generated files. What I don't understand is why they're checking in generated files into Git.
They're using git as a cache. Having generated files stored there means they're available if they're needed (eg in CI) without needing further access controls, they're versioned, and it's a simple and understandable strategy. As the article states, most devs are set up to ignore those files so they're not much of a source of the slowness. It's a common pattern for apps that have to serve lots of different locales.
Re: We Put Half a Million Files in One Git Repository, Here’s What We Learned
#10Earlier quoted context omitted.
From the rest of the article, it sounds like a big chunk of these lines are from generated files. What I don't understand is why they're checking in generated files into Git.
Reading the article they are not generated files, but files that are never touched by developers. Translators will work with those files. I expect that for translators they have a different sparse checkout that only fetches .xlf files for their target languages.
xlf files are usually generated. They're XML. No one wants to write that by hand.