These engineering blogs are intended to promote the engineering culture of the company for recruitment process, and as a correlate an opportunity for the engineers to self-promote and boost their resume/promotion-worthiness internally. The sentiment on this thread, if it is indicative of the greater talent pool, suggests this blog post is having the complete opposite effect. I remember when Uber was proud of their th…
We Put Half a Million Files in One Git Repository, Here’s What We Learned
161–170 of 270 posts
Re: We Put Half a Million Files in One Git Repository, Here’s What We Learned
#162Earlier quoted context omitted.
Sure, how did we ever manage to rename something without monorepos. Oh wait, maybe that's what this "versionning" thing is all about.
Right, it's "I have to send 5 PRs to 5 different repos, get them all cross merged, and then at the end it's wrong anyways so I have to start all over". Multirepo management is extremely frustrating compared to "it's all in the same folder".
Which means that you've got to do independent backwards-compatible changes anyway, and that for anything remotely complex, you are better off with separate branches (and PR/MRs) anyway.
Monorepos mostly have a benefit for trivial changes across all repos (eg. we've decided to rename our "Shop" to "Shoppe"), where it doesn't really take much to explain with multiple repos, but is mostly a lot of tedious work to get all the PRs up and such.
Re: We Put Half a Million Files in One Git Repository, Here’s What We Learned
#163Earlier quoted context omitted.
How would you do refactoring over monorepo if you have sparse checkout?
CI/CD build the entire project. So if you make a breaking change in library the build will fail.
Re: We Put Half a Million Files in One Git Repository, Here’s What We Learned
#164If you don't have a couple of hundreds engineers who would write a custom file system for git (Microsoft), or who would take an existing source control system and nearly fully rewrite it (Facebook), or who would write a custom source control system from scratch (Google, Yandex, etc) DON'T USE MONOREPO. Otherwise you are risking to end up in a situation when hundreds of your engineers have to spend tens of minutes eve…
Re: We Put Half a Million Files in One Git Repository, Here’s What We Learned
#165Hey everyone, author here, the article is a bit misleading in that .xlf files aren't really generated files, they're created through our translation pipeline by real humans. I considered them generated in the sense that they're not directly worked on by engineers who have to deal with them in the repository. The content of these translation files are snapshot in time aligned with the text in our product so simply rem…
Can't those xlf files be stored in a separate repo or in an object storage and let the build system fetch them from there?
Re: We Put Half a Million Files in One Git Repository, Here’s What We Learned
#166Earlier quoted context omitted.
The problem is not the repository, it’s having the translation data in the repo. We have the same issue at $dayjob, the repo is quite large and 80% of it is translation data. Even if they compress ridiculously well (99% last I checked) the number of translation files and the number of exports makes them the vast majority of the cost. However that structure remains a convenient nuisance, and more importantly removing…
But isnt it sort of correct to have translation data, which is text data, and there might be bug fixes, in the repo?
Not to mention transgressions often have their own lifecycle e.g. is common to find translatable strings which are untranslated or incorrectly translated, and want to ship updates independently from the software’s.
As such keeping the translations outside the source is also perfectly defensible.
Re: We Put Half a Million Files in One Git Repository, Here’s What We Learned
#167Earlier quoted context omitted.
The problem is not the repository, it’s having the translation data in the repo. We have the same issue at $dayjob, the repo is quite large and 80% of it is translation data. Even if they compress ridiculously well (99% last I checked) the number of translation files and the number of exports makes them the vast majority of the cost. However that structure remains a convenient nuisance, and more importantly removing…
But isnt it sort of correct to have translation data, which is text data, and there might be bug fixes, in the repo?
Re: We Put Half a Million Files in One Git Repository, Here’s What We Learned
#168Earlier quoted context omitted.
I made another comment as well though tldr is these xlf files are translations tied to texts in code so we can't simply ignore them from the repository. The changes have to be kept so that if we say revert to a certain commit, all the translations match with the texts of headers, buttons, etc...
Could these translations be moved to another repository? Maybe they could be published as a separate NPM package that devs could install if they needed to look at the translations?
Re: We Put Half a Million Files in One Git Repository, Here’s What We Learned
#169Earlier quoted context omitted.
monorepo until it becomes too big -> then, splitting it into 2-3 repos, until each one also gets too big to manage...
The problem is not the repository, it’s having the translation data in the repo. We have the same issue at $dayjob, the repo is quite large and 80% of it is translation data. Even if they compress ridiculously well (99% last I checked) the number of translation files and the number of exports makes them the vast majority of the cost. However that structure remains a convenient nuisance, and more importantly removing…
Does this data change as often as the code does? If not then get it out of the repo.
Re: We Put Half a Million Files in One Git Repository, Here’s What We Learned
#170The fact that Canva has a `Source Control team` with at least 5 people on it (going by the thanks at the bottom of the article), means they should probably try a different approach. I think it's a cool company, with a good product, but they're WAY too small to be having a "source control team" on staff. That's at least 1.2MM a year salary / benefits cost.
That bit surprised me also. I don't have much experience with companies of this size so I have no real idea; how many people would you expect to work on source control at a company of this size? I had a look at Linkedin, they have 4,345 employees.