Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We shrunk our Javascript monorepo git size

jonathancreamer.com

Post reply on HN
Print view View original

91–100 of 226 posts

Re: We shrunk our Javascript monorepo git size

#91

Post by atombender »

clktmr wrote: post #76
tux3 wrote: post #32

For those wondering where this new git-survey command is, it's actually not in git.git yet! The author is using microsoft's git fork, they've added this new command just this summer: https://github.com/microsoft/git/pull/667

[flagged]

Why? You do realize their fork is open source?

The fix described in this post have been submitted as a patch to the official Git project. The fix is improving a legitimate inefficiency in Git, and does nothing towards "embracing", "extending", or "extinguishing" anything.

Re: We shrunk our Javascript monorepo git size

#92

Post by johnisgood »

triyambakam wrote: post #9

> we have folks in Europe that can't even clone the repo due to it's size. What is it about Europe that makes it more difficult? That internet in Europe isn't as good? Actually, I have heard that some primary schools in Europe lack internet. My grandson's elementary school in rural California (population <10k) had internet as far back as 1998.

Some countries in Europe (even Poland) definitely offer faster Internet and for cheaper than the US, and without most of the privacy issues that US ISPs have.

Re: We shrunk our Javascript monorepo git size

#93

Post by jamalaramala »

jakub_g wrote: post #67

Paraphrasing meat of the article: - When you have multiple files in the repo which have the same trailing 16 characters in the repo path, git may wrongly calculate deltas, mixing up between those files. In here they had multiple CHANGELOG.md files mixed up. - So if those files are big and change often, you end up with massive deltas and inflated repo size. - There's a new git option (in Microsoft git fork for now) an…

Thank you to the AI that summarised the article. ;-)

Re: We shrunk our Javascript monorepo git size

#94

Post by heisenbit »

triyambakam wrote: post #9

> we have folks in Europe that can't even clone the repo due to it's size. What is it about Europe that makes it more difficult? That internet in Europe isn't as good? Actually, I have heard that some primary schools in Europe lack internet. My grandson's elementary school in rural California (population <10k) had internet as far back as 1998.

Effective bandwidth can be influenced by roundtrip time. Fewer IP4 numbers means more NAT with more delay and yet another point where occasionally something can go wrong. Last but not least there are some areas in the EU like the Canary Islands where the internet feels like going over a sat.

Re: We shrunk our Javascript monorepo git size

#95

Post by mattlondon »

I recently had a similar moment of WTF for git in a JavaScript repo.

Much much smaller of course though. A raspberry pi had died and I was trying to recover some projects that had not been pushed to GitHub for a while.

Holy crap. A few small JavaScript projects with perhaps 20 or 30 code files, a few thousand lines of code for a couple of 10s of KBs of actual code at most had 10s of gigabytes of data in the .git/ folder. Insane.

In the end I killed the recovery of the entire home dir and had to manually select folders to avoid accidentally trying to recover a .git/ dir as it was taking forever on a poorly SD card that was already in a bad way and I did not want to finally kill it for good by trying to salvage countless gigabytes of trash for git.

Re: We shrunk our Javascript monorepo git size

#96

Post by badgersnake »

_joel wrote: post #83
badgersnake wrote: post #65

Earlier quoted context omitted.

Not in the UK. Still on 80Mbit VDSL here.

You must be unlucky, according to Openreach "fibre broadband is already available in more than 96.59 per cent of the UK."

Coming next year apparently. I won’t hold my breath.

Re: We shrunk our Javascript monorepo git size

#97

Post by masklinn »

kreetx wrote: post #77
jakub_g wrote: post #67

Paraphrasing meat of the article: - When you have multiple files in the repo which have the same trailing 16 characters in the repo path, git may wrongly calculate deltas, mixing up between those files. In here they had multiple CHANGELOG.md files mixed up. - So if those files are big and change often, you end up with massive deltas and inflated repo size. - There's a new git option (in Microsoft git fork for now) an…

I don't think GitHub, or any other git host, will have objections to using it once it's part of mainline git? Also, thank you for the TLDR!

> I don't think GitHub, or any other git host, will have objections to using it once it's part of mainline git?

Fixing an existing repository requires a full repack, and for a repository as big as Chromium it still takes more than half a day (56000 seconds is 15h30), even if that's an improvement over the previous 3 days it's a lot of compute.

From my experience of previous attempts, trying to get Github to run a full repack with harsh settings is extremely difficult (possibly because their infrastructure relies on more loosely packed repositories), I tried to get that for $dayjob's primary repository whose initial checkout had gotten pretty large and got nowhere.

As of right now, said repository is ~9.5GB on disk on initial clone (full, not partial, excluding working copy). Locally running `repack -adf --window 250` brings it down to ~1.5GB, at the cost of a few hours of CPU.

The repository does have some of the attributes described in TFA, so I'm definitely looking forward to trying these changes out.

Re: We shrunk our Javascript monorepo git size

#99

Post by ahartmetz »

holowoodman wrote: post #69
0points wrote: post #57

Earlier quoted context omitted.

What european countries still dont have fiber? //Confused swede with 10G fiber all over the place. Writing from literally the countryside next to nowhere.

Germany. Deutsche Telekom is the former monopoly that was half-privatized around 1995 or something. The state still owns quite a large stake of it. They milk their ancient copper crap for everything they can while keeping prices high. They are refusing useful backbone interconnects to monopolize access to their customers (Actually they are not allowed to refuse. They just offer interconnections only in their data cen…

It is almost as bad as you say, except that I recently noticed several instances of competitors offering cheaper fiber than Telekom and surviving. Still, overall fiber buildout is low, like... I looked it up, reportedly 36% now.

Re: We shrunk our Javascript monorepo git size

#100

Post by matrss »

wodenokoto wrote: post #25
bubblesnort wrote: post #15

> We work in a very large Javascript monorepo at Microsoft we colloquially call 1JS. I used to call it office.com.. Teams is the worst offender there. Even a website with a cryptominer on it runs faster than that junk.

We were all impressed with google docs, but office.com is way more impressive. Collaborative editing between a web app, two mobile anpps and a desktop app with 30 years of backwards compatibility and it pretty much just works. No wonder that took a lot of JavaScript!

> [...] and it pretty much just works.

I beg to differ. Last time I had to use PowerPoint (granted, that was ~3 years ago), math on the slides broke when you touched it with a client that wasn't of the same type as the one that initially put it there. So you would need to use either the web app or the desktop app to edit it, but you couldn't switch between them. Since we were working on the slides with multiple people you also never knew what you had to use if someone else wrote that part initially.

Post reply on HN