Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We shrunk our Javascript monorepo git size

jonathancreamer.com

Post reply on HN
Print view View original

211–220 of 226 posts

Re: We shrunk our Javascript monorepo git size

#211

Post by spankalee »

wavemode wrote: post #181
crazygringo wrote: post #174

Earlier quoted context omitted.

> all while still arguing monorepos are easy and great I don't know anyone who says monorepos are easy. To the contrary, the tooling is precisely the hard part. But the point is that the difficulty of the tooling is a lot less than the difficulty of managing compatibility conflicts between tons of separate repos. Each esoteric bug in C only needs to be fixed once. Whereas your version compatibility conflict this week…

At Amazon, there is no monorepo. And the tooling to handle this is not even particularly conceptually complicated - a "versionset" is a set of versions - a set of pointers to a particular commit of a repository. When you build and deploy an application, what you're building is a versionset containing the correct versions of all its dependencies. And pull requests can span across multiple repositories. Working at Amaz…

> And pull requests can span across multiple repositories

This bit is doing a lot of work here.

How do you make commits atomic? Is there a central commit queue? Do you run the tests of every dependent repo? How do you track cross-repo dependencies to do that? Is there a central database? How do you manage rollbacks?

Re: We shrunk our Javascript monorepo git size

#212

Post by rurban »

tazjin wrote: post #194
avianlyric wrote: post #186

Earlier quoted context omitted.

I do t think there’s any country in Europe with internet infrastructure as underdeveloped as the US. Most of Europe has fibre-to-the-premise, and all of Europe has consumer internet packages that are faster and cheaper than you’re gonna find anywhere in the U.S.

There's (almost) no FTTH in Germany. The US used to be as bad as Germany, but it has improved significantly and is actually pretty decent these days (though connection speed is unevenly distributed). Both countries are behind e.g. Sweden or Russia, but Germany by a much larger margin. There's some trickery done in official statistics (e.g. by factoring in private connections that are unavailable to consumers) to make…

The east has fibre everywhere, but the west is still a developing country(side). Shipping code on a truck would be faster, if you are not on some academic fibre net

Re: We shrunk our Javascript monorepo git size

#213

Post by rurban »

js2 wrote: post #193
eviks wrote: post #7

upd: silly mistake - file name does not include its full path The explanation probably got lost among all the gifs, but the last 16 chars here are different: > was actually only checking the last 16 characters of a filename > For example, if you changed repo/packages/foo/CHANGELOG.md, when git was getting ready to do the push, it was generating a diff against repo/packages/bar/CHANGELOG.md!

> file name does not include its full path No, it is the full path that's considered. Look at the commit message on the first commit in the `--full-name-hash` PR: https://github.com/git-for-windows/git/pull/5157/commits/d5c... Excerpt: "/CHANGELOG.json" is 15 characters, and is created by the beachball [1] tool. Only the final character of the parent directory can differntiate different versions of this file, but als…

Sounds like it needs to be fixed to FNV1a

Re: We shrunk our Javascript monorepo git size

#214

Post by jonathancreamer »

lastdong wrote: post #48
tux3 wrote: post #41

Earlier quoted context omitted.

Derrick provides a better explanation in this cover letter: https://lore.kernel.org/git/pull.1785.git.1725890210.gitgitg... (See also the path-walk API cover letter: https://lore.kernel.org/all/pull.1786.git.1725935335.gitgitg... ) The example in the blog post isn't super clear, but Git was essentially taking all the versions of all the files in the repo, putting the last 16 bytes of the path (not filename) in a hash…

Thank you! I ended up having to look at the PR to make any sense of the blog post, but your explanation and links makes things much clearer

I'll update the post with this clarity too. Thanks!

Re: We shrunk our Javascript monorepo git size

#215

Post by jonathancreamer »

yashap wrote: post #14
triyambakam wrote: post #9

> we have folks in Europe that can't even clone the repo due to it's size. What is it about Europe that makes it more difficult? That internet in Europe isn't as good? Actually, I have heard that some primary schools in Europe lack internet. My grandson's elementary school in rural California (population <10k) had internet as far back as 1998.

They’re probably downloading from a server in the states, being much further away makes a big difference with a massive download.

This.

Re: We shrunk our Javascript monorepo git size

#216

Post by jonathancreamer »

eviks wrote: post #7

upd: silly mistake - file name does not include its full path The explanation probably got lost among all the gifs, but the last 16 chars here are different: > was actually only checking the last 16 characters of a filename > For example, if you changed repo/packages/foo/CHANGELOG.md, when git was getting ready to do the push, it was generating a diff against repo/packages/bar/CHANGELOG.md!

Sorry about the gifs. Haha. And yeah I guess my understanding wasn't quite right either reading the reply to this thread, I'll try to clean it up in the post.

Re: We shrunk our Javascript monorepo git size

#217

Post by [deleted] »

rurban wrote: post #213
js2 wrote: post #193

Earlier quoted context omitted.

> file name does not include its full path No, it is the full path that's considered. Look at the commit message on the first commit in the `--full-name-hash` PR: https://github.com/git-for-windows/git/pull/5157/commits/d5c... Excerpt: "/CHANGELOG.json" is 15 characters, and is created by the beachball [1] tool. Only the final character of the parent directory can differntiate different versions of this file, but als…

Sounds like it needs to be fixed to FNV1a

[deleted]

Re: We shrunk our Javascript monorepo git size

#218

Post by mb7733 »

jasonm23 wrote: post #200
mb7733 wrote: post #185

Earlier quoted context omitted.

VSCode is _their_ product. It doesn't make sense to say that they are EEEing their own product. EEE is when you take some existing open standard, support it in proprietary a product, and then extend it in proprietary ways, thereby taking over the standard. It doesn't apply for a product that you originally created.

.... Fork not an original creation that's how effective eee is, you don't know (or likely care) where MS ripped all this code. This is not an accident. It's the point.

Are you saying that VSCode is a fork of a non-Microsoft product? Which one?

Re: We shrunk our Javascript monorepo git size

#219

Post by 0points »

zelphirkalt wrote: post #72
0points wrote: post #57

Earlier quoted context omitted.

What european countries still dont have fiber? //Confused swede with 10G fiber all over the place. Writing from literally the countryside next to nowhere.

If you really need it pointed out, take it from a German neighbor: Telekom is running some extortion scheme or so here. Oh we could have gotten fiber to our house already ... if we paid them 800+ Euro! So we rather stick with our 100MBits or so connection that is not fiber but copper. If the German state does not intervene here, or the practices of ISPs and whoever has the power to build fiber changes, we will for th…

FWIW, Sweden subsidized fiber digging but we still had to pay 2000 EUR to get it connected.

Government will pay the extra fees, which can easily end up close to 10000 EUR due to large distances.

If all you need to pay is 800 EUR, then I don't understand what is your issue? Just pay it.

Post reply on HN