We need a new generation of source control • Hacker News Board

Re: We need a new generation of source control

#61

Post by sebazzz » Wed, Apr 24, 2019, 4:56 PM UTC

hliyan wrote: post #19

Are we mistaking a dependency control problem as a revision control problem? In a previous life, before microservices, CI/CD etc. existed, we did just fine with 20-30 CVS repositories, each representing a separate component (a running process) in a very large distributed system. The only difference was that we did not have to marshal a large number of 3rd party dependencies that were constantly undergoing version cha…

Monorepo are used because of internal dependencies, but there are already very good solutions for that. We have as an org a lot of projects (50+) but also split out common functionality (as it makes sense) into components which are shared between projects. How do we share those? In our case (.NET) we have an internal NuGet source which contains the components in question. Each project can upgrade to the later version of an component at its own schedule, just like 3rd party dependencies are updated when necessary.

It does not have to be complicated.

Re: We need a new generation of source control

#62

Post by majewsky » Wed, Apr 24, 2019, 5:19 PM UTC

> Each side of this debate classifies the other as zealous extremists (as only developers can!), but both of them miss the crux of the matter.

I take it the author has never had an interest in politics.

Re: We need a new generation of source control

#63

Post by qznc » Wed, Apr 24, 2019, 5:19 PM UTC

arianvanp wrote: post #22

The argument of why monorepos suck seems to largely rely on "CI Sucks" in this article. But I beg to differ. Monorepos only work in combination with a build system that tracks dependencies carefully. I contribute a lot to Nixpkgs, which is a monorepo with almost 50000 subcomponents [1], but because the build tool and CI track changes through hashes, changing a package only triggers rebuilds of other packages that dep…

Subversion.

Also proprietary systems like PlasticSCM.

Re: We need a new generation of source control

#64

Post by majewsky » Wed, Apr 24, 2019, 5:25 PM UTC

intertextuality wrote: post #6

booleandilemma wrote: post #2

I’m just now getting comfortable with Git, please don’t do this to me.

Your opinions should not be swayed by one article. Have some resolve in what you do. Git is great. It also has issues. Scaling has issues. Changing a tool won't solve scaling issues. Anecdotally I think submodules work just fine, although the git submodule tool is not intuitive. Then again, I work in a very small team on small projects compared to these mammoths being discussed with monorepos and the like.

> Your opinions should not be swayed by one article.

Opinions should ideally not be swayed by the volume of text making arguments, but by the coherence and logic of those arguments.

Re: We need a new generation of source control

#65

Post by majewsky » Wed, Apr 24, 2019, 5:26 PM UTC

joshjb17 wrote: post #23

The answer to your problem is Nix: https://nixos.org/nix/

Fun fact: "Nix" is a colloquial variant of the German word for "nothing". Whenever we talk about NixOS at our hackerspace, hilariousness ensues.

Re: We need a new generation of source control

#66

Post by majewsky » Wed, Apr 24, 2019, 5:29 PM UTC

Tempest1981 wrote: post #46

hashkb wrote: post #10

Earlier quoted context omitted.

> There are good technical reasons why you would use Perforce or even Subversion days. Can you say more? What are some of those reasons? Or link to some data or examples?

With a monorepo, how do you avoid the situation where almost every time you want to commit, you have to pull-and-rebase first? Because somebody has always pushed a change, every minute or two.

In a repo that large, you don't want to have random people pushing to master anyway. Have people commit to branches, and then automation merges the approved branches into master. ("Automation" may be as simple as the "Merge" button in Github's UI, or more complex if necessary.)

Re: We need a new generation of source control

#67

Post by apostacy » Wed, Apr 24, 2019, 5:32 PM UTC

This is such a pain point for me.

I would LOVE something between subtrees and submodules.

I have explored this many times, and if I had the ability to write something like this, I would.

I would love it if I could have a child repo that did not require an external remote and could be bundled and stored within a parent repo, unlike a submodule. But I would also like it if it could be more decoupled from the history unlike a subtree.

I can get most of what I want from submodules and subtrees, but not really enough.

It might be possible without even having to change git. Perhaps if there were a way to have branch namespaces of some kind, and I could have a subtree have completely separate history, but have it checked out within the same working tree. Many of my projects that are submodules only make sense within their parent repo, and it is really redundant to have an external repo for them. But I also don't like to have to do expensive surgery to deal with subtrees, and it would be nice to not have it be completely merged.

My dream is to be able to drop a repo inside another repo and have git just treat it as if it were part of the parent repo. And then to be able to bundle the child repo to the parent and push it.

I know that it is mostly possible to do this already, but it is not easy or intuitive.

Re: We need a new generation of source control

#68

Post by grillermo » Wed, Apr 24, 2019, 5:40 PM UTC

khazhou wrote: post #48

netheril96 wrote: post #33

Google uses a monorepo for most of its code and I find it a much better experience than what I’ve had in the past. But that good experience is predicated on a lot of Google only internal tools. If Google open sources enough of those such that people outside can have the same experience, maybe the debate will end decisively in favor of monorepo.

What kinds of tools?

You can read about all the tools in this article https://cacm.acm.org/magazines/2016/7/204032-why-google-stor...

Re: We need a new generation of source control

#69

Post by lucozade » Wed, Apr 24, 2019, 5:44 PM UTC

I know it's wrong of me. Genuinely, I know. But when your second paragraph states that Google and Netflix pioneered horizontally scalable processes...

It makes it so hard to read the remainder untainted by a certain amount of scepticism.

Fortunately, he's not actually saying anything much in the article so I don't think my irrational reaction to ignorance will mean I've missed something important. But still...

Re: We need a new generation of source control

#70

Post by klodolph » Wed, Apr 24, 2019, 5:51 PM UTC

falsedan wrote: post #52

klodolph wrote: post #41

Earlier quoted context omitted.

How do you prevent users from reading certain parts of the repository, though? This was what I meant by "there's only so much you can add"... you can reject pushes that change parts of the repo, but you can't prevent reads without breaking everything.

> can't prevent reads without breaking everything I don't understand, you can lie to git-upload-pack and send anything you want to the user? but when we used gitolite, we put sensitive stuff in a separate server and restricted reads to trusted users/deployment tools edit oh I see, you want to let some people clone the repo but with some stuff redacted and still be able to make changes to the non-redacted stuff. I'd u…

> I'd used LFS and move the ACLs to the file server, if using a single repo was a hard requirement

If you're putting a few large files in LFS, or maybe a couple sensitive files, I can understand and I'd say you're still using Git, but with some extensions.

If you're putting an entire sensitive subtree in LFS, I don't think you're really using Git any more, in the sense that many of your standard Git workflows will have to be different.