Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We need a new generation of source control

rookout.com

Post reply on HN
Print view View original

71–80 of 90 posts

Re: We need a new generation of source control

#71

Post by klodolph »

falsedan wrote: post #55
klodolph wrote: post #45

Earlier quoted context omitted.

That's not really a working copy, though. What some need is a tool that lets you check out a part of the repository as a working copy, without checking out the rest. By "part" we might mean more than one directory and its descendants (i.e. not a single root).

hmm, you could frankenstein together a bunch of trees to make it look like a partial checkout, but you couldn't make a new commit without all the parent tree objects up to the root. This sounds just like a subtree to be honest. Are you frequently checking out a subdir of a repo and committing changes to it? Is it config?

The particular case I'm thinking of involves a repository which is large enough that full checkouts are slow, so you do partial checkouts to make things faster.

Maybe at some point there will be tools that let you do this with Git, maybe built on top of something like Git VFS. At the moment it just kind of sucks. Subversion and Perforce both handle it just fine.

Re: We need a new generation of source control

#72

Post by intertextuality »

majewsky wrote: post #64
intertextuality wrote: post #6

Earlier quoted context omitted.

Your opinions should not be swayed by one article. Have some resolve in what you do. Git is great. It also has issues. Scaling has issues. Changing a tool won't solve scaling issues. Anecdotally I think submodules work just fine, although the git submodule tool is not intuitive. Then again, I work in a very small team on small projects compared to these mammoths being discussed with monorepos and the like.

> Your opinions should not be swayed by one article. Opinions should ideally not be swayed by the volume of text making arguments, but by the coherence and logic of those arguments.

Sure. But in order to see all sides of the argument and make an informed decision, one should read at least a few different sources, no?

Re: We need a new generation of source control

#73

Post by zamalek »

> Take no prisoners: mono-repos suck too

We are transitioning to multi-repos because we have been burned so hard by our mono-repo. Builds used to take 3 hours on the monster and we managed to get them down to 30 minutes, but we are truly at the bottom of the barrel. God help us if there is a build failure, every subsequent build fails while we scramble to identify the problem (and we can only sample success or failure every 30 minutes). It's a house of cards and it's horrible.

> Shots fired: multi-repos suck

We've already had debugging woes with this combined with internal package feeds (you have to pull down the code, build it, remove the package and replace it with the local code), which has made us very bearish on code re-use. That rigmarole sucks way less than mono-repos.

> You can’t have your cake and Git it too

Combine version control and package managers IMO. Go does one half of this. If you work under GOPATH with all your code, you can easily jump across repos to make changes and have those changes immediately propagate to the initial repo. Your hard-disk becomes the mono-repo. What Go doesn't do is pull binaries down from package feeds. There needs to be some simple mechanism to switch between builds and code.

Re: We need a new generation of source control

#74

Post by Rotareti »

apostacy wrote: post #67

This is such a pain point for me. I would LOVE something between subtrees and submodules. I have explored this many times, and if I had the ability to write something like this, I would. I would love it if I could have a child repo that did not require an external remote and could be bundled and stored within a parent repo, unlike a submodule. But I would also like it if it could be more decoupled from the history un…

> My dream is to be able to drop a repo inside another repo and have git just treat it as if it were part of the parent repo. And then to be able to bundle the child repo to the parent and push it.

I'm not sure if I understand you right, but I think I made what you describe: https://github.com/feluxe/gitsub

It's a simple wrapper around git, that allows nested git repositories, with almost no overhead.

I use it for a private library (the parent-repo), which itself contains modules (the child-repos) that I open sourced on github. It works fine for my use case. I wrote it, because I found "submodule" and "subtree" too complicated. 'gitsub' is still in alpha.

Re: We need a new generation of source control

#75

Post by WorldMaker »

sebazzz wrote: post #61
hliyan wrote: post #19

Are we mistaking a dependency control problem as a revision control problem? In a previous life, before microservices, CI/CD etc. existed, we did just fine with 20-30 CVS repositories, each representing a separate component (a running process) in a very large distributed system. The only difference was that we did not have to marshal a large number of 3rd party dependencies that were constantly undergoing version cha…

Monorepo are used because of internal dependencies, but there are already very good solutions for that. We have as an org a lot of projects (50+) but also split out common functionality (as it makes sense) into components which are shared between projects. How do we share those? In our case (.NET) we have an internal NuGet source which contains the components in question. Each project can upgrade to the later version…

The article also points to the issue of multiple repository management, and even includes links to three possible options to solve it.

The questions around "which repositories do I need?" and "how do I update all of them?" and "how do I make an atomic transaction [commit, branch, PR] across all of them?" are interesting questions in a multi-repo situation, but there are plenty of possible answers as well.

Some of them are just social in nature (read the README, watch/follow the whole GitHub organization, etc), so they aren't are as interesting technically as monorepo or "meta-repo" tools.

Re: We need a new generation of source control

#76

Post by boleary-gl »

headcanon wrote: post #21
kadendogthing wrote: post #4

As I've stated in another post on here, what's the point of these articles? It just says everything sucks, but doesn't really dive into why or how we could possibly fix any issues they may directly point out. Also it kind of sounds like the author really doesn't have any idea what GitLab is or does, so maybe he should check it out. But allow me to retort these bald assertions presented in the article: Monorepos are g…

One thing about GitLab tooling is that they have features that apply only on a per-repo basis, for example GitLab CI. Suppose for example we have 2 distinct projects - a backend and frontend, which each have their own testing and deployment strategy. GitLab CI only allows one CI pipeline config per-repository. While we could take care of that with scripting, that can easily get out of hand as we increas the number of…

GitLab Product Manager for CI here

Thank you for this feedback - it’s something we’re thinking about a lot too. We’ve made some improvements for monorepos (`changes:` keyword) and for micro-services/multi-repo (`trigger:` and `dependency:` keywords) but we’re not satisfied!

We have two open Epics - one for making CI lovable for monorepos (https://gitlab.com/groups/gitlab-org/-/epics/812) and one for making CI lovable for microservices (https://gitlab.com/groups/gitlab-org/-/epics/813). Would love community feedback on the direction those will take us and how we can up level lovability even more.

Re: We need a new generation of source control

#77

Post by WorldMaker »

mikece wrote: post #5

I like the idea of creating a source control protocol that can be implemented with any number of tools rather than having wars over particular implementations of source control products. (And would Git really have beaten Mercurial if GitHub had been HgHub instead? GitHub's success was more about process than the technology of Git, IMO.)

> And would Git really have beaten Mercurial if GitHub had been HgHub instead?

Bitbucket for Mercurial launched about the exact same time as GitHub.

Re: We need a new generation of source control

#78

Post by jsty »

arianvanp wrote: post #57
ljm wrote: post #32

Earlier quoted context omitted.

I agree with the first part of this. If by CI you mean something like Circle or Google Cloud Build or Travis, then your CI is pretty much limited to whatever you can fit in a YAML file, and what the CI service will support in that. YAML in and of itself is not the easiest thing to parse when you have multiple layers of nesting and a lot of lines. I don't really want to see what a CircleCI config would look like for N…

I'm honestly surprised that Google Cloud doesn't offer a "CloudBazel" product!

Something along those lines seems to be in the works:

https://blog.bazel.build/2018/10/05/remote-build-execution.h...

Re: We need a new generation of source control

#79

Post by jingwen »

jsty wrote: post #78
arianvanp wrote: post #57

Earlier quoted context omitted.

I'm honestly surprised that Google Cloud doesn't offer a "CloudBazel" product!

Something along those lines seems to be in the works: https://blog.bazel.build/2018/10/05/remote-build-execution.h...

More information on how to get access to Remote Build Execution for Bazel on GCP: https://docs.bazel.build/versions/master/remote-execution.ht...

(Disclaimer: I'm a engineer on the Bazel team)

Re: We need a new generation of source control

#80

Post by jsmith45 »

apostacy wrote: post #67

This is such a pain point for me. I would LOVE something between subtrees and submodules. I have explored this many times, and if I had the ability to write something like this, I would. I would love it if I could have a child repo that did not require an external remote and could be bundled and stored within a parent repo, unlike a submodule. But I would also like it if it could be more decoupled from the history un…

Well git can most certainly have multiple separate history trees within one repository that are all technically unreleated.

One could even use the existing submodule feature to reference unrelated history in the same repository, but the submodule tooling would want create a seperate .git folder, and duplicate everything, and would want a url to identify the repo, rather than knowing to use the parent repo.

It ought to be possible to modify git submodule to be able to specify that the submodule is actually just all the refs in namespace "blah" of the physically containing repo. Basically you would only need an expanded version of the .git "symlink file" feature that lets you specifify both "gitdir" and a ref namespace to use for all operations. Then poof, you would have self-contained submodules.

You would still have the problem that namespaced refs do not get cloned by default.

You also have some risk of pushing some refs of the parent that have module entries that reference objects only in unpushed refs in a ref namespace, meaning that if somebody cloned the repo, and tried to expand the self-contained submodules, it would discover that the commits are not present. I'd not be surprised if regular submodules also had that limitation. (I've never really used them in a manner where i might modify and commit in the submodule.)

Post reply on HN