Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We need a new generation of source control

rookout.com

Post reply on HN
Print view View original

51–60 of 90 posts

Re: We need a new generation of source control

#51

Post by organsnyder »

bluGill wrote: post #36
organsnyder wrote: post #29

Earlier quoted context omitted.

My preferred approach for a microservice architecture: - Contract-first API development - All API contract definition files (OpenAPI/Swagger, .proto, .wsdl...) in a single repo, which has a CICD pipeline to bundle them into artifacts for various platforms (Maven, Nuget, NPM, gem...) - Consumers and producers import the "api-contracts" dependency; this is the only coupling between components - Consumers and producers…

I agree with your last point in theory, but in practice consumers start to rely on bugs and implementation details, and eventually it is easier to change the contract that fix the clients.

Yeah, that is always a danger. From a purist standpoint, I'd argue that any behavior not defined explicitly in the API contract is subject to change at any time, and clients relying on it are by definition buggy. But I recognize that that often doesn't matter when the client code is owned by a team under a director with more clout than yours, a valuable customer, etc.

One possible solution would be to bump the major version (assuming semver) of the API contract, and support multiple versions of the API simultaneously. Of course, that has its own challenges and costs.

Re: We need a new generation of source control

#52

Post by falsedan »

klodolph wrote: post #41
falsedan wrote: post #39

Earlier quoted context omitted.

> it doesn't have access controls (and there's only so much you can add) so there's a lot of drawbacks to using gitolite but we were able to customise access controls down to allowing some users the ability to only change lines of checked-in config only to certain values

How do you prevent users from reading certain parts of the repository, though? This was what I meant by "there's only so much you can add"... you can reject pushes that change parts of the repo, but you can't prevent reads without breaking everything.

> can't prevent reads without breaking everything

I don't understand, you can lie to git-upload-pack and send anything you want to the user?

but when we used gitolite, we put sensitive stuff in a separate server and restricted reads to trusted users/deployment tools

edit oh I see, you want to let some people clone the repo but with some stuff redacted and still be able to make changes to the non-redacted stuff. I'd used LFS and move the ACLs to the file server, if using a single repo was a hard requirement

Re: We need a new generation of source control

#53

Post by organsnyder »

joshuamorton wrote: post #49
bluGill wrote: post #36

Earlier quoted context omitted.

I agree with your last point in theory, but in practice consumers start to rely on bugs and implementation details, and eventually it is easier to change the contract that fix the clients.

Hyrum's law and all. But I don't think that's what he was saying so much as your clients shouldn't depend on the server code , only the api definition. Which is true and possible in general.

That is what I was saying; but [s]he's right that implementation details always seep in by way of assumptions that clients make. It's extremely expensive to write an API definition that encompasses every possible edge case—probably only feasible in certain life/money-critical applications.

Re: We need a new generation of source control

#54

Post by jrockway »

1023bytes wrote: post #38

I think that Git submodules are to blame. I think that the idea is great, but the implementation is cumbersome. If they were better to use it would solve a lot of these problems.

I think it's irrelevant. If your build depends on inputs from two repositories, it's the same level of complexity as having the inputs all come from one repository. You might have two .git directories, but if the code is coupled, it's just a monorepo in two directories.

Ultimately the problem is in scaling the number of build inputs, not the number of .git directories.

Re: We need a new generation of source control

#55

Post by falsedan »

klodolph wrote: post #45
falsedan wrote: post #35

Earlier quoted context omitted.

git archive is ... ok, not pretty great, but it works

That's not really a working copy, though. What some need is a tool that lets you check out a part of the repository as a working copy, without checking out the rest. By "part" we might mean more than one directory and its descendants (i.e. not a single root).

hmm, you could frankenstein together a bunch of trees to make it look like a partial checkout, but you couldn't make a new commit without all the parent tree objects up to the root. This sounds just like a subtree to be honest.

Are you frequently checking out a subdir of a repo and committing changes to it? Is it config?

Re: We need a new generation of source control

#56

Post by weberc2 »

masklinn wrote: post #24
weberc2 wrote: post #8

Earlier quoted context omitted.

> And would Git really have beaten Mercurial if GitHub had been HgHub instead? GitHub's success was more about process than the technology of Git, IMO. Hg is a much better user experience than git, that's for sure. Git won because of Github, which may have beaten any HgHub simply because Git has an actual API while Mercurial's "API" is "use subprocessing". In other words, if Mercurial gave a damn about the developer…

> which may have beaten any HgHub simply because Git has an actual API Git doesn't though. A bunch of shell scripts calling shell scripts calling a few native binaries is pretty much "use subprocessing". libgit came much later, it wasn't part of the original git. However what git did provide was an open, stable, fairly simple and officially supported physical model with which you could easily interact directly, and p…

It looks like you're right about the history. According to the git repo, libgit's first commit was in October of 2008 while Github was incorporated in early 2008 (according to Wikipedia).

Github's popularity was probably due to Git's popularity in the Ruby community which may have been due to the official support of the physical model and simple protocols.

That said, an "officially supported physical model" is an API even if I originally had libgit in mind. Also, none of this invalidates the broader point, which is that Git won because of Github, not because of user experience.

Re: We need a new generation of source control

#57

Post by arianvanp »

ljm wrote: post #32
arianvanp wrote: post #22

The argument of why monorepos suck seems to largely rely on "CI Sucks" in this article. But I beg to differ. Monorepos only work in combination with a build system that tracks dependencies carefully. I contribute a lot to Nixpkgs, which is a monorepo with almost 50000 subcomponents [1], but because the build tool and CI track changes through hashes, changing a package only triggers rebuilds of other packages that dep…

I agree with the first part of this. If by CI you mean something like Circle or Google Cloud Build or Travis, then your CI is pretty much limited to whatever you can fit in a YAML file, and what the CI service will support in that. YAML in and of itself is not the easiest thing to parse when you have multiple layers of nesting and a lot of lines. I don't really want to see what a CircleCI config would look like for N…

I'm honestly surprised that Google Cloud doesn't offer a "CloudBazel" product!

Re: We need a new generation of source control

#58

Post by niftich »

The field does suffer a bit from version control, dependency management, language compilers, and build and packaging tools all being single-purpose tools that are layered, where one can't introspect others beyond the public API, and manual effort or simplistic not-always-true assumptions have to be used to bridge information from one to the other.

It's tempting to imagine an integrated system where making changes to a piece of source code automatically commits every change, every commit will attempt to compile and build, every successful build auto-packages into a new artifact with a new build version. The language and the build system would ensure that all builds are reproducible. Because of this, all builds can be addressed by identity (content hash) too, not just a name and a build number within some namespace.

When any dependency of the current project has newer builds, one could choose to pull up an interactive diff experience to step through the code of newer versions. This would aid in selecting a different version on which to depend, if desired. If a different version of a dependency is picked up, a new build gets triggered too, and a successful build gets a new build version.

The strong linkage between source code revision and build version, the deterministic builds, and content-based artifact addressing work together to ease the traceability of changes and the reusability of artifacts, and sidesteps concerns about the hosting and namespacing of source code and build artifacts interfering with the project's "single source of truth", because any copy of an artifact, known by any name, irrelevant of its location, will share the same hash.

There will still be usability problems with such a system too. There would be no way to strip data out. A shelve, replay, and cherry-pick frontend would be necessary to allow the doctoring of input before it's committed permanently -- but in such a system, only permanently committed code can be built. The workflow to prepare a project for public consumption would be to author and test all the changes in a 'scratch' project that doesn't auto-disseminate its build artifacts elsewhere, and cherry-pick the changes into a public project. Public projects could only have public dependencies.

Configuration files, data files, and pieces making up a larger environment may need a different approach. Nonetheless, a lot of these problems take the same shape: some input should deterministically produce some output, and a running system may choose to alter its own state by interfacing with a stateful outside world (e.g. load or write files, communicate through a network). The sensible places of drawing a boundary between the inside world and outside world will differ for every use-case.

Re: We need a new generation of source control

#59

Post by ZeroCool2u »

khazhou wrote: post #48
netheril96 wrote: post #33

Google uses a monorepo for most of its code and I find it a much better experience than what I’ve had in the past. But that good experience is predicated on a lot of Google only internal tools. If Google open sources enough of those such that people outside can have the same experience, maybe the debate will end decisively in favor of monorepo.

What kinds of tools?

Parent comment is likely referencing bazel[1], which is the external version name, among a number of other tools.

[1]: https://bazel.build/

Re: We need a new generation of source control

#60

Post by zaphar »

hliyan wrote: post #19

Are we mistaking a dependency control problem as a revision control problem? In a previous life, before microservices, CI/CD etc. existed, we did just fine with 20-30 CVS repositories, each representing a separate component (a running process) in a very large distributed system. The only difference was that we did not have to marshal a large number of 3rd party dependencies that were constantly undergoing version cha…

Most of the mono-repo advocates aren't talking about dependency management. They are talking about the advantages around continuous integration that a good mono-repo tool can bring.

The article actually complains less about mono-repos and more about mono-repos on Git and the associated tooling around Git.

Post reply on HN