Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Monorepo: please do

medium.com

Post reply on HN
Print view View original

81–90 of 166 posts

Re: Monorepo: please do

#81

Post by jacques_chester »

mlthoughts2018 wrote: post #54
woolvalley wrote: post #32

My org went from polyrepo 10 commit semver dependency hell when updating an internal API to monorepo and it saves a lot of time. Unmigrated semver breaking changes are a form of technical debt, and it takes a lot more total man hours to do the 'proper' one by one many commit poly repo migration than the other way around. If we had the tooling to do multirepo atomic commits and reviews then maybe we would of stuck wit…

My org went from a monorepo where every project had to obey the same CI model and you could not introduce entirely new CI tools for new prototypes over to a polyrepo with separated semver library repos for shared dependencies, and it simplified everything so much. Adding additional PRs across different repos is functionally no different than the same PR with scattered dependencies in a monorepo, except that separatin…

> except that separating the PRs makes each isolated set of changes more atomic and focused

It makes it less atomic if you need simultaneous changes in multiple repositories.

> Had we been able to set this up as a separate repo where there were no global rules over how all compute jobs must be organized, and used our own choice of deployment (containers) with no concern over whatever other projects were using / doing, we could have solved it in a matter of a few days.

I think this was an organisational problem, but I accept the argument that monorepos will provide a seed around which such pathologies can crystallise. But I don't believe it's the only such seed and I don't think it's an inevitable outcome from monorepos.

Re: Monorepo: please do

#82

Post by jacques_chester »

iainmerrick wrote: post #76
jacques_chester wrote: post #72

Earlier quoted context omitted.

I want to like Bazel. I really do. But on first encounter the syntax is filled with sigils that don't seem to have obvious differences or purpose for existence. Then it turns out that I and others have spent as much time fighting it as using it. Lastly the coverage of ecosystems is sparse and there does not seem to be a lot of activity around extending them -- doing the boring, tedious, unloved work of dealing with e…

I can’t speak to the actual implementation, but I’m surprised at your description of the syntax as “filled with sigils”, as the syntax is basically Python -- isn’t that about as easy as you can get? I find Bazel’s syntax much easier to deal with than other build languages that use JSON (essentially the same Python syntax but with lots of extra quotes everywhere and extra fussiness about where commas are allowed).

    bazel build //main:hello-world
I'm sure the double slashes and colon have important differences. It is not obvious what they are.

    cc_binary(
      name = "hello-world",
      srcs = ["hello-world.cc"],
      deps = [
        ":hello-greet",
        "//lib:hello-time",
      ],
    )
It's not instantly obvious why one is :hello-greet and the other is //lib:hello-time.

I could swear I've seen @ floating around as well.

As I said above, I am sure these are all very sensible. But I am just tired of memorising minilanguages embedded in strings. I don't want to any more.

Re: Monorepo: please do

#83

Post by joshuamorton »

mlthoughts2018 wrote: post #77
joshuamorton wrote: post #74

Earlier quoted context omitted.

>If a tool begets using it wrong all the time This is the (wrong) assumption. Like I said, there's nothing about a monorepo that "begets" draconian policy. Your anecdotal experience is not a rule. The monorepo I work with doesn't have draconian policies about how tooling must work. There are apis and recommended tools, and if those don't fit your needs (which is unusual), the teams that maintain those tools are willi…

There absolutely is something about monorepos that begets monopolicies: exactly the very thing that makes them co-occur. It doesn’t matter if it’s sociological or technological, the co-occurrence itself is the thing. I’d flip it around and say instead that you are assuming the properties by which to compare the two approaches ought to be properties that are roughly like “first principles” and that no first principles…

>There absolutely is something about monorepos that begets monopolicies

Correlation (and a weak one at that) is not causation.

I can just as easily suggest that monopolicies beget monorepos, and that indeed makes a lot more sense. Its easier to enforce global standards when there's a single repo. So companies who wish to enforce draconian standards may move in that direction. That says nothing about companies that don't wish to enforce draconian standards though.

Re: Monorepo: please do

#84

Post by jessaustin »

skj wrote: post #14
benmarten wrote: post #4

Earlier quoted context omitted.

It's not, why do I have to checkout terabyte of code that I don't need, even if the code is modularized?

Sounds like a tooling problem. We shouldn't use the current state of tooling as an excuse.

Isn't the entire argument about the current (or maybe "immediately foreseeable") state of tooling? We don't really care one way or the other, in a philosophical sense. What works?

Re: Monorepo: please do

#85

Post by malkia »

jacques_chester wrote: post #82
iainmerrick wrote: post #76

Earlier quoted context omitted.

I can’t speak to the actual implementation, but I’m surprised at your description of the syntax as “filled with sigils”, as the syntax is basically Python -- isn’t that about as easy as you can get? I find Bazel’s syntax much easier to deal with than other build languages that use JSON (essentially the same Python syntax but with lots of extra quotes everywhere and extra fussiness about where commas are allowed).

bazel build //main:hello-world I'm sure the double slashes and colon have important differences. It is not obvious what they are. cc_binary( name = "hello-world", srcs = ["hello-world.cc"], deps = [ ":hello-greet", "//lib:hello-time", ], ) It's not instantly obvious why one is :hello-greet and the other is //lib:hello-time. I could swear I've seen @ floating around as well. As I said above, I am sure these are all ve…

Piper, google's source control system has roots in Perforce. In perforce, depot roots are starting with //

The ":" is a bit different, e.g. just "//lib" means "//lib:lib" - e.g. points to the "lib" target in /lib/BUILD file, while "//lib:hello-time" points to "hello-time" target in /lib/BUILD file. So not having the ":name" in "//dir:name" means name="dir" - e.g. "//dir:dir" - at first this is strange, but then you get used to it. Your default target is named after the folder it's sitting in.

Re: Monorepo: please do

#86

Post by jkaplowitz »

jacques_chester wrote: post #82
iainmerrick wrote: post #76

Earlier quoted context omitted.

I can’t speak to the actual implementation, but I’m surprised at your description of the syntax as “filled with sigils”, as the syntax is basically Python -- isn’t that about as easy as you can get? I find Bazel’s syntax much easier to deal with than other build languages that use JSON (essentially the same Python syntax but with lots of extra quotes everywhere and extra fussiness about where commas are allowed).

bazel build //main:hello-world I'm sure the double slashes and colon have important differences. It is not obvious what they are. cc_binary( name = "hello-world", srcs = ["hello-world.cc"], deps = [ ":hello-greet", "//lib:hello-time", ], ) It's not instantly obvious why one is :hello-greet and the other is //lib:hello-time. I could swear I've seen @ floating around as well. As I said above, I am sure these are all ve…

Completely valid concern not to want to keep memorizing mini-languages.

In this case, the double slashes are absolute "paths" relative to the top of the workspace, and the part after the colon is a relative "path" to another Bazel target.

I put "paths" in quotes because these are meaningfully different from the true filesystem equivalents; avoiding confusion with real absolute and relative filesystem paths is probably why they made their own syntactic mini-language.

[The sibling reply to mine, referencing Piper and Perforce, goes into a bit more detail on the specifics and the origin of the // prefix.]

What would the better way have been for them to do this?

Re: Monorepo: please do

#87

Post by malkia »

rhacker wrote: post #36
benmarten wrote: post #4

Earlier quoted context omitted.

It's not, why do I have to checkout terabyte of code that I don't need, even if the code is modularized?

It's not for everyone, but damn, why is there a TERABYTE of code? Just curious - assets? checking in binaries?

Test protos. Evaluated configs. Golden data. JAR archives, etc.

Re: Monorepo: please do

#88

Post by mlthoughts2018 »

jacques_chester wrote: post #81
mlthoughts2018 wrote: post #54

Earlier quoted context omitted.

My org went from a monorepo where every project had to obey the same CI model and you could not introduce entirely new CI tools for new prototypes over to a polyrepo with separated semver library repos for shared dependencies, and it simplified everything so much. Adding additional PRs across different repos is functionally no different than the same PR with scattered dependencies in a monorepo, except that separatin…

> except that separating the PRs makes each isolated set of changes more atomic and focused It makes it less atomic if you need simultaneous changes in multiple repositories. > Had we been able to set this up as a separate repo where there were no global rules over how all compute jobs must be organized, and used our own choice of deployment (containers) with no concern over whatever other projects were using / doing…

> It makes it less atomic if you need simultaneous changes in multiple repositories.

No, each individual set of changes is more atomic (smaller in scope, mutating a system from one state of functionality to a new state of functionality).

The problem is that it’s a linguistic fallacy to act like in the monorepo case “the system” is the sum of a bunch of separate systems (it isn’t, because they are not logically required to depend on simultaneously transitioning). So in that monorepo case, to move subcomponent A from some state of functionality to a new state of functionality, you unfortunately have to also make sure you include totally unrelated (from subcomponent A’s point of view) changes that also correctly transition subcomponent B to a new state of functionality, and subcomponent C, etc., which is exactly less atomic (to transition states, you are required to have simultaneous other transitions that are not logically required for any reason other than the superficial sake of the monorepo).

Re: Monorepo: please do

#89

Post by pbalau »

hocuspocus wrote: post #65
pbalau wrote: post #55

Earlier quoted context omitted.

> With 500 it becomes nearly impossible to do anything involving a build. Both FB and Google have more than 500 devs and are using a monorepo.

At what cost? Both FB and Google employ hundreds of devs to work on internal tooling only. For most companies this isn't feasible.

Afaik, fb employs less than 20 people to take care of the monorepo.

Re: Monorepo: please do

#90

Post by jessaustin »

Benjammer wrote: post #31
shados wrote: post #18

Earlier quoted context omitted.

> Fighting back against monorepo design is dangerous What's dangerous about it? Monorepos have a lot of benefits, and should absolutely be considered. Maybe even by most. But right now in the community it's almost pushed as the "only true way with all benefits and no drawbacks", and that's absolutely not true. To the point the knowledge of why and how to poly repo is already starting to get lost. That's dangerous.

The real danger here is anyone talking about any system architectures or tooling as "dangerous" (or "not dangerous") absent any other context... What do you even mean by "dangerous"? To a business? To your health? What is the deal with people trying to make these sorts of global assertions in a vacuum about what's "good" and "bad"? This doesn't make any engineering sense in any way to me. You have a problem and you f…

This complaint belongs one level higher in the thread.
Post reply on HN