Earlier quoted context omitted.
Yeah I'm surprised by that as well. As far as I remember, Rosie started out in 2010 and people were using in 2012. Maybe the clustering/splitting didn't support this use-case or it wasn't well-known enough?
From my memory, extremely large but semantically simple changes still used global approvers since it wasn't considered worth the effort to get approval from hundreds or thousands of individual OWNERs using rosie. Also, with a change of this magnitude, especially one affecting BUILD files, it might not have been possible to create the mega-CL that rosie uses, but I don't remember specific limitations on that.
Reformatting 100k Files at Google in 2011
101–110 of 162 posts
Re: Reformatting 100k Files at Google in 2011
#102Earlier quoted context omitted.
The thing about configurable vs non-configurable in this case is that when its configurable then people will spend time debating how exactly they should configure it.
Yes, that's why I love gofmt. There's nothing to debate!
I actually like how gofumpt formats stuff but ... nobody else on the team would have it, so it would make things worse.
Re: Reformatting 100k Files at Google in 2011
#103Earlier quoted context omitted.
Yes, that's why I love gofmt. There's nothing to debate!
Helpfully, you can debate between "gofmt", "gofmt -s" and "gofumpt". I actually like how gofumpt formats stuff but ... nobody else on the team would have it, so it would make things worse.
Re: Reformatting 100k Files at Google in 2011
#104Earlier quoted context omitted.
> It really is dumb to be arguing over tabs vs spaces, after all. In an in-house dialect of Haskell I used to work with, we solved this problem by just making tabs a syntax error. Never had any problems. (I think tabs might be have been allowed inside strings.)
Arguably it would be a problem for a coworker that wanted to use tabs. “We’ll just force everyone to do things one way” kind of ignores the point I was making. It shouldn’t be necessary for you to care how anyone else formats their code, same as you don’t care what font their code displays in or what text editor they use. It feels like a vestigial aspect of programming that we have to concern ourselves with it in 202…
Wasn't a problem in practice. Just like we never had any problems with anyone wanting to use eg Pascal or so.
Re: Reformatting 100k Files at Google in 2011
#105Autoformatting is so nice. Crazy to think that formatters only became popular after `gofmt`. I also found this related quote from Russ Cox intriguing: "Most people think that we format Go code with gofmt to make code look nicer or to end debates among team members about program layout. But the most important reason for gofmt is that if an algorithm defines how Go source code is formatted, then programs, like goimport…
I’ve wondered before whether the world would be well served by a programming language (or source control system, I suppose) that just stores ASTs in files rather than text code. When users open the file the editor formats to whatever their personal preference is, then saves edits back to the AST. It really is dumb to be arguing over tabs vs spaces, after all.
Blank lines is an obvious one: where do you insert them to "group" sections of a 30 line function? Or are there no blank lines at all?
Line length: just "wrap at columns X" (or never wrap) is not enough, because people can and do wrap at specific locations for specific reasons, because that makes more semantic sense or looks nicer than cramming as much as possible.
Re: Reformatting 100k Files at Google in 2011
#106Earlier quoted context omitted.
> It really is dumb to be arguing over tabs vs spaces, after all. In an in-house dialect of Haskell I used to work with, we solved this problem by just making tabs a syntax error. Never had any problems. (I think tabs might be have been allowed inside strings.)
Simple, elegant, and wrong.
In a language like C your outermost layers of leading whitespace are always indentation, and then you might have some alignment inside.
But in Haskell you might want to align arguments to a function, but some of the arguments can have blocks inside of them.
Mixing up tabs and spaces is technically possible, but it's too much of a pain in practice to bother.
Re: Reformatting 100k Files at Google in 2011
#107Earlier quoted context omitted.
I was one of the global approvers (and also on the Python team until my role at Google was eliminated recently). I no longer have access to the stats, but from my memory, there are currently 50+ global approvers depending on how many are still considered active. Typically, code authors would create a proposal by filling out a doc template. It's usually light weight and also accompanied with examples or full set of th…
Just curious, what kind of global changes do you usually make? And what is the process of becoming an approver?
New approvers are nominated by an existing member and then LGTM'ed by other three. Usually they have gained a lot of large scale change experiences on the other side, and we recognize that we could use more help on the committee side. Especially we want a good coverage on various languages, tech stacks, and time zones.
Re: Reformatting 100k Files at Google in 2011
#108I'm going to take a contrarian view here. Code formatting is amazing in a corporate environment where nobody truly cares about their code -- it's just a means to get a paycheck. It's also great for beginners to a language who are still trying to get a handle of the syntax. But where you are nearly the sole owner of a small library and you are crafting that library to be beautiful and understandable... there is someth…
I think you're doing yourself a disservice by having to play with the formatting itself instead of being able to express what you mean with the tools the language already provides you. As soon as anyone else looks at your code or wants to participate in the development process, what is meaningful to you about the arrangement simply won't translate into their view; what is the use of a programming language if not to u…
Re: Reformatting 100k Files at Google in 2011
#109Earlier quoted context omitted.
It meant to illustrate the same human weakness in consistency and repeatability disadvantages both manual testing and manual formatting. In essence manually formatting is a baker making every bread by hand. Some bread will be better then others. In contrast to an automated factory turning out the same quality bread every day.
Ah, makes sense. I think there are use-cases for both, even personal hobby projects. There's some languages with so much damn syntax that I don't think I could even begin to write code without a formatter (I'm looking at you rust and Java) and others where it stays out of your way (python, Lua, etc)
Re: Reformatting 100k Files at Google in 2011
#110But back to the point... formatting rules without firm, incredibly strict enforcement ends up being a tax on the janitors - the people who clean the code base and do large scale changes. That makes me sad. These are the people who care a great deal about code health, and their work is hindered by the lint checks that we have imposed.
Let me give an example.
I'm trying to eliminate a constraint in the build system. It's a "small" large change - only O(30K) instances. (Yeah, Google scale is different). I have an incredible wealth of tools available to me to automate the process. For the benefit of the Googlers, I can identify Blaze targets to change, use buildozer to fix them, and ship off CLs to review. But the changes I want to make are often ones which should be reviewed by the code owners, and not globally approved. So possibly O(10K) individuals might be involved in reviews.
Let's explore the problem. First, shouts to y2mango for bringing up incremental formatters. This should be the default for all tools. And another to flymasterv for raising the question of "why not just format as each person touches a BUILD file". Here's the situation.
1. buildozer is really good at rewriting BUILD files syntactically correctly. 2. It has an unfortunate side effect of not being incremental. It calls buildifier to rewrite the entire file. 3. We update the formatting rules to make them stricter over time. That means that a "correct" BUILD file on January 1, might require changes on March 1. 4. Buildifier findings are advisory, rather than mandatory. 5. No team is staffed with repeating the monumental work this post started with.
The reality on the ground is that little touched BUILD files become stale, and would require a formatting update over time. It is actually worse than that, because many teams take the path of ignoring buildifier warnings and committing their working code anyway. Without continual BUILD file reformatting there is a lot of stale floating around. [Root cause: We could fix this by promoting people for doing that repeat work. But we don't. We promote for the initial sprint.]
And then a janitor comes along.
I use bulldozer to fix a problem. It reformats an ancient BUILD file completely (not incremental). I send it to the code owner. They see changes far beyond my 2 line fix. They reject it, or ask for a change to only the two lines that actually mattered. Sure. I can hand build the change once or twice. But not for a few hundred, or thousands files. So.... I have to hack up an incremental format. Or, it turns out that users are very happy if I don't bother with formatting at all, and just change single lines. It's not that any individual is right or wrong. It is that they all have a choice and a preference and Google created a policy that allowed individual teams to have a choice of strict compliance or not. That is the failure.
If you are going to have a policy about code formatting: - make it hard mandatory for everything except a "break glass" situation - if the policy can evolve, staff a team with enforcing it globally
The fact that Google, as a company, does not reward this behavior does not take away from any individual's accomplishments. This post may sound grumpy to an outsider, but I am constantly amazed at the tools I have available to fix things on an enormous scale. The friction is usually only where we have good intentions, without the policy teeth to enforce alignment with the intentions. That's a management problem, not a technical one.