Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Reformatting 100k Files at Google in 2011

laurent.le-brun.eu

Post reply on HN
Print view View original

51–60 of 162 posts

Re: Reformatting 100k Files at Google in 2011

#51

Post by plorkyeran »

rowanG077 wrote: post #38
wonger_ wrote: post #2

Autoformatting is so nice. Crazy to think that formatters only became popular after `gofmt`. I also found this related quote from Russ Cox intriguing: "Most people think that we format Go code with gofmt to make code look nicer or to end debates among team members about program layout. But the most important reason for gofmt is that if an algorithm defines how Go source code is formatted, then programs, like goimport…

I never thought it only got popular with go. I remember using formatters before go. What go did do is that it shows how good it is if it is ubiquitous.

I think mandatory code formatting was popularized by Go? Plenty of code formatting tools had been around for quite a while by the time Go came out, but they were the sort of thing you ran on a codebase once to clean up a mess, not something you checked on each commit. A common sentiment was that it was a good idea but was only viable because Go had it from the beginning, which fortunately turned out to be wrong.

Re: Reformatting 100k Files at Google in 2011

#52

Post by y2mango »

bsmith0 wrote: post #32

I'd be curious what role "global approvers" at Google/Google's scale typically have/how many are there/what's the process?

I was one of the global approvers (and also on the Python team until my role at Google was eliminated recently). I no longer have access to the stats, but from my memory, there are currently 50+ global approvers depending on how many are still considered active.

Typically, code authors would create a proposal by filling out a doc template. It's usually light weight and also accompanied with examples or full set of the pending code changes. Then 1-3 of us will review and LGTM the proposal. As part of the review, we also determine whether the changes should be sent to local code owners, or "globally approved" by one of us. The default option is to use "global approval", unless the changes need local code owner's knowledge during the code review. Said in another way, when sent to local code owners, their role is not gate keeping the changes, but to provide necessary local knowledge where we as global approvers don't have.

Refactoring changes, such as formatting or API migrations, shouldn't bother local code owners because 1) it would just be a waste of their time to review and approve; 2) in practice, we find a central code reviewer for the same large set of code changes is more likely to catch bugs (with review automation tooling) than local reviewers.

We consider ourselves as facilitators rather than approvers or gatekeepers of the code changes. Our goal is to make these changes done more efficiently and save engineering time when possible.

If you like stats: over the past 5 years, I have reviewed ~300 such proposals and ~40K changelists (equivalent to PRs). One changelist/PR typically contains 10s to 100s of files depending on the nature of the change. When I was most active, I was about ~5th-ish when ranking the number of changes we were approving. There are many global approvers who have approved more than 100K changelists, which is a milestone we celebrate with a cake. Too bad I didn't have the chance to have my cake.

Re: Reformatting 100k Files at Google in 2011

#53

Post by y2mango »

flymasterv wrote: post #12

I don’t understand why they had to format 100k files. You enforce the format with a presubmit and let the code get formatted in the next change. I have long felt that Google’s strength has always been making a bad architectural choice and then executing on it flawlessly. So many systems are designed in ways that require incredible technical execution to make them workable, and they do it.

Foremost: formatting these BUILD files was the correct decision as rsc already explained.

Then, there are also other formatters that support "incremental formatting", meaning it only formats lines that are changed in your commit.

Disclaimer: I authored https://github.com/google/pyink and replaced Google Python's YAPF formatter with this Black fork and also implemented the "incremental formatting" feature in Pyink and upstreamed to Black.

When we were rolling out the formatter change, we chose to NOT format the Python files mainly because 1) not all teams at Google enforce Python formatting at presubmit time; 2) the formatter supports "incremental formatting" to minimize the diffs introduced by the formatter.

There are of course less ideal cases where even incremental formatting has to touch not-changed-lines, such as a large Python dictionary/list/set literal that spans across dozens or even hundreds of lines. It's a tradeoff in the end.

Re: Reformatting 100k Files at Google in 2011

#55

Post by coolcoder613 »

afavour wrote: post #23
wonger_ wrote: post #2

Autoformatting is so nice. Crazy to think that formatters only became popular after `gofmt`. I also found this related quote from Russ Cox intriguing: "Most people think that we format Go code with gofmt to make code look nicer or to end debates among team members about program layout. But the most important reason for gofmt is that if an algorithm defines how Go source code is formatted, then programs, like goimport…

I’ve wondered before whether the world would be well served by a programming language (or source control system, I suppose) that just stores ASTs in files rather than text code. When users open the file the editor formats to whatever their personal preference is, then saves edits back to the AST. It really is dumb to be arguing over tabs vs spaces, after all.

Many BASIC dialects did this.

Re: Reformatting 100k Files at Google in 2011

#57

Post by laurentlb »

troad wrote: post #22

The use of light grey text on a darker grey background strains my eyes and makes this unnecessarily unpleasant to read. I'd respectfully suggest increasing the contrast dramatically. I keep a quick little scriptlet in my bookmark bar for cases like this: javascript:(function(){ $('head').append(' *{color:#101010 !important; background:#f0f0f0 !important;} '); }()); (A ten second hack job; suggested improvements from…

Thanks, I've increased the contrast (not as much as your suggestion, but I hope it's better).

Re: Reformatting 100k Files at Google in 2011

#58

Post by laurentlb »

rsc wrote: post #19

My notes say it was 193k at the start. The final dashboard when we stopped said "216,626 / 216,890 = 99.8%; 264 to go". The other correction I would make is that this post does not mention Nilton Volpato, who had written an earlier Buildifier and graciously accepted replacing his implementation with a new one and then taking over ownership for that new implementation as well. (Eventually ownership moved to Laurent's…

Thanks for the precisions! I've added some updates at the bottom of the post.

Re: Reformatting 100k Files at Google in 2011

#59

Post by randomifcpfan »

lopkeny12ko wrote: post #56
yegle wrote: post #54

There are 3 tools that that makes maintaining BUILD files enjoyable: buildifier, buildozer and build_cleaner (internal only unfortunately).

What is build_cleaner?

A tool for updating bazel build target dependencies. It inspects build files and source code, then adds/removes dependencies from build targets as needed. It requires using global include paths in C/C++ sources. It is not perfect, but it is pretty nice!

Re: Reformatting 100k Files at Google in 2011

#60

Post by paulddraper »

vitiral wrote: post #7

I'm going to take a contrarian view here. Code formatting is amazing in a corporate environment where nobody truly cares about their code -- it's just a means to get a paycheck. It's also great for beginners to a language who are still trying to get a handle of the syntax. But where you are nearly the sole owner of a small library and you are crafting that library to be beautiful and understandable... there is someth…

I don't find the mental burden of formatting code to have a positive ROI.

I want to write syntacially-valid code, without worrying about the visual presentation of it. (I want a good presentation, but I don't want to put forth the effort to create it.)

Post reply on HN