Live data from Hacker News

The story of Google Guava and patches

plus.google.com

41–50 of 50 posts

Re: The story of Google Guava and patches

#41
post #26
post #14

Am I the only one who is really annoyed by links to Google+ that can only be seen after signing in? I thought it was considered bad style to do that for NYT links here. Maybe the same holds true for G+ links?

This post should be publicly visible without being logged in (at least it is for me). But this is the second HN thread in a week where I've seen a comment like this, so can you send me your details (web browser, etc), so I can debug, please?

Asked to sign in on iphone4s. Quite annoying and uncalled for.

Re: The story of Google Guava and patches

#42
Branching.

A simple feature everyone is trying to avoid, but sometimes it's like am open door from golden cage.

Guava is the "from inside out" project presented as "take it or leave it". I would not personally beat Google because of their attitude towards changes. But they have to state it explicitly, otherwise more contributors - after so much work they invested - will feel betrayed!

Re: The story of Google Guava and patches

#43

Earlier quoted context omitted.

I'm not sure where you came up with any of these assertions, as they are all simply false. First, plenty of Google projects allow outside contributions. There are over 1400 open source Google projects (The number is actually much larger, but i've only counted those that wanted to be identified specifically as Google projects), and >98% of them allow contributions the last time I looked. In fact, it's easier to simply…

What do you mean exactly by "allow outside contributions"? Do you have stats on the number of projects with at least one commiter that doesn't work for google? The number of projects with non-trival LoC written by non-employees? The number of projects with an external mailing list as the place where project decisions are hashed out?

1. I mean are willing to accept outside contributions of code if people submit them, and put them in the codebase if they are acceptable.

2. I have the stats, but given that the vast majority of open source projects (Google or otherwise) don't ever grow past a few people, I don't see why it would be relevant? I also don't see why it's relevant whether they work for Google or not. It's not like we require projects follow a different process for Googler committers vs non, so i don't see how it's any different from a project where the committers are all really good friends who work on an OSS project together. We also hire a lot of committers to our open source projects. I'm guessing you want to make a distinction between "corporate open source projects", and "non-corporate open source projects", but in reality, making such a distinction would be a mistake, because the typical differences are in policies and preferences, and they apply equally well to either. IE What matters is the policies the project applies to committers and contributors, not whether they all work for the same company.

3. Again, I have stats, but for the vast majority of open source projects, just because most are willing to accept them, doesn't mean anyone ever contributes. This has nothing to do with Google, of course. If you look at the hundreds of thousands of projects on say, sourceforge, you will find the number that have either at least one not-same-email-domain-as-owner committer or non-trivial LOC written by a not-same-email-domain-as-owner committer else is quite low.

4. This seems to be a governance and social issue, I don't track it formally, as it would be quite difficult to do so. It would also be wrong for us to try to force a model on folks. We give folks info about what we thinks are best practices, and in fact, free copies of the producing OSS book (Karl used to work with us :P), how they run their projects is generally up to them. We are happy to give them advise when asked, and are happy to consult in general.

Re: The story of Google Guava and patches

#44
post #17

I'll call bullshit. Either they care about external developers or they don't. This is saying they don't. Google's culture seems insular and elitist. Besides Guava, they did the same thing with GWT (which, as much as I love GWT, didn't work out in the project's best interests, IMO), and now are doing the same thing with Dart (AFAICT). Maybe in the 90s you could get away with this. But now if you don't have an active e…

I used to an be external contributor to open-source GWT, and then became a Google employee to work on the GWT team and the situation really has nothing to do with elitism or culture. It basically boils down to a matter of resources. When I started as an open source contributor to GWT, it was used by external developers, but not really used internally by Google, so changes made by external committers couldn't possibly…

Hi Ray. You make good points, more than I can justly respond to right now.

Briefly, I did not know GWT was initially not used widely within Google--I had assumed it was from near-day-1.

The re-open sourcing sounds cool, although it would be interesting to see how it could play nicely with the internal build-from-head system you guys have. E.g. to avoid effectively forked projects.

Speaking of building from head, I'm sure it's a net win, but, as an outsider, the handcuffs of backwards compatibility seem overly tight. More frequent major-point releases that could clean up cruft might be nice. Not sure how you guys handle major-point releases internally? If at all?

And, yeah, elitist was too strong, especially to apply to individual developers. However, not just recently, but over the lifespan of GWT, it hasn't spawned an external dev community (AFAIK), so it seems like something is off.

Re: The story of Google Guava and patches

#45
post #39

Earlier quoted context omitted.

I used to an be external contributor to open-source GWT, and then became a Google employee to work on the GWT team and the situation really has nothing to do with elitism or culture. It basically boils down to a matter of resources. When I started as an open source contributor to GWT, it was used by external developers, but not really used internally by Google, so changes made by external committers couldn't possibly…

Google could carry this suggestion further and re-opensource all projects in the same manner including Android.. Not that it would be an easy transition but the long-term value from all the communities would be in Google's favor..

If only I was a senior VP. :) There are many many Googlers who would like projects to be more open source than they are, since the company has aggressively hired people who value open culture. It's not always possible, I don't know the Android team's constraints, but I would gather if you're working with an OEM on a new secret device, they don't want hints of it leaking in the public repository. :(

Re: The story of Google Guava and patches

#46
post #40

Earlier quoted context omitted.

I used to an be external contributor to open-source GWT, and then became a Google employee to work on the GWT team and the situation really has nothing to do with elitism or culture. It basically boils down to a matter of resources. When I started as an open source contributor to GWT, it was used by external developers, but not really used internally by Google, so changes made by external committers couldn't possibly…

Why was GWT team reduced? Developers were moved to Dart project? Is there any schedule for 2.5? We are waiting for SourceMaps, since debug mode is working slow in Chrome & IE, and with Firefox there always is a version gap. Safari plugin is also broken since version 5.

I can't fully explain why it was reduced, but it's several factors:

1) Some people left, for various reasons (like being at Google for 6+ years), for example, 5 left to go to different startups.

2) Some went to work on Dart

3) Some when to work on Chrome

4) Some went to work on a secret project

5) Maturity. At Google, 20 is considered a "big" team. The Closure Compiler, on which GMail, Docs, Google +, Google Search, Google Maps, etc is based is not even staffed, it's just the result of 20% contributions. GWT was, from an organizational standpoint, even in the wrong division. It should have been in infrastructure/dev-tools/etc, but instead it was under a management chain that cared about other stuff.

GWT 2.5 will be released very soon. I am trying to collect all external contributions that have been languishing and land them first.

Re: The story of Google Guava and patches

#47
post #44

Earlier quoted context omitted.

I used to an be external contributor to open-source GWT, and then became a Google employee to work on the GWT team and the situation really has nothing to do with elitism or culture. It basically boils down to a matter of resources. When I started as an open source contributor to GWT, it was used by external developers, but not really used internally by Google, so changes made by external committers couldn't possibly…

Hi Ray. You make good points, more than I can justly respond to right now. Briefly, I did not know GWT was initially not used widely within Google--I had assumed it was from near-day-1. The re-open sourcing sounds cool, although it would be interesting to see how it could play nicely with the internal build-from-head system you guys have. E.g. to avoid effectively forked projects. Speaking of building from head, I'm…

The history of GWT was that it was started by Joel and Bruce and acquired by Google. Internally, at the time, Google had been using Closure Compiler and millions of lines of Javascript code, so just from inertia, there would not have been much use in the beginning, because it's not like the GMail team is going to rewrite GMail in GWT over night. Really, the first high profile consumer facing project done with GWT was Wave. AdWords is also GWT, but not very sexy.

I come from a background of using Maven to build my projects, and Google's internal build system is somewhat maven like, but it doesn't let you specify versions in dependencies, so you always end up depending on HEAD. To me, this is the root problem making it hard for projects that live simultaneously in the open and closed worlds.

It would be interesting to see how the Guice team handles it, but maybe they're patch velocity is small.

As for why GWT didn't get a huge external community of committers? I do think it has something to do with the fact that it is a gated community, that people feel like they don't "own" it, Google does. Maybe re-open sourcing it and rebranding it as "Open Web Toolkit" or "Community Web Toolkit" would somewhat remove those mental blocks.

I would love for the open community to be true owners of GWT, and Google as just a contributor. I've been lobbying to make it happen, and I hope it does. Too many external people have put in a lot of work, they deserve it.

Re: The story of Google Guava and patches

#48

Earlier quoted context omitted.

What do you mean exactly by "allow outside contributions"? Do you have stats on the number of projects with at least one commiter that doesn't work for google? The number of projects with non-trival LoC written by non-employees? The number of projects with an external mailing list as the place where project decisions are hashed out?

1. I mean are willing to accept outside contributions of code if people submit them, and put them in the codebase if they are acceptable. 2. I have the stats, but given that the vast majority of open source projects (Google or otherwise) don't ever grow past a few people, I don't see why it would be relevant? I also don't see why it's relevant whether they work for Google or not. It's not like we require projects fol…

Thanks for your reply. I appreciate the engagement.

---

I think you set up a false dichotomy between corporate open source and non-corporate open source. A better division would be between single organization projects and multi-organization projects. For example, OpenJDK is very much a corporate project - but in addition to Oracle, IBM, Apple and RedHat all have people heavily involved in development. That means that if Oracle were for whatever reason to lose interest the project wouldn't necessarily die (leaving aside patent issues).

On the other hand look at GWT, see in particular your colleague cromwellian excellent comments upthread. Here was a technology that Google was at one time devoting a great deal of resources to and was moving quickly and in exciting directions. A lot of companies built businesses on top of the GWT library. Now, for what seem like very good and valid reasons, Google has scaled back its efforts on the project. That's fine, but there is no community ready to pick up the slack. The reasons there is no one to pick up the slack are not technical or legal, but as you describe it "governance and social issue[s]".

When there is no path to becoming a committer, you are walled off from discussions about the future of the project and your patches are accepted reluctantly at best - why spend the time to deeply familiarize yourself with a codebase?

Except by being hired away by Google, which isn't exactly going to make your company thrilled to sponsor your work on a project.

Re: The story of Google Guava and patches

#49
post #17

I'll call bullshit. Either they care about external developers or they don't. This is saying they don't. Google's culture seems insular and elitist. Besides Guava, they did the same thing with GWT (which, as much as I love GWT, didn't work out in the project's best interests, IMO), and now are doing the same thing with Dart (AFAICT). Maybe in the 90s you could get away with this. But now if you don't have an active e…

I used to an be external contributor to open-source GWT, and then became a Google employee to work on the GWT team and the situation really has nothing to do with elitism or culture. It basically boils down to a matter of resources. When I started as an open source contributor to GWT, it was used by external developers, but not really used internally by Google, so changes made by external committers couldn't possibly…

I'm with stephen on that. You say each of you working in the GWT team, as a person, didn't have enough time to do review of submissions. To you there was not enough bandwidth.

I say your employer didn't allocate enough bandwidth. It prioritized internal needs so far over the free software approach that the free software aspect died off completely.

You are right in saying none of the developers was an elitist; you are not right in saying that Google in itself wasn't elitist. Your manager's manager's manager knew exactly what he was doing when he allocated this many people to your team, and not more; when he didn't allocate people to do full-time community submission review when there was obviously enough work for multiple such positions.

That's one issue with "sponsored but free" projects: they never are really free because the work gets prioritized that benefits the sponsor, not the work that benefits the project as a whole. Projects are forcibly steered to keep their center of gravity and biggest source of submission inhouse, under the control of the sponsor. It's a very simple strategy of making a "free" project non-free; you can immediately pull the plug on a project which misbehaves: first you make sure the developers not under your control get stalled and discouraged, and that their work becomes harder; they cannot keep up. They couldn't continue the work being done on their own. Then, you make sure the others are so engrossed in work you, the sponsor, assign to them that they cannot keep a project healthy through intellectual exchange with externals. You build a glass wall around the project. You make internal documentation that does not need to be redistributed with your "free software" because it is not a part of it according to the law. The (complicated) build system isn't, either. Finally you have complete control of the direction, progress, and life of the project. You, the sponsor, can submit the project to your neurotic desires, which we, developers, all know. The typical corporate bullshit-o-rama that never fails to make a good project go to hell.

Open source is not a salvation.

The only way to prevent really good projects which have received huge amounts of contribution from skilled developers from becoming useless wastes of talent is to make sure that external requests, contributions, and requirements are taken at face value and are given a chance to be evaluated with the same attention as internal problems. This cannot happen in a corporation where the management performance is counted as a function of successfully resolved projects coming from internal clients. This cannot happen at all if there are internal clients, because "internal client" is a term made exactly with the point of being able to work better with "internal" clients than "external" clients. Sponsored software is not free software unless it is something akin to a donation, where the sponsor is not given creative or otherwise rule over the project.

Re: The story of Google Guava and patches

#50

Maybe I'm alone in thinking this but having been in the position of reviewing more than a few non-trivial bugfix patches myself I think I might tend to agree with Kevin. Sure it's great people are excited and want to contribute but all that excitement is due to the love and care people sweated in to making every single line in that codebase as perfect / performant / easy to understand as possible. Patches almost neve…

Gee, how does Linux ever make it! All of it is contributed! Oh: Torvalds just sits down and comes through on looking at the submissions.
Post reply on HN