Live data from Hacker News

Software Engineering at Google (2017)

arxiv.org

221–230 of 319 posts

Re: Software Engineering at Google (2017)

#221
post #89

Earlier quoted context omitted.

That's what I meant in my question with "share code by other means". It works but in my opinion it is a large pain and I can't believe people at Google work by sending patches back and forth.

It's not like we (I am a Googler) email patch files around. Everything is integrated into the system. You create a CL (change list), it automatically gets a number. People can review it, test it, or fork it (make a new CL using your CL as a starting point) as much as they want, all from that CL number.

Functionally, this may be different but I am struggling to see how conceptually this makes any difference in the development process.

Re: Software Engineering at Google (2017)

#222
post #5

Buried in the "2.11 Frequent rewrites" section, but a great hack for "productivity via a sense of ownership": "In addition, rewriting code is a way of transferring knowledge and a sense of ownership to newer team members. This sense of ownership is crucial for productivity: engineers naturally put more effort into developing features and fixing problems in code that they feel is “theirs”."

Thanks for highlighting this. To me it seems an important idea that contradicts conventional wisdom, similar in the way that most people over-encourage DRY, blind to the fact it increases coupling.

Wait, huh? How does DRY increase coupling?

I mean, I guess the duplicate code/class is now coupled to the two places that use it, but I have a hard time seeing how that is worse than two duplicate instances of the code.

Re: Software Engineering at Google (2017)

#223
post #126
post #9

Earlier quoted context omitted.

I frequently find myself drastically refactoring code to understand it. I don't commit those changes because it's not worth the effort to justify the cleanup to people who treat these rules as gospel. Apparently me spending half a day reading code is no big deal but cleaning it up is a waste of time. Shrug.

Five people understand the system. You refactor it. Now one person understands the system.

Initially the system took 8 hours of reading and 2 days of refactoring to grasp.

Now it takes 2 hours.

I’ll take that.

Re: Software Engineering at Google (2017)

#224
post #147

I was the maintainer of a third-party library used by thousands of dependent applications at Google. I have to admit, I still have not seen on the outside a system that allows me to change the version of numpy, and know that thousands of dependent applications either work or break, within an hour of making my change. Being able to write and use a mapreduce with a high level of confidence that my code would continue t…

People who like multirepos are always saying how easy it is to pin dependencies but like you I haven’t seen anyone doing it right since I left Google. The monorepo third-party system works well in practice. Ps thanks for getting scipy into third_party all those years ago.

I think the problem is more just a reversal of perspective. It’s up to each different app to detect and respond to a breaking change from a dependency.

The idea of wanting to make a change to a third party library and then see all the downstream consumers who would be broken by that change if they updated to start consuming that change is an incredibly stupid thing to want, and it’s no measure of success whatsoever to build something that gives you that information.

It’s like the most giant case of coupling you can imagine (letting the statuses of thousands of consumer apps act as any type of constraint on the developer choices of the third party library, as opposed to all those consumer apps opting in to changes on their own terms by updating their dependencies).

Imagine if I have shared a bunch of copies of my resume with a bunch of recruiters. They are out there selling me as a candidate or whatever. Now I decide I want to change my resume, but I don’t know if it’s going to upset the approach some recruiter is taking.

If I can’t update my resume unless I first consult a big oracle that tells me which recruiters will be negatively impacted, that’s a problem, and not at all some type of live-with-able “customer service” positive thing. It’s just plain old bad coupling.

Creating such a system that could automatically diff the old resume’s usage constraints against my proposed changes would be a gigantic waste of time. The exact opposite of something to celebrate.

I say this as someone who routinely writes in-house software libraries used by dozens or hundreds of other apps, various teams, and even a few that are open source.

The primary thing gauging the health of our development is that we are decoupled from any consumers. We are free to make whatever changes we want, and whether downstream teams would like to receive those changes is wholly an opt-in process with versioned dependencies and easy rollbacks controlled by those consumers.

Re: Software Engineering at Google (2017)

#225
post #91
post #5

Buried in the "2.11 Frequent rewrites" section, but a great hack for "productivity via a sense of ownership": "In addition, rewriting code is a way of transferring knowledge and a sense of ownership to newer team members. This sense of ownership is crucial for productivity: engineers naturally put more effort into developing features and fixing problems in code that they feel is “theirs”."

If you have engineers with physiological problem of “not invented here”, you have a very serious issue. I am currently seeing this in real time in one of the projects and I was told almost exact same words as “reason” to recreate what we already have and working beautifully. It was clear to me that some developers are just too lazy to dive in to complex system. They get ticked off by one imperfection here and other o…

To be clear, "Not invented here" isn't about "don't rebuild things", it's about "don't build things that exist outside of the company".

It's perfectly reasonable (for the reasons in the submission) to rewrite code that exists already.

Re: Software Engineering at Google (2017)

#227
post #200

Earlier quoted context omitted.

People who like multirepos are always saying how easy it is to pin dependencies but like you I haven’t seen anyone doing it right since I left Google. The monorepo third-party system works well in practice. Ps thanks for getting scipy into third_party all those years ago.

> The monorepo third-party system works well in practice. It's worth noting that this is only viable at Google because they don't use git. Git's insistence on every client having a full copy of all history of every file in the repository makes monorepo much more expensive. I see conflicting reports over whether google use Perforce or something proprietary called "piper"?

I used a git wrapper for google3 repo. It wasn't great. There are a number of semantic differences between piper/perforce and git that made it awkward. Especially code review- git doesn't handle code review well (I still find this to be an issue with github and other sites that have code review). but it was not an officially supported solution and I believe the replacement for it is based on another DVCS, Mercurial, for some silly software engineering reasons I don't like.

Re: Software Engineering at Google (2017)

#228
post #169

Earlier quoted context omitted.

Dropbox started in python (and is still using it a lot) ...and we all know how horribly they failed.

Whether a startup will succeed has nothing to do with the language used. Google started with Python and TikTok with PHP (wtf). However when you start, go with the better choice since every line of code becomes a liability later.

Well this makes me feel good, because I apparently write better Python than the average Googler, since my Python is entirely maintainable.

Agree about the monorepo thing though, it just seems like people are optimizing for the wrong things with monorepos.

Re: Software Engineering at Google (2017)

#229
post #200

Earlier quoted context omitted.

> The monorepo third-party system works well in practice. It's worth noting that this is only viable at Google because they don't use git. Git's insistence on every client having a full copy of all history of every file in the repository makes monorepo much more expensive. I see conflicting reports over whether google use Perforce or something proprietary called "piper"?

Afaik it's a custom system with download-and-cache-on-demand. Note that there's nothing forbidding you from writing a virtual git filesystem that fetches objects from some centralized repo as files are open()ed. Git on cloud steroids.

Microsoft actually had done it and pushed the whole Windows repo into git.

https://github.com/Microsoft/VFSForGit

Re: Software Engineering at Google (2017)

#230
post #147

I was the maintainer of a third-party library used by thousands of dependent applications at Google. I have to admit, I still have not seen on the outside a system that allows me to change the version of numpy, and know that thousands of dependent applications either work or break, within an hour of making my change. Being able to write and use a mapreduce with a high level of confidence that my code would continue t…

Are you allowed to share if the internal repository a custom version control system or is it one of the open source ones?

it's already been published. Google used Perforce for at least a decade and then cloned the perforce wire protocol when it was clear perforce wouldn't keep scaling (there's a great presentation online about google using RAMSANs to store the perforce index), but backed the repo in something like bigtable or spanner.

I'll give google credit for one thing: it can change backends a lot without too much user visible pain.

Post reply on HN