Live data from Hacker News

Move Fast and Fix Things

githubengineering.com

91–95 of 95 posts

Re: Move Fast and Fix Things

#91
post #23

I'll highlight something I've learned in both succeeding and failing at this metric: When rewriting something, you should generally strive for a drop-in replacement that does the same thing, in some cases, even matching bug-for-bug, or, as in the article, taking a very close look at the new vs. the old bugs. It's tempting to throw away the old thing and write a brand new bright shiny thing with a new API and a new da…

I've learned a variation on this theme, which is more specific:

You have some code that, for whatever reason, you think is not very good. There is a reason why you have ended up with code that is not very good.

If your action is to sit down and write it all again, you should anticipate getting a very similar result (and this has been the outcome of every "big rewrite" effort I've ever seen: they successfully reproduced all the major problems of the old system).

The reasons why this happen are probably something to do with the way you're doing development work, and not the codebase that you're stuck with. Until you learn how to address those problems, you should not anticipate a better outcome. Once you have learned how to address those problems, you are likely to be able to correct the problem without doing a "big rewrite" (most commonly by fixing them one piece at a time).

Sometimes I see people attempt a "big rewrite" after replacing all of the people, thinking that they can do a better job. The outcome of this appears to me to invariably be that the second team who tried to build the system with no real experience end up following a very similar path to the first team that did the same thing (guided by the map that the first team left them, and again reproducing all the same problems).

From these observations I draw one key conclusion: the important thing that you get from taking smaller steps is that you amplify your ability to learn from things that have already been done, and avoid repeating the mistakes that were made the last time. The smaller the step, the easier it becomes to really understand how this went wrong last time and what to do instead. Yes, the old codebase is terrible, but it still contains vitally important knowledge: how to not do that again. Neither writing new code from scratch without touching the old, nor petting the old code for years without attempting to fix it, are effective ways to extract that knowledge. The only approach I've ever really seen work is some form of "take it apart, one piece at at time, understand it, and then change it".

Re: Move Fast and Fix Things

#92

Earlier quoted context omitted.

That is my dream. Right now, many Windows GUIs (SmartGit/SourceTree) use 'git.exe' to manage the actual Git repositories. If libgit2 is fully mature, I can imagine more GUIs/tools will be built to manage/analyze the git repositories.

I believe github windows client is going that route https://github.com/blog/1127-github-for-windows . I also remember some blog post going into sync/async details of git.exe vs libgit2 stuff. Will try to google it.

This is the article you speak of: http://githubengineering.com/git-concurrency-in-github-deskt...

The AsyncReaderWriterLock mentioned in blog does not directly show up in Google, but it appears to be based on the one in this blog post: http://blogs.msdn.com/b/pfxteam/archive/2012/02/12/building-...

Re: Move Fast and Fix Things

#93
Humans will always reverberate around truths like this.

The emphasis shift on breaking vs fixing looks like a good example of how fashion trends in tech create artificial struggles that help new people understand the "boundaries" of $things.

Fashion's like a tool for teaching via discussion

Edit: I'm just commenting on what I percieve as a fashionable title not the article.

Re: Move Fast and Fix Things

#94
post #72

Earlier quoted context omitted.

You can break this down even more. As we speak, I'm "replacing" old code by just writing a wrapper around it with the new API it should have. Then I'll rewrite it without the wrapper, bug-for-bug. And then I'll actually fix the bugs.

There's a simpler method than this that provides even more surety, used by e.g. LibreSSL: 1. Start writing your new implementation (or heavily refactoring your old implementation, whichever), but in parallel, for each legacy function you remove, write an equivalent "legacy wrapper" function that implements the old API (and ABI; you have to return the same structs and all) in terms of the new API. 2. As you develop th…

This is a textbook example of 'How to Kill an OSS project'

#1, #4, and #5 are completely unnecessary.

If it's desirable to 'kill with fire' the old codebase, you could always create a fork and merge the breaking changes on the next major release.

Creating a second library is a terrible idea for 2 reasons...

You lose the legacy source control history which is (arguably) more valuable than the current source because it can be used to research solutions to old problems.

You split the community which is devastating to the culture and survivability of an OSS project. Even something as a simple name change will have massive negative impacts on the level of contributions. Only the most popular and actively developed projects can get away with forking the community.

LibreSSL will likely survive because everything in the world that requires crypto uses OpenSSL. Even then, that was the absolutely wrong way to go about things.

The only solid justification for a rename and complete rewrite, is if there are license/copyright issues.

Re: Move Fast and Fix Things

#95
post #72

Earlier quoted context omitted.

There's a simpler method than this that provides even more surety, used by e.g. LibreSSL: 1. Start writing your new implementation (or heavily refactoring your old implementation, whichever), but in parallel, for each legacy function you remove, write an equivalent "legacy wrapper" function that implements the old API (and ABI; you have to return the same structs and all) in terms of the new API. 2. As you develop th…

This is a textbook example of 'How to Kill an OSS project' #1, #4, and #5 are completely unnecessary. If it's desirable to 'kill with fire' the old codebase, you could always create a fork and merge the breaking changes on the next major release. Creating a second library is a terrible idea for 2 reasons... You lose the legacy source control history which is (arguably) more valuable than the current source because it…

> You lose the legacy source control history which is (arguably) more valuable than the current source because it can be used to research solutions to old problems.

No reason for that. Both projects—the wrapper and the new core—can branch off the original. Create a commit that removes one half of the files on one side, and the other half of the files on the other, and make each new commit "master" of its repo, and now you've got two projects with one ancestor. Project mitosis.

> You split the community

How so? I'm presuming a scenario here where either 1. you were the sole maintainer for the old code, and it's become such a Big Ball of Mud that nothing's getting done; or 2. the maintainer of the old code is someone else who is really, really bad at their job, and you're "forking to replace" with community buy-in (OpenSSL, Node.js leading to the io.js fork, gcc circa 1997 with the egcs fork, MySQL leading to MariaDB, etc.).

In both scenarios, development of the old code has already basically slowed to a halt. There is no active community contributing to it; or if there is, it is with great disgust and trepidation, mostly just engineers at large companies that have to fix upstream bugs to get their own code working (i.e. "I'm doing it because they pay me.") There are a lot of privately-maintained forks internal to companies, too, sharing around patches the upstream just won't accept for some reason. The ecosystem around the project is unhealthy†.

When you release the new legacy wrapper, it replaces the old library—the legacy wrapper is now the only supported "release" of the old library. It's there as a stopgap for Enterprise Clients with effectively-dead projects which have ossified around the old library's ABI, so these projects can continue to be kept current with security updates et al. It's not there for anyone to choose as a target for their new project! No new features will ever be added to the wrapper. It's a permanent Long-Term Support release, with (elegant, automatic) backporting of security updates, and that's it. Nobody starting a project would decide to build on it any more than they'd build on e.g. Apache 1.3, or Ubuntu 12.04.

> Even something as a simple name change will have massive negative impacts on the level of contributions.

Names are IP, obviously (so if you're a third party, you have to rename the project), but they're more than that—names are associated in our brains with reflexes and conventions for how we build things.

The reason Perl 6, Python 3, etc. have so much trouble with adoption is that people come into them expecting to be able to reuse the muscle-memory of the APIs of Perl 5/Python 2. They'd have been much better off marketed as completely new languages, that happened to be package-ecosystem-compatible with the previous language, like Elixir is to Erlang or Clojure is to Java.

If these releases were accompanied by their creators saying "Python/Perl is dead, long live _____!" then there'd have been a much more dramatic switchover to the new APIs. Managers understand "the upstream is dead and we have to switch" much more easily than they understand "the upstream has a new somewhat-incompatible major version with some great benefits."

One good example: there's a reason Swift wasn't released as "Objective-C 3.0". As it is, ObjC is "obviously dead" (even though Apple hasn't said anything to that effect!) and Swift is "the thing everyone will be using from here on, so we'd better move over to it." In a parallel reality, we'd have this very slow shift from ObjC2 to ObjC3 that would never fully complete.

---

† If the ecosystem were healthy, obviously you don't need the legacy wrapper. As you say, just release the new library as the new major version of the old library—or call the new library "foo2", as many projects have done—and tell people to switch, and they will.

It's easy to find healthy projects like this when you live close enough to the cutting-edge that all your downstream consumers are still in active development, possibly pre-1.0 development. The Node, Elixir, Go and Rust communities look a lot like this right now; any project can just "restart" and that doesn't trouble anybody. Everyone rewrites bits and pieces of their code all the time to track their upstreams' fresh-new-hotness APIs. That's a lot of what people mean when they talk about using a "hip new language": the fact that they won't have to deal with stupid APIs for very long, because stupid APIs get replaced.

But imagine trying to do the same thing to, say, C#, or Java, or any other language with Enterprise barnacles. Imagine trying to tell people consuming Java's DateTime library that "the version of DateTime in Java 9 is now JodaTime, and everyone has to rewrite their date-handling code to use the JodaTime API." While the end results would probably have 10x fewer bugs, because JodaTime is an excellent API whose UX makes the pertinent questions obvious and gives devs the right intuitions... a rewrite like that just ain't gonna happen. Java 9 needs a DateTime that looks and acts like DateTime.

Post reply on HN