Live data from Hacker News

Move Fast and Fix Things

githubengineering.com

51–60 of 95 posts

Re: Move Fast and Fix Things

#51
post #23

I'll highlight something I've learned in both succeeding and failing at this metric: When rewriting something, you should generally strive for a drop-in replacement that does the same thing, in some cases, even matching bug-for-bug, or, as in the article, taking a very close look at the new vs. the old bugs. It's tempting to throw away the old thing and write a brand new bright shiny thing with a new API and a new da…

This is probably very context dependent, because I've learned the opposite.

For example, I was rewriting/consolidating a corner of the local search logic for Google that was spread throughout multiple servers in the stack. Some of the implementation decisions were clearly made because of the convenience of doing so in a particular server. But when consolidating the code into a single server, the data structures and partial results available were not the same, so re-producing the exact same logic and behavior would have been hard. Realizing which parts of the initial implementation were there because it was convenient, and which were there for product concerns let me implement something much simpler that still satisfied the product demands, even if the output was not bitwise identical.

Re: Move Fast and Fix Things

#52

Earlier quoted context omitted.

I think jerf was trying to say that when re-implementing something, re-implement it first then improve it, don't do both at once.

> re-implement it first then improve it Right. I'm saying (poorly, I guess) that that doesn't always work. If the technical debt is sprinkled throughout your system or your organization is sunk deeply into a broken paradigm, you're in trouble. Sometimes you need to provide a completely new technology that serves many of the same use cases in a different way. I'm not saying we should reach for rewriting things as our…

One strategy that I've seen work for this kind of deep architectural change is to write the new system, then write a shim that provides a compatibility layer (however hacky and ugly) to emulate the old system. This lets you test the new system without then having to also test everything that the system interacts with. And then, start replacing usages of the shim with direct interaction with the newer prettier system.

Think the win32 to NPAPI transition.

Re: Move Fast and Fix Things

#53
post #39

Earlier quoted context omitted.

> ...that is a high-risk approach that is usually without correspondingly high payoffs That's probably true, but it's also true that over a long enough timescale (100 years, to trigger a reductio ad absurdum) there is a very high risk that not replacing or rewriting that code will sink your technology and possibly your organization. Just because the risk will be realized in the long run doesn't mean it's not a risk.…

>but it's also true that over a long enough timescale (100 years, to trigger a reductio ad absurdum) there is a very high risk that not replacing or rewriting that code will sink your technology and possibly your organization. Could you elaborate on what basis you claim this as a truth?

It's largely conjecture, admittedly. But there are very few pieces of software that last 40 years. Most companies don't plan to be out of business in the next 40 years. And many tech companies have gone out of business in even the last ten years because they weren't agile enough to adapt quickly to new technological changes.

Re: Move Fast and Fix Things

#54
How does Scientist work with code that produces side effects? In the example, presumably both the new and old each create a merge commit. Maybe these two merge commits are done in in-memory copies of the repo so that the test result can just be discarded, but what about in the general case where a function produces an output file or some other external effect?

Re: Move Fast and Fix Things

#55

Earlier quoted context omitted.

> ...that is a high-risk approach that is usually without correspondingly high payoffs That's probably true, but it's also true that over a long enough timescale (100 years, to trigger a reductio ad absurdum) there is a very high risk that not replacing or rewriting that code will sink your technology and possibly your organization. Just because the risk will be realized in the long run doesn't mean it's not a risk.…

I think jerf was trying to say that when re-implementing something, re-implement it first then improve it, don't do both at once.

This koan also keeps your change history coherent which will likely be useful for the person that has to fix it later.

Re: Move Fast and Fix Things

#56
post #23

I'll highlight something I've learned in both succeeding and failing at this metric: When rewriting something, you should generally strive for a drop-in replacement that does the same thing, in some cases, even matching bug-for-bug, or, as in the article, taking a very close look at the new vs. the old bugs. It's tempting to throw away the old thing and write a brand new bright shiny thing with a new API and a new da…

> It hardly seems worth even having a bug system if the frequency of from-scratch rewrites always outstrips the pace of bug fixing. Why not be honest and resign yourself to the fact that version 0.8 is followed by version 0.8, which is then followed by version 0.8?

https://www.jwz.org/doc/cadt.html

Re: Move Fast and Fix Things

#57
post #52

Earlier quoted context omitted.

> re-implement it first then improve it Right. I'm saying (poorly, I guess) that that doesn't always work. If the technical debt is sprinkled throughout your system or your organization is sunk deeply into a broken paradigm, you're in trouble. Sometimes you need to provide a completely new technology that serves many of the same use cases in a different way. I'm not saying we should reach for rewriting things as our…

One strategy that I've seen work for this kind of deep architectural change is to write the new system, then write a shim that provides a compatibility layer (however hacky and ugly) to emulate the old system. This lets you test the new system without then having to also test everything that the system interacts with. And then , start replacing usages of the shim with direct interaction with the newer prettier system…

I believe you are describing http://www.martinfowler.com/bliki/StranglerApplication.html and particularly http://www.martinfowler.com/bliki/AssetCapture.html

Re: Move Fast and Fix Things

#58
post #15

Earlier quoted context omitted.

Author here. 5 years ago I would have agreed with you and logged e.g. 10 million merge requests to replay them offline. But one thing I've found over the years (which may seem obvious in retrospect) is that staging environment are not identical to production. Particularly not when it comes to finding sneaky bugs and performance regressions -- the code doesn't run on the same exact environment it will run when it is d…

> why would you run these experiments offline when you can run them online? It probably doesn't apply in this case, but many bugs cannot be replicated in the production environment. At least, you don't want to break things in production to see if your software does the right thing in adverse scenarios.

Some would say that's a really good strategy. Keeps you on your toes...

http://techblog.netflix.com/2012/07/chaos-monkey-released-in...

Re: Move Fast and Fix Things

#59

This is tangential, but given the increasing functionality and maturity of libgit2, I wonder if it would yet be feasible to replace the Git command-line program with a new one based on libgit2, and written to be as portable as libgit2. Then there would be just one Git implementation, across the command line, GUIs, and web-based services like GitHub. Also, the new CLI could run natively on Windows, without MSYS.

That is my dream. Right now, many Windows GUIs (SmartGit/SourceTree) use 'git.exe' to manage the actual Git repositories. If libgit2 is fully mature, I can imagine more GUIs/tools will be built to manage/analyze the git repositories.

I believe github windows client is going that route https://github.com/blog/1127-github-for-windows.

I also remember some blog post going into sync/async details of git.exe vs libgit2 stuff. Will try to google it.

Re: Move Fast and Fix Things

#60
post #6

Earlier quoted context omitted.

I haven't read the whole article yet, so I might have missed something; but how do we know that people weren't reporting these issues? I've always had to report issues to GitHub via email as they do not have a public issue tracker (something I've always found a bit ironic).

It's interesting that git is the same. [EDIT:] ...in that all issues and PRs are emailed rather than entered into a web app.

I'm not familiar with the Git project's inner workings, but their website git-scm.com tells me they are hosted on GitHub, which has a public issue tracker: https://github.com/git/git-scm.com/issues

GitHub, however, only allows issues to be reported via private mail. I'm not aware of a 'public' issue tracker for GitHub anywhere (even if a mailing list).

Post reply on HN