We decided to go for the big rewrite
tech.channable.com
We decided to go for the big rewrite
1–10 of 75 posts
Re: We decided to go for the big rewrite
#2This doesn't really sound like a big-bang rewrite as such but an incremental development process. The sort of rewrite that would be a major strategic mistake is where you start with a new repository and begin re-implementing the entire product, but this seems all perfectly sensible to me.
This just sounds like the sort of incremental Ship of Theseus [0] development that many of us are doing. The product I'm working on has had enough key internal portions rewritten over a long enough time (including interestingly the job management system) that you could say it's a rewrite compared to the product from 2 years ago.
Re: We decided to go for the big rewrite
#3This doesn't really sound like a big-bang rewrite as such but an incremental development process. The sort of rewrite that would be a major strategic mistake is where you start with a new repository and begin re-implementing the entire product, but this seems all perfectly sensible to me. This just sounds like the sort of incremental Ship of Theseus [0] development that many of us are doing. The product I'm working o…
I'd like to see some idea of what they mean by big.
An order of magnitude in terms of lines of code and the number of engineers they had for the job and how long it all took versus how long the original took to write etc.
One of the things that scares me about something like this are all the seemingly illogical bits of code that were added over time and are actually that way for a reason because they fix some edge case or whatever. Hard to see those not getting swept away in a rewrite as well as a whole bunch of new rarely occurring issues added.
A complete rewrite strikes me as something that's almost guaranteed to be buggier than what it replaces, at least for awhile.
Re: We decided to go for the big rewrite
#4This doesn't really sound like a big-bang rewrite as such but an incremental development process. The sort of rewrite that would be a major strategic mistake is where you start with a new repository and begin re-implementing the entire product, but this seems all perfectly sensible to me. This just sounds like the sort of incremental Ship of Theseus [0] development that many of us are doing. The product I'm working o…
I'd like to see some idea of what they mean by big. An order of magnitude in terms of lines of code and the number of engineers they had for the job and how long it all took versus how long the original took to write etc. One of the things that scares me about something like this are all the seemingly illogical bits of code that were added over time and are actually that way for a reason because they fix some edge ca…
> A complete rewrite strikes me as something that's almost guaranteed to be buggy
Likely buggy, but buggy in different ways. The real test will be in measuring the number and quality and severity of bugs, as well as the ability to identify and repair them in both systems. A testable system with good test code written may still be 'buggy', but will help foster a sense of confidence when making the fixes required.
Re: We decided to go for the big rewrite
#5> Prematurely designing systems “for scale” is just another instance of premature optimization
> Examples abound: (...) using a distributed database when Postgres would do
This is the only part of the article that bugged me a little, because in my experience the choice between single-machine and distributed databases is not so much about scale as it is about availability and avoiding a single point of failure.
Even if your database server is fairly stable (a VM in a robust cloud for instance), if you use Postgres or MySQL and you need to upgrade to a newer version of the database (let say for an urgent security update), you have no choice but to completely stop the service for a few seconds / minutes (assuming the service cannot work without its database).
Depending on the service and its users, this mandatory down-time might or might not be acceptable.
Anecdotally I suspect services requiring high SLAs are more common than ones requiring petabyte scale storage.
Re: We decided to go for the big rewrite
#6> Prematurely designing systems “for scale” is just another instance of premature optimization > Examples abound: (...) using a distributed database when Postgres would do This is the only part of the article that bugged me a little, because in my experience the choice between single-machine and distributed databases is not so much about scale as it is about availability and avoiding a single point of failure. Even i…
High availability Postgres setups are a minimum for a production system and a staging system to understand how your system behaves during a failure event. These failure scenarios should be tested not necessarily on every commit but often enough that there’s confidence during a failover you’re not going to drop queries on the floor and pretend it’s all good as well as your monitoring systems report on the event for the sake of event reporting.
Re: We decided to go for the big rewrite
#7> Prematurely designing systems “for scale” is just another instance of premature optimization > Examples abound: (...) using a distributed database when Postgres would do This is the only part of the article that bugged me a little, because in my experience the choice between single-machine and distributed databases is not so much about scale as it is about availability and avoiding a single point of failure. Even i…
[deleted]
Re: We decided to go for the big rewrite
#8This doesn't really sound like a big-bang rewrite as such but an incremental development process. The sort of rewrite that would be a major strategic mistake is where you start with a new repository and begin re-implementing the entire product, but this seems all perfectly sensible to me. This just sounds like the sort of incremental Ship of Theseus [0] development that many of us are doing. The product I'm working o…
[deleted]
Re: We decided to go for the big rewrite
#9> Prematurely designing systems “for scale” is just another instance of premature optimization > Examples abound: (...) using a distributed database when Postgres would do This is the only part of the article that bugged me a little, because in my experience the choice between single-machine and distributed databases is not so much about scale as it is about availability and avoiding a single point of failure. Even i…
High availability Postgres setups are a minimum for a production system and a staging system to understand how your system behaves during a failure event. These failure scenarios should be tested not necessarily on every commit but often enough that there’s confidence during a failover you’re not going to drop queries on the floor and pretend it’s all good as well as your monitoring systems report on the event for th…
Yeah. I guess after having been bitten myself a few times with failed MySQL failovers and especially after having read the GitHub October 2018 incident postmortem [1], I stopped considering failover solutions as a reliable availability solution altogether.
However this is just a personal opinion that I might revisit at some point.
[1] https://github.blog/2018-10-30-oct21-post-incident-analysis/
Re: We decided to go for the big rewrite
#10This doesn't really sound like a big-bang rewrite as such but an incremental development process. The sort of rewrite that would be a major strategic mistake is where you start with a new repository and begin re-implementing the entire product, but this seems all perfectly sensible to me. This just sounds like the sort of incremental Ship of Theseus [0] development that many of us are doing. The product I'm working o…
I'd like to see some idea of what they mean by big. An order of magnitude in terms of lines of code and the number of engineers they had for the job and how long it all took versus how long the original took to write etc. One of the things that scares me about something like this are all the seemingly illogical bits of code that were added over time and are actually that way for a reason because they fix some edge ca…
>I'd like to see some idea of what they mean by big. An order of magnitude in terms of lines of code and the number of engineers they had for the job and how long it all took versus how long the original took to write etc.
Yeah, at my last place before I left, they were looking to do a rewrite of their main project that had been the result of years of hundreds of developers working on it. The time-scale was wooly, but was expected to be between 5 and 10 years and probably require ~200 developers and testers.
This was rewriting millions of lines of code, very little would be reusable.
I don't know if they went ahead with it or not, but it was looking like it would descend into chaos.