I was doing this a year ago. Now I'm redoing it. In my case, what I missed is that if the app is complex enough you should treat the deployer like a separate product. Different teams might want to use it for different things, deploy different versions with it, etc. So it should be in a separate repo with its own tests and its own version string. Otherwise iterating on it is hell because you're doing so in a space tha…
> So it should be in a separate repo If it's in the same repo, that means all of the features of the deployer can match the deployed code automatically - the deployer is made for and tested for that version. With different repos, you add complexity. Now you have to support deploying multiple versions so there are more code paths and there's version checking etc. And more complexity also means you need more tests.
- you don't care about tests that span multiple versions of the app (e.g. patterns of usage around database upgrades)
- you don't care about git bisect sessions that evaluate a new test against old versions of the app
- you never want to try the new app on the old infra or the old app on the new infra
- you're not worried about tests written by people not on your team whose failures indicate scenarios that don't require action on your part (but might require action on theirs).