Live data from Hacker News

Ask HN: How do you keep track of releases/deployments of dozens micro-services?

news.ycombinator.com

1–10 of 56 posts

Re: Ask HN: How do you keep track of releases/deployments of dozens micro-services?

#2
What do you mean by keep track? Do you want to be aware of deployments?

A Slack notification could do it. Or do you want to correlate deployments with other metrics?

In this case we instrument our deployments into our monitoring stack (influxdb/grafana) and use this as annotations for the rest of our monitoring.

We can also graph the number of releases per project on different aggregates.

Re: Ask HN: How do you keep track of releases/deployments of dozens micro-services?

#3
Something called 'integration testing' that has to be done before the final build which clearly flags off any compatibility issues between components.

Every component comes with a major/minor release no., which tells about the nature of change that has gone in. For ex: Major rel is incremented for a change that usually introduces a new feature/interface. Minor release no are reserved for bug fixes/optimizations, that are more internal to the component.

The build manager can go through the list of all the delivered fixes and cherry pick the few which can go to the final build.

Re: Ask HN: How do you keep track of releases/deployments of dozens micro-services?

#4
We wrote https://github.com/kiwicom/crane which posts and updates a nicely formatted Slack message with the status of releases. It also posts release events to Datadog (in a version we're publishing soon) and to an API that records them in a Postgres DB we keep for analytics queries.

Re: Ask HN: How do you keep track of releases/deployments of dozens micro-services?

#7
Service discovery contains all the versions and who should be directed at what.

We also store stats in the service discovery app so versions can be promoted to "production" for a customer once the account management team has reviewed and updated their internal training.

Re: Ask HN: How do you keep track of releases/deployments of dozens micro-services?

#8
Keep track as having a version controlled state of all revisions/versions deployed? That's something I would be interested in solutions to too, especially in a kubernetes environment with CI.

I could probably snapshot the kubernetes state to have an trail I can use to rollback to a point in time. Alternatively I thought about having CI updatemanifests in an integration repo and deploy from there, so that every change to the cluster is reflected by a commit in this repository.

Re: Ask HN: How do you keep track of releases/deployments of dozens micro-services?

#9
At the guardian we use https://github.com/guardian/riff-raff

It takes a build from your build system (typically team city, but not exclusively) deploys it and record the deployment.

You can then check later what's currently deployed, or what was deployed at some point in time in order to match it with logs etc.

Not sure how useable it would be outside of our company though.

Re: Ask HN: How do you keep track of releases/deployments of dozens micro-services?

#10
In the company i worked for, they had their own CI/CD system which tracked information about each service and the systems onto which it has to deploy. Once it was all configured, it was basically button pushes. Also the system tracked feedback after deployment to confirm if the build went good or needed to be fixed - if certain parameters were unwell, basically it did a role back ! Also there were canary deployments to make sure code was deployed only to portion of systems to make sure it indeed pushed correctly and worked. If not, they are rolled back !
Post reply on HN