Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: How do you keep track of releases/deployments of dozens micro-services?

news.ycombinator.com

Post reply on HN
Print view View original

131–140 of 140 posts

Re: Ask HN: How do you keep track of releases/deployments of dozens micro-services?

#131

Post by killtimeatwork »

lordgilman wrote: post #47
killtimeatwork wrote: post #44

Earlier quoted context omitted.

> Production issues are a part of life. Only if you accept them. The alternative is to do very few, rigorously tested releases per year. This way you don't have production issues. That's how industries like banking make sure bank transfers and card payments work and people's money is not randomly lost... It's a shame many other industries just accept their product failing for users as something normal/inevitable.

I can't say my experience echoes your comment. I'm a former employer of a financial services (billing) company built around a mainframe code base started in the 70s. We probably qualify for the sort of business you had in mind with your comment. We did four releases a year, across the entire organization (so mainframe and more modern platforms), on Saturday nights/early Sunday mornings. There was plenty of testing bu…

> We did four releases a year, across the entire organization (so mainframe and more modern platforms), on Saturday nights/early Sunday mornings. There was plenty of testing but there was still plenty of errors only found on the day of, and rushed to fix in the wee hours or daylight hours of Sunday morning.

This way, you had bugs in prod for less than a day once every quarter, as opposed to having buggy prod all the time, as is common in organizations doing Continuous Deployment.

Re: Ask HN: How do you keep track of releases/deployments of dozens micro-services?

#132

Post by sdevonoes »

We don't. I'm just waiting the day we reach more than 100 microservices and my company realises that microservices was a bad idea to begin with. That's usually the way it works: learning the hard way.

To elaborate:

- I do think there is value in "utility microservices". For example: a microservice to send email, a microservice to filter spam, etc. These are the next level libraries (because they do need to run as services 24/7). Management usually don't like these kind of microservices because these "domains" usually don't belong to any particular team, so managers cannot "own" their success.

- I don't think there's much value in building microservices for the core of your business (e.g., a checkout microservice, a payments microservice, etc.). The usual argument management gives is: "we'll make teams more independent and they will be able to delivery stuff faster than with a monolith!". While this is sometimes true, "faster software delivery" is not on my top list of prioritites when it comes to build software.

Re: Ask HN: How do you keep track of releases/deployments of dozens micro-services?

#133

Post by sokoloff »

sokoloff wrote: post #77
atrandom wrote: post #39

Earlier quoted context omitted.

This sounds interesting! is there any more detailed write up that you link me to? Thanks!

I looked briefly (and I could have sworn I posted our "nine rules" on HN years ago, but I couldn't find it in a quick search). I'll look again later tonight more thoroughly to see if I've posted the mechanisms and restrictions publicly anywhere before. If I haven't, I'll try to dig it out of our old dev doc system and post them here, but I can't make any promises as the docs I recall are now over a decade old, so I'm…

The internal docs for this are not on any of our documentation systems that we've moved to zero-trust (as they're 12 years old and unchanged for 5+ years). I will probably be able to retrieve them when we're back in the offices; shoot me an email (in my profile) and I'll find a way to get something over to you with some significant delay.

Re: Ask HN: How do you keep track of releases/deployments of dozens micro-services?

#134

Post by taneq »

bhouston wrote: post #4

We just made all the microservices into one big monorepo and we deploy all at the same time. To be honest we tried to avoid the monorepo but it was hellish. Maybe if each microservices was larger and our team was larger but then are they microservices any more?

I was going to make some sarcastic response to the effect of "iT's EvErGReeN" but this is basically the only answer. You don't do individual releases of your microservices any more than you do individual releases of the classes/functions in your C++ project.

Re: Ask HN: How do you keep track of releases/deployments of dozens micro-services?

#135

Post by computershit »

klohto wrote: post #97
computershit wrote: post #48

Earlier quoted context omitted.

Have you looked into Jenkins-X at all? I'm at a point where I'm starting to adopt GitOps and I'm torn between Flux and (what I consider) a far more opinionated but pretty elegant solution in JX.

I did, it’s overly complicated for what I need (single team, apply YAMLs in git repo, specific branch, tagging). I see the industry using mostly Flux and ArgoCD and I really don’t want anything Jenkins related in infra again.

Yeah I am leaning that direction too. Thanks for the reminder about Argo.

Re: Ask HN: How do you keep track of releases/deployments of dozens micro-services?

#136

Post by brianwawok »

giantg2 wrote: post #123
brianwawok wrote: post #121

Earlier quoted context omitted.

Lunch hour. After bedtime. Morning. You are responsible for making yourself better. Do it.

Nah, I've done everything right for years. No point in wasting my time if there's no reward.

Okay, I read your post as you were stuck in a dead end job with no alternatives.

If you are fine to stay in that exact role till you die, then of course there is no reason to do anything else. It does get harder and harder to find time to learn as you age though, saw this first hand with 60 year old cobol programmers 20 years out of date trying to learn Java...

Re: Ask HN: How do you keep track of releases/deployments of dozens micro-services?

#137

Post by k8s_Hero »

Have you heard of Komodor? They just held a joint webinar with Epsagon regarding this very issue! You can see a recording of the webinar here: https://www.youtube.com/watch?v=J32ZoiRVvPg Or the product overview here: https://www.youtube.com/watch?v=Qgio3vF1sPE&t=6s

Re: Ask HN: How do you keep track of releases/deployments of dozens micro-services?

#138

Post by selphy1 »

We use Octopus Deploy. On commit it autodeploys to dev and sends the team a slack message ([environment] version x (previous was y) deployed by z). Prod deployments are also done through octopus but "manually" by the team when we are ready to make a release. Usually every week or two.

Re: Ask HN: How do you keep track of releases/deployments of dozens micro-services?

#140

Post by JensRantil »

bhouston wrote: post #4

We just made all the microservices into one big monorepo and we deploy all at the same time. To be honest we tried to avoid the monorepo but it was hellish. Maybe if each microservices was larger and our team was larger but then are they microservices any more?

"Microservices" is a tool to solve problems. The goal isn't microservices, the goal is to solve those problems. Example of problems it might solve: Unclear ownership, slow innovation, unstable software or orthogonal scaling of features.
Post reply on HN