Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: How do you keep track of releases/deployments of dozens micro-services?

news.ycombinator.com

Post reply on HN
Print view View original

71–80 of 140 posts

Re: Ask HN: How do you keep track of releases/deployments of dozens micro-services?

#71

Post by bhouston »

lovedswain wrote: post #14
bhouston wrote: post #4

We just made all the microservices into one big monorepo and we deploy all at the same time. To be honest we tried to avoid the monorepo but it was hellish. Maybe if each microservices was larger and our team was larger but then are they microservices any more?

The biggest difficulty I've experienced is "librification", where some common code ends up in a little library, and soon that library is not so little any more, and not long after starts to look like half of every service. I can maintain discipline when working on small systems alone, but on a team there will always be one lazy person or urgent need which means eventually some shared component gains enough gravity to…

These libraries already exist whether you write them or you use someone else's. In our case most of our micro services are node.js based so Koa is in every microservice and we use middleware for authentication -- and thus if the authentication system evolves (moving to JWT or a microservice gateway) we have to evolve that middleware everywhere.

Same with our consistent logging system.

Libraries are better than unique code everywhere for the same task - allows you to fix a bug once and to do consistency checking.

Re: Ask HN: How do you keep track of releases/deployments of dozens micro-services?

#72

Post by giantg2 »

edoceo wrote: post #70
giantg2 wrote: post #51

Earlier quoted context omitted.

"If you need to keep track its probably too late. What makes them services is that they should be able to be deployed without a bunch of orchestration of other services." If only you could tell my bosses/architects that. They won't listen to me. Edit: why downvote?

I'm not a downvoter but maybe its cause "they wont listen" my theory is your presentation is not compelling. Was your CBA clear? What risk/reward metrics did you highlight?

I'm a midlevel dev at a large company. I don't even have access to make a presentation to the true stakeholders. I can only make suggestions to my lead/boss and have them move it up the chain. On a side note, I've tried moving up th chain when I felt appropriate, but apparently a SQL injection vulnerability with full schema level privileges that was not being prioritized for remediation was not important enough to waste my department head's time.

Sometimes they take my suggestion, sometimes they are already working on it behind the scenes, and sometimes they go nowhere. In this case, they are already measuring most of the metrics like cycle times and mean time to recovery, etc. They already stated that they want microservices and have been building them out. The problem is that they implement it wrong and have no interest in re-architecting. Many of the apps are rewrites of legacy apps. Instead of evaluating the underlying business process, they just want us to build it the same in the new tech but use "microservices". The problem is that some of the business process was designed around the restrictions of the old technology or old industry norma. We should be evaluating the business process before building the technical system, otherwise we will continue to bake in these old constraints and not fully leverage the capabilities of new technology.

Edit: looks like I made someone angry since this is downvoted too.

Re: Ask HN: How do you keep track of releases/deployments of dozens micro-services?

#74

Post by theptip »

johnx123-up wrote: post #55

From the previous discussion https://news.ycombinator.com/item?id=16166645 1. https://github.com/gocd/gocd - 6.1k stars 2. https://github.com/Shopify/shipit-engine - 1.2k stars 3. https://github.com/guardian/riff-raff - 252 stars 4. https://github.com/ankyra/escape - 201 stars 5. https://github.com/kiwicom/crane - 92 stars 6. https://github.com/tim-group/orc - 34 stars 7. https://github.com/wballard/starphleet - 19 s…

ArgoCD could be on the list too.

Re: Ask HN: How do you keep track of releases/deployments of dozens micro-services?

#75

Post by drbojingle »

twh270 wrote: post #35
drbojingle wrote: post #5

I'm curious to see how micro service management evolves over time and learn whether or not it will become viable for small companies. Hopefully one day its as cheap as writing a function is. As it stands, with what I've seen and heard about microservices, I'd say the best way to deal with micro service anything is to use a monolith 90% of the time and for the rest of the time make sure your micro service could stand…

Right now I don't think microservice management is 'viable' even at larger companies. The custom deployment scripts & yaml to manage building, package/artifact repository, versioning, and deployment tends to be Too Damn Big, at least at the shops I've seen.

Oof. I haven't seen micro services much but, what I have seen makes me wonder what value people are actually expecting to get back. I hope the next iterations of this idea work better.

Re: Ask HN: How do you keep track of releases/deployments of dozens micro-services?

#76

Post by beastcoast »

BurningFrog wrote: post #43
bhouston wrote: post #4

We just made all the microservices into one big monorepo and we deploy all at the same time. To be honest we tried to avoid the monorepo but it was hellish. Maybe if each microservices was larger and our team was larger but then are they microservices any more?

One "wisdom" I hear is that the benefit of microservices is organizational: One microservice per team, so you cut down on intra-team friction, and the team can manage their own releases.

What people miss is that micro services/SOA is an organizational concept more than anything. At Amazon, SOA is tied to the concept of the two pizza team. Each 2PT completely owns a collection of related services, owns the roadmap for those services, and can make deployments independently of any other team. If your company doesn’t have enough engineers to justify at least say 5-10 scrum teams then you might not be large enough to need microservices.

Re: Ask HN: How do you keep track of releases/deployments of dozens micro-services?

#77

Post by sokoloff »

atrandom wrote: post #39
sokoloff wrote: post #20

Earlier quoted context omitted.

Not in the context of micro-services, but we ran our production DB for years in a “both N and N+1 work” by following a few simple rules which turn out to be not that restrictive in practice. Short version: have DB1 hold the transactional data (data generated while running the system). Have DB2a have the release-bound data (data about and connected to the code itself-settings, prices, whatever). Have DB2a have views o…

This sounds interesting! is there any more detailed write up that you link me to? Thanks!

I looked briefly (and I could have sworn I posted our "nine rules" on HN years ago, but I couldn't find it in a quick search).

I'll look again later tonight more thoroughly to see if I've posted the mechanisms and restrictions publicly anywhere before. If I haven't, I'll try to dig it out of our old dev doc system and post them here, but I can't make any promises as the docs I recall are now over a decade old, so I'm not fully sure they exist any more. :)

Re: Ask HN: How do you keep track of releases/deployments of dozens micro-services?

#78

Post by meritt »

wikibob wrote: post #73

Don’t have dozens of micro services. This is a serious comment.

But that negatively impacts the revenue of the various cloud providers and puts devops engineers back into the same unemployment line when their job title was sysadmin.

Re: Ask HN: How do you keep track of releases/deployments of dozens micro-services?

#79

Post by ethbr0 »

giantg2 wrote: post #72
edoceo wrote: post #70

Earlier quoted context omitted.

I'm not a downvoter but maybe its cause "they wont listen" my theory is your presentation is not compelling. Was your CBA clear? What risk/reward metrics did you highlight?

I'm a midlevel dev at a large company. I don't even have access to make a presentation to the true stakeholders. I can only make suggestions to my lead/boss and have them move it up the chain. On a side note, I've tried moving up th chain when I felt appropriate, but apparently a SQL injection vulnerability with full schema level privileges that was not being prioritized for remediation was not important enough to wa…

Meta comments tend to get downvoted. Try editing the complaints out?

Re: Ask HN: How do you keep track of releases/deployments of dozens micro-services?

#80

Post by jupp0r »

giantg2 wrote: post #72
edoceo wrote: post #70

Earlier quoted context omitted.

I'm not a downvoter but maybe its cause "they wont listen" my theory is your presentation is not compelling. Was your CBA clear? What risk/reward metrics did you highlight?

I'm a midlevel dev at a large company. I don't even have access to make a presentation to the true stakeholders. I can only make suggestions to my lead/boss and have them move it up the chain. On a side note, I've tried moving up th chain when I felt appropriate, but apparently a SQL injection vulnerability with full schema level privileges that was not being prioritized for remediation was not important enough to wa…

Time for a job change? Lots of red flags.
Post reply on HN