Why Segment Went Back to a Monolith • Hacker News Board

Re: Why Segment Went Back to a Monolith

#181

Post by FpUser » Wed, Apr 29, 2020, 4:25 PM UTC

ChrisMarshallNY wrote: post #74

FpUser wrote: post #34

Earlier quoted context omitted.

"Turning everything into an object can make a small program into a big program, so it’s maybe not such a good idea for small-scale stuff." In my experience OOP actually makes programs smaller. Assuming of course they have good programmers/architects and the program itself is larger than "Hello world".

Don't get me wrong. I love OOP, and have been using it since before it was cool. It's been a standard wrench in my toolbox for decades. In fact, I have been running into folks, these days, that don't understand it, as, apparently, OOP is becoming "uncool." I've always been a "right tool for the right job" kind of guy. I started off with ML (Machine Language , not Machine Learning ). I am quite comfortable, sitting do…

Lots of words. What's the conclusion? OOP is bad? Or maybe it is incompetent people who manage to f.. things up no matter what you give them or people with the agenda going on holy crusades?

Re: Why Segment Went Back to a Monolith

#182

Post by dx034 » Wed, Apr 29, 2020, 4:26 PM UTC

musingsole wrote: post #121

hedora wrote: post #105

Are there any case studies where microservices went well? From an end user perspective, Netflix runs in “constantly degraded” mod. From an engineering perspective, they track “number of successful stream starts”, instead of percentage of the time 100% of their services are working. That’s a huge red flag. As a researcher, the monitoring and fault-propagation / modeling work they’ve done to get it to stay up at all is…

> Are there any case studies where microservices went well? You answered your own question with Netflix. While you're right that it's not clear Netflix would've needed to develop their chaos monkey tooling and the like, it's not clear at all at that an equivalent system is possible as a monolith. Even if a monolithic Netflix system were technically possible, it's not clear if a monolithic system would be organization…

Why wouldn't it be possible? It doesn't mean you can't modularize the application. And scaling can also work well for monoliths. I don't see how Netflix' service (browsing the catalog, serving content, reencoding videos) can't be done in a monolith.

Re: Why Segment Went Back to a Monolith

#183

Post by dx034 » Wed, Apr 29, 2020, 4:27 PM UTC

gowld wrote: post #174

hedora wrote: post #105

Are there any case studies where microservices went well? From an end user perspective, Netflix runs in “constantly degraded” mod. From an engineering perspective, they track “number of successful stream starts”, instead of percentage of the time 100% of their services are working. That’s a huge red flag. As a researcher, the monitoring and fault-propagation / modeling work they’ve done to get it to stay up at all is…

If Netflix were a monolithic, then the whole system would collapse instead of degrading.

A monolith can run distributed and be scaled across data centers. It's not a mainframe application with one host. One process can crash without affecting system stability.

Re: Why Segment Went Back to a Monolith

#184

Post by FpUser » Wed, Apr 29, 2020, 4:27 PM UTC

dmix wrote: post #80

FpUser wrote: post #30

Earlier quoted context omitted.

So you are saying something in line of: let's increase our development staff X-fold and then we can finally do the same thing that way fewer people doing just fine right now?

Just because monoliths may have diminishing returns at certain team/project scale doesn't mean the scale itself is the problem...

The problem is with people trying to do "cool" things when completely unwarranted

Re: Why Segment Went Back to a Monolith

#185

Post by jillesvangurp » Wed, Apr 29, 2020, 4:30 PM UTC

This is a discussion on pretty much every team I've been on for the last 5 or so years. I agree mostly this stuff is done for the wrong reasons.

IMHO it doesn't matter if you replace microservices by components, corba objects, rpc objects, soap services, etc. It all boils down to chopping your software into smaller bits that than immediately start having a need for sending messages between them, finding each other, defending their boundaries, etc.

So, the first mistake would be assuming this is a new problem to think about. It's not. You can find similar debates about how to chop up software ever since people moved beyond just having their code ship in punch card form.

The right discussion to have would be first deciding whether you want to break down by your logical architecture so that your deployment architecture reflects that or your organization diagram (aka. Conway's law). Then the next step is deciding whether your primary goal is network isolation of unrelated chunks of code or enabling asynchronous development of these chunks of code (if so, there are other solutions). Usually it boils down to, again, Conway's law: different teams just don't want their stuff to depend on shit happening in another team because of internal bureaucracy and hassle.

Now say you have a valid business reason or technical reason for actually wanting to have different stuff be isolated (e.g. for scaling reasons or security reasons). The next step is deciding whether this means you also want to break up your code base. Monorepos and microservices are a thing. Look at e.g. lerna for node.js, or multi module gradle projects on the jvm. In Go this is well supported as well. If you're really sure that you don't want micro services because of Conway's law there are lots of valid reasons for having a well structured mono repo with a bit of reuse of shared functionality, a simplified review process and more visibility in what is happening.

IMHO people do this for completely the wrong reasons; like wanting to try out some new language, organizational issues, etc. that ultimately result in fragmented code bases, lots of devops overhead and complexity (it's never simple or cheap), lots of project management overhead, etc. You pay a price.

Re: Why Segment Went Back to a Monolith

#186

Post by capableweb » Wed, Apr 29, 2020, 4:31 PM UTC

DrScientist wrote: post #165

ajsharp wrote: post #157

I see quite a few people defending microservices; the org is the problem, they must not have written the software correctly, etc. Most org structures are not great. Most software is not great. If you expect the exception to be the rule you're setting yourself up for a career full of disappointment. Microservices are a modern re-branding of service-oriented architecture, but 'microservices' sounds cuter and less like…

> split along domain lines with minimum inter-service dependencies. Exactly, and done right that quite often means big 'microservices'. All too often I see the 'functional programming disease' where the aim is to deconstruct to the smallest possible reusable functions ( 'micro' services right? ), often prematurely, creating high levels of compositional complexity and with zero tools to help you understand how the act…

"Yep each single microservice is simple - ..." but the whole is not.

I always find it more interesting what's _not_ in the single microservices, the stuff you do see. When you make a diagram with boxes and arrows, the interesting stuff would be the arrows, not the boxes themselves.

Re: Why Segment Went Back to a Monolith

#187

Post by dx034 » Wed, Apr 29, 2020, 4:31 PM UTC

Autowired wrote: post #92

pestaa wrote: post #78

Earlier quoted context omitted.

2 pizza team? Well finally I might get my own microservice after all.

This metric is also unsuitable for Europe, where generally pizzas are individual.

You can get party pizzas but in that case the two pizza team might be a bit large.

Re: Why Segment Went Back to a Monolith

#188

Post by hinkley » Wed, Apr 29, 2020, 4:32 PM UTC

username3 wrote: post #166

hinkley wrote: post #152

Earlier quoted context omitted.

The last place I worked that split the devs into UI and backend teams was in a sort of slapstick comedy situation. Nothing ever shipped on time because the front end and backend could never quite talk to each other or needed elaborate conversations to do the simplest of things. This was our new flagship project, I got consolidated in from another team and ended up as a lead not long after. We had been doing some UML…

Can you share examples of your data flow diagrams? Do any open source projects share these documents?

Mostly these were white boarded, but essentially I/we would draw a collaboration diagram (although I could have sworn these used to be called something else). They showed what data was needed to make certain decisions (eg, a conditional drop down that is populated based on another piece of data, or complex validation steps) and where to get data that already existed.

Re: Why Segment Went Back to a Monolith

#189

Post by ljm » Wed, Apr 29, 2020, 4:32 PM UTC

nickcw wrote: post #19

I think that the problem here was that they were fighting against Conway's Law: https://en.wikipedia.org/wiki/Conway%27s_law > Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure. I think microservices work well in organizations that are big enough to have a team per microservice. However if you've just split your monol…

In all honesty, I think the monolith/microservice distinction misses the point a little bit.

It's inevitable that the longer the codebase exists, the more difficult it is to maintain. It's a battle that you can't necessarily win and it's turtles all the way down as your dependencies, and their dependencies, tackle the same issues.

All it takes is one or two roughly defined APIs and you've already created the nucleation point for ever-more tech debt, and while you'll be able to tame some of it you won't manage all of it due to business requirements, or other teams depending on private APIs to save time, or whatever else you can imagine. Switch the architecture and you'll either have all your problems bunched in one codebase, or you'll have distributed your problems all over the place.

I'd go as far as saying that a perfect monolith and a perfect distributed architecture are theoretical ideals that require perfect communication to build them.

Re: Why Segment Went Back to a Monolith

#190

Post by strictfp » Wed, Apr 29, 2020, 4:32 PM UTC

mjburgess wrote: post #48

strictfp wrote: post #44

Earlier quoted context omitted.

Noo! Building teams around software components cements your architecture and prevents most cross-cutting improvements. I'll claim that splitting a well-structured monolith into microservices will always make it less maintanable, but it might be worth it if you need to for some reason like elasticity or failure tolerance. But for the love of god, keep the design open. Don't tie the existence of internal software compo…

> Don't tie the existence of internal software components to peoples livelihoods. The claim is that such ties, at the macro-structure level, are inevitable and exist regardless. The point is then to determine the best way either to restructure the organisation, or, the code base, to cope.

I think the ties arise because people are actively seeking areas of responsibility. Software components are an obvious grab if your eyes are on the software specifically. But there are other ways of dividing your teams; based on for instance customers, use-cases, aspects of the code (performance, security).

The problem is that the software usually keeps expanding until programmers find it hard to cope. If you split teams up so that some people are only concerned with a certain part of the codebase, chances are you are going to grow the size of the codebase by a quite large factor.

I think there should be an incentive in place to keep the codebase small and understandable by most.