Live data from Hacker News

Why Segment Went Back to a Monolith

infoq.com

291–300 of 328 posts

Re: Why Segment Went Back to a Monolith

#291

Earlier quoted context omitted.

The last place I worked that split the devs into UI and backend teams was in a sort of slapstick comedy situation. Nothing ever shipped on time because the front end and backend could never quite talk to each other or needed elaborate conversations to do the simplest of things. This was our new flagship project, I got consolidated in from another team and ended up as a lead not long after. We had been doing some UML…

Can you share examples of your data flow diagrams? Do any open source projects share these documents?

Aside, take a look at tools like PlantUML as a way to create your diagrams. It's higher-level than, say, rolling everything with Graphviz, while easier to share and edit than a bunch of PowerPoint/Visio/etc. files.

The great thing about generated-diagrams is that you can easily store and version the original text representation along with the code it describes or applies to.

https://plantuml.com/

Re: Why Segment Went Back to a Monolith

#292
post #208
post #170

Earlier quoted context omitted.

> Why can't you have both independently deployed microservices and a shared code base? This is what everyone does, so I can't even comprehend what Segment was doing. Maybe they were deploying a fleet of microservices inside a monolithic deployment? If so, there's no wonder it failed.

We do separate code repos, my last place did separate repos, place before that did monolith(s) but still did separate repos for anything not in the same monolith. I'm pretty sure it's more common to do separate repos, rather than mono-repo, for separate services. Seems to me, though, the problem is people trying so hard to reuse code. That's the main problem cited in the article. People get really gung-ho about reusi…

"People get really gung-ho about reusing code and creating shared libraries, but reusing code is actually bad most of the time. You should strive to only depend on things that you can reasonably expect to not change" -- things changing is one of the main reasons you want to share code.

Re: Why Segment Went Back to a Monolith

#293
post #203

I see a lot of places that seem to either think that: 1. Microservices will let them ship things faster or 2. It's microservices everywhere or nothing Microservices might let you ship faster if you are really good at deciding where to draw the lines between services and really good at managing multiple deployment pipelines and all the infra - that's a pretty tough ask. Also, if you have a monolith it's perfectly fine…

I'm yet to do micro-services at all at work or outside. I count myself lucky :-).

Re: Why Segment Went Back to a Monolith

#294
post #19

I think that the problem here was that they were fighting against Conway's Law: https://en.wikipedia.org/wiki/Conway%27s_law > Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure. I think microservices work well in organizations that are big enough to have a team per microservice. However if you've just split your monol…

Yeah, the problem with microservices is because the organisation structure is wrong. I’ve literally heard every excuse about microservices at this point. My architecture is better but it doesn’t have a snappy name; it’s called the smallest possible number of services that can be reasoned about and network partitions are NOT necessary to create bounded contexts in a codebase, often just a directory is FINE.

Or an NPM package works nicely (or .NET assembly, Ruby Gem, Java whatever, etc.)

Re: Why Segment Went Back to a Monolith

#295
post #242
post #203

I see a lot of places that seem to either think that: 1. Microservices will let them ship things faster or 2. It's microservices everywhere or nothing Microservices might let you ship faster if you are really good at deciding where to draw the lines between services and really good at managing multiple deployment pipelines and all the infra - that's a pretty tough ask. Also, if you have a monolith it's perfectly fine…

This, this, this! It's been said elsewhere in these comments, but the term "micro"-services really do them a disservice, like it's expected that you need to break your application up into little pieces, to eliminate complexity. But many applications are inherently complex, and splitting them up isn't going to get you anywhere. I've been trying to advocate for a "solar system model of services", where you have a big c…

I’ve been pushing for a similar model for years as well but called but used rings to model it. Might have to try your solar system model and see if I have better luck.

Re: Why Segment Went Back to a Monolith

#296

If you take a look at some of Segment's open source code, it isn't hard to see why they wound up struggling with microservices. It looks like they subscribe to the "left-pad" style of software development. They have tons of repositories that have less than 10 lines of code. They have a two line repository for calling preventDefault [0], a four line repository for getting the url of a page[1], and a eight line reposit…

What does segment.io do and what does your company do?

Re: Why Segment Went Back to a Monolith

#297

If you take a look at some of Segment's open source code, it isn't hard to see why they wound up struggling with microservices. It looks like they subscribe to the "left-pad" style of software development. They have tons of repositories that have less than 10 lines of code. They have a two line repository for calling preventDefault [0], a four line repository for getting the url of a page[1], and a eight line reposit…

Wow, I figured there was more to it than the article was saying. This is insane!

Re: Why Segment Went Back to a Monolith

#298
post #179

Earlier quoted context omitted.

A distinction without a difference.

There is a difference. OOP is just one of many tools to help accomplish a task. Many other tools as well. It is up to the people how to use tools for a job and what tools for what job. You equating OOP with the dangerous things that should be kept away has no basis in programming.

I want taking a side here, just noting a tradeoff inherent to the choice in programming paradigm. When you make effective abstractions, you make your program bendy in all the right places. It's easy to add new objects where you will need them, and it's easy to extend behavior in places you need to do that.

Of course, if you guess wrong, you're totally fucked. Well, either that, or you are smart and see it coming in time to rewrite the code that put the complexity in the wrong place.

Hope that helps with the context. I'm not some anti-OOP zealot, and those do exist.

Re: Why Segment Went Back to a Monolith

#299
post #296

If you take a look at some of Segment's open source code, it isn't hard to see why they wound up struggling with microservices. It looks like they subscribe to the "left-pad" style of software development. They have tons of repositories that have less than 10 lines of code. They have a two line repository for calling preventDefault [0], a four line repository for getting the url of a page[1], and a eight line reposit…

What does segment.io do and what does your company do?

Sure. I'm the founder of freshpaint.io.

The premise of segment.io is that there are lots of tools that take user behavior data from your site and it's a lot of work to integrate them all. For example, when a user signs up, you may tell multiple different tools that a user signed up:

  - You tell Mixpanel so you can create graphs of how many people signed up.
  - You tell Google Ads so Google knows a specific ad just resulted in a conversion.
  - You tell Optimizely so it knows a specific page from an A/B test just converted.
Before Segment, you would need to write code for each tool separately. This doesn't sound so bad, but it becomes a pain when you have dozens of different tools and dozens of different events you want to track. With Segment, you only need to tell Segment that someone logged in. Segment will then send that event to all your other tools. You can think of it as like a multiplexer for user behavior data. Instead of integrating 10 tools, you just integrate Segment.

The challenge with Segment is you need to write custom code for every action you want to send into Segment. This is bad for two reasons. Usually the end user of Mixpanel/Google Ads/Optimizely is a non-technical person that doesn't know how to write code. What they have to do is file a Jira ticket for an engineer to add a new bit of tracking to the website. Depending on the size of the organization, that person can end up waiting two weeks or more in order to start tracking a new bit of data from the website.

The other challenge is people often don't know what to track ahead of time or forget to track something important. For example, if you launched a new feature two weeks ago and forgot to setup tracking on it, there's no way to get that data back.

Freshpaint solves these problems by automatically collecting every user action upfront. Anytime someone clicks a button on your site, that fires an event in Freshpaint that someone clicked that button. You can then use Freshpaint's point and click UI to say that whenever someone clicks that button that is a "login" event. Then you can send that event into different tools. This is great because the point and click UI allows a non-technical user to send data into different tools and because we track everything up front, even if you forgot to track something, Freshpaint will still have recorded every instance of that action. That way, even if you decided you want to start tracking some action today, you can use our "time travel" functionality and recover every instance of that action since you installed Freshpaint.

Re: Why Segment Went Back to a Monolith

#300
post #183
post #174

Earlier quoted context omitted.

If Netflix were a monolithic, then the whole system would collapse instead of degrading.

A monolith can run distributed and be scaled across data centers. It's not a mainframe application with one host. One process can crash without affecting system stability.

Where you can get into trouble is if the problem affects all your instances across the entire cluster. If all your organization's code is running in the same process, if any of that code has a severe memory leak or other serious issue, it could impact the stability of everything else.

Not to say that wouldn't happen for SPOF microservices (e.g. your auth servers), but the surface area is potentially larger for monoliths.

Post reply on HN