Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Goodbye Microservices: From 100s of problem children to 1 superstar

segment.com

Post reply on HN
Print view View original

681–690 of 782 posts

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#681

Post by wsy »

crimsonalucard wrote: post #514
wsy wrote: post #366

Earlier quoted context omitted.

It's a bit like asking to replace medical doctors with science and theory. We need science and theory, and skilled architects who can apply it to the problem at hand.

The practices of Doctors are based off of science and theory. Design patterns and microservices while technical are not based off of science. They are ideas without quantitiative basis or science.

I agree that we need more empirical investigations of software 'diseases' and 'cures'.

That being said, practical medicine is much less scientific than many think. There is a lot of master/apprentice learning going on, just as in software engineering.

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#682

Post by ergo14 »

ng12 wrote: post #189
ergo14 wrote: post #82

Earlier quoted context omitted.

Yes, because between jQuery and react there was nothing in between, and functional programming or singletons for state were unheard of. /S On other hand you still have things broken in react not to mention few years with uncertain patent situation. https://twitter.com/sveltejs/status/999704064937156611 sums this up pretty well. Programming world is in a constant flux, now you have lit-html that gives you react in 5kb…

> lit-html that gives you react in 5kb without vdom overhead Uh, lit-html seems like just another templating library. Marginally better than something like Soy. Calling it "React" is pretty reductive.

With lit-element you get native elements that are similar to react elements. And only parts of the dom that changed actually do mutations - like with vdom.

https://github.com/Polymer/lit-element#minimal-example

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#683

Post by base698 »

lolive wrote: post #614
ChicagoDave wrote: post #605

I’ve been tracking the comments and my sense is that almost no one here believes the business domain drives the technical solution. Microservices, when constructed from a well-designed model, provides a level of agility I’ve never seen in 33 years of software development. It also walls off change control between domains. My take from the Segment article is that they never modeled their business and just put services…

As a microservice agnostic, i wonder how you can deal elegantly with transactions across services, concurrent access & locks, etc. [Disclaimer: i have not read the article yet]

There are patterns to do with this: Saga Pattern and Command Query Responsibility.

In general the separation of the domain should be such that you don't need a transaction across services.

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#684

Post by nomercy400 »

HolyHaddock wrote: post #644
lolive wrote: post #614

Earlier quoted context omitted.

As a microservice agnostic, i wonder how you can deal elegantly with transactions across services, concurrent access & locks, etc. [Disclaimer: i have not read the article yet]

You may also want to look into the Saga pattern - I found https://www.youtube.com/watch?v=xDuwrtwYHu8 to be a handy high level overview for applying it to microservices. Although personally, I've never felt the need to try and apply it specifically, but the idea is interesting.

Now that was an interesting talk. Thank you.

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#685

Post by inetknght »

testplzignore wrote: post #139
athenot wrote: post #4

Earlier quoted context omitted.

Agreed. Reading about their setup and comparing with some truly large scale services I work with, I'm left with the idea that Segment's service is roughly the size of one microservice on our end. Perhaps the takeaway is don't go overboard with fragmenting services when they conceptually fulfill the same business role. And regardless of the architecture of the system, there are hard state problems to deal with in asso…

The most telling fact is that it "took milliseconds to complete running the tests for all 140+ of our destinations". I've never worked on a single service whose tests ran that fast, given that the time spent by the overhead of the test framework and any other one-time initialization can take a few seconds just itself. It's great to have tests that run fast, but that's a bit ridiculous. Some rules of thumb I just came…

> I've never worked on a single service whose tests ran that fast, given that the time spent by the overhead of the test framework and any other one-time initialization can take a few seconds just itself. It's great to have tests that run fast, but that's a bit ridiculous.

It's not ridiculous. It's good.

I work on an analysis pipeline with thousands of individual tests across a half dozen software programs. Running all of the tests takes just a few seconds. They run in under a second if I run tests in parallel.

If your tests don't run that fast then I suggest you start making them that fast.

I'd be willing to bet that if you learned (or hired someone with the knowledge of) how to optimize your code, you could get some astounding performance increases in your product.

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#686

Post by inetknght »

NickBusey wrote: post #72
thermodynthrway wrote: post #3

I've always said if the Linux kernel can be a giant monolith, in C no less, than there's maybe 100 web applications in the world that need to be split into multiple services. I've worked with microservices a lot. It's a never-ending nightmare. You push data consistency concerns out of the database and between service boundaries. Fanning out one big service in parallel with a matching scalable DB is by far the most sa…

Right, but the thing that makes Linux actually useful isn't really the kernel is it? I would say what makes it useful is all the various small, targeted programs (some might call them microservices) it lets you interact with to solve real world problems. If Linux tried to be an entire computing system all in one code base, (sed, vim, grep, top, etc., etc.) what do you think that would look like code base/maintainabil…

> I would say what makes it useful is all the various small, targeted programs

You mean GNU coreutils?

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#687

Post by inetknght »

alexrbarlow wrote: post #2

I'm not sure what they've been left with is a monolith after all. I would say they just have a new service, which is the size of what they should have originally attempted before splitting. In particular, as to their original problem, the shared library seems to be the main source of pain and that isn't technically solved by a monolith, along with not following the basic rule of services "put together first, split la…

> the basic rule of services "put together first, split later"

Is this rule mentioned or discussed somewhere? A quick google search links to a bunch of dating suggestions about splitting the bill. Searching for the basic rule of services "put together first, split later" reveals nothing useful.

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#688

Post by rsynnott »

9935c101ab17a66 wrote: post #529
barkingcat wrote: post #364

Earlier quoted context omitted.

FTP workflows are so extremely widely used in today's world that it's inconceivable you think it's behind us.

Maybe in frontenddev/small companies/small largely static websites FTP pervades, but I'd argue that most developers that deal with any real complexity probably use some kind of source control system to 'share' files rather than some kind of FTP program. And I'd be very surprised if relative FTP usage hadn't decreased significantly over the last 3-4 years (But this is all quite anecdotal, and I could be wrong). Someth…

In some industries FTP is used quite heavily as part of batch workflows. Payment processors and things seem to love it.

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#689

Post by inetknght »

mekazu wrote: post #667
mpweiher wrote: post #628

Earlier quoted context omitted.

> never worked on a single service whose tests ran that fast I'd say you've never had good tests. I have a test-suite for a bunch of my frameworks that dates to the mid 90s, with tests added regularly with new functionality. It currently takes 4 seconds total for 6 separate frameworks and 1000 individual tests. Which is actually a bit slower than it should be, it used to take around 1-2 seconds, so might have to dig…

Unit tests that don’t read or write to disk and don’t try thousands of repetitions of things should be bleeding fast, but the most useful integration tests that actually help find faults (usually with your assumptions about the associated APIs) often need interaction with your disk or database or external service and tend to take a bit more than a few seconds. I find you need both.

I have tests which verify DNA analysis. The test data vectors are large -- a few hundred MB here, a couple GB there. The hundreds of tests that use these test vectors still run in a few seconds.

If you're using a tape drive or SD cards, sure. But even a 10 year old 5400RPM on an IDE connection should be able to satisfy your tests' requirements in a few seconds or less.

I suspect your tests are just as monolithic as you think microservices shouldn't be. Break them down into smaller pieces. If it's hard to do that, then redesign your software to be more easily testable. Learn when and how to provide static data with abstractions that don't let your software know that the data is static. Or, if you're too busy, then hire a dedicated test engineer. No, not the manual testing kind of engineer. The kind of engineer who actually writes tests all day, has written thousands (or hundreds of thousands) of individual tests during their career. And listen to them about any sort of design decisions.

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#690

Post by MPSimmons »

stickfigure wrote: post #581
Floegipoky wrote: post #558

It seems like splitting into separate repos was a rash response to low-value automated tests. If tests don't actually increase confidence in the correctness of the code they're negative value. Maybe they should have deleted or rewritten a bunch of tests instead. Which is what they did in the end anyway. >> A huge point of frustration was that a single broken test caused tests to fail across all destinations. When we…

it seems like they made some mistakes, microed their services in a knee-jerk attempt to alleviate the symptoms of the mistakes, realized microservices didn't fix their mistakes, finally addressed the mistakes, then wrote a blog post about microservices. That seems... appropriate? This is the general problem with the microservices bandwagon: Most of the people touting it have no idea when or why it's appropriate. I on…

Agreed. Classic case of cargo-cult.
Post reply on HN