Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Goodbye Microservices: From 100s of problem children to 1 superstar

segment.com

Post reply on HN
Print view View original

691–700 of 782 posts

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#691

Post by inetknght »

hinkley wrote: post #319
noir_lord wrote: post #16

Earlier quoted context omitted.

Not to mention you have 2^100 states of on/off (~1.2676506e+30)

I work on that project. Every time some idiot starts talking about 'code coverage' my face turns red. Our code coverage is 1e-10%. Don't talk to me about this 70% bullshit.

Said with less vitriol:

It's not just code coverage that matters. It's the code path selection that matters. If you have a ton of branches and you've evaluated all of them once then yeah you sure might have 100% "coverage". But you have 0% path selection coverage since a single invokation of your API might choose true branch on one statement, false branch on another statement, and a second invokation might choose false branch on the first and true on the second.

While the code was 100% tested, the scenarios were not. What happens if you have true/true or false/false? That's not tested.

There's a term for this but I forgot what it is and don't care to go spelunking to find it.

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#692

Post by mpweiher »

mekazu wrote: post #667
mpweiher wrote: post #628

Earlier quoted context omitted.

> never worked on a single service whose tests ran that fast I'd say you've never had good tests. I have a test-suite for a bunch of my frameworks that dates to the mid 90s, with tests added regularly with new functionality. It currently takes 4 seconds total for 6 separate frameworks and 1000 individual tests. Which is actually a bit slower than it should be, it used to take around 1-2 seconds, so might have to dig…

Unit tests that don’t read or write to disk and don’t try thousands of repetitions of things should be bleeding fast, but the most useful integration tests that actually help find faults (usually with your assumptions about the associated APIs) often need interaction with your disk or database or external service and tend to take a bit more than a few seconds. I find you need both.

Sure. I'd venture to say that integration tests should be fewer than unit tests, see hexagonal etc. Hopefully those external interfaces are also more stable, so they don't need to be run as often.

I tend to use my integration tests also as characterization tests that verify the simulator/test-double I use for any external systems within my unit tests.

See also: the testing pyramid[1] and "integrated tests are a scam"[2], which is a tad click-bait, but actually quite good.

[1] https://martinfowler.com/bliki/TestPyramid.html

[2] https://www.youtube.com/watch?v=VDfX44fZoMc

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#693

Post by agentultra »

crimsonalucard wrote: post #569
agentultra wrote: post #542

Earlier quoted context omitted.

Model checking is based off of mathematics and can be verified or, with some elbow grease, proven . I'll take a TLA+ specification over a diagram any day.

What do you mean by model checking? Usually anything with the keyword "Design" like design patterns for microservices have no science or mathematics to back it up.

A model checker is a software program you use to validate a given mathematical model of a system. If the proposed properties of the model hold the model checker will let you know. More interestingly if the properties fail to hold you'll get a trace back into where your assumptions fell apart.

TLA+ is one such system that includes a language for writing models and a model checker to verify them for you. In the context of microservices you would write a model of your services and the checker would help you to verify that certain properties of your model will hold for all possible executions of the model. Properties people seem to be interested in are consistency and transaction isolation. You can develop a model of your proposed microservices architecture and work out the errors in your design before you even write a lick of code.

Or if you already have a microservice system you could write a model of it and find if there are flaws in its design causing those annoying error reports.

Amazon wrote a paper about how they use it within the AWS team [0]. Highly worth the read. And if any of this sounds interesting I suggest checking out Hillel Wayne's course he's building [1].

[0] https://lamport.azurewebsites.net/tla/formal-methods-amazon.... [1] https://learntla.com/

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#694

Post by inetknght »

HeavyStorm wrote: post #674

I am a strong believer that microservices is over hyped. I usually resist when senior management asks for us to use it (that proves the point of hype). But by reading the first paragraphs of the article you see that the guys from Segment made a series of grave mistakes on their "microservices architecture", the most important one being the use of a shared library on many services. The goal of microservices is to achi…

> I am a strong believer that microservices is over hyped.

In general, I fully concur: there are very few services that actually warrant a microservice architecture.

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#695

Post by aeorgnoieang »

dalbasal wrote: post #595
Floegipoky wrote: post #558

It seems like splitting into separate repos was a rash response to low-value automated tests. If tests don't actually increase confidence in the correctness of the code they're negative value. Maybe they should have deleted or rewritten a bunch of tests instead. Which is what they did in the end anyway. >> A huge point of frustration was that a single broken test caused tests to fail across all destinations. When we…

This is a common pattern, when it come to semi-idealistic memes like microservices or agile. I think it's a bad idea to have such hairy, abstract ideas travel too far and wide. They become a bucket of clichés and abstract terms. Clichéd descriptions of problems you're encountering, like deployments being hard. Clichéd descriptions of the solutions. This let's everyone in on the debate, whether they actually understan…

[Taboo Your Words](https://www.lesswrong.com/posts/WBdvyyHLdxZSAMmoz/taboo-your...)

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#696

Post by dvogel »

inetknght wrote: post #689
mekazu wrote: post #667

Earlier quoted context omitted.

Unit tests that don’t read or write to disk and don’t try thousands of repetitions of things should be bleeding fast, but the most useful integration tests that actually help find faults (usually with your assumptions about the associated APIs) often need interaction with your disk or database or external service and tend to take a bit more than a few seconds. I find you need both.

I have tests which verify DNA analysis. The test data vectors are large -- a few hundred MB here, a couple GB there. The hundreds of tests that use these test vectors still run in a few seconds. If you're using a tape drive or SD cards, sure. But even a 10 year old 5400RPM on an IDE connection should be able to satisfy your tests' requirements in a few seconds or less. I suspect your tests are just as monolithic as y…

Sounds like you have tests that need to read (probably cached) data files while the parent poster has tests that need to write to disks (probably in a database transaction). Those are different enough that run times won't ever be comparable.

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#697

Post by amirouche »

ivan_gammel wrote: post #145
jrs95 wrote: post #91

Earlier quoted context omitted.

And running an integration test on a single modular application is easy . Doing it for a distributed system is very hard .

Running E2E blackbox test is equally simple for all kinds of architectures, especially today, when it's so easy to create a clean test environment with multiple containers even on developer's machine. It may be harder to automate this process for a distributed system, but, frankly speaking, I don't see a big difference between a docker-compose file or a launch script for monolith - I've been writing such tests for di…

> it's much easier to process the test output and debug the microservices than monolithic applications.

You easier to debug end-to-end tests of a microservice architecture that monolith? That's not my experience. How do you manage to put side by side all the events when they are in dozen of files?

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#698

Post by dragonwriter »

eksemplar wrote: post #612
ChicagoDave wrote: post #605

I’ve been tracking the comments and my sense is that almost no one here believes the business domain drives the technical solution. Microservices, when constructed from a well-designed model, provides a level of agility I’ve never seen in 33 years of software development. It also walls off change control between domains. My take from the Segment article is that they never modeled their business and just put services…

Wait till you get older. I’ve seen the hype swing from micro services to soa and back to micro services a few times already. I think the key is doing something that works, for you, and care a little less about what other people are doing. Most paradigms have their advantages and disadvantages and it’s really just about working around that to best utilize the stuff you have.

> I’ve seen the hype swing from micro services to soa and back to micro services a few times already.

SOA and microservices are the same thing; microservices is just a new name coined when the principles were repopularized so that it didn't sound like a crusty old thing.

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#699

Post by amirouche »

radisb wrote: post #668
thermodynthrway wrote: post #3

I've always said if the Linux kernel can be a giant monolith, in C no less, than there's maybe 100 web applications in the world that need to be split into multiple services. I've worked with microservices a lot. It's a never-ending nightmare. You push data consistency concerns out of the database and between service boundaries. Fanning out one big service in parallel with a matching scalable DB is by far the most sa…

"I've worked with microservices a lot. It's a never-ending nightmare. You push data consistency concerns out of the database and between service boundaries." This exactly. Me too. Data consistency concerns in sufficiently large real world projects can be practically dealt with only 2 ways IMO: transactions or spec changes.

what about idempotent updates?

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#700

Post by [deleted] »

jjeaff wrote: post #332
thermodynthrway wrote: post #3

I've always said if the Linux kernel can be a giant monolith, in C no less, than there's maybe 100 web applications in the world that need to be split into multiple services. I've worked with microservices a lot. It's a never-ending nightmare. You push data consistency concerns out of the database and between service boundaries. Fanning out one big service in parallel with a matching scalable DB is by far the most sa…

>Fanning out one big service in parallel with a matching scalable DB is by far the most sane way to build things. Agreed but I also prefer to keep any changing storage data as a separate concern (s3 or similar). So the trinity of services would be DB, storage, application.

[deleted]
Post reply on HN