Goodbye Microservices: From 100s of problem children to 1 superstar • Hacker News Board

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#701

Post by inetknght » Wed, Jul 11, 2018, 2:24 PM UTC

dvogel wrote: post #696

inetknght wrote: post #689

Earlier quoted context omitted.

I have tests which verify DNA analysis. The test data vectors are large -- a few hundred MB here, a couple GB there. The hundreds of tests that use these test vectors still run in a few seconds. If you're using a tape drive or SD cards, sure. But even a 10 year old 5400RPM on an IDE connection should be able to satisfy your tests' requirements in a few seconds or less. I suspect your tests are just as monolithic as y…

Sounds like you have tests that need to read (probably cached) data files while the parent poster has tests that need to write to disks (probably in a database transaction). Those are different enough that run times won't ever be comparable.

I have tests that need to read. I have tests that need to write. All data written must also be read and verified. You're right, the data is probably cached.

If you need to access a database in your tests you're probably doing it wrong. Build a mock-up of your database accessor API to provide static data, or build a local database dedicated for testing.

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#702

Post by organsnyder » Wed, Jul 11, 2018, 2:25 PM UTC

mburst wrote: post #491

jarfil wrote: post #236

The whole article reads like BS to me. So the initial problem was a single queue? Well, then split the queue, no need to go all crazy splitting all the code. Switching to 100+ microservices? There is no need to switch to 100+ repos too, runtime services don't need to have one repo per service, just use a modular approach, or even feature flags. 100+ microservices, some of them with much lower load than others? Then c…

> Library inconsistencies between services? No, just no, always use the same library version for all services. Automate importing/updating the libraries if you need to. If you did this then you'd have to go and update all services whenever you wanted to introduce a breaking change. I don't think what you're suggesting is as easy as it sounds.

Depends on how the library is used. If the change to the library will change how the service behaves externally (to either upstream or downstream services), then you indeed need to update them all simultaneously. However, that has the smell of poorly-defined API contracts (or poor implementations) if that is possible.

Otherwise, just update to the latest library versions whenever you touch a codebase.

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#703

Post by rcxdude » Wed, Jul 11, 2018, 2:26 PM UTC

gaius wrote: post #354

thermodynthrway wrote: post #3

I've always said if the Linux kernel can be a giant monolith, in C no less, than there's maybe 100 web applications in the world that need to be split into multiple services. I've worked with microservices a lot. It's a never-ending nightmare. You push data consistency concerns out of the database and between service boundaries. Fanning out one big service in parallel with a matching scalable DB is by far the most sa…

I've always said if the Linux kernel can be a giant monolith, in C no less, than there's maybe 100 web applications in the world that need to be split into multiple services. Similarly it’s made with make. If anyone has a project more complex than the Linux kernel or GCC I’ll gladly listen to why they need some exotic build system... never met anyone yet...

There's a decent amount of custom supporting machinery around make to create the linux build system.

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#704

Post by _ea1k » Wed, Jul 11, 2018, 2:27 PM UTC

ljw1001 wrote: post #230

copperx wrote: post #190

Earlier quoted context omitted.

What is that called, then? Having small programs, each responding to a different part of the API, all storing data in a single shared DB?

(a) a prototype (b) a set of applications that share a database. You would have to have an oddly disconnected schema if modifications to the program don't result in programs accessing parts of the database that other programs are already accessing. If this isn't a problem it means you're using your database as nature intended and letting it provide a language-neutral, shared repository with transactional and consiste…

I think you are exactly right. Microservice architectures are definitely not automatically good, and there is nothing wrong with a well architected "monolith".

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#705

Post by organsnyder » Wed, Jul 11, 2018, 2:28 PM UTC

jacquesm wrote: post #407

lagadu wrote: post #390

Earlier quoted context omitted.

Storytime?

Every microservice on its own little cluster of VMs for HA and performance... A couple of hundred VMs is nothing in a scenario like that. Good luck trying to debug anything.

That's basically how microservices are operated on orchestrators like Kubernetes—just substitute "container" for "VM", which is a mostly-academic difference from the perspective of your application. Operations tooling—distributed tracing, monitoring, logging...—is essential.

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#706

Post by draw_down » Wed, Jul 11, 2018, 2:29 PM UTC

slivym wrote: post #646

Floegipoky wrote: post #558

It seems like splitting into separate repos was a rash response to low-value automated tests. If tests don't actually increase confidence in the correctness of the code they're negative value. Maybe they should have deleted or rewritten a bunch of tests instead. Which is what they did in the end anyway. >> A huge point of frustration was that a single broken test caused tests to fail across all destinations. When we…

As an engineer in a large company this seems very similar to management structure. Every 6-12 months there's a re-organisation to split the business into vertically aligned business units, and then to horizontally aligned capabilities. Then back again. It's always fun to watch. In reality this process has absolutely nothing to do with the structure of the organisation. It's true purpose is to shuffle out people who a…

Lots of insight here. Great post, thank you.

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#707

Post by organsnyder » Wed, Jul 11, 2018, 2:30 PM UTC

andy_ppp wrote: post #647

Oh man, this is just the tip of the iceberg with Microservices. There is nothing Micro about them, they are so difficult to deal with that it becomes impossible to actually iterate or build user value and introduces loads of difficult to debug problems. We have architects here that dictate the design of the system but IMO they have not done the simplest implementation of anything. We have Kafka to provide ways of mak…

Ugh. Sounds like a severe case of resume-driven architecture.

In my experience, you won't know whether you need microservices until you're on at least v2.0 of your application. By then, you have a better understanding of what your real problems are.

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#708

Post by xkgt » Wed, Jul 11, 2018, 2:32 PM UTC

dnautics wrote: post #267

rstefanic wrote: post #242

Earlier quoted context omitted.

How is this an example of the Hegelian Dialectic?

exactly, root comment's point is that 'synthesis' is missing, so there is no forward progress unlike the dialectic!

I kind of get what systemizer is saying. People may think of evolution of technologies as cycles but it is never that. A new technology 'Y' is always developed because the incumbent 'X' has some shortcomings. And even after a period of disillusionment when we revert back to 'X', it is not always the same. We synthesize the good points of 'Y' back to 'X'.

Coming to this topic, I see Microservices as a solution to the problem of Continuous Delivery which is necessary in some business models. I can't see those use cases reverting back to Monolith architecture. For such scenarios, the problems associated with Microservices are engineering challenges and not avoidable architecture choices.

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#709

Post by dalbasal » Wed, Jul 11, 2018, 2:32 PM UTC

itsmenotyou wrote: post #627

dalbasal wrote: post #595

Earlier quoted context omitted.

This is a common pattern, when it come to semi-idealistic memes like microservices or agile. I think it's a bad idea to have such hairy, abstract ideas travel too far and wide. They become a bucket of clichés and abstract terms. Clichéd descriptions of problems you're encountering, like deployments being hard. Clichéd descriptions of the solutions. This let's everyone in on the debate, whether they actually understan…

Interesting perspective. I think that seeking and naming patterns "microservices", "agile", etc. is useful. It provides something like a domain specific language that allows a higher level conversation to take place. The problem, as your identify, is that once a pattern has been identified people too easily line up behind it and denigrate the "contrasting" pattern. The abstraction becomes opaque. We're used to simpli…

I agree that it's useful, I even think more people should do it more often. Creating your own language (and learning other people's) is a way of having deep thoughts, not just expressing. Words for patterns (or abstract ions generally) are a quanta of language.

I just think there can be downsides to them. These are theories as well as terms and they become parts of our worldview, even identity. This can engage our selective reasoning, cognitive biases and our "defend the worldview!" mechanisms in general. At some point, it's time for new words.

Glad people seem ok with this. I've expressed similar views before (perhaps overstating things) with fairly negative responses. I think part of it might be language nuance. The term "ideology" carries less baggage in Europe, where "idealist" is what politicians hope to be perceived as while "ideologue" is a common political insult statesside, meaning blinded and fanatic.

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#710

Post by eropple » Wed, Jul 11, 2018, 2:43 PM UTC

loliboli wrote: post #611

eropple wrote: post #534

Earlier quoted context omitted.

I have, actually. This week has made me decide to move away from it. I have been using it, and filed an issue[0] asking them to document the schemas emitted by their entity-tree code; I was then told that I was doing it wrong and that I should use their code-first/synchronization feature and trust in their magic rather than writing my own explicit migrations which are informed by my (tbh, pretty extensive) understand…

I’d recommend you to have a look at Objection.js: https://github.com/Vincit/objection.js/ Objection.js is an ORM for Node.js that aims to stay out of your way and make it as easy as possible to use the full power of SQL and the underlying database engine while keeping magic to a minimum. ^^ copy+pasted from github

This looks interesting. Some quick thoughts:

- I was worried at first because I progressively trust non-TypeScript projects less and less, but the test suite looks fine and they have official typings so there's that mitigation at least.

- I really don't love the use of static properties everywhere in order to define model schema. Which is probably a little hypocritical, because one of my own projects[0] does the same thing, but IMO decorators are a cleaner way to do it that reads better to a human.

- Require loop detection in model relations is cool. I like that.

- In general it's a little too string-y for my tastes. I think `eager()` should take a model class, for example, rather than a string name. Maybe behind the scenes it uses the model class and pulls the string name out of it? But I think using objects-as-objects is a better way to do things than using strings-as-references.

Overall, though, it seems very low-magic and I did understand most of it from a five minute peek, so I kind of like this. I think it's a little too JavaScript-y (rather than TypeScript-y) for my tastes, but maybe that can be addressed by layering on a bit of an adapter...I'll need to look deeper.

[0] - https://github.com/eropple/taskbotjs