Goodbye Microservices: From 100s of problem children to 1 superstar • Hacker News Board

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#661

Post by johnlbevan2 » Wed, Jul 11, 2018, 11:28 AM UTC

It will be interesting to read your post in a few months time, when you hit the issues caused by using a monolith which could be resolved by splitting that service apart.

Introducing components to match on various forms of value seems odd. Ideally standardise on those names; but if that's not possible (and you're using microservices), why not have a service to correct the name, rather than a library deployed to every endpoint? Then you call that service with data containing `first_name` or `givenname` and it returns that message with the standardised form. Or have a "global key" service; where you send the source system name and value, and have that translated for the destination via a lookup, allowing any field names or defined values to be translated by a generic reusable component:

  Entity    | System Name | System Value | GlobalKey
  -------------------------------------------------------------------------------------------
  Boolean   | HR          | TRUE         | 52582622-4322-445b-bb7a-8ca118d0ca2b
  Boolean   | HR          | FALSE        | 688c2298-6b99-4c31-a356-1ab9b1caacde
  Boolean   | Finance     | 1            | 52582622-4322-445b-bb7a-8ca118d0ca2b
  Boolean   | Finance     | 0            | 688c2298-6b99-4c31-a356-1ab9b1caacde
  Boolean   | Sales       | Yes          | 52582622-4322-445b-bb7a-8ca118d0ca2b
  Boolean   | Sales       | No           | 688c2298-6b99-4c31-a356-1ab9b1caacde
  FieldName | HR          | GivenName    | 3de31cff-e4bb-4d4a-a819-fd96c7c5032e
  FieldName | HR          | Surname      | e799b891-1eeb-4598-85a5-a9534d3a3a4c
  FieldName | Finance     | First_Name   | 3de31cff-e4bb-4d4a-a819-fd96c7c5032e
  FieldName | Finance     | Last_Name    | e799b891-1eeb-4598-85a5-a9534d3a3a4c
  FieldName | Sales       | FirstName    | 3de31cff-e4bb-4d4a-a819-fd96c7c5032e
  FieldName | Sales       | Surname      | e799b891-1eeb-4598-85a5-a9534d3a3a4c

As for "handcrafted XML"... the alternative looks like handcrafted code... For translation a language like XSLT which was designed for translating data from one format to another seems like a good choice. You can also use this to get around your translation issue by defining a central "universal" format; so you can have XSLTs to translate messages from source systems to the universal format, then other XSLTs to translate from universal to the destination; so that adding or removing a system (/service) only impacts that system / you don't need to rewrite every point-to-point interaction of that service with another (OK, not point to point since we have microservices; but if they're not adding value by abstracting you away from point to point then essentially you've just got complex point-to-point interactions rather than simple point-to-point).

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#662

Post by tchaffee » Wed, Jul 11, 2018, 11:34 AM UTC

s73v3r_ wrote: post #446

eropple wrote: post #282

Earlier quoted context omitted.

The notion that the node.js ecosystem moves way too fast is a time-traveling statement from, like, 2012. When it really was, in a lot of ways, true. But today it really has congregated on a relatively small set of best practices and the tools you see today are mostly the tools you would have seen two years ago. They're also mostly the same tools you would have seen five years ago, and plenty of people use "old" tools…

"But today it really has congregated on a relatively small set of best practices and the tools you see today are mostly the tools you would have seen two years ago." Has it? I'm pretty sure if I were to ask what those best practices and tools are, I'd get a number of different answers.

React is more than five years old.

webpack is six years old.

npm is eight years old.

nodejs is nine years old.

Anecdotal, but I personally feel like things have stabilized a great deal.

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#663

Post by rimliu » Wed, Jul 11, 2018, 11:35 AM UTC

closed wrote: post #351

bumholio wrote: post #280

Earlier quoted context omitted.

> Now that everyone forgot why the new technology was bad, we're free to begin the cycle again. We can very easily break the cycle by training a deep learning TensorFlow brain in the cloud, that will be fed the daily mouse gestures and key presses of all developers in the world. It's an awesome new technology that can solve any problem. Pretty soon the global brain will start to see patterns emerging, for example whe…

5 years later, tensor flow brain writes a medium post about why its very creation was a bad idea, then follows up with a quick rebuttal article about how it is just using itself incorrectly.

  > In the beginning the Universe was created.
  > This has made a lot of people very angry
  > and been widely regarded as a bad move.”

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#664

Post by dalbasal » Wed, Jul 11, 2018, 11:35 AM UTC

hohenheim wrote: post #645

dalbasal wrote: post #595

Earlier quoted context omitted.

This is a common pattern, when it come to semi-idealistic memes like microservices or agile. I think it's a bad idea to have such hairy, abstract ideas travel too far and wide. They become a bucket of clichés and abstract terms. Clichéd descriptions of problems you're encountering, like deployments being hard. Clichéd descriptions of the solutions. This let's everyone in on the debate, whether they actually understan…

Thank you for writing up a concise text about the actual problem. While reading the article I consistently felt bothered by the terminology thrown around but couldn't really pin point why. We really like to think in silos, categorize everything to make them feel familiar and approachable. Which is useful, but sometimes we need to shake them off so we can actually see the problems.

I agree exactly. A concept like "agile" is a great way of organising your own thoughts. Inventing words and concepts is a powerful tool. But... we have to remember that we invented them. They aren't real. That's easier when you invented them yourself.

After a while... it's like the cliché about taxi drivers investing in startups... Sign it's time to get out. When people I know have no idea start talking about the awesomeness of some abstract methodology... I'm out.

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#665

Post by machinecoffee » Wed, Jul 11, 2018, 11:38 AM UTC

buckminster wrote: post #410

dualogy wrote: post #17

Earlier quoted context omitted.

"Need to" and "sane" are among my favourite subjective terms! (Further below, I'll go into in which contexts I'd agree with your assessment and why. But for now the other side of the coin.) In the real world, current-day, why do many enterprises and IT departments and SME shops go for µservice designs, even though they're not multimillion-user-scale? Not for Google/Netflix/Facebook scale, not (primarily/openly) for h…

I was doing this with COM twenty years ago. It had the same advantages of modularity and language independence but without the unnecessary headaches of a distributed system. I take your point, but it saddens me that there aren't better ways of achieving this modularity nowadays.

I was too, of course a distributed system could also be built with DCOM and MTC (later COM+), and the DTC (Distributed Transaction Coordinator) could be used when you needed a transaction across services (or DBs, or MQs). Obviously the DTC was developed in recognition of the fact that distributed transactional service calls were a real requirement - something that current microservice architectures over HTTP REST don't seem to support.

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#666

Post by [deleted] » Wed, Jul 11, 2018, 11:39 AM UTC

mpweiher wrote: post #628

testplzignore wrote: post #139

Earlier quoted context omitted.

The most telling fact is that it "took milliseconds to complete running the tests for all 140+ of our destinations". I've never worked on a single service whose tests ran that fast, given that the time spent by the overhead of the test framework and any other one-time initialization can take a few seconds just itself. It's great to have tests that run fast, but that's a bit ridiculous. Some rules of thumb I just came…

> never worked on a single service whose tests ran that fast I'd say you've never had good tests. I have a test-suite for a bunch of my frameworks that dates to the mid 90s, with tests added regularly with new functionality. It currently takes 4 seconds total for 6 separate frameworks and 1000 individual tests. Which is actually a bit slower than it should be, it used to take around 1-2 seconds, so might have to dig…

[deleted]

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#667

Post by mekazu » Wed, Jul 11, 2018, 11:39 AM UTC

mpweiher wrote: post #628

testplzignore wrote: post #139

Earlier quoted context omitted.

The most telling fact is that it "took milliseconds to complete running the tests for all 140+ of our destinations". I've never worked on a single service whose tests ran that fast, given that the time spent by the overhead of the test framework and any other one-time initialization can take a few seconds just itself. It's great to have tests that run fast, but that's a bit ridiculous. Some rules of thumb I just came…

> never worked on a single service whose tests ran that fast I'd say you've never had good tests. I have a test-suite for a bunch of my frameworks that dates to the mid 90s, with tests added regularly with new functionality. It currently takes 4 seconds total for 6 separate frameworks and 1000 individual tests. Which is actually a bit slower than it should be, it used to take around 1-2 seconds, so might have to dig…

Unit tests that don’t read or write to disk and don’t try thousands of repetitions of things should be bleeding fast, but the most useful integration tests that actually help find faults (usually with your assumptions about the associated APIs) often need interaction with your disk or database or external service and tend to take a bit more than a few seconds. I find you need both.

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#668

Post by radisb » Wed, Jul 11, 2018, 11:50 AM UTC

thermodynthrway wrote: post #3

I've always said if the Linux kernel can be a giant monolith, in C no less, than there's maybe 100 web applications in the world that need to be split into multiple services. I've worked with microservices a lot. It's a never-ending nightmare. You push data consistency concerns out of the database and between service boundaries. Fanning out one big service in parallel with a matching scalable DB is by far the most sa…

"I've worked with microservices a lot. It's a never-ending nightmare. You push data consistency concerns out of the database and between service boundaries."

This exactly. Me too. Data consistency concerns in sufficiently large real world projects can be practically dealt with only 2 ways IMO: transactions or spec changes.

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#669

Post by organsnyder » Wed, Jul 11, 2018, 11:56 AM UTC

DCoder wrote: post #584

foobarian wrote: post #322

Earlier quoted context omitted.

I hated that in our monolith Java web app, any random team could come along and pepper our team's module with global variable lookups and short term hacks that never get cleaned up. Even though we see the changes often times they are urgent changes needed right now, accompanied with promises to clean up later (which usually doesn't happen). The nice thing about a SOA architecture is that it makes it harder to do this…

Isn't that a technical solution to an organizational problem?

Yes. And that's okay, sometimes. To me, the difference is whether it's an organizational challenge or an organizational dysfunction. There will always be challenges inherent to operating a business—and technical solutions often play a critical part. However, if the issue is really an organizational malady that should be addressed, technical solutions probably aren't the right approach.

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#670

Post by ahoka » Wed, Jul 11, 2018, 11:59 AM UTC

VBprogrammer wrote: post #175

kgwgk wrote: post #161

Earlier quoted context omitted.

Between “database servers” and “database tables” there is another level of granularity: “database users”.

While that's true. It also means you are only one grant away from sharing database tables. Maybe with good discipline you will be ok but all it take is for one dev to take that one shortcut.

If you have different database servers then you are only a "connection string" away from breaking your separation. Using a different login and schema is perfectly fine in a lot of architectures.