Live data from Hacker News

Goodbye Microservices: From 100s of problem children to 1 superstar

segment.com

741–750 of 782 posts

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#741

Earlier quoted context omitted.

The fix, then, is to put that commonality into a new microservice the other microservices call. The more I read about the problems people have with microservices, the more I'm convinced they've never read about flow-based programming.

Doesn’t that get excessive, making network calls to do what could be more naturally expressed as a method call? If you have two different teams collaborating or you expand beyond what a single box can do, create services. But if you can express things reasonably as a single service, why make things more complicated and error prone?

They were already breaking things into separate services. They violated DRY by putting the same or similar code into all of them. Then instead of cleaning up the DRY violations by refactoring into a shared service or refactoring into a shared library as a static asset, they factored it out into a mushy shared library still in lots of flux.

Whether they were wanting to factor into another service with a defined and mostly static API or a common library with a defined and mostly static API, it was a failure to factor common code into an amorphous blob that gates the release of all the other services. Instead, they've de-modularized the code and called that a success.

If you're drawing hard lines between services and having them talk to one another, having one of these models for at least parts of that makes a lot of sense: * filter system of the flow-based nature * REST API that does a transformation and returns transformed data * message broker with producer/consumer model where the consumer of one queue does a transform and puts the data into another queue * a full actor model * a full flow-based model

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#742

Earlier quoted context omitted.

I strongly disagree with your assessment of SQL. All of the questions that you asked have actual answers, but I read (from what I perceive as your tone) that you think these are all mysteries generated by an unknowable black box. While some of them may be implementation dependent (like "What algorithm does a select search execute?"), others have common and knowable answers. > Why does using select * slow down a query…

Those are rhetorical questions. I ask them because these are questions you need to know in order to work well with SQL. I cannot become a master of SQL by only learning SQL. I have to learn the specific implementations, I have to run EXPLAIN... etc... A good abstraction only requires you to know the abstraction not what lies underneath. What we have with SQL is a leaky abstraction. My argument that a high level leaky…

These problems are not unique to SQL, they are issues with any datastore. And if you're building your architecture correctly, the datastore is always going to be your bottleneck.

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#743

Earlier quoted context omitted.

With the last sentence, we now award the best hypothesis to dictation. https://www.youtube.com/watch?v=LJfowXTXOfU

That was the goal, thank you :) It's amazing how many stupid to the point of crazyness situations seem perfectly natural nowadays. Thank computers!

One of my favorite YouTubers nicknames computers, "confusers."

https://www.youtube.com/watch?v=LWH5bfpivSU

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#744

Earlier quoted context omitted.

Running E2E blackbox test is equally simple for all kinds of architectures, especially today, when it's so easy to create a clean test environment with multiple containers even on developer's machine. It may be harder to automate this process for a distributed system, but, frankly speaking, I don't see a big difference between a docker-compose file or a launch script for monolith - I've been writing such tests for di…

> it's much easier to process the test output and debug the microservices than monolithic applications. You easier to debug end-to-end tests of a microservice architecture that monolith? That's not my experience. How do you manage to put side by side all the events when they are in dozen of files?

I just don't use files for anything (services should be designed with the assumption that container can be destroyed any time, so files are simply not an option here). If you are talking about the logs, there are solutions like Graylog to aggregate and analyze them.

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#745

Earlier quoted context omitted.

What exactly are you trying to accomplish? If you are testing a single microservice and don’t want to test the dependent microservice - if you are trying to do a unit test and not an integration test, you are going to run against mock services. If you are testing a monolith you are going to create separate test assemblies/modules that call your subject under test with mock dependencies. They are both going to be part…

> What exactly are you trying to accomplish? Good test must verify the contract on the system boundaries: in case of the API, it's verification done by calling the API. We are discussing two options here: integrated application, hosting multiple APIs, and microservice architecture. Verification on the system boundaries means running the app, not running a unit test (unit tests are good, but serve different purpose).…

Verification on the system boundaries means running the app, not running a unit test

What is an app at the system boundaries if not a piece of code with dependencies?

If you have a microservice - FooService that calls BarService. The "system boundary" you are trying to test is FooService using a fake BarService. I'm assuming that you're calling FooService via HTTP using a test runner like Newman and test results.

In a monolithic application you have class FooModule that depends on BarModule that implements IBarModule. In your production application you use create FooModule:

var x = FooModule(new BarModule)

y = x.Baz(5);

In your Unit tests, you create your FooModuleL

var x = FooModule(new FakeBarModule) actual= x.Baz(5) Assert.AreEqual(10,actual)

And run your tests with a runner like NUnit.

There is no functional difference.

Of course FooModule can be at whatever level of the stack you are trying to test - even the Controller.

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#746

Earlier quoted context omitted.

Those are rhetorical questions. I ask them because these are questions you need to know in order to work well with SQL. I cannot become a master of SQL by only learning SQL. I have to learn the specific implementations, I have to run EXPLAIN... etc... A good abstraction only requires you to know the abstraction not what lies underneath. What we have with SQL is a leaky abstraction. My argument that a high level leaky…

These problems are not unique to SQL, they are issues with any datastore. And if you're building your architecture correctly, the datastore is always going to be your bottleneck.

I'm not talking about solving the bottleneck. No api can change that. I'm talking about mitigating the effects of this bottleneck. Namely, SQL is a bad design decision for this area of web dev.

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#747

Earlier quoted context omitted.

Running E2E blackbox test is equally simple for all kinds of architectures, especially today, when it's so easy to create a clean test environment with multiple containers even on developer's machine. It may be harder to automate this process for a distributed system, but, frankly speaking, I don't see a big difference between a docker-compose file or a launch script for monolith - I've been writing such tests for di…

> it's much easier to process the test output and debug the microservices than monolithic applications. You easier to debug end-to-end tests of a microservice architecture that monolith? That's not my experience. How do you manage to put side by side all the events when they are in dozen of files?

Using only files for logging is the last thing I would do in 2018.

I use Serilog for structured logging. Depending on the log destination, your logs are either stored in an RDMS (I wouldn’t recommend it) or created as JSON with name value pairs that can be sent directly to a JSON data store like ElasticSearch or Mongo where you can do adhoc queries.

https://stackify.com/what-is-structured-logging-and-why-deve...

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#748

Earlier quoted context omitted.

"But today it really has congregated on a relatively small set of best practices and the tools you see today are mostly the tools you would have seen two years ago." Has it? I'm pretty sure if I were to ask what those best practices and tools are, I'd get a number of different answers.

React is more than five years old. webpack is six years old. npm is eight years old. nodejs is nine years old. Anecdotal, but I personally feel like things have stabilized a great deal.

I've seen a lot of comments praising Vue in place of React.

Likewise, yarn in place of npm.

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#749
post #607

Earlier quoted context omitted.

Lol, of course you need a general understanding of how your database works on the inside. There are a million different ways you can store your data and I would argue that choosing your data storage is the most important and tricky decision we have to make as software engineers. Back at my first big tech company, I remember reading the best document I have ever read related to software engineering. It was entirely de…

>Lol, of course you need a general understanding of how your database works on the inside. There are a million different ways you can store your data and I would argue that choosing your data storage is the most important and tricky decision we have to make as software engineers. You aren't responding to my argument, everything you said is something I already know. So lol to you. You're making a remark and extending…

> I'm saying that the fact that you need "a general understanding of how a database works on the inside" is a design flaw. It's a leaky abstraction.

And I just said that "choosing a database is all about tradeoffs" which you need to understand (aka: the leaky abstractions).

> A C++ for loop has virtually the same performance across all systems/implementations

> For "SELECT * FROM TABLE", I have to understand implementation.

No you don't, it has the same performance: a for loop. However, by grouping all of your data onto 1 server, for loops are much more costly than the likely orders of magnitude more regular servers you have than a database. Fortunately, your SQL database supports indexes which speed up those queries. Granted, I'm no database expert, but adding the right indexes and making sure your queries utilize them have solved pretty much every scaling problem I have thrown at them.

> It would be smart to have a language api for the database to be highly optimize-able. The problem with SQL is that it is a high level leaky abstraction so optimizing SQL doesn't involve using complexity theory to write a tighter algorithm. It involves memorizing SQL hacks and gotchas and understanding implementation details.

It is optimizable and 90% of those optimizations I have made simply involve adding an index and then running a few explains/tests to make sure you are using them properly.

If you'll only answer me this though, what database would you recommend than? I'm dying to know since you think you know better and google, a company that probably has more scaling problems than anyone else, doubled down on SQL with spanner which from what I have read, requires even more actual fine tuning.

Re: Goodbye Microservices: From 100s of problem children to 1 superstar

#750
post #748

Earlier quoted context omitted.

React is more than five years old. webpack is six years old. npm is eight years old. nodejs is nine years old. Anecdotal, but I personally feel like things have stabilized a great deal.

I've seen a lot of comments praising Vue in place of React. Likewise, yarn in place of npm.

Once every five years? I'm OK with that.

I actually see Vue as more filling the gap for Angular. React uses JavaScript as the templating language instead of trying to use HTML attributes to turn HTML into a programming language like both Vue and Angular do. For that reason alone I have zero interest in Vue and learning yet another templating language. I'm pretty sure React will be here for a few more years. User base is still growing in fact.

And Yarn is a drop in replacement for npm. Took me 10 minutes to learn. If you already know npm, there is almost zero learning curve.

Post reply on HN