Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Microservices are a tax your startup probably can't afford

nexo.sh

Post reply on HN
Print view View original

241–250 of 272 posts

Re: Microservices are a tax your startup probably can't afford

#241

Post by Seb-C »

JackSlateur wrote: post #196
roguecoder wrote: post #164

Earlier quoted context omitted.

I haven't seen a broken build in at least nine years, not since I left the company with a merge process built out of bash scripts that took three hours and required manual hand-holding. I am genuinely curious what situations you are seeing where builds are making it through CI and then don't compile. It isn't always worth investing in quality, but when it is it is entirely possible to write essentially bug-free softw…

You are absolutely right. Of course, if people wrote bug-free code, then there would be no bug ! Bug-free code in the actual code, or bug-free code in the test code, this is the same story. If you write stuff and never have any bug, then either: - you are lying - you do not write much - you only write really simple things - you are Jesus, came back from heaven to shine his light on us, poor souls The more complicated…

I disagree. I write complex code, and it is essentially bug-free.

And no, I'm not Jesus, I just care a lot about quality and have spent the last 20 years finding ways and strategies to improve it.

Reducing the number of bugs does not mean being a god that writes bug-free code on the first draft. It means being able to detect and fix issues as early as possible. In my case I aim to always do that before letting myself push any code to git.

IMO, it only comes down to how much someone really cares about the quality, but here are some examples of what can be done and is very effective:

- Plan ahead your functional and technical design

- Carefully research existing code to confirm the feasibility of the design

- Use a statically typed language

- Use advanced static-analysis tools

- Avoid magic, write explicit code. Especially avoid runtime checks such as reflection. Ideally, everything should be checked statically one way or another.

- Never let a code path/branch/corner case be unhandled, however unlikely it is (and go back to step one to refine the design if a code path has been forgotten in the current design)

- Always have automated testing. The bare minimum is to unit-test all business logic, including all possible code paths. Ideally e2e tests are nice, but not always a good investment. Tests must be 100% independent and never depend on an external environment, otherwise it's going to be flaky at some point.

- Always manually test every feature and path related to my changes (especially don't skip testing the ones that I think are going to be ok) before pushing anything to git.

- Warnings and "optional" notices are unacceptable and must always be fixed (or disabled), otherwise the list will just keep growing, which reduces the visibility of any issue and normalizes having problems.

- Have a CI integration that applies all the automated checks mentioned in this list and make everything mandatory.

Each one of those actions does on it's own significantly reduce the number of bugs. If you combine them all, you can effectively reduce the number of bugs to pretty-much zero. And since the earlier you find a bug, the cheaper is it overall to fix, I've also found-out that in terms of productivity it's always worth the investment (despite many people pretending the opposite).

Re: Microservices are a tax your startup probably can't afford

#242

Post by MrJohz »

quesera wrote: post #216
JackSlateur wrote: post #196

Earlier quoted context omitted.

You are absolutely right. Of course, if people wrote bug-free code, then there would be no bug ! Bug-free code in the actual code, or bug-free code in the test code, this is the same story. If you write stuff and never have any bug, then either: - you are lying - you do not write much - you only write really simple things - you are Jesus, came back from heaven to shine his light on us, poor souls The more complicated…

This seems a bit much. In the DVCS era, we have inexpensive branching. Do as thou wilt on your topic or epic branches. Rebase them against main/master before merging upwards. Fix what must be fixed first. Main/master branch should never fail CI. If it does, there is something seriously wrong with your branch lifecycle and/or deployment process.

Sure, but "never fails CI" is a different assertion from "has no bugs", or even "deploys correctly in production".

Re: Microservices are a tax your startup probably can't afford

#243

Post by const_cast »

eloisant wrote: post #42
abirch wrote: post #30

Earlier quoted context omitted.

Microservices are GREAT when 1 team owns each service. I haven't seen a good use case when you have 1 team supporting multiple microservices.

1 team supporting multiple services is not great, but a monolith with more than 50 developers working on it (no matter how you split your teams) isn't great either. That's why I don't like the term "microservice", as it suggests each service should be very small. I don't think it's the case. You can have a distributed system of multiple services of a decent size. I know "services of a decent size" isn't as catchy as…

I work on a monolith with ~1500 developers and it works pretty great.

The secret is that you're able to break a monolith apart, just like you can with microservices. You have APIs and modules of the monolith are responsible for their own thing. APIs are your contracts, just like in a microservice architecture.

The difference is that you can check if APIs are broken at compile time. In addition, you can view the API right in your IDE. In addition, your API isn't returning wishy-washy json with a half-assed OpenAPI spec - it's returning real types in a full-featured type system. And, cherry on top - you don't have to communicate over the network. Oh my god, you don't realize how many bugs and thousands of hours are wasted just working around that until you no longer have to. It's an immediate productivity boost.

But the best part is probably deployments. It's just so, so much more straightforward with one codebase.

Re: Microservices are a tax your startup probably can't afford

#244

Post by const_cast »

frollogaston wrote: post #202
tauoverpi wrote: post #193

Earlier quoted context omitted.

Statically typed languages, when used correctly, save engineering time both as you extend your service and when thing go wrong as the compiler helps you check that the code you've written, to some degree, meets your specification of the problem domain. With a weak type system you can't specify much of the problem domain without increased labour but with a more expressive type system (and a team that understands how t…

The type system almost never catches a bug that proper testing would miss. And if the code has such nasty untested edge cases that you don't even notice a wrong type going somewhere, it'd probably behave wrongly even with the right types. Indeed "any" breaks type checking all around it, but it can be contained more easily in a helper func with a simple return type. Most common case is your helper does a SQL query, an…

> The type system almost never catches a bug that proper testing would miss.

This is true, but the difference is you don't have to write a compiler, it's already written for you. The testing, you have to write, and do so correctly.

A lot of the woes of statically typed languages can be mitigated with tooling. Don't want to repetitively create types from an OpenAPI spec? Generate the code. Don't want to create types from SQL records? Generate the code. Don't want to write types everywhere? Deduce them.

You get all the benefits of static typing, but none of the work. It's so advanced these days that lots of statically-typed languages look dynamically-typed when you see the code. But they're not, everything has a type if you hover over them. The type deduction is just that good.

Re: Microservices are a tax your startup probably can't afford

#245

Post by Scarblac »

xcskier56 wrote: post #116
Scarblac wrote: post #86

Earlier quoted context omitted.

Splitting off a few services from an application is not the same as using micro services. With microservices you split off basically everything that would be a module in a normal application.

I think that really depends on your definition. But I will also contend that even splitting your system into 2 or 3 services if it's not for strong reasons will 100% slow you down and cause long term headaches. One project that I helped design had to split out a segment of the system b/c the data was eligibility records coming from health plans. This data had very different security and lifecycle requirements (e.g. w…

> But I will also contend that even splitting your system into 2 or 3 services if it's not for strong reasons will 100% slow you down and cause long term headaches.

Even tiny backends usually have, from the start, a database server, some web server / proxy to handle incoming requests and load balance or cache them, and then something running your actual code that requests are passed on to. That's so normal that we don't even think of them as separate services anymore, but they are.

Re: Microservices are a tax your startup probably can't afford

#246

Post by Scarblac »

codr7 wrote: post #139
Scarblac wrote: post #86

Earlier quoted context omitted.

Splitting off a few services from an application is not the same as using micro services. With microservices you split off basically everything that would be a module in a normal application.

If you split off a small, isolated part of the application; that's pretty much the definition of a microservice.

Splitting off a few larger parts because they are self contained, have their own Ops characteristics and might be usable by other projects on their own, vs creating a service for every single entity in your domain modal (or whatever), that's quite a difference.

We have an auth service, a database service for raster geo data, a generic task runner. Those are services. They all have their own pretty unique architecture and stand on their own, they are used by but are separate from the project that they were made for.

Different services for customers, for orders, for invoices, for items in the shop, for items on an order, for promotions, that's microservices. They all look very similar and belong to the application.

(in my opinion this morning)

Re: Microservices are a tax your startup probably can't afford

#247

Post by mrgriffin »

tauoverpi wrote: post #221
frollogaston wrote: post #202

Earlier quoted context omitted.

The type system almost never catches a bug that proper testing would miss. And if the code has such nasty untested edge cases that you don't even notice a wrong type going somewhere, it'd probably behave wrongly even with the right types. Indeed "any" breaks type checking all around it, but it can be contained more easily in a helper func with a simple return type. Most common case is your helper does a SQL query, an…

The type system doesn't replace unit/snapshot/property/simulation tests as it's only job is specification. The type system is meant to be used in addition to testing to reduce the set of possible inputs to a smaller domain such that it's easier to reason about what is possible and what isn't. The same would be true even if you go as far as formal verification of programs, you always need to test even when you have po…

> `foo :: Semigroup a, Traversable t => t a -> a` I already know that whatever is passed to this function can be traversed and have it's sum computed. It's impossible to pass something which doesn't satisfy both of those constraints as this is a specification given at the type level which is checked at compile-time.

To add to your point, I don't think foo can even be implemented (more accurately: is not total) because neither `Semigroup a` or `Traversable t` guarantee a way to get an `a`.

I think you'd need either `Monoid a` which has `mempty`, or `(Foldable1 t, Traversable t)` which guarantees that there's at least one `a` available.

Re: Microservices are a tax your startup probably can't afford

#248

Post by DanielHB »

dimal wrote: post #114
asim wrote: post #2

> Microservices only pay off when you have real scaling bottlenecks, large teams, or independently evolving domains. Before that? You’re paying the price without getting the benefit: duplicated infra, fragile local setups, and slow iteration. For example, Segment eventually reversed their microservice split for this exact reason — too much cost, not enough value. Basically this. Microservices are a design pattern for…

I saw one startup with about fifty engineers, and dozens of services. They had all of the problems that the post describes. Getting anything done was nearly impossible until you were in the system for at least six months and knew how to work around all the issues. Here’s the kicker: They only had a few hundred MAUs. Not hundreds of thousands. Hundreds of users. So all this complexity was for nothing. They burned thro…

I have worked with a company with ~100k MAUs with ~4 teams, even then it often feels the system is over-microserviced (about two dozen services I think).

Definitely some stuff makes sense (especially since it has a lot of IoT stuff), but micro-service added was used mostly as a way to develop new stuff without having to deal with the legacy monolith. The core of the application could easily be a single service backed by one big RDBMS with a few ancillary services around it.

The legacy monolith is still there kicking and screaming, it didn't need "breaking up" it needed (and I assume still do) need a major refactoring.

Re: Microservices are a tax your startup probably can't afford

#249

Post by DanielHB »

elevatedastalt wrote: post #52
singron wrote: post #12

Earlier quoted context omitted.

> You'll have a monolith, it might break out into frontend, backend and a separate service for async background jobs And when you break these out, you don't actually have to split your code at all. You can deploy your normal monolith with a flag telling it what role to play. The background worker can still run a webserver since it's useful for healthchecks and metrics and the loadbalancer will decide what "roles" get…

If you are building the same binary for all microservices you lose the dependency-reduction benefit microservices provide, since your build will still break because of some completely unrelated team's code.

You have a point, but I wouldn't say this is a big deal unless there is a mammoth dependency somewhere that slow down things to a crawl. Then maybe that one part of the codebase can be broken into its own separate service.

But even then there are ways around this kind of problem with dynamic linking pre-built binaries and caching, but it is extra complexity that could be worse than managing multiple services. Docker cache can usually handle this pretty well though.

Re: Microservices are a tax your startup probably can't afford

#250

Post by DanielHB »

elktown wrote: post #183
bcrosby95 wrote: post #163

Earlier quoted context omitted.

Note that the game's industry uses the term 'developer' differently. If a game has X developers, the vast majority of those people are not programmers. Engines also do a lot to empower non-programmers to implement logic in video games, taking lots of the workload off of programmers.

Sure, but there are enough programmers to make the point.

Maybe look up the game credits, I sometimes do and I often see like 10 UI programmers (in games with a ton to 2d UI), 5 gameplay programmers, 5 environment scripting, etc. Sure it is not small amount of people, but it is not an army.

Also those programmers seem to be neatly segregated in different areas of the project which I imagine work similarly to boundaries between the microservices at keeping the logic isolated between teams.

: I do AM surprised just how much QA people are credited in games, QA for major games sure do feel like an army.

Post reply on HN