Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Simple Systems Have Less Downtime

gkogan.co

Post reply on HN
Print view View original

181–190 of 271 posts

Re: Simple Systems Have Less Downtime

#181

Post by petschge »

downerending wrote: post #177

As a sub-case of this, almost every HA (high availability) system I've ever seen has been less reliable than the original system was (without HA). It sounds like a good idea, but the extra complexity kills it. One non-software system was a rather expensive UPS/generator. It was meant to trip on a power loss and provide X minutes of stable power. In reality, it was incredibly sensitive to minute power fluctuations and…

I said the same thing about UPS causing more trouble than preventing it. Then I moved to Africa and quickly learned that a local UPS can be VERY helpful.

Re: Simple Systems Have Less Downtime

#182

Post by ipnon »

munchbunny wrote: post #175
nickbauman wrote: post #114

Earlier quoted context omitted.

They chose Erlang. A language built for communication and managing wire protocols at scale. Which describes WhatsApp itself. That was probably the biggest impact single decision for WhatsApp technically.

That was probably the biggest impact single decision for WhatsApp technically. As a rule of thumb based on my own experiences and the opinions of more experienced engineers I've had the good fortune to work with, language choice is far less important than the quality of the team using it. While I have no doubt that trying to build WhatsApp in a language that would be the wrong tool for the job (say... PHP) would have…

Converse: Can you imagine WhatsApp parallel universe serving with Node?

Re: Simple Systems Have Less Downtime

#183

Post by hyperion2010 »

pjmlp wrote: post #34
hyperion2010 wrote: post #25

The only language that I have worked with that realizes that in the real world simple is _always_ a lie, is common lisp. It is the only language that has actually embraced the fact that its designers/committee were not geniuses and provided the tools for dealing with the complexity of the system. When unix tools fail, hope that you are on a system where it is possible to get the symbols and/or the source code, and ev…

Forgetting Smalltalk here? Mesa/Cedar, Oberon System were also like that by the way. Java and .NET also share a bit of it, after all Java is half-way to Lisp (as per Guy Steele), and .NET follows up on it.

Sort of. I have a bad habit of forgetting Smalltalk in these kinds of conversations, but that is possibly because Smalltalk images exist in their own happy little worlds. I love working in Smalltalk environments, it is always a mind blowing experience, the system is completely homogeneous, accessible, introspectable, modifiable, etc. -- until you hit the hard boundary between the image and the host system, be it hardware or software. That boundary leaves quite a gap that smalltalks tend to be unable to fill by themselves (mostly due to a relative lack of resources). I can imagine a smalltalk system that could go all the way down to modern assembly, but unfortunately no such system exists today (that I'm aware of). If you can live inside the image then yes, Smalltalk is possibly even better.

Re: Simple Systems Have Less Downtime

#184

Post by jniedrauer »

downerending wrote: post #177

As a sub-case of this, almost every HA (high availability) system I've ever seen has been less reliable than the original system was (without HA). It sounds like a good idea, but the extra complexity kills it. One non-software system was a rather expensive UPS/generator. It was meant to trip on a power loss and provide X minutes of stable power. In reality, it was incredibly sensitive to minute power fluctuations and…

This has absolutely not been my experience, at least for well designed cloud infrastructure. High availability doesn't have to be complex or hard to understand. In most cases, automatic failovers and restarts on well designed architecture fix problems with zero end-user impact. It happens so seamlessly that my standard procedure these days is to fail over infrastructure if I want to restart it.

Re: Simple Systems Have Less Downtime

#185

Post by dnautics »

barrkel wrote: post #178
dnautics wrote: post #154

Earlier quoted context omitted.

Not necessarily, you can design erlang systems with more coupling than you'd expect in traditional microservices. Microservices mean that your codebase is segmented along business concern lines.

I would rather argue that microservices are best implemented when each service owns a bounded context in the lingo of domain-driven design. Business concerns might align with that, or they may not, depending on how technically complex (vs product complexity) the product is.

Yes, that is a better definition, but nonetheless you can design erlang systems with a single domain, or design erlang systems as a monolith, even if it has a billion actors running around underneath.

Re: Simple Systems Have Less Downtime

#186

Post by skrebbel »

munchbunny wrote: post #175
nickbauman wrote: post #114

Earlier quoted context omitted.

They chose Erlang. A language built for communication and managing wire protocols at scale. Which describes WhatsApp itself. That was probably the biggest impact single decision for WhatsApp technically.

That was probably the biggest impact single decision for WhatsApp technically. As a rule of thumb based on my own experiences and the opinions of more experienced engineers I've had the good fortune to work with, language choice is far less important than the quality of the team using it. While I have no doubt that trying to build WhatsApp in a language that would be the wrong tool for the job (say... PHP) would have…

Iirc Slack's backend is PHP.

Re: Simple Systems Have Less Downtime

#187

Post by shoo »

downerending wrote: post #177

As a sub-case of this, almost every HA (high availability) system I've ever seen has been less reliable than the original system was (without HA). It sounds like a good idea, but the extra complexity kills it. One non-software system was a rather expensive UPS/generator. It was meant to trip on a power loss and provide X minutes of stable power. In reality, it was incredibly sensitive to minute power fluctuations and…

> almost every HA (high availability) system I've ever seen has been less reliable than the original system was (without HA).

I've seen similar things with rabbitmq, in situation where the load & number of events to process is tiny but HA is some checkbox item to deliver without a serious QA process to measure if the setup is actually working effectively. Number of events in production we genuinely needed multiple rabbitmq nodes in the cluster to cope with failure of one or more nodes: 0 . Number of times when rabbitmq got too excited processing large messages that it delayed heart beating & subsequently decided that there was a network partition when there was in fact no such thing, leading the prod support team to step in and manually recover the cluster: > 0

Re: Simple Systems Have Less Downtime

#188

Post by jayd16 »

sgift wrote: post #145
nickbauman wrote: post #111

Earlier quoted context omitted.

But microservices are an example of simpler systems. Each microservice does far less than the whole monolith does. You can read all the code in ~15 minutes. I've worked at companies that have monoliths that are 50x more difficult to work on because of the size. Some of them millions of lines of code. Nobody really knows how they work anymore.

> But microservices are an example of simpler systems. Each microservice does far less than the whole monolith does. You can read all the code in ~15 minutes. Microservice usually means distributed system. A distributed system is more complex than a non-distributed system since it has to do everything the non-distributed system has to do and, additionally, handle all the distributed problems. Microservices just hide…

>Microservice usually means distributed system. A distributed system is more complex than a non-distributed system since it has to do everything the non-distributed system has to do and, additionally, handle all the distributed problems. Microservices just hide the complexity in places where people don't see them if they take a cursory look over the code, e.g. what is a function call in a monolith can be a call to a completely different machine in a microservice architecture. Often they look the same on the outside, but behave very differently.

This is a false assumption. Some problems are distributed. Sometimes you'll have an external data store or you'll need to deal with distribution across instances of the monolith. You really run into pain when you build a distributed system in your single monolithic code base and your monolithic abstractions start falling apart.

In my experience you end up solving these problems eventually, monolith or not. You might as well embrace the idea that you're deploying multiple services and some form of cross service communication. You don't need to go crazy with it though.

Re: Simple Systems Have Less Downtime

#189

Post by jniedrauer »

monus wrote: post #7

Usually, being easy is confused with being simple. Easy to use systems usually tend to be very complex but they'd only hide it from the average user. Once you go one step ahead, do something outside of quick start, you find yourself helpless. In that sense, the article is on point. Not only the management of the ship is simple, machinery behind the scenes is simple as well. This applies to programming languages, too.…

In real life, there are monsters lurking behind Go's "simplicity." An example: errors as values. It forces you to deal with errors as they come up, right? Not exactly... People just start ignoring errors. The extremely opinionated linter doesn't care if you just assign an error to _, or even just don't handle the return values at all. And it's not something you can easily spot in a pull request either. Then you end up with nil pointers or spaghetti errors that only show up at runtime and can be quite difficult to trace, where a Python exception would make debugging trivial.

This isn't necessarily a knock on Go. I've enjoyed working in it full time for several years now. I don't think there's a such thing as a simple programming language. Go just hides the complexity from initial inspection.

Re: Simple Systems Have Less Downtime

#190

Post by majormajor »

barrkel wrote: post #179
papito wrote: post #164

Earlier quoted context omitted.

As someone said, microservices is a technical solution to a people problem. Devs don't want to talk to each other so they wall off behind their own API. Boom, no need to talk to each other. Ever. Or is there?

Requiring everyone to talk to everyone so everyone has global context isn't just "devs don't want to talk to each other"; it is actually an information dissemination and coordination problem which scales non-linearly (at least n^2), and needs some kind of modularity to be tractable to normal humans. Microservices are like modules but for SaaS rather than shrinkwrap, and are where you end up when you follow SLAs, enca…

"Microservices" don't guarantee that everyone doesn't have to talk to everyone. Good thoughtful design is necessary regardless of how you're building/organizing/deploying/operating the code.
Post reply on HN