Live data from Hacker News

Simple Systems Have Less Downtime (2020)

gkogan.co

141–150 of 220 posts

Re: Simple Systems Have Less Downtime (2020)

#141

I'm going to make a controversial claim for the sake of argument. We can and will debate what programs are "simpler" all day, it's all subjective... except for one metric: line count. Line count is objective. Take pause before rejecting any change that uses fewer lines of code. It takes an awfully good abstraction to beat simply having less code.

It seems like it should be objective, but it isn't. Because you get people writing ridiculous "clever" one-liners that are difficult to parse and understand. That clever one-liner could be re-written to do the exact same thing and be much more clear and readable, but it would take, say, 8 lines instead of 1. Then there's the subjectiveness of what you actually consider a "line".

and then there's "syntactic sugar"

Re: Simple Systems Have Less Downtime (2020)

#142

This is sort of like the old "Water is wet; news at 11." thing. But simple is not easy. In my experience, it often comes after complexity. My general approach to simplification is to first get it working; even in a complicated fashion, then start removing stuff, until it stops working. If I can't get it working on the latest simplicity, I am forced to add the last thing back. But I can usually get it going, which may…

"First, make it correct. Then, make it beautiful. Then, if you need to, make it performant. Because 9 times out of 10, making it beautiful also makes it performant enough."

- Joe Armstrong

Re: Simple Systems Have Less Downtime (2020)

#143
post #124

Earlier quoted context omitted.

> What boggles the mind is people are often sold the cloud even though their org isn't serving millions of users, and could just as easily operate as a box in a closet. Capex vs opex may play a part here. Also, companies in general seems to have transitioned to using services in general. 20/30 years ago companies had cleaners in them. Now everyone uses a cleaning service.

I have read about a dozen and a half articles about capex vs. opex, and it still makes absolutely no sense to me why companies prefer paying over double for opex what they'd pay for equivalent capex over the same lifecycle. Either corporate accounting is some mystical art that makes money appear where there isn't if certain practices are followed, or there's some collective mass delusion that opex is just better? I d…

For a startup it makes perfect sense to avoid high CAPEX: it is not worth to invest into own infrastructure which will have positive ROI in 3 years, if you are not sure that you startup will still be alive in 3 years.

Why big established companies prefer high OPEX to lower CAPEX is less clear.

Re: Simple Systems Have Less Downtime (2020)

#144
post #117

I gave a talk at CU Boulder[1] in 2019 on complexity in cloud infrastructure and the design of rsync.net. What I wrote, and then said several times was that: "Simple systems fail in boring ways. Complex systems fail in fascinating ways." I gave examples from Chernobyl, Air France Flight 447, etc. as examples of such. It's a topic that fascinates me and I had hoped to give several more such talks - but alas, the pande…

I'd love to see/read that talk if you happen to youtube it or write it up.

Re: Simple Systems Have Less Downtime (2020)

#145
Not building what doesn't need to exist won't get you promoted. You don't get promoted by avoiding entire classes of problems with a simple, reliable system. You get promoted by pulling heroics to build some absurdly complex cloud Kubernetes bullshit, and then pulling more heroics to fix the endless stream of production issues that will result from your overengineered nightmare.

Re: Simple Systems Have Less Downtime (2020)

#146
post #52
post #49

I fell for the everything must be a microservice / distributed across as many servers as possible trap. Even though I've read so many warnings about it here on HA and knew upfront I might have to rewind everything. The setup : - Distributed file system using GlusterFS - DNS load balancing using Amazon Route 53 - PostgreSQL HA clusters using Patroni - A WireGuard mesh topology between all instances. Even though it was…

> Even though it was so much fun That’s the root of quite a bit of evil in software. A closely related motive is a desire to show off. Play is good for learning but not for production systems. You’ll regret it later when you are up at 4am on a Sunday morning troubleshooting some Byzantine stack. The antidote is the realization that simplicity is harder than complexity. Simple but highly effective systems are the ones…

The more I think about it the more it seems to me 'simplicity' is just another name for 'encapsulated/isolated/modularised complexity'

Re: Simple Systems Have Less Downtime (2020)

#147

Not building what doesn't need to exist won't get you promoted. You don't get promoted by avoiding entire classes of problems with a simple, reliable system. You get promoted by pulling heroics to build some absurdly complex cloud Kubernetes bullshit, and then pulling more heroics to fix the endless stream of production issues that will result from your overengineered nightmare.

http://web.mit.edu/nelsonr/www/Repenning%3DSterman_CMR_su01_...

I've posted this here before, though got no comments. But yes, firefighter/arsonists get more promotions and kudos than careful thinkers who don't bother setting fires to fight.

EDIT:

One past discussion here (2015): https://news.ycombinator.com/item?id=8940820

Re: Simple Systems Have Less Downtime (2020)

#148

Earlier quoted context omitted.

Many companies in the EU actually blanket ban any storage of their sensitive information in the cloud, especially the Atlassian cloud after the Australian espionage law change.

That's a good point! Then we raise the question one level up: "does the current law regarding storage of sensitive information make sense?" It's a never ending story for those who like to delve in these sort of things.

For all what it's worth: yes. The larger the cloud, the larger the motivation for enemies to attack it. And in this case, the enemy is China (and, for European companies, also the US/FVEY alliance) - all of which have already been caught multiple times doing industrial espionage.

Core company secrets belong on premises of the company owning the secret, not in a cloud where it is fair game for security services and criminals of all kinds and nations.

Re: Simple Systems Have Less Downtime (2020)

#149
post #16

I'm an architect and I keep things simple too. The problem I have is people around me expect more complex/intricate solutions. They think simple is not sophisticated and savvy. They think the competition is ahead of our 'old, simple solutions'. I have to continually justify and explain that KISS is always the right approach.

If they are just looking to sell things to make money without regard to quality, then they're probably right. People underestimate how large of a market that is.

Re: Simple Systems Have Less Downtime (2020)

#150
post #144
post #117

I gave a talk at CU Boulder[1] in 2019 on complexity in cloud infrastructure and the design of rsync.net. What I wrote, and then said several times was that: "Simple systems fail in boring ways. Complex systems fail in fascinating ways." I gave examples from Chernobyl, Air France Flight 447, etc. as examples of such. It's a topic that fascinates me and I had hoped to give several more such talks - but alas, the pande…

I'd love to see/read that talk if you happen to youtube it or write it up.

No, it was neither recorded nor transcribed.

Some of my observations on complexity were touched upon in this interview, however:

https://console.dev/interviews/rsync-john-kozubik/

Specifically:

"rsync.net has no firewalls and no routers. In each location we connect to our IP provider with a dumb, unmanaged switch.

This might seem odd, but consider: if an rsync.net storage array is a FreeBSD system running only OpenSSH, what would the firewall be ? It would be another FreeBSD system with only port 22 open. That would introduce more failure modes, fragility and complexity without gaining any security."

... and I recommend these console.dev interviews in general - they're usually very interesting.

Post reply on HN