Squeeze the hell out of the system you have
blog.danslimmon.com
Squeeze the hell out of the system you have
1–10 of 383 posts
Re: Squeeze the hell out of the system you have
#2> We should always put off significant complexity increases as long as possible.
Reminds me of the mantra that I’ve read here to easily go for reversible things and very careful when going for irreversible things.
Re: Squeeze the hell out of the system you have
#3It's the boring solution. It should also only be the default answer if you are not building a super critical system to life and limb. But it certainly gives a much lower total cost of ownership. If you don't have the resources for some big redundant system, I've too often seen the complexity added by the redundant system be the issue then focusing on simplicity. If you need to add a bunch of people to support complexity but both the money and the risk assessment doesn't call for it, simpler is much better. I won't say I haven't seen the issue where eventually it was only a huge project to go forward, but I tend to think sometimes even that is less then the sum of having dealt with complexity to that point, it's dependent on a lot about what you are building.
Re: Squeeze the hell out of the system you have
#4I suppose it seems obvious in hindsight that your first move should always be to investigate potential causes before a wholesale redesign that adds potentially unnecessary complexity to your system.
Re: Squeeze the hell out of the system you have
#5> The real cost of increased complexity – often the much larger cost – is attention.
...or just mental load. I'm tired of working on micro-service systems that still have downtime, but no one knows how it all works. Most are actually just distributed monoliths so changes often touch multiple services and have to be rolled out in order. Data has to be duplicated, tasks have to be synchronized, state has to be shared, etc...
Re: Squeeze the hell out of the system you have
#6The bit on the database performance issues leads me to my hottest, flamiest take for new projects:
- Design your application's hot path to never use joins. Storage is cheap, denormalize everything and update it all in a transaction. It's truly amazing how much faster everything is when you eliminate joins. For your ad-hoc queries you can replicate to another database for analytical purposes.
On this note, I have mixed feelings about Amazon's DynamoDB, but one things about it is to use it properly you need to plan your use first, and schema second. I think there's something you can take from this even with a RDBMS.
In fact, I'd go as far to say as joins are unnecessary for nonanalytical purposes these days. Storage is so mind booglingly cheap and the major DBs have ACID properties. Just denormalize, forreal.
- Use something more akin to UUIDs to prevent hot partitions. They're not a silver bullet and have their own downsides, but you'll already be used to the consistently "OK" performance that can be horizontally scaled rather than the great performance of say integers that will fall apart eventually.
/hottakes
my sun level take would be also to just index all columns. but that'll have to wait for another day.
Re: Squeeze the hell out of the system you have
#7No mention of caching? If your database is getting hammered with SELECTs, isn't putting a cache in front of it something that should at least be considered?
Re: Squeeze the hell out of the system you have
#8Love this post. I’ve been trying to tell my manager the same message for the last few months (with little success).
We’re about to embark on a massive migration to “next-gen infrastructure” (read: three different Redshift clusters managed by CDK) because our overloaded Redshift cluster (already maxed out with RA3 nodes) has melted down one too many times. The next-gen infra is significantly more complex than our existing setup and I’m not convinced this migration will be the silver bullet everyone is hoping for.
Re: Squeeze the hell out of the system you have
#9Ugh. I had a colleague that addressed any scaling problem by putting a cache in front of the DB. Praised for solving the immediate problem, but shouldered none of the costs.
I admit in the face of finding Prod/market fit, you do the expedient thing, but damned if I'm not often at the receiving end of these sorts of decisions.
Re: Squeeze the hell out of the system you have
#10Isn’t Rails wasteful in its database access patterns?