Strangely obvious advice?
Squeeze the hell out of the system you have
11–20 of 383 posts
Re: Squeeze the hell out of the system you have
#12Strangely obvious advice?
That no one likes to follow.
Re: Squeeze the hell out of the system you have
#13The bit on the database performance issues leads me to my hottest, flamiest take for new projects: - Design your application's hot path to never use joins. Storage is cheap, denormalize everything and update it all in a transaction. It's truly amazing how much faster everything is when you eliminate joins. For your ad-hoc queries you can replicate to another database for analytical purposes. On this note, I have mixe…
Normalization is not only about data storage but most importantly, data integrity.
Re: Squeeze the hell out of the system you have
#14> We should always put off significant complexity increases as long as possible. Reminds me of the mantra that I’ve read here to easily go for reversible things and very careful when going for irreversible things.
Amazon’s one way vs two way door decisions echo the sentiment
Re: Squeeze the hell out of the system you have
#15Strangely obvious advice?
Well, the advice is rarely taken in practice. It is (in my experience, and it seems common from others based on what I've heard) very very common to jump to the complicated solution at the first hint of capacity issues "because we'll need to do it eventually anyway."
The advice is obvious when you're thinking at that level of abstraction. Which suggests that, in practice, people who are architecting such systems rarely think at that level of abstraction. Which is why it is nice to have posts like this, that periodically remind us to get our heads out of the daily minutiae and consider the bigger picture (of complexity tradeoffs, realistic projections, staffing and availability, etc.)
Re: Squeeze the hell out of the system you have
#16Agree wholeheartedly with the conclusion of the article.
But the post makes it seem that there was no real query-level monitoring for the Postgres instance in place, other than perhaps the basic CPU/memory ones provided by the cloud provider. Using an ORM without this kind of monitoring is sure way to shoot yourself in the foot with n+1 queries, queries not using indexes/missing indexes etc
The other thing that is amazing that everyone immediately reached for redesigning the system without analyzing the cause of the issues. A single postgres instance can do a lot!
Re: Squeeze the hell out of the system you have
#17No mention of caching? If your database is getting hammered with SELECTs, isn't putting a cache in front of it something that should at least be considered?
I've been in the OP's situation, and this exact suggestion was made in my case. Welcome to one of the hardest problems in CS: cache invalidation.
If you have a dataset for which cache invalidation is easy (e.g., data that is written and never updated), yeah, absolutely go for this.
In our case, and most cases I've seen, it wasn't so simple, and "split this off to a DB better suited to it" was less complex (maybe still a lot of work, but conceptually simple) than figuring out cache invalidation.
Re: Squeeze the hell out of the system you have
#18The bit on the database performance issues leads me to my hottest, flamiest take for new projects: - Design your application's hot path to never use joins. Storage is cheap, denormalize everything and update it all in a transaction. It's truly amazing how much faster everything is when you eliminate joins. For your ad-hoc queries you can replicate to another database for analytical purposes. On this note, I have mixe…
Normalization is not only about data storage but most importantly, data integrity.
Yes, but I assert that it's possible to use transactions to update everything consistently. Serializable transactions weren't really common when MySQL/Postgres first came out, but now that they're common in new DBs + ACID, I think it's not possible to do with reasonable difficulty. If you agree with this, than its easy to prove that denormalized tables performance increase is well worth the annoyance of updating everything to transactionally update the dependencies.
I won't say that it's trivial to update all of your business logic to do this, but I think it's definitely worth it for a new project at least.
Re: Squeeze the hell out of the system you have
#19ah... rails easy mode discovers rails is only performant if you don't stray too far from hello world...
Re: Squeeze the hell out of the system you have
#20No mention of caching? If your database is getting hammered with SELECTs, isn't putting a cache in front of it something that should at least be considered?
They mentioned adding a DB replica for reads.