Really curious how much can be attributed to using an ORM.
None. The ORM didn't design and type up the code.
Squeeze the hell out of the system you have
191–200 of 383 posts
Re: Squeeze the hell out of the system you have
#192The bit on the database performance issues leads me to my hottest, flamiest take for new projects: - Design your application's hot path to never use joins. Storage is cheap, denormalize everything and update it all in a transaction. It's truly amazing how much faster everything is when you eliminate joins. For your ad-hoc queries you can replicate to another database for analytical purposes. On this note, I have mixe…
That is a hot take... ;) But joins should never impact performance in a large way if they're on the same server and properly indexed. "It's truly amazing how much faster everything is when you eliminate joins" is just not true if you're using joins correctly. Sadly, many developers simply never bother to learn. On the other hand, having to write a piece of data to 20 different spots instead of 1 is going to be dramat…
Storage might be cheap, but memory and bandwidth aren’t.
Memory is the new disk, and disk is the new tape.
You want everything to remain resident in memory, and spool backups and transaction logs to disk.
If you’re joining from disk, you’ve probably done something wrong.
E.g.: compression is often a net win because while it uses more CPU, it allows more data to fit into memory. And if it doesn’t fit, it reduces the disk I/O required.
This is why I look upon JSON-based document databases in horror. They’re bloating the data out many times over by denormalizing and then expand that into a verbose and repetitive text format.
This is why we have insanity like placeholders for text on web apps now — they’re struggling to retrieve a mere kilobyte of data!
Re: Squeeze the hell out of the system you have
#193Earlier quoted context omitted.
Rumsfeld's got some great quotes, most of which were delivered in the context of explaining how the Iraq war turned into such a clusterfuck, and boy could that whole situation have used the kind of leadership Donald Rumsfeld's quotes would lead you to believe the man could've provided.
If I remember it correctly (it was a long time ago), he never fully supported the war. It didn't take a genius to notice that the goals set by the presidency were (literally) impossible and not the kind of thing you do achieve a war. But whatever position he had, Iraq turning into a clusterfuck wasn't a sign of bad leadership by his part. It was a sign of bad ethics, but not leadership. His options were all of gettin…
As history, this is completely incorrect, but beyond that, if you don’t believe in the mission of the President in committing an act of war, you have a responsibility to resign, and it can’t be bracketed as “bad ethics”.
Anyway, another historical point besides what the other commenters have said is that Rumsfeld believed in “transformation” which meant you could do more with less in modern war. He was totally wrong about it.
It wasn’t his fault Turkey didn’t let the US attack from the north, but other than that, the fuck up is his responsibility, among others.
Re: Squeeze the hell out of the system you have
#194Earlier quoted context omitted.
Rumsfeld's got some great quotes, most of which were delivered in the context of explaining how the Iraq war turned into such a clusterfuck, and boy could that whole situation have used the kind of leadership Donald Rumsfeld's quotes would lead you to believe the man could've provided.
> Rumsfeld's got some great quotes, most of which were delivered in the context of explaining how the Iraq war turned into such a clusterfuck If by “explaining how” you mean “deflecting (often preemptively) responsibility for”, yes.
> If by “explaining how” you mean “deflecting (often preemptively) responsibility for”, yes.
There's no reason to think you can't do both of those with the same statement.
Re: Squeeze the hell out of the system you have
#195This can work long term, as long as your system is not a group of humans. But certainly, making what appear like small adjustments can sometimes tremendously lower required resources.
Re: Squeeze the hell out of the system you have
#196The bit on the database performance issues leads me to my hottest, flamiest take for new projects: - Design your application's hot path to never use joins. Storage is cheap, denormalize everything and update it all in a transaction. It's truly amazing how much faster everything is when you eliminate joins. For your ad-hoc queries you can replicate to another database for analytical purposes. On this note, I have mixe…
That is a hot take... ;) But joins should never impact performance in a large way if they're on the same server and properly indexed. "It's truly amazing how much faster everything is when you eliminate joins" is just not true if you're using joins correctly. Sadly, many developers simply never bother to learn. On the other hand, having to write a piece of data to 20 different spots instead of 1 is going to be dramat…
I agree but I’m talking in the context where you can’t vertically scale anymore.
I also don’t think it’s worth the trouble “never using joins” for an existing project. Denormalize as necessary. But for a green one I honestly think since our access patterns can be understood as you continue you can completely get rid of joins.
Again, assuming your new project can’t fit on a single machine. If it can you’re best just following the “traditional” advice, or better yet keep everything in memory.
Re: Squeeze the hell out of the system you have
#197Ugh. I had a colleague that addressed any scaling problem by putting a cache in front of the DB. Praised for solving the immediate problem, but shouldered none of the costs. I admit in the face of finding Prod/market fit, you do the expedient thing, but damned if I'm not often at the receiving end of these sorts of decisions.
“Just put a cache in front of it to speed it up” is up there along with everyone’s favourite misunderstood “(premature) optimisation is the root of all evil” when it comes to performance and efficient software in my opinion.
Caching is mostly a lie. Redis will not “make” your application faster. It will just let you pretend the problem doesn’t exist for a while, and then when the caching eventually falls over, you will be even more stuck, because you’re now past your scaling limits, with no more defenses left and software that has been (continually) built without the understanding of its performance requirements.
Re: Squeeze the hell out of the system you have
#198Re: Squeeze the hell out of the system you have
#199I’m reminded of one of my favorite sayings: You go to war with the army you have, not the army you might want or wish to have at a later time. You may want to ignore that this this comes from Donald Rumsfeld (he has some great ones though: “unknown unknowns …”, etc.) I think about this a lot when working on teams. Everyone is not perfectly agreeable or has the same understanding or collective goals. Some may be subop…
Every time I hear the name Rumsfeld, I am reminded of the time when, for over 10 minutes, he refused to deny being a lizard: https://www.youtube.com/watch?v=XH_34tqxAjA
Haha. Thanks for sharing that. Rumsfeld definitely has a sense of humor.
He’s also one of the best candidates for that type of conspiracy theory. His career history is flabbergasting.
Check out https://en.wikipedia.org/wiki/Donald_Rumsfeld#Corporate_conn...
In addition to all the Bohemian Club, RAND corp, defense and government posts, in the 70s the guy was a CEO in the pharmaceuticals and electronics industries, was a director in aerospace, media and tech.
Definitely the type of resume that lets the imagination run wild with, “… wait, was he a lizard person …?”
Re: Squeeze the hell out of the system you have
#200The bit on the database performance issues leads me to my hottest, flamiest take for new projects: - Design your application's hot path to never use joins. Storage is cheap, denormalize everything and update it all in a transaction. It's truly amazing how much faster everything is when you eliminate joins. For your ad-hoc queries you can replicate to another database for analytical purposes. On this note, I have mixe…
There are "tall" applications and "wide" applications. Almost all advice you ever read about database design and optimization is for "tall" applications. Basically, it means that your application is only doing one single thing, and everything else is in service of that. Most of the big tech companies you can think of are tall. They have only a handful of really critical, driving concepts in their data model. Facebook…
Very insightful, thank you for writing this.