Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Squeeze the hell out of the system you have

blog.danslimmon.com

Post reply on HN
Print view View original

101–110 of 383 posts

Re: Squeeze the hell out of the system you have

#101

Post by dragonwriter »

roughly wrote: post #72
i_like_apis wrote: post #34

I’m reminded of one of my favorite sayings: You go to war with the army you have, not the army you might want or wish to have at a later time. You may want to ignore that this this comes from Donald Rumsfeld (he has some great ones though: “unknown unknowns …”, etc.) I think about this a lot when working on teams. Everyone is not perfectly agreeable or has the same understanding or collective goals. Some may be subop…

Rumsfeld's got some great quotes, most of which were delivered in the context of explaining how the Iraq war turned into such a clusterfuck, and boy could that whole situation have used the kind of leadership Donald Rumsfeld's quotes would lead you to believe the man could've provided.

> Rumsfeld's got some great quotes, most of which were delivered in the context of explaining how the Iraq war turned into such a clusterfuck

If by “explaining how” you mean “deflecting (often preemptively) responsibility for”, yes.

Re: Squeeze the hell out of the system you have

#102

Post by [deleted] »

a_seattle_ian wrote: post #99
KnobbleMcKnees wrote: post #56

Earlier quoted context omitted.

That was Donald Rumsfeld!? I always assumed this came from some techie or agile guru given how much it's used as a concept in project planning.

That it came from Donald Rumsfeld in the context of what we know now and what he surely knew then is why it's such a good quote. The words basically say nothing but are also true about everything. So it can implicit be a warning that there is probably some bullshit going on or someone has a sense of humor and is also warning people while also avoiding the subject - of course just my opinion. How people actually use i…

[deleted]

Re: Squeeze the hell out of the system you have

#103

Post by gillh »

Prioritized load shedding works well as a last resort [0]. The idea is simple -

- Detect overload/congestion build-up at the database

- Apply queueing at the gateway service and schedule requests based on their priority

- Shed excess requests after a timeout

[0]: https://docs.fluxninja.com/blog/protecting-postgresql-with-a...

Re: Squeeze the hell out of the system you have

#104

Post by endisneigh »

tibbetts wrote: post #47
endisneigh wrote: post #6

The bit on the database performance issues leads me to my hottest, flamiest take for new projects: - Design your application's hot path to never use joins. Storage is cheap, denormalize everything and update it all in a transaction. It's truly amazing how much faster everything is when you eliminate joins. For your ad-hoc queries you can replicate to another database for analytical purposes. On this note, I have mixe…

Premature denomalization is expensive complexity. Denormalization is a great tool, maybe an under-used tool. But you should wait until there are hot paths before using it.

I agree. To be clear I'm not suggesting anyone start denormalizing everything. I'm saying if you're fortunate enough to be on a green project, you should design the schema around the access patterns which will surely be "denomarlized." as opposed to designing a normalized schema and designing your access patterns around those.

Re: Squeeze the hell out of the system you have

#105

Post by waprin »

roughly wrote: post #76
midasuni wrote: post #73

Earlier quoted context omitted.

And unknown unknowns is a great way to communicate with stakeholders too

Žižek has a followup to that quote: "What he forgot to add was the crucial fourth term: the "unknown knowns," the things we don't know that we know." I've found it's really critical during the project planning phase to get to not just where the boundaries of our knowledge are, but also where are the things we're either tacitly assuming or not even aware that we've assumed. An awful lot of postmortems I've been a part…

I really enjoy the concept of unknown knowns, but I don’t agree with your example, which is an unknown unknown.

To me the corporate version of the unknown known is when a a project is certainly doomed, for reasons everyone on the ground knows about, yet nobody wants to say anything and be the messenger that inevitably gets killed, as long as paycheck keeps clearing. An exec ten thousand feet from the ground sets a “vision” which can’t be blown off course by minor details such as reality, until the day it does.

Theranos is a famous example of this but I’ve had less extreme versions happen to me many times throughout my career.

Another example of unknown knowns might be the conflict between companies stated values (Focus on the User) and the unstated values that are often much more important (Make Lots of Money)

Re: Squeeze the hell out of the system you have

#106

Post by endisneigh »

walterbell wrote: post #100
sainez wrote: post #68

Earlier quoted context omitted.

> what this might mean in practice is that you do mockups of all of the expected pages and what data is necessary on each one. then you design a schema that results in you never having to do joins on the majority, if not all, of them. Great suggestion! I had a role where I helped a small team develop a full stack, data-heavy application. I felt pretty good about the individual layers but I felt we could have done a b…

2001, "Denormalization effects on performance of RDBMS", by G. L. Sanders and Seungkyoon Shin, https://www.semanticscholar.org/paper/Denormalization-effect... > We have suggested using denormalization as an intermediate step between logical and physical modeling, to be used as an analytic procedure for the design of the applications requirements criteria ... The guidelines and methodology presented are sufficiently g…

yeah, exactly. in my experience the vast majority of access patterns are designed around a normalized schema, where it really should be that the schema is designed around the access patterns and generously "denormalize" (which doesn't make sense in this context of a new database) as necessary.

Re: Squeeze the hell out of the system you have

#107

Post by wtetzner »

endisneigh wrote: post #6

The bit on the database performance issues leads me to my hottest, flamiest take for new projects: - Design your application's hot path to never use joins. Storage is cheap, denormalize everything and update it all in a transaction. It's truly amazing how much faster everything is when you eliminate joins. For your ad-hoc queries you can replicate to another database for analytical purposes. On this note, I have mixe…

I'd say that probably depends on what your hot path is. If it's write-heavy, then you'll probably end up with performance issues when you need to write the same data to multiple tables in a single transaction. And if all of those columns are indexed, it'll be even worse.

Re: Squeeze the hell out of the system you have

#108

Post by killjoywashere »

KnobbleMcKnees wrote: post #56
i_like_apis wrote: post #34

I’m reminded of one of my favorite sayings: You go to war with the army you have, not the army you might want or wish to have at a later time. You may want to ignore that this this comes from Donald Rumsfeld (he has some great ones though: “unknown unknowns …”, etc.) I think about this a lot when working on teams. Everyone is not perfectly agreeable or has the same understanding or collective goals. Some may be subop…

That was Donald Rumsfeld!? I always assumed this came from some techie or agile guru given how much it's used as a concept in project planning.

As a military officer who was watching CNN live from inside an aircraft carrier (moored) when he said that, being in charge of anti-terrorism on the ship at the time, it was absolutely foundational to my approach to so many things after that. Here's the actual footage: https://www.youtube.com/watch?v=REWeBzGuzCc

Rumsfeld was complicated, but there's no doubt he was very effective at leading the Department. I think most people fail to realize how sophisticated the Office of the Secretary of Defense is. Their resources reel the mind, most of all the human capital, many with PhDs, many very savvy political operators with stunning operational experiences. As a small example, as I recall, Google's hallowed SRE system was developed by an engineer who had come up through the ranks of Navy nuclear power. That's but one small component reporting into OSD.

Not a Rumsfeld apologist, by any means. Errol Morris did a good job showing the man for who he is, and it's not pretty (1). But reading HN comments opining about the leadership qualities of a Navy fighter pilot who was both the youngest and oldest SECDEF makes me realize how the Internet lets people indulge in a Dunning-Kruger situation the likes of which humanity has never seen.

https://www.amazon.com/Known-Donald-Rumsfeld/dp/B00JGMJ914

Re: Squeeze the hell out of the system you have

#109

Post by zengid »

The solution they went with, squeezing juice out of the system by finding performance optimizations, brings me so much joy.

It reminds me of Richard L. Sites's book _Understanding_Software_Dynamics_ where he basically teaches how to measure and fix latency issues, and how at large scales, reducing latency can have tremendous savings.

Measuring and reasoning about those issues are hard, but the solutions are often simple. For example, on page 9 he mentions that "[a] simple change paid for 10 years of my salary."

I hope to someday make such an impactful optimization!

Re: Squeeze the hell out of the system you have

#110

Post by hluska »

roughly wrote: post #72
i_like_apis wrote: post #34

I’m reminded of one of my favorite sayings: You go to war with the army you have, not the army you might want or wish to have at a later time. You may want to ignore that this this comes from Donald Rumsfeld (he has some great ones though: “unknown unknowns …”, etc.) I think about this a lot when working on teams. Everyone is not perfectly agreeable or has the same understanding or collective goals. Some may be subop…

Rumsfeld's got some great quotes, most of which were delivered in the context of explaining how the Iraq war turned into such a clusterfuck, and boy could that whole situation have used the kind of leadership Donald Rumsfeld's quotes would lead you to believe the man could've provided.

I like to remind myself that very few people reach positions of great power after mediocre lives. Rather there’s a thread of talent that runs through government.

Once they’re in, the predilections that led to power often rear their dark long tails. But they’re all (even the ones I disagree with) talented.

Post reply on HN