Live data from Hacker News

Squeeze the hell out of the system you have

blog.danslimmon.com

181–190 of 383 posts

Re: Squeeze the hell out of the system you have

#181

The bit on the database performance issues leads me to my hottest, flamiest take for new projects: - Design your application's hot path to never use joins. Storage is cheap, denormalize everything and update it all in a transaction. It's truly amazing how much faster everything is when you eliminate joins. For your ad-hoc queries you can replicate to another database for analytical purposes. On this note, I have mixe…

Materialized views provide a way to have joins that are accessed like tables!

Re: Squeeze the hell out of the system you have

#182
post #130

Earlier quoted context omitted.

"No battle plan survives contact with the enemy." https://www.google.com/search?q=no+battle+plan+survives

Mike Tyson said it more simply: "Everybody has a plan until you get hit in the face."

Nope.

Let's juxtapose them and see:

Von Moltke:

"No battle plan survives contact with the enemy."

Tyson:

"Everybody has a plan until you get hit in the face."

Pretty much the same meaning, and Von Moltke's quote is three words shorter, so no, Tyson's quote is not simpler.

Also, Tyson was ungrammatical, IMO:

"Everybody" vs. "you" in the same sentence, referring to the same entity.

Grammar experts, correct me if I am wrong.

Re: Squeeze the hell out of the system you have

#183
post #130

Earlier quoted context omitted.

Mike Tyson said it more simply: "Everybody has a plan until you get hit in the face."

Nope. Let's juxtapose them and see: Von Moltke: "No battle plan survives contact with the enemy." Tyson: "Everybody has a plan until you get hit in the face." Pretty much the same meaning, and Von Moltke's quote is three words shorter, so no, Tyson's quote is not simpler. Also, Tyson was ungrammatical, IMO: "Everybody" vs. "you" in the same sentence, referring to the same entity. Grammar experts, correct me if I am w…

And anyway, simplicity, although I favor it a lot in my work, is not a virtue in itself. As for many, if not most issues, the answer is "It all depends.".

Re: Squeeze the hell out of the system you have

#184

Earlier quoted context omitted.

"No battle plan survives contact with the enemy." https://www.google.com/search?q=no+battle+plan+survives

https://en.m.wikipedia.org/wiki/Helmuth_von_Moltke_the_Elder Moltke's thesis was that military strategy had to be understood as a system of options, since it was possible to plan only the beginning of a military operation. As a result, he considered the main task of military leaders to consist in the extensive preparation of all possible outcomes.[3] His thesis can be summed up by two statements, one famous and one l…

I’m reminded of the Eisenhower line: “plans are worthless, but planning is everything.”

Re: Squeeze the hell out of the system you have

#185
post #163

Earlier quoted context omitted.

Can't say I've ever come across a scenario where a join itself was the performance bottleneck. If there's any single principle I have observed is "don't let a table get too big". More often than not it's historical-record type tables that are the issue - but the amount of data you need for day-to-day operations is usually a tiny fraction of what's actually in the table, and you're bound to start finding operations on…

In enterprise clients you commonly run into issues where the company thinks you have to save all data forever. Very often this runs in a pattern where the application is lightly used a first then uptake increases over time. Then you run into the slowness issue. They typically expand DB sizing, bit eventually run into the problem where archiving is needed. This can be a huge problem when it's an after thought instead…

It's more that once it gets to a certain size (say, 100s of 1000s of rows), doing anything with the table is painfully slow, often requiring you to take your application offline for considerable periods. Even deleting rows can take 10s of minutes at a time, and it can certainly take a very long time to work out what indexes need to be added and whether they're actually helping.

Yes, sometimes the pressure comes from management etc., but more often than not it would be premature optimisation to add the archiving, so it's a matter of finding a balance and "predicting" at what point the archiving needs to happen.

Table partitioning can help too but only so much.

Re: Squeeze the hell out of the system you have

#186
post #86

I wonder if moving the db on beefy dedicated hardware with ton of ram and nvme would solve the problem. Preferably physicaly connected to the web serveurs. Cost: a fraction of the developper cost. I see so many things done on the cloud that 10X their complexity because of it. Modern hardware in increadibly powerfull.

If they were on the biggest instance I doubt they want to setup a network connection to another DC with another provider which has all kinds of business/privacy policy/etc concerns.

They likely already had 24.xlarge or something lol

Re: Squeeze the hell out of the system you have

#187

I’m reminded of one of my favorite sayings: You go to war with the army you have, not the army you might want or wish to have at a later time. You may want to ignore that this this comes from Donald Rumsfeld (he has some great ones though: “unknown unknowns …”, etc.) I think about this a lot when working on teams. Everyone is not perfectly agreeable or has the same understanding or collective goals. Some may be subop…

I’m not sure who came up with it first but the nautical expression is, “you sail with the crew you have”

Re: Squeeze the hell out of the system you have

#188

I’m reminded of one of my favorite sayings: You go to war with the army you have, not the army you might want or wish to have at a later time. You may want to ignore that this this comes from Donald Rumsfeld (he has some great ones though: “unknown unknowns …”, etc.) I think about this a lot when working on teams. Everyone is not perfectly agreeable or has the same understanding or collective goals. Some may be subop…

That was Donald Rumsfeld!? I always assumed this came from some techie or agile guru given how much it's used as a concept in project planning.

It's from a post WWII psychological theory the 'Johari Window'. Rumsfeld brought the phrase into wider consciousness.

https://en.m.wikipedia.org/wiki/Johari_window

Re: Squeeze the hell out of the system you have

#189

Earlier quoted context omitted.

Lotus notes is wide… I imagine their scope creep checker was just a sticky note that said Absolutely!!

I miss notes - it was really a better way to organize companies than anything later. Historical valuable data, records of why decisions were made, ephemeral email like things but for groups, user programmable if it didn't quite match your needs, robust encryption, it had it all.

oh I always nust assumed Lotus Notes was just lesser Outlook. can you give examples - such has how did it capture why decisions were made - that sounds ... hard or just "someone wrote it down"

Re: Squeeze the hell out of the system you have

#190
post #130

Earlier quoted context omitted.

Mike Tyson said it more simply: "Everybody has a plan until you get hit in the face."

Nope. Let's juxtapose them and see: Von Moltke: "No battle plan survives contact with the enemy." Tyson: "Everybody has a plan until you get hit in the face." Pretty much the same meaning, and Von Moltke's quote is three words shorter, so no, Tyson's quote is not simpler. Also, Tyson was ungrammatical, IMO: "Everybody" vs. "you" in the same sentence, referring to the same entity. Grammar experts, correct me if I am w…

[flagged]
Post reply on HN