Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Squeeze the hell out of the system you have

blog.danslimmon.com

Post reply on HN
Print view View original

251–260 of 383 posts

Re: Squeeze the hell out of the system you have

#251

Post by justinlloyd »

smarkov wrote: post #177
justinlloyd wrote: post #147

Squeeze what you've got, as hard as you can, then realize after squeezing for a while that if you squeezed here, here, and also... here, by changing how you think about a problem, suddenly you've got a lot left to get. I spent two or so months optimizing the crap out of a majestic monolith and went from under 2K RPS when the PM thought, and the team repeatedly reported, that everything had been squeezed as much as it…

Very curious to learn more about what the monolith was doing so incredibly poorly that you managed to squeeze that much performance out of it. Poorly written queries? Too many queries? Lack of any caching? Doing things synchronously when they could've been done concurrently?

Some of that, some other bad practices. Lots of low-hanging fruit, then more esoteric changes.

https://justinlloyd.li/blog/how-much-cache-you-got-on-you/

Re: Squeeze the hell out of the system you have

#252

Post by sbuk »

fuzztester wrote: post #182
sbuk wrote: post #130

Earlier quoted context omitted.

Mike Tyson said it more simply: "Everybody has a plan until you get hit in the face."

Nope. Let's juxtapose them and see: Von Moltke: "No battle plan survives contact with the enemy." Tyson: "Everybody has a plan until you get hit in the face." Pretty much the same meaning, and Von Moltke's quote is three words shorter, so no, Tyson's quote is not simpler. Also, Tyson was ungrammatical, IMO: "Everybody" vs. "you" in the same sentence, referring to the same entity. Grammar experts, correct me if I am w…

It was midnight and a few beers after celebrating a birthday. I'm sorry I offended your grammatical sensibilities. But you really did go full orange site there, didn't you! I will admit to misquoting Mike Tyson; "Everyone has a plan until they get punched in the mouth.", which I hope goes someway to restoring peace and order over a tiny, drunken grammatical slip-up.

Re: Squeeze the hell out of the system you have

#253

Post by justinlloyd »

therealdrag0 wrote: post #242
justinlloyd wrote: post #147

Squeeze what you've got, as hard as you can, then realize after squeezing for a while that if you squeezed here, here, and also... here, by changing how you think about a problem, suddenly you've got a lot left to get. I spent two or so months optimizing the crap out of a majestic monolith and went from under 2K RPS when the PM thought, and the team repeatedly reported, that everything had been squeezed as much as it…

You gotta share more lol. These are insane gains.

I cannot go too deep. It was some client work.

https://justinlloyd.li/blog/how-much-cache-you-got-on-you/

Re: Squeeze the hell out of the system you have

#254

Post by franckpachot »

yashap wrote: post #249
crazygringo wrote: post #159

Earlier quoted context omitted.

That is a hot take... ;) But joins should never impact performance in a large way if they're on the same server and properly indexed. "It's truly amazing how much faster everything is when you eliminate joins" is just not true if you're using joins correctly. Sadly, many developers simply never bother to learn. On the other hand, having to write a piece of data to 20 different spots instead of 1 is going to be dramat…

Joins are not inherently expensive, but they can lead to expensive queries. For example, say I want to find the 10 most recent users with a phone number as their primary contact method: SELECT … FROM User JOIN ContactMethod on ContactMethod.userId = User.id WHERE ContactMethod.priority = ‘primary’ AND ContactMethod.type = ‘phoneNumber’ ORDER BY User.createdAt DESC LIMIT 10 If there are a very large number of users, a…

With an index on User (createdAt, id) and one on ContactMethod ( primary,ContactMethod,userId), it should be fast (check that the the execution plan starts with User). Except if lot of recent users have no phones, but that will not be better in a single table (except if columnar storage)

Re: Squeeze the hell out of the system you have

#255

Post by [deleted] »

yashap wrote: post #249
crazygringo wrote: post #159

Earlier quoted context omitted.

That is a hot take... ;) But joins should never impact performance in a large way if they're on the same server and properly indexed. "It's truly amazing how much faster everything is when you eliminate joins" is just not true if you're using joins correctly. Sadly, many developers simply never bother to learn. On the other hand, having to write a piece of data to 20 different spots instead of 1 is going to be dramat…

Joins are not inherently expensive, but they can lead to expensive queries. For example, say I want to find the 10 most recent users with a phone number as their primary contact method: SELECT … FROM User JOIN ContactMethod on ContactMethod.userId = User.id WHERE ContactMethod.priority = ‘primary’ AND ContactMethod.type = ‘phoneNumber’ ORDER BY User.createdAt DESC LIMIT 10 If there are a very large number of users, a…

[deleted]

Re: Squeeze the hell out of the system you have

#256

Post by Turskarama »

yashap wrote: post #249
crazygringo wrote: post #159

Earlier quoted context omitted.

That is a hot take... ;) But joins should never impact performance in a large way if they're on the same server and properly indexed. "It's truly amazing how much faster everything is when you eliminate joins" is just not true if you're using joins correctly. Sadly, many developers simply never bother to learn. On the other hand, having to write a piece of data to 20 different spots instead of 1 is going to be dramat…

Joins are not inherently expensive, but they can lead to expensive queries. For example, say I want to find the 10 most recent users with a phone number as their primary contact method: SELECT … FROM User JOIN ContactMethod on ContactMethod.userId = User.id WHERE ContactMethod.priority = ‘primary’ AND ContactMethod.type = ‘phoneNumber’ ORDER BY User.createdAt DESC LIMIT 10 If there are a very large number of users, a…

In my experience with SQL, a query like that should return in under a second even if you have 100k or more users.

There are some other tricks you can use if you're clever/lucky as well. If you're just using integer IDs (which is reasonable if your system isn't distributed) then you could order by userid on you ContactMethod table and still get the same speed as you would with no join.

Re: Squeeze the hell out of the system you have

#257

Post by re-thc »

theptip wrote: post #211
endisneigh wrote: post #6

The bit on the database performance issues leads me to my hottest, flamiest take for new projects: - Design your application's hot path to never use joins. Storage is cheap, denormalize everything and update it all in a transaction. It's truly amazing how much faster everything is when you eliminate joins. For your ad-hoc queries you can replicate to another database for analytical purposes. On this note, I have mixe…

Honestly I couldn’t disagree more. I built a startup and paid little attention to perf for years 1-5, and finally in year 6 we started to get bitten by some perf issues in specific tables, and spent a few engineer-months optimizing. In terms of tech debt it would have been way more expensive to make everything perform well from the start, we would have moved much slower and probably failed during a few crunch points.…

> I built a startup and paid little attention to perf for years 1-5, and finally in year 6 we started to get bitten by some perf issues in specific tables, and spent a few engineer-months optimizing.

This screams of if I don’t see it it the problem doesn’t exist view of the world.

How do you know it’s not a problem? Perhaps customers would have signed up if it as faster?

The problem is also treating it in terms of business value and/or cost.

A lot of things are “free” and yet it’s ignored.

For most people, in simple cases like turning on http3, brotli, switching to newer instances and many others are all quick wins that I see ignored 90% of the time.

A good design, implementing some good practices etc are performance specific and don’t always cost more.

Re: Squeeze the hell out of the system you have

#258

Post by therealdrag0 »

justinlloyd wrote: post #253
therealdrag0 wrote: post #242

Earlier quoted context omitted.

You gotta share more lol. These are insane gains.

I cannot go too deep. It was some client work. https://justinlloyd.li/blog/how-much-cache-you-got-on-you/

Thank you! Even that was insightful.

Re: Squeeze the hell out of the system you have

#259

Post by yafbum »

fastball wrote: post #231
endisneigh wrote: post #196

Earlier quoted context omitted.

I agree but I’m talking in the context where you can’t vertically scale anymore. I also don’t think it’s worth the trouble “never using joins” for an existing project. Denormalize as necessary. But for a green one I honestly think since our access patterns can be understood as you continue you can completely get rid of joins. Again, assuming your new project can’t fit on a single machine. If it can you’re best just f…

How is it that your new project can't fit on a single machine?

Could be plenty of reasons, but say you want to support spikey traffic like whatever is related to events, news, realtime information, etc., or highly seasonal stuff like a school portal or selling flowers on mother's day, you want the ability to scale horizontally very quickly to accommodate spikes

Re: Squeeze the hell out of the system you have

#260

Post by yafbum »

feoren wrote: post #115
endisneigh wrote: post #6

The bit on the database performance issues leads me to my hottest, flamiest take for new projects: - Design your application's hot path to never use joins. Storage is cheap, denormalize everything and update it all in a transaction. It's truly amazing how much faster everything is when you eliminate joins. For your ad-hoc queries you can replicate to another database for analytical purposes. On this note, I have mixe…

There are "tall" applications and "wide" applications. Almost all advice you ever read about database design and optimization is for "tall" applications. Basically, it means that your application is only doing one single thing, and everything else is in service of that. Most of the big tech companies you can think of are tall. They have only a handful of really critical, driving concepts in their data model. Facebook…

Doesn't that also say something like, it's an easier road to success if you find the tall way to market and scale, scale, scale once you find it? What is the "wide" success story to take inspiration from?
Post reply on HN