Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How not to structure database-backed web apps: performance bugs in the wild

blog.acolyer.org

Post reply on HN
Print view View original

251–260 of 319 posts

Re: How not to structure database-backed web apps: performance bugs in the wild

#251

Post by mklepaczewski »

Cthulhu_ wrote: post #216
projectramo wrote: post #185

The real joy of this article was the following: Some academics (Yang, Subramanian, Lu, Yan, Cheung) were able to produce massive improvements in about a dozen large, mature, battle tested open source projects using just a few lines of code. This should give hope to all those tepidly trying to get into open source. Just go and take a look at the dozens of open source projects in Django or whatever and you could improv…

What's intimidating to me is that to make a change like that, you first need to get the code, get it and the tests running, and actually understand what the code does - and whether your fix has the equivalent result or is subtly broken. They didn't just change a single line in an application, they wrote a huge benchmark suite and dug through miles of code to find issues like this. I've no clue how much time they spen…

This is not necessarily true. You would be surprised how many absolutely trivial performance issues can be found in almost every project. Sometimes it's really just about moving computation of a constant value out of a 'for' loop.

Re: How not to structure database-backed web apps: performance bugs in the wild

#252

Post by dgritsko »

mklepaczewski wrote: post #248
latch wrote: post #23

I've worked on moderately busy backend platforms (~10K-20k rps handled on a ~4 e5-2650 and aiming for 5ms 95p response times). It greatly depends on what you're doing, but for the majority of systems which are read heavy (and that most certainly includes "dynamic" sites like Amazon or Wikipedia), I hold to two major beliefs: 1 - Have very long TTLs on your internal cache servers with a way to proactively purge (messa…

1 - TTL should be infinite. If one needs finite TTL it means that cache invalidation logic is bogus.

Or it could mean that the cost of cache invalidation is greater than the cost of allowing stale data to be read for the remainder of the TTL, or any other number of justifications for finite TTL. The point is, it depends on your application - while it's true that shorter TTLs can mask faulty invalidation logic, I don't think it's correct to say that "TTL should be infinite" with no qualifiers whatsoever.

Re: How not to structure database-backed web apps: performance bugs in the wild

#253

Post by dgritsko »

godelmachine wrote: post #250
latch wrote: post #23

I've worked on moderately busy backend platforms (~10K-20k rps handled on a ~4 e5-2650 and aiming for 5ms 95p response times). It greatly depends on what you're doing, but for the majority of systems which are read heavy (and that most certainly includes "dynamic" sites like Amazon or Wikipedia), I hold to two major beliefs: 1 - Have very long TTLs on your internal cache servers with a way to proactively purge (messa…

By TTL, are you referring to Transistor Transistor Logic?

TTL in this context is "time-to-live" - "a mechanism that limits the lifespan or lifetime of data in a computer or network" (https://en.wikipedia.org/wiki/Time_to_live). E.g., the duration of a cache entry.

Re: How not to structure database-backed web apps: performance bugs in the wild

#254

Post by snowwrestler »

mklepaczewski wrote: post #248
latch wrote: post #23

I've worked on moderately busy backend platforms (~10K-20k rps handled on a ~4 e5-2650 and aiming for 5ms 95p response times). It greatly depends on what you're doing, but for the majority of systems which are read heavy (and that most certainly includes "dynamic" sites like Amazon or Wikipedia), I hold to two major beliefs: 1 - Have very long TTLs on your internal cache servers with a way to proactively purge (messa…

1 - TTL should be infinite. If one needs finite TTL it means that cache invalidation logic is bogus.

True but cache invalidation is famously hard to get perfect.

A nicer way to put it might be that finite TTLs should be viewed as an opportunity to optimize, not ideal or standard.

Re: How not to structure database-backed web apps: performance bugs in the wild

#255

Post by patmcguire »

tootie wrote: post #215
mnutt wrote: post #212

Earlier quoted context omitted.

It’s susceptible to Thundering Herd whereby more requests come in for the same cache key before the initial computation is finished, and so you end up with lots of cache misses. The fix is usually to lock the cache key and have subsequent requests wait on the original computation but it’s a bit more complex to code.

I've heard it called Cache Stampede. Any decent framework for memoizing method calls would cover this case though.

It depends on the backing, if you're using just memcached there's no way to completely lock the key. None of the popular rails caching frameworks I've seen handle this either.

Re: How not to structure database-backed web apps: performance bugs in the wild

#256

Post by Ramiuz »

knesek wrote: post #82

When I was inexperienced I feared ORMs because of the negative performance impacts I've read they could have. I constantly worried about what would happen if the amount of data increased and I hit ORM induced problem that I could not resolve without major rewrite of data access layer. However, whenever I've actually hit those problems in production, I found the similar thing the authors of the article did - ORM induc…

How would you test a CRUD app with a DB?

The way that I do it now it's just use some simple functional test to GET / POST and measure the time in a pytest script, I'm sure there are better ways to do it.

Re: How not to structure database-backed web apps: performance bugs in the wild

#257

Post by mklepaczewski »

dgritsko wrote: post #252
mklepaczewski wrote: post #248

Earlier quoted context omitted.

1 - TTL should be infinite. If one needs finite TTL it means that cache invalidation logic is bogus.

Or it could mean that the cost of cache invalidation is greater than the cost of allowing stale data to be read for the remainder of the TTL, or any other number of justifications for finite TTL. The point is, it depends on your application - while it's true that shorter TTLs can mask faulty invalidation logic, I don't think it's correct to say that "TTL should be infinite" with no qualifiers whatsoever.

If that's the desired behavior then cache layer should allow one to use stale (invalidated ) content. Cache layer (or application layers above it) should be aware of the status of cached data.

It's fine if stale data is used deliberately, problems arise when stale data is assumed to be 'fresh'.

Re: How not to structure database-backed web apps: performance bugs in the wild

#258

Post by Jach »

ndrr wrote: post #234
tootie wrote: post #215

Earlier quoted context omitted.

I've heard it called Cache Stampede. Any decent framework for memoizing method calls would cover this case though.

Do you have an example of one? I'm curious what features they provide over ad hoc memoization mechanics.

If you're in the Java ecosystem, the CacheBuilder in Guava is pretty good: https://google.github.io/guava/releases/19.0/api/docs/com/go...

By default it handles the case of concurrent retrieval on the same key (the second one will just wait for the first one to finish and use that value rather than starting a duplicate computation). It also lets you configure more interesting things like eviction strategies, removal notifications, and statistics.

Last year was a lesson for me in why caches are a hard problem as I had to debug many cache issues from other people not thinking things through... (At least one of the issues was my own fault. :)) Since then whenever someone suggests we use a cache I instinctively pull out a set of questions[0] to ask. The three questions Guava has you consider can also lead you to using memcached or the like instead but my set tries to answer the question "Do you even need a cache?" and if so, generating helpful design documentation.

    Is the code path as fast as it can possibly be without the cache? Do you have numbers?
    Will the cache have a maximum size, or could it grow without bound?
    If it grows without bound, either because of unbounded entries or because of unbounded memory for any particular entry, under what conditions will it consume all system memory?
    If it has a maximum size, how does it evict things when it reaches that size?
    Are you trying to cache something that could change?
    If so, how is the cache invalidated?
    How can it be invalidated / evicted manually by another thread or signal? (Debuggability, testability, inspectability/monitorability, hit rate and other statistics?)
    Is there a race condition possibility for concurrent stores, retrieves, and evicts?
    How constrained are your cache keys, that is, what does it need to know about to create one?
    Do they need to take into account global info?
    Do they need to take into account contextual information (like organization ID if your application server runs in a multi-tenant system, or user ID, or browser type, or requested-language)?
    Or do they only depend on direct inputs to that code path?
[0] https://www.thejach.com/view/2017/6/caches_are_evil -- need to update it a bit but not much...

Re: How not to structure database-backed web apps: performance bugs in the wild

#259

Post by mklepaczewski »

snowwrestler wrote: post #254
mklepaczewski wrote: post #248

Earlier quoted context omitted.

1 - TTL should be infinite. If one needs finite TTL it means that cache invalidation logic is bogus.

True but cache invalidation is famously hard to get perfect. A nicer way to put it might be that finite TTLs should be viewed as an opportunity to optimize, not ideal or standard.

True, caching even simple data dependant on 2-3 variables may require writing dozens of test cases - in practice, it often turns out that the data doesn't rely on 2-3 but 5-10+ variables.

However, just because it's hard it doesn't mean we shouldn't do it the right way.

Re: How not to structure database-backed web apps: performance bugs in the wild

#260

Post by spraak »

dustingetz wrote: post #228
latch wrote: post #23

I've worked on moderately busy backend platforms (~10K-20k rps handled on a ~4 e5-2650 and aiming for 5ms 95p response times). It greatly depends on what you're doing, but for the majority of systems which are read heavy (and that most certainly includes "dynamic" sites like Amazon or Wikipedia), I hold to two major beliefs: 1 - Have very long TTLs on your internal cache servers with a way to proactively purge (messa…

The amazing thing about 1 is that modern immutable database patterns get it for free. When things like https://www.datomic.com/ get mainstream the world is gonna be in a very different place. Not as in today's apps get faster, but as in they are so much faster that new kinds of apps become possible that couldn't have been dreamed of before. My startup http://www.hyperfiddle.net is an experiment in this space

I hope you can hire a designer and frontend engineer soon! I understand it's an experiment but the site looks 15 years outdated. In any case it seems interesting, good luck!
Post reply on HN