Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

In praise of memcached

jchri.st

Post reply on HN
Print view View original

71–80 of 113 posts

Re: In praise of memcached

#71

Post by gnz11 »

foobarian wrote: post #12
dvt wrote: post #9

Memcached is meant to be a lightweight memory cache, which makes sense, but contrary to the article's claim that "Redis is brought into a stack as a cache, and it is run with the assumption that people treat it that way"—I have very very rarely experienced this. Redis is brought into a stack because (most importantly!) it's fast and (almost as importantly!) because it's simple. I don't think this article is written b…

The memcache slab pools are a leaky abstraction that you may end up having to manage operationally, and it's another way Redis is simpler.

Agreed. Memcache is great until you get into the business of having to configure slabs. Most people just reach for redis at that point.

Re: In praise of memcached

#72

Post by vshulcz »

foobarian wrote: post #12
dvt wrote: post #9

Memcached is meant to be a lightweight memory cache, which makes sense, but contrary to the article's claim that "Redis is brought into a stack as a cache, and it is run with the assumption that people treat it that way"—I have very very rarely experienced this. Redis is brought into a stack because (most importantly!) it's fast and (almost as importantly!) because it's simple. I don't think this article is written b…

The memcache slab pools are a leaky abstraction that you may end up having to manage operationally, and it's another way Redis is simpler.

[flagged]

Re: In praise of memcached

#73

Post by 0xCAP »

downsplat wrote: post #64

Redis is a great piece of tech but it suffers from trying to be good at two different jobs (persistent data structures, volatile cache) which should not be combined. And indeed in Redis itself they don't combine well - persistence is globally on or off. Personally I'd use memcached or some equivalent for strictly cacheing, and then bring on Redis with persistence if you need its data structures for e.g scoreboards. A…

> We could change the fs layer to memcached but so far it's working great.

This so much. (Ab)Using a db table as a k/v store + the FS can do so much before even considering paying the price of setting up a dedicated caching store. I’ve fought countless foes in the engineering world when proposing solutions like yours just because (incompetent) people feel like caching should live in its dedicated store.

Re: In praise of memcached

#74

Post by [deleted] »

bawolff wrote: post #36

I like memcached, but its really not redis's fault if you set it up as a volatile cache but people treat it as a persistent data store. The comparison is especially weird as memcached is also not persistent.

[deleted]

Re: In praise of memcached

#75

Post by jitl »

memcached is about a bazillion times faster than redis at doing the simple KV cache job. it’s got threads. it’s highly optimized to do its one job super well, where redis is more a arbitrary shared Python heap kinda thing with all the data structures and single thread and whatnot.

at notion we use redis for a lot of things, but actual caching we leave to memcached

Re: In praise of memcached

#76

Post by roryirvine »

downsplat wrote: post #64

Redis is a great piece of tech but it suffers from trying to be good at two different jobs (persistent data structures, volatile cache) which should not be combined. And indeed in Redis itself they don't combine well - persistence is globally on or off. Personally I'd use memcached or some equivalent for strictly cacheing, and then bring on Redis with persistence if you need its data structures for e.g scoreboards. A…

I had some experience wrangling Memcachedb (memcache + bdb for persistence) in the late 2000s, and came to much the same conclusion.

Redis was definitely more featureful, and antirez is both an engaging character and admirably humble, so I can see why redis overtook it in popularity - but, for me, memcached has always been the pinnacle of "choose boring technology".

As a platform engineer, I'm happy to support either - but when developers start using some of the more advanced redis features (persistence, replication, clustering), I try to make sure that they've fully understood the downsides of that decision.

Re: In praise of memcached

#77

Post by msandford »

AussieWog93 wrote: post #66
hosteur wrote: post #51

Earlier quoted context omitted.

I am currently in the process of starting a project with Flask, SQLAlchemy, Celery. Say more about why I should avoid Celery and what to use instead.

Things like chaining, groups, named queues just don't work the way you'd think they would. There's a lot of footguns and things require weird workarounds. Error reporting is misleading. It's not bad enough where I had to pull it from the old project that used it, but going forward the new ones used a vibecoded queueing system that was genuinely more reliable than Celery but consumed a lot of memory (RSS inflation). H…

Huh that's interesting! I found celery to mostly match my expectations. I used it in a couple of django apps. My only real foot gun was around having to set an EAGER setting for local development or tasks never got executed.

How did you find your expectations and celery's actual semantics to be different? I'm trying to document well and it seems like I might have some implicit assumptions that I could make explicit, but I don't know what they are since they're already in my head and matching celery it seems.

Re: In praise of memcached

#78

Post by WJW »

alt227 wrote: post #54
AussieWog93 wrote: post #39

I've done a bunch of Flask work over the past couple of years - not full time but as part of the tech stack for my small eCommerce business. Have run into all kinds of footguns and weirdness with MongoEngine, SQLAlchemy, Celery (seriously, if you value your sanity, don't use Celery!), the Python stacks for Google, eBay and Shopify but never Redis. Perhaps that's because I'm not giving admin access to random people wh…

> every time I need to do something slightly weird, there's a sensible and well thought out way to achieve it. In my world cache systems like memcached and redis are just that, a cache to put and get from. Possibly use some invalidation system like tagging. What can you do with a cache system that is 'wierd'? What are people doing with caches other than just caching data? Genuinely interested.

Non-caching things I regularly see people do with Redis:

- Rate limits for API endpoints via the leaky bucket algorithm

- Feature flags and stats tracking

- Websocket pub/sub

- Background job queue

In general, lots of things that need to survive deploys (so they can't be in-memory in the app) and/or they need to be coordinated across multiple horizontally scaled servers and/or things that prefer to be in a data structure which is slightly awkward to stick in a database table.

Re: In praise of memcached

#79

Post by WJW »

alt227 wrote: post #57
boesboes wrote: post #55

Earlier quoted context omitted.

I’ve done moving window rate limiting using redis to do atomic rate calculations etc. That requires some weirdness

> moving window rate limiting So does that mean you are tracking how many times data is being entered into redis, and rejecting it if the entry rate is too high? Why would you not track this before, at the point of calculating the data to enter into redis, rather than querying redis to see how much data is entered in a given timeframe? Again, genuinely curious as to the reason for architectural decisions.

Not GP, but I think they mean usecases like limiting how many times any given IP address can access an API to a certain amount of calls per minute. For example, you might want to restrict login attempts to at most 10 per minute per IP to prevent people trying out lists of common passwords.

This is fairly easy to do if your apps runs on a single server, but many companies run multiple servers and load balance requests among them. Those servers need some sort of coordination mechanism to keep track of the rate limits and their current state. Redis has dedicated instructions these days to do this, and in the old days there were plethora of libraries that use embedded Lua scripts to do the same thing.

Re: In praise of memcached

#80

Post by bloppe »

hosteur wrote: post #51
AussieWog93 wrote: post #39

I've done a bunch of Flask work over the past couple of years - not full time but as part of the tech stack for my small eCommerce business. Have run into all kinds of footguns and weirdness with MongoEngine, SQLAlchemy, Celery (seriously, if you value your sanity, don't use Celery!), the Python stacks for Google, eBay and Shopify but never Redis. Perhaps that's because I'm not giving admin access to random people wh…

I am currently in the process of starting a project with Flask, SQLAlchemy, Celery. Say more about why I should avoid Celery and what to use instead.

What are you using Celery for? Do you need to be able to recover from a reboot or crash with your queues intact? Is it a distributed system as opposed to a single machine? Do you have complex multi-step workflows?

If you answered no to each question, just `import Queue from queue`.

Post reply on HN