Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: File-based cache for slow Python functions

docs.sweep.dev

Post reply on HN
Print view View original

1–10 of 60 posts

Re: Show HN: File-based cache for slow Python functions

#3

Post by skp1995 »

This is a pretty good implementation. I like the simplicity of it, reminds me of SQLite backed storage decorators we used to have, where the data was persisted to a DB instead of the file system (altho thats just a different storage engine)

Does this also take care of the thundering heard problem? That was one of the cases where lru_cache really blows

Re: Show HN: File-based cache for slow Python functions

#4

Post by williamzeng0 »

kapilsinha wrote: post #2

I like the simplicity. I definitely get the payoff for standalone Python scripts, where once the script errors out the memory is cleared. But do you see a similar payoff for Jupyter notebooks (or similar)?

I think the marginal gain would be a lot less for Jupyter notebooks, but I've definitely rerun individual cells and wasted time there before.

I think it could help if you forget to save the output of a function within a single cell like this:

1. print(f(x)) # -> check what happened 2. out = f(x) # -> turns out we want to save this, so we have to wait again

Re: Show HN: File-based cache for slow Python functions

#5

Post by williamzeng0 »

skp1995 wrote: post #3

This is a pretty good implementation. I like the simplicity of it, reminds me of SQLite backed storage decorators we used to have, where the data was persisted to a DB instead of the file system (altho thats just a different storage engine) Does this also take care of the thundering heard problem? That was one of the cases where lru_cache really blows

Unfortunately it doesn't, we typically don't expect to handle high load with this cache and actually disable it in production with another envvar.

Sometimes caching can actually be slower for certain functions, because just performing that operation is faster than pickle.load/pickle.dump.

Re: Show HN: File-based cache for slow Python functions

#7

Post by quickslowdown »

rmholt wrote: post #6

I have extensively used https://pypi.org/project/diskcache/ . Is there a reason you decided to make an in house solution?

I found DiskCache sometime last year, it's amazing. Very simple to set up and works great as a cache for so many different things.

Re: Show HN: File-based cache for slow Python functions

#8

Post by dotancohen »

quickslowdown wrote: post #7
rmholt wrote: post #6

I have extensively used https://pypi.org/project/diskcache/ . Is there a reason you decided to make an in house solution?

I found DiskCache sometime last year, it's amazing. Very simple to set up and works great as a cache for so many different things.

What are you using it for? A disk based cache seems almost contradictory for my use cases, I would love to hear yours. Anything that I would store on disk, even as a cache, I can generally put in SQLite.

Re: Show HN: File-based cache for slow Python functions

#9

Post by wildermuthn »

If you aren’t caching LLM functions during development, then you’re an even greater glutton for punishment than the normal engineer.

My local file cache Python decorator also allows the decorator to define the hash manually, either by the decorator’s parameter function call that plucks a value from the cached function params, or by calling a global function from anywhere with any arbitrary value.

What’s cool about caching results locally to files during development is the ease of invalidating caches — just delete the file named after the function and key you want.

Post reply on HN