11–20 of 60 posts
Earlier quoted context omitted.
I found DiskCache sometime last year, it's amazing. Very simple to set up and works great as a cache for so many different things.
What are you using it for? A disk based cache seems almost contradictory for my use cases, I would love to hear yours. Anything that I would store on disk, even as a cache, I can generally put in SQLite.
I have extensively used https://pypi.org/project/diskcache/ . Is there a reason you decided to make an in house solution?
How it fares with several millions of cached objects?
It's a very mature library, too, nice and polished, I've never once experienced a bug with it.
- from your code it seems you're not sorting kwargs, I would strongly recommend sorting them so that whether you call f(a=1, b=2) or f(b=2, a=1) the cache key is the same
- I use inspect.signature to convert all args to kwargs, this way it doesn't matter how a function gets called, the cache logic is always consistent. I know this is relatively slow but it only gets called once per function (I call it outside the wrapper) and the DX benefits are nice (in this same note, you could probably move the inspect.getsource call outside your wrapper fn for a speed boost)
I also took the opposite approach to ignore_params, and made the __dict__ params that get hashed opt-in, which works well when caching instance methods
I have extensively used https://pypi.org/project/diskcache/ . Is there a reason you decided to make an in house solution?
mature and works well
If you aren’t caching LLM functions during development, then you’re an even greater glutton for punishment than the normal engineer. My local file cache Python decorator also allows the decorator to define the hash manually, either by the decorator’s parameter function call that plucks a value from the cached function params, or by calling a global function from anywhere with any arbitrary value. What’s cool about ca…
I recently wrote a version of this that I use in my projects, some things I do differently that you may or may not care about: - from your code it seems you're not sorting kwargs, I would strongly recommend sorting them so that whether you call f(a=1, b=2) or f(b=2, a=1) the cache key is the same - I use inspect.signature to convert all args to kwargs, this way it doesn't matter how a function gets called, the cache…
If you aren’t caching LLM functions during development, then you’re an even greater glutton for punishment than the normal engineer. My local file cache Python decorator also allows the decorator to define the hash manually, either by the decorator’s parameter function call that plucks a value from the cached function params, or by calling a global function from anywhere with any arbitrary value. What’s cool about ca…
This is also why in my custom cache I back it with sqlite – much easier to delete one db file than thousands of pickle files.
I recently wrote a version of this that I use in my projects, some things I do differently that you may or may not care about: - from your code it seems you're not sorting kwargs, I would strongly recommend sorting them so that whether you call f(a=1, b=2) or f(b=2, a=1) the cache key is the same - I use inspect.signature to convert all args to kwargs, this way it doesn't matter how a function gets called, the cache…
Very insightful comment, but can I ask what DX stands for? Maybe I'm missing something obvious.
I recently wrote a version of this that I use in my projects, some things I do differently that you may or may not care about: - from your code it seems you're not sorting kwargs, I would strongly recommend sorting them so that whether you call f(a=1, b=2) or f(b=2, a=1) the cache key is the same - I use inspect.signature to convert all args to kwargs, this way it doesn't matter how a function gets called, the cache…
Very insightful comment, but can I ask what DX stands for? Maybe I'm missing something obvious.