What's the difference to using joblibs Memory class similar to this implementation:
https://github.com/stanfordnlp/dspy/blob/main/dsp/modules/ca...
Show HN: File-based cache for slow Python functions
51–60 of 60 posts
Re: Show HN: File-based cache for slow Python functions
#52Earlier quoted context omitted.
Using md5 for this seems like an odd choice. Sha1 is a better choice even for non-cryptographic use cases, it's quite a bit faster than md5. Even better would be something like xxhash! According to a quick bash script I wrote to benchmark the popular hash functions, md5 comes out last compared to sha1, sha256, sha512, and blake2, and by a decent margin! A good rule of thumb is to never use md5 at all. Not even for no…
That sounds great, I'm going to see how Sweep does on this issue: https://github.com/sweepai/sweep/issues/3333
I think python objects have a __hash__ method available on them as well that can be used for hashing. That should be even much faster than sha1, but for this use case I'm not sure how much it really matters. Would be interesting to benchmark!
Re: Show HN: File-based cache for slow Python functions
#53Recently, I experimented with various techniques to cache some JSON responses from FastAPI, using Python decorators for both in-memory and disk caching on a single machine. After benchmarking the performance, I found the results somewhat disappointing (500 req/s vs 5k req/s). While caching did lead to a tenfold improvement in speed compared to no caching, I believe the primary bottleneck was Python's inherent performance limitations, which made it X times slower than a comparable program written in C. Consequently, I decided to remove the cache decorator and instead put a simple nginx caching reverse proxy in front of FastAPI. This resulted in performance gains that were an order of magnitude better (60k req/s) than those achieved with Python based caching.
Re: Show HN: File-based cache for slow Python functions
#54What's the difference to using joblibs Memory class similar to this implementation: https://github.com/stanfordnlp/dspy/blob/main/dsp/modules/ca...
I was going to mention this as well. It's fairly similar:
memory = joblib.memory.Memory(...)
@memory.cache
def slow_func(...):
...Re: Show HN: File-based cache for slow Python functions
#55What's the difference to using joblibs Memory class similar to this implementation: https://github.com/stanfordnlp/dspy/blob/main/dsp/modules/ca...
I was going to mention this as well. It's fairly similar: memory = joblib.memory.Memory(...) @memory.cache def slow_func(...): ...
The diskcache docs state:
""" Caching Libraries
joblib.Memory provides caching functions and works by explicitly saving the inputs and outputs to files. It is designed to work with non-hashable and potentially large input and output data types such as numpy arrays.Re: Show HN: File-based cache for slow Python functions
#56Earlier quoted context omitted.
I was going to mention this as well. It's fairly similar: memory = joblib.memory.Memory(...) @memory.cache def slow_func(...): ...
The diskcache docs state: """ Caching Libraries joblib.Memory provides caching functions and works by explicitly saving the inputs and outputs to files. It is designed to work with non-hashable and potentially large input and output data types such as numpy arrays. """ From https://pypi.org/project/diskcache/
This is great! I see it also supports an 'ignore' parameter.
Re: Show HN: File-based cache for slow Python functions
#57Recently, I experimented with various techniques to cache some JSON responses from FastAPI, using Python decorators for both in-memory and disk caching on a single machine. After benchmarking the performance, I found the results somewhat disappointing (500 req/s vs 5k req/s). While caching did lead to a tenfold improvement in speed compared to no caching, I believe the primary bottleneck was Python's inherent perform…
We also found a lot of cases where caching ends up being actually slower than doing the operation. The 100% solution would probably be to use a SQL db the way diskcache does it, but this is easier to use for us.
Re: Show HN: File-based cache for slow Python functions
#58Reminds me of a little prototype I wrote a while ago that tried to do something similar with Javascript's Proxy class. https://github.com/emileindik/cashola
The main difference is that it stores the state of an object, not a function.
If your data is JSON serializable then it could be a cool way to save and resume application state.
Re: Show HN: File-based cache for slow Python functions
#59Earlier quoted context omitted.
I found DiskCache sometime last year, it's amazing. Very simple to set up and works great as a cache for so many different things.
What are you using it for? A disk based cache seems almost contradictory for my use cases, I would love to hear yours. Anything that I would store on disk, even as a cache, I can generally put in SQLite.
I write some toy Python scripts/apps for a couple APIs (the Pokeapi, Spacetraders, etc). I use the HTTPX library as a request client, and wasn't aware of the Hishel library for caching requests.
I used DiskCache to cache responses for ~15 minutes so I wasn't sending live requests every time I tested the app.
I'm not building anything "cloud scale," most things I run are off my local machine. Having a convenient, fast local cache that's simple to use (DiskCache) has so many uses, it's hard to think of them all! I might use a cache with no expiration to store some configs, or to store a serialized object for later retrieval. I might use it as an in-memory object cache while the program loads, so I don't have to spin up a Redis server.
Re: Show HN: File-based cache for slow Python functions
#60This looks cool :). A while ago, I wrote something similar that analyzes bytecode and invalidates the cache if the bytecode changes.