Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: File-based cache for slow Python functions

docs.sweep.dev

Post reply on HN
Print view View original

21–30 of 60 posts

Re: Show HN: File-based cache for slow Python functions

#21

Post by [deleted] »

AlecSchueler wrote: post #17
mpeg wrote: post #14

I recently wrote a version of this that I use in my projects, some things I do differently that you may or may not care about: - from your code it seems you're not sorting kwargs, I would strongly recommend sorting them so that whether you call f(a=1, b=2) or f(b=2, a=1) the cache key is the same - I use inspect.signature to convert all args to kwargs, this way it doesn't matter how a function gets called, the cache…

Very insightful comment, but can I ask what DX stands for? Maybe I'm missing something obvious.

[deleted]

Re: Show HN: File-based cache for slow Python functions

#22

Post by mpeg »

AlecSchueler wrote: post #18
mpeg wrote: post #16

Earlier quoted context omitted.

This is also why in my custom cache I back it with sqlite – much easier to delete one db file than thousands of pickle files.

Globs are a thing?

Weird comment... yes, they are, but what is faster for me to type `rm .cache_dir/function_name*.pickle` or just to delete the one sqlite file in my file manager / vscode file tree.

Regardless, there are other reasons why sqlite is nice for this, you gain control over locking and thread safety without having to implement it all from scratch

Re: Show HN: File-based cache for slow Python functions

#23

Post by rmholt »

dotancohen wrote: post #8
quickslowdown wrote: post #7

Earlier quoted context omitted.

I found DiskCache sometime last year, it's amazing. Very simple to set up and works great as a cache for so many different things.

What are you using it for? A disk based cache seems almost contradictory for my use cases, I would love to hear yours. Anything that I would store on disk, even as a cache, I can generally put in SQLite.

It pretty much does that, if I was to be a little reductive, DiskCache is just a wrapper around sqlite and pickle

Re: Show HN: File-based cache for slow Python functions

#24

Post by epr » 

  def hash_code(code):
      return hashlib.md5(code.encode()).hexdigest()
Be warned. The above function is used as part of the hash. The ostensible purpose is to prevent using cached values of functions who's code has changed, but it does not handle dependencies of that function.

Re: Show HN: File-based cache for slow Python functions

#25

Post by canadiantim »

wildermuthn wrote: post #9

If you aren’t caching LLM functions during development, then you’re an even greater glutton for punishment than the normal engineer. My local file cache Python decorator also allows the decorator to define the hash manually, either by the decorator’s parameter function call that plucks a value from the cached function params, or by calling a global function from anywhere with any arbitrary value. What’s cool about ca…

I'm sure this is a stupid question, but why is it much better to be caching LLM functions during development?

Re: Show HN: File-based cache for slow Python functions

#26

Post by CGamesPlay »

rmholt wrote: post #6

I have extensively used https://pypi.org/project/diskcache/ . Is there a reason you decided to make an in house solution?

I was curious to see an alternative to this, but how is this an alternative? You're saying I can implement my own caching of function calls that invalidates when the arguments or source code change...? These feel like entirely separate layers. Did I miss where diskcache does this stuff?

Re: Show HN: File-based cache for slow Python functions

#27

Post by martinky24 »

epr wrote: post #24

def hash_code(code): return hashlib.md5(code.encode()).hexdigest() Be warned. The above function is used as part of the hash. The ostensible purpose is to prevent using cached values of functions who's code has changed, but it does not handle dependencies of that function.

How do you suggest one might fix that issue? Also pin the cache to a hash of all dependency versions? And then if one minor update And let's say the dependency did change, but it's generally inert (more error handling around edge cases, for example), how do you factor that in? Blow up the whole cache?

Your example isn't really a problem with OPs utility, but a specific example of a broader dependency management problem that affects just about everything. The answers usually boil down to 1) invest heavily in a kick ass test suite, 2) never upgrade or 3) upgrade and pray nothing breaks.

Re: Show HN: File-based cache for slow Python functions

#28

Post by nicwolff »

CGamesPlay wrote: post #26
rmholt wrote: post #6

I have extensively used https://pypi.org/project/diskcache/ . Is there a reason you decided to make an in house solution?

I was curious to see an alternative to this, but how is this an alternative? You're saying I can implement my own caching of function calls that invalidates when the arguments or source code change...? These feel like entirely separate layers. Did I miss where diskcache does this stuff?

https://grantjenks.com/docs/diskcache/api.html#diskcache.Cac...

Decorator to wrap callable with memoizing function using cache. Repeated calls with the same arguments will lookup result in cache and avoid function evaluation.

Re: Show HN: File-based cache for slow Python functions

#29

Post by p10_user »

rmholt wrote: post #23
dotancohen wrote: post #8

Earlier quoted context omitted.

What are you using it for? A disk based cache seems almost contradictory for my use cases, I would love to hear yours. Anything that I would store on disk, even as a cache, I can generally put in SQLite.

It pretty much does that, if I was to be a little reductive, DiskCache is just a wrapper around sqlite and pickle

The degree of reduction is nice considering the countless times in the past where I wrote my own file cache logic using if/else statements, temporary files, pickle, and bespoke sqlite databases.

Why give myself the headache of maintaining so much extra code when someone already wrote it.

Re: Show HN: File-based cache for slow Python functions

#30

Post by p10_user »

canadiantim wrote: post #25
wildermuthn wrote: post #9

If you aren’t caching LLM functions during development, then you’re an even greater glutton for punishment than the normal engineer. My local file cache Python decorator also allows the decorator to define the hash manually, either by the decorator’s parameter function call that plucks a value from the cached function params, or by calling a global function from anywhere with any arbitrary value. What’s cool about ca…

I'm sure this is a stupid question, but why is it much better to be caching LLM functions during development?

Because they are generally incredibly computationally expensive operations that can take hours/days to complete (?more)
Post reply on HN