Live data from Hacker News

Show HN: Pocache, preemptive optimistic caching for Go

github.com

21–30 of 33 posts

Re: Show HN: Pocache, preemptive optimistic caching for Go

#21
post #20
post #11

Earlier quoted context omitted.

I'm confused by the decision in DoChan to return a channel (instead of accepting one supplied by the caller) and then, given that, also not to close that channel (is something else going to be sent to the channel in the future?). Both seem like strange/unnecessary design decisions.

Returning a channel avoids questions of what happens if sending to a caller-supplied channel blocks. DoChan returns a channel with a single-element buffer, so a single send to the channel will always succeed without blocking, even if the caller has lost interest in the result and discarded the channel. DoChan doesn't close the channel because there isn't any reason to do so.

A non-blocking send would work just as well for that issue, is a standard part of the language, and would support user-supplied channels, but it would still be at risk of panicking when sending to a closed channel. I think there ought to be a safe way to send to a closed channel, but the language authors disagree, so that's not really on the library authors (though they could still recover from the panic).

However, not closing the channel you specifically chose to control all sending to is just lazy/rude. Even though the caller should receive from the channel once and then forget about it, closing the channel after sending would prevent incorrect subsequent receives from hanging forever.

All this having been said, contributing to these libraries seems better than complaining about them, but I don't know how the golang.org/x stuff is maintained; looks like this one is here: https://github.com/golang/sync

Re: Show HN: Pocache, preemptive optimistic caching for Go

#22
post #21
post #20

Earlier quoted context omitted.

Returning a channel avoids questions of what happens if sending to a caller-supplied channel blocks. DoChan returns a channel with a single-element buffer, so a single send to the channel will always succeed without blocking, even if the caller has lost interest in the result and discarded the channel. DoChan doesn't close the channel because there isn't any reason to do so.

A non-blocking send would work just as well for that issue, is a standard part of the language, and would support user-supplied channels, but it would still be at risk of panicking when sending to a closed channel. I think there ought to be a safe way to send to a closed channel, but the language authors disagree, so that's not really on the library authors (though they could still recover from the panic). However, n…

A non-blocking send doesn't work in this case. Consider: User provides DoChan an unbuffered channel, and then reads a value from it. If the send is nonblocking and occurs before the user reads from the channel, the value is lost.

Re: Show HN: Pocache, preemptive optimistic caching for Go

#23
post #16

Earlier quoted context omitted.

@zimpenfish yes you are right. refetch is initiated on the first Get between 9-10mins, and the timer is reset as soon as the back fetch is successful

One optimization for background refresh is coalescing the individual reloads into a batch operation based on a time/space window. Here is how we do it in the Java world. [1] [1] https://github.com/ben-manes/caffeine/tree/master/examples/c...

Thank you for your OSS work! I used Caffeine many years ago.

Re: Show HN: Pocache, preemptive optimistic caching for Go

#24
post #23
post #16

Earlier quoted context omitted.

One optimization for background refresh is coalescing the individual reloads into a batch operation based on a time/space window. Here is how we do it in the Java world. [1] [1] https://github.com/ben-manes/caffeine/tree/master/examples/c...

Thank you for your OSS work! I used Caffeine many years ago.

Oh thank you, I’m glad it’s been helpful.

Re: Show HN: Pocache, preemptive optimistic caching for Go

#25

I have implemented my own SIEVE cache, with TTL support. It solves all these issues and requires no background workers. Author, or anyone else interested in this, should read the SIEVE paper/website and implement their own.

https://github.com/opencoff/go-sieve

https://github.com/scalalang2/golang-fifo

Re: Show HN: Pocache, preemptive optimistic caching for Go

#26
post #21
post #20

Earlier quoted context omitted.

Returning a channel avoids questions of what happens if sending to a caller-supplied channel blocks. DoChan returns a channel with a single-element buffer, so a single send to the channel will always succeed without blocking, even if the caller has lost interest in the result and discarded the channel. DoChan doesn't close the channel because there isn't any reason to do so.

A non-blocking send would work just as well for that issue, is a standard part of the language, and would support user-supplied channels, but it would still be at risk of panicking when sending to a closed channel. I think there ought to be a safe way to send to a closed channel, but the language authors disagree, so that's not really on the library authors (though they could still recover from the panic). However, n…

Closing the channel is pointless. I don't understand why people get obsessive about closing channels.

It's not needed by the garbage collector, it's not good practice. It's explicitly called out in the official go guide as unnecessary most of the time. [0]

If you have a channel that is only used a single time and then discarded, closing it is literally just wasting CPU cycles. And definitely not "lazy/rude".

[0] https://go.dev/tour/concurrency/4

Re: Show HN: Pocache, preemptive optimistic caching for Go

#27
post #16

Earlier quoted context omitted.

@zimpenfish yes you are right. refetch is initiated on the first Get between 9-10mins, and the timer is reset as soon as the back fetch is successful

One optimization for background refresh is coalescing the individual reloads into a batch operation based on a time/space window. Here is how we do it in the Java world. [1] [1] https://github.com/ben-manes/caffeine/tree/master/examples/c...

aha yes! It's in my todo list to introduce bulk updates. On the other hand, I'll be publishing a batcher package soon which does something very close to what you suggested here. Thank you

Re: Show HN: Pocache, preemptive optimistic caching for Go

#28
post #5

PSA: You can also use singleflight[1] to solve the problem. This prevents the thundering herd problem. Pocache is an interesting/alternative way to solve thundering herd indeed! [1]: https://pkg.go.dev/golang.org/x/sync/singleflight

thank you for the recommendation, was a good read as well. I could even use it to replace how I'm handling the call suppression/debounce mechanism. Though I think Pocache does 1 extra thing, which is to keep the cache updated before it expires, i.e. for keys which are frequently fetched it'd serve up to date data always from the cache. If we only relied on call suppression, then the concurrent requests would just have to wait during the update stage, or the read-through mechanism would keep hitting the main database.

Re: Show HN: Pocache, preemptive optimistic caching for Go

#29
post #6

You may be interested in Groupcache's method for filling caches, it solves the same problem that I believe this project is aimed at. Groupcache has a similar goal of limiting the number of fetches required to fill a cache key to one—regardless of the number of concurrent requests for that key—but it doesn't try to speculatively fetch data, it just coordinates fetching so that all the routines attempting to query the…

hey thank you for sharing this. Based on what I understand, this package focuses on the underlying storage mechanism itself rather than helping with the cache strategy. It seems like a solid storage extension which can be used Pocache!

Re: Show HN: Pocache, preemptive optimistic caching for Go

#30

I have implemented my own SIEVE cache, with TTL support. It solves all these issues and requires no background workers. Author, or anyone else interested in this, should read the SIEVE paper/website and implement their own.

that was an interesting read (https://junchengyang.com/publication/nsdi24-SIEVE.pdf), thanks for the recommendation. It looks like a good fit for replacing the underlying storage mechanism of Pocache, instead of the LRU. Though I do not think it addresses the thundering herd problem, where the underlying database would be flooded with calls when the cache expires. I think Pocache is focusing more on the caching strategy itself rather than the cache eviction or storage mechanisms. Hence the store is configurable for Pocache.
Post reply on HN