21–27 of 27 posts
Elements added during the iteration may be returned, or not, at random. I can't be the only one who thinks this is crazy behaviour.
It is actually very reasonable. Other alternative is that iterator returns an error if underlying collection was modified (fail fast iterators).
Elements added during the iteration may be returned, or not, at random. I can't be the only one who thinks this is crazy behaviour.
Elements added during the iteration may be returned, or not, at random. I can't be the only one who thinks this is crazy behaviour.
Schema migrations involving tens of millions of keys can be challenging. The KEYS command returns all matching keys, so you can roll your own iterator with KEYS AAA* then KEYS AAB* etc. But keys aren't evenly distributed so some result sets are zero and others are still too large.
I've been indexing keys with Redis lists and iterate through the index with LRANGE. That uses extra memory and causes more roundtrip calls.
I can deal with this compromise since SCAN makes Redis much more useful for my applications.
Elements added during the iteration may be returned, or not, at random. I can't be the only one who thinks this is crazy behaviour.
How is this any different from an iterator in Java? IIRC, if you have the iterator of a Collection and add elements to it during iteration, the results are undefined. You can, however, safely remove elements even while iterating.
Elements added during the iteration may be returned, or not, at random. I can't be the only one who thinks this is crazy behaviour.
Encode the size of the collection in the first 8 bits of the cursor value, and when the client gives you an cursor value with the first 8 bits set to a smaller size than the current size of the collection, start it over from scratch with the new size as the top 8 bits.
If the collection has shrunk, no big deal, just keep going from where the client was because of the mentioned properties of using a reversed bit set.
This is cool and helpful, but not a game-changer. For things like garbage-collecting I've been using probabilistic techniques (just get a random element and check it, make sure you check enough to have guarantees you need) with great success. The new scanning doesn't provide any tight guarantees (it can't, really, without sacrificing a lot of what Redis stands for), so it will be more of a convenience than a really n…
I agree that for some data type like Sets you could use SRANDMEMBER, however to sample all the elements by random sampling you need to perform a lot more work. For example for a 10000 elements collection in average you are going to need around 10 times the requests.