11–20 of 27 posts
One thing you can do is use :ets.slot/2 to do random probing of the cache and evict entries. The way to do so is simple, just have a process every few milliseconds check a random set of keys (say, 50 keys), and check if they expire. If >25% of keys were about to expire, repeat this process again instantly, until the % of keys expired in the batch is This is how Redis versions prior to 6.0 implemented key expiry. It's…
You can even get a simple cache running quite quickly with a Genserver and ETS. About the post, I'm not sure what dcache offers over Nebulex though.
Nebulex looks more complicated than dcache, just comparing the getting started info: "In order to give more flexibility and loading only needed dependencies, Nebulex makes all its dependencies as optional." Then it goes on to describing shards, decorators, telemetry and external adapters like Cachex and Redis. So before I can run `mix deps.get` to include this package I now have to make decisions or at least read thru what those things are for this package.
I prefer picking simple dependencies and add complexity later on if needed.
"nil is greater than any integer", that was astounding. Which languages have a no-data is bigger than max_int rule?
There is a total order on (almost) all data types in the beam. This is useful for arbitrary sorting and comparison of different types. I say almost because floats and their equivalent integer are different items but they are equal in the total order. In practice this is fine, and I have never heard of this causing a problem.
Also, for some languages, not failing with an exception when comparing different strong types, could be a source of bugs, generally.
The purging is a scan with `next()` and a `lookup()` for each item. Since the logic is a simple `<` comparison, could this be done with a single `match_delete` instead? (Unfortunately, ETS match specifications are quite wonky, so without a lot of fiddling myself, I can't suggest exactly what the comparison logic would look like, but I know that you can express certain things like comparisons with it.) That would cut…
Matchspecs are gnarly! Very annoying to wrap your head around, and it's closer to being an AST than anything.
Shameless plug: I had written a library at one point to make matchspecs with Mnesia easier, but it might also work with ETS (haven't tested unfortunately): https://github.com/queer/lethe
The purging is a scan with `next()` and a `lookup()` for each item. Since the logic is a simple `<` comparison, could this be done with a single `match_delete` instead? (Unfortunately, ETS match specifications are quite wonky, so without a lot of fiddling myself, I can't suggest exactly what the comparison logic would look like, but I know that you can express certain things like comparisons with it.) That would cut…
One thing you can do is use :ets.slot/2 to do random probing of the cache and evict entries. The way to do so is simple, just have a process every few milliseconds check a random set of keys (say, 50 keys), and check if they expire. If >25% of keys were about to expire, repeat this process again instantly, until the % of keys expired in the batch is This is how Redis versions prior to 6.0 implemented key expiry. It's…
Earlier quoted context omitted.
There is a total order on (almost) all data types in the beam. This is useful for arbitrary sorting and comparison of different types. I say almost because floats and their equivalent integer are different items but they are equal in the total order. In practice this is fine, and I have never heard of this causing a problem.
For me, no data is either NaN or zero, but +infinity is was the astounding part. Also, for some languages, not failing with an exception when comparing different strong types, could be a source of bugs, generally.
Elixir/BEAM is strongly typed. There is no coercion going on when you are doing comparisons.
[0] note that in elixir, "if" is in the stdlib, it's sugar over case, and the falsiness of nil is software-level (only false and nil are falsy in elixir, which is sane).
You can even get a simple cache running quite quickly with a Genserver and ETS. About the post, I'm not sure what dcache offers over Nebulex though.
To keep it simple, you could rewrite this post using ex_shards as it handles scaling ETS (which IIRC is the backend by default for Nebulex).
That said, I was able to write a distributed session store with Nebulex pretty quickly. Then extended it to support live sessions, though it's not up to feature parity with its ETS counterpart if anyone wants to help out with my PR https://github.com/pentacent/phoenix_live_session/pull/14
This why I like hobby projects. In my day-to-day work, my job is to deliver value and "get stuff done". However, in hobby projects, I can feel free to dive into an optimization rabbit hole and learn a whole bunch of stuff. Even if I don't ship my hobby projects, I still gain a lot from the background knowledge.
One thing you can do is use :ets.slot/2 to do random probing of the cache and evict entries. The way to do so is simple, just have a process every few milliseconds check a random set of keys (say, 50 keys), and check if they expire. If >25% of keys were about to expire, repeat this process again instantly, until the % of keys expired in the batch is This is how Redis versions prior to 6.0 implemented key expiry. It's…
Why not use a min heap to track expiry times? (Or a doubly-linked list, if lifetimes are uniform?)