Frugal streaming. For estimating a median or percentile in a stream, using constant memory. It's very simple: if the current value is higher than our estimate, then increase our estimate by one. Else decrease by one. It will converge over long enough time. This is called the Frugal-1U algorithm. The Frugal-2U is a slightly more advanced version that modifies the step size along the way, plus other optimizations, to c…
Ask HN: What are some cool but obscure data structures you know about?
391–400 of 772 posts
Re: Ask HN: What are some cool but obscure data structures you know about?
#392Fenwick Trees (which, despite the name, are implemented using an array) allow counting prefix sums AND updating prefix sums in O(log n) time. Very useful when n is in the order of millions. I have used them a few times in Project Euler problems. https://en.wikipedia.org/wiki/Fenwick_tree
Re: Ask HN: What are some cool but obscure data structures you know about?
#393The union-find data structure / algorithm is useful and a lot of fun. The goal is a data structure where you can perform operations like "a and b are in the same set", "b and c are in the same set" and then get answers to questions like "are a and c in the same set?" (yes, in this example.) The implementation starts out pretty obvious - a tree where every element either points at itself or some thing it was merged wi…
I have always wanted to really understand this data structure. Sure, I can follow the analysis with the potential functions and all, but I never really understood how Tarjan came up with the functions in the first place. Does anybody have a resource which intuitively explains the analysis?
Re: Ask HN: What are some cool but obscure data structures you know about?
#394I wrote a Python library implementing them a number of years ago: https://github.com/benhoyt/pybktree
Re: Ask HN: What are some cool but obscure data structures you know about?
#395Here's one I don't know if I've ever seen documented anywhere. If anyone knows a proper name for this one, let me know! Imagine it's for a text editor, and you want to map Line Numbers to Byte Positions. But then you want to insert a byte somewhere, and you need to add 1 to all your Byte Position values. Instead of actually keeping a big array of Byte Position values, you have a hierarchical array. The convention is…
Ted Nelson named them Enfilades. Guy Steele called them Monoid Cached Trees.
Re: Ask HN: What are some cool but obscure data structures you know about?
#396Equality graphs (e-graphs) for theorem proving and equality saturation and other equality-related things. They're awesome data structures that efficiently maintain a congruence relation over many expressions > At a high level, e-graphs extend union-find to compactly represent equivalence classes of expressions while maintaining a key invariant: the equivalence relation is closed under congruence. e.g. If I were to re…
Isnt that the building blocks of logic based programming languages like Prolog?
https://souffle-lang.github.io/docs.html http://www.philipzucker.com/souffle-egg4/ https://arxiv.org/pdf/2004.03082.pdf
Re: Ask HN: What are some cool but obscure data structures you know about?
#397Not a very deep CS-y one, but still one of my favourite data structures: Promise Maps. It only works in languages where promises/futures/tasks are a first-class citizen. Eg JavaScript. When caching the result of an expensive computation or a network call, don't actually cache the result, but cache the promise that awaits the result. Ie don't make a Map but a Map > This way, if a new, uncached key gets requested twice…
do you mean
setmetatable({}, {__index = function(tab, key)
local co = coroutine.running()
uv.read(fd, function(err, result)
tab[key] = result
return coroutine.resume(result)
end)
return coroutine.yield()
end })
This is an incomplete implementation, clearly: but I've never missed Promises with coroutines and libuv.Re: Ask HN: What are some cool but obscure data structures you know about?
#398I did an append only list of URL's with RAW files where each bit says something is true/false about the url at that offset. For example 26x26 RAW files asking if the page contains the letter combination "aa" or "ab" all the way up to "zy" and "zz". When one types a search query after 2 letters a file is pulled, at the 3rd letter we have a second 2 letter combination. Then do the AND operation. It is much like a bloom…
Re: Ask HN: What are some cool but obscure data structures you know about?
#399I've had a situation where I needed to stream blocks of data from a remote computer in soft-realtime, but still needed to share the same data with many different consumers. The code was simple, but effective (Go): import "sync" type Muxer[T any] struct { ret T mut sync.RWMutex } func (c *Muxer[T]) Multiplex(fn func() T) (ret T) { if c.mut.TryLock() { defer c.mut.Unlock() ret = fn() c.ret = ret } else { c.mut.RLock()…
I don't get it. It seems overengineered to me, but I can't formulate my reasoning well enough. Why isn't the original value that you're returning protected by a RWLock, and all goroutines will just need to acquire a read lock, instead of using a write lock for what is basically a getter function? Yeah, I don't get it.
1) If no other goroutine is currently executing the Multiplex() function, then Multiplex() acquires a write lock, executes the getter and caches the result.
2) If some other goroutine is currently executing the Multiplex() function, then Multiplex() waits to acquire the read lock, then reads the cached value.
So in case of a single goroutine executing the Multiplex() function in a loop, it will be nothing more than a wrapper. But when many goroutines are executing the same Multiplex() function concurrently, only one goroutine will actually execute the getter, and all others will just wait for the first one to finish, then take its cached result.
The point is that you don't have to think about who executes the getter and who copies the cached values, how often to execute the getter, how much time will getter take... You just call it like an ordinary function, and every goroutine will get the most recent value it can get.
Hopefully this clears things out. I'm willing to elaborate further if you have any more questions.
Re: Ask HN: What are some cool but obscure data structures you know about?
#400[Ordered] Minimal Perfect Hash Functions with O(N) construction time. They have been around for 30 years and I don't see them used in practice.