Some recent academic work suggests implementing caches directly in network switches. Tofino switches are programmable enough that academics can implement this today. OrbitCache is one example, described in this paper: https://www.usenix.org/system/files/nsdi25-kim.pdf It should solve the thundering herd problem, because the switch would "know" what outstanding cache misses it has pending, and the switch would park su…
> This has an advantage compared to a multi-threaded CPU-based cache, because it avoids performance overheads associated with multiple threads having to synchronize with each other to realize they are about to start a stampede. The switch presumably also has multiple cores which still need to do this work, no? Or is the claim that moving this synchronization to the router behind a network hop saves CPU cycles on the…
I wrote a brief description of RMT here: https://danglingpointers.substack.com/p/scaling-ip-lookup-to...