I know there are classics that get posted every time this question comes around, so bias them towards more recent ones :)
Ask HN: What's the best paper you've read in 2020?
1–10 of 197 posts
Re: Ask HN: What's the best paper you've read in 2020?
#2For me, it was "Erasure Coding in Windows Azure Storage" from Microsoft Research (2016) [0]
The idea that you can achieve the same practical effect of a 3x replication factor in a distributed system, but only increasing the cost of data storage by 1.6x, by leveraging some clever information theory tricks is mind bending to me.
If you're operating a large Ceph cluster, or you're Google/Amazon/Microsoft and you're running GCS/S3/ABS, if you needed 50PB HDDs before, you only need 27PB now (if implementing this).
The cost savings, and environmental impact reduction that this allows for are truly enormous, I'm surprised how little attention this paper has gotten in the wild.
[0] https://www.microsoft.com/en-us/research/wp-content/uploads/...
Re: Ask HN: What's the best paper you've read in 2020?
#3Fresh today: Chromosome-scale, haplotype-resolved assembly of human genomes ;)
Re: Ask HN: What's the best paper you've read in 2020?
#4Meaningful Availability, Hauer et al.: https://www.usenix.org/system/files/nsdi20spring_hauer_prepu...
A good incremental improvement in service level indicator measurements for large-scale cloud services.
Obligatory The Morning Paper post: https://blog.acolyer.org/2020/02/26/meaningful-availability/
Re: Ask HN: What's the best paper you've read in 2020?
#5A Conceptual Introduction to Hamiltonian Monte Carlo (2017) https://arxiv.org/abs/1701.02434
Re: Ask HN: What's the best paper you've read in 2020?
#6Re: Ask HN: What's the best paper you've read in 2020?
#7For me, it was "Erasure Coding in Windows Azure Storage" from Microsoft Research (2016) [0] The idea that you can achieve the same practical effect of a 3x replication factor in a distributed system, but only increasing the cost of data storage by 1.6x, by leveraging some clever information theory tricks is mind bending to me. If you're operating a large Ceph cluster, or you're Google/Amazon/Microsoft and you're runn…
The Google File System (GFS) paper from 2003 mentions erasure codes. Which isn't to say they did it then, but rather that the technique of using erasure coding was known back then.
(And surely before GFS too, I just picked it as an example of a large data storage system that used replication and a direct predecessor to the systems you mentioned.)
https://static.googleusercontent.com/media/research.google.c...
Re: Ask HN: What's the best paper you've read in 2020?
#8For me, it was "Erasure Coding in Windows Azure Storage" from Microsoft Research (2016) [0] The idea that you can achieve the same practical effect of a 3x replication factor in a distributed system, but only increasing the cost of data storage by 1.6x, by leveraging some clever information theory tricks is mind bending to me. If you're operating a large Ceph cluster, or you're Google/Amazon/Microsoft and you're runn…
I think for the major players you mentioned the 2016 paper was retrospective. Everyone was already doing it. Even mid-tier players like Dropbox Magic Pocket were using erasure coding by 2016, and their scheme was mostly written by ex-Google engineers influenced by Colossus.
Re: Ask HN: What's the best paper you've read in 2020?
#9I'm a former biophysicist bumbling my way into distributed systems; was learning rust and bumped into Frank McSherry's blog posts.
Thought the Naiad project is really cool!
https://cs.stanford.edu/~matei/courses/2015/6.S897/readings/...
Re: Ask HN: What's the best paper you've read in 2020?
#10Snap: a Microkernel Approach to Host Networking