Viewing profile — bassp

> Some problems are straightforward to specify. A file system is a good example. I’ve got to disagree with this - if only specifying a file system were easy! From the horse’s mouth…

Yes! There’s a canonical algorithm called the “Blelloch scan” for prefix sum (aka prefix scan, because you can generalize “sum” to “any binary associative function”) that’s very gp…

Sure, but that’s not what the person responding to my original comment was suggesting :). They suggested that you serialize entire data structures (bloom filters, lists, sets, etc……

They can (and that's probably the right choice for a lot of use cases, especially for small data structures and infrequently updated ones), but serializing and storing them in a da…

I agree with the author 100% (the TanTan anecdote is great, super clever work!), but.... sometimes you do need Redis, because Redis is the only production-ready "data structure ser…

I use Kafka for a low-message-volume use case because it lets my downstream consumers replay messages… but yeah in most cases, it’s over kill

Your network programming guide really saved my bacon back when I was taking a networking class, I appreciate all your hard work!

That's really clever! Kudos. I'm gonna set aside some time this week to dive into the implementation

IME, yes. Here a couple problems I've run into using GQL for backend to backend communication: * Auth. Good GQL APIs think carefully about permission management on a per-field basi…

Sorry if I was sloppy with my wording, instruction issuance is what I meant :) I thought that warps weren't issued instructions unless they were ready to execute (ie had all the da…

You can request up to 1024-2048 threads per block depending on the gpu; each SM can execute between 32 and 128 threads at a time! So you can have a lot more threads assigned to an …

I was taught that you want, usually, more threads per block than each SM can execute, because SMs context switch between threads (fancy hardware multi threading!) on memory read st…

Not the OP, but Hillel Wayne’s course/tutorial ( https://www.learntla.com/ ) is fantastic. It’s focused on building practical skills, and helped me build enough competence to write…

It’s not all or nothing! I work on a very “product-y” back end that isn’t fully specified, but I have formally specified parts of it. For instance, I property-based-tested a partic…

I’m surprised that this piece mentioned Microsoft, but didn’t touch on Microsoft’s solution to this problem: project silica ( https://www.microsoft.com/en-us/research/project/proje…

That’s a good point! I think I overstated the case a little, I definitely don’t think automated reasoning is some “secret reliability sauce” that nobody else can replicate; it does…

Minimally, the two examples I cited: Shardstore and Shuttle. The former is a (lightweight) formally verified key value store used by S3, and the latter is a model checker for concu…

Only tangentially related to the article, but I’ve never understood how R2 offers 11 9s of durability. I trust that S3 offers 11 9s because Amazon has shown, publicly, that they ca…

Gotcha! Thanks for the answer; so the tl;dr is, if I’m understanding: “All fsync-ed writes will eventually make it to S3, but fsync successfully returning only guarantees that writ…

For sure! Upon reflection, maybe I’m less curious about crash consistency (corruption or whatever) per-se, and more about what kinds of durability guarantees I can expect in the pr…

This feels, intuitively, like it would be very hard to make crash consistent (given the durable caching layer in between the client and S3). How are you approaching that?