41–50 of 128 posts
It doesn't matter how many technical blog posts they put out, I will still never host my code with Elon Musk.
There’s a trend of doing impressive things by pushing many of the hard problems into S3 and assuming S3 “just works”, then not bothering to explain how S3 works. I guess we do the same thing all the time with other miraculous solutions to hard problems, like assuming that superscalar out-of-order processors “just work”, but in cases like this it makes for an unsatisfying explanation. Especially because S3 is a propri…
It’s hard to overstate the reputation of the author of this post. Everything good about GitHub’s internal systems seemed to have his name all over it (I realize that today this statement hits different than a few years ago). Our times at GH didn’t overlap much but hearing the fact that he’s working at cursor increases my estimation of their engineering org by leaps and bounds.
[flagged]
It doesn't matter how many technical blog posts they put out, I will still never host my code with Elon Musk.
Same. Sometimes you have to live with inconvenience to keep a moral stand. Those down-voting don't understand why any sacrifice of convenience or tech is worth it.
> Those down-voting don't understand why any sacrifice of convenience or tech is worth it.
FWIW my comment was actually highly upvoted but it looks like X employees and/or Musk fans showed up and downvoted all criticism in this thread.
It’s hard to overstate the reputation of the author of this post. Everything good about GitHub’s internal systems seemed to have his name all over it (I realize that today this statement hits different than a few years ago). Our times at GH didn’t overlap much but hearing the fact that he’s working at cursor increases my estimation of their engineering org by leaps and bounds.
This solution with WAL and compact resembled the database internals as taught by Martin Kleppmann.
I asked Claude about this observation, and learned that the author (Vicent Martí) wasn't only core Github developer but also worked on Vitess from PlanetScale.
So no surprise these two contexts resulted in the design of Continuity as it is.
https://vitess.io/ - Vitess is a database clustering system for horizontal scaling of MySQL.
WAL on S3 is the new normal.
Took me some thinking to figure out where this design is not optimal. The first step is to separate the problems. The content addressable object storage and storing the refs. libgit2 has two backends for those already. Then use appropriate distributed dbs for the two backends. With large scale something like Cassandra for the objects (AP) and FoundationDB (CP) for the refs (or the equivalent Amazon services). For per…
Part of the problem is that the API you're supposed to provide externally depends on git packs which you might have to reconstruct on the fly from such a system.
I feel like the article is a bit light on the design of the WAL and maintenance operations, but maybe it's me.
Anyway, I'd love to have a friendly chat with the author.
There’s a trend of doing impressive things by pushing many of the hard problems into S3 and assuming S3 “just works”, then not bothering to explain how S3 works. I guess we do the same thing all the time with other miraculous solutions to hard problems, like assuming that superscalar out-of-order processors “just work”, but in cases like this it makes for an unsatisfying explanation. Especially because S3 is a propri…
at this point, s3 just means object storage and doesnt mean it actually has to be hosted on aws. theres plenty of other companies that provide s3 compatible storage apis.