GitHub availability this week

Post by dustym » Sat, Sep 15, 2012, 10:20 PM UTC

donavanm wrote: post #50

dustym wrote: post #33

Earlier quoted context omitted.

We use S3 behind 1 second max-age cloudfront to serve The Verge liveblog. It's been nothing but rock solid. We essentially create a static site and push up JSON blobs. See here: http://product.voxmedia.com/post/25113965826/introducing-syl...

Could you say more about using both the Cache-Control and a query string of epoch time? In particular the query string has me puzzled. On it's face it seems to decrease your cache hit ratio, with no/little benefit. Im assuming the epoch time is the clients local time. The clock skew across the client population increases the number of cache keys active at any one time. The incrementing query string also forces a new…

Great point. I don't speak for the guys that made the decision to append the timestamp to the query, but I assume our concern is in intermediate network caches that don't honor low TTLs. Though I don't know how founded that is, we won't ever have to deal with the issue if we take control of it with the url string.

It'd be interesting to see how wide the key space is due to clock skew. I suppose we could specify some number and consider it a global counter that is incremented every second, then when someone comes in for the first time they can by synced in with the global incrementing counter. That counter is used to ensure a fresh cloudfront hit.

I think at end of the day, these issues haven't been a huge concern for a one month emergency project, but they are good points.

Post by gbog » Sun, Sep 16, 2012, 3:37 AM UTC

holman wrote: post #60

gbog wrote: post #57

Genuine question: github is built upon git, which is a rock solid system for storing dataand in these reports we read that github relies a lot on MySQL, so... Did the github guys ponder using git as their data store? Just an example, in git one can add comments on commits, would it be possible to use it for the github comment function? Or maybe it is?

Generally, Git will be way too slow for that. Git is typically our bottleneck, since you're dealing with so much overhead and disk access to perform functions. Databases are best for, well, performing relational queries. In the case of commenting on a commit, if you store them only in the repository it becomes non-trivial to ask "show me all of the comments by this user" unless you have an intermediary cache layer (i…

Thanks for answering. Tell me if I'm wrong but MySQL would be behind a caching layer anyway, so the choice would be between cached git or cached git + mysql.

In git, logging commits on a file from an author is also a kind of join, and it is surprisingly fast, so using git as a data store is a weird idea that I cannot take out of my head.

Re: GitHub availability this week

Re: GitHub availability this week