Live data from Hacker News

GitHub major issues - repos have "lost" commits and site is erroring

support.github.com

51–60 of 91 posts

Re: GitHub major issues - repos have "lost" commits and site is erroring

#51
post #41

Earlier quoted context omitted.

It's amazing to me that you feel more confident in your own abilities to keep a server running than GitHub... you must be a really great sysadmin! The GitHub folks are pretty hardcore! I, for one, am a positively mediocre sysadmin. And the amount of time I have to spend waiting for GitHub to fix bugs is almost certainly far, far, FAR less than the amount of time I'd spend scrambling (but not waiting!) to fix my own.…

It's not that hard to install a source control server, whether it's git, svn or mercurial.

Yes, I can confirm that it's surprisingly easy to run git with gitosis over ssh on your own server for a few dollars. Takes a couple of minutes to setup. And of course you can make it public and you can share it with other people with keys.

Re: GitHub major issues - repos have "lost" commits and site is erroring

#52
Suggestion for creating a startup:

For private repositories gitolite is really awesome, but I think there should be a company about it. What I mean is that github or some other company should offer hosted gitolite instances, in the following form:

You pay something like price_of_small_linode_instace+price_of_service, to get:

A virtual machine with gitolite up and running, with a good web interface and online documentation. I think that the web interface part is the piece this company should not opensource. With this web interface you can create repositories and users, and modify the gitolite settings in general.

A backup service, in some other different VM and network, so that if something goes wrong with your gitolite VM you can ask a free (and automated) restore, possibly with hour granularity for the latest 24 hours, and less granularity for bigger periods and so forth.

Note that this gives the user virtually infinite repositories... if they don't upload movies to their instance, as there is a lot of space even in a small linode instance for git repositories.

This way this is a "pure product" as there is no need to repair or install manually virtual machines, and the company can focus into evolving the offering, developing a better version of gitolite, of the web interface, and so forth.

p.s. I mentioned linode but can well be ec2 or whatever is better suited for this stuff.

Re: GitHub major issues - repos have "lost" commits and site is erroring

#53
post #43

Earlier quoted context omitted.

It's amazing to me that you feel more confident in your own abilities to keep a server running than GitHub... you must be a really great sysadmin! The GitHub folks are pretty hardcore! I, for one, am a positively mediocre sysadmin. And the amount of time I have to spend waiting for GitHub to fix bugs is almost certainly far, far, FAR less than the amount of time I'd spend scrambling (but not waiting!) to fix my own.…

The beauty os a dvcs is that I have a full copy on my machine as well. But seriously, if you use git over ssh there is no sys admin work other than running apt-get upgrade

Maybe for a few months. When you run a server for 2 or 3 years the distro goes out of support, software upgrades are getting behind, before you know it you are compiling patches from source, ...

I've done this several times over the last 5 years and am now finally moving everything to specialized hosting services. Just hosting a web site, photo repo, mail and svn on a machine (vps) is enough to make it a serious hassle Moving all those to specialized services costs the same, is less work and more reliable.

Re: GitHub major issues - repos have "lost" commits and site is erroring

#54

Is it just me or has guthub's performance and reliability been getting progressively worse over the past month or two? Even putting this issue aside, and the outages, pushing to a private repo takes _minutes_ now, and file lists in the web interface often don't load without a page refresh.

I haven't had any trouble, but I have seen more interruptions happening recently. I relate it to growth, because as you grow, these things sometimes happen more often; "growing pains" if you will. Overall I am happy with Github, and their downtime has not cost me any time or money.

Re: GitHub major issues - repos have "lost" commits and site is erroring

#55

Earlier quoted context omitted.

Git has full cryptographic history (and data) authentication, so any changes to the history would be easily detectable. And you can sign known-good commits, so that even if you've never pulled before, you can still verify the part of the history that's been signed. SVN and CVS are missing this key feature, which is why the sf.net hacking is scary.

I'm a Git n00b, so thanks for pointing that out! No wonder git has killed off SVN

It hasn't yet, but it is making great headway, and I for one can't wait!

Re: GitHub major issues - repos have "lost" commits and site is erroring

#56
post #43

Earlier quoted context omitted.

It's amazing to me that you feel more confident in your own abilities to keep a server running than GitHub... you must be a really great sysadmin! The GitHub folks are pretty hardcore! I, for one, am a positively mediocre sysadmin. And the amount of time I have to spend waiting for GitHub to fix bugs is almost certainly far, far, FAR less than the amount of time I'd spend scrambling (but not waiting!) to fix my own.…

The beauty os a dvcs is that I have a full copy on my machine as well. But seriously, if you use git over ssh there is no sys admin work other than running apt-get upgrade

There's always sysadmin work.

1) You need to keep current on your security patches.

2) You need to upgrade your OS when it reaches end-of-life.

3) You need to make backups, and more importantly, verify that those backups will actually restore. (For git, this is most critical if you have a large team and dozens of repos.)

You can ignore a server for years, but eventually you'll get compromised or loose a hard drive. (If you use RAID, you'll eventually lose or an entire RAID array. Not fun at all, nor cheap.)

Re: GitHub major issues - repos have "lost" commits and site is erroring

#57

Is it just me or has guthub's performance and reliability been getting progressively worse over the past month or two? Even putting this issue aside, and the outages, pushing to a private repo takes _minutes_ now, and file lists in the web interface often don't load without a page refresh.

I have two repos living in "Repository temporarily unavailable" status for several days now. I understand that shit happens, but what's really frustration is inquiry about what can or will be done about it and getting no response. Do I wait? Do I just move stuff to my own server? Go back to Gitorious? It's not that I can't keep working, but the main reason I use Github is to make my OSS projects available. For other…

Responding is key.

I believe the easiest way to build a successful business (software and/or service business anyway) is to respond promptly. Even if you don't have an answer yet. If you do, your customers will stick with you through almost anything.

Re: GitHub major issues - repos have "lost" commits and site is erroring

#58

Just posted to the support page. "Sorry guys, there was a Javascript error displaying an error when there was none. We've deployed a fix for this and everything should be good to go."

For what it's worth, I was pulling & pushing just fine while the error was displayed on my repositories, but other people reported commits lost or mangled. Maybe there was something more to the Javascript display error?

Re: GitHub major issues - repos have "lost" commits and site is erroring

#59
post #50

Earlier quoted context omitted.

It's amazing to me that you feel more confident in your own abilities to keep a server running than GitHub... you must be a really great sysadmin! The GitHub folks are pretty hardcore! I, for one, am a positively mediocre sysadmin. And the amount of time I have to spend waiting for GitHub to fix bugs is almost certainly far, far, FAR less than the amount of time I'd spend scrambling (but not waiting!) to fix my own.…

> The GitHub folks are pretty hardcore! What gives you that idea? I run several cronjobs that deal with their service, and it's down a lot compared to other similar services. My jobs deal with that gracefully, but if I wanted to I could easily set up a more reliable service than GitHub if my only goal was to host *.git repositories, not have any of the other nice things they provide.

They've struggled and dealt with a number of hard engineering problems, which they've blogged extensively about. In particular, their architectural (both hardware and software) changes since leaving Engine Yard demonstrate more than a simple "working knowledge" of how to build large-scale systems.

Re: GitHub major issues - repos have "lost" commits and site is erroring

#60
post #47

Earlier quoted context omitted.

This is a straw man argument. The question isn't whether Git could be more intuitive/user-friendly (Hint: It should be, in fact I bet my company on it [see my profile]), or whether it is more secure/cheap to host your own repo. If you have a million people `pull`ing from your repo, of course you should have be hosting your own public access point. But, in 80% of cases, people can't be bothered to figure out how to se…

This comment is totally unrelated and is itself a strawman. Yes, it is easier to use GitHub: I will not argue that fact. However, using GitHub will cause people to be pulling from GitHub, and GitHub may go down. This is a tradeoff, and is one people use a lot: you use a shared platform and give up control of the URL to get easier outsourced hosting. But to argue that git's decentralization solves that problem is disi…

Web services go down. It happens. You make the choice to use them anyway.

In this case, Github is very responsive about outages and clearly strives to eliminate or reduce them as much as possible.

Post reply on HN