Live data from Hacker News

Speed Up Git Pull

interrobeng.com

11–20 of 50 posts

Re: Speed Up Git Pull

#12

Meta question: assuming that lots of GH users do this (nice trick), would GH have loads of dormant SSH connections? At scale, this could be a huge number. Would this be an issue?

No. I have this enabled and Github closes my connections after a very short period. I use it primarily for SSHing into my cluster of EC2 instances (which does massively speed things up).

Re: Speed Up Git Pull

#13
...and keep your development copy of the project on ramdisk. Have a script that you launch as you start work that makes the ramsdisk and puts files from persistent location there using rsync, then periodically launches rsync to copy changes you make on our ramdisk back to persistent location.

I used this setup for almost a year. It saved me lot of time and sanity.

Re: Speed Up Git Pull

#14

Meta question: assuming that lots of GH users do this (nice trick), would GH have loads of dormant SSH connections? At scale, this could be a huge number. Would this be an issue?

No. I have this enabled and Github closes my connections after a very short period. I use it primarily for SSHing into my cluster of EC2 instances (which does massively speed things up).

Same here. I do a

    (ssh -fqN -o "StrictHostKeyChecking no" git@bitbucket.org >&/dev/null &)
with bitbucket and github in my zshrc and bitbucket stays open but github is getting closed at some point. It used to stay open however.

Re: Speed Up Git Pull

#15

If you're a heavy SSH user, using multiplexing in this manner can have negative consequences [1]. Downsides include having all your multiplexed connections exiting if the master exits! [1] http://www.anchor.com.au/blog/2010/02/ssh-controlmaster-the-...

More recent SSH clients can use "ControlPersist" to establish the master connection in the background, so the first session doesn't control the lifetime of the connection. This makes using ControlMaster workable. I usually set ControlPersist to 30 seconds, which may not be long enough for people hoping to get performance improvements from GitHub, . Setting it to too large a value increases the risk that you'll have s…

> This makes using ControlMaster workable.

Best part of reading this article. I had turned off connection sharing because of this.

So what, in more details, are the downsides to ControlPersist?

Re: Speed Up Git Pull

#16

...and keep your development copy of the project on ramdisk. Have a script that you launch as you start work that makes the ramsdisk and puts files from persistent location there using rsync, then periodically launches rsync to copy changes you make on our ramdisk back to persistent location. I used this setup for almost a year. It saved me lot of time and sanity.

https://github.com/graysky2/anything-sync-daemon

Re: Speed Up Git Pull

#17
post #5

A 50x speedup is pretty cool in its own right. Kudos. However, I wonder if this isn't treating a symptom versus a root cause. Is saving that 5s round-trip so common in your workflow that you needed to optimize it, and would it be more productive to refactor the app so you and collaborators are working on different files? Also, this has the entirely valuable guidance that pushing to a local server is much faster than…

http://xkcd.com/1205/

If you pull 10 times a day (conservative for me) then you’ll save ~5 hours over a year.

Re: Speed Up Git Pull

#18

...and keep your development copy of the project on ramdisk. Have a script that you launch as you start work that makes the ramsdisk and puts files from persistent location there using rsync, then periodically launches rsync to copy changes you make on our ramdisk back to persistent location. I used this setup for almost a year. It saved me lot of time and sanity.

https://github.com/graysky2/anything-sync-daemon

Cool. I still prefer my 5 line bash script manually started in separate console.

Re: Speed Up Git Pull

#19
post #15

Earlier quoted context omitted.

More recent SSH clients can use "ControlPersist" to establish the master connection in the background, so the first session doesn't control the lifetime of the connection. This makes using ControlMaster workable. I usually set ControlPersist to 30 seconds, which may not be long enough for people hoping to get performance improvements from GitHub, . Setting it to too large a value increases the risk that you'll have s…

> This makes using ControlMaster workable. Best part of reading this article. I had turned off connection sharing because of this. So what, in more details, are the downsides to ControlPersist?

One that I frequently run into is that if you use SSH tunneling (like -L), you have to specify it the first time you ssh to that machine (i.e. when the ControlMaster is connected) and can't change it later. Using -L on later ssh's to the same machine silently fail, which can be infuriating if you don't realise it's happening. The best you can do at that point is to kill the ControlMaster ssh (disconnecting you across all your sessions), and then reconnecting with the right -L.
Post reply on HN