121–130 of 287 posts
Just to make sure I have this right, this has to do with the _amount_ of files in their repo and not the _size_ of the files? So projects like git annex and LFS would not help the speed of the git repos?
I'd assume this GVFS would work hand in hand with Git LFS for the use case of large files.
Earlier quoted context omitted.
Also don't think that this is a good idea. Git is a Distributed Version Control https://en.wikipedia.org/wiki/Distributed_version_control , the main benefit of which is "allows many software developers to work on a given project without requiring them to share a common network". Seems like with GVFS they are making DVC to be a CVS ( https://en.wikipedia.org/wiki/Concurrent_Versions_System ) again. What is the point?…
> I believe there are plenty bridges between CVS and git already implemented, which also allows you to checkout only part of the CVS tree. How many of them have you used? I've used a couple, to interact with large code bases on the rough order of 300GB. In my experience they don't work very well, because you have to be hygienic about the commands you run or some part of your Git state gets out of sync with some part…
My point was that with GVFS they are not really solving the problem they had - git status still takes 4-5 seconds, to be that is a lot.
How on Earth can anybody work like that?
I'd have thought you may as well ditch git at that point, since nobody's going to be using it as a tool, surely?
git commit -m 'Add today\'s work - night all!' && git push; shutdownI remember few years ago Git under Windows was very slow, is it still true?
> when you run “git checkout” and it takes up to 3 hours, or even a simple “git status” takes almost 10 minutes to run. That’s assuming you can get past the “git clone”, which takes 12+ hours. How on Earth can anybody work like that? I'd have thought you may as well ditch git at that point, since nobody's going to be using it as a tool, surely? git commit -m 'Add today\'s work - night all!' && git push; shutdown
Earlier quoted context omitted.
> Seems like with GVFS they are making DVC to be a CVS again. What is the point? It sounds like they answered that: > In a repo that is this large, no developer builds the entire source tree. Instead, they typically download the build outputs from the most recent official build, and only build a small portion of the sources related to the area they are modifying. Therefore, even though there are over 3 million files…
I'm curious to dig a bit further in, but from the blog post I get the impression that they are also still cloning the full commit history, just not the full file trees attached to the commits and definitely not the full worktree of HEAD, leaving those to be lazily fetched. If that is the case, that sounds like an interesting compromise on the git model and something verging on some of the speculative ideas I've seen…
My sysadmin: "we won't switch to git because it can't handle binary files and our code base is too big" Our whole codebase is 800MB.
Otherwise, I hope you replaced your sysadmin.
Earlier quoted context omitted.
Unfortunately that alone would not have allowed Git to be fast on such a huge repository. Normally (without tools as sparse-checkouts) Git would read all files for example on git status. Therefore IPFS would also download all files locally, making it a moot addition.
You would probably still need the changes they made to Git itself. But fundamentally IPFS is also a filesystem virtualization layer (so should be able to do everything their file system virtualization is doing - if it doesn't already), and inherently has lazy checkouts. The main added benefit is that if your friend on the LAN has also checked out the parts you need you can get them directly from them rather than some…
It's interesting how all the cool things seem to come from Microsoft these days. I still think we need something better than Git, though. It brought some very cool ideas and the inner workings are reasonably understandable, but the UI is atrociously complicated. And yes, dealing with large files is a very sore point. I'd love to see a second attempt at a distributed version control system. But I applaud MS's initiati…
Maybe something that has the data models of git but has a more consistent interface? Today on Git Merge there was a presentation about http://gitless.com/ For example one of the goals is to always allow you to switch branches. Stash and stash pop would happen automatically and it would even work if you're in the middle of a merge.
"Most developers access Piper through a system called Clients in the Cloud, or CitC, which consists of a cloud-based storage backend and a Linux-only FUSE13 file system. Developers see their workspaces as directories in the file system, including their changes overlaid on top of the full Piper repository. CitC supports code browsing and normal Unix tools with no need to clone or sync state locally. Developers can browse and edit files anywhere across the Piper repository, and only modified files are stored in their workspace. This structure means CitC workspaces typically consume only a small amount of storage (an average workspace has fewer than 10 files) while presenting a seamless view of the entire Piper codebase to the developer."
This is a very powerful model when dealing with large code bases, as it solves the issue of downloading all the code to each client. Kudos to Microsoft for open sourcing it, and under the MIT license no less.