Microsoft is moving away from source depo to git it seems. I think its fantastic that a company like Microsoft is adapting git for its big king and queen projects such as office and windows. Also open sourcing the underlying magic tells a lot about the new Microsoft. They're really moving away from not-invented here syndrome
Announcing GVFS: Git Virtual File System
221–230 of 287 posts
Re: Announcing GVFS: Git Virtual File System
#222Earlier quoted context omitted.
Our codebase (latest tree) is similar, but switching to git it's the total history size that is the problem. Our history is well over 25GB which git doesn't handle very gracefully.
History shouldn't be a problem, you can do a shallow checkout. But you will have to store the working tree at least on your workstation. This solves the next scaling problem of avoiding managing the whole working tree. (without requiring narrow clones which have significant downsides)
Yeah, the working tree works well to have locally, and that's what's done with svn currently.
The problem is that I also want a fast log/blame for any file back to the beginning of time - but I'm ok with that requiring devs connecting to the server containing the history (as with svn).
I also haven't found a way to make git work smoothly in shallow mode as the default, e.g can I make checkout of a branch always remember it must be shallow? Can I make log use remote history when necessary etc? I don't want to fight the tool all the time because I'm using a nonstandard approach.
Re: Announcing GVFS: Git Virtual File System
#223Earlier quoted context omitted.
That still only solves half the problem with large binary blobs. The other half is that almost all of the binary formats can't be merged and so you need a mechanism to lock them to prevent people from wiping out other people's changes. Unfortunately that runs pretty much counter the idea of DCVS.
When git-annex finds a conflict it can't solve, it gives back to you the two versions of the same file with the SHA of the original versions suffixed. This way you can look at both and resolve the conflict.
That's the exact point I'm making, fundamentally a large portion of these formats(PSD, JPG, PNG, MA, 3DS) can't be resolved.
If two people touch the same file at the same time someone is going to drop their work on the floor and that's a bad thing(tm). You need to synchronize their work with a locking mechanism that informs the user at the edit(not sync) point in the workflow.
Re: Announcing GVFS: Git Virtual File System
#224It's interesting how all the cool things seem to come from Microsoft these days. I still think we need something better than Git, though. It brought some very cool ideas and the inner workings are reasonably understandable, but the UI is atrociously complicated. And yes, dealing with large files is a very sore point. I'd love to see a second attempt at a distributed version control system. But I applaud MS's initiati…
What a change of cash-cow placement can do ...
Re: Announcing GVFS: Git Virtual File System
#225This is similar to what Google uses internally. See http://cacm.acm.org/magazines/2016/7/204032-why-google-store... : "Most developers access Piper through a system called Clients in the Cloud, or CitC, which consists of a cloud-based storage backend and a Linux-only FUSE13 file system. Developers see their workspaces as directories in the file system, including their changes overlaid on top of the full Piper reposit…
Google is far more advanced than this. They have one giant monorepo (Piper) that's backed by Bigtable (or at least it was, when I was there). Piper was mostly created in response to Perforce's inability to scale and be fault tolerant. Until Piper came along, they would have to periodically restart The Giant Perforce Server in Mountain View. Piper is 24x7x365 and doesn't need any restarts at all. But the key bit here…
I'm a Microsoft employee on the Git team. We do have a distributed, caching, incremental build system and a distributed test system. Right now, they're completely internal - like Google. They're called CloudBuild and CloudTest. They're very fast and no one thinks twice about kicking off a build.
Re: Announcing GVFS: Git Virtual File System
#226Does this article imply that Microsoft itself is also moving towards Git? Instead of e.g. using their own product like TFS?
I'm a Microsoft employee on the Git team.
Microsoft is moving to Git and we use Team Services / TFS as our Git server for all private repositories. GitHub is only used for OSS since that's where the OSS community is.
Re: Announcing GVFS: Git Virtual File System
#227Earlier quoted context omitted.
Google is far more advanced than this. They have one giant monorepo (Piper) that's backed by Bigtable (or at least it was, when I was there). Piper was mostly created in response to Perforce's inability to scale and be fault tolerant. Until Piper came along, they would have to periodically restart The Giant Perforce Server in Mountain View. Piper is 24x7x365 and doesn't need any restarts at all. But the key bit here…
I'm a Microsoft employee on the Git team. We do have a distributed, caching, incremental build system and a distributed test system. Right now, they're completely internal - like Google. They're called CloudBuild and CloudTest. They're very fast and no one thinks twice about kicking off a build.
Google employs a distributed, caching, incremental build system and a distributed test system across the majority of their code base. I worked in Windows Store and I can assure you that most people there don't use CloudBuild and CloudTest, let along know what they are. I would be confident in saying the majority of people at Microsoft are in that boat.
Re: Announcing GVFS: Git Virtual File System
#228Using git with large repos and large (binary blob) files has been a pain point for quite a while. There have been several attempts to solve the problem, none of which have really taken off. I think all the attempts have been (too) proprietary – without wide support, it doesn’t get adopted. I'll be watching this to see if Microsoft can break the logjam. By open sourcing the client and protocol, there is potential... O…
git-annex is, IMHO, by far the best solution. Pros of git-annex: - it is conceptually very simple: use symlinks instead of ad-hoc pointer files, virtual files system, etc. to represent symbolic pointer that point to the actual blob file; - you can add support for any backend storage you want. As long as it support basic CRUD operations, git-annex can have it as a remote; - you can quickly clone a huge repo by just cl…
Git Annex is only a partial solution, since it only solves issues with binary blobs. It doesn't solve problems with large repos.
Re: Announcing GVFS: Git Virtual File System
#229I think they could have picked a name that doesn't conflict with GNOME Virtual File System (GVfs).
I understand that sometimes two products have the same name, but they usually have very different scopes/usages.
In this case, they're both called GVFS AND three of the letters have the same meaning, and they both do relatively similar things.
Even the tooling, and the output of `mount` is bound to be incredible confusing.
Re: Announcing GVFS: Git Virtual File System
#230I think they could have picked a name that doesn't conflict with GNOME Virtual File System (GVfs).
When they picked the "Windows" product name, they could have picked a name that didn't conflict with the use of windows. Picking on an obscure file system doesn't even register in comparison. X Windows NeWS - https://en.m.wikipedia.org/wiki/NeWS Remember the mess on usenet? comp.windows.x.motif comp.windows.new - not news about Microsoft Windows
In this particular case, the name is THE SAME (GVFS / GVFS). And they're both virtual filesystems, so there's lots of rooms for confusion.
I can image people at a forum:
"Hey, GVFS isn't working for me. It crashes with error -504" when I try to mount /nfs/company_data".
Try guessing which GVFS that is.