Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Announcing GVFS: Git Virtual File System

blogs.msdn.microsoft.com

Post reply on HN
Print view View original

251–260 of 287 posts

Re: Announcing GVFS: Git Virtual File System

#251

Post by foobiekr »

tcbawo wrote: post #248
wyldfire wrote: post #50

I'm immediately reminded of MVFS and clearcase. Lots of companies still use clearcase, but IMO it's not the best tool for the job. git is superior in most dimensions. From what this article says, it's not quite the same as clearcase but there's certainly some hints of similarities. The biggest PITA with clearcase was keeping their lousy MVFS kernel module in sync with ever-advancing linux distros. I really liked Clea…

I used Clearcase (on Solaris) in 1999 and was not a fan. It slowed our build times by at least 10x. I'm sure it was probably set up wrong, but this was a Fortune 100 company with lots of dedicated resources.

Clearcase performance for many builds was specifically impacted by the very poor performance of stat(). You could make very real improvements on build times by reducing the number of calls to stat(). It was sort of amazing.

Clearcase also suffered, at least in my experience, from a clumsy and ugly merging process and deeply unintuitive command set which meant everyone who "used clearcase" actually tended to use some terrible homegrown wrapper scripts.

Still, considering it was the last remaining vestige of the Apollo Domain OS, not bad.

Re: Announcing GVFS: Git Virtual File System

#252

Post by klodolph »

jauco wrote: post #119
klodolph wrote: post #91

Earlier quoted context omitted.

> I believe there are plenty bridges between CVS and git already implemented, which also allows you to checkout only part of the CVS tree. How many of them have you used? I've used a couple, to interact with large code bases on the rough order of 300GB. In my experience they don't work very well, because you have to be hygienic about the commands you run or some part of your Git state gets out of sync with some part…

A bit of a topic highjack, but I'm always curious how "many people committing very often on the same branch" works in practice. I'd expect there to be livelock like scenario's. Do you need to do anything special? Or is this just a non-issue? Doy you push to master or do you use some sort of pull request gui (like github or phabricator)

Not sure what livelock you're talking about. If you have a bunch of people pushing to Git master, sure, you'll get conflicts and have to rebase (if we're talking about TBD). But the conflicts are always caused by someone successfully pushing to master, so some progress will always be made.

But I was always just using Git as an interface to something else, usually Perforce or something similar. When pushing with these tools, you'd only get conflicts if other people changed the same files. Git was just used to create a bunch of intermediate commits and failed experiments on your workstation, which is something that it really excels at.

The only real problem is when the file you're changing is modified by many people on different teams, which often means that it's used for operations, and when that becomes a bottleneck it'll get refactored into multiple files or the data will be moved out of source control.

Re: Announcing GVFS: Git Virtual File System

#253

Post by codesnik »

general_ai wrote: post #215
kevincox wrote: post #204

Earlier quoted context omitted.

I think this would be better the piper+citc. While the virtual filesystem aspect is nice the perforce model is far inferior to git's. (IMO) Of course Google has tools on top of it. But it's not fair to compare a VCS and interface to it to a complete development infrastructure. Hell, with the content addressing of git it would even make it easy to build something similar.

> perforce model is far inferior to git's That's just, like, your opinion, man. There are other bits of infra that integrate with it quite nicely, and would integrate with something like Git quite poorly. One of those things is their code review system. The closest thing I could find to it outside Google is Gerrit, but it's a tremendous pain to set up and use, and it's but a pale shadow of Google's internal tool (Cri…

I wonder how do you know what parts you CAN disclose?

Re: Announcing GVFS: Git Virtual File System

#254

Post by general_ai »

haberman wrote: post #241
mooman219 wrote: post #227

Earlier quoted context omitted.

Google employs a distributed, caching, incremental build system and a distributed test system across the majority of their code base. I worked in Windows Store and I can assure you that most people there don't use CloudBuild and CloudTest, let along know what they are. I would be confident in saying the majority of people at Microsoft are in that boat.

Just to give others an idea of what this is like: I work on the Protocol Buffers team at Google. Almost all software at Google depends on my team's code. If I have a pending change, I can test it against the whole codebase before submitting (this is a "global presubmit"). If something breaks in global presubmit, I can build and run any target in the codebase with my change in a single command, and this will take O(10…

Dunno how it is now, but years ago it'd take them a _few weeks_ to just propagate commits into the stable branch through a series of elaborate branch integrations, so yeah, you couldn't change something and test it on a whim. Plus build of just windows alone would take overnight, and rebuilding everything to test a Windows change was not logistically, politically, or technically possible.

Re: Announcing GVFS: Git Virtual File System

#255

Post by klodolph »

outcoldman wrote: post #122
klodolph wrote: post #91

Earlier quoted context omitted.

> I believe there are plenty bridges between CVS and git already implemented, which also allows you to checkout only part of the CVS tree. How many of them have you used? I've used a couple, to interact with large code bases on the rough order of 300GB. In my experience they don't work very well, because you have to be hygienic about the commands you run or some part of your Git state gets out of sync with some part…

I used one for TFS, when I worked in Microsoft, and git p4 when I worked at Splunk. Certainly enjoying that we are 100% git now. My point was that with GVFS they are not really solving the problem they had - git status still takes 4-5 seconds, to be that is a lot.

So you're saying that GVFS isn't a good idea because it's not good enough for working on the Windows repository?

Well, yeah. It's pre-production, and let Microsoft worry about their own problems anyway. But it sounds like GVFS will be killer for people who have large repos that aren't as large as the Windows repo. Even if 4-5s for the 270GB/3.5M file repo is too long, 400-500ms for the 27GB repo is fantastic.

At some point you ask yourself, "Would I split this repo if my tools could handle the combined repo just fine?" If the answer is no, then you're going to be happy that the tools are getting better at handling big repos. Microsoft's choice to exploit the filesystem-as-API and funnel all filesystem interaction through the VCS is a smart choice and there are a ton of opportunities for optimization that don't exist when you're just writing to a plain filesystem.

Re: Announcing GVFS: Git Virtual File System

#256

Post by general_ai »

codesnik wrote: post #253
general_ai wrote: post #215

Earlier quoted context omitted.

> perforce model is far inferior to git's That's just, like, your opinion, man. There are other bits of infra that integrate with it quite nicely, and would integrate with something like Git quite poorly. One of those things is their code review system. The closest thing I could find to it outside Google is Gerrit, but it's a tremendous pain to set up and use, and it's but a pale shadow of Google's internal tool (Cri…

I wonder how do you know what parts you CAN disclose?

Whatever is already public. You can watch a video about Piper and some other systems on YouTube, and read about other things from Google's own blogs, papers, etc.

Re: Announcing GVFS: Git Virtual File System

#258

Post by patio11 »

oz wrote: post #181
ska wrote: post #146

Earlier quoted context omitted.

It would be naive to assume large companies always make the right decisions, which is why few people do that. However, when you see that a large company is doing something in a way that you thing is silly or strange, the logical thing to ask first is "what do they understand that I don't understand?". It won't always be the case, of course, but most of the time it will turn out you were missing something. Assuming of…

>the logical thing to ask first is "what do they understand that I don't understand?". There's even a name for this: "Going from D to C - from disparagement to curiosity". I think I first heard it from @patio11.

Doesn't sound like the way I would phrase the thought, so I checked: http://www.kalzumeus.com/2012/09/17/ramit-sethi-and-patrick-... <-- Ramit said it here, although I suspect it might be older.

Re: Announcing GVFS: Git Virtual File System

#259

Post by pjmlp »

contextfree wrote: post #232
pjmlp wrote: post #43

Quite nice use of C# and C++/CX for a virtual system implementation.

looks like C++/CLI (C++/CX reused its syntax and maybe parsing code, but they're still distinct)

Yes, hence why Microsoft had lots of trouble to convince developers that don't read documentation, that they are distinct and C++/CX gets compiled to just pure native code, as they were spreading misinformation about it.

In any case, when C++/WinRT gets feature parity, I imagine it will eventually be deprecated, depending which one gets more developer love.

Re: Announcing GVFS: Git Virtual File System

#260

Post by sdesol »

sdesol wrote: post #233
krupan wrote: post #205

Earlier quoted context omitted.

Holy cow, it sounds like they reinvented Clearcase!

If they get this right, this can be MASSIVE for Microsoft in Enterprise. ClearCase was the reason why IBM was able to charge $1000+ per developer license fees. ClearCase did what a lot of Enterprise companies needed at the time, and most importantly, it created hooks, that were mostly too difficult to remove. Once you create deep integration with ClearCase, you are very much committed to using it long term.

I was planning on responding to the person that had replied, but their comment has since been removed, and because I can't edit my post, I'll add some insight, as to why I used ClearCase as an example.

For those who have never worked with/administered ClearCase before, you may not fully appreciate how insanely complex it is. In order to use it, you first have to apply kernel patches from IBM, which shows how committed you had to be. ClearCase provided something that others couldn't, which is why it was so expensive. With Git, everything has changed.

Since nobody owns Git and its implementation, the differentiating factor right now is mostly superficial. There really isn't anything, other than hosting repos at a massive scale, that can't be easily duplicated. Git hosting, in my opinion, is now officially a commodity product. And what differentiates GitHub, GitLab, Bitbucket, etc. is mostly marketing.

With GVFS, things could change. This could be the first step, in Microsoft owning the hard part, that can't be easily duplicated by others. I really don't know what is on their roadmap, but views in ClearCase were pretty powerful and if they are looking at the level of integration, then it could be tough for GitLab, GitHub and others to follow.

Post reply on HN