EdenSCM – A cross-platform, scalable source control management system
1–10 of 33 posts
Re: EdenSCM – A cross-platform, scalable source control management system
#2Interesting. This might be a step in the direction of being able to store big files in repositories without hassle.
However, perhaps OS level support would be preferred. Imagine you have a type of symbolic link that is not just followed, but executed when you access it. That would be really powerful and would allow this kind of optimization. And you wouldn't even need to install or run anything.
Re: EdenSCM – A cross-platform, scalable source control management system
#3Interesting. This might be a step in the direction of being able to store big files in repositories without hassle. However, perhaps OS level support would be preferred. Imagine you have a type of symbolic link that is not just followed, but executed when you access it. That would be really powerful and would allow this kind of optimization. And you wouldn't even need to install or run anything.
Sounds a lot like FUSE?
Re: EdenSCM – A cross-platform, scalable source control management system
#4Facebook rewrote Mercurial, while Microsoft has essentially expanded git with their own virtual file system VFSforGit [0] and a bunch of performance improvements.
Re: EdenSCM – A cross-platform, scalable source control management system
#5There is more tooling needed in general - just one recursive grep will populate the entire EdenFS.
I suppose FB has better tools. But I won’t touch this until the ecosystem is sufficient (and also, because git and hg are perfectly sufficient for the monorepo I oversee)
Re: EdenSCM – A cross-platform, scalable source control management system
#6Does this work together with a build cache? My dream setup for dealing with building huge code bases is a file system integrated with the version control system to only download files when they are accessed (which sounds like what this does) but also employing a build cache and module system, so it doesn't even need to download and compile any module that has not been touched, it just downloads the result from the build cache instead.
Re: EdenSCM – A cross-platform, scalable source control management system
#7Facebook rewrote Mercurial, while Microsoft has essentially expanded git with their own virtual file system VFSforGit [0] and a bunch of performance improvements. 0 - https://vfsforgit.org/
Curious why facebook went for mercurial?
Re: EdenSCM – A cross-platform, scalable source control management system
#8There is more tooling needed in general - just one recursive grep will populate the entire EdenFS. I suppose FB has better tools. But I won’t touch this until the ecosystem is sufficient (and also, because git and hg are perfectly sufficient for the monorepo I oversee)
The file system, the VCS, and the ability to index/grep the repo are just the tip of the iceberg. Pretty much everything that needs to access the filesystem or which depends on the contents/structure of the repo in some way needs to be re-built from scratch or otherwise dramatically customized to operate in this new landscape. That means a long catalog of tools and many years of effort.
This was already starting to become true even before FB switched to Eden, and before it switched to Mercurial (grepping, for example, was already back in the Git days being served by a custom grep service).
Re: EdenSCM – A cross-platform, scalable source control management system
#9Facebook rewrote Mercurial, while Microsoft has essentially expanded git with their own virtual file system VFSforGit [0] and a bunch of performance improvements. 0 - https://vfsforgit.org/
Curious why facebook went for mercurial?
They explain it here: https://engineering.fb.com/core-data/scaling-mercurial-at-fa...
The official reason is that the "internals of Git" weren't conduce to the kinds of invasive changes they needed/wanted. But I think the truth is closer to being that it was going to be too hard/slow to get those invasive changes past the Git mailing list.
Here's an example of a FB eng reaching out to the mailing list: http://git.661346.n2.nabble.com/Git-performance-results-on-a...
Re: EdenSCM – A cross-platform, scalable source control management system
#10Does this work together with a build cache? My dream setup for dealing with building huge code bases is a file system integrated with the version control system to only download files when they are accessed (which sounds like what this does) but also employing a build cache and module system, so it doesn't even need to download and compile any module that has not been touched, it just downloads the result from the bu…
Seems like they can be separated systems. For example, Google's Bazel supports build caches.
https://docs.bazel.build/versions/master/remote-caching.html