Earlier quoted context omitted.
And a few of them think that it is corporate structure and development models that are actually the problem. I'm just asking questions!
I don't understand what you're trying to imply. If you're making the point that there are multiple confounding factors to just about any non-trivial problem, then I agree.
Lore – Open source version control system designed for scalability
471–480 of 717 posts
Re: Lore – Open source version control system designed for scalability
#472Re: Lore – Open source version control system designed for scalability
#473For context, since a lot of people on HN haven't worked on games - this is not intended to compete with Git for general software development. This is a competitor with Perforce for game development. Git is fine for text based files like code, but it's really bad at stuff like textures, 3D models, audio files, and other non-text files that game developers need to collaborate on. For example, one artist might need to o…
However combining some of the flexibility and workflows of git with the ability to deal more efficiently and effectively with large asset files is something that a hell of a lot game engineers would be interested in. And having the virtual checkout as a feature out of the box for folks used to half terabyte repo sizes is definitely a huge plus.
This announcement is definitely a big deal, and if the promises for lore actually measure up, we could be seeing the beginning of a switch over to open source version control for larger asset heavy games where git was still not a great fit.
Will be interesting to see what the public code hosting platforms do (github), and whether there will be any major structural changes to git (I'm thinking not likely). I wonder if Epic will make a play to take business away from github or gitlabs.
Re: Lore – Open source version control system designed for scalability
#474Re: Lore – Open source version control system designed for scalability
#475Earlier quoted context omitted.
Strikes me as bizarre that payment code would be sensitive, unless it's a security by obscurity thing (which would also be concerning). Keys, secrets, etc. yes. But code? What am I missing here?
As others have said, it's Apple and they do not take kindly to other people leaking their technology/announcements ahead of time. See also: the time that ATI's CEO told his employees that their chips would be powering Apple's to-be-announced hardware a few days before the announcement. Steve Jobs responded by pulling all of ATI's hardware from its demo units at the announcement, not mentioning ATI at all, cancelling…
Re: Lore – Open source version control system designed for scalability
#476Earlier quoted context omitted.
> this is not intended to compete with Git for general software development. This is a competitor with Perforce for game development. Well, it is intended to compete with Git for version control. It's just that Git happens to be so bad at some aspects of version control that it isn't used much in those cases. There's no good reason that Git couldn't be good at versioning binary files, or splitting up large projects.…
It feels like a Pareto Principle problem. 80% of source control is text files that can be three-way merged as text files, but a lot of hard problems are in that 20% that isn't. Git does very well at the 80% and with tools like custom merge tools and git lfs/annex and git sparse "cone" checkouts can get pretty close to hitting the 90 or 95% case. But yeah, so many of those extra tools in that 80 to 90% area are awful…
The only alternative Git offers, which is slightly better IMO, is monorepos. But they're only slightly better - they also have really significant downsides.
I'm 100% sure there's a much nicer solution to the kinds of problems people use submodules for that isn't submodules, but as far as I can tell zero people are trying to find it, despite it being such a universal problem.
Re: Lore – Open source version control system designed for scalability
#477For context, since a lot of people on HN haven't worked on games - this is not intended to compete with Git for general software development. This is a competitor with Perforce for game development. Git is fine for text based files like code, but it's really bad at stuff like textures, 3D models, audio files, and other non-text files that game developers need to collaborate on. For example, one artist might need to o…
Edit: I do feel a bit uneasy about epic games, though.
Re: Lore – Open source version control system designed for scalability
#478Earlier quoted context omitted.
I completely agree with every single point you've said here. Running a P4 server is an exercise in self torture, but it's the only option with UE5. > I wish I could use a better code review tool than Swarm. With one caveat. I think swarm is a great tool. It could do with being easier to configure, but as a code review tool it's simple, straightforward and gets the job done. > I wish I could integrate SSO without weir…
I'm actually using helix-authentication-service. Swarm "works with it" in theory, but in practice I had to patch the loginWithSSO function to shell out to the p4 process to handle auth, because P4PHP doesn't expose ClientSSO at all and it's required for a clean login with keycloak. EntraID is a different flow that they've probably nailed, from what I hear from other studios.
Re: Lore – Open source version control system designed for scalability
#479Earlier quoted context omitted.
Is Google a big 3 media/entertainment studio? While I agree with you, not at Google anymore, I do not consider them the top 3 engine providers that I am referring to in my post. No no, Unreal/Unity/Red/EA/SC it’s all perforce.
Ahhhh. Okay. I didn't realize you meant game studios. Wasn't familiar with that "big 3" term until now. Thanks for clearing that up.
Re: Lore – Open source version control system designed for scalability
#480Earlier quoted context omitted.
What do you do instead of chunking your snapshots? Storing diffs is usually the other approach.
I tried to give that section of the doc a fair read. Looks like operational transforms to me. The doc claims it's the first with this technique. A 30 second search reminded me of Darcs, and taught me about Pijul, and Weave. And yes, Google Docs storage works the same way - there are probably papers documenting how efficient Google Docs storage is, but it's not wrapped up in a full VCS that folks can use. The example…
Sort of. I add provenance, which helps properly identify collisions, and require a well-defined order by stacking [1] changelists.
> The doc claims it's the first with this technique.
More like the first with the particular angle on the technique. I specifically mention patch theory as another side of the same coin.
> A 30 second search reminded me of Darcs, and taught me about Pijul, and Weave.
Darcs is Pijul's ancestor, and I mentioned Pijul. I also mentioned the weave and how reference sets scale better.
> The example in the doc uses text, and unfortunately I think it's for a reason.
Readability. Nothing more. The real stuff will be a compact binary format.
> I think with large, binary game assets, the most common operation is going to be strings of "replace A with B", and depending on your chunk size relative to the distribution of changes you make on your assets, I see it as pretty close to a wash, for efficiency.
Yore will dedup change data instead because as the Lore document itself identifies, dedupping content is hard using chunks; you either get dedupping or canonical addresses. Change data doesn't have one canonical address; the address is in the commit data instead.
Dedupping changes has another benefit. If most instances are "replace A with B," and A replaces B in multiple places, Yore will be able to store just one instance of A, no matter its size. This matters because the larger the chunk, the less likely it will match any other chunk.
> Especially considering that content-addressable blocks also solves de-duplication, which for a multi-game studio is probably going to be significant. Especially if they're managing multiple releases, patches, development branches, etc.
True, but that should be table stakes. The fact that Git does not is a poor reflection on Git, not an innovation in Lore.