Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Lore – Open source version control system designed for scalability

lore.org

Post reply on HN
Print view View original

81–90 of 717 posts

Re: Lore – Open source version control system designed for scalability

#81

Post by jaapz »

TheBigSalad wrote: post #64
throw2ih020 wrote: post #24

For context, since a lot of people on HN haven't worked on games - this is not intended to compete with Git for general software development. This is a competitor with Perforce for game development. Git is fine for text based files like code, but it's really bad at stuff like textures, 3D models, audio files, and other non-text files that game developers need to collaborate on. For example, one artist might need to o…

That sucks, git is so absolutely horrible. It's crazy to me that nobody has made anything better yet. Although I could start that myself and yet have not.

turns out version control is hard

Re: Lore – Open source version control system designed for scalability

#82

Post by spelunker »

niek_pas wrote: post #22

Just today as I pushed some changes to Github, I was thinking how user-unfriendly Git's UI is: Enumerating objects: 5, done. Counting objects: 100% (5/5), done. Delta compression using up to 10 threads Compressing objects: 100% (3/3), done. Writing objects: 100% (3/3), 290 bytes | 290.00 KiB/s, done. Total 3 (delta 2), reused 0 (delta 0), pack-reused 0 remote: Resolving deltas: 100% (2/2), completed with 2 local obje…

The lights are blinking, so everything must be working!

Re: Lore – Open source version control system designed for scalability

#83

Post by cwillu »

falcor84 wrote: post #59
analog8374 wrote: post #30

Earlier quoted context omitted.

All Data is Lore. I mean lore is a superset of data. I mean data is lore with a special attribute. I'm not just picking nits here. And this is not cynicism. so there you go.

I'm not the parent, but I suppose it was a joke reference to Star Trek where Data (an android character) discovers he has a brother named Lore [0]. [0] https://en.wikipedia.org/wiki/List_of_minor_Star_Trek:_The_N...

an evil brother

Re: Lore – Open source version control system designed for scalability

#84

Post by Wowfunhappy »

Aurornis wrote: post #46
chadgpt3 wrote: post #41

Earlier quoted context omitted.

Jonathan Blow found it convenient to represent all assets in a large number of text files, to enable merging. For instance he'd have one text file per entity on a map. The game and editor could read either this or the compiled binary version.

Jonathan Blow works with extremely small team sizes relative to the big studios. When you only have a couple people working on a project you don’t need all of the same coordination features.

He's also the sort of person who I suspect works in a very idiosyncratic way, which is great for him and his mind but probably not everyone else. (This is not a criticism.)

Re: Lore – Open source version control system designed for scalability

#85

Post by stevefan1999 »

LugosFergus wrote: post #62
throw2ih020 wrote: post #24

For context, since a lot of people on HN haven't worked on games - this is not intended to compete with Git for general software development. This is a competitor with Perforce for game development. Git is fine for text based files like code, but it's really bad at stuff like textures, 3D models, audio files, and other non-text files that game developers need to collaborate on. For example, one artist might need to o…

Something else that git isn't good at: permissions. In gamedev, you might have proprietary work that you want to restrict to certain users. In P4, you can add restrictions to certain directories for only those who have signed the required NDAs. That's not something that you can do in git: it's all or nothing. Maybe you can set something up with submodules, but that's going to upend your repository if you hadn't plann…

That's by design of git, you can't forget that git is first developed for a bazaar model of information flow, especially with a big decentrailized project like the Linux Kernel, not the silo and isolated corporate NDA and closed model you described. Git prefers open information and discourages information closures and segregation of information by placing restrictions exactly like this.

Git enthusiast would often tell you to do this separately with a submodule, and set permission on the version control forge software level (which means Gitea/Github private RBAC access to certain repos for cloning), sure, but that is also painful as hell.

But my point is that all of this is exactly by design from Linus Torvalds's need for Linux Kernel to replace BitKeeper. Git simply isn't the tool for everything, it was developed for a software project with liberalism in mind, but corporate stuff is monoculture and prefers proprietary, shut-in model, and the eat your own dog food mindset, and no wonder it is so painful to deal with.

Re: Lore – Open source version control system designed for scalability

#86

Post by flohofwoe »

rootlocus wrote: post #33
throw2ih020 wrote: post #24

For context, since a lot of people on HN haven't worked on games - this is not intended to compete with Git for general software development. This is a competitor with Perforce for game development. Git is fine for text based files like code, but it's really bad at stuff like textures, 3D models, audio files, and other non-text files that game developers need to collaborate on. For example, one artist might need to o…

Git LFS has file locking, and no VCS can provide you with the tools for diffing binary assets. I don't see any meaningful difference between Perforce, Diversion or Lore and git + LFS + file locking. Unless there's a meaningful performance impact for large projects (I only work on small / medium projects), the capabilities are the same. However, I get excellent git support for code in any editor, as opposed to Diversi…

I guess you never worked with anything but git? The devil is in the details, and those details generally suck more in git (or generally: distributed version control systems) than in traditional centralized VCS's.

Also git-lfs is a crutch that breaks more often than it works :/

(I agree though that for small game projects, git is mostly 'good enough', even without lfs).

Re: Lore – Open source version control system designed for scalability

#87

Post by frollogaston »

Lucasoato wrote: post #52

> Git’s content-addressed revision graph is excellent, but it treats binary files as second-class citizens—large files require bolted-on LFS rather than first-class chunked storage, sparse checkouts have sharp edges in offline use, and there is no native multi-tenant isolation. I'm trying to figure out what Lore can accomplish that git+LFS can't. I've read about big binaries chunking, native interface and permission,…

The fact that it's even referred to as git+LFS instead of just git... If I needed to work with large files frequently, I wouldn't want such a basic feature bolted on. Not a criticism of git, just can see why Epic doesn't use git.

Re: Lore – Open source version control system designed for scalability

#88

Post by penciltwirler »

The premise is that Git-LFS sucks, so we need to build a new data versioning system (in Rust, from scratch). While I mostly agree with this premise, but there are already lots of existing (mature) data versioning systems with the same tricks under the hood:

- Pachyderm (Go): https://github.com/pachyderm/pachyderm

- XetHub (acquired by HuggingFace): https://huggingface.co/blog/xethub-joins-hf

- LakeFS (Go): https://github.com/treeverse/lakeFS

- Oxen (Rust): https://github.com/Oxen-AI/Oxen

I guess with AI, anyone can vibe code a content-addressed, chunk-level deduped, versioning system in Rust these days...

But jokes aside, Lore seems really cool! What's interesting is the realization that different domains/industries have similar problems, but they don't seem to be cross-polinating. In this case AI and Gaming both need a storage system that can version control large binary files at scale. I think there's lots of opportunities to share ideas here, but perhaps the lack of idea sharing (currently) creates opportunity!

Re: Lore – Open source version control system designed for scalability

#89

Post by frollogaston »

simcop2387 wrote: post #19
glouwbug wrote: post #13

I’ve always wanted a git with five commands, and maybe with AST based diffing

The five command part isn't really possible but you can use custom diffs for merges, git diff, etc. pretty easily. There are projects like diffsitter ( https://github.com/afnanenayet/diffsitter ) for doing more intelligent diffs like this for supported languages. EDIT: and then an example for the merge stuff I couldn't find while typing before: https://mergiraf.org/ and HN discussion a few years ago: https://news.yco…

clone, pull, push, branch, merge, add, commit are the ones I use, but that's 7

Re: Lore – Open source version control system designed for scalability

#90

Post by troupo »

rootlocus wrote: post #56
Aurornis wrote: post #46

Earlier quoted context omitted.

Jonathan Blow works with extremely small team sizes relative to the big studios. When you only have a couple people working on a project you don’t need all of the same coordination features.

He and Casey Muratory make a lot of cool instructive content, but their condescending attitude towards the industry always made me thing "Huh, must be really nice working alone and making all the decisions yourself."

And then they usually back their words by doing things like "you claim that outputting text to a terminal emulator is a PhD level problem, so here I did it in a weekend".

Huge teams are more often than not the sign of bloat and inefficiences.

Post reply on HN