Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Reproducible Git Bundles

baecher.dev

Post reply on HN
Print view View original

31–38 of 38 posts

Re: Reproducible Git Bundles

#31

Post by rlkf »

RcouF1uZ4gsC wrote: post #29

>The naive solution of simply backing up the entire file-system tree is clearly not desirable since that would clutter the backup with useless build artifacts. Just ignore the files in .gitignore and backup the entire file-system tree. Don’t be clever. This is a backup of source code that took many hours/days/weeks of effort to create. Since git is mainly source code, it is not that big of space hog. Disk space is ch…

> Disk space is cheap. Time is not.

At the same time, this train of though is why VSCode dumps 8 GiB into the ~/.config directory, causing not only a lot longer time to wait for any backup to finish, but also incurring time in getting more and more backup disks, figuring out where to physically store them, etc. Better spend more time up front getting the storage right, and save space and time for those that use it later.

Re: Reproducible Git Bundles

#32

Post by everybodyknows »

stepupmakeup wrote: post #8

Why not just use tar or any other archive tool on the repository .git folder? Unless your repository is a un-gc'd mess with millions of unpacked objects...

Does not meet stated goal of the author:

> The naive solution of simply backing up the entire file-system tree is clearly not desirable since that would clutter the backup with useless build artifacts.

Build artifacts can be filtered out with tar --exclude patterns, but this is a language-dependent set that will require curation.

Re: Reproducible Git Bundles

#33

Post by Hackbraten »

pyrolistical wrote: post #7
Hackbraten wrote: post #2

[flagged]

It easily answers one of the most important questions. Are these two thing equal? With reproducibility you just need to compare bytes. Without, you need additional logic

We're talking about software releases. Those have an identifier already: the version number. You can even pin that identifier to a hash over the release tarball, all without even beginning to consider reproducible builds.

Re: Reproducible Git Bundles

#34

Post by Hackbraten »

dartos wrote: post #3
Hackbraten wrote: post #2

[flagged]

This comment is baffling. Reproducibility in general is a key part of all sciences. Practically speaking, it’s important from a security perspective. If you can make a secure configuration of some system you’d want to be 100% certain you can reproduce that same configuration on some other system. Deep and provable reproducibility doesn’t matter too much for most web servers, unless security is a top priority.

Do you know an example for a security property that a reproducible build would be able to guarantee, and where pinning a dependency against a version identifier plus digest of the release tarball wouldn't accomplish the same thing?

Re: Reproducible Git Bundles

#35

Post by Hackbraten »

makeworld wrote: post #5
Hackbraten wrote: post #2

[flagged]

In this case, it accomplishes efficient backups.

What's efficient (or less efficient) about backing up two identical files vs. two different files with comparable sizes?

If one has a backup scheme where files are stored in a content-adressable store, I can see the efficiency. But I have never heard of such a scheme in the context of backups.

If one has a backup scheme with automatic deduplication over arbitrary byte ranges (such as ZFS's deduplication feature), I can see the efficiency. But I have never seen anyone enable that feature in the context of backups due to its massive RAM usage requirements.

Re: Reproducible Git Bundles

#36

Post by cesarb »

everybodyknows wrote: post #32
stepupmakeup wrote: post #8

Why not just use tar or any other archive tool on the repository .git folder? Unless your repository is a un-gc'd mess with millions of unpacked objects...

Does not meet stated goal of the author: > The naive solution of simply backing up the entire file-system tree is clearly not desirable since that would clutter the backup with useless build artifacts. Build artifacts can be filtered out with tar --exclude patterns, but this is a language-dependent set that will require curation.

Wouldn't all these useless build artifacts be outside the .git folders?

Re: Reproducible Git Bundles

#37

Post by cesarb »

Hackbraten wrote: post #35
makeworld wrote: post #5

Earlier quoted context omitted.

In this case, it accomplishes efficient backups.

What's efficient (or less efficient) about backing up two identical files vs. two different files with comparable sizes? If one has a backup scheme where files are stored in a content-adressable store, I can see the efficiency. But I have never heard of such a scheme in the context of backups. If one has a backup scheme with automatic deduplication over arbitrary byte ranges (such as ZFS's deduplication feature), I c…

> If one has a backup scheme where files are stored in a content-adressable store, I can see the efficiency. But I have never heard of such a scheme in the context of backups.

One example of a system which does that is borgbackup. But even a simple "rsync with snapshots" benefits from identical files, since with common rsync options identical files are not transferred again (which means the data is kept shared between the snapshots).

Re: Reproducible Git Bundles

#38

Post by dartos »

Hackbraten wrote: post #34
dartos wrote: post #3

Earlier quoted context omitted.

This comment is baffling. Reproducibility in general is a key part of all sciences. Practically speaking, it’s important from a security perspective. If you can make a secure configuration of some system you’d want to be 100% certain you can reproduce that same configuration on some other system. Deep and provable reproducibility doesn’t matter too much for most web servers, unless security is a top priority.

Do you know an example for a security property that a reproducible build would be able to guarantee, and where pinning a dependency against a version identifier plus digest of the release tarball wouldn't accomplish the same thing?

I think pinning a dependency against a version identifier and a digest of a tarball is a great way to ensure reproducibility.

You’re controlling which dependency you’re using very specifically.

Reproducibility like this protects against supply chain attacks.

This may be a bit contrived, but It could prevent a malicious package maintainer from releasing a modified version of OpenSSL that you depend on without you noticing that change in your dependencies.

Post reply on HN