Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Disorderfs: FUSE-based filesystem that introduces non-determinism into metadata

salsa.debian.org

Post reply on HN
Print view View original

11–20 of 33 posts

Re: Disorderfs: FUSE-based filesystem that introduces non-determinism into metadata

#11

Post by Spivak »

amelius wrote: post #6
danbst wrote: post #5

Sorry for plugging in. NixOS has a plan to use DisorderFS in making 100% reproducible build. https://r13y.com/ As I understand it, currently several packages are not reproducible (like, python, pytest, gcc), so it is not priority, but when those large packages will be done, r13y will start using DisorderFS to uncover remaining reproducibility bugs. This is too idealistic, but gives lots of pleasure about package spac…

Instead of introducing non-determinism, shouldn't it instead try to enforce determinism in any possible way? (E.g. running all processes under ptrace or by using virtualization and thereby making the OS behave in a deterministic way during a build).

It’s going to be part of the testing suite. The packages should be able to build exactly the same even in a hostile environment. You want the environment “outside” your build system to be as chaotic as possible so you know there aren’t any accidental dependencies you don’t catch.

Re: Disorderfs: FUSE-based filesystem that introduces non-determinism into metadata

#12

Post by ithkuil »

infogulch wrote: post #7
EdSchouten wrote: post #4

Buildbarn, a build cluster implementation for Bazel that I maintain, can also run build actions (compilation steps, unit tests) in a FUSE file system. Though the primary motivator for this is that it reduces the time to construct a build action's file system to nearly instant, it has the advantage that I can also do things similar to disorderfs. Shuffling directory listings is actually something that I also added. Pr…

If you are confident that certain build steps are deterministic, can you enable Dockerfile-like caching for intermediate steps? The way docker does it is take a hash of the "input" filesystem and the command, and see if there's an associated "result" filesystem, and if there is then just jump to evaluating the next command with the previous result as input.

That's precisely what bazel does (and remote build execution systems like buildbarn allow this cache to be effectively shared among users)

Re: Disorderfs: FUSE-based filesystem that introduces non-determinism into metadata

#13

Post by lambda_obrien »

ithkuil wrote: post #10
amelius wrote: post #6

Earlier quoted context omitted.

Instead of introducing non-determinism, shouldn't it instead try to enforce determinism in any possible way? (E.g. running all processes under ptrace or by using virtualization and thereby making the OS behave in a deterministic way during a build).

The goal is to ensure that builds can be deterministic despite non-determinism. Once such criterion is enforced, then everybody can reproduce the build with that set of source files and build instructions, without requiring a special environment that forces a specific order of events.

> The goal is to ensure that builds can be deterministic despite non-determinism

...by using non-determinism?

Very mind bending for me, I'm not sure I understand, but I'm glad smart people are figuring this stuff out.

Re: Disorderfs: FUSE-based filesystem that introduces non-determinism into metadata

#14

Post by agwa »

lambda_obrien wrote: post #13
ithkuil wrote: post #10

Earlier quoted context omitted.

The goal is to ensure that builds can be deterministic despite non-determinism. Once such criterion is enforced, then everybody can reproduce the build with that set of source files and build instructions, without requiring a special environment that forces a specific order of events.

> The goal is to ensure that builds can be deterministic despite non-determinism ...by using non-determinism? Very mind bending for me, I'm not sure I understand, but I'm glad smart people are figuring this stuff out.

You use disorderfs as part of a CI process. The CI builds the package once without disorderfs, and once with disorderfs. If they produce the same output, the package is reproducible (at least with respect to filesystem order). Otherwise, something in the build process is depending on filesystem order and should be fixed to sort directory entries before using them. You wouldn't use disorderfs when building a package normally.

(At least this is how Debian uses disorderfs. I wrote the first version of disorderfs 6 years ago in a hacking session at DebConf15 in Heidelberg. I never expected to see it on the front page of HN!)

Re: Disorderfs: FUSE-based filesystem that introduces non-determinism into metadata

#15

Post by gopalv »

amelius wrote: post #6
danbst wrote: post #5

Sorry for plugging in. NixOS has a plan to use DisorderFS in making 100% reproducible build. https://r13y.com/ As I understand it, currently several packages are not reproducible (like, python, pytest, gcc), so it is not priority, but when those large packages will be done, r13y will start using DisorderFS to uncover remaining reproducibility bugs. This is too idealistic, but gives lots of pleasure about package spac…

Instead of introducing non-determinism, shouldn't it instead try to enforce determinism in any possible way? (E.g. running all processes under ptrace or by using virtualization and thereby making the OS behave in a deterministic way during a build).

> shouldn't it instead try to enforce determinism in any possible way

That would only fix the build machine's problem, it wouldn't fix anyone else's builds.

A repeatable build without determinism is a fix for all people everywhere.

Re: Disorderfs: FUSE-based filesystem that introduces non-determinism into metadata

#16

Post by sneak »

lambda_obrien wrote: post #13
ithkuil wrote: post #10

Earlier quoted context omitted.

The goal is to ensure that builds can be deterministic despite non-determinism. Once such criterion is enforced, then everybody can reproduce the build with that set of source files and build instructions, without requiring a special environment that forces a specific order of events.

> The goal is to ensure that builds can be deterministic despite non-determinism ...by using non-determinism? Very mind bending for me, I'm not sure I understand, but I'm glad smart people are figuring this stuff out.

It's to root out "works [deterministically] on my machine" bugs earlier.

Re: Disorderfs: FUSE-based filesystem that introduces non-determinism into metadata

#17

Post by myself248 »

From the headline, I was picturing EXIF tags, which would be pretty amusing. Store your photos at this mountpoint here, read 'em from that mountpoint over there, and their locations and camera details get anonymized. But the real data still lives on disk.

Alas that's not what this is about, but now I wonder how hard it would be to make the thing I had in my head.

Re: Disorderfs: FUSE-based filesystem that introduces non-determinism into metadata

#18

Post by lambda_obrien »

agwa wrote: post #14
lambda_obrien wrote: post #13

Earlier quoted context omitted.

> The goal is to ensure that builds can be deterministic despite non-determinism ...by using non-determinism? Very mind bending for me, I'm not sure I understand, but I'm glad smart people are figuring this stuff out.

You use disorderfs as part of a CI process. The CI builds the package once without disorderfs, and once with disorderfs. If they produce the same output, the package is reproducible (at least with respect to filesystem order). Otherwise, something in the build process is depending on filesystem order and should be fixed to sort directory entries before using them. You wouldn't use disorderfs when building a package n…

Thanks! That makes more sense now.

Re: Disorderfs: FUSE-based filesystem that introduces non-determinism into metadata

#19

Post by ampdepolymerase »

amelius wrote: post #6
danbst wrote: post #5

Sorry for plugging in. NixOS has a plan to use DisorderFS in making 100% reproducible build. https://r13y.com/ As I understand it, currently several packages are not reproducible (like, python, pytest, gcc), so it is not priority, but when those large packages will be done, r13y will start using DisorderFS to uncover remaining reproducibility bugs. This is too idealistic, but gives lots of pleasure about package spac…

Instead of introducing non-determinism, shouldn't it instead try to enforce determinism in any possible way? (E.g. running all processes under ptrace or by using virtualization and thereby making the OS behave in a deterministic way during a build).

It's a form of fuzzing. A bug free build system would not be affected by non-determinism.

Re: Disorderfs: FUSE-based filesystem that introduces non-determinism into metadata

#20

Post by amelius »

agwa wrote: post #14
lambda_obrien wrote: post #13

Earlier quoted context omitted.

> The goal is to ensure that builds can be deterministic despite non-determinism ...by using non-determinism? Very mind bending for me, I'm not sure I understand, but I'm glad smart people are figuring this stuff out.

You use disorderfs as part of a CI process. The CI builds the package once without disorderfs, and once with disorderfs. If they produce the same output, the package is reproducible (at least with respect to filesystem order). Otherwise, something in the build process is depending on filesystem order and should be fixed to sort directory entries before using them. You wouldn't use disorderfs when building a package n…

I think you need to build many times to be sure.

Therefore (the original question), instead of using "disorderfs", why not write and use an "orderedfs" for every build?

Post reply on HN