To run the check build manually:
nix-build /nix/store/???.drv --check --keep-failedAnd to find out why a derivation is part of the build closure, use:
git clone https://github.com/nixos/nixpkgs.git cd nixpkgs git checkout 5a2500a5e80cb462fe1cc45b26a9e5584eeabef1 nix-store -q --tree $(nix-instantiate nixos/release-combined.nix -A nixos.iso_minimal.x86_64-linux)
paths in the nixos.iso_minimal.x86_64-linux build closure are reproducible! 0 paths remained unchecked
These derivations produced a different result compared to the official cache, revealing a reproducibility problem. See the Project for progress on fixing those, or add an issue for those that are not tracked yet.
Each build is run twice, at different times, on different hardware running different kernels.
Fairly. We don't currently inject randomness at the filesystem layer, but many of the reproducibility issues are being exercised already. It isn't possible to guarantee a package is reproducible, just like it isn't possible to prove software is bug-free. It is possible there is nondeterminism in a package source, waiting for some specific circumstance.
This is why we run these tests: to track how we are doing over time, to submit bug fixes for nondeterminism when we find them.
There are further steps we could take. For example, the next likely step is using disorderfs which injects additional nondeterminism by reordering directory entries.
Nix has built-in support for checking a path is reproducible. There are two routes.
Pretending you are debugging a nondeterminism bug in
hello. To check it, you build the package, and then
build it again with
--check --keep-failed. This will
provide the differing output in a separate directory which you can
$ nix-build . -A hello $ nix-build . -A hello --check --keep-failed [...snip...] error: derivation '/nix/store/...hello.drv' may not be deterministic: output '/nix/store/...-hello' differs from '/nix/store/...hello.check' $ diffoscope /nix/store/...hello /nix/store/...hello.check
.check output is not a valid store path, and
will automatically be deleted on the next run of the Nix garbage
There is support for an automatic
Nix 2, but it is much more complicated to set up. If you would like to
work on this, or need help setting it up,
contact us on Matrix.
We can work together to write docs on how to use it.