Skip to content

buffrs publish forces --preserve-mtime, defeating its own idempotent-republish path #321

Open

Description

@overby-me

Summary

package and publish disagree on the default for preserve_mtime: it is false for package (main.rs:105-108) and true for publish (main.rs:129-133). Because it is a plain bool clap flag, publish has no way to turn it off — neither --no-preserve-mtime nor --preserve-mtime=false is accepted.

The result is that buffrs package is byte-reproducible but buffrs publish is not, and the divergence breaks a code path buffrs already has.

Why this matters

Artifactory::publish already treats republish as idempotent (artifactory.rs:328-382): it fetches the existing artifact, compares SHA256, and returns Ok(()) with "... is already published, skipping" when the hashes match. That fast path is unreachable in practice, because the tarball publish builds embeds local filesystem mtimes and so never matches a tarball built from a different checkout. Instead you land in the sibling branch:

unable to publish <pkg> to artifactory: package is already published with a different hash

Reproduction

Two checkouts, identical content, different mtimes:

mkdir -p a && cd a && buffrs init --lib my-protos
printf 'syntax = "proto3";\npackage my_protos;\nmessage Foo { string bar = 1; }\n' > proto/my_protos.proto
cd .. && cp -r a b
touch -t 202601011200 a/proto/my_protos.proto
touch -t 202606151530 b/proto/my_protos.proto

package (default, preserve_mtime=false) is reproducible:

$ (cd a && buffrs package --output-directory /tmp/d-a)
$ (cd b && buffrs package --output-directory /tmp/d-b)
$ sha256sum /tmp/d-a/*.tgz /tmp/d-b/*.tgz
e9df1478...  /tmp/d-a/my-protos-0.1.0.tgz
e9df1478...  /tmp/d-b/my-protos-0.1.0.tgz     # identical

The same packaging as performed by publish is not:

$ (cd a && buffrs package --preserve-mtime --output-directory /tmp/p-a)
$ (cd b && buffrs package --preserve-mtime --output-directory /tmp/p-b)
$ sha256sum /tmp/p-a/*.tgz /tmp/p-b/*.tgz
babc25a0...  /tmp/p-a/my-protos-0.1.0.tgz
e2f4bb87...  /tmp/p-b/my-protos-0.1.0.tgz     # differs

mtimes are the only delta:

$ tar tvf /tmp/p-a/*.tgz
-r--r--r-- 0/0    94 Jan  1  1970 Proto.toml
-r--r--r-- 0/0    70 Jan  1  2026 my_protos.proto
$ tar tvf /tmp/p-b/*.tgz
-r--r--r-- 0/0    94 Jan  1  1970 Proto.toml
-r--r--r-- 0/0    70 Jun 15 15:30 my_protos.proto

And the flag cannot be disabled on publish:

$ buffrs publish --no-preserve-mtime ...
error: unexpected argument '--no-preserve-mtime' found
$ buffrs publish --preserve-mtime=false ...
error: unexpected value 'false' for '--preserve-mtime' found; no more were expected

Expected

publish produces the same tarball as package for the same sources, so republishing an unchanged version hits the existing "already published, skipping" path.

Actual

publish always embeds mtimes, so the hash-match branch is dead code for any cross-checkout republish and the mismatch error fires instead.

Impact

Breaks retry-safety. In a workspace, a mid-publish failure leaves some members published; the rerun cannot succeed because no retry clears a hash mismatch. Recovery means manually deleting partial artifacts from the registry.

Suggested fix

Flip publish's default to false to match package. The packaging path is otherwise already reproducibility-conscious — gzip mtime is pinned to 0 (compressed.rs:130) and entries are sorted via BTreeMap with an explicit comment to that effect — so preserve_mtime is the only remaining source of nondeterminism.

If the current default must be kept for compatibility, at minimum make it disableable, e.g. #[clap(long, action = ArgAction::Set, default_value_t = true)] or a --no-preserve-mtime counterpart. Normalising to SOURCE_DATE_EPOCH when set would also be reasonable.

Activity

  1. changed the title [-]`buffrs publish` produces non-reproducible tarballs[/-] [+]`buffrs publish` forces `--preserve-mtime`, defeating its own idempotent-republish path[/+] on Sep 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions