Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/publish-checks-tagged-source.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
verwalter: minor
---

`publish` fails a package whose files changed since its tag, before its publish command runs. The reason names `ref=<tag>`: a run by hand from a later commit no longer publishes newer source under the tagged version. `publish --allow-changed-source` turns the check off. Versions the registry has are skipped before the check.
2 changes: 1 addition & 1 deletion docs/commands.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
| `verwalter pr [--remote origin]` | Push the branch and open or update the PR. |
| `verwalter release [--asset <file\|dir>]... [--package <name>] [--publish]` | Tag and create GitHub releases for every untagged version that has a changelog entry. Uploads assets to drafts or a named package. A directory stands for every file directly in it. `--publish` flips a draft. |
| `verwalter release --abandon <tag>` | Delete a GitHub release and its tag. |
| `verwalter publish [--releases <json>] [--jobs <n>]` | Run each package's publish command for the packages in `--releases` (the `releases` output of `release`), or every tagged package without it or with an empty list, dependencies first, up to `n` at once. A failure blocks dependents. A version on the registry is skipped. |
| `verwalter publish [--releases <json>] [--jobs <n>] [--allow-changed-source]` | Run each package's publish command for the packages in `--releases` (the `releases` output of `release`), or every tagged package without it or with an empty list, dependencies first, up to `n` at once. A failure blocks dependents. A version on the registry is skipped. A package whose files changed since its tag fails, unless `--allow-changed-source` is set. |
| `verwalter pre-release [--target <branch>]` | Open an rc PR for every package with a pending changeset and the fixed groups they are in, from `target`. Changesets stay pending. |

All commands take `--config <file>` and an optional checkout path
Expand Down
7 changes: 4 additions & 3 deletions docs/features.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ What the tool does, what it catches, and what to do when a step fails.
| One release PR | Every package with a pending bump goes in one PR. Each push regenerates the branch from `main` and updates the PR in place. With nothing pending, the PR is closed. | `pr`, `[release].branch_prefix` |
| Dependent bumps | A package whose in-repo dependency is released gets a bump too, transitively. Its dependency floor moves in its manifest and the lock files. | `[versioning].dependent_bump_level`, `[workspace].pin_files` |
| Dependency order | Releases, tags and the handoff list go dependencies first. A failure stops before any dependent. | `release` |
| Publish scheduler | Runs each package's publish command for the packages the run released, from the `releases` output, dependencies first, independent packages in parallel. A package with `publish.registry = "none"` and no command is released on GitHub only and left out. A failure blocks its transitive dependents and nothing else. A version the registry has is skipped before the command runs. Without `releases`, every tagged package runs. | `publish`, `[publish]`, `publish.registry`, `publish.command` |
| Publish scheduler | Runs each package's publish command for the packages the run released, from the `releases` output, dependencies first, independent packages in parallel. A package with `publish.registry = "none"` and no command is released on GitHub only and left out. A failure blocks its transitive dependents and nothing else. A version the registry has is skipped before the command runs. A package whose files changed since its tag fails before the command runs. Without `releases`, every tagged package runs. | `publish`, `[publish]`, `publish.registry`, `publish.command`, `publish --allow-changed-source` |
| Workspace publish | One recursive command for every package on the default registry, run once from the root, with per-package reporting from the registry afterwards. Per-package commands resolve workspace ranges too: the npm default is `pnpm publish` in a pnpm workspace. | `[publish].workspace` |
| Registry check | One GET per registry answers whether a version exists. 200 skips, 404 publishes, anything else fails. On `goproxy` the GET is the publish: 200 is published, 404 or 410 fails and is re-run. `status` shows the answer. | `publish.registry`, `publish.exists_url` |
| Fixed groups | Members release at one version: the highest in the group, bumped by the highest level any member needs. A member without a changeset gets a "Released with" line. | `[versioning].fixed` |
Expand Down Expand Up @@ -48,6 +48,7 @@ What the tool does, what it catches, and what to do when a step fails.
| Two release runs race on the same tag | The run that loses the create sees the existing release and reports it. |
| A publish fails for one package | Its transitive dependents are blocked and reported with the cause. Every other package publishes. The run exits non-zero. |
| A publish is re-run | Versions the registry has are skipped before the command runs. The rest publish. |
| A run by hand publishes from a commit after the release | A package whose files changed since its tag fails before its command runs. The reason names `ref=<tag>`. Its dependents are blocked. `publish --allow-changed-source` publishes the newer source anyway. |
| The registry is down during a publish | A 503 is an error, not "not there yet". The package fails, its dependents are blocked. |

### When a step fails
Expand Down Expand Up @@ -75,8 +76,8 @@ it keeps the merge commit. A run by hand (`workflow_dispatch`) takes the
workflow file from the branch, so use it when the workflow file needed
a fix, and set its `ref` input to the release tag. Without `ref`, the
run checks out the branch head. The tool then tags that commit if the
tag is missing, and the publish command uploads its source under the
released version.
tag is missing. If the tag exists and the branch head changed a package
since the tag, `publish` fails that package and names the tag.

Running locally needs push rights and `gh auth login`. Fetch first so tags
and `origin/<base>` are current.
Expand Down
4 changes: 4 additions & 0 deletions docs/release-workflows.md
Original file line number Diff line number Diff line change
Expand Up @@ -148,6 +148,10 @@ Three terms:
- A run by hand checks out the branch head. Give it a `ref` input,
as the example workflow does, and set it to the merge commit or to
any tag of that release.
- `verwalter publish` fails a package whose files changed since its
tag, and the reason names `ref=<tag>`. It does not compare files
outside the package path, such as a root lock file, so pin the
commit anyway.
- A later release run can finish an older release: verwalter reports
a draft on every run until it is public. A job that finishes such
a draft checks out the tag of that release, not the commit of the
Expand Down
17 changes: 14 additions & 3 deletions specs/versioning.allium
Original file line number Diff line number Diff line change
Expand Up @@ -875,8 +875,15 @@ surface PreReleaseRequest {
-- empty list counts as absent: it reaches publish only from a re-run
-- by hand after a run that released nothing, and the re-run is for the
-- versions a registry lacks.
-- PublishBuildsTheTaggedSource: changed_since_tag(p) is true when HEAD
-- changed a file of p since its tag, files of a package nested in p
-- excluded. A run by hand from a later commit would otherwise publish
-- newer source under the tagged version. The registry check comes
-- first: a version it has runs no command. A goproxy package is not
-- compared, because the proxy serves the tag. allow_changed_source,
-- from `publish --allow-changed-source`, turns the check off.
rule WorkflowPublishesInDependencyOrder {
when: WorkflowRunsPublish(workflow, repository, releases?)
when: WorkflowRunsPublish(workflow, repository, releases?, allow_changed_source)
let candidates = filter(Packages, p => p.repository = repository and p.release_mode != none and GitTags.any(t => t.package = p and t.version = p.version) and (releases = none or p in releases))
requires: candidates.all(p => p.publish_command != null or p.in_workspace_publish)
ensures:
Expand All @@ -887,6 +894,10 @@ rule WorkflowPublishesInDependencyOrder {
-- packages run at the same time, up to the job limit.
if p.on_registry(p.version) = true:
PublishOutcome.created(package: p, status: `exists`)
else if not allow_changed_source and changed_since_tag(p):
-- PublishBuildsTheTaggedSource. The command builds HEAD,
-- so HEAD must hold the tagged source of the package.
PublishOutcome.created(package: p, status: failed)
else if p.dependencies.any(d => d in candidates and PublishOutcomes.any(o => o.package = d and o.status in {failed, blocked})):
PublishOutcome.created(package: p, status: blocked)
else if publish_command_succeeded(p):
Expand All @@ -906,7 +917,7 @@ entity PublishOutcome {
-- outcome: the job fails, and a re-run finds the version on the registry
-- and tries the flip again.
rule PublishFlipsDrafts {
when: WorkflowRunsPublish(workflow, repository, releases?)
when: WorkflowRunsPublish(workflow, repository, releases?, allow_changed_source)
for o in PublishOutcomes where o.status in {published, `exists`}:
for release in Releases where release.package = o.package and release.status = drafted:
ensures:
Expand Down Expand Up @@ -936,7 +947,7 @@ surface ToolRun {

provides:
ToolRunStarted(repository, named, publish_drafts)
WorkflowRunsPublish(workflow, repository, releases)
WorkflowRunsPublish(workflow, repository, releases, allow_changed_source)
}

-- Every release the run tagged, created, published or attached assets
Expand Down
2 changes: 1 addition & 1 deletion src/check.rs
Original file line number Diff line number Diff line change
Expand Up @@ -269,7 +269,7 @@ fn in_package(file: &str, package_path: &str) -> bool {
/// file, those with the longest path. A file in a nested package, such as
/// `webrtc-sys/build` in `webrtc-sys`, is not a change of the package
/// around it. Two packages at one path both own the file.
fn owners<'a>(config: &'a Config, file: &str) -> Vec<&'a String> {
pub(crate) fn owners<'a>(config: &'a Config, file: &str) -> Vec<&'a String> {
fn depth(path: &str) -> usize {
match path.trim_end_matches('/') {
"." => 0,
Expand Down
10 changes: 9 additions & 1 deletion src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,9 @@ enum Command {
/// Packages to publish at once
#[arg(long, value_name = "N")]
jobs: Option<usize>,
/// Publish even when HEAD changed a package's files since its tag. Without it, such a package fails: check out the tag instead
#[arg(long)]
allow_changed_source: bool,
},
/// Open an rc PR for every package with a pending changeset
PreRelease {
Expand Down Expand Up @@ -331,11 +334,16 @@ fn run(cli: Cli) -> Result<()> {
}
Ok(())
}
Command::Publish { releases, jobs } => {
Command::Publish {
releases,
jobs,
allow_changed_source,
} => {
let options = verwalter::publish::Options {
jobs,
dry_run,
only: only(releases),
allow_changed_source,
};
let outcomes =
verwalter::publish::run(&repo, &options, &verwalter::registry::curl, gh)?;
Expand Down
69 changes: 64 additions & 5 deletions src/publish.rs
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,9 @@ pub struct Options {
/// from the `release` step's output. None publishes every tagged
/// package, for a re-run by hand; the CLI maps an empty list to None.
pub only: Option<BTreeSet<String>>,
/// Spec: PublishBuildsTheTaggedSource. Run the command even when HEAD
/// changed the package's files since its tag.
pub allow_changed_source: bool,
}

#[derive(Debug, Clone, PartialEq, Eq, Serialize)]
Expand Down Expand Up @@ -162,6 +165,25 @@ fn candidates(repo: &Repo, only: Option<&BTreeSet<String>>) -> Result<BTreeMap<S
Ok(out)
}

/// Spec: PublishBuildsTheTaggedSource. True when HEAD changed a file of
/// `name` since `tag`. A file of a package nested in it is not its file.
fn changed_since_tag(repo: &Repo, name: &str, tag: &str) -> Result<bool> {
let path = &repo.config.packages[name].path;
let files = git::run(
&repo.root,
&["diff", "--name-only", "--relative", tag, "HEAD", "--", path],
)?;
Ok(files.lines().any(|f| {
crate::check::owners(&repo.config, f)
.iter()
.any(|o| *o == name)
}))
}

fn changed_reason(tag: &str) -> String {
format!("HEAD changed the source since {tag}: publish from ref={tag}")
}

fn outcome(
package: &str,
version: &crate::Version,
Expand Down Expand Up @@ -339,16 +361,15 @@ pub fn run(
}
if name == WORKSPACE {
// Skip the command only when every member is there.
let mut all = true;
for (m, v, _) in &c.members {
let mut missing = Vec::new();
for (m, v, t) in &c.members {
if registry::exists(registry::version_url(&repo.config, m, v), http)?
!= Some(true)
{
all = false;
break;
missing.push((m, t));
}
}
if all {
if missing.is_empty() {
done.insert(name.clone(), "exists");
for (m, v, t) in &c.members {
done.insert(m.clone(), "exists");
Expand All @@ -357,8 +378,34 @@ pub fn run(
resolved = true;
continue;
}
// One changed member stops the command for every member.
let mut changed = BTreeSet::new();
if !options.allow_changed_source {
for (m, t) in missing {
if changed_since_tag(repo, m, t)? {
changed.insert(m.clone());
}
}
}
if let Some(first) = changed.first() {
done.insert(name.clone(), "failed");
for (m, v, t) in &c.members {
let reason = if changed.contains(m) {
changed_reason(t)
} else {
format!(
"the workspace command did not run: HEAD changed {first} since its tag"
)
};
done.insert(m.clone(), "failed");
outcomes.push(outcome(m, v, t, "failed", reason));
}
resolved = true;
continue;
}
} else if c.command.is_empty() {
// goproxy: asking for the version makes the proxy fetch it.
// The proxy serves the tag, not HEAD: nothing to compare.
let url = registry::version_url(&repo.config, name, &c.version);
let (status, reason) = if options.dry_run {
("dry-run", format!("GET {}", url.unwrap_or_default()))
Expand Down Expand Up @@ -393,6 +440,18 @@ pub fn run(
resolved = true;
continue;
}
if !options.allow_changed_source && changed_since_tag(repo, name, &c.tag)? {
done.insert(name.clone(), "failed");
outcomes.push(outcome(
name,
&c.version,
&c.tag,
"failed",
changed_reason(&c.tag),
));
resolved = true;
continue;
}
}
let first = name != WORKSPACE
&& crate::release::first_release(repo, name, &c.version, http)?;
Expand Down
Loading
Loading