Skip to content

Commit b84ddc8

Browse files
danielmeppielDaniel MeppielCopilot
authored
feat(marketplace): filter-driven CLI, map-based manifest, JSON output (#1324)
* feat(marketplace): add map-form outputs parsing + profile validation (#1317 phase-3a) - Add path_env_var field to MarketplaceOutputProfile - Add _validate_profile() guard: reserved names, bad chars, env-var shape - Add MarketplaceOutputSpec dataclass for resolved per-format specs - Parse outputs: as map (new) or list/string (deprecated with warning) - Detect sibling-vs-map path conflicts (sibling wins, with warning) - Wire output_specs + warnings fields on MarketplaceConfig - Add 36 new tests covering profiles + map-form parsing Refs: design.final.md §1 (data model), §5.4 (sibling conflict) Test IDs: T-3a-01..26 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat(marketplace): add BuildReport JSON serialization (#1317 phase-3b) - Add to_json_dict() on BuildReport: §4 JSON contract shape {ok, dry_run, warnings[], errors[], marketplace:{outputs:[]}, bundle:null} - Add failure_to_json_dict() classmethod for pre-build failures - 8 new tests covering success/failure/multi-output/dry-run Refs: design.final.md §4 (JSON contract) Test IDs: T-3b-01..08 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat(marketplace): add --marketplace, --marketplace-path, --json CLI flags (#1317 phase-3c) - Add -m/--marketplace FORMATS filter flag (comma-separated, 'all'/'none') - Add --marketplace-path FORMAT=PATH repeatable override - Add --json flag: emits §4 JSON to stdout, logs to stderr - Deprecate --marketplace-output (hidden, warns, auto-translates) - All error paths emit JSON under --json mode (no broken pipes) - Update existing test for hidden deprecated flag - 7 new CLI flag tests Refs: design.final.md §3 (CLI surface), §4 (JSON contract), §5 (failures) Test IDs: T-3c-01..12 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat(marketplace): update init template to map-form outputs (#1317 phase-3d) - Replace list-form 'outputs: [claude, codex]' scaffold comment with explicit map-form 'outputs: {claude: {}}' per design.final.md §6 - Add CI tip comment showing '--marketplace=... --json | jq' usage - Keep codex section as commented-out example with path override Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs: update pack reference, marketplace guide, and CHANGELOG for #1317 - pack.md: document --marketplace, --marketplace-path, --json flags; mark --marketplace-output as deprecated/hidden; update YAML examples to map form; add JSON mode behavior bullet - publish-to-a-marketplace.md: update outputs examples to map form; mention --json for CI - CHANGELOG: add 5 'Added' + 1 'Changed' entries under [Unreleased] Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * style: apply ruff format to marketplace output UX files Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: exclude build/apm-*/ from version control The apm pack output directory was accidentally committed. Add glob pattern to .gitignore to prevent recurrence. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: address review panel blockers + add E2E tests (#1317) BLOCKING fixes: - Route CommandLogger/Rich output to stderr under --json via set_console_stderr() — no stdout contamination from logs - Wire --marketplace filter to BuildOrchestrator: marketplace_formats and marketplace_path_overrides fields added to BuildOptions, filter applied in MarketplaceProducer.produce() - Extract _emit_json_error_or_raise() helper — eliminates triplicated json-error emission pattern SECURITY: - Add validate_path_segments() to --marketplace-path CLI values, rejecting path traversal (../) attacks on CLI surface UX + DOCS: - Remove 'Repeatable.' from -m docs (comma-separated, not multiple=True) - Fix CHANGELOG: env-var overrides planned for v0.15, not shipped - Add docstring to path_env_var: declared for validation only, not consumed - Deprecation warning uses click.echo(err=True) — stderr, not stdout - Consistent JSON envelope on both success and error paths - Remove dead 'return' after ctx.exit(1) E2E tests (7 new): - JSON envelope has all required keys on success + error - No stdout contamination under --json - Path traversal rejected (plain + json mode) - Deprecation warning does not corrupt JSON stdout - --marketplace=none produces empty outputs array Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Daniel Meppiel <copilot-rework@github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent 894d73a commit b84ddc8

17 files changed

Lines changed: 1376 additions & 48 deletions

File tree

.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -77,3 +77,4 @@ scout-pipeline-result.png
7777
.playwright-mcp/
7878
server.pid
7979
.docs-rewrite-plan/
80+
build/apm-*/

CHANGELOG.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
1212
- `apm install --target claude` now preserves self-defined stdio MCP `env` values from `apm.yml` and writes non-string values such as `PORT: 3000` and `DEBUG: false` as MCP-compatible strings. (#1222)
1313
- Non-skill integrators (agent, instruction, prompt, command, hook script-copy) silently adopt byte-identical pre-existing files so a degraded `deployed_files=[]` lockfile no longer permanently blocks installs gated by `required-packages-deployed`. (#1313)
1414
- `apm audit` drift check now returns skip-with-info (`passed=True`) when the install cache is cold, instead of failing the audit; bare `apm audit` surfaces the skip reason on stderr so CI pipelines that have not yet run `apm install` are not incorrectly red-marked. (#1289)
15+
16+
### Added
17+
18+
- `apm pack --marketplace=FORMATS` filters which marketplace formats are built in a single run; accepts comma-separated names and sentinels `all`/`none`. (#1317)
19+
- `apm pack --marketplace-path FORMAT=PATH` overrides the output path for a specific marketplace format at invocation time. Env var overrides (`APM_MARKETPLACE_<FORMAT>_PATH`) are planned for v0.15. (#1317)
20+
- `apm pack --json` emits a stable JSON contract to stdout (`{ok, dry_run, warnings, errors, marketplace: {outputs: [{format, path, ...}]}}`); all logs move to stderr so downstream tooling can `jq` the output safely. (#1317)
21+
- `marketplace.outputs` in `apm.yml` now accepts a map form keyed by format name (`outputs: {claude: {}, codex: {path: ...}}`), replacing the deprecated list form; the list form still parses with a one-cycle deprecation warning. (#1317)
22+
- `apm marketplace init` now scaffolds the explicit map-form `outputs: {claude: {}}` so the default state is observable in the manifest. (#1317)
23+
24+
### Changed
25+
26+
- `--marketplace-output PATH` is now hidden from `--help` and emits a stderr deprecation warning; it auto-translates to `--marketplace-path claude=PATH`. Removal tracked in #1318. (#1317)
1527
- `extends: org` now correctly layers `dependencies.require` and `dependencies.deny` from the parent policy when the child omits the `dependencies:` block entirely; `None` signals "no opinion" (transparent) while `[]` signals explicit override. (#1290)
1628
- CI self-check job now uses `setup-only: true` + `apm audit --ci --no-drift` so managed files are not overwritten by `apm install` before `content-integrity` runs; documented the audit-only CI pattern and the install-before-audit blind spot in the enterprise and CI/CD guides. (#1291)
1729
- Pin `Path.home()` under unit tests via a session-scoped autouse conftest fixture, fixing 56 Windows runner failures on the new `windows-2025-vs2026` GitHub-hosted image where `USERPROFILE`/`HOMEDRIVE`+`HOMEPATH` are not seeded for pytest workers; also patch the `_check_and_notify_updates` import binding in the disabled-self-update test so it no longer races on the version-check cache. (#1270)

docs/src/content/docs/producer/publish-to-a-marketplace.md

Lines changed: 11 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -71,7 +71,8 @@ marketplace:
7171
name: acme-org
7272
url: https://github.com/acme-org
7373

74-
outputs: [claude] # default; add codex for Codex repo output
74+
outputs: # map form (recommended)
75+
claude: {} # default; add codex for Codex repo output
7576

7677
claude:
7778
output: .claude-plugin/marketplace.json
@@ -108,13 +109,19 @@ For the full field reference (every key on every entry, including
108109
`pluginRoot`, `outputs`, `claude`, `codex`, and pass-through fields
109110
like `tags`, `author`, `license`), see the reference below.
110111

111-
Marketplace output targets use a selector-list pattern:
112+
Marketplace output targets use a map-form pattern:
112113

113114
```yaml
114115
marketplace:
115-
outputs: [claude, codex]
116+
outputs:
117+
claude: {}
118+
codex:
119+
path: .agents/plugins/marketplace.json
116120
```
117121

122+
The legacy list form (`outputs: [claude, codex]`) still parses with a
123+
deprecation warning but new projects should use the map form above.
124+
118125
Claude output is selected by default for backwards compatibility. The
119126
legacy `marketplace.output` field remains supported as shorthand for
120127
`marketplace.claude.output`; when both are set, the explicit
@@ -143,6 +150,7 @@ apm pack --dry-run # resolve and print; do not write
143150
apm pack --offline # cached refs only
144151
apm pack --include-prerelease # allow pre-release tags
145152
apm pack -v # per-entry resolution detail
153+
apm pack --marketplace=claude --json # JSON output for CI pipelines
146154
```
147155

148156
Exit codes: `0` build succeeded, `1` build error (network, missing

docs/src/content/docs/reference/cli/pack.md

Lines changed: 19 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,10 @@ Bundles are target-agnostic. The consumer's project decides where files land at
3535
| `--verbose`, `-v` | off | Show per-file paths and detailed packer output. |
3636
| `--offline` | off | Marketplace: resolve version ranges from cached refs only; skip `git ls-remote`. |
3737
| `--include-prerelease` | off | Marketplace: allow pre-release tags to satisfy version ranges. |
38-
| `--marketplace-output PATH` | `.claude-plugin/marketplace.json` | Marketplace legacy compatibility: override only the Claude/Anthropic output path. Prefer `marketplace.claude.output` in `apm.yml`. |
38+
| `-m`, `--marketplace FORMATS` | all configured | Comma-separated list of marketplace formats to build. Sentinels: `all` (every configured format), `none` (skip marketplace entirely). |
39+
| `--marketplace-path FORMAT=PATH` | manifest default | Override the output path for a specific format. Repeatable. Example: `--marketplace-path codex=./dist/codex.json`. |
40+
| `--json` | off | Emit machine-readable JSON to stdout. All logs move to stderr. Shape: `{ok, dry_run, warnings, errors, marketplace: {outputs: [...]}}`. |
41+
| `--marketplace-output PATH` | _(hidden)_ | **Deprecated.** Translates to `--marketplace-path claude=PATH` with a stderr warning. Will be removed in v0.15 (see #1318). |
3942
| `--legacy-skill-paths` | off | Bundle skills under per-client paths (e.g. `.cursor/skills/`) instead of the converged `.agents/skills/`. Compatibility flag. |
4043
| `--target`, `-t VALUE` | auto-detect | **Deprecated.** Recorded as informational `pack.target` metadata only; ignored by `apm install`. Will be removed in a future release. |
4144

@@ -54,6 +57,15 @@ apm pack --format apm -o ./dist # legacy APM bundle layout
5457
```bash
5558
apm pack
5659
apm pack --offline --dry-run
60+
61+
# Build only Claude format, output as JSON for CI:
62+
apm pack --marketplace=claude --json
63+
64+
# Override codex output path:
65+
apm pack --marketplace-path codex=./dist/codex-marketplace.json
66+
67+
# Build all formats, preview paths:
68+
apm pack --marketplace=all --json | jq -r '.marketplace.outputs[].path'
5769
```
5870

5971
### Both artifacts in one run
@@ -67,11 +79,10 @@ apm pack --archive --offline
6779

6880
```yaml
6981
marketplace:
70-
outputs: [claude, codex]
71-
claude:
72-
output: ./build/claude-marketplace.json
73-
codex:
74-
output: ./build/codex-marketplace.json
82+
outputs:
83+
claude: {}
84+
codex:
85+
path: ./build/codex-marketplace.json
7586
```
7687
7788
### Preview without writing
@@ -124,7 +135,8 @@ Configure marketplace artifact paths in `apm.yml`: `marketplace.claude.output` c
124135
- **Empty bundle warning.** If no files match (e.g. nothing was installed yet), `apm pack` emits a warning and exits `0` with an empty bundle. Verbose mode prints a hint to run `apm install` first.
125136
- **Share line.** On success, `apm pack` prints `Share with: apm install <bundle-path>` so the produced bundle is immediately copy-pasteable.
126137
- **Marketplace fallback.** With no `marketplace:` block in `apm.yml`, a legacy `marketplace.yml` file is read with a deprecation warning. Both files present is a hard error.
127-
- **Marketplace outputs.** `marketplace.outputs` defaults to `[claude]`. Add `codex` to also write `.agents/plugins/marketplace.json`; when selected, each package must define `category`.
138+
- **Marketplace outputs.** Configure via `marketplace.outputs` map (keyed by format). Claude is included by default. The legacy list form (`outputs: [claude]`) still parses with a deprecation warning. Use `--marketplace=` to filter which formats are built in a given invocation.
139+
- **JSON mode.** `--json` makes `apm pack` machine-friendly: stdout is a single JSON object, all human-readable logs move to stderr. Combine with `--marketplace=` for selective CI matrix builds.
128140

129141
## Exit codes
130142

src/apm_cli/commands/pack.py

Lines changed: 139 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
"""Click commands for ``apm pack`` and ``apm unpack``."""
22

3+
import json as json_mod
34
import sys
45
from pathlib import Path
56

@@ -14,6 +15,7 @@
1415
)
1516
from ..core.command_logger import CommandLogger
1617
from ..core.target_detection import TargetParamType
18+
from ..utils.console import set_console_stderr
1719

1820
_PACK_HELP = """\
1921
Pack distributable artifacts from your APM project.
@@ -52,6 +54,21 @@
5254
"""
5355

5456

57+
def _emit_json_error_or_raise(ctx, json_output: bool, code: str, message: str):
58+
"""Emit a JSON error envelope to stdout or raise ClickException."""
59+
if json_output:
60+
from ..marketplace.builder import BuildReport
61+
62+
click.echo(
63+
json_mod.dumps(
64+
BuildReport.failure_to_json_dict(errors=[{"code": code, "message": message}])
65+
)
66+
)
67+
ctx.exit(1)
68+
else:
69+
raise click.ClickException(message)
70+
71+
5572
@click.command(name="pack", help=_PACK_HELP)
5673
@click.option(
5774
"--format",
@@ -104,11 +121,38 @@
104121
"marketplace_output",
105122
type=click.Path(),
106123
default=None,
124+
hidden=True,
125+
help=("[Deprecated] Override Claude output path. Use --marketplace-path claude=PATH instead."),
126+
)
127+
@click.option(
128+
"-m",
129+
"--marketplace",
130+
"marketplace_filter",
131+
type=str,
132+
default=None,
133+
help=(
134+
"Comma-separated marketplace outputs to build (e.g. 'claude,codex'). "
135+
"Use 'all' for every configured output, 'none' to skip marketplace. "
136+
"Default: build all configured outputs."
137+
),
138+
)
139+
@click.option(
140+
"--marketplace-path",
141+
"marketplace_path_overrides",
142+
type=str,
143+
multiple=True,
107144
help=(
108-
"Marketplace legacy compatibility: override only the Claude/Anthropic "
109-
"output path. Prefer marketplace.claude.output in apm.yml."
145+
"Override output path for a format: FORMAT=PATH (repeatable). "
146+
"Example: --marketplace-path claude=dist/marketplace.json"
110147
),
111148
)
149+
@click.option(
150+
"--json",
151+
"json_output",
152+
is_flag=True,
153+
default=False,
154+
help="Emit machine-readable JSON to stdout; logs go to stderr.",
155+
)
112156
@click.option(
113157
"--legacy-skill-paths",
114158
"legacy_skill_paths",
@@ -133,10 +177,79 @@ def pack_cmd(
133177
offline,
134178
include_prerelease,
135179
marketplace_output,
180+
marketplace_filter,
181+
marketplace_path_overrides,
182+
json_output,
136183
legacy_skill_paths,
137184
):
138185
"""Pack APM artifacts: bundle and/or marketplace.json."""
186+
from ..marketplace.output_profiles import known_output_names
187+
from ..utils.path_security import validate_path_segments
188+
189+
# -- Stream discipline: under --json, route ALL output to stderr --
190+
if json_output:
191+
set_console_stderr(True)
192+
139193
logger = CommandLogger("pack", verbose=verbose, dry_run=dry_run)
194+
195+
# -- Deprecation: --marketplace-output → --marketplace-path claude=PATH --
196+
if marketplace_output is not None:
197+
translated = f"--marketplace-path claude={marketplace_output}"
198+
click.echo(
199+
f"Warning: --marketplace-output is deprecated and will be removed in v0.15. "
200+
f"Use {translated} instead.",
201+
err=True,
202+
)
203+
marketplace_path_overrides = (
204+
*marketplace_path_overrides,
205+
f"claude={marketplace_output}",
206+
)
207+
marketplace_output = None
208+
209+
# -- Parse --marketplace-path overrides --
210+
path_overrides: dict[str, str] = {}
211+
for override in marketplace_path_overrides:
212+
if "=" not in override:
213+
msg = f"--marketplace-path must be FORMAT=PATH, got: {override!r}"
214+
_emit_json_error_or_raise(ctx, json_output, "cli_error", msg)
215+
return
216+
fmt_name, path_val = override.split("=", 1)
217+
fmt_name = fmt_name.strip()
218+
path_val = path_val.strip()
219+
if fmt_name not in known_output_names():
220+
msg = (
221+
f"Unknown marketplace format '{fmt_name}' in --marketplace-path. "
222+
f"Known formats: {', '.join(sorted(known_output_names()))}"
223+
)
224+
_emit_json_error_or_raise(ctx, json_output, "unknown_format", msg)
225+
return
226+
# Security: validate path to prevent traversal attacks
227+
try:
228+
validate_path_segments(path_val, context="--marketplace-path", allow_current_dir=True)
229+
except Exception as exc:
230+
_emit_json_error_or_raise(ctx, json_output, "path_error", str(exc))
231+
return
232+
path_overrides[fmt_name] = path_val
233+
234+
# -- Parse --marketplace filter --
235+
marketplace_formats: tuple[str, ...] | None = None
236+
if marketplace_filter is not None:
237+
if marketplace_filter.strip().lower() == "none":
238+
marketplace_formats = ()
239+
elif marketplace_filter.strip().lower() == "all":
240+
marketplace_formats = None # all configured
241+
else:
242+
requested = [f.strip() for f in marketplace_filter.split(",") if f.strip()]
243+
known = known_output_names()
244+
for r in requested:
245+
if r not in known:
246+
msg = (
247+
f"Unknown marketplace format '{r}' in --marketplace. "
248+
f"Known formats: {', '.join(sorted(known))}"
249+
)
250+
_emit_json_error_or_raise(ctx, json_output, "unknown_format", msg)
251+
return
252+
marketplace_formats = tuple(requested)
140253
project_root = Path(".").resolve()
141254
# Issue #1207 D1: when --target is not given, detect the project's
142255
# actual target so the embedded ``pack.target`` reflects what was
@@ -169,15 +282,37 @@ def pack_cmd(
169282
bundle_force=force,
170283
marketplace_offline=offline,
171284
marketplace_include_prerelease=include_prerelease,
172-
marketplace_output=Path(marketplace_output) if marketplace_output else None,
285+
marketplace_output=None,
286+
marketplace_formats=marketplace_formats,
287+
marketplace_path_overrides=path_overrides if path_overrides else None,
173288
dry_run=dry_run,
174289
verbose=verbose,
175290
)
176291

177292
try:
178293
result = BuildOrchestrator().run(options, logger=logger)
179294
except BuildError as exc:
180-
raise click.ClickException(str(exc)) # noqa: B904
295+
_emit_json_error_or_raise(ctx, json_output, "build_error", str(exc))
296+
return
297+
298+
# -- JSON output mode: consistent envelope --
299+
if json_output:
300+
envelope = {
301+
"ok": True,
302+
"dry_run": dry_run,
303+
"warnings": [],
304+
"errors": [],
305+
"marketplace": {"outputs": []},
306+
"bundle": None,
307+
}
308+
for sub in result.producer_results:
309+
if sub.kind is OutputKind.MARKETPLACE and sub.payload is not None:
310+
payload = sub.payload.to_json_dict()
311+
envelope["warnings"] = payload.get("warnings", [])
312+
envelope["marketplace"] = payload.get("marketplace", {"outputs": []})
313+
break
314+
click.echo(json_mod.dumps(envelope, indent=2))
315+
return
181316

182317
for sub in result.producer_results:
183318
if sub.kind is OutputKind.BUNDLE:

src/apm_cli/core/build_orchestrator.py

Lines changed: 19 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,8 @@ class BuildOptions:
4646
marketplace_offline: bool = False
4747
marketplace_include_prerelease: bool = False
4848
marketplace_output: Path | None = None
49+
marketplace_formats: tuple[str, ...] | None = None
50+
marketplace_path_overrides: dict[str, str] | None = None
4951
# Common options
5052
dry_run: bool = False
5153
verbose: bool = False
@@ -182,7 +184,13 @@ def _warn(msg: str) -> None:
182184
resolve_result = None
183185
output_reports = []
184186
outputs: list[Path] = []
185-
for output_name in config.outputs:
187+
188+
# Apply --marketplace filter: skip outputs not in the requested set
189+
active_outputs = list(config.outputs)
190+
if options.marketplace_formats is not None:
191+
active_outputs = [o for o in active_outputs if o in options.marketplace_formats]
192+
193+
for output_name in active_outputs:
186194
profile = MARKETPLACE_OUTPUTS.get(output_name)
187195
if profile is None:
188196
valid_targets = ", ".join(sorted(MARKETPLACE_OUTPUTS))
@@ -198,7 +206,16 @@ def _warn(msg: str) -> None:
198206
configured_output_value = getattr(config, profile.config_attr).output
199207
configured_output = Path(configured_output_value)
200208
output_path = project_root / configured_output
201-
if profile.supports_cli_output_override and options.marketplace_output is not None:
209+
210+
# Apply --marketplace-path override
211+
if (
212+
options.marketplace_path_overrides
213+
and output_name in options.marketplace_path_overrides
214+
):
215+
output_path = project_root / options.marketplace_path_overrides[output_name]
216+
elif (
217+
profile.supports_cli_output_override and options.marketplace_output is not None
218+
):
202219
output_path = options.marketplace_output
203220

204221
output_report = builder.write_output(

0 commit comments

Comments
 (0)