Skip to content

[code-infra] Modify publish script to filter packages#1236

Merged
brijeshb42 merged 2 commits intomasterfrom
publish-exclude
Mar 26, 2026
Merged

[code-infra] Modify publish script to filter packages#1236
brijeshb42 merged 2 commits intomasterfrom
publish-exclude

Conversation

@brijeshb42
Copy link
Copy Markdown
Contributor

@brijeshb42 brijeshb42 commented Mar 23, 2026

being published. The flag is same as pnpm -F and does the same thing
internally to call pnpm with the flag to get the list of packages to
publish.

This change is done to allow use to do two phase publishing in core
where we want one workflow job to publish only internal packages and the
other job to exclude such packages.

Refactored the code related to transitive dependency resolution and moved
it to pnpm utils module.

Tested locally with dry-run.

Related to - mui/material-ui#47952

@brijeshb42 brijeshb42 requested a review from a team March 23, 2026 11:33
@brijeshb42 brijeshb42 added the scope: code-infra Involves the code-infra product (https://www.notion.so/mui-org/5562c14178aa42af97bc1fa5114000cd). label Mar 23, 2026
@Janpot
Copy link
Copy Markdown
Member

Janpot commented Mar 23, 2026

Alternatively, what if we support the pnpm -F filter expressions in getWorkspacePackages as an option. Then place all public packages under ./packages/, and all internal packages under ./packages-internal/ and filter on those folders? It feels like a more sustainable approach than having to fiddle with inclusion/exclusion lists in both publish commands. You can just put all public ones under one folder and all internal ones in another folder, nice and clear what the boundaries are. We already started creating these folders, this would enforce it.

@brijeshb42 brijeshb42 changed the title [code-infra] Modify publish script to exclude packages [code-infra] Modify publish script to filter packages Mar 23, 2026
@mui-bot
Copy link
Copy Markdown

mui-bot commented Mar 23, 2026

Bundle size report

Bundle Parsed size Gzip size
@base-ui/react 0B(0.00%) 0B(0.00%)
@mui/x-charts-pro 0B(0.00%) 0B(0.00%)

Details of bundle changes


Check out the code infra dashboard for more information about this PR.

type: 'string',
array: true,
description:
'pnpm --filter expressions to select which packages to publish (same syntax as `pnpm -F`). When omitted, all public packages are published.',
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is that a union or intersection of the filter? How does it work in pnpm?

Copy link
Copy Markdown
Contributor Author

@brijeshb42 brijeshb42 Mar 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Separate filters mean packages that satisfy any of the patterns. So union.

https://pnpm.io/filtering#multiplicity

const publishNames = new Set(packagesToPublish.map((pkg) => pkg.name));
const depsOnly = await getWorkspacePackages({
publicOnly: true,
filter: filter.map((f) => `${f}^...`),
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking, we should probably widen this check. e.g. packages should not depend on workspace: packages that are not in the list returned from getWorkspacePackages, also when no filter is specified. e.g. what if it has a workspace dependency on a private package. I also don't think this trick would work correctly if the filter is something like foo-*.

I think we could probable generalize and reuse getTransitiveDependencies on all the packages, then check if it returned any package not in the list.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the check is reused in multiple places, perhaps we should create a function for it as well:

function getExtraneousWorkspaceDependency(packageList: string[]): string[]

or something, which returns a list of workspace packages that anything in packageList transitively depends on without including it.

@brijeshb42 brijeshb42 force-pushed the publish-exclude branch 4 times, most recently from a4aa4c0 to 64fbd90 Compare March 24, 2026 16:29
@brijeshb42 brijeshb42 requested review from a team and Janpot March 24, 2026 16:32

/**
* @typedef {Object} GetTransitiveDependenciesOptions
* @property {boolean} [includeDev=true] - Whether to include devDependencies in the traversal
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we also add includeProd then?

Comment thread packages/code-infra/src/utils/pnpm.mjs Outdated
* roots depend on it.
*
* @param {string[]} packageNames - Package names to start the traversal from
* @param {Map<string, string>} workspacePathByName - Map of workspace package name to directory path
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this go in the options?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could go either way.

Comment thread packages/code-infra/src/utils/pnpm.mjs Outdated
* @param {GetTransitiveDependenciesOptions} [options]
* @returns {Promise<Set<string>>} All reachable workspace package names, including the input packages themselves
*/
export async function getTransitiveDependencies(packageNames, workspacePathByName, options = {}) {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

workspacePathByName feels very leaky. One thing we could do is create a function that fetches this, but caches the results. Maybe something for follow-up

* Sets of package names that violate the requirements. Both sets are empty when valid.
*/
export async function validatePublishDependencies(packages) {
const allWorkspacePackages = await getWorkspacePackages();
Copy link
Copy Markdown
Member

@Janpot Janpot Mar 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Calling getWorkspacePackages again while it was already called to obtain packages is a bit wasteful. I'm not sure how we can fix this though without creating leaky abstractions. Perhaps some of the commands need to cache their results? We can do it in follow-up

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah. This here is being done to get all public and private packages so we can track if something is not included in the publish list.
I think its fine though since the script code-infra publish or code-infra publish-canary is only called once in a release trigger. Can be looked into it separately.

Copy link
Copy Markdown
Member

@Janpot Janpot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving, but from a maintainability POV, would prefer to keep validation logic internal to validatePublishDependencies

Comment thread packages/code-infra/src/cli/cmdPublish.mjs Outdated
This also includes a refactoring of existing transitive dependency
checker in cmdNetlifyIgnore.

It handles checking if the filtered publishable packages have any
dependency that is not published or is private.
@brijeshb42 brijeshb42 enabled auto-merge (squash) March 26, 2026 07:52
@brijeshb42 brijeshb42 merged commit b989d00 into master Mar 26, 2026
11 of 12 checks passed
@brijeshb42 brijeshb42 deleted the publish-exclude branch March 26, 2026 07:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

scope: code-infra Involves the code-infra product (https://www.notion.so/mui-org/5562c14178aa42af97bc1fa5114000cd).

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants