Ensure that the default workflow tests the source extension installation

[krassowski]

I learned over at https://github.com/krassowski/jupyterlab-lsp/issues/518 that extension authors should not only test the installation of federated extension, but the source version too and do it in a clean environment (without the dev dependencies installed). I think that it is important to have this as a step in CI. I think that the default workflow does not do that:

extension-cookiecutter-ts/.github/workflows/main.yml

Lines 54 to 66 in a161a92

	- name: Create pure frontend extension
	run: |
	cookiecutter . --no-input
	cd myextension
	jlpm install && jlpm run eslint:check
	pip install -e .
	jupyter labextension list 2>&1 | grep -ie "myextension.*OK"
	python -m jupyterlab.browser_check
	jlpm run install:extension
	jupyter labextension build .
	jupyter labextension uninstall myextension
	pip uninstall -y myextension jupyterlab

(jlpm run install:extension is an alias for jupyter labextension develop --overwrite .); even if it does (I may be misunderstanding how the cookicutter influences these steps), it would do so in an environment that is tainted with the dev dependencies. Having a clean smoke test, separated out from the build step as we implemented here, using separate jobs and artifact sharing action might be the best practice for preventing issues like this.

Should such an approach be included in the default workflow?

[jtpio]

Thanks @krassowski.

Should such an approach be included in the default workflow?

Since source extensions are still supported in JupyterLab it would indeed make sense to have a check on CI to do that.

[bollwyvl]

gah... this well is deep. for it to actually be testing where problems often occur for users, it would have to run:

• on all the major platforms (but mostly windows)
• try different shells
• use/not use conda
• have crazy extensions for npm.(exe|cmd|bat)
• have non-unicode characters in and spaces in the path
• have really long paths
• have UNC network paths
• in OneDrive/box/whatever shared folders
• during a "headless" build run by the API, with a bunch of kernels open to simulate open file handle issues
• ...

i feel like if we go down this road, supports_source_install would need to be an opt-in, because on a number of extensions we straight up can't support it, as there are just too many/big/weird files for underpowered windows CI machines to handle.