Move from nbsphinx to myst-nb

[OriolAbril]

My plan is to first make this change, which will work once the widget issue in
pymc-devs/pymc-examples#83 is fixed, @chiral-carbon will
send a PR for this at some point tomorrow. Moving from nbsphinx to myst-nb will allow us
to use all sphinx features from within markdown. Therefore, if we write a glossary,
we'll be able to link to terms defined there with {term}`RV` from within any markdown cell
in notebooks, we'll be able to link to respective api docs with {func}`pymc3.to_inference_data` or {class}`pymc3.Normal`, to define anchors with
(anchor_name)= and reference them with {ref}`anchor_name` and so on.

After that, I will also change the theme to pydata-sphinx-theme, like pandas and ArviZ. Note that
both changes are independent and we could do only one without the other, which is why I'm doing two
separate PRs. I think it would be great to change the appearance of the docs with 4.0, but
it would still be good to have myst already in v3 so we can start using it in pymc-examples without
breaking anything. We can then cherry pick this one onto v4 but not port the theme one.

cc @martinacantaro @ricardoV94

• what are the (breaking) changes that this PR makes? None
• important background, or details about the implementation see above
• linting/style checks have been run
• right before it's ready to merge, mention the PR in the RELEASE-NOTES.md

[OriolAbril]

How do I get pre-commit to be happy with me changing the development dependencies?

[OriolAbril]

PR is now ready to test! As this has significant changes to doc building process, which is not covered anywhere by CI, it has to be tested locally by building the documentation and making sure everything works. This only modifies how docs are build, so they should look the same and have the same pages as the published website (except for differences between v3 and v4 like theano renamed to aesara and so on).

I have tested locally and everything works, but someone else should do too and then merge

[OriolAbril]

@ricardoV94 shoud be ready to merge now.

For everyone else, it works and notebooks are rendered, but some of the css is not compatible with the current theme, so images are not shrinked and look way to large and other technically correct yet unpleasant rendering. I propose to not cherry pick that onto v3 unless necessary. i.e. we start using cool sphinx features in ~2-3 weeks but still have to make a 3.11.4 release 2 months from now. If that were to happen we'd have to build the docs from an old commit in pymc-examples or use myst and get ugly notebooks.

[fonnesbeck]

Looking at this now.

[fonnesbeck]

It builds without error for me, but the examples and tutorials pages are blank, so I'm doing something wrong. I've cloned the pymc-examples submodule and run fast_build_docs.sh.

[OriolAbril]

It builds without error for me, but the examples and tutorials pages are blank, so I'm doing something wrong. I've cloned the pymc-examples submodule and run fast_build_docs.sh

Have you opened the html file or have you use python -m http.server to see local version the website on your browser? The tutorial and example gallery are javascript, so you need to start the server from docs/source/_build/html/, opening the html files as a static website doesn't work.

We went over the whole doc building process yesterday with @ricardoV94, who might be able to document it?

[fonnesbeck]

Yeah, this is running the webserver. The console is showing some file errors:

::1 - - [10/Jun/2021 09:38:46] code 404, message File not found
::1 - - [10/Jun/2021 09:38:46] "GET /_static/gallery_tutorials_contents.js HTTP/1.1" 404 -
::1 - - [10/Jun/2021 09:38:46] "GET /_static/PyMC3.ico HTTP/1.1" 200 -
::1 - - [10/Jun/2021 09:38:46] code 404, message File not found
::1 - - [10/Jun/2021 09:38:46] "GET /_static/gallery_tutorials_contents.js HTTP/1.1" 404 -

The docs build completed without error, but I will go back and scan the output for any clues.

[fonnesbeck]

There are some warnings that may be relevant.

/Users/cfonnesbeck/GitHub/pymc3/docs/source/pymc-examples/examples/case_studies/LKJ.ipynb:70008: WARNING: 'myst' reference target not found: ../api/distributions/multivariate.rst
/Users/cfonnesbeck/GitHub/pymc3/docs/source/pymc-examples/examples/case_studies/LKJ.ipynb:110002: WARNING: 'myst' reference target not found: ../api/math.rst

I must have missed a step. Maybe I should wait for the doc build process documentation that you mentioned. Here's what I did:

1. installed myst_nb, sphinx and dependencies
2. updated the pymc-examples submodule
3. ran the fast_build_docs.sh script
4. started webserver from the html dir

[OriolAbril]

These steps should be enough 🤔

Inside the html folder, do you have a pymc-examples/examples folder with the same structure as the pymc example repo but with html files instead of ipynb?

These 2 sphinx warnings look like an error generated when restructuring the notebooks. They all used to be in the notebooks folder so a path like ../api/rest_of_path pointed to the right locations of other doc files, after reorganizing they don't anymore. With myst we can stop using paths and use anchors (like one would do in rST files) to avoid these issues if we ever reorganize the file tree.

I have no idea what to make of the other server warnings though

[fonnesbeck]

Yes, the pymc-examples/examples folder is there and populated with html files.

The _static subdir seems to be missing a whole bunch of files (mostly .css) that the page is looking for.

[OriolAbril]

What do you have inside the _build/html/_static folder?

[fonnesbeck]

[OriolAbril]

Something went wrong at some point, I see different files and many more images

[fonnesbeck]

This occurs both on Linux and Mac for me, so it is not unique to one platform/environment.

[fonnesbeck]

Solved the issue by running the script a second time.

[michaelosthege]

Curious to see how this myst-nb will play out in practice!

[michaelosthege]

@OriolAbril delete the docs branch?