Notebook introspection

[wstomv]

Is it possible to access the various elements of a notebook from code inside that notebook?

For instance, using the IPython kernel, you can do a very limited kind of introspection using such special variables as In and Out.

It would be useful to have full access to all cells (code, markdown, raw), their sources, all parts of output, metadata, etc.

In particular, In contains a mangled version of the actual input (code cell source). Magics have been replaced by calls to get_ipython().run_cell_magic(). With _i you seem to get an unmangled version of the input of the most recently executed code cell (but is there a mechanism to get this for arbitrary code cells?).

Can this be achieved through JavaScript?

The reason I want this, is that we are developing notebook test frames that contain holes, which will be filled by material from a notebook that students fill. The test frame needs access not only to the execution result (e.g. as left behind in variables), but also to the actually code itself for some static tests that query the code's Abstract Syntax Tree (AST).

[takluyver]

The kernel does not know about the notebook document structure (this is a deliberate design decision). What you're getting with In and other such variables is the execution history.

You can get the raw version of the full execution history from get_ipython().history_manager.input_hist_raw.

You may have more luck with Javascript, because the Javascript does have access to the notebook document structure. Try e.g. Jupyter.notebook.get_cells().

If it's practical to work with the notebook as a file rather than working from inside the notebook's kernel, you can do this readily with Python using the nbformat package.

[wstomv]

We do use nbformat for other processing of notebooks. But it would be good to have dynamic and static test cases together.

Where can one find documentation about how a notebook is stored in the browser (client)? Is there API documentation?

[takluyver]

There's not, unfortunately. We resisted adding it for far too long on the grounds that the API was going to change.

Possibly the best source of information is to find an nbextension that does something similar and look at the code. E.g. the 2to3 converter is an extension that gets the content of code cells and sends them to the kernel, which sounds similar to what you might want.

[lsloan]

I noticed this issue because I would like to use notebook introspection to produce a table of contents at the beginning of the notebook. The TOC would be filled with links to the various headings within the notebook.

[wstomv]

@lsloan There is a separate extension toc2 (part of nbextensions for Jupyter Notebook; not for Jupyter Lab) that does exactly that:

https://jupyter-contrib-nbextensions.readthedocs.io/en/latest/nbextensions/toc2/README.html

[lsloan]

Thanks for pointing that out, @wstomv. However, from the description, it seems to work only in Jupyter Notebooks and nothing else.

I'd like the option of saving the generated TOC as part of the notebook. That is, the first cell calls the code to generate the TOC, which is displayed in that cell's output. When I save and share that notebook online (Bitbucket, GitHub, etc.), the TOC would be displayed from the saved version. Of course, I'd hope that the TOC items would link to the various sections of the document, but that's not critical.

[wstomv]

It does all that, if I recall correctly. On 6 Jun 2018 18:27, Lance E Sloan «UMich» <[email protected]> wrote: Thanks for pointing that out, @wstomv<https://github.com/wstomv>. However, from the description, it seems to work only in Jupyter Notebooks and nothing else. I'd like the option of the generated TOC to be saved as part of the notebook. That is, the first cell calls the code to generate the TOC, which is displayed in that cell's output. When I save and share that notebook online (Bitbucket, GitHub, etc.), the TOC would be displayed from the saved version. Of course, I'd hope that the TOC items would link to the various sections of the document, but that's not critical. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#3101 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AFWec2GHCjgIK5a8Hm6n85W_odKp-v6eks5t6ALRgaJpZM4Qyjhd>.