Notebooks - HTML elements (widgets) in output overflow & get displayed under the editor

[DonJayamanne]

In the Python extension when enabling IPyWidgets:

• Some widgest such as beakerx (have calendar controls)
• These controls are floating HTML elements, however when they overflow the bounds of the widget, they end up getting displayed behind the monaco editor.
• Seems like a zindex issue (monaco editors) are dislayed over the HTML content, hence get displayed over outputs.

@rebornix /cc

[rebornix]

We can't fix this issue completely as long as the outputs and cell input (editors) are rendered in different frames, an output renderer can use position: absolute to put a DOM element anywhere it wants. However we can improve the experience case by case.

For example, on Google Colab, we can use native calendar widget without any issue (the output is rendered in an iframe, and the calendar is not being covered by anything)

The way it works is

• an output is rendered in a dedicated iframe, so its inner native element can escape the iframe and also be on top of editors (while in our architecture, the webview/iframe is a single giant one so it's always below editors)
• every iframe load shim code for ipywidgets, so the widget can talk to the kernel to get/save states
• catch is if you have two widgets linked with the same data entry, you will notice latency when updating one, as it will send data to the kernel first and then communicate back to the other widget

We can probably do the same thing, if an output doesn't need to access outputs' DOM elements directly. Every output can be rendered in its own iframe, and everyone of them talks to the kernel to sync widget states.

Short summary:

• If an output's renderer doesn't need access to the DOM elements out of the output container, we can render it in a separate iframe. For ipywidgets, it syncs states directly through kernel, every iframe will have its own widget manager
• If an output's renderer tries to absolute position a DOM element, it needs to be aware of the output container, otherwise it won't behave properly (same in Google Colab or any other frontend which separates editor and output in different frames)

@DonJayamanne

[rebornix]

Well I spoke a bit early, Chromium is smart enough to render native inputs, even if the zindex of iframe is smaller than others

[rebornix]

Did more investigation into widget rendering in multiple different jupyter frontends, including Jupyter Lab, Notebook and Google Colab, I found that rendering floating properly is a non-trivial problem for renderers. You can easily find bugs with third party renderers in Jupyter Notebook

To position a custom floating widget, there are three approaches (from easy to complex):

1. Absolutely position it next to the pointer/mouse. It can easily be covered by other DOM elements (like above), or cut off by the boundary of the notebook UI (imaging put a jupyter notebook in an iframe)
2. Position it inside the visual viewport.
3. Position it inside the output container.

Approach 2 is mostly good enough but it requires us (VS Code notebook) to render output in dedicated iframe (meaning the visual viewport === output container). Approach 3 is the best but also requires more smartness in renderers.

[rebornix]

Closing as we have come to conclusion that there aren't any we can do to support floating DOMs if they are not native floating elements (input), we would need to contribute to the community to help polish the experience.