Dashboarding

[jasongrout]

To keep this archived somewhere, here are the notes from our discussion on dashboarding at the ipywidgets workshop in Paris in Jan 2018 (see OpenDreamKit/OpenDreamKit#246, and https://www.lri.fr/etherpad/p/jupyter-widgets for the notes).

We can get an overview of current solutions to displaying the widgets on a web page, as sort of a web app. For example, Pascal's ipywidget-server, https://github.com/oschuett/appmode, or #1862

Pascal's solution: a separate server that is instantiated with the code to be run, a specific kernel https://github.com/pbugnion/ipywidgets_server

https://github.com/oschuett/appmode - strip out the classic notebook UI

#1862 (not finished) - read a notebook, execute all cells from the browser, display specific cell outputs

IBM Jupyter Dashboard server - easy drag-and-drop layout of cell outputs in a grid, one-click publish, secure execution of the notebook by not accepting code execution from the browser
https://github.com/jupyter-attic/dashboards_server and https://github.com/jupyter/dashboards and https://github.com/jupyter-attic/dashboards_bundlers

Do we reuse kernels (e.g., you give a link to someone with a widget from a specific running kernel), or start a new kernel for every dashboard view

• Different kinds of uses cases: dashboards as "isolated" kernel execution (each dashboard has own kernel) vs dashboard as "broadcast" enviornment - by interacting with one I am affecting all others.

Can we expose a layout widget that lets you drag and drop to layout things, and then save the layout.

Integration with kernel gateway / remote kernel manager?

It's not obvious we need a live separate kernel. For some usecases, you may want to execute the code in-process (e.g., flask, django).

Should the layout depend on widgets and the kernel? Or should the layout be independent of widgets/kernel.

• Making two layout systems can be confusing (a frontend UI layout, and a widget layout)
• We could make a layout widget that lets you drag and drop children to lay out them. Then save the state of the widget.

Do we force the authoring environment to be a notebook? An authoring environment like shiny?

Perhaps we just have some very specific predefined layouts that can just be populated with children.

If we were able to publish a binder-ready repo for a dashboard, that could take care of a deployment usecase.

Needs:
- Subject-matter experts (know python, not js) easily expose their code/explorations as a webapp to stakeholders with no programming skills, so they don't want any notion of a notebook, code, etc. Code might be running internal software stacks, etc., so we don't have to distribute software, just expose a GUI to use the tools.
- Students exploring learning materials (similar to above usecase)
- Should be usable from different language kernels, such as R
- Layout should be doable programmatically
- E-posters at an academic conference (three-column layout)
- Include markdown/text/html, possibly with variable substitutions
- Security - not being able to execute arbitrary code from the browser - needs to be an option
- Programmatic layouts should use standard layout protocols, e.g., flexbox, grid, etc.
- Other styling, such as colors, fonts, etc.

Here is another issue talking about dashboarding: jupyterlab/jupyterlab#1640

[jasongrout]

Actually, reopening this for further discussion too.

[maartenbreddels]

I hould also mention https://github.com/maartenbreddels/flask-ipywidgets which is a proof of concept of inprocess kernel just doing Comms for flask, which should be able to port to Django and tornado. I think this, ipywidgets-server and appmode could live in 1 repo since they will share a lot of code.

[ixxie]

We have been working on an ipywidgets dashboard as a way to quickly get an internal GUI tool going; our setup (outlined here ) is a Jupyterhub server with Jupyterlab with a shared group owned directory symlinked into users' notebooks directories (I believe this is quite a common approach to sharing notebooks).

So far out major pain-points are:

1. Having to restart the kernel and run all cells to render the dashboard; this has to be done on every page load (at least I haven't found a way around it).
2. It seems that the notebook saves some kind of state to disk, because concurrent use of the notebook by multiple users triggers the 'revert file' dialogue box.
3. We cannot seem to find a reliable and elegant way to share notebook URLs; this seems to work well for the classic tree links but with lab the interface saves some kind of viewport state which gets loaded.

It seems to me - with my addmitedly limited knowledge of Jupyter ecosystem internals - that a sensible and relatively easy to implement solution would be to create an .ipyapp format which is mostly just like a normal .ipynb notebook but:

1. Has only one input cell which is always hidden and run when the page is loaded;*
2. Saves no state to disk, other than whatever the user explicitly tells the dashboard to save;
3. Runs in 'appmode' (i.e. strips out the notebook UI) and has an affiliated reliable URL that launches the app, like the appmode extension;**
4. Gets added to a new 'apps' section of the launcher (with an optional user specified icon).

Does that make any sense?

---

* I guess this means the notebook would be 'trusted'?
** appmode doesn't seem to run correctly on our stack; we get 404 errors.

[ixxie]

I would be happy to try and contribute that feature with some guidance; I have some experience with Python but not so much Java/Typescript.

[oschuett]

Hi @ixxie, it seems that all your pain-points would be addressed by adding support for JupyterLab to appmode. This is something I've been putting off because the API was still in flux. I'll now put it back on my TODO list. However, it'll probably take me a few weeks to carve out the time. So, feel free to take a first shot at it yourself.

[ixxie]

@oschuett I wouldn't know where to begin, but since this is a nice way to get introduced to Jupyter/Lab extension writing, perhaps you can sketch out a rough implementation plan and I could maybe give it a shot over the next weeks?

But I wonder whether appmode - one updated to support JupyterLab - could actually become a core feature of JupyterLab? It seems like something that would make sense to integrate into the core offering, and would make it easy to then specify a .ipyapp format spec.

[ixxie]

On another note, it seems our team's developers are now leaning towards adopting Dash as our dashboarding framework since it seems at least to allow us more flexibility (it binds the React framework into Python); it seems Dash works within Jupyter notebooks as well.

This means that once we get appmode going, we can deploy Dash apps. What if it is equally easy to deploy Flask or Django apps via JupyterLab? With some kind of deployment framework (Docker/Kubernetes integration) we would have something ready to deploy Python apps to production.

I would love to contribute in this area, but it would take me some time as I have limited knowledge of the necessary prerequisites. Maybe starting with an attempt to port appmode to JupyterLab would be the place to start :)

[BexDunn]

interested to hear where this goes