Semantics of iopub messages after an execute_reply or idle status message

[jasongrout]

When threads are created that give output after the main thread is finished, some issues come up about how to treat iopub messages after an execute_reply message. The spec is ambiguous on this point.

The most relevant paragraph of the spec (http://jupyter-client.readthedocs.io/en/latest/messaging.html#execute) says:

Upon completion of the execution request, the kernel always sends a reply, with a status code indicating what happened and additional data depending on the outcome. See below for the possible return codes and associated data.

In the classic notebook, iopub messages after an execute_reply message are processed as normal (i.e., output callbacks still seem to be called so output still shows up in a cell).

In JupyterLab, iopub messages after an execute_reply message are handled until the cell has a new execution. (CC @blink1073; relevant links: https://github.com/jupyterlab/jupyterlab/blob/cec1d9ea9eaae73bba03f6a761b5a2c86b7f1907/packages/outputarea/src/widget.ts#L476 is where code execution is done, which preserves the future until the output has another execution)

I think we should have an official interpretation in the spec of how iopub messages should be handled after execute_reply messages are sent.

Another question is how iopub messages are treated when they are in response to a comm message. It appears in JLab that the iopub callback mechanism is disposed when the idle status is received (https://github.com/jupyterlab/jupyterlab/blob/50be3c38fdf5d82efb06d415dc8ede85aed17edd/packages/services/src/kernel/comm.ts#L140 - disposeOnDone, the third argument, is true). In the classic notebook, it appears that the iopub is still handled when the comm send is finished and the status is idle.

@rgbkrk - how does nteract handle this output from either a comm message or a execute request after the execute_reply/status idle messages?

CC @nleclercq - who was discussing this issue on the jupyter-widgets/Lobby gitter.

[jasongrout]

CC @sccolbert - who may have opinions on the issue as well

[rgbkrk]

The way it works right now in both notebook and nteract is how I expect it to behave -- that all of the messages that are output (except for execute_result) can still come after the status message is idle.

The primary reasoning for this is to be able to handle code that is running on a background thread. In Python, it's code that runs on a thread in a ThreadPool. Within JavaScript, it's literally anything that happens in the background:

fs.readFile('test.txt', (err, data) => {
  console.log(data); // stream message, should be routed to the parent message id
})
"yep" // comes back as execute_result

Same thing for how I want / expect it to work for Scala, especially when working with Spark or other cluster computing tools.

[blink1073]

Looking at the notebook implementation, the most recently executed message is always kept alive. We can do the same in JupyterLab.

[jasongrout]

We can do the same in JupyterLab.

We do essentially do this in jlab for execute request (or even better - the most recently executed per cell), but not for comm messages - which is where we were seeing the issue.

[jasongrout]

Relevant code from @nleclercq:

To test comm message output -

import time
import threading
from IPython.display import display
import ipywidgets as ipw

sleep_time = 4.

out = ipw.Output(layout=dict(border='1px solid red'))
def work(out):
    pb = ipw.FloatProgress(value=0.0, min=0.0, max=1.0, step=0.01)
    with out:
        display(pb)
    for i in range(10):
        with out:
            print('work loop: iteration #{}'.format(i))
        time.sleep(1.)
        pb.value = (i + 1) / 10.
    pb.close()
    #out.clear_output()

def spawn_clicked(b):
    thread = threading.Thread(target=work, args=[out])
    thread.start()
    time.sleep(sleep_time)
    print('spawn_clicked done')

but = ipw.Button(description="spawn worker thread")
but.on_click(spawn_clicked)
display(but)

display(out)
print('main thread done')

For the execute_result message case:

import time
import threading
from IPython.display import display

sleep_time = 4.

def work():
    for i in range(10):
        print('iteration %d'%i)
        time.sleep(1)

def spawn():
    thread = threading.Thread(target=work)
    thread.start()
    time.sleep(sleep_time)

spawn()
print('main thread done\n')

[blink1073]

We have no way to associate a comm future with a cell. I think we should unambiguously say that message replies should meet the spec, but the last message can continue to output in order to handle out of band outputs.

[jasongrout]

We have no way to associate a comm future with a cell.

The key point is that comm futures are automatically disposed when "done". Funny enough, there is an argument for the option at https://github.com/jupyterlab/jupyterlab/blob/50be3c38fdf5d82efb06d415dc8ede85aed17edd/packages/services/src/kernel/comm.ts#L125, but it's not used and hardcoded to dispose on done at https://github.com/jupyterlab/jupyterlab/blob/50be3c38fdf5d82efb06d415dc8ede85aed17edd/packages/services/src/kernel/comm.ts#L140. Also, there's no disposeOnDone option for the futures returned from the open or close methods on a Comm.

[blink1073]

We can turn off the auto-dispose for kernel execution on a cell because we know we are storing the future and disposing of it later. There is no similar controller logic for the comm messages.

[blink1073]

What I'm suggesting is that we remove the distinction and do what the classic notebook is doing entirely.

[jasongrout]

There is no similar controller logic for the comm messages.

My point was that the caller of the comm messages can opt into managing the lifecycle - but then they are responsible for calling dispose when appropriate.

[jasongrout]

But back to the issue at hand - the message spec - I think in the vast majority of cases these days, the idle status message should come after the last output message. That's great because it allows the frontend to clean up message handlers.

We've talked before about letting the backend notify the frontends when there possibly could be messages beyond the status idle message (perhaps a field in the status idle message). I think this is a nice balance between being able to clean up things on the frontend side, and keeping things around.

Another option is to have an explicit end-of-output message.

[minrk]

IPython can never know whether async output may or may not come, so if allowing output after idle is opt-in, then IPython would have to opt-in for every request.

I think what makes the most practical sense is to extend the current logic of execute to comm messages. Preserve the handlers for everything that’s still on the page, and only dispose of them when the destination gets a new msg_id or the target is itself destroyed. That’s easy for execute, because it limits the number of live handlers to the number of cells on the page. For widgets, I would assume that the limited to the number of widgets as well.

[rgbkrk]

My argument for messages coming after idle are going to be for Python that does any background asynchronous work as well as continuing to extend us beyond our Python history and allow for interpreters that have async as part of their core.

For nteract's handling, there is only one event listener for the iopub socket. The way it gets composed is using Observables. All observers are cleaned up when a cell is executed again, the kernel is interrupted, or other kernel lifecycle events. The memory footprint for these is minimal compared to what else is on the page (codemirror, etc.).

How many output handlers do you really have per cell?

Another option is to have an explicit end-of-output message.

This may or may not be intractable.

[jasongrout]

What I'm suggesting is that we remove the distinction and do what the classic notebook is doing entirely.

So have "disposeOnDone" actually mean "disposeWhenNextMessageIsDone"?

I think this works for things like the IPython kernel that already confuse thread output (i.e., there is only a single global output parent message header, and if the main thread switches to a new parent message header, the threads automatically redirect their output to the new parent message - essentially the parent header is dynamically scoped instead of lexically scoped). It doesn't work if a kernel implements threads in a way that has multiple threads sending messages with different past parent message headers.