Cesium Widget full screen does not behave properly in Jupyter

[iMurfyD]

🐞 Problem

Cesium widget has a full-scree button in the lower right hand corner. Pressing it should make the window go full screen, however it has a different behavior. It seems that it makes the entire Jupyter notebook full screen and sets the background outside of the notebook to be the same as a the widget. Screenshot below:

🖥 Please paste the output of following commands
Observed on both Chrome Version 80.0.3987.163 (Official Build) (64-bit) and Opera v67.0.3575.115 on Windows 10 Version 1909.

• conda info -a (only if you have conda)

(poliastroprereqs-nopresintalled) C:\Users\IMDes>conda info -a

     active environment : poliastroprereqs-nopresintalled
    active env location : C:\Users\IMDes\Anaconda3\envs\poliastroprereqs-nopresintalled
            shell level : 1
       user config file : C:\Users\IMDes\.condarc
 populated config files : C:\Users\IMDes\.condarc
          conda version : 4.8.2
    conda-build version : 3.18.9
         python version : 3.7.4.final.0
       virtual packages : __cuda=10.2
       base environment : C:\Users\IMDes\Anaconda3  (writable)
           channel URLs : https://repo.anaconda.com/pkgs/main/win-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/win-64
                          https://repo.anaconda.com/pkgs/r/noarch
                          https://repo.anaconda.com/pkgs/msys2/win-64
                          https://repo.anaconda.com/pkgs/msys2/noarch
          package cache : C:\Users\IMDes\Anaconda3\pkgs
                          C:\Users\IMDes\.conda\pkgs
                          C:\Users\IMDes\AppData\Local\conda\conda\pkgs
       envs directories : C:\Users\IMDes\Anaconda3\envs
                          C:\Users\IMDes\.conda\envs
                          C:\Users\IMDes\AppData\Local\conda\conda\envs
               platform : win-64
             user-agent : conda/4.8.2 requests/2.22.0 CPython/3.7.4 Windows/10 Windows/10.0.18362
          administrator : False
             netrc file : None
           offline mode : False

# conda environments:
#
base                     C:\Users\IMDes\Anaconda3
calc.py                  C:\Users\IMDes\Anaconda3\envs\calc.py
lowthrustsim             C:\Users\IMDes\Anaconda3\envs\lowthrustsim
plasmapydev              C:\Users\IMDes\Anaconda3\envs\plasmapydev
poliastropreinstalled     C:\Users\IMDes\Anaconda3\envs\poliastropreinstalled
poliastroprereqs-nopresintalled  *  C:\Users\IMDes\Anaconda3\envs\poliastroprereqs-nopresintalled

sys.version: 3.7.4 (default, Aug  9 2019, 18:34:13) [...
sys.prefix: C:\Users\IMDes\Anaconda3
sys.executable: C:\Users\IMDes\Anaconda3\python.exe
conda location: C:\Users\IMDes\Anaconda3\lib\site-packages\conda
conda-build: C:\Users\IMDes\Anaconda3\Scripts\conda-build.exe
conda-convert: C:\Users\IMDes\Anaconda3\Scripts\conda-convert.exe
conda-debug: C:\Users\IMDes\Anaconda3\Scripts\conda-debug.exe
conda-develop: C:\Users\IMDes\Anaconda3\Scripts\conda-develop.exe
conda-env: C:\Users\IMDes\Anaconda3\Scripts\conda-env.exe
conda-index: C:\Users\IMDes\Anaconda3\Scripts\conda-index.exe
conda-inspect: C:\Users\IMDes\Anaconda3\Scripts\conda-inspect.exe
conda-metapackage: C:\Users\IMDes\Anaconda3\Scripts\conda-metapackage.exe
conda-render: C:\Users\IMDes\Anaconda3\Scripts\conda-render.exe
conda-server: C:\Users\IMDes\Anaconda3\Scripts\conda-server.exe
conda-skeleton: C:\Users\IMDes\Anaconda3\Scripts\conda-skeleton.exe
conda-verify: C:\Users\IMDes\Anaconda3\Scripts\conda-verify.exe
user site dirs:

AF_PATH: C:\Program Files\ArrayFire\v3
CIO_TEST: <not set>
CONDA_DEFAULT_ENV: poliastroprereqs-nopresintalled
CONDA_EXE: C:\Users\IMDes\Anaconda3\condabin\..\Scripts\conda.exe
CONDA_EXES: "C:\Users\IMDes\Anaconda3\condabin\..\Scripts\conda.exe"
CONDA_PREFIX: C:\Users\IMDes\Anaconda3\envs\poliastroprereqs-nopresintalled
CONDA_PROMPT_MODIFIER: (poliastroprereqs-nopresintalled)
CONDA_PYTHON_EXE: C:\Users\IMDes\Anaconda3\python.exe
CONDA_ROOT: C:\Users\IMDes\Anaconda3
CONDA_SHLVL: 1
CUDA_PATH: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.2
GOPATH: C:\Users\IMDes\go
HOMEPATH: \Users\IMDes
NVTOOLSEXT_PATH: C:\Program Files\NVIDIA Corporation\NvToolsExt\
PATH: C:\Users\IMDes\Anaconda3;C:\Users\IMDes\Anaconda3\Library\mingw-w64\bin;C:\Users\IMDes\Anaconda3\Library\usr\bin;C:\Users\IMDes\Anaconda3\Library\bin;C:\Users\IMDes\Anaconda3\Scripts;C:\Users\IMDes\Anaconda3\bin;C:\Users\IMDes\Anaconda3\envs\poliastroprereqs-nopresintalled;C:\Users\IMDes\Anaconda3\envs\poliastroprereqs-nopresintalled\Library\mingw-w64\bin;C:\Users\IMDes\Anaconda3\envs\poliastroprereqs-nopresintalled\Library\usr\bin;C:\Users\IMDes\Anaconda3\envs\poliastroprereqs-nopresintalled\Library\bin;C:\Users\IMDes\Anaconda3\envs\poliastroprereqs-nopresintalled\Scripts;C:\Users\IMDes\Anaconda3\envs\poliastroprereqs-nopresintalled\bin;C:\Users\IMDes\Anaconda3\condabin;C:\Users\IMDes\Anaconda3;C:\Users\IMDes\Anaconda3\Library\mingw-w64\bin;C:\Users\IMDes\Anaconda3\Library\usr\bin;C:\Users\IMDes\Anaconda3\Library\bin;C:\Users\IMDes\Anaconda3\Scripts;C:\Python27;C:\Python27\Scripts;C:\Program Files\Haskell\bin;C:\Program Files\Haskell Platform\8.6.5\lib\extralibs\bin;C:\Program Files\Haskell Platform\8.6.5\bin;C:\Program Files (x86)\Common Files\Oracle\Java\javapath;C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.2\bin;C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.2\libnvvp;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\iCLS;C:\Program Files\Intel\Intel(R) Management Engine Components\iCLS;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\IPT;C:\Program Files\Intel\Intel(R) Management Engine Components\IPT;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0;C:\Program Files (x86)\Wolfram Research\WolframScript;C:\Program Files\Intel\WiFi\bin;C:\Program Files\Common Files\Intel\WirelessCommon;C:\WINDOWS\System32\OpenSSH;C:\Program Files\MATLAB\R2019b\bin;C:\TDM-GCC-64\bin;C:\Program Files\Microsoft SQL Server\120\Tools\Binn;C:\Program Files\PuTTY;C:\Program Files\NVIDIA Corporation\Nsight Compute 2019.5.0;C:\Program Files\NVIDIA Corporation\NVIDIA NvDLISR;C:\Program Files\Haskell Platform\8.6.5\mingw\bin;C:\Program Files\nodejs;C:\ProgramData\chocolatey\bin;C:\Program Files\Git\cmd;C:\Program Files\Git\mingw64\bin;C:\Program Files\Git\usr\bin;C:\Program Files\CMake\bin;C:\Go\bin;C:\Gcc64\bin;C:\Users\IMDes\AppData\Roaming\cabal\bin;C:\Users\IMDes\AppData\Roaming\local\bin;C:\Users\IMDes\.cargo\bin;C:\Users\IMDes\AppData\Local\Microsoft\WindowsApps;C:\Users\IMDes\AppData\Local\Programs\Microsoft VS Code\bin;C:\Users\IMDes\AppData\Local\GitHubDesktop\bin;C:\Program Files\ArrayFire\v3\lib;C:\Users\IMDes\AppData\Local\Programs\MiKTeX 2.9\miktex\bin\x64;C:\Program Files\JetBrains\PyCharm 2019.3.1\bin;.;C:\Users\IMDes\AppData\Roaming\npm;C:\Users\IMDes\AppData\Local\Box\Box Edit;C:\Users\IMDes\go\bin
PSMODULEPATH: C:\Program Files\WindowsPowerShell\Modules;C:\WINDOWS\system32\WindowsPowerShell\v1.0\Modules
REQUESTS_CA_BUNDLE: <not set>
SSL_CERT_FILE: <not set>
VBOX_MSI_INSTALL_PATH: C:\Program Files\Oracle\VirtualBox\

• conda list (only if you have conda)

(poliastroprereqs-nopresintalled) C:\Users\IMDes>conda list
# packages in environment at C:\Users\IMDes\Anaconda3\envs\poliastroprereqs-nopresintalled:
#
# Name                    Version                   Build  Channel
appdirs                   1.4.3                    pypi_0    pypi
asn1crypto                1.3.0                    py38_0
astropy                   4.0                      pypi_0    pypi
astroquery                0.4                      pypi_0    pypi
atomicwrites              1.3.0                      py_0
attrs                     19.3.0                     py_0
backcall                  0.1.0                    py38_0
beautifulsoup4            4.8.2                    pypi_0    pypi
blas                      1.0                         mkl
bleach                    3.1.0                      py_0
ca-certificates           2020.1.1                      0
certifi                   2019.11.28               py38_1
cffi                      1.14.0           py38h7a1dbc1_0
chardet                   3.0.4                 py38_1003
colorama                  0.4.3                      py_0
console_shortcut          0.1.1                         4
cryptography              2.8              py38h7a1dbc1_0
cycler                    0.10.0                   py38_0
czml3                     0.5.3                    pypi_0    pypi
decorator                 4.4.2                      py_0
defusedxml                0.6.0                      py_0
distlib                   0.3.0                    pypi_0    pypi
entrypoints               0.3                      py38_0
filelock                  3.0.12                   pypi_0    pypi
freetype                  2.9.1                ha9979f8_1
html5lib                  1.0.1                    pypi_0    pypi
icc_rt                    2019.0.0             h0cc432a_1
icu                       58.2                 ha66f8fd_1
idna                      2.9                        py_1
importlib_metadata        1.5.0                    py38_0
intel-openmp              2020.0                      166
ipykernel                 5.1.4            py38h39e3cac_0
ipython                   7.13.0           py38h5ca1d4c_0
ipython_genutils          0.2.0                    py38_0
jedi                      0.16.0                   py38_0
jinja2                    2.11.1                     py_0
jpeg                      9b                   hb83a4c4_2
json5                     0.9.4                      py_0
jsonschema                3.2.0                    py38_0
jupyter_client            5.3.4                    py38_0
jupyter_core              4.6.1                    py38_0
jupyterlab                1.2.6              pyhf63ae98_0
jupyterlab_server         1.1.0                      py_0
keyring                   21.1.1                   pypi_0    pypi
kiwisolver                1.0.1            py38ha925a31_0
libpng                    1.6.37               h2a8f88b_0
libsodium                 1.0.16               h9d3ae62_0
llvmlite                  0.31.0           py38ha925a31_0
m2w64-gcc-libgfortran     5.3.0                         6
m2w64-gcc-libs            5.3.0                         7
m2w64-gcc-libs-core       5.3.0                         7
m2w64-gmp                 6.1.0                         2
m2w64-libwinpthread-git   5.0.0.4634.697f757               2
markupsafe                1.1.1            py38he774522_0
matplotlib                3.1.3                    py38_0
matplotlib-base           3.1.3            py38h64f37c6_0
mistune                   0.8.4           py38he774522_1000
mkl                       2020.0                      166
mkl-service               2.3.0            py38hb782905_0
mkl_fft                   1.0.15           py38h14836fe_0
mkl_random                1.1.0            py38hf9181ef_0
more-itertools            8.2.0                      py_0
msys2-conda-epoch         20160418                      1
nbconvert                 5.6.1                    py38_0
nbformat                  5.0.4                      py_0
notebook                  6.0.3                    py38_0
numba                     0.48.0           py38h47e9c7a_0
numpy                     1.18.1                   pypi_0    pypi
numpy-base                1.18.1           py38hc3f5095_1
openssl                   1.1.1f               he774522_0
packaging                 20.3                       py_0
pandas                    1.0.1            py38h47e9c7a_0
pandoc                    2.2.3.2                       0
pandocfilters             1.4.2                    py38_1
parso                     0.6.1                      py_0
pickleshare               0.7.5                 py38_1000
pip                       20.0.2                   py38_1
plotly                    4.5.2                      py_0
pluggy                    0.13.1                   py38_0
prometheus_client         0.7.1                      py_0
prompt_toolkit            3.0.3                      py_0
py                        1.8.1                      py_0
pycparser                 2.20                       py_0
pygments                  2.5.2                      py_0
pyopenssl                 19.1.0                   py38_0
pyparsing                 2.4.6                      py_0
pyqt                      5.9.2            py38ha925a31_4
pyrsistent                0.15.7           py38he774522_0
pysocks                   1.7.1                    py38_0
pytest                    5.3.5                    py38_0
python                    3.8.1           h5fd99cc_8_cpython
python-dateutil           2.8.1                      py_0
pytz                      2019.3                     py_0
pywin32                   227              py38he774522_1
pywin32-ctypes            0.2.0                    pypi_0    pypi
pywinpty                  0.5.7                    py38_0
pyzmq                     18.1.1           py38ha925a31_0
qt                        5.9.7            vc14h73c81de_0
qtconsole                 4.7.1                      py_0
qtpy                      1.9.0                      py_0
requests                  2.23.0                   py38_0
retrying                  1.3.3                      py_2
scipy                     1.4.1            py38h9439919_0
send2trash                1.5.0                    py38_0
setuptools                46.0.0                   py38_0
sip                       4.19.13          py38ha925a31_0
six                       1.14.0                   py38_0
soupsieve                 2.0                      pypi_0    pypi
sqlite                    3.31.1               he774522_0
tbb                       2020.0               h74a9793_0
terminado                 0.8.3                    py38_0
testpath                  0.4.4                      py_0
toml                      0.10.0                   pypi_0    pypi
tornado                   6.0.4            py38he774522_1
tox                       3.14.6                   pypi_0    pypi
traitlets                 4.3.3                    py38_0
urllib3                   1.25.8                   py38_0
vc                        14.1                 h0510ff6_4
virtualenv                20.0.14                  pypi_0    pypi
vs2015_runtime            14.16.27012          hf0eaf9b_1
w3lib                     1.21.0                   pypi_0    pypi
wcwidth                   0.1.8                      py_0
webencodings              0.5.1                    py38_1
wheel                     0.34.2                   py38_0
win_inet_pton             1.1.0                    py38_0
wincertstore              0.2                      py38_0
winpty                    0.4.3                         4
zeromq                    4.3.1                h33f27b4_3
zipp                      2.2.0                      py_0
zlib                      1.2.11               h62dcd97_3

• pip freeze

(poliastroprereqs-nopresintalled) C:\Users\IMDes>pip freeze
appdirs==1.4.3
asn1crypto==1.3.0
astropy==4.0
astroquery==0.4
atomicwrites==1.3.0
attrs==19.3.0
backcall==0.1.0
beautifulsoup4==4.8.2
bleach==3.1.0
certifi==2019.11.28
cffi==1.14.0
chardet==3.0.4
colorama==0.4.3
cryptography==2.8
cycler==0.10.0
czml3==0.5.3
decorator==4.4.2
defusedxml==0.6.0
distlib==0.3.0
entrypoints==0.3
filelock==3.0.12
html5lib==1.0.1
idna==2.9
importlib-metadata==1.5.0
ipykernel==5.1.4
ipython==7.13.0
ipython-genutils==0.2.0
jedi==0.16.0
Jinja2==2.11.1
json5==0.9.4
jsonschema==3.2.0
jupyter-client==5.3.4
jupyter-core==4.6.1
jupyterlab==1.2.6
jupyterlab-server==1.1.0
keyring==21.1.1
kiwisolver==1.0.1
llvmlite==0.31.0
MarkupSafe==1.1.1
matplotlib==3.1.3
mistune==0.8.4
mkl-fft==1.0.15
mkl-random==1.1.0
mkl-service==2.3.0
more-itertools==8.2.0
nbconvert==5.6.1
nbformat==5.0.4
notebook==6.0.3
numba==0.48.0
numpy==1.18.1
packaging==20.3
pandas==1.0.1
pandocfilters==1.4.2
parso==0.6.1
pickleshare==0.7.5
plotly==4.5.2
pluggy==0.13.1
prometheus-client==0.7.1
prompt-toolkit==3.0.3
py==1.8.1
pycparser==2.20
Pygments==2.5.2
pyOpenSSL==19.1.0
pyparsing==2.4.6
pyrsistent==0.15.7
PySocks==1.7.1
pytest==5.3.5
python-dateutil==2.8.1
pytz==2019.3
pywin32==227
pywin32-ctypes==0.2.0
pywinpty==0.5.7
pyzmq==18.1.1
qtconsole==4.7.1
QtPy==1.9.0
requests==2.23.0
retrying==1.3.3
scipy==1.4.1
Send2Trash==1.5.0
sip==4.19.13
six==1.14.0
soupsieve==2.0
terminado==0.8.3
testpath==0.4.4
toml==0.10.0
tornado==6.0.4
tox==3.14.6
traitlets==4.3.3
urllib3==1.25.8
virtualenv==20.0.14
w3lib==1.21.0
wcwidth==0.1.8
webencodings==0.5.1
win-inet-pton==1.1.0
wincertstore==0.2
zipp==2.2.0

Code to reproduce:

from poliastro.examples import molniya, iss
from poliastro.czml.extract_czml import CZMLExtractor
from czml3.widget import CZMLWidget
from czml3.core import Document
start_epoch = iss.epoch
end_epoch = iss.epoch + molniya.period
N = 10

extractor = CZMLExtractor(start_epoch, end_epoch, N)

extractor.add_orbit(molniya, label_text="Molniya")
extractor.add_orbit(iss, label_text="ISS")

with open('play.czml', 'w') as f:
    for l in extractor.packets:
        f.write(l.__str__())
d = extractor.get_document()
CZMLWidget(d)

Another note:
The missing Earth map in the screenshot is fixed by changing to a different map skin. I think this is due to some subscription access or something. That issue is not related to this issue (as far as I know).

[iMurfyD]

Ran in breakpoint on click in Opera debugger found the following when I stepped through clicking the full screen button in the Cesium widget:

Failed to execute 'requestFullscreen' on 'Element': API can only be initiated by a user gesture. Uncaught (in promise) TypeError: fullscreen error

Uncaught (in promise) TypeError: fullscreen error
    at Object.Fullscreen.requestFullscreen (Cesium.js?v=20200412131125:formatted:8346)
    at Cesium.js?v=20200412131125:formatted:113525
    at FullscreenButtonViewModel.t [as _command] (Cesium.js?v=20200412131125:formatted:110778)
    at HTMLButtonElement.<anonymous> (Cesium.js?v=20200412131125:formatted:109417)
    at HTMLButtonElement.dispatch (jquery.min.js:5237)
    at HTMLButtonElement.elemData.handle (jquery.min.js:5044)

Excerpt from Cesium.js starting at line 8345:

    Fullscreen.requestFullscreen = function(e, t) {
        Fullscreen.supportsFullscreen() && e[_names.requestFullscreen]({
            vrDisplay: t
        })
    }

[astrojuanlu]

Addressing the issue part first:

The missing Earth map in the screenshot is fixed by changing to a different map skin. I think this is due to some subscription access or something.

czml3/src/czml3/widget.py

Line 68 in 328608f

cesium_version = attr.ib(default="1.64")

The latest version has to be specified manually. Can you try CZMLWidget(d, cesium_version="1.68")?

It seems that it makes the entire Jupyter notebook full screen and sets the background outside of the notebook to be the same as a the widget.

Confirmed in my computer.

In the Cesium sandcastle, this works normally:

https://sandcastle.cesium.com/

Googling the error throws things like this Stack Overflow answer:

Imagine the security concerns if that were possible. Malicious websites could "Hijack" a less computer literate person's desktop for all kinds of dubious business.

Which is reasonable, but there was a user event: we clicked the button!

Therefore, perhaps something is intercepting the click event. I see two possibilities:

• We are not embedding Cesium correctly
• There is some fault with Jupyter notebook and Cesium can't be embedded correctly

I would start by looking at the first option, perhaps outside of a notebook environment (see for example #53).

[niharsalunke]

Hi @astrojuanlu , I have worked on this issue. I was able to solve the missing earth problem. We need to generate a Cesium ion token before can proceed to display plots. To generate a free token, I followed this link . And after that, I also made a few modifications to the widgets.py file. Everything seems to be working now.

However, with the fullscreen issue, I have worked up to a point where if you perform "cesium_widget_object.fullscreen()" in a python code, you will be able to trigger the fullscreen mode of the canvas.

I was not satisfied here so I tried to write a bit of javascript event listener code in the CESIUM_TPL string of widgets.py and tried to bind the event listeners on the fullscreen button. Now the functions are being triggered correctly. Meaning, if you click on the "fullscreen" button of the CZMLWidget, a JavaScript code responsible to "FULLSCREEN-ize" the canvas gets triggered. But after all this, I got this error: "Failed to execute 'requestFullscreen' on 'Element': API can only be initiated by a user gesture."

This is not easily solvable since it will require external permissions. More here.

To summarize, we have 1 success, 1 partial success, 1 partial failure. So apart from the 1 partial failure, may I create a pull request?

Oh, and BTW I was able to make czml work for more than just the "simple.py" example given in the czml3 file. So I believe this will also cover Poliastro's issue #706.

Thanks a lot for following through. Would love to elaborate more. 😄

Fix for the CZML3 package:

Code Responsilbe for switching to fullscreen

This code piece can be included with the object itself as I mentioned earlier.

Fullscreen Output

[astrojuanlu]

Copy pasting a comment I left in #92:

I know it might be a challenging problem, but I think adding a method just for this is not the right thing to do.

I have no idea about front-end, so I am really not in a position to give a clear path. But I think we should make the fullscreen button just work.

If the button works on Cesium Sandcastle but it doesn't work on Jupyter notebook, then either:

• We are not embedding Cesium correctly (which is very likely - we should be making an extension instead of spitting a blob of HTML, see Support JupyterLab #61)
• Jupyter notebook has some special layout or behavior that blocks this event
• ?

We should build a minimum reproducible example and ask in https://community.cesium.com/ about this issue.

[TJKoury]

Try C137.js.

It's open source now, Apache 2.0 licensed:

Github - c137.js