Scattergl invisible/transparent points

[Iwo-K]

Hi Everyone,

I am not sure if issue is connected to issue #204: "Can't change number of points displayed in a scattergl", I am sorry for duplicating if this is the case.

I have encountered an unusual behaviour of scattergl (this does not happen with scatter) when called from a dash app. When plotting large number of points (64800 in this example) with many traces (36) the first plot is correct.

However if a plot is updated with a new scattergl object with different traces (e.g 1 or 4 traces) then after coming back to the original 36 traces version some of the point are transparent (points can be detected by hovering), and toggling trace visibility works incorrectly.

Curiously the missing points are within the first trace (1/36) if the switched back from a figure with single trace or with first four traces (1-4/36) if switched back from a figure with 4 traces. If one tries toggling traces afterwards the points are shifted.

I apologize if the explanation is not clear. The following code should reproduce this behaviour. If you flick back and forth between options in the dropdown menu then you should notice the problem.

Thanks a lot for creating such a useful module for python!

Code:

import dash
import dash_core_components as dcc
import dash_html_components as html
import pandas as pd
import plotly.graph_objs as go
import numpy as np

external_stylesheets = ['https://codepen.io/chriddyp/pen/bWLwgP.css']
app = dash.Dash(__name__, external_stylesheets=external_stylesheets)

#Number of points per cluster
n = 1800

#Coordinates
x = np.repeat([1,2,3,4,5,6], n)
x = np.tile(x, 6)
x = x + np.random.normal(0, 0.1, 36*n)

y = np.repeat([1,2,3,4, 5, 6], 6*n)
y = y + np.random.normal(0, 0.1, 36*n)

#dataframe with values by which the data is split
catinfo = pd.DataFrame({'cluster' : np.repeat(list(range(1,37)), n)})
catinfo['random4'] = np.random.choice(np.array(['a', 'b', 'c', 'd']), 36*n)
catinfo['all'] = ['all']*n*36

app.layout = html.Div([
                dcc.Graph(id='maingraph'),
                dcc.Dropdown(
                    id='split_dropdown',
                    options=[{'label' : x, 'value' : x} for x in catinfo.columns.values],
                    value='cluster'
                    )])

@app.callback(dash.dependencies.Output('maingraph', 'figure'),
                [dash.dependencies.Input('split_dropdown', 'value')])
def update_figure(splitbyvalue):

    traces = []
    for i in catinfo[splitbyvalue].unique():

        boolSUB = catinfo[splitbyvalue] == i

        traces.append(go.Scattergl(
            x = x[boolSUB],
            y = y[boolSUB],
            mode = 'markers',
            name = str(i))
                    )

    return {
        'data': traces,
        'layout': {
            'hovermode' : 'closest',
            }
        }

if __name__ == '__main__':
    app.run_server(debug=True)

Software:
Platform: MacOS 10.12.16
Browsers tested: Firefox/Safari/Chrome
Python: 3.6.3
Plotly: 3.5.0
Dash: 0.35.1
dash-core-components: 0.42.0
dash-html-components: 0.13.4
dash-renderer: 0.16.1
Flask: 1.0.2

[chriddyp]

Thanks for reporting! Ive transferred over to the plotly.js repo.

fyi plotly.js - dash-core-components==0.42.0 uses plotly.js 1.43.1

[etpinard]

Could someone try to replicate in JS in a codepen? Thanks for the report.

This sounds similar to #2334

[secretwpn]

@Wintermute12, I've used to have a similar issue using react-plotly.js. In my case some points were invisible, but hovering them would show the hoverbox.

What fixed it for me was inspecting my data and making sure I don't have duplicating items in ids, x, y arrays.
I've had some errors in my logic, leading to adding same points multiple times to the same trace.
Once that was fixed - the issue with disappearing points went away.

[Iwo-K]

@secretwpn Thank you for the suggestion, this sounds very similar to my issue.

@etpinard Unfortunately I don't have any experience with JavaScript. Reproducing the plot was not a problem, but I don't know how to properly implement the variable choice (e.g. a dropdown) and replotting in JavaScript. If anyone would like to follow up: the plot (with option to plot either 36 or 4 clusters, as above) can be found at https://codepen.io/anon/pen/NovOad

There is also a possibility that the issue arises when data is passed from Python objects into JavaScript (perhaps something gets duplicated as @secretwpn pointed out). In which case it may not be possible to replicate this behaviour in JavaScript alone.

[etpinard]

Another report from #3533 (comment)

I don't have a reproducible example of this but a user showed me a scattergl trace with lots of datapoints not displaying markers in Firefox (although Chrome was fine). This was inside a Dash app running dash-core v0.43.0 so plotly.js' version was v.1.44.1. Although the points were invisible, the hoverlabels were properly rendered.

[Donnyvdm]

I don't have a reproducible example right now, but I would like to report that I'm experiencing the same issue, though for me it occured in Chrome too.
When plotting my full dataset (10797 points) the plot is fine, but if I then limit my datapoints using a dash dropdown callback, most (but not all) points are hidden.
If I set a filter as a default, the plot works fine too and I can switch the value of my filter. When I then unselect the filter, the full dataset shows fine, but after that applying any filter shows this bug again.
I'd like to add that when my plot is in this bugged state and I make a selection (e.g. with the box-select tool), the points inside the selection do show up. As reported, the hover labels do work in all cases, also for the hidden points (see redacted screenshot).
Also like previous users reported, if I switch to a regular scatter plot everything works as expected.

Platform: MacOS 10.13.6
dash=0.35.1
dash-core-components==0.42.1
dash-renderer==0.14.1
plotly==3.2.1 (python)

[etpinard]

Can anyone affected by this bug share (at least) their x / y data arrays here? CSV would be fine.

Thank you all!

[Donnyvdm]

Thank you for looking into this, please find a minimal working example below. To see the problem, please run below dash app and click the 'break!' button.
To help with debugging: changing one of the following fixes the plot for me:

• change size from 10001 to 10000 or below
• remove the "mode='markers'" from the plot
• change go.Scattergl to go.Scatter

import dash
import dash_core_components as dcc
import dash_html_components as html
import numpy as np
import pandas as pd
import plotly.graph_objs as go

np.random.seed = 42
size = 10001
df = pd.DataFrame({'x': np.linspace(0, 1, size),
                   'to_filter': np.random.choice([True, False], size)})

app = dash.Dash(__name__)
app.layout = html.Div(children=[
    html.Button('Break!', id='button'),
    dcc.Graph(id='broken-scatter')
])

@app.callback(
    dash.dependencies.Output('broken-scatter', 'figure'),
    [dash.dependencies.Input('button', 'n_clicks')])
def plot_scatter(n_clicks):
    to_plot = df[df.to_filter] if n_clicks else df
    return {'data': [go.Scattergl(x=to_plot.x, y=to_plot.x, mode='markers')]}

if __name__ == '__main__':
    app.run_server(debug=True)

Example is tested under the latest version:
Platform: macOS 10.13.6
Browsers tested: Google Chrome, Firefox, Safari
dash==0.36.0
dash-core-components==0.43.0
dash-renderer==0.18.0
plotly==3.6.1

EDIT: For the non-python/numpy people here: x = y = range between 0 and 1 divided equally in 10001 values. See attached CSV as per request.
testdata.txt

[etpinard]

#3413 (comment) in a codepen: https://codepen.io/etpinard/pen/rPRwOy?editors=1010

Thanks @Donnyvdm !

This is in fact a duplicate of #2334

We'll continue the discussion over there. Thank you for your help!