d3 Renderer Cleanup

[benloh]

Overview

The main goal of this work was to refactor the d3-simplenetgraph renderer so that it could reliably render visual data (VDATA). It introduces a number of new managers that transform the working Net.Create network graph data (NCDATA) into visual data (VDATA).

The end result is a new renderer NCGraphRenderer, with a wrapper React Component NCGraph that manages the visual data. This potentially enables us to create multiple renderers in the same window.

Please see User Input Data Flow v2.x diagram for an overview.

Main Features

Data Flow: NCDATA => FILTEREDNCDATA => VDATA

Data is now better differentiated.
NCDATA -- conceptual Net.Create data, including metadata (e.g. degrees, weight)
FILTEREDNCDATA -- a subset of NCDATA, with nodes and edges removed or marked by user-selected filters
VDATA -- the visual data used by the renderer (e.g. size, opacity).

The flow is basically:

1. server.js loads and broadcasts node and edge updates.
2. nc-logic listens to the broadcasts and constructs NCDATA based on them.
3. filter-mgr listens for NCDATA updates and applies any active filters, and then broadcasts FILTEREDNCDATA
4. edge-mgr listens for FILTEREDNCDATA updates and renders it as VDATA. The reason edge-mgr handles this is because we need to simplify the edges before rendering them. See #285
5. NCGraph listens for updates to VDATA and passes it on to NCGraphRenderer
6. NCGraphRenderer draws the graph.

NCGraph and NCGraphRenderer

• NCGraph replaces NetGraph. It is the new wrapper for the renderer, NCGraphRenderer. It decides what data updates to listen to (e.g. VDATA)
• NCGraphRenderer.js replaces d3-simplenetgraph.js
• Both NetGraph and d3-simplenetgraph.js remain in the repo, but are not being used.

SELECTION vs HILITE

We will eventually need to implement a more centralized selection manager, which is outside of the scope of this PR. However, refactoring the renderer broke some of the existing functionality (when editing an edge, you could no longer select a source/target node). So we had to introduce a new hilite-mgr module to differentiate SELECTION (e.g. clicking on a node) from HILITE (e.g. hovering over a node, hovering over a row in the Node Table, nodes found by search, hovering over an autosuggest selection).

Visually differentiating a selection from a highlight is difficult, especially on a colorful graph. So we've added an experimental animated arrows to indicate the currently selected node. We can revise this and try other approaches if this is too busy, but it really does make the currently selected node much more obvious.

• selection-mgr.js is currently an empty placeholder. We anticipate that the selection methods from nc-logic will eventually be moved here.

Additional Features/Fixes

• Improved highlight visuals -- The system now better handles three simultaneous highlight states (search, autosuggested highlight, selected). Prior to this the "selected" and "autosuggested" states were conflated.
• Improved autosuggestion highlights -- The color of the highlight now matches the color of the hover menu to better connect the two (We used to re-use the "selected node" color for the highlight, which is problematic). Also, the graph no longer jumps as you switch between hovers.
• Improved node clicking -- We refined the renderer so that nodes in the graph no longer move around so much as you select and deselect nodes. This makes the system overall feel much more stable.
• Lint Fixes -- With the revised linting rules, most of the existing files needed to be cleaned up. Mostly this involves auto layout rules, but there were a few code changes as well (e.g. removing ReactDOM.findDOMNode).

The introduction of the rudimentary selection manager ended up breaking Edge Editing. So we had to flesh out the selection and highlight managers even though they were not originally a part of this scope. We also added a few new features along the way:

• Node Table Row highlights now also highlights the corresponding node in the graph -- was quick to implement now that we have a good selection manager and renderer (15min).

[daveseah]

Mermaid Diagram Generated by ChatGPT Test

graph LR
  server.js[server.js] -->|Broadcasts| nc-logic[nc-logic]
  nc-logic -->|NCDATA| filter-mgr[filter-mgr]
  filter-mgr -->|FILTEREDNCDATA| edge-mgr[edge-mgr]
  edge-mgr -->|VDATA| NCGraph[NCGraph]
  NCGraph -->|Visual data updates| NCGraphRenderer[NCGraphRenderer]
  hilite-mgr[hilite-mgr] -- SELECTION/HILITE actions --> NCGraph
  hilite-mgr -- SELECTION/HILITE actions --> NCGraphRenderer

Loading

[benloh]

To Test

This was a pretty deep rewrite, but the application should behave like it used to with a few improvements and no regression errors. So all aspects of the system ought to be tested, e.g.

• Create new nodes
• Create new edges
• View Node Table
• View Edge Table
• Use Template Editor to change the source selection and search highlight color.

A specific test of the selection/hilite system:

1. Open a project with a number of nodes and edges
2. Type "a" in the search field (or some letter shared by multiple nodes -- all of the found nodes should be highlighted, the graph might move a little but not too much.
3. Click in the search field to open up the autosuggest highlight and hover over the individual nodes -- the hovered node should be highlighted in gray, while the search matches remain highlighted -- the graph should not shift while you change highlights
4. Click on a node from the popup -- the node should be selected and displayed, the highlight should be the selection color (not the autosuggest color)
5. Click on a node directly in the graph -- the node should be selected
6. Do another search, and hover over the autosuggestions -- the node selected in step 5 should remain selected, in fact you should see the
7. Open a Node Table
8. Hover over a row in the table -- the corresponding node on the graph should be hilited.

[daveseah]

There is a console error about React:

I think this should be cleaned-up as part of a policy of enforcing clean commits. We are doing this with linting code and format, so in the spirit of this we should clean up console output. If not fixed, then suppressed somehow.

CHECKLIST

• There has been no update to React version (v16.4.2 is from 2018)
• HashRouter needs update
• Route needs update
• Router needs update
• Search needs update
• Switch needs update
• Transition needs update
• Autosuggest needs update
• Autowhatever needs update

[daveseah]

Tested quickly:

• create a node
• create a new edge
• search for a node by name
• used edge table to select
• used node table to select
• changed selection color

[daveseah]

Seems to work overall! I can accept it, and then look into the warning part as a separate fixup pass when we address any remaining linting issues and errors as a separate pull request.

[benloh]

Thanks!

There is a console error about React:

I tried to clean up most of these when they occurred in components I was working on. In these cases, it's a little more complex since they're in different packages.