Add `visualize` command #262

hendrikvanantwerpen · 2023-04-19T11:01:05Z

◀️ #261	#264 ▶️

This PR adds a CLI command to generate a HTML visualization from the database.

This has limited use, as the visualizations quickly break down when they contain a lot of data. But because it is useful for debugging, I still think it is useful to have it. And if a better performing visualization is developed one day, it would be more useful.

This also updates the d3 and d3-dag JS files to newer versions.

github-actions · 2023-04-19T11:18:16Z

Performance Summary

Comparing base c5332d8 with head c49414d on microsoft/[email protected]. For details see workflow artifacts. Note that performance is tested on the last commits with changes in stack-graphs, not on every commit.

Before

--------------------------------------------------------------------------------
Command:            base/target/release/tree-sitter-stack-graphs-typescript index -D base.sqlite --max-file-time=30 --hide-error-details -- test/src/compiler
Massif arguments:   --massif-out-file=perf.out
ms_print arguments: --x=72 --y=12 base-perf-results/perf.out
--------------------------------------------------------------------------------


    GB
2.300^                                   ##                                   
     |                                   #                                    
     |                                   #                                    
     |                                   #                                    
     |                                   #                                    
     |           @                       #                                    
     |     ::    @                      :#                 :                  
     |     :     @        :             :#                 :                  
     |     :     @        :          :  :#                 :                  
     |     :    :@        :        :::  :#        :        :            :     
     |     :    :@   :  : : ::     : :  :#        :        :         :: ::    
     |    @:   ::@:  :: ::: ::  :  : : ::#  :     : :::    :     ::  :::::    
   0 +----------------------------------------------------------------------->Gi
     0                                                                   228.5

After

--------------------------------------------------------------------------------
Command:            head/target/release/tree-sitter-stack-graphs-typescript index -D head.sqlite --max-file-time=30 --hide-error-details -- test/src/compiler
Massif arguments:   --massif-out-file=perf.out
ms_print arguments: --x=72 --y=12 head-perf-results/perf.out
--------------------------------------------------------------------------------


    GB
2.300^                                   #                                    
     |                                   #                                    
     |                                   #                                    
     |                                   #                                    
     |                                   #                                    
     |                                   #                                    
     |                                   #           :    :                   
     |                                   #           :    :                   
     |          :                   :    #           :    :                   
     |     :    :    :   :          :  ::#          ::    :     :     : :     
     |     :    :    : ::: :::     :: :: #          :::   :     :   @::::     
     |     :   @: :::: ::: :::     :: :: #:::   ::: :::  @:   : :   @:::: @   
   0 +----------------------------------------------------------------------->Gi
     0                                                                   236.8

github-actions · 2023-04-19T11:36:24Z

Performance Summary

Comparing base c5332d8 with head 1e5871a on microsoft/[email protected]. For details see workflow artifacts. Note that performance is tested on the last commits with changes in stack-graphs, not on every commit.

Before

--------------------------------------------------------------------------------
Command:            base/target/release/tree-sitter-stack-graphs-typescript index -D base.sqlite --max-file-time=30 --hide-error-details -- test/src/compiler
Massif arguments:   --massif-out-file=perf.out
ms_print arguments: --x=72 --y=12 base-perf-results/perf.out
--------------------------------------------------------------------------------


    GB
2.300^                                   ##                                   
     |                                   #                                    
     |                                   #                                    
     |                                   #                                    
     |                                   #                                    
     |                    @              #                                    
     |     ::             @             :#                                    
     |     :    :         @             :#                                    
     |     :    :         @  ::     :   :#                                    
     |     :    :        :@  :      :   :#          :     :            ::     
     |     :    :      :::@  :     :::  :#          :     :         :: :@     
     |    @@   ::::  : :::@:::  :  ::::::#      :   :     :: :  :  :::::@ :   
   0 +----------------------------------------------------------------------->Gi
     0                                                                   233.2

After

--------------------------------------------------------------------------------
Command:            head/target/release/tree-sitter-stack-graphs-typescript index -D head.sqlite --max-file-time=30 --hide-error-details -- test/src/compiler
Massif arguments:   --massif-out-file=perf.out
ms_print arguments: --x=72 --y=12 head-perf-results/perf.out
--------------------------------------------------------------------------------


    GB
1.234^                                              #                         
     |      @@                                      #     :                   
     |      @                                       #     :                   
     |      @                                       #     :                   
     |      @                                       #     :                   
     |      @    @                  :               #     :                   
     |      @    @                  :         :    :#     :             :     
     |      @    @                  :         :    :#     :           @ :     
     |      @    @       @  :      ::         :    :#:    :          :@::     
     |      @    @ : :   @  :      :: :       : : ::#:   ::  @  :  : :@:::    
     |    @@@    @ : ::  @  :      :: :@     :: : ::#:   ::  @  :  :::@::::   
     |    @ @ : :@:: ::  @  :   :  :: :@ ::  :: : ::#: :::: :@::: ::::@:::@: :
   0 +----------------------------------------------------------------------->Gi
     0                                                                   211.9

github-actions · 2023-04-19T11:39:23Z

Performance Summary

Comparing base c5332d8 with head 7b353b0 on microsoft/[email protected]. For details see workflow artifacts. Note that performance is tested on the last commits with changes in stack-graphs, not on every commit.

Before

--------------------------------------------------------------------------------
Command:            base/target/release/tree-sitter-stack-graphs-typescript index -D base.sqlite --max-file-time=30 --hide-error-details -- test/src/compiler
Massif arguments:   --massif-out-file=perf.out
ms_print arguments: --x=72 --y=12 base-perf-results/perf.out
--------------------------------------------------------------------------------


    GB
2.300^                                   ##                                   
     |                                   #                                    
     |                                   #                                    
     |                                   #                                    
     |                                   #                                    
     |           @                       #                                    
     |     ::    @                      :#                 :                  
     |     :     @        :             :#                 :                  
     |     :     @        :          :  :#                 :                  
     |     :    :@        :        :::  :#        :        :            :     
     |     :    :@   :  : : ::     : :  :#        :        :         :: ::    
     |    @:   ::@:  :: ::: ::  :  : : ::#  :     : :::    :     ::  :::::    
   0 +----------------------------------------------------------------------->Gi
     0                                                                   228.5

After

--------------------------------------------------------------------------------
Command:            head/target/release/tree-sitter-stack-graphs-typescript index -D head.sqlite --max-file-time=30 --hide-error-details -- test/src/compiler
Massif arguments:   --massif-out-file=perf.out
ms_print arguments: --x=72 --y=12 head-perf-results/perf.out
--------------------------------------------------------------------------------


    GB
2.300^                                   #                                    
     |                                   #                                    
     |                                   #                                    
     |                                   #                                    
     |                                   #                                    
     |          @        @@              #                                    
     |     :    @        @              :#                :                   
     |     :    @        @              :#                :                   
     |     :    @        @   @     ::   :#                :                   
     |     :    @   :    @   @     : :  :#:         :     :           : @     
     |     :    @   :: : @  :@     : : ::#:         : :   :         ::::@:    
     |   @@:   @@:: :: ::@ @:@  :  : : ::#:: :  :  ::@:   :  :  :  :::::@:: : 
   0 +----------------------------------------------------------------------->Gi
     0                                                                   238.3

github-actions · 2023-04-19T15:03:15Z

Performance Summary

Comparing base a9af2f2 with head 303bb12 on microsoft/[email protected]. For details see workflow artifacts. Note that performance is tested on the last commits with changes in stack-graphs, not on every commit.

Before

--------------------------------------------------------------------------------
Command:            base/target/release/tree-sitter-stack-graphs-typescript index -D base.sqlite --max-file-time=30 --hide-error-details -- test/src/compiler
Massif arguments:   --massif-out-file=perf.out
ms_print arguments: --x=72 --y=12 base-perf-results/perf.out
--------------------------------------------------------------------------------


    GB
2.300^                                   ##                                   
     |                                   #                                    
     |                                   #                                    
     |                                   #                                    
     |                                   #                                    
     |                                   #                                    
     |     :                             #           :                        
     |     :    :                        #           :                        
     |     :    :            :      :    #           :                        
     |     :    :        :   :      :    #        :  :    :          :  :     
     |     :    :      ::: :::     @::   #        :  :@   :          :: ::    
     |    ::    :::  : ::: :::     @:::@@#  :   : : ::@   :     :::@::: :::   
   0 +----------------------------------------------------------------------->Gi
     0                                                                   230.5

After

--------------------------------------------------------------------------------
Command:            head/target/release/tree-sitter-stack-graphs-typescript index -D head.sqlite --max-file-time=30 --hide-error-details -- test/src/compiler
Massif arguments:   --massif-out-file=perf.out
ms_print arguments: --x=72 --y=12 head-perf-results/perf.out
--------------------------------------------------------------------------------


    GB
1.234^                                               #                        
     |      @                                        #    @                   
     |      @                                        #    @                   
     |      @                                        #    @                   
     |      @    @                  :                #    @                   
     |      @    @                  :                #    @                   
     |      @    @   :              :   :           :#    @     :       :     
     |      @    @   :              :   :           :#    @     :     : :     
     |      @    @   :      :       :   :           :#    @     :    :@::     
     |      @    @ @ :      :       :   :  :     : ::#   :@:   ::  : :@::     
     |    :@@   :@@@::  :: :::  :  ::  :: ::     : ::#   :@:   ::  : :@::::   
     |    :@@   :@@@::: :: :::: ::::: ::: ::  :: : ::# : :@:::::@  :::@::::: :
   0 +----------------------------------------------------------------------->Gi
     0                                                                   216.5

github-actions · 2023-05-05T08:58:09Z

Performance Summary

Comparing base e0bf953 with head b1a2d37 on microsoft/[email protected]. For details see workflow artifacts. Note that performance is tested on the last commits with changes in stack-graphs, not on every commit.

Before

--------------------------------------------------------------------------------
Command:            base/target/release/tree-sitter-stack-graphs-typescript index -D base.sqlite --max-file-time=30 --hide-error-details -- test/src/compiler
Massif arguments:   --massif-out-file=perf.out
ms_print arguments: --x=72 --y=12 base-perf-results/perf.out
--------------------------------------------------------------------------------


    GB
2.300^                                   #                                    
     |                                   #                                    
     |                                   #                                    
     |                                   #                                    
     |                                   #                                    
     |           @                       #                                    
     |      @    @                      :#           :    :                   
     |      @    @                      :#           :    :                   
     |      @   :@                  :   :#           :    :                   
     |     :@   :@   :   :          :   :#        :  :    :             :     
     |     :@   :@   : : : :::     ::   :#        :  :    :          ::::     
     |    ::@  ::@@::: ::: :::     :: : :#: :     : :::   @  ::   : ::::: :   
   0 +----------------------------------------------------------------------->Gi
     0                                                                   230.3

After

--------------------------------------------------------------------------------
Command:            head/target/release/tree-sitter-stack-graphs-typescript index -D head.sqlite --max-file-time=30 --hide-error-details -- test/src/compiler
Massif arguments:   --massif-out-file=perf.out
ms_print arguments: --x=72 --y=12 head-perf-results/perf.out
--------------------------------------------------------------------------------


    GB
1.234^                                               #                        
     |      @         :                              #                        
     |      @         :                              #                        
     |      @         :                              #                        
     |      @         :                              #                        
     |      @    :    :             :                #                        
     |     :@    :    :             :   :           :#    :             :     
     |     :@    :    :             :   :        :  :#    :           : :     
     |     :@    :    :  :  :       :  ::        :  :#    :          @:::     
     |     :@    :    : @:  :       :  ::  :  :  : @:#    :  : ::    @::: :   
     |    ::@   ::    : @:  :    :  :  ::  :  :: : @:#    @  : :::  :@::: :   
     |    ::@:  ::::::: @: :: ::::  :  :: :: ::: : @:# :: @ ::::@: ::@:::::@::
   0 +----------------------------------------------------------------------->Gi
     0                                                                   212.5

github-actions · 2023-05-17T12:12:21Z

Performance Summary

Comparing base e0bf953 with head 230c1ae on microsoft/[email protected]. For details see workflow artifacts. Note that performance is tested on the last commits with changes in stack-graphs, not on every commit.

Before

--------------------------------------------------------------------------------
Command:            base/target/release/tree-sitter-stack-graphs-typescript index -D base.sqlite --max-file-time=30 --hide-error-details -- test/src/compiler
Massif arguments:   --massif-out-file=perf.out
ms_print arguments: --x=72 --y=12 base-perf-results/perf.out
--------------------------------------------------------------------------------


    GB
1.234^                                               #                        
     |      @                            :           #    :                   
     |      @                            :           #    :                   
     |      @                            :           #    :                   
     |      @                            :           #    :                   
     |      @   ::                  ::   :           #    :                   
     |      @   :                   ::   :     :    :#    :             :     
     |      @   :        @          ::   :     : :  :#:   :           : :     
     |      @   :        @  :       :: : :     : :  :#:   :          :: :     
     |      @   :    @   @ ::      ::: : : ::  : : ::#:   :     :  : ::::     
     |    ::@   :  ::@  :@ ::@  :  ::::::::::  : : ::#:   :     :@ : ::::     
     |    : @:  :  ::@ ::@ ::@ ::: :::::::::: :: : ::#:: ::@  : :@ ::::@:::  :
   0 +----------------------------------------------------------------------->Gi
     0                                                                   212.0

After

--------------------------------------------------------------------------------
Command:            head/target/release/tree-sitter-stack-graphs-typescript index -D head.sqlite --max-file-time=30 --hide-error-details -- test/src/compiler
Massif arguments:   --massif-out-file=perf.out
ms_print arguments: --x=72 --y=12 head-perf-results/perf.out
--------------------------------------------------------------------------------


    GB
2.300^                                   #                                    
     |                                   #                                    
     |                                   #                                    
     |                                   #                                    
     |                                   #                                    
     |                                   #                                    
     |      @                           :#           :                        
     |      @   :                       :#           :                        
     |      @   :                   @@  :#           :                        
     |      @   :    :   :         :@   :#      :    :          :       :     
     |      @   :    :  :: :       :@   :#   :  :    ::         :   :: :::    
     |    ::@  :: :  :  :: :       :@ :::#   :  :@@ :::       ::: ::::@:::::  
   0 +----------------------------------------------------------------------->Gi
     0                                                                   237.8

rewinfrey · 2023-06-08T02:55:47Z

tree-sitter-stack-graphs/src/cli/visualize.rs

+    Visualizations will only work for very small stack graphs. This command is
+    useful for debugging minimal examples, but running it on any real-world code
+    will most likely result in HTML files that will not load in any browser.
+"#)]


Thank you for including this help text.

hendrikvanantwerpen requested a review from a team as a code owner April 19, 2023 11:01

hendrikvanantwerpen self-assigned this Apr 19, 2023

hendrikvanantwerpen force-pushed the visualize-command branch from 978a52a to 81864d2 Compare April 19, 2023 11:15

hendrikvanantwerpen force-pushed the index-special-files branch from e2eeac6 to 5dd505b Compare April 19, 2023 14:34

hendrikvanantwerpen force-pushed the visualize-command branch from 7b353b0 to 303bb12 Compare April 19, 2023 14:44

hendrikvanantwerpen mentioned this pull request Apr 20, 2023

Use transaction mechanism instead of explicit BEGIN/COMMIT #264

Merged

hendrikvanantwerpen force-pushed the index-special-files branch from 5dd505b to b29da8c Compare May 5, 2023 08:37

hendrikvanantwerpen force-pushed the visualize-command branch from 303bb12 to b1a2d37 Compare May 5, 2023 08:38

hendrikvanantwerpen force-pushed the index-special-files branch from b29da8c to 8ae0463 Compare May 17, 2023 09:32

Add method to load all graphs for a file or directory

f615964

hendrikvanantwerpen force-pushed the visualize-command branch from b1a2d37 to 799e504 Compare May 17, 2023 09:33

hendrikvanantwerpen added 4 commits May 17, 2023 13:55

Add command to visualize graph

235ffa2

Include paths in visualize command

db1ddc9

Add note about limitations

2998319

Update d3 and d3-dag files to new versions

230c1ae

hendrikvanantwerpen force-pushed the visualize-command branch from 35c8747 to 230c1ae Compare May 17, 2023 11:55

hendrikvanantwerpen mentioned this pull request Jun 2, 2023

Ensure special files are also indexed #261

Merged

Base automatically changed from index-special-files to main June 2, 2023 14:39

rewinfrey approved these changes Jun 8, 2023

View reviewed changes

hendrikvanantwerpen merged commit 9de0462 into main Jun 8, 2023

hendrikvanantwerpen deleted the visualize-command branch June 8, 2023 10:53

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add `visualize` command #262

Add `visualize` command #262

Uh oh!

hendrikvanantwerpen commented Apr 19, 2023 •

edited

Loading

Uh oh!

github-actions bot commented Apr 19, 2023

Uh oh!

github-actions bot commented Apr 19, 2023

Uh oh!

github-actions bot commented Apr 19, 2023

Uh oh!

github-actions bot commented Apr 19, 2023

Uh oh!

github-actions bot commented May 5, 2023

Uh oh!

github-actions bot commented May 17, 2023

Uh oh!

rewinfrey Jun 8, 2023

Uh oh!

Uh oh!

Add visualize command #262

Add visualize command #262

Uh oh!

Conversation

hendrikvanantwerpen commented Apr 19, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Apr 19, 2023

Performance Summary

Uh oh!

github-actions bot commented Apr 19, 2023

Performance Summary

Uh oh!

github-actions bot commented Apr 19, 2023

Performance Summary

Uh oh!

github-actions bot commented Apr 19, 2023

Performance Summary

Uh oh!

github-actions bot commented May 5, 2023

Performance Summary

Uh oh!

github-actions bot commented May 17, 2023

Performance Summary

Uh oh!

rewinfrey Jun 8, 2023

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Add `visualize` command #262

Add `visualize` command #262

hendrikvanantwerpen commented Apr 19, 2023 •

edited

Loading