ENH : adding parallel implementations of all_pairs_ algos

[Schefflera-Arboricola]

Please do "Squash and merge" while merging this PR

Added the following :

• all_pairs_dijkstra
• all_pairs_dijkstra_path_length
• all_pairs_dijkstra_path
• all_pairs_bellman_ford_path_length
• all_pairs_shortest_path
• all_pairs_shortest_path_length - no speedups(the standard sequential implementation is already really fast, and even when the graph size was increased till 4000 nodes, there were negligible speedups, and it only took a minute or 2 to generate its heatmap)
• all_pairs_node_connectivity¹
• approximate_all_pairs_node_connectivity¹
• all_pairs_all_shortest_paths

[1] here, default chunking means having chunks of at most 10 nodes, because otherwise, it was taking too long with bigger node_chunks; and I also reduced the number of nodes while timing, but the speedups are pretty consistent(i.e. around 3). This heatmap alone took 2.5 hrs, and ran successfully after several tries. Also no heatmap for approximate_all_pairs_node_connectivity because for a random graph, it will result in the same heatmap as all_pairs_node_connectivity

[Schefflera-Arboricola]

Tests / test (macos-latest, 3.11) is running for more than 4 hrs. It is stuck at:

algorithms/centrality/tests/test_group.py ........................       [ 50%]

[dschult]

That happened to me yesterday too. I don’ think it has anything to do with the backend, but I’m not sure.

[dschult]

Where does this PR stand? Is there more to do here?
Do you feel the improvements here are what you expect -- or are there other questions/issues you want to explore with this PR?

[Schefflera-Arboricola]

I think we can merge this. The only algo not showing any speedups is all_pairs_shortest_path_length and its sequential implementation is already really fast. But, we should think about how to handle such functions in the whole of nx-parallel, i.e the functions that don’t show any speedups for the general random graphs we use in the current timing script and/or whose sequential networkx implementation is already really fast(like number_of_isolates, all_pairs_shortest_path_length, global_reaching_centrality and local_reaching_centrality(#44), etc.). For some such algos setting the minimum chunk size to be really large might help(this didn't work for all_pairs_shortest_path_length), but then the input graph should also be really big. Just a thought, but having something like should_run = True only when the graph is really big might be helpful.