Add Diffusion Graph Network

[rickybalin]

Add diffusion modeling to nekRS-ML in order to generate velocity fields on target graphs/meshes. This work leverages the Dist-GNN model already present.

The diffusion model introduced enough differences to the GNN architecture and the training/inference scripts to warrant the creation of a new directory specific to this model in ./3rd_party/dist-dgn.

The PR also adds an example based on the 3D flow past an extruded cylinder in order to demonstrate the use of the model and validate it on a simple test case.

To Do:

• Add batching with respect to the forward process step r
• Fix bug calling inference routine
• Enable learnable variance in the model (needed for the hybrid loss)
• Add the Hybrid Loss for diffusion models (see the hybrid loss in the dgn4cfd project taken from this paper).
• Investigate importance sampling for diffusion step
• Investigate use of larger batch sizes (32 and above) and larger datasets
• Investigate poor model convergence on ext_cyl case for early diffusion steps
• Implement v and x0 predictions
• Implement min-SNR weighting of the loss
• Validate model on coarse mesh and single GPU
• Add a parallel MSE loss consistent across part boundaries
• Validate model on multiple GPU
• Add transformer based model with element-wise attention
• Expand transformer model as in Appendix A of this paper to add global attention