Move CUDA into an extension

[navidcy]

It'd be nice if we move CUDA functionality into an extension so it's only loaded for CUDA-enabled devices.

@vchuravy made a first attempt to do that at #3066

similar effort but for AMDGPU is at #3475

[navidcy]

Of course, there is a (minor?) downside to this. As things stand now we can do:

using Oceananigans

grid = RectilinearGrid(GPU(), ...)

...

If we move CUDA functionality to an extension we'd have to do:

Of course, there is a (minor?) downside to this. As things stand now we can do:

using Oceananigans
using CUDA

grid = RectilinearGrid(GPU(), ...)

...

[vchuravy]

The benefit is that the user can choose to not install CUDA and just use the CPU implementation of Oceananigans, or use AMDGPU instead of CUDA. So installing Oceanigans will be faster and users on laptops without GPUs won't have to pay for the additional packages

[glwagner]

I think the benefit of faster CPU compilation is worth the price of a using CUDA, not least because we should be concerned about time-to-first-solution and because, though the most important simulations run on GPU, the most number of simulations run on CPU (including a lot of CI). It also seems more forward-looking since there may be many GPU options in the near future.

[ffrancesco94]

I guess this fell through the cracks somehow..?

[navidcy]

Lack of time mostly. Not lack of interest. But if someone leads a PR I can try help!

[ffrancesco94]

I'm not super familiar with the extension system (from the developer side) but let me dig around the documentation and see what I can come up with it!

[glwagner]

There is already a PR open which is very close to completion: #4499