Adds logging [INFO] to TimelineVisualizationCallback and HistoryCallback

[norlandrhagen]

Small PR that:

• Adds info level logging to show the directory that TimelineVisualizationCallback and HistoryCallback write their artifacts to.
• Updates the TimelineVisualizationCallback os.mkdir to the Pathlib mkdir used in the HistoryCallback. This seems to fix the FileNotFoundError in TimelineVisualizationCallback() usage [cubed - cubed/xarray] #544.

example logging:

TimelineVisualizationCallback results saved to directory: history/compute-20240808T135430.657101/
HistoryCallback results saved to directory: history/compute-20240808T135430.657101/

Some remaining Q's:

• The TimelineVisualizationCallback calls an external function (to the class) create_timeline that has args for such as dst=None, which are not a part of TimelineVisualizationCallback. Is this create_timeline function used anywhere else? I ask this b/c the optional dst arg creates a few more path creation options. If it is confined to TimelineVisualizationCallback, the logic could be more similar to HistoryCallback which is a bit cleaner.

• I added info level logging to diagnostics/__init__.py is this OK to set logging here?

[TomNicholas]

Nice! Would this close either of #543 or #544?

[norlandrhagen]

I think it should close both!

[norlandrhagen]

No idea why the Apache Beam tests are failing though 😕

[tomwhite]

Thanks for the PR @norlandrhagen!

• The TimelineVisualizationCallback calls an external function (to the class) create_timeline that has args for such as dst=None, which are not a part of TimelineVisualizationCallback. Is this create_timeline function used anywhere else? I ask this b/c the optional dst arg creates a few more path creation options. If it is confined to TimelineVisualizationCallback, the logic could be more similar to HistoryCallback which is a bit cleaner.

This code was originally based on code from Lithops for producing a timeline image, so some of it may be redundant. Please go ahead and remove code paths that are not used so it's more similar to HistoryCallback.

• I added info level logging to diagnostics/__init__.py is this OK to set logging here?

No, it should go in client code. Perhaps add it to examples/add-random.py and examples/matmul-random.py?

[tomwhite]

No idea why the Apache Beam tests are failing though 😕

Not related to this PR. Opened #549

[norlandrhagen]

• I added info level logging to diagnostics/__init__.py is this OK to set logging here?

No, it should go in client code. Perhaps add it to examples/add-random.py and examples/matmul-random.py?

Do you see any place where logging or optional logging might be useful? Maybe default off, but user enabled --debug flag could output some info about where callback artifacts are written. Also happy to remove it and just mention in the docs of where you can expect them to be written.

ex:

TimelineVisualizationCallback results saved to directory: history/compute-20240809T115943.795332/timeline.svg/
HistoryCallback results saved to directory: history/compute-20240809T115943.795332/

[tomwhite]

It would be useful, but the problem is that if you configure logging then that will interfere with logging that the user has configured (often in non-obvious ways). So I think adding logging to the examples is a good compromise. Also mentioning in the docs.

[norlandrhagen]

Makes sense! Thanks for explaining.

[tomwhite]

Thanks @norlandrhagen! LGTM. I've made one small suggestion. Let me know when you're ready to merge.

[norlandrhagen]

If you're happy with the doc suggestions, then it's ready to merge.