Skip to content

Conversation

@pbrubeck
Copy link
Contributor

@pbrubeck pbrubeck commented Oct 12, 2023


name: "TransferManager: Re-injection of coefficients; no-op for unmodified dat"
description: ''
title: ''
labels: ''
assignees: ''


Description

I'm adding a cache on the arguments of tm.prolong, tm.restrict, tm.inject that holds the dat_versions from the last time the operator was applied on those arguments. The transfer operator will behave as a no-op if the current versions coincide with the ones in the cache.

I also add a coarsen hook at restriction with the injection of all coefficients of the problem at hand. Here we avoid the unnecessary re-injection of coefficients that remain constant.

Associated Pull Requests:

Need to merge this PR first

Fixes Issues:

If issues are fixed by this PR, prepend each of them with the word
"fixes", so they are automatically closed when this PR is merged. For
example "fixes #123, fixes #456".

Checklist for author:

  • I have linted the codebase (make lint in the firedrake source directory).
  • My changes generate no new warnings.
  • All of my functions and classes have appropriate docstrings.
  • I have commented my code where its purpose may be unclear.
  • I have included and updated any relevant documentation.
  • Documentation builds locally (make linkcheck; make html; make latexpdf in firedrake/docs directory)
  • I have added tests specific to the issues fixed in this PR.
  • I have added tests that exercise the new functionality I have introduced
  • Tests pass locally (pytest tests in the firedrake source directory) (useful, but not essential if you don't have suitable hardware).
  • I have performed a self-review of my own code using the below guidelines.

Checklist for reviewer:

  • Docstrings present
  • New tests present
  • Code correctly commented
  • No bad "code smells"
  • No issues in parallel
  • No CI issues (excessive parallelism/memory usage/time/warnings generated)
  • Upstream/dependent branches and PRs are ready

Feel free to add reviewers if you know there is someone who is already aware of this work.

Please open this PR initially as a draft and mark as ready for review once CI tests are passing.

dham
dham previously requested changes Oct 18, 2023
Copy link
Member

@dham dham left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please check for dual safety.

@pbrubeck pbrubeck changed the title TransferManager: no-op for unmodified arguments TransferManager: Re-injection of coefficients; no-op for unmodified dat Oct 25, 2023
Comment on lines 149 to 152
if isinstance(V, firedrake.functionspaceimpl.FiredrakeDualSpace):
V = firedrake.functionspace.DualSpace(mesh, V.ufl_element())
else:
V = firedrake.FunctionSpace(mesh, V.ufl_element())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should add an interface to reconstruct on a new mesh.

@pbrubeck pbrubeck force-pushed the pbrubeck/feature/cache-transfer branch from 0dc8083 to 02ba9d0 Compare November 1, 2023 13:49
Copy link
Contributor

@connorjward connorjward left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This basically looks fine to me.

if problem.Jp is not None:
coefficients = coefficients + problem.Jp.coefficients()
for val in chain(coefficients, (bc.function_arg for bc in problem.bcs)):
if isinstance(val, (function.Function, cofunction.Cofunction)):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a UFL superclass for this that we can use instead right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, ufl.coefficient.BaseCoefficient, but we opted to be more descriptive.

# work right)
for ctx in chain((self.inserted_options(), dmhooks.add_hooks(dm, self, appctx=self._ctx)),
for ctx in chain((self.inserted_options(),
*(dmhooks.add_hooks(dm, self, appctx=self._ctx) for dm in reversed(problem_dms))),
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wence- Apparently the order of the contexts matters, we need to visit the solution DM after all other coefficients. Does this look good?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤷 Probably? Hooks are torn down in LIFO order, but I don't know why that is important in this scenario. What happens if you don't do it this way round?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor, no need to splat the generator since you're in a chain, so:

chain([self.inserted_options()], (dmhooks.add_hooks(...) for dm in ...), self._transfer_operators)

I suppose.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't put in a generator, either I pass options and hooks together in the same tuple as I had before, or I use your suggestion with tuple in front of the generator.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤷 Probably? Hooks are torn down in LIFO order, but I don't know why that is important in this scenario. What happens if you don't do it this way round?

If I don't use reverse then the appctx does not get attached properly and I get the warning "Creating new TransferManager".

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Huh, odd, anyway, I think that is fine.

@dham dham dismissed stale reviews from connorjward and themself November 8, 2023 16:45

He's happy now

@dham dham merged commit 519da0f into master Nov 8, 2023
@dham dham deleted the pbrubeck/feature/cache-transfer branch November 8, 2023 16:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support for local_range

6 participants