[Executorch][to_backend] Introduce preprocess_multimethod #9823
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mcr229
requested review from
JacobSzwejbka,
tarun292 and
larryliu0820
as code owners
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/9823
Note: Links to docs will display an error until the docs builds have been completed. This comment was automatically generated by Dr. CI and updates every 15 minutes. |
facebook-github-bot
added
the
CLA Signed
This was referenced Apr 1, 2025
mcr229
added
ciflow/trunk
release notes: api
cccclai
reviewed
Apr 7, 2025
| def preprocess_multimethod( | ||
| cls, | ||
| edge_programs: Dict[str, List[ExportedProgram]], | ||
| compile_specs: Dict[str, List[List[CompileSpec]]], |
There was a problem hiding this comment.
This is the finest granularity which I believe cover all the use cases. For majority of the use case, we probably can support List[CompileSpec], which will be applied to all edge programs for all methods.
cccclai
reviewed
Apr 7, 2025
| # Users should return a compiled blob - a binary that can run the desired | ||
| # program in the backend. | ||
| pass | ||
|
|
||
| @classmethod |
There was a problem hiding this comment.
I assume it will be the default method if the backend didn't implement their own preprocess multimethod, is it correct? If so, let's add some tests.
cccclai
approved these changes
Apr 7, 2025
|
@cccclai yea let me add a test for the default preprocess_multimethod case |
…_multimethod (#9824) ### Summary We add a new to_backend api which for multi method models. Specifically, we pass in a dictionary mapping MethodName to ExportedProgram, as well as a dictionary mapping MethodName to Partitioner. We then return a dictionary mapping MethodName to the partitioned and lowered exported program. In addition, we also provide a new preprocess API for backends to implement. This API is preprocess_multimethod. The signature of the new method is as follows: ``` def preprocess_multimethod( cls, edge_programs: Dict[str, List[ExportedProgram]], compile_specs: Dict[str, List[List[CompileSpec]]], ) -> Dict[str, list[PreprocessResult]]: """ Runs preprocess on all partitioned Edge Programs across multiple methods. This allows backends to share information across partitioned graphs. Backend can serialize shared data by putting the shared data into the data_store_output of the preprocess results. This will record the shared data used by that specific partition. Default implementation is running the existing preprocess implementation on all Args: edge_programs: Dictionary mapping the method name to a list of all the partitioned edge_programs from that method to be lowered. compile_specs: Dictionary mapping the method name to a list of compile_specs. The list of compile specs maps directly to the list of edge_programs for the same given method name i.e. edge_program[method_name][i] --> compile_specs[method_name][i] Returns: Dictionary mapping the method name to a list of PreprocessResults. The list of PreprocessResults maps directly to the list of edge_programs for the same given method name. i.e. edge_program[method_name][i] --> result[method_name][i] """ ``` This new API enableds backends to preprocess all partitions/methods at once. This way, while processing blobs, they can identify shared components between preprocessed blobs. Shared components can be serialized within the NamedDataStore. The key change in backend infra, is that when partitioning, we now have to identify all the partitioned graphs to be lowered at once, and pass them to preprocess_multimethod at once. Previously, as we found lowerable partitions, we preprocessed and embedded them into the graph. ### Testing python -m unittest exir.backend.test.test_to_backend_multi_method
keyprocedure
pushed a commit
to keyprocedure/executorch
that referenced
this pull request
Apr 21, 2025
### summary We introduce a new preprocess_multimethod. This takes the place of preprocess all in the following discussion: pytorch#8230 This is a class method so that we can allow the default implementation to leverage the static preprocess method. The default implementation just applies preprocess on all the exported programs given. ### Testing Testing done in follow up PR pytorch#9824
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
summary
We introduce a new preprocess_multimethod. This takes the place of preprocess all in the following discussion: #8230
This is a class method so that we can allow the default implementation to leverage the static preprocess method. The default implementation just applies preprocess on all the exported programs given.
Testing
Testing done in follow up PR
#9824