diff --git a/docs/source/debug-backend-delegate.md b/docs/source/debug-backend-delegate.md new file mode 100644 index 00000000000..ebcf94136c7 --- /dev/null +++ b/docs/source/debug-backend-delegate.md @@ -0,0 +1,65 @@ +# Debug Backend Delegate + +We provide a list of util functions to give users insights on what happened to the graph modules during the `to_backend()` stage. + +## Get delegation summary +The `get_delegation_info()` method provides a summary of what happened to the model after the `to_backend()` call: + +```python +from executorch.exir.backend.utils import get_delegation_info +from tabulate import tabulate + +# ... After call to to_backend(), but before to_executorch() +graph_module = edge_manager.exported_program().graph_module +delegation_info = get_delegation_info(graph_module) +print(delegation_info.get_summary()) +df = delegation_info.get_operator_delegation_dataframe() +print(tabulate(df, headers="keys", tablefmt="fancy_grid")) +``` + +Example printout: +``` +Total delegated subgraphs: 86 +Number of delegated nodes: 473 +Number of non-delegated nodes: 430 +``` + + +| | op_type | occurrences_in_delegated_graphs | occurrences_in_non_delegated_graphs | +|----|---------------------------------|------- |-----| +| 0 | aten__softmax_default | 12 | 0 | +| 1 | aten_add_tensor | 37 | 0 | +| 2 | aten_addmm_default | 48 | 0 | +| 3 | aten_arange_start_step | 0 | 25 | +| | ... | | | +| 23 | aten_view_copy_default | 170 | 48 | +| | ... | | | +| 26 | Total | 473 | 430 | + +From the table, the operator `aten_view_copy_default` appears 170 times in delegate graphs and 48 times in non-delegated graphs. Users can use information like this to debug. + +## Visualize delegated graph +To see a more detailed view, use the `print_delegated_graph()` method to display a printout of the whole graph: + +```python +from executorch.exir.backend.utils import print_delegated_graph +graph_module = edge_manager.exported_program().graph_module +print(print_delegated_graph(graph_module)) +``` +It will print the whole model as well as the subgraph consumed by the backend. The generic debug function provided by fx like `print_tabular()` or `print_readable()` will only show `call_delegate` but hide the the subgraph consumes by the backend, while this function exposes the contents inside the subgraph. + +In the example printout below, observe that `embedding` and `add` operators are delegated to `XNNPACK` while the `sub` operator is not. + +``` +%aten_unsqueeze_copy_default_22 : [num_users=1] = call_function[target=executorch.exir.dialects.edge._ops.aten.unsqueeze_copy.default](args = (%aten_arange_start_step_23, -2), kwargs = {}) + %aten_unsqueeze_copy_default_23 : [num_users=1] = call_function[target=executorch.exir.dialects.edge._ops.aten.unsqueeze_copy.default](args = (%aten_arange_start_step_24, -1), kwargs = {}) + %lowered_module_0 : [num_users=1] = get_attr[target=lowered_module_0] + backend_id: XnnpackBackend + lowered graph(): + %aten_embedding_default : [num_users=1] = placeholder[target=aten_embedding_default] + %aten_embedding_default_1 : [num_users=1] = placeholder[target=aten_embedding_default_1] + %aten_add_tensor : [num_users=1] = call_function[target=executorch.exir.dialects.edge._ops.aten.add.Tensor](args = (%aten_embedding_default, %aten_embedding_default_1), kwargs = {}) + return (aten_add_tensor,) + %executorch_call_delegate : [num_users=1] = call_function[target=torch.ops.higher_order.executorch_call_delegate](args = (%lowered_module_0, %aten_embedding_default, %aten_embedding_default_1), kwargs = {}) + %aten_sub_tensor : [num_users=1] = call_function[target=executorch.exir.dialects.edge._ops.aten.sub.Tensor](args = (%aten_unsqueeze_copy_default, %aten_unsqueeze_copy_default_1), kwargs = {}) +``` diff --git a/docs/source/index.rst b/docs/source/index.rst index cb78b012850..210839cf57c 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -187,6 +187,7 @@ Topics in this section will help you get started with ExecuTorch. native-delegates-executorch-vulkan-delegate backend-delegates-integration backend-delegates-dependencies + debug-backend-delegate .. toctree:: :glob: