Debug Backend Delegate

We provide a list of util functions to give users insights on what happened to the graph modules during the to_backend() stage.

Get delegation summary

The get_delegation_info() method provides a summary of what happened to the model after the to_backend() call:

from executorch.devtools.backend_debug import get_delegation_info
from tabulate import tabulate

# ... After call to to_backend(), but before to_executorch()
graph_module = edge_manager.exported_program().graph_module
delegation_info = get_delegation_info(graph_module)
print(delegation_info.get_summary())
df = delegation_info.get_operator_delegation_dataframe()
print(tabulate(df, headers="keys", tablefmt="fancy_grid"))

Example printout:

Total  delegated  subgraphs:  86
Number  of  delegated  nodes:  473
Number  of  non-delegated  nodes:  430

	op_type	occurrences_in_delegated_graphs	occurrences_in_non_delegated_graphs
0	aten__softmax_default	12	0
1	aten_add_tensor	37	0
2	aten_addmm_default	48	0
3	aten_arange_start_step	0	25
	...
23	aten_view_copy_default	170	48
	...
26	Total	473	430

From the table, the operator aten_view_copy_default appears 170 times in delegate graphs and 48 times in non-delegated graphs. Users can use information like this to debug.

Visualize delegated graph

To see a more detailed view, use the format_delegated_graph() method to get a str of printout of the whole graph or use print_delegated_graph() to print directly:

from executorch.exir.backend.utils import format_delegated_graph
graph_module = edge_manager.exported_program().graph_module
print(format_delegated_graph(graph_module)) # or call print_delegated_graph(graph_module)

It will print the whole model as well as the subgraph consumed by the backend. The generic debug function provided by fx like print_tabular() or print_readable() will only show call_delegate but hide the the subgraph consumes by the backend, while this function exposes the contents inside the subgraph.

In the example printout below, observe that embedding and add operators are delegated to XNNPACK while the sub operator is not.

%aten_unsqueeze_copy_default_22 : [num_users=1] = call_function[target=executorch.exir.dialects.edge._ops.aten.unsqueeze_copy.default](args = (%aten_arange_start_step_23, -2), kwargs = {})
  %aten_unsqueeze_copy_default_23 : [num_users=1] = call_function[target=executorch.exir.dialects.edge._ops.aten.unsqueeze_copy.default](args = (%aten_arange_start_step_24, -1), kwargs = {})
  %lowered_module_0 : [num_users=1] = get_attr[target=lowered_module_0]
    backend_id: XnnpackBackend
    lowered graph():
      %aten_embedding_default : [num_users=1] = placeholder[target=aten_embedding_default]
      %aten_embedding_default_1 : [num_users=1] = placeholder[target=aten_embedding_default_1]
      %aten_add_tensor : [num_users=1] = call_function[target=executorch.exir.dialects.edge._ops.aten.add.Tensor](args = (%aten_embedding_default, %aten_embedding_default_1), kwargs = {})
      return (aten_add_tensor,)
  %executorch_call_delegate : [num_users=1] = call_function[target=torch.ops.higher_order.executorch_call_delegate](args = (%lowered_module_0, %aten_embedding_default, %aten_embedding_default_1), kwargs = {})
  %aten_sub_tensor : [num_users=1] = call_function[target=executorch.exir.dialects.edge._ops.aten.sub.Tensor](args = (%aten_unsqueeze_copy_default, %aten_unsqueeze_copy_default_1), kwargs = {})