Refit bug fix #3097

cehongwang · 2024-08-17T19:13:54Z

Description

Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

Fixes # (issue)

Type of change

Please delete options that are not relevant and/or add your own.

Bug fix (non-breaking change which fixes an issue)

github-actions

There are some changes that do not conform to Python style guidelines:

--- /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/_compile.py	2024-08-17 19:14:08.524383+00:00
+++ /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/_compile.py	2024-08-17 19:14:28.914991+00:00
@@ -532,6 +532,6 @@

                with enable_torchbind_tracing():
                    exp_program = torch.export.export(
                        module, tuple(arg_inputs), kwargs=kwarg_inputs, strict=False
                    )
-                    torch.export.save(exp_program, file_path)
\ No newline at end of file
+                    torch.export.save(exp_program, file_path)

github-actions

There are some changes that do not conform to Python style guidelines:

--- /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/_compile.py	2024-08-17 19:15:47.718371+00:00
+++ /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/_compile.py	2024-08-17 19:16:06.105413+00:00
@@ -532,6 +532,6 @@

                with enable_torchbind_tracing():
                    exp_program = torch.export.export(
                        module, tuple(arg_inputs), kwargs=kwarg_inputs, strict=False
                    )
-                    torch.export.save(exp_program, file_path)
\ No newline at end of file
+                    torch.export.save(exp_program, file_path)

github-actions

There are some changes that do not conform to Python style guidelines:

--- /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/_compile.py	2024-08-17 20:57:51.366224+00:00
+++ /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/_compile.py	2024-08-17 20:58:11.913742+00:00
@@ -532,6 +532,6 @@

                with enable_torchbind_tracing():
                    exp_program = torch.export.export(
                        module, tuple(arg_inputs), kwargs=kwarg_inputs, strict=False
                    )
-                    torch.export.save(exp_program, file_path)
\ No newline at end of file
+                    torch.export.save(exp_program, file_path)

github-actions

There are some changes that do not conform to Python style guidelines:

--- /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/_compile.py	2024-08-17 20:58:30.164265+00:00
+++ /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/_compile.py	2024-08-17 20:58:51.729699+00:00
@@ -532,6 +532,6 @@

                with enable_torchbind_tracing():
                    exp_program = torch.export.export(
                        module, tuple(arg_inputs), kwargs=kwarg_inputs, strict=False
                    )
-                    torch.export.save(exp_program, file_path)
\ No newline at end of file
+                    torch.export.save(exp_program, file_path)

github-actions

There are some changes that do not conform to Python style guidelines:

--- /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/_compile.py	2024-08-17 22:48:32.396308+00:00
+++ /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/_compile.py	2024-08-17 22:48:53.776331+00:00
@@ -532,6 +532,6 @@

                with enable_torchbind_tracing():
                    exp_program = torch.export.export(
                        module, tuple(arg_inputs), kwargs=kwarg_inputs, strict=False
                    )
-                    torch.export.save(exp_program, file_path)
\ No newline at end of file
+                    torch.export.save(exp_program, file_path)

github-actions

There are some changes that do not conform to Python style guidelines:

--- /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/_compile.py	2024-08-18 21:21:30.900238+00:00
+++ /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/_compile.py	2024-08-18 21:21:49.713339+00:00
@@ -532,6 +532,6 @@

                with enable_torchbind_tracing():
                    exp_program = torch.export.export(
                        module, tuple(arg_inputs), kwargs=kwarg_inputs, strict=False
                    )
-                    torch.export.save(exp_program, file_path)
\ No newline at end of file
+                    torch.export.save(exp_program, file_path)

github-actions

There are some changes that do not conform to Python style guidelines:

--- /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/_compile.py	2024-08-18 21:25:53.397551+00:00
+++ /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/_compile.py	2024-08-18 21:26:12.396126+00:00
@@ -532,6 +532,6 @@

                with enable_torchbind_tracing():
                    exp_program = torch.export.export(
                        module, tuple(arg_inputs), kwargs=kwarg_inputs, strict=False
                    )
-                    torch.export.save(exp_program, file_path)
\ No newline at end of file
+                    torch.export.save(exp_program, file_path)

narendasan · 2024-08-19T16:09:11Z

@cehongwang can you check in the linting for _compile.py as well even though its unrelated?

narendasan · 2024-08-19T16:13:11Z

py/torch_tensorrt/dynamo/conversion/_TRTInterpreter.py

 from torch_tensorrt.fx.observer import Observer
 from torch_tensorrt.logging import TRT_LOGGER
+from tqdm import tqdm


We already have an optional dependency on rich coming in another PR. We should enable this optionally

Ok I will delete this for now and we can add that later

peri044 · 2024-08-19T16:02:49Z

py/torch_tensorrt/dynamo/_refit.py

+                "weight": 1.0,
+                "bias": 0.0,
+                "running_mean": 0.0,
+                "running_var": 1.0,


why are these hardcoded ?

These are the default value when those doesn't exist

peri044 · 2024-08-19T16:03:20Z

py/torch_tensorrt/dynamo/_refit.py

+        # Debug Use
+        # correct = construct_refit_mapping(new_gm, input_list, settings)
+        # {k: np.allclose(correct[k][0], mapping[k][0].cpu().numpy(), 1e-2, 1e-2) for k in mapping if k in correct}


remove this

py/torch_tensorrt/dynamo/conversion/_TRTInterpreter.py

github-actions

There are some changes that do not conform to Python style guidelines:

--- /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/_compile.py	2024-08-19 17:36:07.179801+00:00
+++ /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/_compile.py	2024-08-19 17:36:27.161982+00:00
@@ -532,6 +532,6 @@

                with enable_torchbind_tracing():
                    exp_program = torch.export.export(
                        module, tuple(arg_inputs), kwargs=kwarg_inputs, strict=False
                    )
-                    torch.export.save(exp_program, file_path)
\ No newline at end of file
+                    torch.export.save(exp_program, file_path)

github-actions

There are some changes that do not conform to Python style guidelines:

--- /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/_compile.py	2024-08-19 17:40:11.059587+00:00
+++ /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/_compile.py	2024-08-19 17:40:31.806668+00:00
@@ -532,6 +532,6 @@

                with enable_torchbind_tracing():
                    exp_program = torch.export.export(
                        module, tuple(arg_inputs), kwargs=kwarg_inputs, strict=False
                    )
-                    torch.export.save(exp_program, file_path)
\ No newline at end of file
+                    torch.export.save(exp_program, file_path)

py/torch_tensorrt/dynamo/conversion/_TRTInterpreter.py

…rect result when varify_output is set to True

py/torch_tensorrt/dynamo/utils.py

facebook-github-bot added the cla signed label Aug 17, 2024

github-actions bot added component: conversion Issues re: Conversion stage component: api [Python] Issues re: Python API component: runtime component: dynamo Issues relating to the `torch.compile` or `torch._dynamo.export` paths labels Aug 17, 2024

github-actions bot requested a review from peri044 August 17, 2024 19:14

github-actions bot requested changes Aug 17, 2024

View reviewed changes

cehongwang force-pushed the Refit-bug-fix branch from f8b6ac6 to 1f307e1 Compare August 17, 2024 19:15

github-actions bot requested changes Aug 17, 2024

View reviewed changes

cehongwang force-pushed the Refit-bug-fix branch 2 times, most recently from 7f7599b to a4a1767 Compare August 17, 2024 20:58

github-actions bot requested changes Aug 17, 2024

View reviewed changes

github-actions bot requested changes Aug 18, 2024

View reviewed changes

cehongwang force-pushed the Refit-bug-fix branch from b288f80 to d9653cd Compare August 18, 2024 21:25

github-actions bot added the component: tests Issues re: Tests label Aug 18, 2024

github-actions bot requested changes Aug 18, 2024

View reviewed changes

narendasan reviewed Aug 19, 2024

View reviewed changes

peri044 reviewed Aug 19, 2024

View reviewed changes

github-actions bot requested changes Aug 19, 2024

View reviewed changes

cehongwang force-pushed the Refit-bug-fix branch from 2762e6e to e12d88d Compare August 19, 2024 17:43

narendasan reviewed Aug 21, 2024

View reviewed changes

py/torch_tensorrt/dynamo/conversion/_TRTInterpreter.py Outdated Show resolved Hide resolved

narendasan reviewed Aug 21, 2024

View reviewed changes

py/torch_tensorrt/dynamo/conversion/_TRTInterpreter.py Outdated Show resolved Hide resolved

cehongwang force-pushed the Refit-bug-fix branch 2 times, most recently from 12d1d80 to b1fdae3 Compare August 21, 2024 19:35

cehongwang added 2 commits August 22, 2024 13:54

Fixed device issue when model is on CPU

36e7cc8

Fixed prolong time of weight name mapping construction

2998c91

cehongwang and others added 5 commits August 22, 2024 13:54

Handled the memory issue

e68a7e3

Seriously optimized weight name map construction

e8dc166

Fixed accuracy issue of fast refit

9b801e7

Made revision according to comments

6bfd978

Fixed style issue

df95f00

cehongwang force-pushed the Refit-bug-fix branch from b1fdae3 to df95f00 Compare August 22, 2024 20:54

github-actions bot removed the component: runtime label Aug 22, 2024

cehongwang force-pushed the Refit-bug-fix branch from c9f1a55 to e732e4c Compare August 23, 2024 00:48

Fixed batchnorm converter change

df121b9

cehongwang force-pushed the Refit-bug-fix branch from e732e4c to df121b9 Compare August 23, 2024 00:51

Moved output check to the util. Redo refit if fast refit yields incor…

da4542c

…rect result when varify_output is set to True

github-actions bot added the component: runtime label Aug 23, 2024

narendasan reviewed Aug 27, 2024

View reviewed changes

py/torch_tensorrt/dynamo/utils.py Outdated Show resolved Hide resolved

cehongwang and others added 3 commits August 28, 2024 00:02

Changed the precision tolerance from 1e-2 to 5e-3

d585a13

Merge branch 'main' into Refit-bug-fix

1ffa050

chore: move threshold values to variables and rebase with main

762d965

peri044 merged commit 60ec67b into main Aug 28, 2024
67 checks passed

zewenli98 mentioned this pull request Aug 29, 2024

🐛 [Bug] Refit bug that state_dict could be empty #3126

Closed

Refit bug fix #3097

Refit bug fix #3097

Uh oh!

Conversation

cehongwang commented Aug 17, 2024

Description

Type of change

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

narendasan commented Aug 19, 2024

Uh oh!

narendasan Aug 19, 2024

Choose a reason for hiding this comment

Uh oh!

cehongwang Aug 19, 2024

Choose a reason for hiding this comment

Uh oh!

peri044 Aug 19, 2024

Choose a reason for hiding this comment

Uh oh!

cehongwang Aug 19, 2024

Choose a reason for hiding this comment

Uh oh!

peri044 Aug 19, 2024

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!