Skip to content
Closed
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 17 additions & 20 deletions src/transformers/utils/output_capturing.py
Original file line number Diff line number Diff line change
Expand Up @@ -64,35 +64,32 @@ class CompileableContextVar:
"""

def __init__(self, name, default):
self.context_var = ContextVar(name, default=default)
self.global_var = default
self.compiling = False
self.context_var = ContextVar(name, default=default) # real thread-safe, local var
self.global_var = default # mirror for compile, global var
self.default = default

def get(self):
# Set was called before and compilation was already detected
if self.compiling:
# Always check the current dynamo state dynamically! See the note below
#
# NOTE: Local attributes/flags may be interpreted once during tracing
# and effectively treated as constant across compiled frames. If we
# cached such a flag, eager frames could incorrectly fall back to the
# shared global value, breaking visibility between eager wrappers and
# compiled hooks

@Cyrilvallez Cyrilvallez Mar 12, 2026 •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't really understand this comment (the final part), so not helpful IMO haha - let's either entirely remove or rewrite!

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will update in a second

if is_torchdynamo_compiling():
return self.global_var
else:
# Set was maybe never called, so still check it here
if is_torchdynamo_compiling():
self.is_compiling = True

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Side note: This was even a typo 👀 self.compiling <-> self.is_compiling

return self.global_var
else:
return self.context_var.get()
return self.context_var.get()

def set(self, value):
self.global_var = value
# Same, check the current state dynamically rather than relying on cached attributes

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we always set the global here? Let's only do when it's compiling!

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is essentially the core change, it seems somewhere along the way, some small graph break happens and when we do not set it in general the state between compiled regions and non-compiled eager regions is messed up

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is concerning, it means some modules are fully skipped by dynamo and not compiled??

if is_torchdynamo_compiling():
self.global_var = value
self.compiling = True
return None
else:
return self.context_var.set(value)
return self.context_var.set(value)

def reset(self, token):
if self.compiling or token is None:
self.global_var = None
self.compiling = False
else:
self.global_var = self.default

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's just reset to None alway no? Instead of saving the default as internal attr

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't mind, but why do we even provide a default in the init at all 😅

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only because ContextVar requires it! Though indeed we could simply explicitly do it in our wrapper's init instead of exposing it

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea changed the init, and forced none in any case

if token is not None:
self.context_var.reset(token)


Expand Down
Loading