You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[failed][ProfilerStep not found in the trace. The analysis result may not be accurate] Using HTA but failed #238
Hi @zejun-chen sorry about the delay.
So are you asking about "ProfilerStep not found in the trace" message? Looks the analysis did run fine otherwise.
The "ProfilerStep" is used to identify training iterations and some analysis might be less accurate if we do not have them.
The annotation is added when "prof.step()" executes.
So, I think you would not see "ProfilerStepXXX" annotations in your trace if you opened them up using chrome://tracing. The reason is you need to put the "prof.step()" within the for loop in your program.
with profile(
activities = [ProfilerActivity.CPU, ProfilerActivity.CUDA],
on_trace_ready = trace_handler,
profile_memory = True,
record_shapes = True,
with_stack = True
) as prof:
input = torch.randn(4096, 4096, device=device)
for idx in range(N):
print('idx = ', idx)
output = train(input)
output = output.cpu()
prof.step() # <<<<< move this to the for loop
prof.step() # now it will execute on every train iteration
You can also use a schedule to capture certain number of iterations.
Also, I would not put profile_memory=True because that is handled using export_memory_timeline, and it is for memory analysis and not execution time analysis.
Please profiler docs for schedule and profile_memory https://pytorch.org/docs/stable/profiler.html
Activity
briancoutinho commentedon Apr 23, 2025
Hi @zejun-chen sorry about the delay.
So are you asking about "ProfilerStep not found in the trace" message? Looks the analysis did run fine otherwise.
The "ProfilerStep" is used to identify training iterations and some analysis might be less accurate if we do not have them.
The annotation is added when "prof.step()" executes.
So, I think you would not see "ProfilerStepXXX" annotations in your trace if you opened them up using
chrome://tracing. The reason is you need to put the "prof.step()" within the for loop in your program.You can also use a schedule to capture certain number of iterations.
Also, I would not put profile_memory=True because that is handled using export_memory_timeline, and it is for memory analysis and not execution time analysis.
Please profiler docs for schedule and profile_memory
https://pytorch.org/docs/stable/profiler.html
Hope this help