Change activation parsing in TRTLLM#11173
Conversation
| elif k == "activation_func": | ||
| if isinstance(v, torch.jit.ScriptFunction): | ||
| nemo_model_config["activation"] = v.name.replace("_", "-") | ||
| nemo_model_config["activation"] = v.name |
There was a problem hiding this comment.
n00b question: could you explain the reasoning for this change? the PR doesn't have any description and it's hard to parse out from this alone. are there any tests that exercise this path?
There was a problem hiding this comment.
There are two kinds of activation in config: JIT function and python function. Original code is to rename "squared_relu"->"squared-relu", but it will only rename JIT function. New code renames it after parsing all config and can handle both JIT and python function. Besides, removing "_"->"-" is to keep some activation like "gelu_fast".
* Fix squared relu * Fix openai-gelu
|
[🤖]: Hi @meatybobby 👋, We wanted to let you know that a CICD pipeline for this PR just finished successfully So it might be time to merge this PR or get some approvals I'm just a bot so I'll leave it you what to do next. //cc @pablo-garay @ko3n1g |
* Fix squared relu * Fix openai-gelu Signed-off-by: Joseph Jennings <jjennings@nvidia.com>
* Fix squared relu * Fix openai-gelu
* Fix squared relu * Fix openai-gelu
* Fix squared relu * Fix openai-gelu
* Fix squared relu * Fix openai-gelu
5 participants
What does this PR do ?
Add a one line overview of what this PR aims to accomplish.
Collection: [Note which collection this PR will affect]
Changelog
Usage
# Add a code snippet demonstrating how to use thisGitHub Actions CI
The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.
The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".
Before your PR is "Ready for review"
Pre checks:
PR Type:
If you haven't finished some of the above items you can still open "Draft" PR.
Who can review?
Anyone in the community is free to review the PR once the checks have passed.
Contributor guidelines contains specific people who can review PRs to various areas.
Additional Information