Skip to content

Add phi 3.5 moe test - #934

Merged
ddilbazTT merged 2 commits into
mainfrom
ddilbaz/phi3p5_moe
Jun 13, 2025
Merged

ddilbazTT merged 2 commits into
mainfrom
ddilbaz/phi3p5_moe

Conversation

@ddilbazTT

@ddilbazTT ddilbazTT commented Jun 12, 2025 •

Copy link
Copy Markdown
Contributor

Ticket

#330

Problem description

Add Phi 3.5 Mixture of Experts test
Note: the model cannot compile end-to-end due to failed to legalize operation 'stablehlo.reduce' error. For more info: https://github.com/tenstorrent/tt-torch/actions/runs/15615422753/job/43987527656

Checklist

  • Add tests/models/phi/test_phi_3p5_moe.py
  • Add tests/models/phi/test_phi_3p5_moe.py to nightly op-by-op test

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds a new test for the Phi 3.5 Mixture of Experts model and integrates it into the nightly op-by-op and full model execution test workflows.

  • Added tests/models/phi/test_phi_3p5_moe.py implementing model load, input preparation, and evaluation logic for the Phi 3.5 MoE test.
  • Updated the op-by-op nightly workflow to include the new phi test.
  • Updated the full model execution workflow to include the new phi test.

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File Description
tests/models/phi/test_phi_3p5_moe.py New test file for the Phi 3.5 MoE model
.github/workflows/run-op-by-op-model-tests-nightly.yml Added phi test to op-by-op nightly tests
.github/workflows/run-full-model-execution-tests-nightly.yml Added phi test to full model execution nightly tests
Comments suppressed due to low confidence (3)

tests/models/phi/test_phi_3p5_moe.py:14

  • The class name 'ThisTester' is not very descriptive; consider renaming it to something that reflects its purpose, like 'PhiMoETester', for better clarity.
class ThisTester(ModelTester):

tests/models/phi/test_phi_3p5_moe.py:63

  • [nitpick] The value 'red' for the variable 'model_group' is ambiguous; it would be clearer to use a name that reflects its function or meaning.
    model_group = "red"

tests/models/phi/test_phi_3p5_moe.py:36

  • [nitpick] The multi-line string for the prompt may include unintended leading whitespace; consider using a dedent function or adjusting the formatting to ensure the prompt content is as intended.
        self.prompt = """

@ddilbazTT

ddilbazTT commented Jun 12, 2025 •

Copy link
Copy Markdown
Contributor Author

It seems like there are some problems with compiling the RoPE embeddings with dynamic=False - I couldn't get it to work anyway and it might be related to huggingface/transformers#32617

However, I changed how to run the model from model(**inputs) or model.generate(**inputs) to pipeline(**intputs) following this blog https://www.datacamp.com/tutorial/fine-tuning-phi-3-5 Seems to be working.

Sonnet is adding the rest of Phi 3 tests separately, this one was having some weird Fake Tensor error from pass_pipeline so I looked into it.

@ddilbazTT
ddilbazTT requested a review from AleksKnezevic June 12, 2025 19:31
@ddilbazTT ddilbazTT changed the title Add working phi 3.5 moe test Add phi 3.5 moe test Jun 12, 2025
@ddilbazTT
ddilbazTT requested review from brataTT and jazpurTT June 12, 2025 19:44
Comment thread tests/models/phi/test_phi_3p5_moe.py Resolved
@ddilbazTT
ddilbazTT requested a review from jazpurTT June 12, 2025 19:56
Comment thread tests/models/phi/test_phi_3p5_moe.py Resolved
@ddilbazTT
ddilbazTT force-pushed the ddilbaz/phi3p5_moe branch from fa20208 to 3288421 Compare June 12, 2025 20:09

@kmabeeTT kmabeeTT left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome - thanks!

@vmilosevic

Copy link
Copy Markdown
Contributor
TestsPassed ✅Skipped ⚠️Failed
TT-Torch Tests766 ran751 passed15 skipped0 failed
TestResult
No test annotations available

@ddilbazTT
ddilbazTT merged commit a960106 into main Jun 13, 2025
@ddilbazTT
ddilbazTT deleted the ddilbaz/phi3p5_moe branch June 13, 2025 13:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants