enable internvl UTs on XPU

[yao-matrix]

pytest -rA tests/models/internvl/test_modeling_internvl.py

137 cases PASSED, 4 failed cases are same as CUDA, will investigate in separate efforts.

FAILED tests/models/internvl/test_modeling_internvl.py::InternVLModelTest::test_resize_embeddings_untied_with_deepspeed
FAILED tests/models/internvl/test_modeling_internvl.py::InternVLModelTest::test_resize_embeddings_untied_with_deepspeed_multi_gpu
FAILED tests/models/internvl/test_modeling_internvl.py::InternVLModelTest::test_resize_tokens_embeddings_with_deepspeed
FAILED tests/models/internvl/test_modeling_internvl.py::InternVLModelTest::test_resize_tokens_embeddings_with_deepspeed_multi_gpu

[github-actions]

Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the Ready for review button (at the bottom of the PR page). This will assign reviewers and trigger CI.

[yao-matrix]

@ydshieh , pls help review, thx

[Rocketknight1]

cc @IlyasMoutawwakil

[yao-matrix]

@ydshieh @IlyasMoutawwakil , could you help review again? thx

[IlyasMoutawwakil]

LGTM !
Only thing I'm a bit opinionated about is the use of "require_deterministic_for_xpu" because it behaves differently than other "require_xxx". We can probably try getting rid of it if ever the problem with "warn_only=True" was fixed ? #30774 (comment)

[yao-matrix]

LGTM ! Only thing I'm a bit opinionated about is the use of "require_deterministic_for_xpu" because it behaves differently than other "require_xxx". We can probably try getting rid of it if every the problem with "warn_only=True" was fixed ? #30774 (comment)

it's a new info for me, let me go back to check with our pytorch team, and if OK, i will make update. And at the same time, we are working w/ our pytorch team to root cause this difference btw xpu and cuda.

[ydshieh]

Thank you. Only 2 nits regarding formatting 🙏

[yao-matrix]

Thank you. Only 2 nits regarding formatting 🙏

@ydshieh thx, done per comments.