Implement CPU plugin just-in-time emitter for IsFinite operation #25237

awayzjj · 2024-06-26T15:28:58Z

Details:

item1
...

Tickets:

CVS-137697

awayzjj · 2024-06-26T15:36:24Z

@eshoguli @dmitry-gorokhov Hi, I have implemented this new emitter. Could you please help me review the PR and trigger the CI? Thank you very much!

awayzjj · 2024-06-27T09:59:39Z

Hello @dmitry-gorokhov @eshoguli , just a gentle reminder to review this PR when you have time.

awayzjj · 2024-06-28T10:30:09Z

@mlukasze Hi, could you please help me trigger the CI?

mlukasze · 2024-06-28T11:32:16Z

build_jenkins

awayzjj · 2024-06-28T16:01:18Z

Hello @dmitry-gorokhov @eshoguli @mlukasze Some tests failed :(

The visible failed test is about JAX FE which seems not related with my PR ?
Could you please help me check other failed tests?

awayzjj · 2024-07-01T06:10:56Z

@dmitry-gorokhov @eshoguli Gentle ping.

awayzjj · 2024-07-03T02:38:20Z

@dmitry-gorokhov @eshoguli Just a friendly reminder for help when you have time. I want to finish this emitter and move to another!

a-sidorova · 2024-07-03T07:00:16Z

@awayzjj there are two failed jobs which are not affected by your changes.

I see that at the moment the both of them are green on the master branch, so I suggest to just rebase your branch to fix CI.

awayzjj · 2024-07-03T07:04:37Z

@a-sidorova Thank you so much! I have rebased my branch, Could you please help me trigger the CI?

mlukasze · 2024-07-03T07:05:42Z

build_jenkins

awayzjj · 2024-07-03T23:48:57Z

@a-sidorova Hi, there are 2 jobs that have been hanging for a long time, and 3 jobs have failed. Could you please provide some suggestions on how to address these issues? Thank you very much!

a-sidorova · 2024-07-04T04:51:53Z

@awayzjj hi! To fix GPU jobs, you can rebase to the latest master to get the commit #25355 which fixes the failed test. As for 2 hanging GHA jobs, there was a queue. I hope rebasing will help you 😃

awayzjj · 2024-07-04T05:31:31Z

@a-sidorova Thank you so much! I have rebased my branch, Could you please help me trigger the CI again?

a-sidorova · 2024-07-04T05:34:23Z

build_jenkins

awayzjj · 2024-07-04T07:51:43Z

@a-sidorova @mlukasze Wow, the CI is full green. Can you merge the PR? Thank you very much!

eshoguli

New emitter handles special values: NaN and Infinity, but tests input tensor generation was not changed. Please, update ActivationLayerCPUTest::generate_inputs to have NaN and Infinity values in test input tensor.
Note, please:

there are several types of NaN values - use different, please,
there is an example for Sign activation here. You can do something similar but use more then one value, please.

src/plugins/intel_cpu/src/emitters/plugin/aarch64/jit_eltwise_emitters.cpp

awayzjj · 2024-07-08T17:35:56Z

@eshoguli Hi, thank you very much for your review! I have incorporated your suggestions, could you please review the PR again?

mlukasze · 2024-07-09T04:47:13Z

build_jenkins

dmitry-gorokhov · 2024-07-09T05:41:41Z

src/plugins/intel_cpu/tests/functional/custom/single_layer_tests/classes/activation.cpp

+                static_cast<float*>(tensor.data())[3] = std::tgamma(-1); // nan
+                static_cast<float*>(tensor.data())[4] = std::log(-1); // nan
+                static_cast<float*>(tensor.data())[5] = std::sqrt(-1); // -nan
+                static_cast<float*>(tensor.data())[6] = 0 / 0.0; // -nan


@awayzjj This line leads to failed Win build on CI:
"openvino/src/plugins/intel_cpu/tests/functional/custom/single_layer_tests/classes/activation.cpp(105): error C2124: divide or mod by zero"

@dmitry-gorokhov Thank you! But I do not have much experirence on windows. Should I just remove the line 105?

You are implementing isFinite operation. So I believe it is enough to have 4 specific values: nan, -nan, inf, -inf.

Thank you, I will fix it later today!

dmitry-gorokhov · 2024-07-09T13:00:30Z

build_jenkins

src/plugins/intel_cpu/tests/functional/custom/single_layer_tests/classes/activation.cpp

dmitry-gorokhov · 2024-07-11T13:37:27Z

@awayzjj Could you please rebase you PR on latest master state? This is required to make CI green.

dmitry-gorokhov · 2024-07-11T13:41:03Z

build_jenkins

awayzjj · 2024-07-11T14:46:40Z

@dmitry-gorokhov Hi, I forgot to update the master before rebase, could you please help me trigger the CI again? Thank you very much!

mg-intel · 2024-07-11T17:29:08Z

build_jenkins

awayzjj · 2024-07-12T01:45:20Z

@dmitry-gorokhov @eshoguli Hi, the CI is still not full green after rebasing :(

dmitry-gorokhov · 2024-07-12T06:59:57Z

@awayzjj CI is green now. Added in merge queue.
Great job!

awayzjj · 2024-07-12T07:20:55Z

@dmitry-gorokhov Thank you so much for your patience and help!

I have only one emitter to implement! Before starting, I have one question: how much performance improvements in terms of speed and throughput can we expect from JIT? Are there approximate numbers you can share?

dmitry-gorokhov · 2024-07-12T09:18:26Z

@awayzjj

I have only one emitter to implement!

We have two more that need contributors. Feel free to take :)

As for performance improvements it depends on the particular operation. As you know except JIT we have 2 backends ACL and reference impls.
JIT usually delivers 10-50% better perf than ACL. Comparing with reference the difference is much more noticable (since reference impl is very slow itself): 10-100x faster.

awayzjj · 2024-07-12T09:45:19Z

@dmitry-gorokhov I've taken these two issues and will work on resolving them as quickly as possible. I hope that in the future, you can release slightly different issues, as I want to become more familiar with our ARM backend! :)

…nvinotoolkit#25237) Closes openvinotoolkit#24416 ### Details: - *item1* - *...* ### Tickets: - [CVS-137697](https://jira.devtools.intel.com/browse/CVS-137697)

awayzjj requested review from a team as code owners June 26, 2024 15:28

github-actions bot added category: IE Tests OpenVINO Test: plugins and common category: CPU OpenVINO CPU plugin labels Jun 26, 2024

sys-openvino-ci added the ExternalPR External contributor label Jun 26, 2024

awayzjj force-pushed the is_finite_emitter branch from 68839ad to 2e11f21 Compare July 3, 2024 07:03

awayzjj force-pushed the is_finite_emitter branch from 2e11f21 to 686d9ae Compare July 4, 2024 05:29

a-sidorova assigned eshoguli Jul 4, 2024

eshoguli reviewed Jul 4, 2024

View reviewed changes

awayzjj force-pushed the is_finite_emitter branch 2 times, most recently from 2958716 to a5c3b10 Compare July 8, 2024 16:33

mlukasze requested a review from eshoguli July 9, 2024 04:47

dmitry-gorokhov reviewed Jul 9, 2024

View reviewed changes

eshoguli reviewed Jul 9, 2024

View reviewed changes

src/plugins/intel_cpu/tests/functional/custom/single_layer_tests/classes/activation.cpp Show resolved Hide resolved

awayzjj force-pushed the is_finite_emitter branch from 166e18f to 2e48dbf Compare July 11, 2024 13:39

dmitry-gorokhov enabled auto-merge July 11, 2024 14:32

awayzjj added 4 commits July 11, 2024 22:44

Implement CPU plugin just-in-time emitter for IsFinite operation

15ba4b8

for ARM32

dd4982e

optimize emitter and add tests

ceb99a6

remove some test cases

0b2b2e9

auto-merge was automatically disabled July 11, 2024 14:44
Head branch was pushed to by a user without write access

awayzjj force-pushed the is_finite_emitter branch from 2e48dbf to 0b2b2e9 Compare July 11, 2024 14:44

dmitry-gorokhov enabled auto-merge July 11, 2024 19:03

dmitry-gorokhov approved these changes Jul 12, 2024

View reviewed changes

dmitry-gorokhov added this pull request to the merge queue Jul 12, 2024

Merged via the queue into openvinotoolkit:master with commit 2bd91ee Jul 12, 2024

mlukasze added this to the 2024.4 milestone Jul 12, 2024

Implement CPU plugin just-in-time emitter for IsFinite operation #25237

Implement CPU plugin just-in-time emitter for IsFinite operation #25237

Uh oh!

Conversation

awayzjj commented Jun 26, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Details:

Tickets:

Uh oh!

awayzjj commented Jun 26, 2024

Uh oh!

awayzjj commented Jun 27, 2024

Uh oh!

awayzjj commented Jun 28, 2024

Uh oh!

mlukasze commented Jun 28, 2024

Uh oh!

awayzjj commented Jun 28, 2024

Uh oh!

awayzjj commented Jul 1, 2024

Uh oh!

awayzjj commented Jul 3, 2024

Uh oh!

a-sidorova commented Jul 3, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

awayzjj commented Jul 3, 2024

Uh oh!

mlukasze commented Jul 3, 2024

Uh oh!

awayzjj commented Jul 3, 2024

Uh oh!

a-sidorova commented Jul 4, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

awayzjj commented Jul 4, 2024

Uh oh!

a-sidorova commented Jul 4, 2024

Uh oh!

awayzjj commented Jul 4, 2024

Uh oh!

eshoguli left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

awayzjj commented Jul 8, 2024

Uh oh!

mlukasze commented Jul 9, 2024

Uh oh!

dmitry-gorokhov Jul 9, 2024

Choose a reason for hiding this comment

Uh oh!

awayzjj Jul 9, 2024

Choose a reason for hiding this comment

Uh oh!

dmitry-gorokhov Jul 9, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

awayzjj Jul 9, 2024

Choose a reason for hiding this comment

Uh oh!

dmitry-gorokhov commented Jul 9, 2024

Uh oh!

Uh oh!

dmitry-gorokhov commented Jul 11, 2024

Uh oh!

dmitry-gorokhov commented Jul 11, 2024

Uh oh!

awayzjj commented Jul 11, 2024

Uh oh!

mg-intel commented Jul 11, 2024

Uh oh!

awayzjj commented Jul 12, 2024

Uh oh!

dmitry-gorokhov commented Jul 12, 2024

Uh oh!

awayzjj commented Jun 26, 2024 •

edited

Loading

a-sidorova commented Jul 3, 2024 •

edited

Loading

a-sidorova commented Jul 4, 2024 •

edited

Loading

eshoguli left a comment •

edited

Loading

dmitry-gorokhov Jul 9, 2024 •

edited

Loading