Skip to content

[TF FE][MOC] Fuse Keras LSTM to LSTMSequence and Optimize TF While with TensorList ops #25170

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
rkazants merged 16 commits into from
Jun 27, 2024
Merged

[TF FE][MOC] Fuse Keras LSTM to LSTMSequence and Optimize TF While with TensorList ops #25170

rkazants merged 16 commits into from
Jun 27, 2024

Conversation

@rkazants
Copy link
Collaborator

@rkazants rkazants commented Jun 22, 2024 •
edited
Loading

Details: Fuse Keras LSTM to LSTMSequence and Optimize TF While with TensorList ops.
Loop operations with TensorListSetItem transformed to ConcatOutput outputs.
Loop operations with TensorListGetItem transformed to SlicedInput inputs.

It helps to fuse six loops with LSTMCell to six LSTM sequence model. It reduces customer model size by twice and increase throughput by 1.97x.

Tickets: TBD

@rkazants rkazants requested a review from a team as a code owner June 22, 2024 11:32
@github-actions github-actions bot added category: TF FE OpenVINO TensorFlow FrontEnd category: TFL FE OpenVINO TensorFlow Lite FrontEnd labels Jun 22, 2024
@rkazants rkazants requested review from itikhono, mvafin and popovaan June 22, 2024 11:32
@rkazants rkazants requested a review from a team as a code owner June 24, 2024 11:24
@github-actions github-actions bot added the category: transformations OpenVINO Runtime library - Transformations label Jun 24, 2024
@rkazants
Fix build
d2c7b1f 
Signed-off-by: Kazantsev, Roman <[email protected]>
@rkazants rkazants requested a review from evkotov June 24, 2024 11:29
@rkazants
Fix build
02605ee 
Signed-off-by: Kazantsev, Roman <[email protected]>
itikhono
itikhono reviewed Jun 24, 2024
View reviewed changes
evkotov
evkotov reviewed Jun 25, 2024
View reviewed changes
Copy link
Contributor

@evkotov evkotov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you have performance measurements? Did this transformation improve?

rkazants added 2 commits June 25, 2024 18:57
@rkazants
Fix LSTMCell pattern
c4dbc34 
Signed-off-by: Kazantsev, Roman <[email protected]>
@rkazants
Avoid deprecated LSTMSequence version
f499406 
Signed-off-by: Kazantsev, Roman <[email protected]>
@rkazants
Copy link
Collaborator Author

rkazants commented Jun 26, 2024

Do you have performance measurements? Did this transformation improve?

Yes, 2x for customer model

rkazants added 4 commits June 26, 2024 18:31
@rkazants
Address code-review comments
d05510b 
Signed-off-by: Kazantsev, Roman <[email protected]>
@rkazants
Address comment to remove get_param_idx routine
2a3374b 
Signed-off-by: Kazantsev, Roman <[email protected]>
@rkazants
Address comment to move get_constant_value to utils
c368d39 
Signed-off-by: Kazantsev, Roman <[email protected]>
@rkazants
Fix build
e4031e0 
Signed-off-by: Kazantsev, Roman <[email protected]>
@rkazants rkazants requested review from evkotov and itikhono June 26, 2024 16:35
rkazants added 2 commits June 27, 2024 02:42
@rkazants
Implement unit-tests for transformations
0b2745b 
Signed-off-by: Kazantsev, Roman <[email protected]>
@rkazants
Add callback
866bea7 
Signed-off-by: Kazantsev, Roman <[email protected]>
itikhono
itikhono approved these changes Jun 27, 2024
View reviewed changes
@rkazants
Copy link
Collaborator Author

rkazants commented Jun 27, 2024

@popovaan, @itikhono, let me fix your comments in separate PR.

@rkazants rkazants added this to the 2024.3 milestone Jun 27, 2024
@andrei-kochin andrei-kochin added this pull request to the merge queue Jun 27, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jun 27, 2024
@rkazants rkazants added this pull request to the merge queue Jun 27, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jun 27, 2024
@rkazants rkazants added this pull request to the merge queue Jun 27, 2024
Merged via the queue into openvinotoolkit:master with commit 07cbea2 Jun 27, 2024
@rkazants rkazants deleted the rkazants/lstm_keras_fusion branch June 27, 2024 19:48
@rkazants rkazants mentioned this pull request Jun 27, 2024
Merged
github-merge-queue bot pushed a commit that referenced this pull request Jun 28, 2024
@rkazants
[TF FE][MOC] Fix leftovers for Keras LSTM fusion transformation (#25268)
f9cf396 
**Details:** Fix leftovers for Keras LSTM fusion transformation
#25170

**Tickets:** TBD

Signed-off-by: Kazantsev, Roman <[email protected]>
github-merge-queue bot pushed a commit that referenced this pull request Jun 28, 2024
@rkazants
[TF FE][MOC] Fix leftovers for Keras LSTM fusion transformation (#25268)
8cdbedc 
**Details:** Fix leftovers for Keras LSTM fusion transformation
#25170

**Tickets:** TBD

Signed-off-by: Kazantsev, Roman <[email protected]>
AsyaPronina pushed a commit to AsyaPronina/openvino that referenced this pull request Jul 1, 2024
@rkazants @AsyaPronina
[TF FE][MOC] Fuse Keras LSTM to LSTMSequence and Optimize TF While wi…
5253d2e 
…th TensorList ops (openvinotoolkit#25170)

**Details:** Fuse Keras LSTM to LSTMSequence and Optimize TF While with
TensorList ops.
Loop operations with TensorListSetItem transformed to ConcatOutput
outputs.
Loop operations with TensorListGetItem transformed to SlicedInput
inputs.

It helps to fuse six loops with LSTMCell to six LSTM sequence model. It
reduces customer model size by twice and increase throughput by 1.97x.

**Tickets:** TBD

---------

Signed-off-by: Kazantsev, Roman <[email protected]>
AsyaPronina pushed a commit to AsyaPronina/openvino that referenced this pull request Jul 1, 2024
@rkazants @AsyaPronina
[TF FE][MOC] Fix leftovers for Keras LSTM fusion transformation (open…
b7c22ef 
…vinotoolkit#25268)

**Details:** Fix leftovers for Keras LSTM fusion transformation
openvinotoolkit#25170

**Tickets:** TBD

Signed-off-by: Kazantsev, Roman <[email protected]>
AsyaPronina pushed a commit to AsyaPronina/openvino that referenced this pull request Jul 1, 2024
@rkazants @AsyaPronina
[TF FE][MOC] Fuse Keras LSTM to LSTMSequence and Optimize TF While wi…
ab1a407 
…th TensorList ops (openvinotoolkit#25170)

**Details:** Fuse Keras LSTM to LSTMSequence and Optimize TF While with
TensorList ops.
Loop operations with TensorListSetItem transformed to ConcatOutput
outputs.
Loop operations with TensorListGetItem transformed to SlicedInput
inputs.

It helps to fuse six loops with LSTMCell to six LSTM sequence model. It
reduces customer model size by twice and increase throughput by 1.97x.

**Tickets:** TBD

---------

Signed-off-by: Kazantsev, Roman <[email protected]>
AsyaPronina pushed a commit to AsyaPronina/openvino that referenced this pull request Jul 1, 2024
@rkazants @AsyaPronina
[TF FE][MOC] Fix leftovers for Keras LSTM fusion transformation (open…
d7d9770 
…vinotoolkit#25268)

**Details:** Fix leftovers for Keras LSTM fusion transformation
openvinotoolkit#25170

**Tickets:** TBD

Signed-off-by: Kazantsev, Roman <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@popovaan popovaan popovaan left review comments

@evkotov evkotov evkotov approved these changes

@mvafin mvafin Awaiting requested review from mvafin

+2 more reviewers

@itikhono itikhono itikhono approved these changes

@andrei-kochin andrei-kochin andrei-kochin approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@AKochin AKochin

@andrei-kochin andrei-kochin

Labels

category: TF FE OpenVINO TensorFlow FrontEnd category: TFL FE OpenVINO TensorFlow Lite FrontEnd category: transformations OpenVINO Runtime library - Transformations

Projects

None yet

Milestone

2024.3

Development

Successfully merging this pull request may close these issues.

6 participants

@rkazants @itikhono @andrei-kochin @popovaan @evkotov @AKochin