Skip to content

Add gru_unit_op #4443

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
guoshengCS merged 5 commits into from
Oct 13, 2017
Merged

Add gru_unit_op #4443

guoshengCS merged 5 commits into from
Oct 13, 2017

Conversation

@guoshengCS
Copy link
Contributor

@guoshengCS guoshengCS commented Sep 27, 2017 •
edited
Loading

Resolves #4213

Rewrite GruStepLayer through Eigen in the new framework and keep parameter format consistent with the original GruStepLayer, but leave some optimization like AVX to be done. https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/gserver/layers/GruStepLayer.cpp

qingqing01
qingqing01 reviewed Oct 9, 2017
View reviewed changes
paddle/operators/gru_unit_op.h Outdated
Copy link
Contributor

@qingqing01 qingqing01 Oct 9, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Many tutorials fixed the Sigmoid activation for the update gate and reset gate, Tanh activation for the memory in the GRU, but other activations like ReLU can also be used. So activation function should not be fixed here.

Copy link
Contributor Author

@guoshengCS guoshengCS Oct 12, 2017 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Refine to support multiple activation types.

paddle/operators/gru_unit_op.h Outdated
Copy link
Contributor

@qingqing01 qingqing01 Oct 9, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same as above, if want to fix it in next PR, please add TODO comments.

Copy link
Contributor Author

@guoshengCS guoshengCS Oct 12, 2017 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Refine to support multiple activation types.

paddle/operators/gru_unit_op.h Outdated
Copy link
Contributor

@qingqing01 qingqing01 Oct 9, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same as above for the tanh activation.

Copy link
Contributor Author

@guoshengCS guoshengCS Oct 12, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Refine to support multiple activation types.

paddle/operators/gru_unit_op.cc Outdated
Copy link
Contributor

@qingqing01 qingqing01 Oct 9, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should follow name convention： CamelCase

https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/operators/name_convention.md#opprotomaker-names

Copy link
Contributor Author

@guoshengCS guoshengCS Oct 12, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@qingqing01 qingqing01 mentioned this pull request Oct 13, 2017
Closed
36 tasks
@guoshengCS guoshengCS merged commit a0af1ee into PaddlePaddle:develop Oct 13, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@qingqing01 qingqing01 qingqing01 approved these changes

Assignees

No one assigned

Labels

OpPorting

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@guoshengCS @qingqing01 @Xreki