Fix elementwise_gradient bug #10150
Merged
Fix elementwise_gradient bug #10150
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
panyx0718
reviewed
Apr 24, 2018
Contributor
panyx0718
left a comment
panyx0718
left a comment
There was a problem hiding this comment.
Can you add your test as well?
chengduoZH
force-pushed
the
fix_elementwise_gradient
branch
from
612685e to
5b345d1
Compare
chengduoZH
force-pushed
the
fix_elementwise_gradient
branch
from
5b345d1 to
d06c79c
Compare
qingqing01
reviewed
Apr 24, 2018
Contributor
Author
There was a problem hiding this comment.
The process of getting gradient involves accumulative operation and the shape of input is too large, x.shape = [2,32,220,220], y.shape = [32], so the diff of the result between Python and C++(CUDA) is bigger.
Python result
array([96771.625, 96801.26 , 96760.63 , 96893.32 , 96946.375, 96911.95 ,
96837.16 , 96716.88 , 96931.305, 96746.445, 96781.81 , 96827.22 ,
97108.19 , 96866.8 , 96894.17 , 96776.36 , 96720.61 , 96992.8 ,
96641.664, 96772.305, 96698.16 , 96675.64 , 96805.86 , 96710.6 ,
96733.37 , 96858.41 , 96771.516, 97033.516, 96820.48 , 96726.09 ,
96784.66 , 96740.76 ], dtype=float32)
C++ result
array([96771.1 , 96801.05 , 96761.125, 96892.69 , 96946.73 , 96910.93 ,
96837.516, 96716.27 , 96930.99 , 96748.18 , 96781.41 , 96828.016,
97108.17 , 96866.62 , 96893.836, 96775.74 , 96720.98 , 96992.24 ,
96642.36 , 96771.14 , 96698.195, 96675.055, 96805.88 , 96711.305,
96732.086, 96858.97 , 96770.734, 97033.92 , 96819.95 , 96725.72 ,
96784.35 , 96740.76 ], dtype=float32)
CUDA result
array([96738.2 , 96891.58 , 96752.25 , 96972.39 , 96706.016, 96755.234,
96731.42 , 96850.01 , 97060.125, 96790.375, 96587.12 , 96833.305,
96709.3 , 96703.7 , 96842.61 , 96727.95 , 96966.586, 96791.33 ,
97077.234, 96715.03 , 96850.26 , 96898.47 , 96780.97 , 96839.516,
96835.61 , 96600.41 , 96517.08 , 96787.58 , 96758.37 , 96555.484,
96882.94 , 96578.2 ], dtype=float32)
So the max diff between Python result and C++ result is 1.28125, the max diff between Python result and CUDA result is 0.015625.
chengduoZH
force-pushed
the
fix_elementwise_gradient
branch
from
e2f89a1 to
3623f02
Compare
… fix_elementwise_gradient
chengduoZH
force-pushed
the
fix_elementwise_gradient
branch
from
3623f02 to
0f5d5b1
Compare
panyx0718
approved these changes
Apr 24, 2018
Contributor
Author
|
fix #10122 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There is a lack of synchronization in
reduceSum, so the results of it have some randomness when the thread block excess 256. But the reason of why randomness doesn't happen when thread block is less than 256 is not clear.