Skip to content

Commit 5cef2fd

Browse files
billmguofacebook-github-bot
authored andcommitted
Back out "Do not use BNNS copy when dtypes differ in CoreML (pytorch#13018)"
Summary: the diff D79416945 make the model inference slow 1. in old 08/01 build runner on Mac , P1905141721 Prefilled 18 tokens @ 250 tokens/second. Generated 23 tokens @ 18.4 tokens/second. 2. in today 0814 build runner, on Mac, P1905142300 refilled 18 tokens @ 36.5112 token/s in 493ms Generated 23 tokens @ 2.25734 token/s in 10189ms Differential Revision: D80362730
1 parent 8e208ad commit 5cef2fd

File tree

1 file changed

+0
-3
lines changed

1 file changed

+0
-3
lines changed

backends/apple/coreml/runtime/delegate/multiarray.mm

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -123,9 +123,6 @@ bool init_bnns_descriptor(BNNSNDArrayDescriptor& bnns_descriptor, const MultiArr
123123
}
124124

125125
bool copy_using_bnns(const MultiArray& src, MultiArray& dst) {
126-
if (src.layout().dataType() != dst.layout().dataType()) {
127-
return false;
128-
}
129126
if (dst.layout().num_bytes() < src.layout().num_bytes()) {
130127
return false;
131128
}

0 commit comments

Comments
 (0)