[tflite] use newer xnnpack related source for M1 bazel build

[freedomtan]

with newer related code, we can build benchmark_model with xnnpack, gpu, and coreml delegates.

On M1 machines,

Either

bazel-3.7.2-arm64 tensorflow/lite/tools/benchmark:benchmark_model --config macos_arm64  --macos_cpus arm64

or

bazel-4.0-arm64 tensorflow/lite/tools/benchmark:benchmark_model --macos_cpus arm64

works.

[freedomtan]

MobileNet V1 1.0 224 inference latency

unit: ms

how	average latency
CPU 1xthread	18.847
CPU XNNPACK 1xthread	29.219
CPU 4xthreads	9.087
CPU XNNPACK 4xthreads	9.079
CPU + Accelerate	5.397
GPU delegate	2.699
CoreML delegate	1.051

c.f. #47605

XNNPACK numbers are not better than non-XNNPACK ones? @Maratyszcza

[freedomtan]

@terryheo With this patch,
I can build arm64 binary on a mac x86_64 machine.

bazel-3.7.2-x86_64 tensorflow/lite/tools/benchmark:benchmark_model --config macos_arm64  --macos_cpus arm64

[abattery]

@terryheo @yyoon @teijeong could you review this PR?

[Maratyszcza]

@freedomtan I suspect TFLite might be calling into Accelerate on Mac. Accelerate uses AMX accelerator, which is not documented, and thus not used in XNNPACK.

[freedomtan]

@Maratyszcza NO, those numbers are not AMX/Accelerate numbers. Accelerate is not enabled (yet) when building bazel.

Inception V3 numbers look more reasonable. I also updated the MobileNet V1 table with Accelerate number.

Inception V3 float from tflite hosted model

how	average latency
CPU 1xthread	183.136
CPU XNNPACK 1xthread	159.613
CPU 2xthreads	103.406
CPU XNNPACK 2xthreads	85.646
CPU 4xthreads	64.729
CPU XNNPACK 4xthreads	46.066
CPU + Accelerate	31.743
GPU delegate	13.449
CoreML delegate	2.834

CPU + Accelerate:

bazel build //tensorflow/lite/tools/benchmark:benchmark_model --macos_cpus=arm64 --copt=-DTF_LITE_USE_CBLAS

and -framework Accelerate is added to benchmark_model's BUILD file.

[freedomtan]

Using cblas from Accelerate for convolution could be enabled on M1 machines with something like the following

diff --git a/tensorflow/lite/kernels/internal/BUILD b/tensorflow/lite/kernels/internal/BUILD
index d1b0505de90..7bac11d8fb6 100644
--- a/tensorflow/lite/kernels/internal/BUILD
+++ b/tensorflow/lite/kernels/internal/BUILD
@@ -286,7 +286,10 @@ cc_library(
         "optimized/sparse_ops/fully_connected.h",
     ],
     compatible_with = get_compatible_with_portable(),
-    copts = tflite_copts(),
+    copts = tflite_copts() + select({
+        "//tensorflow:macos_arm64": ["-DTF_LITE_USE_CBLAS"],
+        "//conditions:default": [],
+    }),
     deps = [
         ":common",
         ":compatibility",
@@ -307,6 +310,13 @@ cc_library(
         "@gemmlowp//:fixedpoint",
         "@ruy//ruy/profiler:instrumentation",
     ],
+    linkopts = select({
+        "//tensorflow:macos_arm64": [
+            "-framework Accelerate",
+        ],
+        "//conditions:default": [],
+    }),
+
 )

[terryheo]

LGTM