Support arm64

[vova-lantsov-dev]

Summary

Support for arm64 processors (including arm64v8).

Motivation

Users can run Sentry on mini PCs like Raspberry Pi or Asus Tinker [edit: as well as on Apple M1s].

Additional Context

There are 13 images used in official Sentry docker-compose configuration.
4 images support arm64 officially:

• memcached
• redis
• postgres
• nginx.

5 images do not support arm64 at all:

• smtp (tianon/exim4)
• geoipupdate (maxmindinc/geoipupdate:latest)
• snuba
• symbolicator
• relay

4 images support arm64 unofficially:

• cp-zookeeper (watershine/cp-zookeeper:6.1.0-arm64) - only 6.1.0 version
• cp-kafka (watershine/cp-kafka:6.1.0-arm64) - only 6.1.0 version
• clickhouse-server (altinity/clickhouse-server:20.10.1.4844-testing-arm) - looks like a test unstable build, that has 20.10.x version instead of 20.3.9.70 suggested by your config
• sentry (daprio/sentry:latest) - I guess this is 1.1.0 version of Sentry

[chadwhitacre]

geoipupdate should (could) be optional, and probably smtp as well. Gonna need snuba, symbolicator, and relay, however. Fortunately those are Sentry-owned, so theoretically easier to achieve arm64 support there. Version 1.1.0 of Sentry is hecka ancient. Sure that's right?

[vova-lantsov-dev]

Not familiar with Sentry versioning, maybe you're right :) Maybe I am wrong, because this docker hub repo is actively supported and gets updates regularly. Anyway, sentry service is supported by Sentry team as well, it would be good to have official arm64 builds for it too.

[BYK]

As much as I find being able to use arm64 on production fascinating, I don't think we can commit to support this for the time being due to following reasons:

1. Our main cloud provider, Google Cloud Platform, does not support arm64 architecture so we don't have any incentives or real-world testing ability for this architecture
2. Our preferred CI platform, GitHub Actions, does not support arm64 either so even if we implemented something, testing these continously would be a challenge.

[ezhevita]

I've created patches for on-premise deploy and all related docker images, it may work while Sentry hasn't introduced official support yet: https://github.com/Sentry-ARM/onpremise

[qiutian00]

I've created patches for on-premise deploy and all related docker images, it may work while Sentry hasn't introduced official support yet: https://github.com/Sentry-ARM/onpremise

Great!

[TuringKi]

I have

I've created patches for on-premise deploy and all related docker images, it may work while Sentry hasn't introduced official support yet: https://github.com/Sentry-ARM/onpremise

Great works!
I have maken some changes adapted to some arm64 cloud server (e.g. huawei's cloud server):
https://github.com/TuringKi/sentry_server_arm64

[chadwhitacre]

Why are we directing M1-related tickets (#1006 #1255) here? This ticket is about arm64, and from the description I thought that meant Raspberry Pis and such, not Macs—I don't see "arm64" mentioned on https://en.wikipedia.org/wiki/Apple_M1. What am I missing? 🐭

[stsandro]

Why are we directing M1-related tickets (#1006 #1255) here? This ticket is about arm64, and from the description I thought that meant Raspberry Pis and such, not Macs—I don't see "arm64" mentioned on https://en.wikipedia.org/wiki/Apple_M1. What am I missing? 🐭

Well, technically the M1 is an arm64 (AArch64) architecture. Maybe some people would like to run Sentry on their M1 Mac minis either in the cloud (AWS, Hetzner) or in their own basement.

[vova-lantsov-dev]

@chadwhitacre you're missing that Apple Silicon (M1, M1 Pro, M1 Max) chips are based on arm64 architecture

[vova-lantsov-dev]

@chadwhitacre on note, arm64 is the same as arm64/v8, referring to your Wiki page:
Instruction set: ARMv8.5-A

[chadwhitacre]

Thanks for setting me straight @vova-lantsov-dev @stsandro. I've edited the ticket description to note M1s.

Is the "Additional Context" section in the description still accurate? My understanding is that Sentry proper runs on M1 now (we made a big push internally [python, docker] to get that done to unblock laptop upgrades for devs), so I would expect snuba, symbolicator, relay, and sentry to "support arm64 officially." No?

If so I believe that leaves only upstream, non-Sentry components, yes?

[muratcorlu]

@chadwhitacre When I run ./install.sh in my M1 Mac Mini, I see this error:

FAIL: The CPU your machine is running on does not support the SSE 4.2 instruction set, which is required for one of the services Sentry uses (Clickhouse). See https://git.io/JvLDt for more info.

So apparently, Clickhouse is one of the blockers here.

[BYK]

@muratcorlu there's a solution to that: getsentry/sentry#29081 (not ideal but works)

[armenzg]

Would you be able to try swapping this

self-hosted/docker-compose.yml

Line 189 in 28d8928

image: "yandex/clickhouse-server:20.3.9.70"

for altinity/clickhouse-server:20.10.1.4844-testing-arm?

Eventually, we hope one of the two companies to produce both architectures of the image and require no changes on your side.

[chadwhitacre]

lolsob I got a new laptop and now I'm in this boat

[chadwhitacre]

Against e3d0801:

diff --git a/docker-compose.yml b/docker-compose.yml
index b01e8e3..aabbfd5 100644
--- a/docker-compose.yml
+++ b/docker-compose.yml
@@ -186,7 +186,8 @@ services:
       test: ["CMD-SHELL", "nc -z localhost 9092"]
   clickhouse:
     <<: *restart_policy
-    image: "yandex/clickhouse-server:20.3.9.70"
+    # image: "yandex/clickhouse-server:20.3.9.70"
+    image: "altinity/clickhouse-server:20.10.1.4844-testing-arm"
     ulimits:
       nofile:
         soft: 262144
diff --git a/install/check-minimum-requirements.sh b/install/check-minimum-requirements.sh
index b05952a..f80bc1b 100644
--- a/install/check-minimum-requirements.sh
+++ b/install/check-minimum-requirements.sh
@@ -38,13 +38,13 @@ fi
 
 #SSE4.2 required by Clickhouse (https://clickhouse.yandex/docs/en/operations/requirements/)
 # On KVM, cpuinfo could falsely not report SSE 4.2 support, so skip the check. https://github.com/ClickHouse/ClickHouse/issues/20#issuecomment-226849297
-IS_KVM=$(docker run --rm busybox grep -c 'Common KVM processor' /proc/cpuinfo || :)
-if [[ "$IS_KVM" -eq 0 ]]; then
-  SUPPORTS_SSE42=$(docker run --rm busybox grep -c sse4_2 /proc/cpuinfo || :)
-  if [[ "$SUPPORTS_SSE42" -eq 0 ]]; then
-    echo "FAIL: The CPU your machine is running on does not support the SSE 4.2 instruction set, which is required for one of the services Sentry uses (Clickhouse). See https://git.io/JvLDt for more info."
-    exit 1
-  fi
-fi
+#IS_KVM=$(docker run --rm busybox grep -c 'Common KVM processor' /proc/cpuinfo || :)
+#if [[ "$IS_KVM" -eq 0 ]]; then
+#  SUPPORTS_SSE42=$(docker run --rm busybox grep -c sse4_2 /proc/cpuinfo || :)
+#  if [[ "$SUPPORTS_SSE42" -eq 0 ]]; then
+#    echo "FAIL: The CPU your machine is running on does not support the SSE 4.2 instruction set, which is required for one of the services Sentry uses (Clickhouse). See https://git.io/JvLDt for more info."
+#    exit 1
+#  fi
+#fi
 
 echo "${_endgroup}"

Results:

Container sentry-self-hosted-zookeeper-1  Creating
Container sentry-self-hosted-redis-1  Creating
Container sentry-self-hosted-clickhouse-1  Creating
Container sentry-self-hosted-redis-1  Created
Container sentry-self-hosted-zookeeper-1  Created
Container sentry-self-hosted-kafka-1  Creating
Container sentry-self-hosted-clickhouse-1  Created
Container sentry-self-hosted-kafka-1  Created
Container sentry-self-hosted-zookeeper-1  Starting
Container sentry-self-hosted-redis-1  Starting
Container sentry-self-hosted-clickhouse-1  Starting
Container sentry-self-hosted-zookeeper-1  Started
Container sentry-self-hosted-zookeeper-1  Waiting
Container sentry-self-hosted-clickhouse-1  Started
Container sentry-self-hosted-redis-1  Started
Container sentry-self-hosted-zookeeper-1  Healthy
Container sentry-self-hosted-kafka-1  Starting
Container sentry-self-hosted-kafka-1  Started
container for service "clickhouse" is unhealthy
An error occurred, caught SIGERR on line 3
Cleaning up...

Full

[chadwhitacre]

Looks like ClickHouse shipped a compatible Docker image week before last.

Trying it out ...

[chadwhitacre]

First snag is "Application: Syntax error: Not a valid floating-point number" tracked to:

self-hosted/clickhouse/config.xml

Line 2 in 6abdcfe

<max_server_memory_usage_to_ram_ratio from_env="MAX_MEMORY_USAGE_RATIO" />

Not sure what's up there, if I specify 0.3 directly rather than going through the envvar then the error goes away so I did that for now. ClickHouse config is designed to be layerable so maybe that's a better way to go (support a local config layer vs. an envvar for this one knob).

Second snag is ClickHouse/ClickHouse#15638. Looks like @armenzg worked around this in getsentry/sentry#29284 by using an Altinity build. Why does this new ClickHouse build still exhibit this behavior? 🤔

[chadwhitacre]

We're not going to get a ClickHouse upgrade anytime soon. Maybe we can hack a conditional like we're doing in dev envs? 🧐

[chadwhitacre]

@ethanhs points out that "You can add --platform linux/amd64 to run an Intel image under emulation."

https://docs.docker.com/desktop/mac/apple-silicon/#known-issues

We'd want to still do this conditionally, though.

[emmatyping]

This was fixed in #1538 🎉

Please open a new issue if you run into any problems.