Skip to content

Implement Synapse to MAS migration #454

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 56 commits into from
Jun 6, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
1c8e8ad
init syn2mas work
gaelgatelement May 16, 2025
41d8aa0
add syn2mas manifests
gaelgatelement May 16, 2025
a011bbb
document how to use preMigrationSynapseHandlesAuth
gaelgatelement May 16, 2025
f5c5716
add syn2mas fixture
gaelgatelement May 16, 2025
9a49719
mas: add readyToHandleAuth case
gaelgatelement May 19, 2025
bbc5d94
matrix-tools: add syn2mas capabilities
gaelgatelement May 19, 2025
6c2c2dd
add syn2mas role binding
gaelgatelement May 19, 2025
2e5dd41
syn2mas: pass templating
gaelgatelement May 19, 2025
06782a3
validate syn2mas manifests
gaelgatelement May 19, 2025
557ad69
pass manifests tests
gaelgatelement May 19, 2025
b0e1a37
run integration tests on syn2mas job
gaelgatelement May 19, 2025
3421b68
run job but fail as DB is missing
gaelgatelement May 19, 2025
1bc868f
synapse: only configure rendez vous if mas is ready to handle auth
gaelgatelement May 20, 2025
30d3ed6
syn2mas values: configure password scheme
gaelgatelement May 20, 2025
8f90a10
syn2mas job: dryRun as post-hook, actual run as pre-hook
gaelgatelement May 20, 2025
438195d
dropme: build matrix tools in tests
gaelgatelement May 20, 2025
4b9a19d
tests: verify that access tokens work after migration
gaelgatelement May 20, 2025
5038143
matrix-tools: log more during upgrade
gaelgatelement May 20, 2025
d164e51
syn2mas: run the actual migration as a pre-upgrade hook and enable in…
gaelgatelement May 20, 2025
64283cc
run MAS Syn2Mas test manifests
gaelgatelement May 20, 2025
204a9fd
fix postgres labels
gaelgatelement May 20, 2025
8d3746e
fix job indentation
gaelgatelement May 20, 2025
9b0e9b5
fix python lint
gaelgatelement May 20, 2025
9e61071
fix copyright dates
gaelgatelement May 20, 2025
db18391
add newsfragments
gaelgatelement May 20, 2025
0e19a26
fix missing extraEnv in syn2mas
gaelgatelement May 20, 2025
d2cef8a
test syn2mas manifests without the ingress tests
gaelgatelement May 20, 2025
6e82b92
test syn2mas in secrets
gaelgatelement May 20, 2025
c67ab6c
fix syn2mas job idempotency
gaelgatelement May 20, 2025
6e80463
fix labels on syn2mas job
gaelgatelement May 20, 2025
e80b745
syn2mas job: enhance synapse & mas context usage
gaelgatelement May 20, 2025
e5dcc82
scripts lint
gaelgatelement May 20, 2025
8ce0f85
syn2mas: use new render-config helpers
gaelgatelement May 27, 2025
6e5edaf
fix tests
gaelgatelement Jun 3, 2025
80cd5e9
fix synapse render-config overrides
gaelgatelement Jun 3, 2025
bd62299
index secret based on hash of their names
gaelgatelement Jun 3, 2025
64f89b3
fix tests
gaelgatelement Jun 3, 2025
719e060
Drop required image tag
gaelgatelement Jun 3, 2025
14a1ace
rename syn2mas manifests
gaelgatelement Jun 4, 2025
2e13b7d
mas: fix hook name
gaelgatelement Jun 4, 2025
fdfcd70
Add deployment markers for syn2mas
gaelgatelement Jun 4, 2025
4b5c03c
tests: add a deploy_with_values_patch helper
gaelgatelement Jun 4, 2025
dab606a
add a helper to get dpeloyment marker
gaelgatelement Jun 4, 2025
6fc4a04
update comment about hook weights in syn2mas
gaelgatelement Jun 5, 2025
8b7c301
syn2mas job: fix worker name
gaelgatelement Jun 5, 2025
ffff805
Update syn2mas comment with marker description
gaelgatelement Jun 5, 2025
72a11b5
syn2mas: enable ingress + read only mode
gaelgatelement Jun 5, 2025
514e72b
syn2mas: check auth_metadata
gaelgatelement Jun 5, 2025
6569b3b
add syn2mas docs
gaelgatelement Jun 5, 2025
7640544
syn2mas: test that we can create a mas user after the migration
gaelgatelement Jun 5, 2025
af37111
syn2mas roles: only needed when actually migrating
gaelgatelement Jun 5, 2025
e4f78e1
syn2mas: use --reuse-values in docs
gaelgatelement Jun 5, 2025
a1dea02
syn2mas: make sure we cant migrate twice with the deployment markers
gaelgatelement Jun 5, 2025
d3c1474
matrix-tools: test syn2mas args
gaelgatelement Jun 5, 2025
8715597
docs: mention synapse & mas password scheme configuration
gaelgatelement Jun 5, 2025
36658e4
matrix-tools: update to 0.5.0
gaelgatelement Jun 5, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# Copyright 2025 New Vector Ltd
#
# SPDX-License-Identifier: AGPL-3.0-only

matrixAuthenticationService:
additional:
password-scheme.yml:
config: |
passwords:
schemes:
- version: 1
algorithm: bcrypt
- version: 2
algorithm: argon2id
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,11 @@ matrixAuthenticationService:
configSecret: "{{ $.Release.Name }}-pytest-admin"
configSecretKey: "admin.yaml"

syn2mas:
extraEnv:
- name: DEBUG_RENDERING
value: "1"

postgres:
podSecurityContext:
runAsGroup: 0
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# Copyright 2025 New Vector Ltd
#
# SPDX-License-Identifier: AGPL-3.0-only

matrixAuthenticationService:
syn2mas:
enabled: true
dryRun: true

additional:
password-scheme.yml:
config: |
passwords:
schemes:
- version: 1
algorithm: bcrypt
- version: 2
algorithm: argon2id
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# Copyright 2025 New Vector Ltd
#
# SPDX-License-Identifier: AGPL-3.0-only

matrixAuthenticationService:
syn2mas:
enabled: true
dryRun: false

additional:
password-scheme.yml:
config: |
passwords:
schemes:
- version: 1
algorithm: bcrypt
- version: 2
algorithm: argon2id

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,22 @@
#
# SPDX-License-Identifier: AGPL-3.0-only
#
# source_fragments: matrix-authentication-service-minimal.yaml matrix-authentication-service-postgres.yaml matrix-authentication-service-postgres-secrets-externally.yaml matrix-authentication-service-secrets-externally.yaml synapse-minimal.yaml synapse-postgres.yaml synapse-postgres-secrets-externally.yaml synapse-secrets-externally.yaml
# source_fragments: matrix-authentication-service-minimal.yaml matrix-authentication-service-postgres.yaml matrix-authentication-service-postgres-secrets-externally.yaml matrix-authentication-service-secrets-externally.yaml synapse-minimal.yaml synapse-postgres.yaml synapse-postgres-secrets-externally.yaml synapse-secrets-externally.yaml matrix-authentication-service-syn2mas-dryrun.yaml
# DO NOT EDIT DIRECTLY. Edit the fragment files to add / modify / remove values

# initSecrets, postgres don't have any required properties to be set and defaults to enabled
elementWeb:
enabled: false
matrixAuthenticationService:
additional:
password-scheme.yml:
config: |
passwords:
schemes:
- version: 1
algorithm: bcrypt
- version: 2
algorithm: argon2id
encryptionSecret:
secret: '{{ $.Release.Name }}-mas-external'
secretKey: encryption
Expand All @@ -34,6 +43,9 @@ matrixAuthenticationService:
rsa:
secret: '{{ $.Release.Name }}-mas-external'
secretKey: keysRSA
syn2mas:
dryRun: true
enabled: true
synapseOIDCClientSecret:
secret: '{{ $.Release.Name }}-mas-external'
secretKey: synapseOIDC
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,22 @@
#
# SPDX-License-Identifier: AGPL-3.0-only
#
# source_fragments: matrix-authentication-service-minimal.yaml matrix-authentication-service-postgres.yaml matrix-authentication-service-postgres-secrets-in-helm.yaml matrix-authentication-service-secrets-in-helm.yaml synapse-minimal.yaml synapse-postgres.yaml synapse-postgres-secrets-in-helm.yaml synapse-secrets-in-helm.yaml
# source_fragments: matrix-authentication-service-minimal.yaml matrix-authentication-service-postgres.yaml matrix-authentication-service-postgres-secrets-in-helm.yaml matrix-authentication-service-secrets-in-helm.yaml synapse-minimal.yaml synapse-postgres.yaml synapse-postgres-secrets-in-helm.yaml synapse-secrets-in-helm.yaml matrix-authentication-service-syn2mas-dryrun.yaml
# DO NOT EDIT DIRECTLY. Edit the fragment files to add / modify / remove values

# initSecrets, postgres don't have any required properties to be set and defaults to enabled
elementWeb:
enabled: false
matrixAuthenticationService:
additional:
password-scheme.yml:
config: |
passwords:
schemes:
- version: 1
algorithm: bcrypt
- version: 2
algorithm: argon2id
encryptionSecret:
value: CHANGEME-ahohhohgiavee5Koh8ahwo
ingress:
Expand Down Expand Up @@ -42,6 +51,9 @@ matrixAuthenticationService:
-----BEGIN RSA PRIVATE KEY-----
MIIEowIBAAKCAQEA6521bYjZ789034nLz+oXJyVWqgUdDmRlKxvTfHsBhFtGpOaAoGCCqGSM49AwEHoUQDQgAE6521bYjZ789034nLz+oXJyVWqgUdDmRlKxvTfHsBhFtGpOaAoGCCqGSM49
------END RSA PRIVATE KEY-----
syn2mas:
dryRun: true
enabled: true
synapseOIDCClientSecret:
value: CHANGEME-eiv6wae8shooPhie4ief8ru2egahbah0
synapseSharedSecret:
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
# Copyright 2024-2025 New Vector Ltd
#
# SPDX-License-Identifier: AGPL-3.0-only
#
# source_fragments: matrix-authentication-service-minimal.yaml matrix-authentication-service-postgres.yaml matrix-authentication-service-postgres-secrets-externally.yaml matrix-authentication-service-secrets-externally.yaml synapse-minimal.yaml synapse-postgres.yaml synapse-postgres-secrets-externally.yaml synapse-secrets-externally.yaml matrix-authentication-service-syn2mas-dryrun.yaml matrix-authentication-service-syn2mas-migrate.yaml
# DO NOT EDIT DIRECTLY. Edit the fragment files to add / modify / remove values

# initSecrets, postgres don't have any required properties to be set and defaults to enabled
elementWeb:
enabled: false
matrixAuthenticationService:
additional:
password-scheme.yml:
config: |
passwords:
schemes:
- version: 1
algorithm: bcrypt
- version: 2
algorithm: argon2id
encryptionSecret:
secret: '{{ $.Release.Name }}-mas-external'
secretKey: encryption
ingress:
host: mas.ess.localhost
postgres:
database: mas
host: postgres
password:
secret: '{{ $.Release.Name }}-mas-external'
secretKey: postgresPassword
user: mas
privateKeys:
ecdsaPrime256v1:
secret: '{{ $.Release.Name }}-mas-external'
secretKey: keysEcdsaPrime256v1
ecdsaSecp256k1:
secret: '{{ $.Release.Name }}-mas-external'
secretKey: keysEcdsaSecp256k1
ecdsaSecp384r1:
secret: '{{ $.Release.Name }}-mas-external'
secretKey: keysEcdsaSecp384r1
rsa:
secret: '{{ $.Release.Name }}-mas-external'
secretKey: keysRSA
syn2mas:
dryRun: false
enabled: true
synapseOIDCClientSecret:
secret: '{{ $.Release.Name }}-mas-external'
secretKey: synapseOIDC
synapseSharedSecret:
secret: '{{ $.Release.Name }}-mas-external'
secretKey: synapseShared
matrixRTC:
enabled: false
serverName: ess.localhost
synapse:
appservices:
- secret: '{{ $.Release.Name }}-synapse-external'
secretKey: bridge_registration.yaml
ingress:
host: synapse.ess.localhost
macaroon:
secret: '{{ $.Release.Name }}-synapse-external'
secretKey: macaroon
postgres:
database: synapse
host: ess-postgres
password:
secret: '{{ $.Release.Name }}-synapse-external'
secretKey: postgresPassword
user: synapse_user
registrationSharedSecret:
secret: '{{ $.Release.Name }}-synapse-external'
secretKey: registrationSharedSecret
signingKey:
secret: '{{ $.Release.Name }}-synapse-external'
secretKey: signingKey
wellKnownDelegation:
enabled: false
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
# Copyright 2024-2025 New Vector Ltd
#
# SPDX-License-Identifier: AGPL-3.0-only
#
# source_fragments: matrix-authentication-service-minimal.yaml matrix-authentication-service-postgres.yaml matrix-authentication-service-postgres-secrets-in-helm.yaml matrix-authentication-service-secrets-in-helm.yaml synapse-minimal.yaml synapse-postgres.yaml synapse-postgres-secrets-in-helm.yaml synapse-secrets-in-helm.yaml matrix-authentication-service-syn2mas-dryrun.yaml matrix-authentication-service-syn2mas-migrate.yaml
# DO NOT EDIT DIRECTLY. Edit the fragment files to add / modify / remove values

# initSecrets, postgres don't have any required properties to be set and defaults to enabled
elementWeb:
enabled: false
matrixAuthenticationService:
additional:
password-scheme.yml:
config: |
passwords:
schemes:
- version: 1
algorithm: bcrypt
- version: 2
algorithm: argon2id
encryptionSecret:
value: CHANGEME-ahohhohgiavee5Koh8ahwo
ingress:
host: mas.ess.localhost
postgres:
database: mas
host: postgres
password:
value: CHANGEME-ooWo6jeidahhei3Hae0eer9U
user: mas
privateKeys:
ecdsaPrime256v1:
value: |
-----BEGIN EC PRIVATE KEY-----
MHcCAQEEIYjZ789034nLz+oXJyVWqgUdDmRlKxvTfHsBhFtGpOaAoGCCqGSM49
AwEHoUQDQgAE6521bYjZ789034nLz+oXJyVWqgUdDmRlKxvTfHsBhFtGpOaAoGCCqGSM49
AwEAAKBcZW5jb2duZXQwgggYMIINL6Ado018734nLz+oXJyVWqgUdDmRlKxvTfHsBhFtGpOaAoGCCqGSM49AwEH
------END EC PRIVATE KEY-----
ecdsaSecp256k1:
value: |
-----BEGIN EC PRIVATE KEY-----
MHcCAQEEZFQZ789034nLz+oXJyVWqgUdDmRlKxvTfHsBhFtGpOaAoGCCqGSM49AwEHoUQDQgAE6521bYjZ789034nLz+oXJyVWqgUdDmRlKxvTfHsBhFtGpOaAoGCCqGSM49
------END EC PRIVATE KEY-----
ecdsaSecp384r1:
value: |
-----BEGIN EC PRIVATE KEY-----
MHcCAQEEZFQZ789034nLz+oXJyVWqgUdDmRlKxvTfHsBhFtGpOaAoGCCqGSM49AwEHoUQDQgAE6521bYjZ789034nLz+oXJyVWqgUdDmRlKxvTfHsBhFtGpOaAoGCCqGSM49
------END EC PRIVATE KEY-----
rsa:
value: |
-----BEGIN RSA PRIVATE KEY-----
MIIEowIBAAKCAQEA6521bYjZ789034nLz+oXJyVWqgUdDmRlKxvTfHsBhFtGpOaAoGCCqGSM49AwEHoUQDQgAE6521bYjZ789034nLz+oXJyVWqgUdDmRlKxvTfHsBhFtGpOaAoGCCqGSM49
------END RSA PRIVATE KEY-----
syn2mas:
dryRun: false
enabled: true
synapseOIDCClientSecret:
value: CHANGEME-eiv6wae8shooPhie4ief8ru2egahbah0
synapseSharedSecret:
value: CHANGEME-iaw8eeSef4zeefie8ii3akien9tiaYah
matrixRTC:
enabled: false
serverName: ess.localhost
synapse:
ingress:
host: synapse.ess.localhost
macaroon:
value: CHANGEME-eek3Eigoh8ux8laeTingeej1
postgres:
database: synapse
host: ess-postgres
password:
value: CHANGEME-ooWo6jeidahhei3Hae0eer9U
user: synapse_user
registrationSharedSecret:
value: CHANGEME-ooWo6jeidahhei3Hae0eer9U
signingKey:
value: ed25519 0 bNQOzBUDszff7Ax81z6w0uZ1IPWoxYaazT7emaZEfpw
wellKnownDelegation:
enabled: false
Loading
Loading