Unable to update ServerTLSCert and ClientTLSCert for Orderers

[gabrieledm]

I have a Hyperledger Fabric Network composed of 5 Orderer a 4 Peers.
There is only one channel.
The components are deployed on a Kubernetes cluster using an NFS as persistence layer.
The certificates that identifies the Orderer and Peers identities are managed by CertManager installed on Kubernets cluster using Helm.
The certificates has been expired and has been renewd by CertManager automatically but after this updates the network is not working.
I've found on the internet that the network must be updated submitting a channel update containing the new certificates for the orderers.

I tried to do the update following these steps

• Get the last configuration definition from the network
• Decode the fetched file and write into a JSON
• Perform the update, using jq, of
  • .channel_group.groups.Orderer.values.ConsensusType.value.metadata.consenters[ORDERER_INDEX].server_tls_cert and
  • .channel_group.groups.Orderer.values.ConsensusType.value.metadata.consenters[ORDERER_INDEX].client_tls_cert properties
• Encode the original configuration file to proto buffer
• Encode the file with the configuration changes to proto buffer
• Compute the modifications between the original file and the modified one
• Decode the proto buffer file to a JSON
• Generate the JSON file containing the submit proposal for network update
• Re-encode the JSON file to submit it to the network
• Sign the proto buffer file representing the new network configuration update using the Orderer identity
• Send the update to the network for approval

These steps are executed sequentially for each Orderer and run correctly until the 3rd Orderer for which I receive this error

Error: got unexpected status: BAD_REQUEST -- error applying config update to existing channel 'consip-channel': consensus metadata update for channel config update is invalid: 3 out of 5 nodes are alive, configuration will result in quorum loss

From this point the network become unusable because the Orderers are no longer able to communicate with each other

Attached there is the renew-cert-channel-update.zip script I used to perfom the described actions

[denyeart]

Please see the Certificate Management guide:
https://hyperledger-fabric.readthedocs.io/en/latest/certs_management.html#renew-orderer-tls-certificates-without-reusing-the-private-key

The process of recovering from expired certs is much more complicated so it is always best to renew before expiry. Additionally, if you re-use the private key you don't even need a channel update which makes it much easier.