Skip to content

How to reconnect after "service unavailable" error and grpc stream end #52

@alice-watershed

Description

@alice-watershed

Hey @pozil,

We're periodically encountering grpc status messages like:

{"code":13,"details":"The service is unavailable. Try again later. rpcId: 45b0e87b-e00f-48c5-94cf-34a043854962","metadata":{"error-code":["sfdc.platform.eventbus.grpc.service.unavailable"],"rpc-id":["45b0e87b-e00f-48c5-94cf-34a043854962"],"type":["Subscribe"]}}

This will happen on one subscription topic (we are currently using 3 subscriptions to subscribe to 3 platform events), and then immediately we will get an "end" message and that one subscription topic will have its grpc stream closed. However, we want always-running subscriptions so we currently handle unexpected disconnections by reconnecting using the same PubSubApi client.

// received an end message
client.close();
// wait for 10 sec
await client.connect()
await client.subscribeFromReplayId(topic 1)
await client.subscribeFromReplayId(topic 2)
await client.subscribeFromReplayId(topic 3)

However, this results in us receiving duplicate messages with the same replayId. For example, if we get the "service unavailable" error message 5 times in a row, then once we've closed and reconnected the client 5 times we'll start receiving 5 events with replayId 500 and then 5 events with replayId 501, and so on.

Is there a better way to handle these unexpected service unavailable messages resulting in closed grpc streams?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions