help request: monitor websocket connection status in apisix

[a1156698162]

Description

We are performing traffic migration, transferring traffic to the APISIX gateway. During the migration process, we noticed a lack of monitoring for WebSocket connection status. How can we monitor the status of WebSocket connections?

Environment

• APISIX version (run apisix version):
• Operating system (run uname -a):
• OpenResty / Nginx version (run openresty -V or nginx -V):
• etcd version, if relevant (run curl http://127.0.0.1:9090/v1/server_info):
• APISIX Dashboard version, if relevant:
• Plugin runner version, for issues related to plugin runners:
• LuaRocks version, for installation issues (run luarocks --version):

[Baoyuantop]

Could you describe what specific metrics you want to monitor?

[a1156698162]

Could you describe what specific metrics you want to monitor?

The simplest way is that I want to see the number of websocket connections at a certain point in time. How should I do that?

[a1156698162]

Could you describe what specific metrics you want to monitor?

The simplest way is that I want to see the number of websocket connections at a certain point in time. How should I do that?

Additionally, I also want to monitor which requests failed during the websocket protocol upgrade and did not establish a connection correctly.

[a1156698162]

Could you describe what specific metrics you want to monitor?

I have developed a WebSocket metrics exposure plugin myself. Do you think this plugin can be submitted as a feature in a PR?

[Baoyuantop]

The Prometheus plugin has some metrics that can be observed, such as apisix_nginx_http_current_connections and apisix_http_status. You can refer to https://docs.api7.ai/hub/prometheus#metrics

[Baoyuantop]

I have developed a WebSocket metrics exposure plugin myself. Do you think this plugin can be submitted as a feature in a PR?

You can describe the specific implementation method and discuss it in the community.

[a1156698162]

The Prometheus plugin has some metrics that can be observed, such as apisix_nginx_http_current_connections and apisix_http_status. You can refer to https://docs.api7.ai/hub/prometheus#metrics

I think that the current metrics collected by Prometheus cannot calculate the number of active WebSocket connections, as apisix_nginx_http_current_connections includes all connection types.

[a1156698162]

I have developed a WebSocket metrics exposure plugin myself. Do you think this plugin can be submitted as a feature in a PR?

You can describe the specific implementation method and discuss it in the community.

I implemented this mainly through two phases in the plugin lifecycle. During the access phase, I record which requests need to be upgraded to the WebSocket protocol. In the log phase, I track metrics for failed protocol upgrades and disconnected protocols.
The pseudocode is as follows

if prometheus instance exists:
  create ws_opened_counter
  create ws_closed_counter
  create ws_upgrade_failed_counter

function access(config, context):
  get request ID
  
  check if it's a WebSocket upgrade request
  if it's a WebSocket request
    record connection information (host, uri)
    increment ws_opened_counter

function log(config, context):
  get connection information for request ID
  
  if connection information exists
    increment ws_closed_counter
    if status code is not 101
      increment ws_upgrade_failed_counter
    clean up connection information

Simply subtracting ws_opened_counter from ws_closed_counter gives the current state of WebSocket connections. As for why a gauge-type metric isn't used, it's mainly to avoid thread safety issues when calculating the metric.