TST: Handle external OPeNDAP outages in network tests

[senanurcetin]

Description

This PR improves the reliability of xarray's PyDAP network integration tests when the external OPeNDAP test server (test.opendap.org) is temporarily unavailable.

Problem

Some network tests depend on datasets hosted by test.opendap.org.

When the external server returns HTTP 503 responses or refuses connections, these tests fail with urllib3.exceptions.MaxRetryError and requests.exceptions.RetryError, even when the xarray implementation is unchanged.

This behavior was observed in the Python 3.14 CI job associated with #11713, where six backend tests failed because of external OPeNDAP connection errors.

xref: #11718

Changes

• Add availability checks for the external OPeNDAP test endpoints.
• Handle HTTP 502, 503, and 504 responses as temporary service outages.
• Handle connection failures, DNS resolution errors, and timeouts.
• Skip affected network tests with an explicit explanation when the external service is unavailable.
• Preserve failures for unexpected HTTP responses, including 401, 404, and 500, and TLS certificate errors.
• Check the actual DAP2 and DAP4 metadata endpoints used by the tests.
• Check all three remote DataTree fixtures used by the affected integration tests.
• Add unit tests for successful responses, temporary outages, unexpected errors, and endpoint selection.

Scope

The changes are limited to four test files:

• xarray/tests/backends/conftest.py
• xarray/tests/backends/test_pydap.py
• xarray/tests/backends/test_datatree.py
• xarray/tests/backends/test_opendap_availability.py

No public APIs or production backend implementations are modified.

Testing

The added unit tests cover the availability checks and their handling of transient versus unexpected failures.

The complete xarray test suite and the Python 3.14 CI job have not yet been independently verified against this branch.

The availability checks are intended to reduce failures caused by external service outages. They do not guarantee that every possible network interruption will be detected.

Related PR

Related to #11713, which encountered six OPeNDAP-related CI failures. This is a separate infrastructure improvement and does not change the documentation contribution in that PR.

AI Disclosure

OpenAI ChatGPT assisted with preparing the implementation, test cases, and initial documentation for this contribution.

The contributor is responsible for independently reviewing, understanding, and validating the final changes.

[headtr1ck]

As discussed here: #11660 (comment) this is currently by design.
These tests are part of the flaky tests, so they are not required to pass to be able to merge a PR.

So I think this direction is not the correct one, ideally we would make yourself independent of pydap test servers.

[headtr1ck]

Btw, I reported the outage here: OPENDAP/test.opendap.org#8 but nobody seems to be looking at that, no idea where the correct place is to ask for help.

[senanurcetin]

@headtr1ck Thanks for clarifying. I agree that skipping these tests when the server is unavailable could hide a persistent problem, especially since the flaky tests are already non-blocking.

I'll reconsider the approach and investigate using local, self-contained PyDAP fixtures instead of relying on the external test server. I also understand the suggestion to move protocol-level integration coverage upstream to PyDAP.

Would replacing the external dependency with local fixtures be an appropriate direction for a revised PR?

[dcherian]

Btw, I reported the outage here: OPENDAP/test.opendap.org#8

cc @Mikejmnez

[headtr1ck]

Forgot to mention that I created #11718.
We should discuss the approach first.