Skip to content

Improve test pyramid: strengthen unit tests alongside integration tests #1699

@dgenio

Description

@dgenio

Description

Summary

The current test suite for the Python SDK appears to lean heavily on integration tests:

  • many tests spin up full servers/clients,
  • numerous tests live under tests/issues/ as end-to-end regressions.

These are valuable, but they make it harder to:

  • get fast feedback from focused unit tests,
  • refactor internal components without breaking many tests at once,
  • pinpoint which component is responsible when a regression occurs.

Proposal

  1. Identify core units suitable for isolated testing

    Examples might include:

    • schema generation and func_metadata() helpers,
    • message parsing/serialization,
    • specific error mapping logic,
    • pure utility modules.
  2. Add targeted unit tests

    • For each such module, add tests that do not require spinning up a full session or transport.
    • Use mocks/fakes at the boundaries to keep tests small and fast.
  3. Keep integration tests, but make them more focused

    • Retain high-value end-to-end tests, especially for bug regressions and cross-component behavior.
    • When possible, scope regression tests to the minimal surface needed to reproduce the bug.

Why this matters

  • Developer velocity: Quicker feedback when changing internals.
  • Refactorability: Internal improvements (e.g., transport abstraction, session state machine) become safer.
  • Signal quality: When tests fail, it’s easier to see which component is responsible.

Acceptance criteria

  • Key modules (e.g., schema generation, error mapping) have unit tests that do not depend on full server/client setup.
  • Integration tests still cover end-to-end behavior and known regressions.
  • New contributions are encouraged to add unit tests where appropriate, not only integration tests.

References

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions