Skip to content

Allow custom chunking strategies (open up the :strategy option) #86

Description

@stuartjohnpage

Background

The README advertises extensibility and TextChunker.ChunkerBehaviour defines the contract for splitting strategies, but the option schema validates :strategy with {:in, [RecursiveChunk]} — so a third-party module implementing the behaviour is rejected by validation. Users cannot actually plug in their own strategy, which contradicts both the behaviour's existence and the README's "Extensibility" feature bullet.

The fix is confined to the option schema in TextChunker.split/2: replace the {:in, ...} validation with a custom NimbleOptions validator that accepts any module implementing the behaviour (e.g. checks function_exported?(mod, :split, 2) after ensuring it is loaded). The strategies themselves need no changes.

Acceptance Criteria

Scenario: Custom strategy module is accepted

  • Given I define a module implementing TextChunker.ChunkerBehaviour
  • When I pass it as strategy: to TextChunker.split/2
  • Then validation passes and my module's split/2 is invoked with the validated options

Scenario: Invalid strategy is rejected with a clear error

  • Given I pass a module that does not implement the behaviour (or a non-module)
  • Then I receive an {:error, message} explaining the module must implement TextChunker.ChunkerBehaviour

Scenario: Documentation matches behavior

  • README and moduledoc describe how to provide a custom strategy

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions