Skip to content

A/B comparison command for meta-harness configs #10

@canivel

Description

@canivel

Problem

Users run baseline vs experimental configs manually, compare manually.

Proposed Solution

kaos mh compare --config-a baseline --config-b ev_voting --benchmark aimo3

Runs both, scores both, outputs comparison table with statistical significance.

Reported by AI agent using KAOS v0.3.0

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Nice to haveai-reportedIssue reported by an AI agent using KAOSenhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions