|
| 1 | +# Filter plugin `reportsmerger` |
| 2 | + |
| 3 | +Aggregates N JSON reports into 1 consolidated test report. |
| 4 | +Supports several strategies for different scenarios. |
| 5 | + |
| 6 | +## Usage |
| 7 | + |
| 8 | +```yaml |
| 9 | +--- |
| 10 | +- name: merge files with default strategy into variable my_var |
| 11 | + ansible.builtin.set_fact: |
| 12 | + my_var: "{{ ['f1.json', 'f2.json'] | reportsmerger }}" |
| 13 | + |
| 14 | +- name: merge files with specific strategy and output to file |
| 15 | + ansible.builtin.set_fact: |
| 16 | + result_path: "{{ ['f1.json', 'f2.json'] | reportsmerger(strategy='normal', output='result.json') }}" |
| 17 | +``` |
| 18 | +
|
| 19 | +## Strategies |
| 20 | +
|
| 21 | +The `reportsmerger` filter provides multiple optimization strategies to handle different scenarios efficiently: |
| 22 | + |
| 23 | +### `normal` (Default) |
| 24 | + |
| 25 | +**Use case**: Standard merging for typical file sizes and counts |
| 26 | +**Behavior**: Sequential processing with full data loading and validation |
| 27 | +**Memory usage**: Moderate - loads all files into memory |
| 28 | +**Performance**: Baseline performance, backward compatible |
| 29 | +**Best for**: Small to medium files (< 100MB), moderate file counts (< 50 files) |
| 30 | + |
| 31 | +```yaml |
| 32 | +- name: standard merge |
| 33 | + ansible.builtin.set_fact: |
| 34 | + merged: "{{ report_files | reportsmerger(strategy='normal') }}" |
| 35 | +``` |
| 36 | + |
| 37 | +### `large` |
| 38 | + |
| 39 | +**Use case**: Memory-efficient processing of large individual files |
| 40 | +**Behavior**: Streaming aggregation with immediate memory cleanup |
| 41 | +**Memory usage**: Low - processes files one at a time with explicit cleanup |
| 42 | +**Performance**: Slightly slower due to aggressive cleanup, but memory-safe |
| 43 | +**Best for**: Large files (> 100MB), memory-constrained environments |
| 44 | + |
| 45 | +```yaml |
| 46 | +- name: merge large files efficiently |
| 47 | + ansible.builtin.set_fact: |
| 48 | + merged: "{{ large_report_files | reportsmerger(strategy='large') }}" |
| 49 | +``` |
| 50 | + |
| 51 | +### `many` |
| 52 | + |
| 53 | +**Use case**: Fast processing of many files using parallel I/O |
| 54 | +**Behavior**: Parallel file processing with thread-safe accumulation |
| 55 | +**Memory usage**: Moderate to high - loads multiple files concurrently |
| 56 | +**Performance**: Fastest for I/O-bound scenarios (max 4 threads) |
| 57 | +**Best for**: Many files (> 50), fast storage, sufficient memory |
| 58 | + |
| 59 | +```yaml |
| 60 | +- name: parallel processing of many files |
| 61 | + ansible.builtin.set_fact: |
| 62 | + merged: "{{ many_report_files | reportsmerger(strategy='many') }}" |
| 63 | +``` |
| 64 | + |
| 65 | +### `shallow` |
| 66 | + |
| 67 | +**Use case**: Statistics-focused operations without detailed test suite data |
| 68 | +**Behavior**: Lazy loading of test suites - only loads when accessed |
| 69 | +**Memory usage**: Very low - defers test suite loading until needed |
| 70 | +**Performance**: Fastest for stats-only operations |
| 71 | +**Best for**: Dashboard summaries, quick statistics, when test suite details aren't immediately needed |
| 72 | + |
| 73 | +```yaml |
| 74 | +- name: get quick statistics without loading full test suites |
| 75 | + ansible.builtin.set_fact: |
| 76 | + stats: "{{ report_files | reportsmerger(strategy='shallow') }}" |
| 77 | + |
| 78 | +# Test suites are loaded only when accessed: |
| 79 | +- debug: |
| 80 | + msg: "Total tests: {{ stats.tests }}, Total suites: {{ stats.test_suites | length }}" |
| 81 | +``` |
| 82 | + |
| 83 | +### `complex` |
| 84 | + |
| 85 | +**Use case**: Lightweight merging that excludes heavy test case details |
| 86 | +**Behavior**: Custom JSON parsing that strips detailed test case data during loading |
| 87 | +**Memory usage**: Low - removes test case details, keeps suite metadata |
| 88 | +**Performance**: Good for complex schemas with heavy test case data |
| 89 | +**Best for**: Files with extensive test case details, when only suite-level information is needed |
| 90 | + |
| 91 | +```yaml |
| 92 | +- name: lightweight merge excluding test case details |
| 93 | + ansible.builtin.set_fact: |
| 94 | + lightweight: "{{ complex_report_files | reportsmerger(strategy='complex') }}" |
| 95 | +``` |
| 96 | + |
| 97 | +## Strategy Selection Guide |
| 98 | + |
| 99 | +| Scenario | Strategy | Reason | |
| 100 | +| ------------------------- | --------- | --------------------------------------- | |
| 101 | +| < 50 files, < 100MB each | `normal` | Balanced performance and compatibility | |
| 102 | +| Large files (> 100MB) | `large` | Memory-efficient streaming | |
| 103 | +| Many files (> 50) | `many` | Parallel I/O processing | |
| 104 | +| Stats/dashboards only | `shallow` | Lazy loading for performance | |
| 105 | +| Heavy test case data | `complex` | Strips unnecessary details | |
| 106 | + |
| 107 | +## Output Options |
| 108 | + |
| 109 | +All strategies support writing output directly to a file: |
| 110 | + |
| 111 | +```yaml |
| 112 | +- name: merge and save to file |
| 113 | + ansible.builtin.set_fact: |
| 114 | + result_path: "{{ files | reportsmerger(strategy='large', output='/tmp/merged.json') }}" |
| 115 | + # Returns the file path, not the data |
| 116 | +``` |
0 commit comments