Skip to content

Commit 591b1b2

Browse files
committed
## adding plugin filter reportsmerger + tests
- code + tests + data - usage examples under `doc/` Signed-off-by: Maxim Kovgan <[email protected]>
1 parent 3c9767c commit 591b1b2

14 files changed

+1394
-0
lines changed
Lines changed: 116 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,116 @@
1+
# Filter plugin `reportsmerger`
2+
3+
Aggregates N JSON reports into 1 consolidated test report.
4+
Supports several strategies for different scenarios.
5+
6+
## Usage
7+
8+
```yaml
9+
---
10+
- name: merge files with default strategy into variable my_var
11+
ansible.builtin.set_fact:
12+
my_var: "{{ ['f1.json', 'f2.json'] | reportsmerger }}"
13+
14+
- name: merge files with specific strategy and output to file
15+
ansible.builtin.set_fact:
16+
result_path: "{{ ['f1.json', 'f2.json'] | reportsmerger(strategy='normal', output='result.json') }}"
17+
```
18+
19+
## Strategies
20+
21+
The `reportsmerger` filter provides multiple optimization strategies to handle different scenarios efficiently:
22+
23+
### `normal` (Default)
24+
25+
**Use case**: Standard merging for typical file sizes and counts
26+
**Behavior**: Sequential processing with full data loading and validation
27+
**Memory usage**: Moderate - loads all files into memory
28+
**Performance**: Baseline performance, backward compatible
29+
**Best for**: Small to medium files (< 100MB), moderate file counts (< 50 files)
30+
31+
```yaml
32+
- name: standard merge
33+
ansible.builtin.set_fact:
34+
merged: "{{ report_files | reportsmerger(strategy='normal') }}"
35+
```
36+
37+
### `large`
38+
39+
**Use case**: Memory-efficient processing of large individual files
40+
**Behavior**: Streaming aggregation with immediate memory cleanup
41+
**Memory usage**: Low - processes files one at a time with explicit cleanup
42+
**Performance**: Slightly slower due to aggressive cleanup, but memory-safe
43+
**Best for**: Large files (> 100MB), memory-constrained environments
44+
45+
```yaml
46+
- name: merge large files efficiently
47+
ansible.builtin.set_fact:
48+
merged: "{{ large_report_files | reportsmerger(strategy='large') }}"
49+
```
50+
51+
### `many`
52+
53+
**Use case**: Fast processing of many files using parallel I/O
54+
**Behavior**: Parallel file processing with thread-safe accumulation
55+
**Memory usage**: Moderate to high - loads multiple files concurrently
56+
**Performance**: Fastest for I/O-bound scenarios (max 4 threads)
57+
**Best for**: Many files (> 50), fast storage, sufficient memory
58+
59+
```yaml
60+
- name: parallel processing of many files
61+
ansible.builtin.set_fact:
62+
merged: "{{ many_report_files | reportsmerger(strategy='many') }}"
63+
```
64+
65+
### `shallow`
66+
67+
**Use case**: Statistics-focused operations without detailed test suite data
68+
**Behavior**: Lazy loading of test suites - only loads when accessed
69+
**Memory usage**: Very low - defers test suite loading until needed
70+
**Performance**: Fastest for stats-only operations
71+
**Best for**: Dashboard summaries, quick statistics, when test suite details aren't immediately needed
72+
73+
```yaml
74+
- name: get quick statistics without loading full test suites
75+
ansible.builtin.set_fact:
76+
stats: "{{ report_files | reportsmerger(strategy='shallow') }}"
77+
78+
# Test suites are loaded only when accessed:
79+
- debug:
80+
msg: "Total tests: {{ stats.tests }}, Total suites: {{ stats.test_suites | length }}"
81+
```
82+
83+
### `complex`
84+
85+
**Use case**: Lightweight merging that excludes heavy test case details
86+
**Behavior**: Custom JSON parsing that strips detailed test case data during loading
87+
**Memory usage**: Low - removes test case details, keeps suite metadata
88+
**Performance**: Good for complex schemas with heavy test case data
89+
**Best for**: Files with extensive test case details, when only suite-level information is needed
90+
91+
```yaml
92+
- name: lightweight merge excluding test case details
93+
ansible.builtin.set_fact:
94+
lightweight: "{{ complex_report_files | reportsmerger(strategy='complex') }}"
95+
```
96+
97+
## Strategy Selection Guide
98+
99+
| Scenario | Strategy | Reason |
100+
| ------------------------- | --------- | --------------------------------------- |
101+
| < 50 files, < 100MB each | `normal` | Balanced performance and compatibility |
102+
| Large files (> 100MB) | `large` | Memory-efficient streaming |
103+
| Many files (> 50) | `many` | Parallel I/O processing |
104+
| Stats/dashboards only | `shallow` | Lazy loading for performance |
105+
| Heavy test case data | `complex` | Strips unnecessary details |
106+
107+
## Output Options
108+
109+
All strategies support writing output directly to a file:
110+
111+
```yaml
112+
- name: merge and save to file
113+
ansible.builtin.set_fact:
114+
result_path: "{{ files | reportsmerger(strategy='large', output='/tmp/merged.json') }}"
115+
# Returns the file path, not the data
116+
```

0 commit comments

Comments
 (0)