Skip to content

Commit 6cdf88d

Browse files
dantavoriyaakovi
authored and
bensterenson
committed
New RN summary (demisto#7133)
* in progress * in progress * in progress * in progress * in progress * in progress * in progress * in progress * changed RN file name * added comments * test in progress * updated script * updated script * updated script * test fix helloworld * in progress * in progress * in progress * in progress * in progress * in progress * in progress * added test * fix * fixed doc * in progress * updated new release notes generator * in progress * in progress * in progress * in progress * fixed ignored release notes validations and added tests * small fixes * restored .gitignore * added new packs to release notes generator * fixed unit tests * fixed lint error * fixed lint error * fixed modified packs collection * Fixed handling of ignored section in RN files * fixed modified release notes filter * start merging pack versions block to single block * added support for merging pack versions block to single block * added unit test for merge release notes block function * fixed small formatting issue * fixed CR notes * fixed content-descriptor.json path * fixed time formats in content-descriptor.json Co-authored-by: syaakovi <[email protected]>
1 parent 01d0c28 commit 6cdf88d

File tree

14 files changed

+749
-2
lines changed

14 files changed

+749
-2
lines changed

‎.circleci/config.yml

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -181,10 +181,14 @@ jobs:
181181
fi
182182
if [ -n "${GITHUB_TOKEN}" ] ;
183183
then
184-
python3 release_notes.py $CONTENT_VERSION $GIT_SHA1 $CIRCLE_BUILD_NUM $SERVER_VERSION --github-token $GITHUB_TOKEN || echo "ignore errors"
184+
# python3 release_notes.py $CONTENT_VERSION $GIT_SHA1 $CIRCLE_BUILD_NUM $SERVER_VERSION --github-token $GITHUB_TOKEN || echo "ignore errors"
185+
# new release notes summary generator in packs format
186+
python3 Utils/release_notes_generator.py $CONTENT_VERSION $GIT_SHA1 $CIRCLE_BUILD_NUM --output $CIRCLE_ARTIFACTS/packs-release-notes.md --github-token $GITHUB_TOKEN
185187
186188
else
187-
python3 release_notes.py $CONTENT_VERSION $GIT_SHA1 $CIRCLE_BUILD_NUM $SERVER_VERSION || echo "ignore errors"
189+
# python3 release_notes.py $CONTENT_VERSION $GIT_SHA1 $CIRCLE_BUILD_NUM $SERVER_VERSION || echo "ignore errors"
190+
# new release notes summary generator in packs format
191+
python3 Utils/release_notes_generator.py $CONTENT_VERSION $GIT_SHA1 $CIRCLE_BUILD_NUM --output $CIRCLE_ARTIFACTS/packs-release-notes.md
188192
fi
189193
- run:
190194
name: Common Server Documentation

‎Tests/scripts/infrastructure_tests/release_notes_generator_test.py

Lines changed: 267 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,267 @@
1+
import os
2+
import re
3+
from Utils.release_notes_generator import (get_release_notes_dict,
4+
generate_release_notes_summary,
5+
get_pack_entities,
6+
read_and_format_release_note,
7+
merge_version_blocks,
8+
EMPTY_LINES_REGEX)
9+
10+
TEST_DATA_PATH = 'Tests/scripts/infrastructure_tests/tests_data/RN_tests_data'
11+
12+
VERSION = 'VERSION'
13+
ASSET_ID = 'ASSET_ID'
14+
15+
16+
class TestReadAndFormatReleaseNote:
17+
def test_sanity(self):
18+
"""
19+
Given
20+
- A release note file with 2 Integrations:
21+
- FakePack1_Integration1
22+
- FakePack1_Integration2
23+
24+
When
25+
- Formatting a release notes file.
26+
27+
Then
28+
- Ensure both integration appear in the formatted string
29+
"""
30+
rn_file = os.path.join(TEST_DATA_PATH, 'FakePack1', 'ReleaseNotes', '1_1_0.md')
31+
formatted_text = read_and_format_release_note(rn_file)
32+
assert 'FakePack1_Integration1' in formatted_text
33+
assert 'FakePack1_Integration2' in formatted_text
34+
35+
def test_ignored_release_notes_block(self):
36+
"""
37+
Given
38+
- A release note file with an Integration and a Script:
39+
- FakePack4_Script1
40+
- FakePack4_Integration1 - should be ignored
41+
42+
When
43+
- Formatting a release notes file.
44+
45+
Then
46+
- Ensure only the script appears in the formatted string
47+
"""
48+
rn_file = os.path.join(TEST_DATA_PATH, 'FakePack4', 'ReleaseNotes', '1_1_0.md')
49+
formatted_text = read_and_format_release_note(rn_file)
50+
assert 'FakePack4_Script1' in formatted_text
51+
assert 'FakePack4_Integration1' not in formatted_text
52+
53+
def test_ignored_entire_release_note(self):
54+
"""
55+
Given
56+
- A release note file with an Integration and a Script:
57+
- FakePack4_Script1
58+
- FakePack4_Integration1
59+
60+
When
61+
- Formatting a release notes file.
62+
63+
Then
64+
- Ensure formatted string is empty.
65+
"""
66+
rn_file = os.path.join(TEST_DATA_PATH, 'FakePack4', 'ReleaseNotes', '1_0_1.md')
67+
formatted_text = read_and_format_release_note(rn_file)
68+
assert formatted_text == ''
69+
70+
71+
class TestGenerateReleaseNotesSummary:
72+
def setup(self):
73+
self._version = VERSION
74+
self._asset_id = ASSET_ID
75+
self._outfile = 'temp.md'
76+
77+
def test_added_pack(self):
78+
"""
79+
Given
80+
- A repository of two new packs:
81+
- FakePack3 version 1.0.0
82+
- FakePack4 version 1.0.0
83+
84+
When
85+
- Generating a release notes summary file.
86+
87+
Then
88+
- Ensure release notes generator creates a valid summary, by checking:
89+
- the release notes summary contains two packs:
90+
- FakePack3 with version 1.0.0
91+
- FakePack4 with version 1.0.0
92+
"""
93+
new_packs_rn = {
94+
'FakePack3': get_pack_entities(os.path.join(TEST_DATA_PATH, 'FakePack3')),
95+
'FakePack4': get_pack_entities(os.path.join(TEST_DATA_PATH, 'FakePack4')),
96+
}
97+
98+
rn_summary = generate_release_notes_summary(new_packs_rn, {}, self._version, self._asset_id, 'temp.md')
99+
100+
assert '## New: FakePack3 Pack v1.0.0' in rn_summary
101+
assert '## New: FakePack4 Pack v1.0.0' in rn_summary
102+
103+
def test_two_packs(self):
104+
"""
105+
Given
106+
- A repository of two packs updates and release notes:
107+
- FakePack1 with versions 1.1.0 and 2.0.0
108+
- FakePack2 version 1.1.0
109+
110+
When
111+
- Generating a release notes summary file.
112+
113+
Then
114+
- Ensure release notes generator creates a valid summary, by checking:
115+
- the output of get_release_notes_dict() is a valid dict of (pack_name, dict(pack_version, release_note)).
116+
- the release notes summary contains two packs with 3 updates:
117+
- FakePack1 with versions 1.1.0 and 2.0.0
118+
- FakePack2 with versions 1.1.0
119+
"""
120+
release_notes_files = [
121+
os.path.join(TEST_DATA_PATH, 'FakePack1', 'ReleaseNotes', '1_1_0.md'),
122+
os.path.join(TEST_DATA_PATH, 'FakePack1', 'ReleaseNotes', '2_0_0.md'),
123+
os.path.join(TEST_DATA_PATH, 'FakePack2', 'ReleaseNotes', '1_1_0.md'),
124+
]
125+
126+
rn_dict = get_release_notes_dict(release_notes_files)
127+
128+
assert '1.1.0' in rn_dict['FakePack1'].keys()
129+
assert '2.0.0' in rn_dict['FakePack1'].keys()
130+
assert '1.1.0' in rn_dict['FakePack2'].keys()
131+
132+
rn_summary = generate_release_notes_summary({}, rn_dict, self._version, self._asset_id, self._outfile)
133+
134+
assert VERSION in rn_summary and ASSET_ID in rn_summary # summary title
135+
assert '### FakePack1 Pack v2.0.0' in rn_summary
136+
assert '##### FakePack1_Integration1' in rn_summary
137+
assert 'This is a fake1 minor release note.' in rn_summary
138+
assert 'This is a fake1 major release note.' in rn_summary
139+
assert '### FakePack2 Pack v1.1.0' in rn_summary
140+
assert '##### FakePack2_Script1' in rn_summary
141+
assert 'This is a fake2 major release note.' in rn_summary
142+
143+
def test_release_notes_summary_with_empty_lines_in_rn(self):
144+
"""
145+
Given
146+
- A repository contains a FakePack3 update with ignored release notes.
147+
148+
When
149+
- Generating a release notes summary file.
150+
151+
Then
152+
- Ensure release notes generator creates a valid summary, by checking:
153+
- the output of get_release_notes_dict() is a dict of (pack_name, dict(pack_version, release_note)).
154+
- empty lines (with dashes) are removed from the release notes summary.
155+
"""
156+
release_notes_files = [
157+
os.path.join(TEST_DATA_PATH, 'FakePack3', 'ReleaseNotes', '1_0_1.md')
158+
]
159+
160+
rn_dict = get_release_notes_dict(release_notes_files)
161+
162+
assert '1.0.1' in rn_dict['FakePack3'].keys()
163+
assert len(rn_dict) == 1
164+
165+
rn_summary = generate_release_notes_summary({}, rn_dict, self._version, self._asset_id, self._outfile)
166+
167+
print(rn_summary)
168+
169+
match = re.search(EMPTY_LINES_REGEX, rn_summary)
170+
assert match is None
171+
172+
def test_release_notes_summary_with_ignored_rns(self):
173+
"""
174+
Given
175+
- A repository of a packs update and release notes:
176+
- FakePack4 with versions 1.0.1 and 1.1.0
177+
178+
When
179+
- Generating a release notes summary file.
180+
181+
Then
182+
- Ensure release notes generator creates a valid summary, by checking:
183+
- the output of get_release_notes_dict() is a valid dict of (pack_name, dict(pack_version, release_note))
184+
- the release notes summary contains one packs with 1 updates:
185+
- FakePack4 version 1.1.0
186+
- the summary does not contain release notes 1.0.1, because it is ignored.
187+
"""
188+
release_notes_files = [
189+
os.path.join(TEST_DATA_PATH, 'FakePack4', 'ReleaseNotes', '1_0_1.md'),
190+
os.path.join(TEST_DATA_PATH, 'FakePack4', 'ReleaseNotes', '1_1_0.md'),
191+
]
192+
193+
rn_dict = get_release_notes_dict(release_notes_files)
194+
195+
assert '1.1.0' in rn_dict['FakePack4'].keys()
196+
assert len(rn_dict) == 1
197+
198+
rn_summary = generate_release_notes_summary({}, rn_dict, self._version, self._asset_id, self._outfile)
199+
200+
assert '### FakePack4 Pack v1.1.0' in rn_summary
201+
assert '##### FakePack4_Script1' in rn_summary
202+
203+
204+
class TestMergeVersionBlocks:
205+
def test_sanity(self):
206+
"""
207+
Given
208+
two changes in foreign content types
209+
210+
When
211+
two pack versions that modified different items.
212+
213+
Then
214+
type sections appears one after the other
215+
"""
216+
release_notes_paths = [
217+
os.path.join(TEST_DATA_PATH, 'FakePack1', 'ReleaseNotes', '1_1_0.md'),
218+
os.path.join(TEST_DATA_PATH, 'FakePack1', 'ReleaseNotes', '2_1_0.md'),
219+
]
220+
221+
pack_versions_dict = {}
222+
for path in release_notes_paths:
223+
with open(path) as file_:
224+
pack_versions_dict[os.path.basename(os.path.splitext(path)[0])] = file_.read()
225+
226+
rn_block = merge_version_blocks('FakePack', pack_versions_dict)
227+
228+
assert 'FakePack1_Playbook1' in rn_block
229+
assert 'FakePack1_Playbook2' in rn_block
230+
assert 'FakePack1_Integration1' in rn_block
231+
assert 'FakePack1_Integration2' in rn_block
232+
assert 'v2_1_0' in rn_block
233+
assert 'v1_1_0' not in rn_block
234+
235+
def test_similiar_entities(self):
236+
"""
237+
Given
238+
two changes in similar content entities
239+
240+
When
241+
two pack versions that modified the same items.
242+
243+
Then
244+
one integration section appears
245+
one entity title for each one with two comments
246+
"""
247+
release_notes_paths = [
248+
os.path.join(TEST_DATA_PATH, 'FakePack1', 'ReleaseNotes', '1_1_0.md'),
249+
os.path.join(TEST_DATA_PATH, 'FakePack1', 'ReleaseNotes', '2_0_0.md'),
250+
]
251+
252+
pack_versions_dict = {}
253+
for path in release_notes_paths:
254+
with open(path) as file_:
255+
pack_versions_dict[os.path.basename(os.path.splitext(path)[0])] = file_.read()
256+
257+
rn_block = merge_version_blocks('FakePack', pack_versions_dict)
258+
259+
assert rn_block.count('Integrations') == 1
260+
assert rn_block.count('FakePack1_Integration1') == 1
261+
assert rn_block.count('FakePack1_Integration2') == 1
262+
assert 'v2_0_0' in rn_block
263+
assert 'v1_1_0' not in rn_block
264+
assert 'fake1 minor' in rn_block
265+
assert 'fake2 minor' in rn_block
266+
assert 'fake1 major' in rn_block
267+
assert 'fake2 major' in rn_block

‎Tests/scripts/infrastructure_tests/tests_data/RN_tests_data/FakePack1/ReleaseNotes/1_1_0.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
2+
#### Integrations
3+
##### __FakePack1_Integration1__
4+
- This is a fake1 minor release note.
5+
##### __FakePack1_Integration2__
6+
- This is a fake2 minor release note.

‎Tests/scripts/infrastructure_tests/tests_data/RN_tests_data/FakePack1/ReleaseNotes/2_0_0.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
2+
#### Integrations
3+
##### __FakePack1_Integration1__
4+
- This is a fake1 major release note.
5+
##### __FakePack1_Integration2__
6+
- This is a fake2 major release note.

‎Tests/scripts/infrastructure_tests/tests_data/RN_tests_data/FakePack1/ReleaseNotes/2_1_0.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
2+
#### Playbooks
3+
##### __FakePack1_Playbook1__
4+
- This is a fake major release note.
5+
##### __FakePack1_Playbook2__
6+
- This is a fake major release note.

‎Tests/scripts/infrastructure_tests/tests_data/RN_tests_data/FakePack1/pack_metadata.json

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
{
2+
"name": "FakePack1",
3+
"currentVersion": "2.0.0"
4+
}

‎Tests/scripts/infrastructure_tests/tests_data/RN_tests_data/FakePack2/ReleaseNotes/1_1_0.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
2+
#### Scripts
3+
##### __FakePack2_Script1__
4+
- This is a fake minor release note.

‎Tests/scripts/infrastructure_tests/tests_data/RN_tests_data/FakePack2/pack_metadata.json

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
{
2+
"name": "FakePack2",
3+
"currentVersion": "1.1.0"
4+
}

‎Tests/scripts/infrastructure_tests/tests_data/RN_tests_data/FakePack3/ReleaseNotes/1_0_1.md

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
2+
#### Scripts
3+
##### __FakePack3_Script1__
4+
- This is a fake minor release note.
5+
-
6+
7+
##### __FakePack3_Script2__
8+
- Another release note.
9+
-
10+

‎Tests/scripts/infrastructure_tests/tests_data/RN_tests_data/FakePack3/pack_metadata.json

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
{
2+
"name": "FakePack3",
3+
"currentVersion": "1.0.1"
4+
}

‎Tests/scripts/infrastructure_tests/tests_data/RN_tests_data/FakePack4/ReleaseNotes/1_0_1.md

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
<!--
2+
#### Scripts
3+
- __FakePack4_Script1__
4+
This is a fake minor release note.
5+
6+
7+
#### Integrations
8+
- __FakePack4_Integration1__
9+
-
10+
-->

‎Tests/scripts/infrastructure_tests/tests_data/RN_tests_data/FakePack4/ReleaseNotes/1_1_0.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
#### Scripts
2+
##### __FakePack4_Script1__
3+
- This is a fake minor release note.
4+
5+
<!--
6+
#### Integrations
7+
- __FakePack4_Integration1__
8+
- ignored
9+
-->

‎Tests/scripts/infrastructure_tests/tests_data/RN_tests_data/FakePack4/pack_metadata.json

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
{
2+
"name": "FakePack4",
3+
"currentVersion": "1.0.1"
4+
}

0 commit comments

Comments
 (0)