New RN summary #7133

dantavori · 2020-05-21T08:35:47Z

Status

In Progress
Ready
In Hold - (Reason for hold)

Related Issues

fixes: https://github.com/demisto/etc/issues/23756

Description

Added the new release notes summary generator (in packs format) to the build process. Currently in Build Content Descriptor step, right after release_notes.py execution.

Must have

Tests
Documentation

Itay4 · 2020-06-22T08:29:21Z

Tests/scripts/infrastructure_tests/release_notes_generator_test.py

+        - Formatting a release notes file.
+
+        Then
+        - Ensure both integration appear in the formatted string


this seems wrong

Itay4 · 2020-06-22T09:13:35Z

release_notes_generator.py

@@ -0,0 +1,408 @@
+from __future__ import print_function


should this file be in content root?

the old release notes generator was there, do you suggest something else?
maybe Utils?

Itay4 · 2020-06-22T09:20:23Z

release_notes_generator.py

+    return pack_paths
+
+
+def get_new_entity_record(entity_path: str) -> (str, str):


Suggested change

def get_new_entity_record(entity_path: str) -> (str, str):

def get_new_entity_record(entity_path: str) -> Tuple[str, str]:

Itay4 · 2020-06-22T09:22:15Z

release_notes_generator.py

+        glob.glob(f'{pack_path}/*/*/*.yml')], [])
+    pack_entities.sort()
+
+    entities_data = {}


Suggested change

entities_data = {}

entities_data: Dict = {}

* add test for get_msg_mail_format with headers set to none * catch all types of exceptions raised in get_msg_format * New RN summary (#7133) * in progress * in progress * in progress * in progress * in progress * in progress * in progress * in progress * changed RN file name * added comments * test in progress * updated script * updated script * updated script * test fix helloworld * in progress * in progress * in progress * in progress * in progress * in progress * in progress * added test * fix * fixed doc * in progress * updated new release notes generator * in progress * in progress * in progress * in progress * fixed ignored release notes validations and added tests * small fixes * restored .gitignore * added new packs to release notes generator * fixed unit tests * fixed lint error * fixed lint error * fixed modified packs collection * Fixed handling of ignored section in RN files * fixed modified release notes filter * start merging pack versions block to single block * added support for merging pack versions block to single block * added unit test for merge release notes block function * fixed small formatting issue * fixed CR notes * fixed content-descriptor.json path * fixed time formats in content-descriptor.json Co-authored-by: syaakovi <[email protected]> * Microsoft azure feed (#7715) * Microsoft azure feed (#7410) * Add pack Azure AD Connect Health Feed * Clean up * Passed all tests * Fix test playbook * Minor fixes * Minor changes * remove changelog * change integration name global * include integration name in demisto.info * add docstring to substitution function * add disclaimer to readme * remove the other changelog * add command_examples.txt and generate readme with demisto-sdk * remove entries in tests/conf.json * remove urls from readme as it fails secrets test * remove trailing space * add entries back to conf.json as it fails pre-commit otherwise * remove exept blocks and apply suggestion to subs function * add suggested changes * Fix issue with results dict * changes to avoid duplicated indicators * remove substitution function as it's not deded Co-authored-by: Bar Katzir <[email protected]> * chnge taskid task.id TO UUIDS Co-authored-by: Guillermo Serrano <[email protected]> Co-authored-by: Bar Katzir <[email protected]> Co-authored-by: bar katzir <[email protected]> * Workday (#7478) * creating workday pack * Worker context build function * Integration Ready but README.md, fine-tuning on unitests and TPB * Added Unittests and README.md * * Removed "Business_Title" from headers * Changed `Phone Number` field source. (and changed expected context accordingly) * Changed Test_module output * Changed username and password params to credentials type. in Workday.yml * Changed TPB according to Shai's review in Workday_-_Test.yml * Added TPB details to conf.json Co-authored-by: Shai Yaakovi <[email protected]> * Update Packs/Workday/Integrations/Workday/Workday.py Co-authored-by: Shai Yaakovi <[email protected]> * Changed verion in README.md * CR Changes * Secrets * Updated * CR changes Added Workday TPB to skipped tests (lack of access) * Update docker image Co-authored-by: Shai Yaakovi <[email protected]> Co-authored-by: Alex Fiedler <[email protected]> * pan-os-edl-docs1 (#7510) * Revert base fix (#7717) * revert 1.0.11 and 1.0.12 versions in Base pack by issuing version 1.0.13 * skiping XDR iocs test (#7719) Co-authored-by: esharf <[email protected]> * Workday - rm TODO lines (#7720) * rm todo * SlackAsk readme - add missing arguments (#7650) * added missing arguments to the README. * docker * fix * Updated * correction * merge * RN * no RN Co-authored-by: Alex Fiedler <[email protected]> * fix typo in integration name (#7698) * bump version 20.6.1 (#7722) * PAN-OS - Added outputs to the `panorama-get-logs` command (#7554) * PAN-OS - Added outputs to the `panorama-get-logs` command * dockerimage update * Update Packs/PAN-OS/ReleaseNotes/1_1_0.md Co-authored-by: Andrew Shamah <[email protected]> Co-authored-by: Andrew Shamah <[email protected]> * Nightly parallel pack installation (#7693) * added multithreading to pack installation for nightly runs * fixed call error * fixed CR notes * added sleep after nightly pack installation * Update 1_1_10.md * CrowdStrike Falcon Streaming v2 - bug fixes and improvments (#7552) * disable session timeout, handle exception better, add log prints and store offset in context * bump docker image tag * output event dump to rawJSON * Updated * store sample events * fix mypy reports * remove unnecessary line from the readme * Update Packs/CrowdStrikeFalconStreamingV2/Integrations/CrowdStrikeFalconStreamingV2/CrowdStrikeFalconStreamingV2.yml Co-authored-by: Guy Lichtman <[email protected]> * Update Packs/CrowdStrikeFalconStreamingV2/Integrations/CrowdStrikeFalconStreamingV2/README.md Co-authored-by: Guy Lichtman <[email protected]> * Update Packs/CrowdStrikeFalconStreamingV2/Integrations/CrowdStrikeFalconStreamingV2/CrowdStrikeFalconStreamingV2.py Co-authored-by: Guy Lichtman <[email protected]> * Update Packs/CrowdStrikeFalconStreamingV2/Integrations/CrowdStrikeFalconStreamingV2/README.md Co-authored-by: Guy Lichtman <[email protected]> * update readme that events are fetched every 1 minute * store integration context every 1 min and check if param is enabled in get samples cmd * remove unused import of union * Updated * Updated * reset last_integration_context_set to now after context set Co-authored-by: Alex Fiedler <[email protected]> Co-authored-by: Guy Lichtman <[email protected]> * Flake8 phase15 (#7707) * Traps flake8 * MailSenderNew_test flake8 * comment rn * flake8 phase 13 (#7705) * ZScalar, PcapMinerV2, QuestKace, Zsacalar, DBotSuggestClassifierMapping, ExtractDomainFromUrlFormat_test flake8 fixes * base rns * fixed typo * fix rn * fix rn * fix metada * comment rn Co-authored-by: Dan Tavori <[email protected]> Co-authored-by: syaakovi <[email protected]> Co-authored-by: content-bot <[email protected]> Co-authored-by: Guillermo Serrano <[email protected]> Co-authored-by: Bar Katzir <[email protected]> Co-authored-by: bar katzir <[email protected]> Co-authored-by: Bargenish <[email protected]> Co-authored-by: Shai Yaakovi <[email protected]> Co-authored-by: Alex Fiedler <[email protected]> Co-authored-by: Andrew Shamah <[email protected]> Co-authored-by: eli sharf <[email protected]> Co-authored-by: esharf <[email protected]> Co-authored-by: Lior Blobstein <[email protected]> Co-authored-by: Bar Hochman <[email protected]> Co-authored-by: roysagi <[email protected]> Co-authored-by: Guy Lichtman <[email protected]>

* in progress * in progress * in progress * in progress * in progress * in progress * in progress * in progress * changed RN file name * added comments * test in progress * updated script * updated script * updated script * test fix helloworld * in progress * in progress * in progress * in progress * in progress * in progress * in progress * added test * fix * fixed doc * in progress * updated new release notes generator * in progress * in progress * in progress * in progress * fixed ignored release notes validations and added tests * small fixes * restored .gitignore * added new packs to release notes generator * fixed unit tests * fixed lint error * fixed lint error * fixed modified packs collection * Fixed handling of ignored section in RN files * fixed modified release notes filter * start merging pack versions block to single block * added support for merging pack versions block to single block * added unit test for merge release notes block function * fixed small formatting issue * fixed CR notes * fixed content-descriptor.json path * fixed time formats in content-descriptor.json Co-authored-by: syaakovi <[email protected]>

* add test for get_msg_mail_format with headers set to none * catch all types of exceptions raised in get_msg_format * New RN summary (demisto#7133) * in progress * in progress * in progress * in progress * in progress * in progress * in progress * in progress * changed RN file name * added comments * test in progress * updated script * updated script * updated script * test fix helloworld * in progress * in progress * in progress * in progress * in progress * in progress * in progress * added test * fix * fixed doc * in progress * updated new release notes generator * in progress * in progress * in progress * in progress * fixed ignored release notes validations and added tests * small fixes * restored .gitignore * added new packs to release notes generator * fixed unit tests * fixed lint error * fixed lint error * fixed modified packs collection * Fixed handling of ignored section in RN files * fixed modified release notes filter * start merging pack versions block to single block * added support for merging pack versions block to single block * added unit test for merge release notes block function * fixed small formatting issue * fixed CR notes * fixed content-descriptor.json path * fixed time formats in content-descriptor.json Co-authored-by: syaakovi <[email protected]> * Microsoft azure feed (demisto#7715) * Microsoft azure feed (demisto#7410) * Add pack Azure AD Connect Health Feed * Clean up * Passed all tests * Fix test playbook * Minor fixes * Minor changes * remove changelog * change integration name global * include integration name in demisto.info * add docstring to substitution function * add disclaimer to readme * remove the other changelog * add command_examples.txt and generate readme with demisto-sdk * remove entries in tests/conf.json * remove urls from readme as it fails secrets test * remove trailing space * add entries back to conf.json as it fails pre-commit otherwise * remove exept blocks and apply suggestion to subs function * add suggested changes * Fix issue with results dict * changes to avoid duplicated indicators * remove substitution function as it's not deded Co-authored-by: Bar Katzir <[email protected]> * chnge taskid task.id TO UUIDS Co-authored-by: Guillermo Serrano <[email protected]> Co-authored-by: Bar Katzir <[email protected]> Co-authored-by: bar katzir <[email protected]> * Workday (demisto#7478) * creating workday pack * Worker context build function * Integration Ready but README.md, fine-tuning on unitests and TPB * Added Unittests and README.md * * Removed "Business_Title" from headers * Changed `Phone Number` field source. (and changed expected context accordingly) * Changed Test_module output * Changed username and password params to credentials type. in Workday.yml * Changed TPB according to Shai's review in Workday_-_Test.yml * Added TPB details to conf.json Co-authored-by: Shai Yaakovi <[email protected]> * Update Packs/Workday/Integrations/Workday/Workday.py Co-authored-by: Shai Yaakovi <[email protected]> * Changed verion in README.md * CR Changes * Secrets * Updated * CR changes Added Workday TPB to skipped tests (lack of access) * Update docker image Co-authored-by: Shai Yaakovi <[email protected]> Co-authored-by: Alex Fiedler <[email protected]> * pan-os-edl-docs1 (demisto#7510) * Revert base fix (demisto#7717) * revert 1.0.11 and 1.0.12 versions in Base pack by issuing version 1.0.13 * skiping XDR iocs test (demisto#7719) Co-authored-by: esharf <[email protected]> * Workday - rm TODO lines (demisto#7720) * rm todo * SlackAsk readme - add missing arguments (demisto#7650) * added missing arguments to the README. * docker * fix * Updated * correction * merge * RN * no RN Co-authored-by: Alex Fiedler <[email protected]> * fix typo in integration name (demisto#7698) * bump version 20.6.1 (demisto#7722) * PAN-OS - Added outputs to the `panorama-get-logs` command (demisto#7554) * PAN-OS - Added outputs to the `panorama-get-logs` command * dockerimage update * Update Packs/PAN-OS/ReleaseNotes/1_1_0.md Co-authored-by: Andrew Shamah <[email protected]> Co-authored-by: Andrew Shamah <[email protected]> * Nightly parallel pack installation (demisto#7693) * added multithreading to pack installation for nightly runs * fixed call error * fixed CR notes * added sleep after nightly pack installation * Update 1_1_10.md * CrowdStrike Falcon Streaming v2 - bug fixes and improvments (demisto#7552) * disable session timeout, handle exception better, add log prints and store offset in context * bump docker image tag * output event dump to rawJSON * Updated * store sample events * fix mypy reports * remove unnecessary line from the readme * Update Packs/CrowdStrikeFalconStreamingV2/Integrations/CrowdStrikeFalconStreamingV2/CrowdStrikeFalconStreamingV2.yml Co-authored-by: Guy Lichtman <[email protected]> * Update Packs/CrowdStrikeFalconStreamingV2/Integrations/CrowdStrikeFalconStreamingV2/README.md Co-authored-by: Guy Lichtman <[email protected]> * Update Packs/CrowdStrikeFalconStreamingV2/Integrations/CrowdStrikeFalconStreamingV2/CrowdStrikeFalconStreamingV2.py Co-authored-by: Guy Lichtman <[email protected]> * Update Packs/CrowdStrikeFalconStreamingV2/Integrations/CrowdStrikeFalconStreamingV2/README.md Co-authored-by: Guy Lichtman <[email protected]> * update readme that events are fetched every 1 minute * store integration context every 1 min and check if param is enabled in get samples cmd * remove unused import of union * Updated * Updated * reset last_integration_context_set to now after context set Co-authored-by: Alex Fiedler <[email protected]> Co-authored-by: Guy Lichtman <[email protected]> * Flake8 phase15 (demisto#7707) * Traps flake8 * MailSenderNew_test flake8 * comment rn * flake8 phase 13 (demisto#7705) * ZScalar, PcapMinerV2, QuestKace, Zsacalar, DBotSuggestClassifierMapping, ExtractDomainFromUrlFormat_test flake8 fixes * base rns * fixed typo * fix rn * fix rn * fix metada * comment rn Co-authored-by: Dan Tavori <[email protected]> Co-authored-by: syaakovi <[email protected]> Co-authored-by: content-bot <[email protected]> Co-authored-by: Guillermo Serrano <[email protected]> Co-authored-by: Bar Katzir <[email protected]> Co-authored-by: bar katzir <[email protected]> Co-authored-by: Bargenish <[email protected]> Co-authored-by: Shai Yaakovi <[email protected]> Co-authored-by: Alex Fiedler <[email protected]> Co-authored-by: Andrew Shamah <[email protected]> Co-authored-by: eli sharf <[email protected]> Co-authored-by: esharf <[email protected]> Co-authored-by: Lior Blobstein <[email protected]> Co-authored-by: Bar Hochman <[email protected]> Co-authored-by: roysagi <[email protected]> Co-authored-by: Guy Lichtman <[email protected]>

* in progress * in progress * in progress * in progress * in progress * in progress * in progress * in progress * changed RN file name * added comments * test in progress * updated script * updated script * updated script * test fix helloworld * in progress * in progress * in progress * in progress * in progress * in progress * in progress * added test * fix * fixed doc * in progress * updated new release notes generator * in progress * in progress * in progress * in progress * fixed ignored release notes validations and added tests * small fixes * restored .gitignore * added new packs to release notes generator * fixed unit tests * fixed lint error * fixed lint error * fixed modified packs collection * Fixed handling of ignored section in RN files * fixed modified release notes filter * start merging pack versions block to single block * added support for merging pack versions block to single block * added unit test for merge release notes block function * fixed small formatting issue * fixed CR notes * fixed content-descriptor.json path * fixed time formats in content-descriptor.json Co-authored-by: syaakovi <[email protected]>

dantavori and others added 29 commits May 19, 2020 18:37

in progress

8acff2c

Merge branch 'master' into rn_summary

e91faf5

in progress

971b4d8

in progress

374b48c

in progress

2812214

in progress

a5db938

in progress

ce254e1

in progress

c3c8d8f

in progress

f5855ce

changed RN file name

bac2515

Merge branch 'master' into rn_summary

b3cb570

added comments

8aa9a83

test in progress

f6bbb88

updated script

0dfdd4f

Merge branch 'master' into rn_summary

e6f8ee9

updated script

c3ffcd3

updated script

27bcfa8

test fix helloworld

184d56e

in progress

88daff8

in progress

260c2ce

in progress

ccf739b

in progress

039808b

in progress

8bb9364

in progress

cf06496

in progress

1dffa82

added test

349ff61

fix

7f78396

fixed doc

a9f7a5e

Merge remote-tracking branch 'origin/master' into rn_summary

f1d4d23

yuvalbenshalom approved these changes Jun 3, 2020

View reviewed changes

yaakovi requested a review from Itay4 June 16, 2020 05:51

yaakovi added the docs-approved label Jun 16, 2020

yaakovi self-assigned this Jun 16, 2020

yaakovi added 10 commits June 16, 2020 09:03

fixed lint error

5990ad9

fixed modified packs collection

9a85264

Fixed handling of ignored section in RN files

43d791a

fixed modified release notes filter

593f0c7

Merge remote-tracking branch 'origin/master' into rn_summary

e369089

start merging pack versions block to single block

d56d217

added support for merging pack versions block to single block

c4f6335

added unit test for merge release notes block function

9e8dcf3

Merge remote-tracking branch 'origin/master' into rn_summary

81a09e7

fixed small formatting issue

11c7b23

Itay4 reviewed Jun 22, 2020

View reviewed changes

Itay4 approved these changes Jun 22, 2020

View reviewed changes

yaakovi added 4 commits June 22, 2020 14:46

fixed CR notes

73e8bca

fixed content-descriptor.json path

30d20e3

Merge remote-tracking branch 'origin/master' into rn_summary

0e8933c

fixed time formats in content-descriptor.json

d9e3b97

yaakovi merged commit 2a4d34e into master Jun 23, 2020

yaakovi deleted the rn_summary branch June 23, 2020 08:52

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

New RN summary #7133

New RN summary #7133

Uh oh!

dantavori commented May 21, 2020

Uh oh!

Itay4 Jun 22, 2020

Uh oh!

Itay4 Jun 22, 2020

Uh oh!

yaakovi Jun 22, 2020

Uh oh!

Itay4 Jun 22, 2020

Uh oh!

Itay4 Jun 22, 2020

Uh oh!

Itay4 Jun 22, 2020

Uh oh!

Uh oh!

		return pack_paths


		def get_new_entity_record(entity_path: str) -> (str, str):

New RN summary #7133

New RN summary #7133

Uh oh!

Conversation

dantavori commented May 21, 2020

Status

Related Issues

Description

Must have

Uh oh!

Itay4 Jun 22, 2020

Choose a reason for hiding this comment

Uh oh!

Itay4 Jun 22, 2020

Choose a reason for hiding this comment

Uh oh!

yaakovi Jun 22, 2020

Choose a reason for hiding this comment

Uh oh!

Itay4 Jun 22, 2020

Choose a reason for hiding this comment

Uh oh!

Itay4 Jun 22, 2020

Choose a reason for hiding this comment

Uh oh!

Itay4 Jun 22, 2020

Choose a reason for hiding this comment

Uh oh!

Uh oh!