Skip to content

[pre-commit MyPy] Align the entire repo with MyPy #3 #29819

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 18 commits into from
Sep 27, 2023
Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
129 changes: 66 additions & 63 deletions Packs/AWS-GuardDuty/Integrations/AWSGuardDuty/AWSGuardDuty.py

Large diffs are not rendered by default.

12 changes: 6 additions & 6 deletions Packs/AWS-GuardDuty/Integrations/AWSGuardDuty/AWSGuardDuty.yml
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,7 @@ configuration:
section: Connect
advanced: true
required: false
description: Amazon Web Services Guard Duty Service (gd)
description: Amazon Web Services Guard Duty Service (gd).
display: AWS - GuardDuty
name: AWS - GuardDuty
script:
Expand Down Expand Up @@ -315,7 +315,7 @@ script:
- PROOF_POINT
- FIRE_EYE
required: true
- description: The URI of the file that contains the IPSet. For example (https://s3.us-west-2.amazonaws.com/my-bucket/my-object-key)
- description: The URI of the file that contains the IPSet. For example (https://s3.us-west-2.amazonaws.com/my-bucket/my-object-key).
name: location
- description: The user friendly name to identify the IPSet. This name is displayed in all findings that are triggered by activity that involves IP addresses included in this IPSet.
name: name
Expand Down Expand Up @@ -457,7 +457,7 @@ script:
name: aws-gd-list-ip-sets
outputs:
- contextPath: AWS.GuardDuty.Detectors.IPSet.IpSetId
description: The unique identifier for an IP Set
description: The unique identifier for an IP Set.
type: Unknown
- arguments:
- auto: PREDEFINED
Expand Down Expand Up @@ -576,7 +576,7 @@ script:
name: aws-gd-list-threatintel-sets
outputs:
- contextPath: AWS.GuardDuty.Detectors.ThreatIntelSet.ThreatIntelSetId
description: The unique identifier for an threat intel set
description: The unique identifier for an threat intel set.
type: string
- arguments:
- description: The detectorID that specifies the GuardDuty service whose ThreatIntelSet you want to update.
Expand Down Expand Up @@ -628,7 +628,7 @@ script:
name: aws-gd-list-findings
outputs:
- contextPath: AWS.GuardDuty.Findings.FindingId
description: The unique identifier for the Finding
description: The unique identifier for the Finding.
type: string
- arguments:
- description: The ID of the detector that specifies the GuardDuty service whose findings you want to retrieve.
Expand Down Expand Up @@ -871,7 +871,7 @@ script:
- contextPath: AWS.GuardDuty.Members.UpdatedAt
description: The time a member was last updated.
type: string
dockerimage: demisto/boto3py3:1.0.0.71685
dockerimage: demisto/boto3py3:1.0.0.75596
isfetch: true
runonce: false
script: '-'
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,16 @@
from CommonServerPython import * # noqa: F401
from AWSApiModule import * # noqa: E402

from typing import Tuple
from typing import TYPE_CHECKING, Tuple
from datetime import datetime, date

import urllib3.util
import boto3
import json

# Disable insecure warnings
urllib3.disable_warnings()
# The following import are used only for type hints and autocomplete.
# It is not used at runtime, and not exist in the docker image.
if TYPE_CHECKING:
from mypy_boto3_guardduty import GuardDutyClient


CLIENT_SERVICE = 'guardduty'
MAX_IDS_PER_REQ = 50
Expand Down Expand Up @@ -52,7 +53,7 @@ def convert_events_with_datetime_to_str(events: list) -> list:
return output_events


def get_events(aws_client: boto3.client, collect_from: dict, collect_from_default: Optional[datetime], last_ids: dict,
def get_events(aws_client: "GuardDutyClient", collect_from: dict, collect_from_default: Optional[datetime], last_ids: dict,
severity: str, limit: int = MAX_RESULTS, detectors_num: int = MAX_RESULTS,
max_ids_per_req: int = MAX_IDS_PER_REQ) -> Tuple[list, dict, dict]:
"""Get events from AWSGuardDuty.
Expand Down Expand Up @@ -91,7 +92,7 @@ def get_events(aws_client: boto3.client, collect_from: dict, collect_from_defaul

response = aws_client.list_detectors(**list_detectors_args)
detector_ids += response.get('DetectorIds', [])
next_token = response.get('NextToken')
next_token = response.get('NextToken', '')

demisto.debug(f"AWSGuardDutyEventCollector - Found detector ids: {detector_ids}")

Expand Down Expand Up @@ -124,7 +125,7 @@ def get_events(aws_client: boto3.client, collect_from: dict, collect_from_defaul
list_finding_args.update({'NextToken': next_token})
list_findings = aws_client.list_findings(**list_finding_args)
finding_ids += list_findings.get('FindingIds', [])
next_token = list_findings.get('NextToken')
next_token = list_findings.get('NextToken', '')

# Handle duplicates and findings updated at the same time.
if last_ids.get(detector_id) and last_ids.get(detector_id) in finding_ids:
Expand Down Expand Up @@ -189,7 +190,7 @@ def main(): # pragma: no cover
aws_role_policy, aws_access_key_id, aws_secret_access_key, verify_certificate,
timeout, retries, sts_endpoint_url=sts_endpoint_url, endpoint_url=endpoint_url)

client = aws_client.aws_session(service=CLIENT_SERVICE, region=aws_default_region)
client: "GuardDutyClient" = aws_client.aws_session(service=CLIENT_SERVICE, region=aws_default_region)

command = demisto.command()
if command == 'test-module':
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -112,7 +112,7 @@ script:
name: limit
description: Manual command used to fetch events and display them.
name: aws-gd-get-events
dockerimage: demisto/boto3py3:1.0.0.71685
dockerimage: demisto/boto3py3:1.0.0.75596
isfetchevents: true
subtype: python3
marketplaces:
Expand Down
10 changes: 10 additions & 0 deletions Packs/AWS-GuardDuty/ReleaseNotes/1_3_30.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@

#### Integrations

##### AWS - GuardDuty Event Collector

Updated the Docker image to: *demisto/boto3py3:1.0.0.75596*.

##### AWS - GuardDuty

Updated the Docker image to: *demisto/boto3py3:1.0.0.75596*.
2 changes: 1 addition & 1 deletion Packs/AWS-GuardDuty/pack_metadata.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"name": "AWS - GuardDuty",
"description": "Amazon Web Services Guard Duty Service (gd)",
"support": "xsoar",
"currentVersion": "1.3.29",
"currentVersion": "1.3.30",
"author": "Cortex XSOAR",
"url": "https://www.paloaltonetworks.com/cortex",
"email": "",
Expand Down
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
import demistomock as demisto # noqa: F401
from CommonServerPython import * # noqa: F401
import datetime as dt
import urllib3
from typing import Iterator
from typing import TYPE_CHECKING, Iterator

from AWSApiModule import *

import boto3
# The following import are used only for type hints and autocomplete.
# It is not used at runtime, and not exist in the docker image.
if TYPE_CHECKING:
from mypy_boto3_securityhub import SecurityHubClient

# Disable insecure warnings
urllib3.disable_warnings()

VENDOR = 'AWS'
PRODUCT = 'Security Hub'
Expand Down Expand Up @@ -50,14 +50,14 @@ def generate_last_run(events: list[dict]) -> dict:
}


def get_events(client: boto3.client, start_time: dt.datetime | None = None,
def get_events(client: "SecurityHubClient", start_time: dt.datetime | None = None,
end_time: dt.datetime | None = None, id_ignore_list: list[str] | None = None,
page_size: int = API_MAX_PAGE_SIZE, limit: int = 0) -> Iterator[list[dict]]:
"""
Fetch events from AWS Security Hub.

Args:
client (boto3.client): Boto3 client to use.
client (SecurityHubClient): Boto3 client to use.
start_time (datetime | None, optional): Start time to fetch events from. Required if end_time is set.
end_time (datetime | None, optional): Time to fetch events until. Defaults to current time.
id_ignore_list (list[str] | None, optional): List of finding IDs to not include in the results.
Expand Down Expand Up @@ -105,7 +105,7 @@ def get_events(client: boto3.client, start_time: dt.datetime | None = None,
response = client.get_findings(**kwargs)
result = response.get('Findings', [])
count += len(result)
yield result
yield result # type: ignore[misc]

if 'NextToken' in response and (limit == 0 or count < limit):
kwargs['NextToken'] = response['NextToken']
Expand All @@ -114,13 +114,13 @@ def get_events(client: boto3.client, start_time: dt.datetime | None = None,
break


def fetch_events(client: boto3.client, last_run: dict, first_fetch_time: dt.datetime | None,
def fetch_events(client: "SecurityHubClient", last_run: dict, first_fetch_time: dt.datetime | None,
page_size: int = API_MAX_PAGE_SIZE, limit: int = 0) -> tuple[list[dict], dict, Exception | None]:
"""
Fetch events from AWS Security Hub and send them to XSIAM.

Args:
client (boto3.client): Boto3 client to use.
client (SecurityHubClient): Boto3 client to use.
last_run (dict): Dict containing the last fetched event creation time.
first_fetch_time (datetime | None, optional): In case of first fetch, fetch events from this datetime.
page_size (int, optional): Number of results to fetch per request. Defaults to API_MAX_PAGE_SIZE.
Expand Down Expand Up @@ -161,13 +161,13 @@ def fetch_events(client: boto3.client, last_run: dict, first_fetch_time: dt.date
return events, next_run, error


def get_events_command(client: boto3.client, should_push_events: bool,
def get_events_command(client: "SecurityHubClient", should_push_events: bool,
page_size: int, limit: int = 0) -> CommandResults:
"""
Fetch events from AWS Security Hub.

Args:
client (boto3.client): Boto3 client to use.
client (SecurityHubClient): Boto3 client to use.
should_push_events (bool): Whether to push events to XSIAM.
page_size (int, optional): Number of results to fetch per request. Defaults to API_MAX_PAGE_SIZE.
limit (int, optional): Maximum number of events to fetch. Defaults to 0 (no limit).
Expand Down Expand Up @@ -241,7 +241,7 @@ def main(): # pragma: no cover
retries=retries,
)

client = aws_client.aws_session(
client: "SecurityHubClient" = aws_client.aws_session(
service='securityhub',
region=aws_default_region,
role_arn=aws_role_arn,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,7 @@ script:
name: limit
description: Fetch events from AWS Security Hub.
name: aws-securityhub-get-events
dockerimage: demisto/boto3py3:1.0.0.71373
dockerimage: demisto/boto3py3:1.0.0.75596
isfetchevents: true
script: '-'
subtype: python3
Expand Down
6 changes: 6 additions & 0 deletions Packs/AWS-SecurityHub/ReleaseNotes/1_3_12.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@

#### Integrations

##### AWS Security Hub Event Collector

Updated the Docker image to: *demisto/boto3py3:1.0.0.75596*.
2 changes: 1 addition & 1 deletion Packs/AWS-SecurityHub/pack_metadata.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"name": "AWS - Security Hub",
"description": "Amazon Web Services Security Hub Service.",
"support": "xsoar",
"currentVersion": "1.3.11",
"currentVersion": "1.3.12",
"author": "Cortex XSOAR",
"url": "https://www.paloaltonetworks.com/cortex",
"email": "",
Expand Down
Loading