-
-
Notifications
You must be signed in to change notification settings - Fork 529
Abusix send email #2283
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Abusix send email #2283
Changes from 26 commits
Commits
Show all changes
114 commits
Select commit
Hold shift + click to select a range
b81444a
Added AbuseSubmitter connector
cristinaascari 1213072
Renamed and updated connector EmailSender
cristinaascari 19ab514
Added monkeypatch to email sender connector
cristinaascari 98c49f7
Updated EmailSender parameters
cristinaascari 00704c1
Added AbuseDomainToAbuseIp pivot
cristinaascari af4c1fe
Updated AbuseDomainToAbuseIp pivot
cristinaascari 6dbf1ec
Updated pivots
cristinaascari 4a8417c
Changed health_check_status to editable
cristinaascari fe1b6d2
Changed health_check_status back to not editable
cristinaascari 6f5d888
Merge branch 'develop' into abusix_send_email
cristinaascari 2c553bb
Merge branch 'develop' into abusix_send_email
cristinaascari 3f6f0c7
receiver update
cristinaascari c97b7e0
subject and body update
cristinaascari 775245c
added abusix update migration
cristinaascari 8014e92
fix investigation creation
cristinaascari 2486a42
updated AbuseDomainToIp should_run
cristinaascari b0d4d74
updated AbuseIpToSubmission should_run
cristinaascari 9d51a12
added plugin migrations
cristinaascari ff7160a
Merge branch 'develop' into abusix_send_email
cristinaascari 4c727dd
fix migrations
cristinaascari a131680
Changed email sender body
cristinaascari 34a680c
fixed migrations dependencies
cristinaascari b3ab294
updates email sender and abuse submitter
cristinaascari 387f6d0
Fix migrations
cristinaascari 9cd641d
Fix
cristinaascari 47613bb
Fix
cristinaascari 915c1aa
Fix playbooks migrations
cristinaascari 7d5186f
Update api_app/connectors_manager/connectors/abuse_submitter.py
cristinaascari 579c542
Added AbuseSubmitter connector
cristinaascari 0cf2a40
Renamed and updated connector EmailSender
cristinaascari 7918a4d
Added monkeypatch to email sender connector
cristinaascari 9a1e1a0
Updated EmailSender parameters
cristinaascari 7993d58
Added AbuseDomainToAbuseIp pivot
cristinaascari 7e36776
Updated AbuseDomainToAbuseIp pivot
cristinaascari fe52c87
Updated pivots
cristinaascari c625171
Changed health_check_status to editable
cristinaascari ce8a3a5
Changed health_check_status back to not editable
cristinaascari b405870
receiver update
cristinaascari 7aa4ddb
subject and body update
cristinaascari dacb89e
added abusix update migration
cristinaascari e489253
fix investigation creation
cristinaascari 9f2a236
updated AbuseDomainToIp should_run
cristinaascari b20f51c
updated AbuseIpToSubmission should_run
cristinaascari d9c3a6f
added plugin migrations
cristinaascari 020b265
fix migrations
cristinaascari 582c297
Changed email sender body
cristinaascari dc83769
fixed migrations dependencies
cristinaascari 09c7535
updates email sender and abuse submitter
cristinaascari 4688f51
Fix migrations
cristinaascari 2d51c73
Fix
cristinaascari 11ab3c8
Fix
cristinaascari 2cbaaa4
Fix playbooks migrations
cristinaascari 4b8985d
added update method
cristinaascari 2704230
Merge remote-tracking branch 'origin/abusix_send_email' into abusix_s…
cristinaascari ba4dad8
added update method
cristinaascari 4482e77
removed AbuseSubmitter values
cristinaascari c605cc8
removed EmailSender values
cristinaascari 2cdf458
Merge branch 'develop' into abusix_send_email
cristinaascari 67c04da
Update should_run
cristinaascari 79fef91
Merge branch 'abusix_send_email' of github.com:intelowlproject/IntelO…
cristinaascari cb79a2a
changed AbuseDomainToAbuseIp to AnyCompare
cristinaascari fac4dfe
update compare
cristinaascari 8f4fcf6
added update
cristinaascari 9a3650b
removed AbuseIpToSubmission
cristinaascari f5bc4e1
Added AbuseSubmitter connector
cristinaascari a734614
Renamed and updated connector EmailSender
cristinaascari f409e6a
Added monkeypatch to email sender connector
cristinaascari d97d63d
Updated EmailSender parameters
cristinaascari d11da67
Added AbuseDomainToAbuseIp pivot
cristinaascari 713d5ed
Updated AbuseDomainToAbuseIp pivot
cristinaascari 263a4ca
Updated pivots
cristinaascari 24f3945
Changed health_check_status to editable
cristinaascari 177f559
Changed health_check_status back to not editable
cristinaascari c829ce8
receiver update
cristinaascari 9450635
subject and body update
cristinaascari 3b43404
added abusix update migration
cristinaascari e8b82af
fix investigation creation
cristinaascari cfe039c
updated AbuseDomainToIp should_run
cristinaascari f492bc4
updated AbuseIpToSubmission should_run
cristinaascari 434e8d6
added plugin migrations
cristinaascari d8873d1
fix migrations
cristinaascari 380c27a
Changed email sender body
cristinaascari f016cb7
fixed migrations dependencies
cristinaascari e46f378
updates email sender and abuse submitter
cristinaascari 818db69
Fix migrations
cristinaascari 01266cb
Fix
cristinaascari f892cac
Fix
cristinaascari 603a8f9
Fix playbooks migrations
cristinaascari de7507d
added update method
cristinaascari 3e0a4aa
Added AbuseSubmitter connector
cristinaascari 26c542b
Changed health_check_status to editable
cristinaascari 1c3387a
Changed health_check_status back to not editable
cristinaascari 5acca3a
added abusix update migration
cristinaascari 9aa4a90
added plugin migrations
cristinaascari cbe9baf
fix migrations
cristinaascari 5d96a2e
fixed migrations dependencies
cristinaascari 74fa5de
Fix migrations
cristinaascari b7f98b8
removed AbuseSubmitter values
cristinaascari c61a002
removed EmailSender values
cristinaascari 53b27b7
Update should_run
cristinaascari c721786
changed AbuseDomainToAbuseIp to AnyCompare
cristinaascari 2e54500
update compare
cristinaascari 3ef0081
added update
cristinaascari baa0680
removed AbuseIpToSubmission
cristinaascari 4f87e73
Fix pivot migrations
cristinaascari f25f408
Merge remote-tracking branch 'origin/abusix_send_email' into abusix_s…
cristinaascari bf9d484
Renamed playbook Abuse_Domain to Takedown_Request
cristinaascari f760eb8
Update api_app/connectors_manager/connectors/abuse_submitter.py
cristinaascari baee808
Update api_app/connectors_manager/connectors/abuse_submitter.py
cristinaascari 3296a86
Added EmailSender header and footer
cristinaascari eae5369
Fix linters
cristinaascari bd66963
Fix test
0ssigeno ac56fce
Fix pivot test
cristinaascari 80137ba
Fix
0ssigeno File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
23 changes: 23 additions & 0 deletions
23
api_app/analyzers_manager/migrations/0081_adjust_abusix.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| from django.db import migrations | ||
|
|
||
|
|
||
| def migrate(apps, schema_editor): | ||
| AnalyzerConfig = apps.get_model("analyzers_manager", "AnalyzerConfig") | ||
|
|
||
| AnalyzerConfig.objects.filter( | ||
| name="Abusix", | ||
| ).update(health_check_status=True) | ||
|
|
||
|
|
||
| def reverse_migrate(apps, schema_editor): | ||
| pass | ||
|
|
||
|
|
||
| class Migration(migrations.Migration): | ||
| dependencies = [ | ||
| ("api_app", "0062_alter_parameter_python_module"), | ||
| ("analyzers_manager", "0080_remove_dns0_names_analyzer"), | ||
| ] | ||
| operations = [ | ||
| migrations.RunPython(migrate, reverse_migrate), | ||
| ] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| from api_app.connectors_manager.connectors.email_sender import EmailSender | ||
|
|
||
|
|
||
| class AbuseSubmitter(EmailSender): | ||
| @property | ||
| def subject(self) -> str: | ||
| return f"Take down domain {self._job.parent_job.parent_job.observable_name}" | ||
|
|
||
| @property | ||
| def body(self) -> str: | ||
| return ( | ||
| f"Domain {self._job.parent_job.parent_job.observable_name} " | ||
| f"has been reported as malicious. We request you to take it down." | ||
cristinaascari marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| ) | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,37 @@ | ||
| from django.core.mail import EmailMessage | ||
|
|
||
| from api_app.connectors_manager.classes import Connector | ||
| from intel_owl.settings import DEFAULT_FROM_EMAIL | ||
| from tests.mock_utils import if_mock_connections, patch | ||
|
|
||
|
|
||
| class EmailSender(Connector): | ||
| sender: str | ||
| subject: str | ||
| body: str | ||
|
|
||
| def run(self) -> dict: | ||
| if self.sender: | ||
| sender = self.sender | ||
| else: | ||
| sender = DEFAULT_FROM_EMAIL | ||
| base_eml = EmailMessage( | ||
| subject=self.subject, | ||
| from_email=sender, | ||
| to=[self._job.observable_name], | ||
| body=self.body, | ||
| ) | ||
| base_eml.send() | ||
| return {"receiver": self._job.observable_name} | ||
|
|
||
| @classmethod | ||
| def _monkeypatch(cls): | ||
| patches = [ | ||
| if_mock_connections( | ||
| patch( | ||
| "django.core.mail.EmailMessage.send", | ||
| return_value="Email sent", | ||
| ) | ||
| ) | ||
| ] | ||
| return super()._monkeypatch(patches=patches) |
212 changes: 212 additions & 0 deletions
212
api_app/connectors_manager/migrations/0030_connector_config_emailsender.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,212 @@ | ||
| from django.db import migrations | ||
| from django.db.models.fields.related_descriptors import ( | ||
| ForwardManyToOneDescriptor, | ||
| ForwardOneToOneDescriptor, | ||
| ManyToManyDescriptor, | ||
| ) | ||
|
|
||
| plugin = { | ||
| "python_module": { | ||
| "health_check_schedule": None, | ||
| "update_schedule": None, | ||
| "module": "email_sender.EmailSender", | ||
| "base_path": "api_app.connectors_manager.connectors", | ||
| }, | ||
| "name": "EmailSender", | ||
| "description": "Send email", | ||
| "disabled": False, | ||
| "soft_time_limit": 60, | ||
| "routing_key": "default", | ||
| "health_check_status": True, | ||
| "maximum_tlp": "AMBER", | ||
| "run_on_failure": True, | ||
| "model": "connectors_manager.ConnectorConfig", | ||
| } | ||
|
|
||
| params = [ | ||
| { | ||
| "python_module": { | ||
| "module": "email_sender.EmailSender", | ||
| "base_path": "api_app.connectors_manager.connectors", | ||
| }, | ||
| "name": "subject", | ||
| "type": "str", | ||
| "description": "Email subject", | ||
| "is_secret": False, | ||
| "required": True, | ||
| }, | ||
| { | ||
| "python_module": { | ||
| "module": "email_sender.EmailSender", | ||
| "base_path": "api_app.connectors_manager.connectors", | ||
| }, | ||
| "name": "body", | ||
| "type": "str", | ||
| "description": "Email body", | ||
| "is_secret": False, | ||
| "required": True, | ||
| }, | ||
| { | ||
| "python_module": { | ||
| "module": "email_sender.EmailSender", | ||
| "base_path": "api_app.connectors_manager.connectors", | ||
| }, | ||
| "name": "sender", | ||
| "type": "str", | ||
| "description": "Email sender", | ||
| "is_secret": False, | ||
| "required": False, | ||
| }, | ||
| ] | ||
|
|
||
| values = [ | ||
| { | ||
| "parameter": { | ||
| "python_module": { | ||
| "module": "email_sender.EmailSender", | ||
| "base_path": "api_app.connectors_manager.connectors", | ||
| }, | ||
| "name": "subject", | ||
| "type": "str", | ||
| "description": "Email subject", | ||
| "is_secret": False, | ||
| "required": True, | ||
| }, | ||
| "analyzer_config": None, | ||
| "connector_config": "EmailSender", | ||
| "visualizer_config": None, | ||
| "ingestor_config": None, | ||
| "pivot_config": None, | ||
| "for_organization": False, | ||
| "value": "Subject", | ||
| "updated_at": "2024-04-22T14:08:49.711495Z", | ||
| "owner": None, | ||
| }, | ||
| { | ||
| "parameter": { | ||
| "python_module": { | ||
| "module": "email_sender.EmailSender", | ||
| "base_path": "api_app.connectors_manager.connectors", | ||
cristinaascari marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| }, | ||
| "name": "body", | ||
| "type": "str", | ||
| "description": "Email body", | ||
| "is_secret": False, | ||
| "required": True, | ||
| }, | ||
| "analyzer_config": None, | ||
| "connector_config": "EmailSender", | ||
| "visualizer_config": None, | ||
| "ingestor_config": None, | ||
| "pivot_config": None, | ||
| "for_organization": False, | ||
| "value": "Body", | ||
| "updated_at": "2024-04-22T14:08:49.712604Z", | ||
| "owner": None, | ||
| }, | ||
| { | ||
| "parameter": { | ||
| "python_module": { | ||
| "module": "email_sender.EmailSender", | ||
| "base_path": "api_app.connectors_manager.connectors", | ||
| }, | ||
| "name": "sender", | ||
| "type": "str", | ||
| "description": "Email sender", | ||
| "is_secret": False, | ||
| "required": False, | ||
| }, | ||
| "analyzer_config": None, | ||
| "connector_config": "EmailSender", | ||
| "visualizer_config": None, | ||
| "ingestor_config": None, | ||
| "pivot_config": None, | ||
| "for_organization": False, | ||
| "value": "", | ||
| "updated_at": "2024-04-22T15:22:44.281836Z", | ||
| "owner": None, | ||
| }, | ||
| ] | ||
|
|
||
|
|
||
| def _get_real_obj(Model, field, value): | ||
| def _get_obj(Model, other_model, value): | ||
| if isinstance(value, dict): | ||
| real_vals = {} | ||
| for key, real_val in value.items(): | ||
| real_vals[key] = _get_real_obj(other_model, key, real_val) | ||
| value = other_model.objects.get_or_create(**real_vals)[0] | ||
| # it is just the primary key serialized | ||
| else: | ||
| if isinstance(value, int): | ||
| if Model.__name__ == "PluginConfig": | ||
| value = other_model.objects.get(name=plugin["name"]) | ||
| else: | ||
| value = other_model.objects.get(pk=value) | ||
| else: | ||
| value = other_model.objects.get(name=value) | ||
| return value | ||
|
|
||
| if ( | ||
| type(getattr(Model, field)) | ||
| in [ForwardManyToOneDescriptor, ForwardOneToOneDescriptor] | ||
| and value | ||
| ): | ||
| other_model = getattr(Model, field).get_queryset().model | ||
| value = _get_obj(Model, other_model, value) | ||
| elif type(getattr(Model, field)) in [ManyToManyDescriptor] and value: | ||
| other_model = getattr(Model, field).rel.model | ||
| value = [_get_obj(Model, other_model, val) for val in value] | ||
| return value | ||
|
|
||
|
|
||
| def _create_object(Model, data): | ||
| mtm, no_mtm = {}, {} | ||
| for field, value in data.items(): | ||
| value = _get_real_obj(Model, field, value) | ||
| if type(getattr(Model, field)) is ManyToManyDescriptor: | ||
| mtm[field] = value | ||
| else: | ||
| no_mtm[field] = value | ||
| try: | ||
| o = Model.objects.get(**no_mtm) | ||
| except Model.DoesNotExist: | ||
| o = Model(**no_mtm) | ||
| o.full_clean() | ||
| o.save() | ||
| for field, value in mtm.items(): | ||
| attribute = getattr(o, field) | ||
| if value is not None: | ||
| attribute.set(value) | ||
| return False | ||
| return True | ||
|
|
||
|
|
||
| def migrate(apps, schema_editor): | ||
| Parameter = apps.get_model("api_app", "Parameter") | ||
| PluginConfig = apps.get_model("api_app", "PluginConfig") | ||
| python_path = plugin.pop("model") | ||
| Model = apps.get_model(*python_path.split(".")) | ||
| if not Model.objects.filter(name=plugin["name"]).exists(): | ||
| exists = _create_object(Model, plugin) | ||
| if not exists: | ||
| for param in params: | ||
| _create_object(Parameter, param) | ||
| for value in values: | ||
| _create_object(PluginConfig, value) | ||
|
|
||
|
|
||
| def reverse_migrate(apps, schema_editor): | ||
| python_path = plugin.pop("model") | ||
| Model = apps.get_model(*python_path.split(".")) | ||
| Model.objects.get(name=plugin["name"]).delete() | ||
|
|
||
|
|
||
| class Migration(migrations.Migration): | ||
| atomic = False | ||
| dependencies = [ | ||
| ("api_app", "0062_alter_parameter_python_module"), | ||
| ("connectors_manager", "0029_4_change_primary_key"), | ||
| ] | ||
|
|
||
| operations = [migrations.RunPython(migrate, reverse_migrate)] | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.