Dns0 api integration (#2078)

* First not tested prototype

* Added _monkeypatch() skeleton

* Fixed a wrong condition in parameter validation

* Added config() function to prepare data for request

* Added hasattr() check for attributes

* First successful run

- Added check on existence of API parameter
- Added authorization header
- Added checks on parameters existence

* Fix _api_name parameter check

* Fixed parameter configuration

* Removed unused if statement

* Added support for UNIX timestamps and relative dates

* Renamed dns0 service to separate services

* Completed checks for not set parameters

* Added checks of existence for all parameters

* Added default Accept format header

* Simplified regex to match single relative date

* Fixed date check by forcing format

* Modified _api_key to be non required

* Made _api_key optional to support 10 free queries/day

* Added /rrsets API

* Moved files into separate dns0/ folder

* Created module structure and base file

* Extracted a common Mixin and applied to subclasses

* Resolved performance issue flagged by deepsource

* Added absolute import path

* Substituted dateutils with dateparser and updated requirements

* Updated Usage.md

* Added explanation in Advanced-Usage.md

* Fix dump

Signed-off-by: 0ssigeno <[email protected]>

* Black

Signed-off-by: 0ssigeno <[email protected]>

* Added analyzer config migrations

* Added new analyzers to FREE_TO_USE_ANALYZERS playbook

* Removed dns query type check

* Removed unused variable 'query'

* Added explicit default value in 'limit' parameter

* Added DNS0 api docs reference

* Solved deepsource warnings

* Added custom user agent

* Removed DNS0_rrsets_data

* Solved deepsource warning

* Added example JSON data

* Generalization of ids

Signed-off-by: 0ssigeno <[email protected]>

* Fix dumpplugin migration generation

Signed-off-by: 0ssigeno <[email protected]>

* Excluding id

Signed-off-by: 0ssigeno <[email protected]>

* Excluding id in migration

Signed-off-by: 0ssigeno <[email protected]>

* Added default parameter in test

* Added link to the DNS0 doc in plugin description

* Fix pipeline test #1

* Fix pipeline test n2

* Fix pipeline test n3

* Fix pipeline test n4

* Fix pipeline test n5

* Fix pipeline test n6

* Fix pipeline test n7

* Fix pipeline test n8

* Fix pipeline test n9

* Fix pipeline test n10

* Fix pipeline test n11

* Fix pipeline test n12

* Fix pipeline test n13

* Fix pipeline test n14

* Fix pipeline test n15

* Fix pipeline test n16

* Fix pipeline test n17

* Removed unused q parameter

* Solved merge conflict

* Added migration for removing URL from types

* Fixed migration to remove URL

---------

Signed-off-by: 0ssigeno <[email protected]>
Co-authored-by: 0ssigeno <[email protected]>

Author: fgibertoni

api_app/analyzers_manager/migrations/0059_alter_analyzer_config_dns0_rrsets_data.py

@@ -0,0 +1,36 @@
+from django.db import migrations
+
+from api_app.analyzers_manager.constants import ObservableTypes
+
+
+def migrate(apps, schema_editor):
+    AnalyzerConfig = apps.get_model("analyzers_manager", "AnalyzerConfig")
+    config = AnalyzerConfig.objects.get(name="DNS0_rrsets_data")
+    config.observable_supported = [
+        ObservableTypes.DOMAIN,
+        ObservableTypes.GENERIC,
+        ObservableTypes.IP,
+    ]
+    config.full_clean()
+    config.save()
+
+
+def reverse_migrate(apps, schema_editor):
+    AnalyzerConfig = apps.get_model("analyzers_manager", "AnalyzerConfig")
+    config = AnalyzerConfig.objects.get(name="DNS0_rrsets_data")
+    config.observable_supported = [
+        ObservableTypes.DOMAIN,
+        ObservableTypes.URL,
+        ObservableTypes.GENERIC,
+        ObservableTypes.IP,
+    ]
+    config.full_clean()
+    config.save()
+
+
+class Migration(migrations.Migration):
+    dependencies = [
+        ("analyzers_manager", "0058_4_change_primary_key"),
+    ]
+
+    operations = [migrations.RunPython(migrate, reverse_migrate)]