intelowlproject
diff --git a/‎.github/CHANGELOG.md‎
Lines changed: 16 additions & 0 deletions b/‎.github/CHANGELOG.md‎
Lines changed: 16 additions & 0 deletions
diff --git a/‎.github/pull_request_template.md‎
Lines changed: 4 additions & 4 deletions b/‎.github/pull_request_template.md‎
Lines changed: 4 additions & 4 deletions
diff --git a/‎.github/workflows/codeql-analysis.yml‎
Lines changed: 3 additions & 3 deletions b/‎.github/workflows/codeql-analysis.yml‎
Lines changed: 3 additions & 3 deletions
diff --git a/‎.github/workflows/pull_request_automation.yml‎
Lines changed: 1 addition & 1 deletion b/‎.github/workflows/pull_request_automation.yml‎
Lines changed: 1 addition & 1 deletion
diff --git a/‎.github/workflows/scorecard.yml‎
Lines changed: 1 addition & 1 deletion b/‎.github/workflows/scorecard.yml‎
Lines changed: 1 addition & 1 deletion
diff --git a/‎.github/workflows/sphinx_build_on_release.yml‎
Lines changed: 1 addition & 1 deletion b/‎.github/workflows/sphinx_build_on_release.yml‎
Lines changed: 1 addition & 1 deletion
diff --git a/‎api_app/analyzers_manager/migrations/0052_analyzer_config_dns0_names.py‎
Lines changed: 209 additions & 0 deletions b/‎api_app/analyzers_manager/migrations/0052_analyzer_config_dns0_names.py‎
Lines changed: 209 additions & 0 deletions
@@ -2,8 +2,24 @@
 
 [**Upgrade Guide**](https://intelowl.readthedocs.io/en/latest/Installation.md#update-to-the-most-recent-version)
 
+## [v5.2.2](https://github.com/intelowlproject/IntelOwl/releases/tag/v5.2.2)
+
+This release has been done mainly to adjusts a broken database migration introduced in the previous release.
+
+**Main Improvements**
+* Added new analyzers for [DNS0](https://docs.dns0.eu/) PassiveDNS data
+* Added the chance to collect metrics ([Business Intelligence](https://intelowl.readthedocs.io/en/develop/Advanced-Configuration.html#business-intelligence) regarding Plugins Usage and send it to an ElasticSearch instance.
+* Added new buttons to test ["Healthcheck" and "Pull" operations](https://intelowl.readthedocs.io/en/latest/Usage.html#special-plugins-operations) for each Plugin (A feature introduced in the previous version)
+
+**Other improvements**
+* Various generic fixes and adjustments in the GUI
+* dependencies upgrades
+* adjusted contribution guides
+
 ## [v5.2.1](https://github.com/intelowlproject/IntelOwl/releases/tag/v5.2.1)
 
+!!! This release has been found with a broken database migration !!! Please upgrade to v5.2.2 to fix the problem.
+
 **General improvements**
 * Incremented wait time of containers' healthchecks to avoid to break clean installations
 * Improvements to the "Scan page":
 
@@ -19,17 +19,17 @@ Please delete options that are not relevant.
 - [ ] A new plugin (analyzer, connector, visualizer, playbook, pivot or ingestor) was added or changed, in which case:
     - [ ] I strictly followed the documentation ["How to create a Plugin"](https://intelowl.readthedocs.io/en/latest/Contribute.html#how-to-add-a-new-plugin)
     - [ ] [Usage](https://github.com/intelowlproject/IntelOwl/blob/master/docs/source/Usage.md) file was updated.
-    - [ ] [Advanced-Usage](./Advanced-Usage.md) was updated (in case the plugin provides additional optional configuration).
+    - [ ] [Advanced-Usage](https://github.com/intelowlproject/IntelOwl/blob/master/docs/source/Advanced-Usage.md) was updated (in case the plugin provides additional optional configuration).
     - [ ] If the plugin requires mocked testing, `_monkeypatch()` was used in its class to apply the necessary decorators.
-    - [ ] I have dumped the configuration from Django Admin using the `dumpplugin` command and added it in the project as a data migration. [Doc](["How to create a Plugin"](https://intelowl.readthedocs.io/en/latest/Contribute.html#how-to-add-a-new-plugin))
+    - [ ] I have dumped the configuration from Django Admin using the `dumpplugin` command and added it in the project as a data migration. (["How to share a plugin with the community"](https://intelowl.readthedocs.io/en/latest/Contribute.html#how-to-share-your-plugin-with-the-community))
     - [ ] If a File analyzer was added and it supports a mimetype which is not already supported, you added a sample of that type inside the archive `test_files.zip` and you added the default tests for that mimetype in [test_classes.py](https://github.com/intelowlproject/IntelOwl/blob/master/tests/analyzers_manager/test_classes.py).
-    - [ ] If you created a new analyzer and it is free (does not require API keys), please add it in the `FREE_TO_USE_ANALYZERS` playbook in `playbook_config.json`.
+    - [ ] If you created a new analyzer and it is free (does not require API keys), please add it in the `FREE_TO_USE_ANALYZERS` playbook by following [this guide](https://intelowl.readthedocs.io/en/latest/Contribute.html#how-to-modify-a-plugin).
     - [ ] Check if it could make sense to add that analyzer/connector to other [freely available playbooks](https://intelowl.readthedocs.io/en/develop/Usage.html#list-of-pre-built-playbooks).
     - [ ] I have provided the resulting raw JSON of a finished analysis and a screenshot of the results.
 - [ ] If external libraries/packages with restrictive licenses were used, they were added in the [Legal Notice](https://github.com/certego/IntelOwl/blob/master/.github/legal_notice.md) section.
 - [ ] Linters (`Black`, `Flake`, `Isort`) gave 0 errors. If you have correctly installed [pre-commit](https://intelowl.readthedocs.io/en/latest/Contribute.html#how-to-start-setup-project-and-development-instance), it does these checks and adjustments on your behalf.
 - [ ] I have added tests for the feature/bug I solved (see `tests` folder). All the tests (new and old ones) gave 0 errors.
-- [ ] If changes were made to an existing model/serializer/view, the docs were updated and regenerated (check [CONTRIBUTE.md](./Contribute.md)).
+- [ ] If changes were made to an existing model/serializer/view, the docs were updated and regenerated (check [CONTRIBUTE.md](https://github.com/intelowlproject/IntelOwl/blob/master/docs/source/Contribute.md)).
 - [ ] If the GUI has been modified:
     - [ ] I have a provided a screenshot of the result in the PR.
     - [ ] I have created new frontend tests for the new component or updated existing ones.
 
@@ -44,7 +44,7 @@ jobs:
         fetch-depth: 2
 
     - name: Set up Python
-      uses: actions/setup-python@v4.7.0
+      uses: actions/setup-python@v5.0.0
       with:
         python-version: '3.9'
 
@@ -65,7 +65,7 @@ jobs:
 
     # Initializes the CodeQL tools for scanning.
     - name: Initialize CodeQL
-      uses: github/codeql-action/init@v2
+      uses: github/codeql-action/init@v3
       with:
         languages: python
         # Override the default behavior so that the action doesn't attempt
@@ -93,4 +93,4 @@ jobs:
     #   make release
 
     - name: Perform CodeQL Analysis
-      uses: github/codeql-action/analyze@v2
+      uses: github/codeql-action/analyze@v3
@@ -37,7 +37,7 @@ jobs:
         uses: actions/[email protected]
 
       - name: Set up Python
-        uses: actions/setup-python@v4.7.0
+        uses: actions/setup-python@v5.0.0
         with:
           python-version: 3.9
 
 
@@ -67,6 +67,6 @@ jobs:
 
       # Upload the results to GitHub's code scanning dashboard.
       - name: "Upload to code-scanning"
-        uses: github/codeql-action/upload-sarif@807578363a7869ca324a79039e6db9c843e0e100 # v2.1.27
+        uses: github/codeql-action/upload-sarif@03e7845b7bfcd5e7fb63d1ae8c61b0e791134fab # v2.22.11
         with:
           sarif_file: results.sarif
@@ -14,7 +14,7 @@ jobs:
     steps:
       - uses: actions/[email protected]
       - name: Set up Python
-        uses: actions/setup-python@v4.7.0
+        uses: actions/setup-python@v5.0.0
         with:
           python-version: 3.9
       - name: Install docs requirements
 
@@ -0,0 +1,209 @@
+from django.db import migrations
+from django.db.models.fields.related_descriptors import (
+    ForwardManyToOneDescriptor,
+    ForwardOneToOneDescriptor,
+    ManyToManyDescriptor,
+)
+
+plugin = {
+    "name": "DNS0_names",
+    "python_module": {
+        "module": "dns0.dns0_names.DNS0Names",
+        "base_path": "api_app.analyzers_manager.observable_analyzers",
+    },
+    "description": "Run advanced searches on billions of current and historical domain names. [API](https://docs.dns0.eu/dns-api/names).",
+    "disabled": False,
+    "soft_time_limit": 60,
+    "routing_key": "default",
+    "health_check_status": True,
+    "type": "observable",
+    "docker_based": False,
+    "maximum_tlp": "RED",
+    "observable_supported": ["url", "domain", "generic"],
+    "supported_filetypes": [],
+    "run_hash": False,
+    "run_hash_type": "",
+    "not_supported_filetypes": [],
+    "health_check_task": None,
+    "model": "analyzers_manager.AnalyzerConfig",
+}
+
+params = [
+    {
+        "name": "api_key",
+        "type": "str",
+        "description": "",
+        "is_secret": True,
+        "required": False,
+        "python_module": {
+            "module": "dns0.dns0_names.DNS0Names",
+            "base_path": "api_app.analyzers_manager.observable_analyzers",
+        },
+    },
+    {
+        "name": "root",
+        "type": "bool",
+        "description": "Limit results to root domains.",
+        "is_secret": False,
+        "required": False,
+        "python_module": {
+            "module": "dns0.dns0_names.DNS0Names",
+            "base_path": "api_app.analyzers_manager.observable_analyzers",
+        },
+    },
+    {
+        "name": "fuzzy",
+        "type": "list",
+        "description": "Apply fuzziness to q. Accepts a comma-separated list of fuzzy algorithms, or all to apply them all.",
+        "is_secret": False,
+        "required": False,
+        "python_module": {
+            "module": "dns0.dns0_names.DNS0Names",
+            "base_path": "api_app.analyzers_manager.observable_analyzers",
+        },
+    },
+    {
+        "name": "from",
+        "type": "str",
+        "description": "Limit results to names seen after this date.",
+        "is_secret": False,
+        "required": False,
+        "python_module": {
+            "module": "dns0.dns0_names.DNS0Names",
+            "base_path": "api_app.analyzers_manager.observable_analyzers",
+        },
+    },
+    {
+        "name": "to",
+        "type": "str",
+        "description": "Limit results to names seen before this date.",
+        "is_secret": False,
+        "required": False,
+        "python_module": {
+            "module": "dns0.dns0_names.DNS0Names",
+            "base_path": "api_app.analyzers_manager.observable_analyzers",
+        },
+    },
+    {
+        "name": "not_before",
+        "type": "str",
+        "description": "Limit results to names not seen before this date.",
+        "is_secret": False,
+        "required": False,
+        "python_module": {
+            "module": "dns0.dns0_names.DNS0Names",
+            "base_path": "api_app.analyzers_manager.observable_analyzers",
+        },
+    },
+    {
+        "name": "sort",
+        "type": "str",
+        "description": "Available sorts are first_seen (the default) or last_seen. Both are descending sorts (most recent first).",
+        "is_secret": False,
+        "required": False,
+        "python_module": {
+            "module": "dns0.dns0_names.DNS0Names",
+            "base_path": "api_app.analyzers_manager.observable_analyzers",
+        },
+    },
+    {
+        "name": "format",
+        "type": "str",
+        "description": "Available formats are json and dig. Default format is based on the Accept HTTP header.",
+        "is_secret": False,
+        "required": False,
+        "python_module": {
+            "module": "dns0.dns0_names.DNS0Names",
+            "base_path": "api_app.analyzers_manager.observable_analyzers",
+        },
+    },
+    {
+        "name": "limit",
+        "type": "int",
+        "description": "Limit the number of results.",
+        "is_secret": False,
+        "required": False,
+        "python_module": {
+            "module": "dns0.dns0_names.DNS0Names",
+            "base_path": "api_app.analyzers_manager.observable_analyzers",
+        },
+    },
+    {
+        "name": "offset",
+        "type": "int",
+        "description": "Used for pagination.",
+        "is_secret": False,
+        "required": False,
+        "python_module": {
+            "module": "dns0.dns0_names.DNS0Names",
+            "base_path": "api_app.analyzers_manager.observable_analyzers",
+        },
+    },
+]
+
+values = []
+
+
+def _get_real_obj(Model, field, value):
+    if (
+        type(getattr(Model, field))
+        in [ForwardManyToOneDescriptor, ForwardOneToOneDescriptor]
+        and value
+    ):
+        other_model = getattr(Model, field).get_queryset().model
+        # in case is a dictionary, we have to retrieve the object with every key
+        if isinstance(value, dict):
+            real_vals = {}
+            for key, real_val in value.items():
+                real_vals[key] = _get_real_obj(other_model, key, real_val)
+            value = other_model.objects.get_or_create(**real_vals)[0]
+        # it is just the primary key serialized
+        else:
+            value = other_model.objects.get(pk=value)
+    return value
+
+
+def _create_object(Model, data):
+    mtm, no_mtm = {}, {}
+    for field, value in data.items():
+        if type(getattr(Model, field)) is ManyToManyDescriptor:
+            mtm[field] = value
+        else:
+            value = _get_real_obj(Model, field, value)
+            no_mtm[field] = value
+    try:
+        o = Model.objects.get(**no_mtm)
+    except Model.DoesNotExist:
+        o = Model(**no_mtm)
+        o.full_clean()
+        o.save()
+        for field, value in mtm.items():
+            attribute = getattr(o, field)
+            attribute.set(value)
+
+
+def migrate(apps, schema_editor):
+    Parameter = apps.get_model("api_app", "Parameter")
+    PluginConfig = apps.get_model("api_app", "PluginConfig")
+    python_path = plugin.pop("model")
+    Model = apps.get_model(*python_path.split("."))
+    _create_object(Model, plugin)
+    for param in params:
+        _create_object(Parameter, param)
+    for value in values:
+        _create_object(PluginConfig, value)
+
+
+def reverse_migrate(apps, schema_editor):
+    python_path = plugin.pop("model")
+    Model = apps.get_model(*python_path.split("."))
+    Model.objects.get(name=plugin["name"]).delete()
+
+
+class Migration(migrations.Migration):
+    dependencies = [
+        ("api_app", "0051_pythonmodule_health_check_schedule_and_more"),
+        ("analyzers_manager", "0051_analyzerreport_parameters"),
+    ]
+
+    operations = [migrations.RunPython(migrate, reverse_migrate)]