Use Django-storages for scripts and Replace STORAGE_BACKEND and STORAGE_CONFIG configuration parameters with Django's STORAGES

[arthanson]

NetBox version

v4.2.5

Feature type

Change to existing functionality

Proposed functionality

NetBox currently uses the local file system for saving and running scripts - this would change it to use Django-storages which would allow uploading and running scripts from S3 or other remote storages.

This would require more flexibility in configuration of STORAGES than is currently provided by the config setting STORAGE_BACKEND, so using STORAGES directly in configuration.py directly would be better. NetBox currently exposes two configuration parameters for configuring file storage:

• STORAGE_BACKEND - Sets the storage backend (e.g. django-storages)
• STORAGE_CONFIG - A dictionary of backend-specific configuration parameters

We can replace these by instead exposing Django's STORAGES setting directly.

Use case

There are some use cases where local file system is not available or directly accessible on the instance of NetBox (some docker configurations and cloud). This can be the case for ephemeral storage, read-only lockdown on Docker containers or stricter permission issues.

When using Kubernetes for deployment the instances can be ephemeral which means scripts are not directly copied to each node, so another layer is needed to sync the scripts (eventual consistency) which can lead to problems with the copying process not being completed by the time the script is run.

S3FS can be used in some cases to mount S3 as a local drive, but it has issues as it does not support Amazon Linux and can have permission issues.

Changing the script code to use Django-storages would fix these issues and allow scripts to be run from remote storages.

Note: Django's native STORAGE_BACKEND setting was removed in v5.1 and superseded by STORAGES. There is no longer a need to handle the backend selection and configuration via separate configuration parameters.

The extent of these changes would be:

• Allow settings STORAGES directly in configuration.py instead of STORAGE_BACKEND and STORAGE_CONFIG, depreciate these two settings.
• Switch the script file upload code to use Django-storages
• Allow the script running code (specifically the inspect python module specific code that loads modules) to use Django-storages. Note: inspect only directly supports direct file system access, so some changes will need to be made to support it using Django-storages.

Database changes

N/A

External dependencies

Django-storages