Skip to content

Version 1.5.1 #119

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Jun 30, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/CODEOWNERS
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
# Default owners for all files in this repository
* @glennmatthews @dgarros
* @glennmatthews @Kircheneer @chadell
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -289,3 +289,6 @@ fabric.properties

## Sphinx Documentation ##
docs/build

## Secrets
creds.env
7 changes: 2 additions & 5 deletions .readthedocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,6 @@ sphinx:
fail_on_warning: false

python:
version: 3.7
version: "3.7"
install:
- method: "pip"
path: "."
extra_requirements:
- "docs"
- requirements: "docs/requirements.txt"
15 changes: 15 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,20 @@
# Changelog

## v1.5.1 - 2022-06-30

### Added

- #111 - Added example 6, regarding IP prefixes.

### Changed

- #107 - Updated example 5 to use the Redis backend store.

### Fixed

- #115 - Fixed ReadTheDocs rendering pipeline
- #118 - Fixed a regression in `DiffSync.get(modelname, identifiers)` introduced in 1.5.0

## v1.5.0 - 2022-06-07

### Added
Expand Down
8 changes: 4 additions & 4 deletions diffsync/store/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -194,9 +194,9 @@ def _get_object_class_and_model(
"""Get object class and model name for a model."""
if isinstance(model, str):
modelname = model
if not hasattr(self, model):
if not hasattr(self.diffsync, model):
return None, modelname
object_class = getattr(self, model)
object_class = getattr(self.diffsync, model)
else:
object_class = model
modelname = model.get_type()
Expand All @@ -216,7 +216,7 @@ def _get_uid(
uid = object_class.create_unique_id(**identifier)
else:
raise ValueError(
f"Invalid args: ({model}, {identifier}): "
f"either {model} should be a class/instance or {identifier} should be a str"
f"Invalid args: ({model}, {object_class}, {identifier}): "
f"either {object_class} should be a class/instance or {identifier} should be a str"
)
return uid
5 changes: 5 additions & 0 deletions docs/requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
m2r2==0.2.7
mistune==0.8.4
sphinx==4.5.0
toml==0.10.2
sphinx-rtd-theme==1.0.0
8 changes: 8 additions & 0 deletions docs/source/api/diffsync.rst
Original file line number Diff line number Diff line change
Expand Up @@ -16,3 +16,11 @@ API Reference
diffsync.helpers
diffsync.logging
diffsync.utils

Subpackages
-----------

.. toctree::
:maxdepth: 4

diffsync.store
7 changes: 7 additions & 0 deletions docs/source/api/diffsync.store.local.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
diffsync.store.local
====================

.. automodule:: diffsync.store.local
:members:
:undoc-members:
:show-inheritance:
7 changes: 7 additions & 0 deletions docs/source/api/diffsync.store.redis.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
diffsync.store.redis
====================

.. automodule:: diffsync.store.redis
:members:
:undoc-members:
:show-inheritance:
14 changes: 14 additions & 0 deletions docs/source/api/diffsync.store.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
API Reference
=============

.. automodule:: diffsync.store
:members:
:undoc-members:
:show-inheritance:


.. toctree::
:maxdepth: 4

diffsync.store.local
diffsync.store.redis
2 changes: 2 additions & 0 deletions docs/source/examples/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,3 +8,5 @@ For each example, the complete source code is `available in Github <https://gith
.. mdinclude:: ../../../examples/02-callback-function/README.md
.. mdinclude:: ../../../examples/03-remote-system/README.md
.. mdinclude:: ../../../examples/04-get-update-instantiate/README.md
.. mdinclude:: ../../../examples/05-nautobot-peeringdb/README.md
.. mdinclude:: ../../../examples/06-ip-prefixes/README.md
16 changes: 8 additions & 8 deletions docs/source/template/api/package.rst_t
Original file line number Diff line number Diff line change
Expand Up @@ -23,13 +23,6 @@
{{ automodule(pkgname, automodule_options) }}
{% endif %}

{%- if subpackages %}
Subpackages
-----------

{{ toctree(subpackages) }}
{% endif %}

{%- if submodules %}
{% if separatemodules %}
{{ toctree(submodules) }}
Expand All @@ -43,9 +36,16 @@ Subpackages
{%- endif %}
{%- endif %}

{%- if subpackages %}
Subpackages
-----------

{{ toctree(subpackages) }}
{% endif %}

{%- if not modulefirst and not is_namespace %}
Module contents
---------------

{{ automodule(pkgname, automodule_options) }}
{% endif %}
{% endif %}
14 changes: 14 additions & 0 deletions examples/05-nautobot-peeringdb/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
ARG PYTHON_VER=3.8.10

FROM python:${PYTHON_VER}-slim

RUN apt-get update \
&& apt-get install -y --no-install-recommends git \
&& apt-get purge -y --auto-remove \
&& rm -rf /var/lib/apt/lists/*

WORKDIR /local
COPY . /local

RUN pip install --upgrade pip \
&& pip install -r requirements.txt
49 changes: 32 additions & 17 deletions examples/05-nautobot-peeringdb/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,45 +4,61 @@

The goal of this example is to synchronize some data from [PeeringDB](https://www.peeringdb.com/), that as the name suggests is a DB where peering entities define their facilities and presence to facilitate peering, towards [Nautobot Demo](https://demo.nautobot.com/) that is a always on demo service for [Nautobot](https://nautobot.readthedocs.io/), an open source Source of Truth.

In Peering DB there is a model that defines a `Facility` and you can get information about the actual data center and the city where it is placed. In Nautobot, this information could be mapped to the `Region` and `Site` models, where `Region` can define hierarchy. For instance, Barcelona is in Spain and Spain is in Europe, and all of them are `Regions`. And, finally, the actual datacenter will refer to the `Region` where it is placed.
In Peering DB there is a model that defines a `Facility` and you can get information about the actual data center and the city where it is placed. In Nautobot, this information could be mapped to the `Region` and `Site` models, where `Region` can depend from other `Region` and also contain `Site` as children. For instance, Barcelona is in Spain and Spain is in Europe, and all of them are `Regions`. And, finally, the actual datacenter will refer to the `Region` where it is placed.

Because of the nature of the demo, we will focus on syncing from PeeringDB to Nautobot (we can assume that PeeringDB is the authoritative System of Record) and we will skip the `delete` part of the `diffsync` library.
Because of the nature of the demo, we will focus on syncing from PeeringDB to Nautobot (we assume that PeeringDB is the authoritative System of Record) and we will skip the `delete` part of the `diffsync` library, using diffsync flags.

We have 3 files:

- `models.py`: defines the reference models that we will use: `RegionMode` and `SiteModel`
- `adapter_peeringdb.py`: defines the PeeringDB adapter to translate via `load()` the data from PeeringDB into the reference models commented above. Notice that we don't define CRUD methods because we will sync from it (no to it)
- `adapter_nautobot.py`: deifnes the Nautobot adapter with the `load()` and the CRUD methods
- `adapter_nautobot.py`: defines the Nautobot adapter with the `load()` and the CRUD methods

> The source code for this example is in Github in the [examples/05-nautobot-peeringdb/](https://github.com/networktocode/diffsync/tree/main/examples/05-nautobot-peeringdb) directory.

## Install dependencies
## Get PeeringDB API Key (optional)

To ensure a good performance from PeeringDB API, you should provide an API Key: https://docs.peeringdb.com/howto/api_keys/

Then, copy the example `creds.example.env` into `creds.env`, and place your new API Key.

```bash
python3 -m venv .venv
source .venv/bin/activate
pip3 install -r requirements.txt
$ cp examples/05-nautobot-peeringdb/creds.example.env examples/05-nautobot-peeringdb/creds.env

```

## Run it interactively
> Without API Key it might also work, but it could fail due to API rate limiting.

```python
from IPython import embed
embed(colors="neutral")
## Set up local docker environment

# Import Adapters
from diffsync.enum import DiffSyncFlags
```bash
$ docker-compose -f examples/05-nautobot-peeringdb/docker-compose.yml up -d --build

$ docker exec -it 05-nautobot-peeringdb_example_1 python
```

## Interactive execution

```python
from adapter_nautobot import NautobotRemote
from adapter_peeringdb import PeeringDB
from diffsync.enum import DiffSyncFlags
from diffsync.store.redis import RedisStore

store_one = RedisStore(host="redis")
store_two = RedisStore(host="redis")

# Initialize PeeringDB adapter, using CATNIX id for demonstration
peeringdb = PeeringDB(ix_id=62)
peeringdb = PeeringDB(
ix_id=62,
internal_storage_engine=store_one
)

# Initialize Nautobot adapter, pointing to the demo instance (it's also the default settings)
nautobot = NautobotRemote(
url="https://demo.nautobot.com",
token="aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
token="a" * 40,
internal_storage_engine=store_two
)

# Load PeeringDB info into the adapter
Expand All @@ -55,12 +71,11 @@ peeringdb.dict()
nautobot.load()

# Let's diffsync do it's magic
diff = nautobot.diff_from(peeringdb)
diff = nautobot.diff_from(peeringdb, flags=DiffSyncFlags.SKIP_UNMATCHED_DST)

# Quick summary of the expected changes (remember that delete ones are dry-run)
diff.summary()

# Execute the synchronization
nautobot.sync_from(peeringdb, flags=DiffSyncFlags.SKIP_UNMATCHED_DST)

```
Loading