Skip to content

Prototype/example of an adapter that doesn't leverage the internal datastore #58

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 3 commits into from

Conversation

dgarros
Copy link
Contributor

@dgarros dgarros commented Jun 20, 2021

This PR is a prototype to explore if it's possible to build an adapter that is not leveraging the internal datastore (related to #57)

For this example I modified example3 (still in review #53) to bypass the internal datastore by reimplementing the 5 main methods on the adapter that are interacting with the datastore (get / get_all / get_by_uids / add / remove).
I moved the logic of get / get_all and get_by_uids to each model class instead.

it's working but it's significantly slower, especially when doing a sync with a very small diff, I need to investigate further why but it's a good start.
One thing I noticed is that DiffSync is currently expecting get_all and get_by_uids to return a list where it would be useful to accept of generator instead to improve the processing and the memory consumption.

I'd love to get some feedback on this experiment and how we could package that into DiffSync if we want to move forward.

@dgarros dgarros requested a review from glennmatthews June 20, 2021 20:22
Copy link
Collaborator

@glennmatthews glennmatthews left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting design! Definitely worth exploring further. What issues did you run into when trying to return a generator rather than a list from get_all and get_by_uids? Have you done any sort of memory profiling to see what sort of difference is observed?

@dgarros dgarros force-pushed the dga-examples-nautobot branch 2 times, most recently from 9f56d1f to 5288b5f Compare October 11, 2021 13:04
@chadell chadell mentioned this pull request Apr 13, 2022
3 tasks
@chadell
Copy link
Collaborator

chadell commented May 26, 2022

superseded by #106

@chadell chadell closed this May 26, 2022
@jdrew82 jdrew82 deleted the dga-storeless branch March 27, 2025 22:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants