-
Notifications
You must be signed in to change notification settings - Fork 4
Folder monitor #72
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
ajCameron
wants to merge
29
commits into
mainline
Choose a base branch
from
folder_monitor
base: mainline
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Folder monitor #72
Changes from all commits
Commits
Show all changes
29 commits
Select commit
Hold shift + click to select a range
28e8f57
Initial work on FileSystemIO
ajCameron 0db0e88
Adding more comprehensive tests to fix the post run file creation issues
ajCameron 3cfdedb
About to completely re-write using asyncinotify instead of watchfiles
ajCameron f5dad36
Prototype is in a functional state
ajCameron ab7309d
Broken file_input down into compoents
ajCameron 246e502
Added explicit events for target object creation
ajCameron bb234df
Encountered an annoying bug during testing
ajCameron c1d5113
windows input is now tested and working
ajCameron c464921
Output tested and working
ajCameron d4e66d3
Added and tested a linux file monitor
ajCameron 71626a3
Updating tests and monitors to try and produce (somewhat similar resu…
ajCameron 5bbe610
Additional tests for the linux file monitor
ajCameron 5d31c81
Windows side tests are passing
ajCameron 517695c
File input mode is now being properly tested on linux
ajCameron 1c7ff21
File input mode is now being properly tested on linux
ajCameron 9b65d04
Testing complete
ajCameron ae6a222
Modified test due to _yet another_ unexpected case
ajCameron 2623cf2
Commit that will probably be re-write
ajCameron a74ed1a
here we go again
ajCameron 172190e
Re-writing to DRY the code base
ajCameron 10968ba
Refactoring code to enable deletion of specific test cases
ajCameron 0fa358c
Timeout added to queue get
ajCameron f1c9784
Linux versions of the tests now working after the refactor.
ajCameron 89d25c8
DRYing out monitors.py
ajCameron 74a48b2
Continuing working towards Sonarcloud compliance
ajCameron de13df7
Attempted fix to address Sonarcloud bugs
ajCameron aa18b10
Sonarcloud line duplicate requirements may be met
ajCameron 0352ee0
Merge branch 'mainline' into folder_monitor
ajCameron 11046c9
Merge branch 'mainline' into folder_monitor
ajCameron File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -31,3 +31,4 @@ examples/TrivialDiscordBot/secrets.json | |
/reports/ | ||
/.pytest_cache/ | ||
/junit.xml | ||
.coverage* |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
Catch all place for individual developers to store notes they don't mind being public |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,11 @@ | ||
|
||
https://github.com/topics/fediverse?l=python | ||
|
||
Standardization and upgrade run - next step | ||
|
||
- standardize all setter values to value | ||
- There's a few places - e.g. in FileSystem - where strings could be enums for clarity | ||
- And type(self).__name__ could be used in a bunch of different places to save on changes after a class rename | ||
- Consistent and present docstrings everywhere | ||
|
||
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
The aim of this folder is to include notes written by the developers as they were working. | ||
These will - probably - eventually - end up on some kind of wiki. | ||
|
||
Feel free to write as much (or as little) as you like. | ||
|
||
Personally, these are my working notes. So expect some digressions and random other useful stuff. |
Large diffs are not rendered by default.
Oops, something went wrong.
Empty file.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,110 @@ | ||
#!/usr/bin/env python3 | ||
# pylint: disable=duplicate-code | ||
# this is an example - duplication for emphasis is desirable | ||
|
||
from __future__ import annotations | ||
|
||
from typing import Any, Dict, Set, Type | ||
|
||
import logging | ||
|
||
from mewbot.api.v1 import Trigger, Action | ||
from mewbot.core import InputEvent, OutputEvent, OutputQueue | ||
from mewbot.io.file_system import ( | ||
CreatedDirFSInputEvent, | ||
UpdatedDirFSInputEvent, | ||
MovedDirFSInputEvent, | ||
DeletedDirFSInputEvent, | ||
CreatedFileFSInputEvent, | ||
UpdatedFileFSInputEvent, | ||
MovedFileFSInputEvent, | ||
DeletedFileFSInputEvent, | ||
InputFileDirCreationInputEvent, | ||
InputFileDirDeletionInputEvent, | ||
) | ||
|
||
|
||
class DirSystemAllCommandTrigger(Trigger): | ||
""" | ||
Nothing fancy - just fires whenever there is a dir related FSInputEvent | ||
""" | ||
|
||
@staticmethod | ||
def consumes_inputs() -> Set[Type[InputEvent]]: | ||
return { | ||
CreatedDirFSInputEvent, | ||
UpdatedDirFSInputEvent, | ||
MovedDirFSInputEvent, | ||
DeletedDirFSInputEvent, | ||
CreatedFileFSInputEvent, | ||
UpdatedFileFSInputEvent, | ||
MovedFileFSInputEvent, | ||
DeletedFileFSInputEvent, | ||
InputFileDirCreationInputEvent, | ||
InputFileDirDeletionInputEvent, | ||
} | ||
|
||
def matches(self, event: InputEvent) -> bool: | ||
|
||
print("-------\n", "event seen by matches - ", event, "\n-------") | ||
|
||
if not isinstance( | ||
event, | ||
( | ||
CreatedDirFSInputEvent, | ||
UpdatedDirFSInputEvent, | ||
MovedDirFSInputEvent, | ||
DeletedDirFSInputEvent, | ||
CreatedFileFSInputEvent, | ||
MovedFileFSInputEvent, | ||
UpdatedFileFSInputEvent, | ||
DeletedFileFSInputEvent, | ||
InputFileDirCreationInputEvent, | ||
InputFileDirDeletionInputEvent, | ||
), | ||
): | ||
return False | ||
|
||
return True | ||
|
||
|
||
class DirSystemInputPrintResponse(Action): | ||
""" | ||
Print every DirSystem Dir related InputEvent. | ||
""" | ||
|
||
_logger: logging.Logger | ||
_queue: OutputQueue | ||
|
||
def __init__(self) -> None: | ||
super().__init__() | ||
self._logger = logging.getLogger(__name__ + type(self).__name__) | ||
|
||
@staticmethod | ||
def consumes_inputs() -> Set[Type[InputEvent]]: | ||
return { | ||
CreatedDirFSInputEvent, | ||
UpdatedDirFSInputEvent, | ||
MovedDirFSInputEvent, | ||
DeletedDirFSInputEvent, | ||
CreatedFileFSInputEvent, | ||
UpdatedFileFSInputEvent, | ||
MovedFileFSInputEvent, | ||
DeletedFileFSInputEvent, | ||
InputFileDirCreationInputEvent, | ||
InputFileDirDeletionInputEvent, | ||
} | ||
|
||
@staticmethod | ||
def produces_outputs() -> Set[Type[OutputEvent]]: | ||
return set() | ||
|
||
async def act(self, event: InputEvent, state: Dict[str, Any]) -> None: | ||
""" | ||
Construct a DiscordOutputEvent with the result of performing the calculation. | ||
""" | ||
if not isinstance(event, InputEvent): | ||
self._logger.warning("Received wrong event type %s", type(event)) | ||
return | ||
|
||
print("-------\n", "event seen by action - ", event, "\n-------") |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,24 @@ | ||
kind: IOConfig | ||
implementation: mewbot.io.file_system.FileSystemIO | ||
uuid: aaaaaaaa-aaaa-4aaa-0001-aaaaaaaaaa00 | ||
properties: | ||
input_path: C:\mewbot_dev\new_dir | ||
input_path_type: dir | ||
--- | ||
|
||
kind: Behaviour | ||
implementation: mewbot.api.v1.Behaviour | ||
uuid: aaaaaaaa-aaaa-4aaa-0001-aaaaaaaaaa01 | ||
properties: | ||
name: 'Target Dir Change Alerts' | ||
triggers: | ||
- kind: Trigger | ||
implementation: examples.file_system_bots.dir_input_monitor_bot.DirSystemAllCommandTrigger | ||
uuid: aaaaaaaa-aaaa-4aaa-0001-aaaaaaaaaa02 | ||
properties: {} | ||
conditions: [] | ||
actions: | ||
- kind: Action | ||
implementation: examples.file_system_bots.dir_input_monitor_bot.DirSystemInputPrintResponse | ||
uuid: aaaaaaaa-aaaa-4aaa-0001-aaaaaaaaaa03 | ||
properties: {} |
23 changes: 23 additions & 0 deletions
23
examples/file_system_bots/file_input_None_path_monitor_bot.yaml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,23 @@ | ||
kind: IOConfig | ||
implementation: mewbot.io.file_system.FileSystemIO | ||
uuid: aaaaaaaa-aaaa-4aaa-0001-aaaaaaaaaa00 | ||
properties: | ||
input_path: None | ||
--- | ||
|
||
kind: Behaviour | ||
implementation: mewbot.api.v1.Behaviour | ||
uuid: aaaaaaaa-aaaa-4aaa-0001-aaaaaaaaaa01 | ||
properties: | ||
name: 'Target File Change Alerts - but with None path - so it just sits here - hopefully not eating all resources' | ||
triggers: | ||
- kind: Trigger | ||
implementation: examples.file_system_bots.file_input_monitor_bot.FileSystemAllCommandTrigger | ||
uuid: aaaaaaaa-aaaa-4aaa-0001-aaaaaaaaaa02 | ||
properties: {} | ||
conditions: [] | ||
actions: | ||
- kind: Action | ||
implementation: examples.file_system_bots.file_input_monitor_bot.FileSystemInputPrintResponse | ||
uuid: aaaaaaaa-aaaa-4aaa-0001-aaaaaaaaaa03 | ||
properties: {} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
@@ -0,0 +1,81 @@ | ||||||
#!/usr/bin/env python3 | ||||||
# pylint: disable=duplicate-code | ||||||
# this is an example - duplication for emphasis is desirable | ||||||
|
||||||
from __future__ import annotations | ||||||
|
||||||
from typing import Any, Dict, Set, Type | ||||||
|
||||||
import logging | ||||||
|
||||||
from mewbot.api.v1 import Trigger, Action | ||||||
from mewbot.core import InputEvent, OutputEvent, OutputQueue | ||||||
from mewbot.io.file_system import ( | ||||||
FSInputEvent, | ||||||
UpdatedFileFSInputEvent, | ||||||
InputFileFileCreationInputEvent, | ||||||
InputFileFileDeletionInputEvent, | ||||||
) | ||||||
|
||||||
|
||||||
class FileSystemAllCommandTrigger(Trigger): | ||||||
""" | ||||||
Nothing fancy - just fires whenever there is a file related FSInputEvent | ||||||
""" | ||||||
|
||||||
@staticmethod | ||||||
def consumes_inputs() -> Set[Type[InputEvent]]: | ||||||
return { | ||||||
InputFileFileCreationInputEvent, | ||||||
UpdatedFileFSInputEvent, | ||||||
InputFileFileDeletionInputEvent, | ||||||
} | ||||||
|
||||||
def matches(self, event: InputEvent) -> bool: | ||||||
|
||||||
if not isinstance( | ||||||
event, | ||||||
( | ||||||
InputFileFileCreationInputEvent, | ||||||
UpdatedFileFSInputEvent, | ||||||
InputFileFileDeletionInputEvent, | ||||||
), | ||||||
): | ||||||
return False | ||||||
|
||||||
return True | ||||||
|
||||||
|
||||||
class FileSystemInputPrintResponse(Action): | ||||||
""" | ||||||
Print every FileSystem File related InputEvent. | ||||||
""" | ||||||
|
||||||
_logger: logging.Logger | ||||||
_queue: OutputQueue | ||||||
|
||||||
def __init__(self) -> None: | ||||||
super().__init__() | ||||||
self._logger = logging.getLogger(__name__ + type(self).__name__) | ||||||
|
||||||
@staticmethod | ||||||
def consumes_inputs() -> Set[Type[InputEvent]]: | ||||||
return { | ||||||
InputFileFileCreationInputEvent, | ||||||
UpdatedFileFSInputEvent, | ||||||
InputFileFileDeletionInputEvent, | ||||||
} | ||||||
|
||||||
@staticmethod | ||||||
def produces_outputs() -> Set[Type[OutputEvent]]: | ||||||
return set() | ||||||
|
||||||
async def act(self, event: InputEvent, state: Dict[str, Any]) -> None: | ||||||
""" | ||||||
Construct a DiscordOutputEvent with the result of performing the calculation. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
""" | ||||||
if not isinstance(event, FSInputEvent): | ||||||
self._logger.warning("Received wrong event type %s", type(event)) | ||||||
return | ||||||
|
||||||
print(event) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,24 @@ | ||
kind: IOConfig | ||
implementation: mewbot.io.file_system.FileSystemIO | ||
uuid: aaaaaaaa-aaaa-4aaa-0001-aaaaaaaaaa00 | ||
properties: | ||
input_path: 'C:\mewbot_dev\test_file.md' | ||
input_path_type: 'file' | ||
--- | ||
|
||
kind: Behaviour | ||
implementation: mewbot.api.v1.Behaviour | ||
uuid: aaaaaaaa-aaaa-4aaa-0001-aaaaaaaaaa01 | ||
properties: | ||
name: 'Target File Change Alerts' | ||
triggers: | ||
- kind: Trigger | ||
implementation: examples.file_system_bots.file_input_monitor_bot.FileSystemAllCommandTrigger | ||
uuid: aaaaaaaa-aaaa-4aaa-0001-aaaaaaaaaa02 | ||
properties: {} | ||
conditions: [] | ||
actions: | ||
- kind: Action | ||
implementation: examples.file_system_bots.file_input_monitor_bot.FileSystemInputPrintResponse | ||
uuid: aaaaaaaa-aaaa-4aaa-0001-aaaaaaaaaa03 | ||
properties: {} |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.