Skip to content

Add events command to legal-hold #264

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 34 commits into from
Apr 15, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
b285a68
Legal Hold work to meet Issue 176
maddie-vargo Dec 28, 2020
ed72879
Fix to Changelog
maddie-vargo Dec 28, 2020
ea4381a
Minor fix to CHANGELOG
maddie-vargo Dec 29, 2020
a0ebe57
Added to legal hold user guide
maddie-vargo Dec 29, 2020
077dea1
Adjusting build parameters to bypass 3.5 for this PR
maddie-vargo Dec 29, 2020
03a3814
Merge branch 'master' into iss176
maddie-vargo Dec 29, 2020
f174992
Fix low hanging fruit for initial PR review
maddie-vargo Jan 4, 2021
205d559
Move development from legal-hold command to devices command, add new …
maddie-vargo Jan 29, 2021
02d6530
remove whitespaces that are coming through as edits
maddie-vargo Feb 2, 2021
e5784cf
fix changes identfied by tox style run
maddie-vargo Feb 2, 2021
9bfcd42
remove duplication in setup.py - file should have no edits
maddie-vargo Feb 2, 2021
2e303de
remove duplication in setup.py - file should have no edits
maddie-vargo Feb 2, 2021
94fe3b9
refactor membership function to use generator and remove NaNs from ou…
maddie-vargo Feb 11, 2021
0b79911
fix tox style run issue
maddie-vargo Feb 11, 2021
1bd1e2f
Fix tox style run x2
maddie-vargo Feb 11, 2021
e4725c7
flipping back to using NaN, awaiting PR #245
maddie-vargo Feb 16, 2021
237ea31
Adding --include-total-storage option, which calculates total number …
maddie-vargo Feb 18, 2021
7212fbf
Remove V2 archives from storage calcuation; rename columns
maddie-vargo Feb 22, 2021
2a917bd
fix small change to the incldue/excluded archive types
maddie-vargo Feb 23, 2021
2d1db8c
reword
maddie-vargo Feb 25, 2021
9a0afcb
conflict reconciliation in changelog, part I
maddie-vargo Feb 26, 2021
a3dd28f
conflict reconciliation in changelog, part II (repulled from upstream…
maddie-vargo Feb 26, 2021
9677f23
fix style run
maddie-vargo Feb 26, 2021
c328588
Initial development for command
maddie-vargo Apr 1, 2021
2764bf0
Fix to changelog
maddie-vargo Apr 1, 2021
537289f
Refactored matter option, help text updates, run tests off generator …
maddie-vargo Apr 6, 2021
2ab1ae0
Add test for option and remove list-based mocks
maddie-vargo Apr 6, 2021
73daebb
removed excess fixture/object and updated variable names
maddie-vargo Apr 7, 2021
d8921c5
Fix CHANGELOG.md conflicts
maddie-vargo Apr 7, 2021
285a1ec
Fix CHANGELONG.md conflicts
maddie-vargo Apr 7, 2021
5e49055
Merge branch 'master' into iss176-events
maddie-vargo Apr 7, 2021
e7d6a5d
help text change
maddie-vargo Apr 8, 2021
8e55a4a
Merge branch 'iss176-events' of https://github.com/maddie-vargo/code4…
maddie-vargo Apr 8, 2021
693126f
Reworking CHANGELOG.md after today's bug-fix release
maddie-vargo Apr 15, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 12 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,20 @@ how a consumer would use the library (e.g. adding unit tests, updating documenta

## Unreleased

### Added

- `code42 legal-hold search-events` command:
- `--matter-id` filters based on a legal hold uid.
- `--begin` filters based on a beginning timestamp.
- `--end` filters based on an end timestamp.
- `--event-type` filters based on a list of event types.

## 1.4.1 - 2021-04-15

### Fixed

- Arguments/options that read data from files now attempt to autodetect file encodings. Resolving a bug where CSVs written
on Windows with Powershell would fail to be read properly.
- Arguments/options that read data from files now attempt to autodetect file encodings.
Resolving a bug where CSVs written on Windows with Powershell would fail to be read properly.

## 1.4.0 - 2021-03-09

Expand Down
8 changes: 8 additions & 0 deletions docs/userguides/legalhold.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,4 +93,12 @@ To view all custodians (including inactive) for a legal hold matter, enter

`code42 legal-hold show <matterID> --include-inactive`

### List legal hold events

To view a list of legal hold administrative events, use the following command:

`code42 legal-hold search-events`

This command takes the optional filters of a specific matter uid, beginning timestamp, end timestamp, and event type.

Learn more about the [Legal Hold](../commands/legalhold.md) commands.
90 changes: 73 additions & 17 deletions src/code42cli/cmds/legal_hold.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
import json
from collections import OrderedDict
from functools import lru_cache
from pprint import pformat

Expand All @@ -14,33 +13,52 @@
from code42cli.file_readers import read_csv_arg
from code42cli.options import format_option
from code42cli.options import sdk_options
from code42cli.options import set_begin_default_dict
from code42cli.options import set_end_default_dict
from code42cli.output_formats import OutputFormat
from code42cli.output_formats import OutputFormatter
from code42cli.util import format_string_list_to_columns


_MATTER_KEYS_MAP = OrderedDict()
_MATTER_KEYS_MAP["legalHoldUid"] = "Matter ID"
_MATTER_KEYS_MAP["name"] = "Name"
_MATTER_KEYS_MAP["description"] = "Description"
_MATTER_KEYS_MAP["creator_username"] = "Creator"
_MATTER_KEYS_MAP["creationDate"] = "Creation Date"
_MATTER_KEYS_MAP = {
"legalHoldUid": "Matter ID",
"name": "Name",
"description": "Description",
"creator_username": "Creator",
"creationDate": "Creation Date",
}
_EVENT_KEYS_MAP = {
"eventUid": "Event ID",
"eventType": "Event Type",
"eventDate": "Event Date",
"legalHoldUid": "Legal Hold ID",
"actorUsername": "Actor Username",
"custodianUsername": "Custodian Username",
}
LEGAL_HOLD_KEYWORD = "legal hold events"
LEGAL_HOLD_EVENT_TYPES = [
"MembershipCreated",
"MembershipReactivated",
"MembershipDeactivated",
"HoldCreated",
"HoldDeactivated",
"HoldReactivated",
"Restore",
]
BEGIN_DATE_DICT = set_begin_default_dict(LEGAL_HOLD_KEYWORD)
END_DATE_DICT = set_end_default_dict(LEGAL_HOLD_KEYWORD)


@click.group(cls=OrderedGroup)
@sdk_options(hidden=True)
def legal_hold(state):
"""Add and remove custodians from legal hold matters."""
pass


matter_id_option = click.option(
"-m",
"--matter-id",
required=True,
type=str,
help="Identification number of the legal hold matter the custodian will be added to.",
)
def matter_id_option(required, help):
return click.option("-m", "--matter-id", required=required, type=str, help=help)


user_id_option = click.option(
"-u",
"--username",
Expand All @@ -51,7 +69,10 @@ def legal_hold(state):


@legal_hold.command()
@matter_id_option
@matter_id_option(
True,
"Identification number of the legal hold matter the custodian will be added to.",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@annie-payseur When you have chance, could you help review these option help texts? I will tag you in spots.

)
@user_id_option
@sdk_options()
def add_user(state, matter_id, username):
Expand All @@ -60,7 +81,10 @@ def add_user(state, matter_id, username):


@legal_hold.command()
@matter_id_option
@matter_id_option(
True,
"Identification number of the legal hold matter the custodian will be removed from.",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thanks!

)
@user_id_option
@sdk_options()
def remove_user(state, matter_id, username):
Expand Down Expand Up @@ -124,6 +148,30 @@ def show(state, matter_id, include_inactive=False, include_policy=False):
echo("")


@legal_hold.command()
@matter_id_option(False, "Filter results by legal hold UID.")
@click.option(
"--event-type",
type=click.Choice(LEGAL_HOLD_EVENT_TYPES),
help="Filter results by event types.",
)
@click.option("--begin", **BEGIN_DATE_DICT)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should use the begin_option defined in code42cli.options
There will be several advantages to doing this.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same with end_option

Copy link
Contributor Author

@maddie-vargo maddie-vargo Apr 5, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, I avoided this option initially because the begin option requires a cursor initialized for state, which I didn't think was necessary. I've added a LegalHoldEvents cursor to the cursor_store.py to make it work, but not sure if that's appropriate.

Also, the begin_option is required, which is should not be.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we should implement the --use-checkpoint option for this command and probably also make a send-to. As I could see customers wanting to be able to automate sending this data into a SIEM.

Copy link
Contributor

@antazoey antazoey Apr 6, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh! @maddie-vargo Sorry, I did forget about that when I made my initial comments.

I do like the idea from @timabrmsn, supporting checkingpointing would be a nice feature. We can always add it later though

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@unparalleled-js would it be possible to use the begin_option but somehow designate it as optional? I left the original options in there for now.

I left checkpoint'ing off for now.

@click.option("--end", **END_DATE_DICT)
@format_option
@sdk_options()
def search_events(state, matter_id, event_type, begin, end, format):
"""Tools for getting legal hold event data."""
formatter = OutputFormatter(format, _EVENT_KEYS_MAP)
events = _get_all_events(state.sdk, matter_id, begin, end)
if event_type:
events = [event for event in events if event["eventType"] == event_type]
if len(events) > 10:
output = formatter.get_formatted_output(events)
click.echo_via_pager(output)
else:
formatter.echo_formatted_list(events)


@legal_hold.group(cls=OrderedGroup)
@sdk_options(hidden=True)
def bulk(state):
Expand Down Expand Up @@ -230,6 +278,14 @@ def _get_all_active_matters(sdk):
return matters


def _get_all_events(sdk, legal_hold_uid, begin_date, end_date):
events_generator = sdk.legalhold.get_all_events(
legal_hold_uid, begin_date, end_date
)
events = [event for page in events_generator for event in page["legalHoldEvents"]]
return events


def _print_matter_members(username_list, member_type="active"):
if username_list:
echo("\n{} matter members:\n".format(member_type.capitalize()))
Expand Down
94 changes: 93 additions & 1 deletion tests/cmds/test_legal_hold.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
import datetime

import pytest
from py42.exceptions import Py42BadRequestError
from py42.response import Py42Response
Expand All @@ -6,9 +8,9 @@

from code42cli import PRODUCT_NAME
from code42cli.cmds.legal_hold import _check_matter_is_accessible
from code42cli.date_helper import convert_datetime_to_timestamp
from code42cli.main import cli


_NAMESPACE = "{}.cmds.legal_hold".format(PRODUCT_NAME)
TEST_MATTER_ID = "99999"
TEST_LEGAL_HOLD_MEMBERSHIP_UID = "88888"
Expand All @@ -18,6 +20,8 @@
INACTIVE_TEST_USERNAME = "[email protected]"
INACTIVE_TEST_USER_ID = "54321"
TEST_POLICY_UID = "66666"
_CREATE_EVENT_ID = "564564654566"
_MEMBERSHIP_EVENT_ID = "74533457745"
TEST_PRESERVATION_POLICY_UID = "1010101010"
MATTER_RESPONSE = """
{
Expand Down Expand Up @@ -169,6 +173,42 @@
]
}
"""
TEST_EVENT_PAGE = {
"legalHoldEvents": [
{
"eventUid": "564564654566",
"eventType": "HoldCreated",
"eventDate": "2015-05-16T15:07:44.820Z",
"legalHoldUid": "88888",
"actorUserUid": "12345",
"actorUsername": "[email protected]",
"actorFirstName": "john",
"actorLastName": "doe",
"actorUserExtRef": None,
"actorEmail": "[email protected]",
},
{
"eventUid": "74533457745",
"eventType": "MembershipCreated",
"eventDate": "2019-05-17T15:07:44.820Z",
"legalHoldUid": "88888",
"legalHoldMembershipUid": "645576514441664433",
"custodianUserUid": "12345",
"custodianUsername": "[email protected]",
"custodianFirstName": "kim",
"custodianLastName": "jones",
"custodianUserExtRef": None,
"custodianEmail": "[email protected]",
"actorUserUid": "1234512345",
"actorUsername": "[email protected]",
"actorFirstName": "john",
"actorLastName": "doe",
"actorUserExtRef": None,
"actorEmail": "[email protected]",
},
]
}
EMPTY_EVENTS_RESPONSE = """{"legalHoldEvents": []}"""
EMPTY_MATTERS_RESPONSE = """{"legalHolds": []}"""
ALL_MATTERS_RESPONSE = """{{"legalHolds": [{}]}}""".format(MATTER_RESPONSE)
LEGAL_HOLD_COMMAND = "legal-hold"
Expand Down Expand Up @@ -212,6 +252,15 @@ def active_and_inactive_legal_hold_memberships_response(mocker):
return [_create_py42_response(mocker, ALL_ACTIVE_AND_INACTIVE_CUSTODIANS_RESPONSE)]


@pytest.fixture
def empty_events_response(mocker):
return _create_py42_response(mocker, EMPTY_EVENTS_RESPONSE)


def events_list_generator():
yield TEST_EVENT_PAGE


@pytest.fixture
def get_user_id_success(cli_state):
cli_state.sdk.users.get_by_username.return_value = {
Expand Down Expand Up @@ -246,6 +295,11 @@ def check_matter_accessible_failure(cli_state, custom_error):
)


@pytest.fixture
def get_all_events_success(cli_state):
cli_state.sdk.legalhold.get_all_events.return_value = events_list_generator()


@pytest.fixture
def user_already_added_response(mocker):
mock_response = mocker.MagicMock(spec=Response)
Expand Down Expand Up @@ -575,6 +629,44 @@ def test_list_with_csv_format_returns_no_response_when_response_is_empty(
assert "Matter ID,Name,Description,Creator,Creation Date" not in result.output


def test_search_events_shows_events_that_respect_type_filters(
runner, cli_state, get_all_events_success
):

result = runner.invoke(
cli,
["legal-hold", "search-events", "--event-type", "HoldCreated"],
obj=cli_state,
)

assert _CREATE_EVENT_ID in result.output
assert _MEMBERSHIP_EVENT_ID not in result.output


def test_search_events_with_csv_returns_no_events_when_response_is_empty(
runner, cli_state, get_all_events_success, empty_events_response
):
cli_state.sdk.legalhold.get_all_events.return_value = empty_events_response
result = runner.invoke(cli, ["legal-hold", "events", "-f", "csv"], obj=cli_state)

assert (
"actorEmail,actorUsername,actorLastName,actorUserUid,actorUserExtRef"
not in result.output
)


def test_search_events_is_called_with_expected_begin_timestamp(runner, cli_state):
expected_timestamp = convert_datetime_to_timestamp(
datetime.datetime.strptime("2017-01-01", "%Y-%m-%d")
)
command = ["legal-hold", "search-events", "--begin", "2017-01-01T00:00:00"]
runner.invoke(cli, command, obj=cli_state)

cli_state.sdk.legalhold.get_all_events.assert_called_once_with(
None, expected_timestamp, None
)


Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we get test(s) for begin_date / end_date validation?
It could be to just assert the values were passed into py42 correctly.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll work on this. I had tried this initially, but had trouble getting a test to recognize the date inputs in the runner.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a test titled test_search_events_is_called_with_expected_begin_timestamp

@pytest.mark.parametrize(
"command, error_msg",
[
Expand Down