User update fix

[adorton-adobe]

Summary

This is to fix a regression found in v2.8.0 which prevents the sync tool from updating a user's email address. This was likely caused by the introduction of the email address to the static user key.

The fix I propose is to stop indexing user information using this static composite key. It never worked super well because of how the UMAPI and admin console identify users by email address under most circumstances but also require username in others.

I've created a system that indexes user info by multiple keys, but only one key is required to identify a user when looking it up in the index. This is implemented in a new class in the UMAPI engine file called MultiIndex.

More information can be found in the class's docstring:

    This data structure replaces the old convention of caching users in a
    dictionary indexed by a static composite key. The MultiIndex structure
    consists of a simple list (self.data) consisting of one or more dictionaries
    that follow a regular structure (e.g. a list of directory users or UMAPI
    users).

    This list is indexed by one or more keys. Each key should point to one
    record in self.data.

    When a record is fetched from the index, a record is returned if at least
    one key matches a record. This allows partial matches - i.e. when retrieving
    information for a user where the email address matches a record but the
    username does not (or vice versa).

    Example:

    >>> data = [{"key1": "foo", "key2": "bar", "other": "data"}]
    >>> mi = MultiIndex(data=data, key_names=["key1", "key2"])
    >>> mi.get(key1="foo", key2="bar")
    {"key1": "foo", "key2": "bar", "other": "data"}
    >>> mi.get(key1="foo", key2="invalid")
    {"key1": "foo", "key2": "bar", "other": "data"}
    >>> mi.get(key1="invalid", key2="invalid")
    None

    MultiIndex supports the addition of new individual records and
    the ability to update existing records. It does not support
    deletion because that would require a full reindex for each
    deletion.

Testing Steps

I've tested this with one UMAPI target, primary + secondary and secondary only (which produces the expected UMAPI errors). I've also tested the update with Adobe-only CSV files (read/write).

Fixes #810

[Luci2015]

new line 960 checks:
if directory_user is None:
at this stage we know that an umapi user_key was resolved from the list of directory_users.
To solve the fact that the username can't be updated in this version we could do at this stage the check if:
normalize_string(umapi_user['username']) != normalize_string(directory_user['username'])
and log a warning under the self.logger.debug("Adobe user matched on customer side: %s", user_key)
self.logger.warning('Username differences found: %s / %s',umapi_user['username'], directory_user['username'])
This bit of code can be removed after the username updates will be possible. Sounds OK?

[adorton-adobe]

Since we agreed to plan to release the username update feature in the same release as this fix, I don't think we need the temporary warning.

Here's the feature branch for username update: https://github.com/adobe-apiplatform/user-sync.py/tree/feature/username-update

Once I get a chance to test it more I'll work on the configuration feature to select which fields to update.

[adorton-adobe]

@Luci2015 - are you still testing? I hoped to have this merged a week ago. What can I help you with to get this finished up?

[Luci2015]

works with csv input