User Sync Tool v2.4.2

Release Notes for User Sync Tool Version 2.4.2

These notes apply to v2.4.2 of 2019-04-03.

New Features

#368 --adobe-users command line option to whitelist specific Adobe groups for sync. Invocation option also added.

Bug Fixes

#438 Log a better error if directory connector not correctly configured

#450 Fix regression introduced in 2.4

Compatibility with Prior Versions

All configuration and command-line arguments accepted in prior releases work in this release.

Known Issues

Python 3.7 is not supported at this time. See #376.

Additional Build Information

#448 Test system overhaul

v2.4.2 - 2019-04-03

User Sync Tool v2.4

Release Notes for User Sync Tool Version 2.4

These notes apply to v2.4 of 2019-01-28.

New Features

#398 max_adobe_only_users can be set to a percentage of total users.

#323 Two-step group lookup. Certain LDAP systems do not support group membership queries. This feature adds new config options to connector-ldap.yml to enable a two-step LDAP user lookup workflow.

#385 Support for users that have a different email-type username and email address. Users of this type are synced by specifying both a user_username_format and user_email_format in connector-ldap.yml. The username field must contain only email-type usernames. Users with alphanumeric usernames will not be synced. See the "Advanced Configuration" section of the User Manual for more information.

#339 Dynamic mapping of additional groups and automatic group creation. Introduces an optional config option to identify additional groups that a user directly belongs to. Additional groups are matched with a list of one or more regular expressions. These groups can be dynamically mapped to Adobe groups using regular expression substitution strings. In addition, Adobe groups targeted by this method, as well as the standard mapping or extension config, can be automatically created by the sync tool. New groups are created as user groups. See the documentation for more details.

#405 Additional enhancements and fixes to group sync

• Log "additional group" rule mapping
• Don't allow multiple source rules to map to same target group
• Catch regex substitution errors
• Remove some superfluous and confusing checks
• Secondary org support

Bug Fixes

#379 --user-filter and invocation default

#381 Invocation Defaults doesn't work for "--users file"

• Not actually a bug, but user-sync-config.yml was updated to clarify how to specify user input file in invocation_defaults

#396 LDAP error when running user-sync-v2.4rc1-win64-py2715

Documentation Updates

#403 Add documentation for Azure AD / UST

#426 Ergonomic tweaks to template configs

• Removed Number from the sample template
• Connector-umapi.yml
  • set private key path to just private.key
• Connector-ldap.yml
  • set page size to 1000 (Active Directory Default)
  • user_username_format example to just {sAMAccountName}
• User-Sync-Config.yml
  • Default to FederatedID
  • Tweaked the example to match with current use case
  • Enable Logging by Default
  • Default Invocation - Set to Process-group and Users Mapped to avoid accidentally directory dump to Admin console.

Compatibility with Prior Versions

All configuration and command-line arguments accepted in prior releases work in this release.

Known Issues

Python 3.7 is not supported at this time. See #376.

#384 UMAPI returns truncated group list for users assigned to a large amount of groups. This doesn't prevent the new additional group functionality from working, but it does result in unnecessary API calls to assign users to groups they already may belong to.

Additional Build Information

User Sync is now built with umapi_client 2.12, which supports the following new features

• Add new user groups
• Update existing user groups
• Delete user groups
• Create users with different email-type usernames and email addresses

v2.4 - 2019-01-28

User Sync Tool v2.4rc3

Third Release Candidate of UST 2.4

Unstable Warning

This is an unstable pre-release intended for testing and feature integration. If you don't need any of the new features or bug fixes listed here, please use the latest stable release.

New Features

#385 Support for users that have a different email-type username and email address. Users of this type are synced by specifying both a user_username_format and user_email_format in connector-ldap.yml. The username field must contain only email-type usernames. Users with alphanumeric usernames will not be synced. See the "Advanced Configuration" section of the User Manual for more information.

#339 Dynamic mapping of additional groups and automatic group creation. Introduces an optional config option to identify additional groups that a user directly belongs to. Additional groups are matched with a list of one or more regular expressions. These groups can be dynamically mapped to Adobe groups using regular expression substitution strings. In addition, Adobe groups targeted by this method, as well as the standard mapping or extension config, can be automatically created by the sync tool. New groups are created as user groups. See the documentation for more details.

#405 Additional enhancements and fixes to group sync

• Log "additional group" rule mapping
• Don't allow multiple source rules to map to same target group
• Catch regex substitution errors
• Remove some superfluous and confusing checks
• Secondary org support

Bug Fixes

#379 --user-filter and invocation default

#381 Invocation Defaults doesn't work for "--users file"

• Not actually a bug, but 1 user-sync-config.yml was updated to clarify how to specify user input file in invocation_defaults

#396 LDAP error when running user-sync-v2.4rc1-win64-py2715

Documentation Updates

#403 Add documentation for Azure AD / UST

#426 Ergonomic tweaks to template configs

• Removed Number from the sample template
• Connector-umapi.yml
  • set private key path to just private.key
• Connector-ldap.yml
  • set page size to 1000 (Active Directory Default)
  • user_username_format example to just {sAMAccountName}
• User-Sync-Config.yml
  • Default to FederatedID
  • Tweaked the example to match with current use case
  • Enable Logging by Default
  • Default Invocation - Set to Process-group and Users Mapped to avoid accidentally directory dump to Admin console.

Compatibility with Prior Versions

All configuration and command-line arguments accepted in prior releases work in this release.

Known Issues

Python 3.7 is not supported at this time. See #376.

#384 UMAPI returns truncated group list for users assigned to a large amount of groups. This doesn't prevent the new additional group functionality from working, but it does result in unnecessary API calls to assign users to groups they already may belong to.

Additional Build Information

User Sync is now built with umapi_client 2.12, which supports the following new features

• Add new user groups
• Update existing user groups
• Delete user groups
• Create users with different email-type usernames and email addresses

User Sync Tool v2.4rc2

Second Release Candidate of UST 2.4

Unstable Warning

This is an unstable pre-release intended for testing and feature integration. If you don't need any of the new features or bug fixes listed here, please use the latest stable release.

New Features

#339 Dynamic mapping of additional groups and automatic group creation. Introduces an optional config option to identify additional groups that a user directly belongs to. Additional groups are matched with a list of one or more regular expressions. These groups can be dynamically mapped to Adobe groups using regular expression substitution strings. In addition, Adobe groups targeted by this method, as well as the standard mapping or extension config, can be automatically created by the sync tool. New groups are created as user groups. See the documentation for more details.

#405 Additional enhancements and fixes to group sync

• Log "additional group" rule mapping
• Don't allow multiple source rules to map to same target group
• Catch regex substitution errors
• Remove some superfluous and confusing checks
• Secondary org support

Bug Fixes

#379 --user-filter and invocation default

#381 Invocation Defaults doesn't work for "--users file"

• Not actually a bug, but 1 user-sync-config.yml was updated to clarify how to specify user input file in invocation_defaults

#396 LDAP error when running user-sync-v2.4rc1-win64-py2715

Documentation Updates

#403 Add documentation for Azure AD / UST

#426 Ergonomic tweaks to template configs

• Removed Number from the sample template
• Connector-umapi.yml
  • set private key path to just private.key
• Connector-ldap.yml
  • set page size to 1000 (Active Directory Default)
  • user_username_format example to just {sAMAccountName}
• User-Sync-Config.yml
  • Default to FederatedID
  • Tweaked the example to match with current use case
  • Enable Logging by Default
  • Default Invocation - Set to Process-group and Users Mapped to avoid accidentally directory dump to Admin console.

Compatibility with Prior Versions

All configuration and command-line arguments accepted in prior releases work in this release.

Known Issues

Python 3.7 is not supported at this time. See #376.

#384 UMAPI returns truncated group list for users assigned to a large amount of groups. This doesn't prevent the new additional group functionality from working, but it does result in unnecessary API calls to assign users to groups they already may belong to.

Additional Build Information

User Sync is now built with umapi_client 2.11, which can add, update, and delete user groups.

User Sync Tool v2.4rc1

First Release Candidate of UST 2.4

Unstable Warning

This is an unstable pre-release intended for testing and feature integration. If you don't need any of the new features or bug fixes listed here, please use the latest stable release.

New Features

#339 Dynamic mapping of additional groups and automatic group creation. Introduces an optional config option to identify additional groups that a user directly belongs to. Additional groups are matched with a list of one or more regular expressions. These groups can be dynamically mapped to Adobe groups using regular expression substitution strings. In addition, Adobe groups targeted by this method, as well as the standard mapping or extension config, can be automatically created by the sync tool. New groups are created as user groups. See the documentation for more details.

Bug Fixes

#379 --user-filter and invocation default

#381 Invocation Defaults doesn't work for "--users file"

• Not actually a bug, but 1 user-sync-config.yml was updated to clarify how to specify user input file in invocation_defaults

Compatibility with Prior Versions

All configuration and command-line arguments accepted in prior releases work in this release.

Known Issues

Python 3.7 is not supported at this time.

#384 UMAPI returns truncated group list for users assigned to a large amount of groups. This doesn't prevent the new additional group functionality from working, but it does result in unnecessary API calls to assign users to groups they already may belong to.

Additional Build Information

User Sync is now built with umapi_client 2.11, which can add, update, and delete user groups.

User Sync Tool v2.3

Release Notes for User Sync Tool Version 2.3

These notes apply to v2.3 of 2018-07-31.

New Features

User Sync can now connect to Okta enterprise directories. Create an Okta configuration and use the new --connector okta command-line argument to select that connector. See the docs for details.

There is a new command-line argument --connector for specifying whether to get directory information via LDAP file, by reading a CSV file, or via the Okta connector. The default connector is ldap. For CSV users, who formerly had to specify their input source with the --users argument, this optional argument offers the chance to specify --users mapped or --users group ... (since the CSV input can be specified with --connector). See the docs for details.

#292 You can now specify the log file name as well as the log file directory in your configuration file. The name is specified by giving a Python format string which, when applied to a Python datetime value at the start of the run, produces the name of the log file. The default value of this string is backwards-compatible with prior User Sync behavior. See the docs for details.

#299 You can now use an invocation_defaults section to specify desired values for command-line arguments in the main configuration file. This can make it a lot easier to repeat runs with a stable set of arguments, even when running interactively rather than from a script. The sample main configuration file specifies the configuration parameters to use as well as the syntax for specifying values. See the docs for full details.

#322, #319 As it has been with email, you can now use formatted combinations of ldap/okta attributes for the Adobe-side first name, last name, and country. (See the sample configuration files for details.) You can also specify the country code in lower case.

Bug Fixes

#305 General issues with Okta connector.

#306 v2.2.2 crashes if country code not specified.

#308 docs are unclear about how to set PEX_ROOT.

#314 invocation_defaults section should be optional.

#315 Can't specify --user-filter or other string-valued args.

#318 Fix the README build instructions regarding dbus.

#324 Handle LDAP servers with no support for PagedResults.

#325 Adding '--process-groups' doesn't override the default.

#364 Okta decode error

#365 Using adobe-only-user-list does not work

Compatibility with Prior Versions

All configuration and command-line arguments accepted in prior releases work in this release. The --users file argument is still accepted, and is equivalent to (although more limited than) specifying --connector csv.

Known Issues

On the Win64 platform, there are very long pathnames embedded in the released build artifact user-sync.pex, which will cause problems unless you are on Windows 10 and are either running Python 3.6 or have enabled long pathnames system-wide (as described in this Microsoft Dev Center article). To work around this issue on older platforms, set the PEX_ROOT environment variable (as described in the docs here) to be a very short path (e.g., set PEX_ROOT=C:\pex).

Each release on each platform is built with a specific version of Python. Typically this is the latest available for that platform (from the OS vendor, if they provide one, from python.org otherwise). In general, and especially on Windows, you should use the same Python to run User Sync as it was built with.

Additional Build Information

User Sync is now built with PyLDAP 2.4.45.

User Sync is now built with umapi_client 2.10. This allows mocking the UMAPI connection for use with a test framework. See the test_framework directory in the source tree for more details.

v2.3 - 2018-08-01

Fourth release candidate for v2.3

These notes apply to v2.3rc4 of 2018-01-29.

New Features

User Sync can now connect to Okta enterprise directories. Create an Okta configuration and use the new --connector okta command-line argument to select that connector. See the docs for details.

There is a new command-line argument --connector for specifying whether to get directory information via LDAP file, by reading a CSV file, or via the Okta connector. The default connector is ldap. For CSV users, who formerly had to specify their input source with the --users argument, this optional argument offers the chance to specify --users mapped or --users group ... (since the CSV input can be specified with --connector). See the docs for details.

#292 You can now specify the log file name as well as the log file directory in your configuration file. The name is specified by giving a Python format string which, when applied to a Python datetime value at the start of the run, produces the name of the log file. The default value of this string is backwards-compatible with prior User Sync behavior. See the docs for details.

#299 You can now use an invocation_defaults section to specify desired values for command-line arguments in the main configuration file. This can make it a lot easier to repeat runs with a stable set of arguments, even when running interactively rather than from a script. The sample main configuration file specifies the configuration parameters to use as well as the syntax for specifying values. See the docs for full details.

#322, #319 As it has been with email, you can now use formatted combinations of ldap/okta attributes for the Adobe-side first name, last name, and country. (See the sample configuration files for details.) You can also specify the country code in lower case.

Bug Fixes

#305 General issues with Okta connector.

#306 v2.2.2 crashes if country code not specified.

#308 docs are unclear about how to set PEX_ROOT.

#314 invocation_defaults section should be optional.

#315 Can't specify --user-filter or other string-valued args.

#318 Fix the README build instructions regarding dbus.

#324 Handle LDAP servers with no support for PagedResults.

#325 Adding '--process-groups' doesn't override the default.

Compatibility with Prior Versions

All configuration and command-line arguments accepted in prior releases work in this release. The --users file argument is still accepted, and is equivalent to (although more limited than) specifying --connector csv.

Known Issues

On the Win64 platform, there are very long pathnames embedded in the released build artifact user-sync.pex, which will cause problems unless you are on Windows 10 and are either running Python 3.6 or have enabled long pathnames system-wide (as described in this Microsoft Dev Center article). To work around this issue on older platforms, set the PEX_ROOT environment variable (as described in the docs here) to be a very short path (e.g., set PEX_ROOT=C:\pex).

Each release on each platform is built with a specific version of Python. Typically this is the latest available for that platform (from the OS vendor, if they provide one, from python.org otherwise). In general, and especially on Windows, you should use the same Python to run User Sync as it was built with.

Additional Build Information

User Sync is now built with PyLDAP 2.4.45.

User Sync is now built with umapi_client 2.10. This allows mocking the UMAPI connection for use with a test framework. See the test_framework directory in the source tree for more details.

Third release candidate for v2.3

These notes apply to v2.3rc3 of 2017-12-10.

New Features

User Sync can now connect to Okta enterprise directories. Create an Okta configuration and use the new --connector okta command-line argument to select that connector. See the docs for details.

There is a new command-line argument --connector for specifying whether to get directory information via LDAP file, by reading a CSV file, or via the Okta connector. The default connector is ldap. For CSV users, who formerly had to specify their input source with the --users argument, this optional argument offers the chance to specify --users mapped or --users group ... (since the CSV input can be specified with --connector). See the docs for details.

#292 You can now specify the log file name as well as the log file directory in your configuration file. The name is specified by giving a Python format string which, when applied to a Python datetime value at the start of the run, produces the name of the log file. The default value of this string is backwards-compatible with prior User Sync behavior. See the docs for details.

#299 You can now use an invocation_defaults section to specify desired values for command-line arguments in the main configuration file. This can make it a lot easier to repeat runs with a stable set of arguments, even when running interactively rather than from a script. The sample main configuration file specifies the configuration parameters to use as well as the syntax for specifying values. See the docs for full details.

Bug Fixes

#305 General issues with Okta connector.

#306 v2.2.2 crashes if country code not specified.

#314 invocation_defaults section should be optional.

#315 Can't specify --user-filter or other string-valued args.

Compatibility with Prior Versions

All configuration and command-line arguments accepted in prior releases work in this release. The --users file argument is still accepted, and is equivalent to (although more limited than) specifying --connector csv.

Known Issues

On the Win64 platform, there are very long pathnames embedded in the released build artifact user-sync.pex, which will cause problems unless you are on Windows 10 and are either running Python 3.6 or have enabled long pathnames system-wide (as described in this Microsoft Dev Center article). To work around this issue on older platforms, set the PEX_ROOT environment variable (as described in the docs here) to be a very short path (e.g., set PEX_ROOT=C:\pex).

Each release on each platform is built with a specific version of Python. Typically this is the latest available for that platform (from the OS vendor, if they provide one, from python.org otherwise). In general, and especially on Windows, you should use the same Python to run User Sync as it was built with.

Additional Build Information

User Sync is now built with PyLDAP 2.4.45.

User Sync is now built with umapi_client 2.10. This allows mocking the UMAPI connection for use with a test framework. See the test_framework directory in the source tree for more details.

Second release candidate for v2.3

These notes apply to v2.3rc2 of 2017-12-03.

New Features

User Sync can now connect to Okta enterprise directories. Create an Okta configuration and use the new --connector okta command-line argument to select that connector. See the docs for details.

There is a new command-line argument --connector for specifying whether to get directory information via LDAP file, by reading a CSV file, or via the Okta connector. The default connector is ldap. For CSV users, who formerly had to specify their input source with the --users argument, this optional argument offers the chance to specify --users mapped or --users group ... (since the CSV input can be specified with --connector). See the docs for details.

#292 You can now specify the log file name as well as the log file directory in your configuration file. The name is specified by giving a Python format string which, when applied to a Python datetime value at the start of the run, produces the name of the log file. The default value of this string is backwards-compatible with prior User Sync behavior. See the docs for details.

#299 You can now use an invocation_defaults section to specify desired values for command-line arguments in the main configuration file. This can make it a lot easier to repeat runs with a stable set of arguments, even when running interactively rather than from a script. The sample main configuration file specifies the configuration parameters to use as well as the syntax for specifying values. See the docs for full details.

Bug Fixes

#305 General issues with Okta connector.

#306 v2.2.2 crashes if country code not specified.

Compatibility with Prior Versions

All configuration and command-line arguments accepted in prior releases work in this release. The --users file argument is still accepted, and is equivalent to (although more limited than) specifying --connector csv.

Known Issues

Because the release on Windows is built with a pre-compiled version of pyldap, we have to specify a specific version to be used in each release (see the setup.py file for the specific version). This may not always be the latest version.

On the Win64 platform, there are very long pathnames embedded in the released build artifact user-sync.pex, which will cause problems unless you are on Windows 10 and are either running Python 3.6 or have enabled long pathnames system-wide (as described in this Microsoft Dev Center article). To work around this issue on older platforms, set the PEX_ROOT environment variable (as described in the docs here) to be a very short path (e.g., set PEX_ROOT=C:\pex).

Each release on each platform is built with a specific version of Python. Typically this is the latest available for that platform (from the OS vendor, if they provide one, from python.org otherwise). In general, and especially on Windows, you should use the same Python to run User Sync as it was built with.

okta connector: v2.3rc1

These notes apply to v2.3rc1 of 2017-11-20. (There are still bugs and enhancements slated for v2.3, so there will be at least one more release candidate.)

New Features

User Sync can now connect to Okta enterprise directories. Create an Okta configuration and use the new --connector okta command-line argument to select that connector. See the docs for details.

There is a new command-line argument --connector for specifying whether to get directory information via LDAP, by reading a CSV file, or via the Okta connector. The default connector is ldap. For CSV users, who formerly had to specify their input source with the --users argument, this optional argument offers the chance to specify --users mapped or --users group ... (since the CSV input can be specified with --connector). See the docs for details.

Bug Fixes

#305 General issues with Okta connector.

#306 v2.2.2 crashes if country code not specified.

Compatibility with Prior Versions

All configuration and command-line arguments accepted in prior releases work in this release. The --users file argument is still accepted, and is equivalent to (although more limited than) specifying --connector csv.

Known Issues

Because the release on Windows is built with a pre-compiled version of pyldap, we have to specify a specific version to be used in each release (see the setup.py file for the specific version). This may not always be the latest version.

On the Win64 platform, there are very long pathnames embedded in the released build artifact user-sync.pex, which will cause problems unless you are on Windows 10 and are either running Python 3.6 or have enabled long pathnames system-wide (as described in this Microsoft Dev Center article). To work around this issue on older platforms, set the PEX_ROOT environment variable (as described in the docs here) to be a very short path (e.g., set PEX_ROOT=C:\pex).

Each release on each platform is built with a specific version of Python. Typically this is the latest available for that platform (from the OS vendor, if they provide one, from python.org otherwise). In general, and especially on Windows, you should use the same Python to run User Sync as it was built with.