Skip to content

Commit dd0cec7

Browse files
authored
Merge pull request #317 from adobe-apiplatform/v2
prepare for build of v2.3rc3
2 parents 74fa212 + a96e560 commit dd0cec7

File tree

246 files changed

+10059
-32
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

246 files changed

+10059
-32
lines changed

README.md

+12-17
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,6 @@ The [User Sync Documentation](https://adobe-apiplatform.github.io/user-sync.py/)
1919
- [User Manual](https://adobe-apiplatform.github.io/user-sync.py/en/user-manual/)
2020
- [Step-by-Step Setup](https://adobe-apiplatform.github.io/user-sync.py/en/success-guide/)
2121

22-
2322
## System Requirements
2423

2524
To run User Sync, you must have an up-to-date 64-bit Python installed on your system, either Python 2.7 or Python 3.4+. In addition you must have User Management API Credentials for your organization (see [the official documentation](https://www.adobe.io/products/usermanagement/docs/gettingstarted))
@@ -66,24 +65,20 @@ We pre-package releases on Ubuntu, so the advice here is definitely accurate for
6665
sudo apt-get install build-essential
6766
```
6867
* Make sure you use the system package manager to install the following packages (and their dependencies):
69-
* python-dev
70-
* python-pip (server variants often don't have it pre-installed)
71-
* python-virtualenv (ditto)
68+
* python-dev (or python3-dev if you are doing python3 builds)
69+
* python-pip (not needed for python3)
70+
* python-virtualenv (not needed for python3)
7271
* pkg-config
73-
* python-devel
74-
* python-pip (see above)
75-
* python-virtualenv
76-
* pkgconfig
77-
* python-dbus
7872
* libssl-dev
7973
* libldap2-dev
8074
* libsasl2-dev
8175
* libdbus-glib-1-dev
8276
* libffi-dev
8377
* For convenience, you can copy and paste this command:
8478
```bash
85-
sudo apt-get install python-dev python-pip python-virtualenv pkg-config python-dbus libssl-dev libldap2-dev libsasl2-dev libdbus-glib-1-dev libffi-dev
79+
sudo apt-get install python-dev python-pip python-virtualenv pkg-config libssl-dev libldap2-dev libsasl2-dev libdbus-glib-1-dev libffi-dev
8680
```
81+
* You don't need the python-dbus package to _build_ user-sync, but you will need it to run user-sync if you use the dbus secure store for your credentials.
8782
8883
### CentOS and other RedHat variants
8984
@@ -101,19 +96,19 @@ We pre-package releases on CentOS, so the advice here is definitely accurate for
10196
sudo yum install epel-release
10297
```
10398
* Make sure you use the system package manager to install the following packages (and their dependencies):
104-
* python-devel
105-
* python-pip (see above)
106-
* python-virtualenv
99+
* python-devel (or python3-devel, if you are doing python3 builds)
100+
* python-pip (not needed for python3)
101+
* python-virtualenv (not needed for python3)
107102
* pkgconfig
108-
* python-dbus
109103
* openssl-devel
110104
* openldap-devel (includes sasl)
111105
* dbus-glib-devel
112106
* libffi-devel
113107
* For convenience, you can copy and paste this command:
114108
```bash
115-
sudo yum install python-devel python-pip python-virtualenv pkgconfig python-dbus openssl-devel openldap-devel dbus-glib-devel libffi-devel
109+
sudo yum install python-devel python-pip python-virtualenv pkgconfig openssl-devel openldap-devel dbus-glib-devel libffi-devel
116110
```
111+
* You don't need the python-dbus package to _build_ user-sync, but you will need it to run user-sync if you use the dbus secure store for your credentials.
117112
118113
### Mac OS X
119114
@@ -148,8 +143,8 @@ In general, regardless of how you get your Python, you will need:
148143
149144
Windows is the trickiest platform because you need a command line development environment and package manager, and many of the dependencies don't have Windows builds available on PyPI. So rather than building from Windows on scratch, we recommend the following procedure:
150145

151-
* Install [Cygwin](https://www.cygwin.com/) to get a bash command-line, together with basic tools such as `git` and `make`.
152-
* Use the [python.org](https://python.org) installers for the desired version of Python.
146+
* Install [Cygwin](https://www.cygwin.com/) to get a bash command-line, together with basic tools such as `git` and `make`. You will need to specify in the Cygwin installer that you want `git` and `make` installed, because they are not defaults. Alternatively, you can install the entire Cygwin suite of development tools, but that's probably more than you need.
147+
* Use the [python.org](https://python.org) *64-bit* installers for the desired version of Python.
153148
* Install the [latest Visual C++ Redistributable Libraries](https://support.microsoft.com/en-us/help/2977003/the-latest-supported-visual-c-downloads) from Microsoft (because python3 will need these to run in a virtual environment).
154149
* For the User Sync dependencies that don't have Windows 64-bit wheels on [PyPI](https://pypi.python.org/), get builds from [Christoph Guelke's excellent site](http://www.lfd.uci.edu/~gohlke/pythonlibs/). We have stashed the ones needed for the current release in the `external` directory, and that's where the `Makefile` looks for them, so if you go fetch your own be sure to put them in that same directory.
155150

RELEASE_NOTES.md

+11-3
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Release Notes for User Sync Tool Version 2.3
22

3-
These notes apply to v2.3rc2 of 2017-12-03.
3+
These notes apply to v2.3rc3 of 2017-12-10.
44

55
## New Features
66

@@ -18,14 +18,22 @@ There is a new command-line argument `--connector` for specifying whether to get
1818

1919
[#306](https://github.com/adobe-apiplatform/user-sync.py/issues/306) v2.2.2 crashes if country code not specified.
2020

21+
[#314](https://github.com/adobe-apiplatform/user-sync.py/issues/314) invocation_defaults section should be optional.
22+
23+
[#315](https://github.com/adobe-apiplatform/user-sync.py/issues/315) Can't specify --user-filter or other string-valued args.
24+
2125
## Compatibility with Prior Versions
2226

2327
All configuration and command-line arguments accepted in prior releases work in this release. The `--users file` argument is still accepted, and is equivalent to (although more limited than) specifying `--connector csv`.
2428

2529
## Known Issues
2630

27-
Because the release on Windows is built with a pre-compiled version of pyldap, we have to specify a specific version to be used in each release (see the setup.py file for the specific version). This may not always be the latest version.
28-
2931
On the Win64 platform, there are very long pathnames embedded in the released build artifact `user-sync.pex`, which will cause problems unless you are on Windows 10 and are either running Python 3.6 or have enabled long pathnames system-wide (as described in this [Microsoft Dev Center article](https://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx)). To work around this issue on older platforms, set the `PEX_ROOT` environment variable (as described [in the docs here](https://adobe-apiplatform.github.io/user-sync.py/en/user-manual/setup_and_installation.html)) to be a very short path (e.g., `set PEX_ROOT=C:\pex`).
3032

3133
Each release on each platform is built with a specific version of Python. Typically this is the latest available for that platform (from the OS vendor, if they provide one, from [python.org](http://python.org) otherwise). In general, and especially on Windows, you should use the same Python to run User Sync as it was built with.
34+
35+
## Additional Build Information
36+
37+
User Sync is now built with PyLDAP 2.4.45.
38+
39+
User Sync is now built with umapi_client 2.10. This allows mocking the UMAPI connection for use with a test framework. See the test_framework directory in the source tree for more details.

docs/en/user-manual/command_parameters.md

+5-5
Original file line numberDiff line numberDiff line change
@@ -29,12 +29,12 @@ specific behavior in various situations.
2929
|------------------------------|------------------|
3030
| `-h`<br />`--help` | Show a help message and exit. |
3131
| `-v`<br />`--version` | Show program's version number and exit. |
32-
| `-t`<br />`--test-mode`<br />`-T`<br />`--no-test-mode` | Specifying `-t` or `--test-mode` causes User Sync to run API action calls in _test mode_ (that is, the server does syntax/semantics checking on the calls but does not execute any requested changes). User Sync still logs all actions, making this very useful for previewing what a run would have requested. Starting with version 2.3, specifying `-T` or `--no-test-mode` can be specified to turn off test mode. |
32+
| `-t`<br />`--test-mode`<br />`-T`<br />`--no-test-mode` | Specifying `-t` or `--test-mode` causes User Sync to run API action calls in _test mode_ (that is, the server does syntax/semantics checking on the calls but does not execute any requested changes). User Sync still logs all actions, making this very useful for previewing what a run would have requested. Starting with version 2.3, specifying `-T` or `--no-test-mode` can be specified to turn off test mode, which is useful to override a default value specified in the configuration file (see below). |
3333
| `-c` _filename_<br />`--config-filename` _filename_ | The complete path to the main configuration file, absolute or relative to the working folder. Default filename is "user-sync-config.yml" |
3434
| `--users` `all`<br />`--users` `file` _input_path_<br />`--users` `group` _grp1,grp2_<br />`--users` `mapped` | Specify the users to be selected for sync. The default is `all` meaning all users found in the directory. Specifying `file` means to take input user specifications from the CSV file named by the argument. Specifying `group` interprets the argument as a comma-separated list of groups in the enterprise directory, and only users in those groups are selected. Specifying `mapped` is the same as specifying `group` with all groups listed in the group mapping in the configuration file. This is a very common case where just the users in mapped groups are to be synced.|
3535
| `--user-filter` _regex\_pattern_ | Limit the set of users that are examined for syncing to those matching a pattern specified with a regular expression. See the [Python regular expression documentation](https://docs.python.org/2/library/re.html) or [for Python 3](https://docs.python.org/3/library/re.html) for information on constructing regular expressions in Python. The user name must completely match the regular expression.|
36-
| `--update-user-info`<br />`--no-update-user-info` | Specifying `--update-user-info` synchronizes user information. If the information differs between the enterprise directory side and the Adobe side, the Adobe side is updated to match. This includes the firstname and lastname fields. Starting with User Sync 2.3, `--no-update-user-info` can be specified to prevent this synchronization. |
37-
| `--process-groups`<br />`--no-process-groups` | Specifying `--process-groups` causes synchronization of group membership information: if the membership in mapped groups differs between the enterprise directory side and the Adobe side, the group membership is updated on the Adobe side to match. This includes removal of group membership for Adobe users not listed in the directory side (unless the `--adobe-only-user-action exclude` option is also selected). Starting with User Sync 2.3, `--no-process-groups` can be specified to prevent this synchronization. |
36+
| `--update-user-info`<br />`--no-update-user-info` | Specifying `--update-user-info` synchronizes user information. If the information differs between the enterprise directory side and the Adobe side, the Adobe side is updated to match. This includes the firstname and lastname fields. Starting with User Sync 2.3, `--no-update-user-info` can be specified to prevent this synchronization, which is useful to override a default value specified in the configuration file (see below). |
37+
| `--process-groups`<br />`--no-process-groups` | Specifying `--process-groups` causes synchronization of group membership information: if the membership in mapped groups differs between the enterprise directory side and the Adobe side, the group membership is updated on the Adobe side to match. This includes removal of group membership for Adobe users not listed in the directory side (unless the `--adobe-only-user-action exclude` option is also selected). Starting with User Sync 2.3, `--no-process-groups` can be specified to prevent this synchronization, which is useful to override a default value specified in the configuration file (see below). |
3838
| `--adobe-only-user-action preserve`<br />`--adobe-only-user-action remove-adobe-groups`<br />`--adobe-only-user-action remove`<br />`--adobe-only-user-action delete`<br /><br/>`--adobe-only-user-action write-file`&nbsp;filename<br/><br/>`--adobe-only-user-action exclude` | When supplied, if user accounts are found on the Adobe side that are not in the directory, take the indicated action. <br/><br/>`preserve`: no action concerning account deletion is taken. This is the default. There may still be group membership changes if the `--process-groups` option was specified.<br/><br/>`remove-adobe-groups`: The account is removed from user groups and product configurations, freeing any licenses it held, but is left as an active account in the organization.<br><br/>`remove`: In addition to remove-adobe-groups, the account is also removed from the organization, but the user account, with its associated assets, is left in the domain and can be re-added to the organization if desired.<br/><br/>`delete`: In addition to the action for remove, the account is deleted if its domain is owned by the organization.<br/><br/>`write-file`: No action concerning account deletion is taken. The list of user accounts present on the Adobe side but not in the directory is written to the file indicated. You can then pass this file to the `--adobe-only-user-list` argument in a subsequent run. There may still be group membership changes if the `--process-groups` option was specified.<br/><br/>`exclude`: No update of any kind is applied to users found only on the Adobe side. This is used when doing updates of specific users via a file (`--users file f`) where only users needing explicit updates are listed in the file and all other users should be left alone.<br/><br>Only permitted actions will be applied. Accounts of type adobeID are owned by the user so the delete action will do the equivalent of remove. The same is true of Adobe accounts owned by other organizations. |
3939
| `--adobe-only-user-list` _filename_ | Specifies a file from which a list of users will be read. This list is used as the definitive list of "Adobe only" user accounts to be acted upon. One of the `--adobe-only-user-action` directives must also be specified and its action will be applied to user accounts in the list. The `--users` option is disallowed if this option is present: only account removal actions can be processed. |
4040
| `--config-file-encoding` _encoding_name_ | Optional. Specifies the character encoding for the contents of the configuration files themselves. This includes the main configuration file, "user-sync-config.yml" as well as other configuration files it may reference. Default is `utf8` for User Sync 2.2 and later and `ascii` for earlier versions.<br />Character encoding in the user source data (whether csv or ldap) is declared by the connector configurations, and that encoding can be different than the encoding used for the configuration files (e.g., you could have a latin-1 configuration file but a CSV source file that uses utf-8 encoding).<br />The available encodings are dependent on the Python version used; see the documentation [here for Python 2.7](https://docs.python.org/2.7/library/codecs.html#standard-encodings) or [here for Python 3.6](https://docs.python.org/3.6/library/codecs.html#standard-encodings) for more information. |
@@ -59,8 +59,8 @@ invocation_defaults:
5959
6060
As you can see from the example:
6161
62-
* Each command-line parameter can be specified using a configuration option whose name is the same, but with hyphens replaced by underscores (e.g., the command-line parameter `process-groups` is specified by the configuration option `process_groups`). Only those command-line parameters which control the loading of configuration files (`--config-filename`, `--config-file-encoding`) cannot be specified as configuration options, because they take effect _before_ the configuration file is loaded.
63-
* Command-line parameters that take zero arguments because they specify Yes/No (boolean) options (`--test-mode`, `--process-groups`, `--update-user-info`) can be specified as a having a value of Yes/True or No/False (case-insensitive), since YAML syntax treats these all as booleans. The example above contains configuration options that use both formats.
62+
* Each command-line parameter can be specified using a configuration option whose name is the same, but with hyphens replaced by underscores; for example, the command-line parameter `process-groups` is specified by the configuration option `process_groups`. (YAML doesn't allow hyphens in option names.) Only those command-line parameters which control the loading of configuration files (`--config-filename`, `--config-file-encoding`) cannot be specified as configuration options, because they take effect _before_ the configuration file is loaded.
63+
* Command-line parameters that take zero arguments because they specify Yes/No (boolean) options (`--test-mode`, `--process-groups`, `--update-user-info`) can be specified as having a value of Yes/True or No/False (case-insensitive), since YAML syntax treats these all as booleans. The example above contains configuration options that use both formats.
6464
* Command-line parameters that are being given a single string argument should have the desired string specified as their value, as shown for the `users` option above.
6565
* Command-line parameters that are being given multiple string arguments should have a list of the desired strings specified as their value. YAML supports two options for specifying lists of values, one of which (single-line) is shown for the `connector` option above and the other of which (multi-line) is shown for the `adobe_only_user_action` above. (A list containing a single string is treated the same as a single string argument.)
6666

-1.05 MB
Binary file not shown.
-1.05 MB
Binary file not shown.
-1.05 MB
Binary file not shown.
-1.05 MB
Binary file not shown.
1.05 MB
Binary file not shown.
1.05 MB
Binary file not shown.
1.05 MB
Binary file not shown.
1.05 MB
Binary file not shown.

setup.py

+2-2
Original file line numberDiff line numberDiff line change
@@ -49,10 +49,10 @@
4949
'okta==0.0.3.1',
5050
'psutil',
5151
'pycryptodome',
52-
'pyldap==2.4.37',
52+
'pyldap==2.4.45',
5353
'PyYAML',
5454
'six',
55-
'umapi-client>=2.9',
55+
'umapi-client>=2.10',
5656
],
5757
extras_require={
5858
':sys_platform=="linux" or sys_platform=="linux2"':[

test_framework/Makefile

+21
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
RM := rm -rf
2+
3+
ifeq ($(OS),Windows_NT)
4+
output_file_extension = .pex
5+
rm_path := $(shell python -c "import distutils.spawn; print(distutils.spawn.find_executable('rm'))")
6+
ifeq ($(rm_path),None)
7+
RM := rmdir /S /Q
8+
endif
9+
endif
10+
11+
output_dir = dist
12+
output_filename = user-sync-test
13+
14+
pex:
15+
pip install --upgrade pip
16+
pip install pex requests wheel
17+
-$(RM) $(output_dir)
18+
pex \
19+
-v -o $(output_dir)/$(output_filename)$(output_file_extension) -m user_sync_test.app \
20+
--disable-cache \
21+
--not-zip-safe .

0 commit comments

Comments
 (0)