From fbc3d7260c32870c2c5405517302ff1448f6ed69 Mon Sep 17 00:00:00 2001 From: Steven Silvester Date: Mon, 4 Sep 2023 13:28:18 -0500 Subject: [PATCH 01/44] DRIVERS-2616 OIDC-SASL Follow-Up --- source/auth/tests/mongodb-oidc.rst | 334 ++++++----------------------- 1 file changed, 64 insertions(+), 270 deletions(-) diff --git a/source/auth/tests/mongodb-oidc.rst b/source/auth/tests/mongodb-oidc.rst index 724841e0e0..f6496210c3 100644 --- a/source/auth/tests/mongodb-oidc.rst +++ b/source/auth/tests/mongodb-oidc.rst @@ -5,31 +5,38 @@ MongoDB OIDC Drivers MUST test the following scenarios: - ``Callback-Driven Auth`` -- ``AWS Automatic Auth`` - ``Callback Validation`` -- ``Cached Credentials`` - ``Speculative Authentication`` - ``Reauthentication`` +- ``Separate Connections Avoid Extra Callback Calls`` .. sectnum:: -Drivers MUST be able to authenticate using either authentication or provider -name if there are multiple principals configured on the server. Note that -``directConnection=true`` and ``readPreference=secondaryPreferred`` are needed because the server is a secondary on a replica set, on port ``27018``. +Drivers MUST be able to authenticate against a server configured with either one or two configured identity providers. + +Note that typically the preconfigured Atlas Dev clusters are used for testing. The URIs can be fetched +from the ``drivers/oidc`` Secrets vault, see vault instructions . Use ``OIDC_ATLAS_URI_SINGLE`` for ``MONGODB_URI_SINGLE`` and +``OIDC_ATLAS_URI_MULTI`` for ``OIDC_ATLAS_URI_MULTI``. + +If using local servers is preferred, using the ``drivers-evergreen-tools`` Local Testing method , +use ``mongodb://localhost/?authMechanism=MONGODB-OIDC`` for ``MONGODB_URI_SINGLE`` and +``mongodb://localhost:27018/?authMechanism=MONGODB-OIDC&directConnection=true&readPreference=secondaryPreferred`` +for ``MONGODB_URI_MULTI`` because the other server is a secondary on a replica set, on port ``27018``. + +The driver MUST generate valid local tokens at the location given by ``OIDC_TOKEN_DIR``, see ``drivers-evergreen-tools`` for example . +. Drivers will need to be able to internally query and clear the cached credentials to verify usage for testing purposes. Clearing the cache means removing all data from the cache, including ``OIDCMechanismServerStep1`` information. -Drivers MUST set the ``AWS_WEB_IDENTITY_TOKEN_FILE`` environment variable -to the location of valid ``test_user1`` credentials at the beginning of each -test, unless otherwise specified. - -Unless otherwise specified, tests will use a URL -of the form ``mongodb://localhost/?authMechanism=MONGODB-OIDC``. +The default OIDC client used in the tests will be configured with ``MONGODB_URI_SINGLE`` and a valid request callback handler +that returns the ``test_user1`` local token. +https://wiki.corp.mongodb.com/display/DRIVERS/Using+AWS+Secrets+Manager+to+Store+Testing+Secrets +https://github.com/mongodb-labs/drivers-evergreen-tools/blob/master/.evergreen/auth_oidc/README.md Callback-Driven Auth ==================== @@ -39,149 +46,65 @@ is one principal configured. Single Principal Implicit Username ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -- Clear the cache. -- Create a request callback returns a valid token. -- Create a client that uses the default OIDC url and the request callback. +- Create default OIDC client. - Perform a ``find`` operation. that succeeds. - Close the client. Single Principal Explicit Username ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -- Clear the cache. -- Create a request callback that returns a valid token. -- Create a client with a url of the form ``mongodb://test_user1@localhost/?authMechanism=MONGODB-OIDC`` and the OIDC request callback. +- Create a client with ``MONGODB_URI_SINGLE``, a username of ``test_user1``, and the OIDC request callback. - Perform a ``find`` operation that succeeds. - Close the client. Multiple Principal User 1 ~~~~~~~~~~~~~~~~~~~~~~~~~ -- Clear the cache. -- Create a request callback that returns a valid token. -- Create a client with a url of the form ``mongodb://test_user1@localhost:27018/?authMechanism=MONGODB-OIDC&directConnection=true&readPreference=secondaryPreferred`` and a valid OIDC request callback. +- Create a client with ``MONGODB_URI_MULTI``, a username of ``test_user1``, and the OIDC request callback. - Perform a ``find`` operation that succeeds. - Close the client. Multiple Principal User 2 ~~~~~~~~~~~~~~~~~~~~~~~~~ -- Clear the cache. - Create a request callback that reads in the generated ``test_user2`` token file. -- Create a client with a url of the form ``mongodb://test_user2@localhost:27018/?authMechanism=MONGODB-OIDC&directConnection=true&readPreference=secondaryPreferred`` and a valid OIDC request callback. +- Create a client with ``MONGODB_URI_MULTI``, a username of ``test_user2``, and the OIDC request callback. - Perform a ``find`` operation that succeeds. - Close the client. Multiple Principal No User ~~~~~~~~~~~~~~~~~~~~~~~~~~ -- Clear the cache. -- Create a client with a url of the form ``mongodb://localhost:27018/?authMechanism=MONGODB-OIDC&directConnection=true&readPreference=secondaryPreferred`` and a valid OIDC request callback. +- Create a client with ``MONGODB_URI_MULTI``, no username, and the OIDC request callback. - Assert that a ``find`` operation fails. - Close the client. Allowed Hosts Blocked ~~~~~~~~~~~~~~~~~~~~~ -- Clear the cache. -- Create a client that uses the OIDC url and a request callback, and an - ``ALLOWED_HOSTS`` that is an empty list. +- Create a default OIDC client, with an ``ALLOWED_HOSTS`` that is an empty list. - Assert that a ``find`` operation fails with a client-side error. - Close the client. - Create a client that uses the url ``mongodb://localhost/?authMechanism=MONGODB-OIDC&ignored=example.com`` a request callback, and an - ``ALLOWED_HOSTS`` that contains ["example.com"]. + ``ALLOWED_HOSTS`` that contains ``["example.com"]``. - Assert that a ``find`` operation fails with a client-side error. - Close the client. -Lock Avoids Extra Callback Calls -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -- Clear the cache. -- Create a request callback that returns a token that will expire soon, and - a refresh callback. Ensure that the request callback has a time delay, and - that we can record the number of times each callback is called. -- Spawn two threads or async operations that do the following: - - Create a client with the callbacks. - - Run a find operation that succeeds. - - Close the client. - - Create a new client with the callbacks. - - Run a find operation that succeeds. - - Close the client. -- Join the two threads or simultaneously call the async operations -- Ensure that the request callback has been called once, and the refresh - callback has been called twice, or that no async function has been - entered simultaneously. - -AWS Automatic Auth -================== - -Drivers MUST be able to authenticate using the "aws" provider workflow -simulating an EC2 instance with an enabled web identity token provider, -generated by Drivers Evergreen Tools. - -Single Principal -~~~~~~~~~~~~~~~~ -- Create a client with a url of the form ``mongodb://localhost/?authMechanism=MONGODB-OIDC&authMechanismProperties=PROVIDER_NAME:aws``. -- Perform a ``find`` operation that succeeds. -- Close the client. - -Multiple Principal User 1 -~~~~~~~~~~~~~~~~~~~~~~~~~ -- Create a client with a url of the form ``mongodb://localhost:27018/?authMechanism=MONGODB-OIDC&authMechanismProperties=PROVIDER_NAME:aws&directConnection=true&readPreference=secondaryPreferred``. -- Perform a ``find`` operation that succeeds. -- Close the client. - -Multiple Principal User 2 -~~~~~~~~~~~~~~~~~~~~~~~~~ -- Set the ``AWS_WEB_IDENTITY_TOKEN_FILE`` environment variable - to the location of valid ``test_user2`` credentials. -- Create a client with a url of the form ``mongodb://localhost:27018/?authMechanism=MONGODB-OIDC&authMechanismProperties=PROVIDER_NAME:aws&directConnection=true&readPreference=secondaryPreferred``. -- Perform a ``find`` operation that succeeds. -- Close the client. - -Allowed Hosts Ignored -~~~~~~~~~~~~~~~~~~~~~ -- Create a client with a url of the form ``mongodb://localhost/?authMechanism=MONGODB-OIDC&authMechanismProperties=PROVIDER_NAME:aws``, and an - ``ALLOWED_HOSTS`` that is an empty list. -- Assert that a ``find`` operation succeeds. -- Close the client. - Callback Validation =================== Valid Callbacks ~~~~~~~~~~~~~~~ -- Clear the cache. -- Create request and refresh callback that validate their inputs and return - a valid token. The request callback must return a token that expires in - one minute. +- Create request callback that validates its inputs and returns a valid token. - Create a client that uses the above callbacks. - Perform a ``find`` operation that succeeds. Verify that the request callback was called with the appropriate inputs, including the timeout parameter if possible. Ensure that there are no unexpected fields. - Close the client. -- Create a new client with the same configuration. -- Perform a ``find`` operation that succeeds. Verify that the refresh - callback was called with the appropriate inputs, including the timeout - parameter if possible. -- Close the client. Request Callback Returns Null ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -- Clear the cache. - Create a client with a request callback that returns ``null``. - Perform a ``find`` operation that fails. - Close the client. -Refresh Callback Returns Null -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -- Clear the cache. -- Create request callback that returns a valid token that will expire in a - minute, and a refresh callback that returns ``null``. -- Perform a ``find`` operation that succeeds. -- Close the client. -- Create a new client with the same configuration. -- Perform a ``find`` operation that fails. -- Close the client. - Request Callback Returns Invalid Data ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -- Clear the cache. - Create a client with a request callback that returns data not conforming to the ``OIDCRequestTokenResult`` with missing field(s). - Perform a ``find`` operation that fails. @@ -191,107 +114,12 @@ Request Callback Returns Invalid Data - Perform a ``find`` operation that fails. - Close the client. -Refresh Callback Returns Missing Data -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -- Clear the cache. -- Create request callback that returns a valid token that will expire in a - minute, and a refresh callback that returns data not conforming to - the ``OIDCRequestTokenResult`` with missing field(s). -- Create a client with the callbacks. -- Perform a ``find`` operation that succeeds. -- Close the client. -- Create a new client with the same callbacks. -- Perform a ``find`` operation that fails. -- Close the client. - -Refresh Callback Returns Extra Data -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -- Clear the cache. -- Create request callback that returns a valid token that will expire in a - minute, and a refresh callback that returns data not conforming to - the ``OIDCRequestTokenResult`` with extra field(s). -- Create a client with the callbacks. -- Perform a ``find`` operation that succeeds. -- Close the client. -- Create a new client with the same callbacks. -- Perform a ``find`` operation that fails. -- Close the client. - -Cached Credentials -================== - -Drivers MUST ensure that they are testing the ability to cache credentials. -Unless otherwise specified, the tests MUST be performed with the callback-driven workflow with a provided request and refresh callback. If -desired, the caching tests MAY be done using mock server responses. -The following tests assume a global cache is in use. If a different -cache scheme is in use, appropriate tests MUST be written to ensure that -the cache is performing as intended. - -Cache with refresh -~~~~~~~~~~~~~~~~~~ -- Clear the cache. -- Create a new client with a request callback that gives credentials that - expire in on minute. -- Ensure that a ``find`` operation adds credentials to the cache. -- Close the client. -- Create a new client with the same request callback and a refresh callback. -- Ensure that a ``find`` operation results in a call to the refresh callback. -- Close the client. - -Cache with no refresh -~~~~~~~~~~~~~~~~~~~~~ -- Clear the cache. -- Create a new client with a request callback that gives credentials that - expire in one minute. -- Ensure that a ``find`` operation adds credentials to the cache. -- Close the client. -- Create a new client with the a request callback but no refresh callback. -- Ensure that a ``find`` operation results in a call to the request callback. -- Close the client. - -Cache key includes callback -~~~~~~~~~~~~~~~~~~~~~~~~~~~ -If the driver does not support using callback references or hashes as part of -the cache key, skip this test. This test ensures that the callback is -considered as part of the cache key. - -- Clear the cache. -- Create a new client with a request callback that does not give an - ```expiresInSeconds``` value. -- Ensure that a ``find`` operation adds credentials to the cache. -- Close the client. -- Create a new client with a different request callback. -- Ensure that a ``find`` operation replaces the one-time use entry and adds a new entry to the cache. -- Close the client. - -Error clears cache -~~~~~~~~~~~~~~~~~~ -- Clear the cache. -- Create a new client with a valid request callback that gives credentials - that expire within 5 minutes and a refresh callback that gives invalid - credentials. -- Ensure that a ``find`` operation adds a new entry to the cache. -- Close the client. -- Create a new client with the same parameters. -- Ensure that a subsequent ``find`` operation results in an error. -- Ensure that the cache value cleared. -- Close the client. - -AWS Automatic workflow does not use cache -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -- Clear the cache. -- Create a new client that uses the AWS automatic workflow. -- Ensure that a ``find`` operation does not add credentials to the cache. -- Close the client. - Speculative Authentication ========================== We can only test the successful case, by verifying that ``saslStart`` is not called. -- Clear the cache. -- Create a client with a request callback that returns a valid token - that will not expire soon. +- Create a client with a request callback that returns a valid token. - Set a fail point for ``saslStart`` commands of the form: .. code:: javascript @@ -316,10 +144,6 @@ is not called. - Perform a ``find`` operation that succeeds. - Close the client. -- Create a new client with the same properties without clearing the cache. -- Set a fail point for ``saslStart`` commands. -- Perform a ``find`` operation that succeeds. -- Close the client. Reauthentication ================ @@ -329,15 +153,12 @@ operation. Succeeds ~~~~~~~~ -- Clear the cache. -- Create request and refresh callbacks that return valid credentials - that will not expire soon. -- Create a client with the callbacks and an event listener. The following +- Create a default OIDC client and add an event listener. The following assumes that the driver does not emit ``saslStart`` or ``saslContinue`` events. If the driver does emit those events, ignore/filter them for the purposes of this test. - Perform a ``find`` operation that succeeds. -- Assert that the refresh callback has not been called. +- Assert that the request callback has been called once. - Clear the listener state if possible. - Force a reauthenication using a ``failCommand`` of the form: @@ -362,7 +183,7 @@ Succeeds remove the ``failCommand`` after the test to prevent leakage. - Perform another find operation that succeeds. -- Assert that the refresh callback has been called once, if possible. +- Assert that the request callback has been called twice. - Assert that the ordering of list started events is [``find``], , ``find``. Note that if the listener stat could not be cleared then there will and be extra ``find`` command. @@ -370,69 +191,44 @@ Succeeds - Assert that a ``find`` operation failed once during the command execution. - Close the client. -Retries and Succeeds with Cache -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -- Clear the cache. -- Create request and refresh callbacks that return valid credentials - that will not expire soon. -- Perform a ``find`` operation that succeeds. -- Force a reauthenication using a ``failCommand`` of the form: - -.. code:: javascript - - { - "configureFailPoint": "failCommand", - "mode": { - "times": 2 - }, - "data": { - "failCommands": [ - "find", "saslStart" - ], - "errorCode": 391 - } - } - -- Perform a ``find`` operation that succeeds. -- Close the client. - -Retries and Fails with no Cache -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -- Clear the cache. -- Create request and refresh callbacks that return valid credentials - that will not expire soon. -- Perform a ``find`` operation that succeeds (to force a speculative auth). -- Clear the cache. -- Force a reauthenication using a ``failCommand`` of the form: +Fails +~~~~~ +- Create a default OIDC client. +- Perform a find operation that succeeds (to force a speculative auth). +- Assert that the request callback has been called once. +- Force a reauthenication using a failCommand of the form: .. code:: javascript - - { - "configureFailPoint": "failCommand", - "mode": { - "times": 2 - }, - "data": { - "failCommands": [ - "find", "saslStart" - ], - "errorCode": 391 - } + { + "configureFailPoint": "failCommand", + "mode": { + "times": 2 + }, + "data": { + "failCommands": [ + "find", "saslStart" + ], + "errorCode": 391 } + } -- Perform a ``find`` operation that fails. +- Perform a find operation that fails. +- Assert that the request callback has been called twice. - Close the client. Separate Connections Avoid Extra Callback Calls ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -- Clear the cache. -- Create request and refresh callbacks that return tokens that will not expire - soon. Ensure that we can record the number of times each callback is called. -- Create two clients using the callbacks -- Peform a find operation on each client that succeeds. -- Ensure that the request callback has been called once and the refresh - callback has not been called. -- Force a reauthenication on the first client using a ``failCommand`` of the +The following test assumes that the driver will be able to share a cache between +two MongoClient objects, or ensure that the same MongoClient is used with two +different connections. If that is not possible, the test may be skipped. + +- Create a request callback that returns valid, and ensure that we can record the number + of times the callback is called. +- Create two clients using the callbacks, or a single client and two connection objects. +- Peform a find operation on each client/connection that succeeds. +- If using a single client, share the underlying cache between clients. +- Ensure that the request callback has been called twice. +- Force a reauthenication on the first client/connection using a ``failCommand`` of the form: .. code:: javascript @@ -451,9 +247,7 @@ Separate Connections Avoid Extra Callback Calls } - Perform a ``find`` operation that succeds. -- Ensure that the request callback has been called once and the refresh - callback has been called once. -- Repeat the ``failCommand`` and ``find`` operation on the second client. -- Ensure that the request callback has been called once and the refresh - callback has been called once. -- Close both clients. \ No newline at end of file +- Ensure that the request callback has been called three times. +- Repeat the ``failCommand`` and ``find`` operation on the second client/connection. +- Ensure that the request callback has been called three times. +- Close all clients/connections. From f01c6788463398e477439995e34c2d3f6950d6b0 Mon Sep 17 00:00:00 2001 From: Steven Silvester Date: Mon, 4 Sep 2023 20:51:35 -0500 Subject: [PATCH 02/44] finish refactor --- source/auth/auth.rst | 409 ++++++++---------- .../auth/tests/legacy/connection-string.yml | 33 +- source/auth/tests/mongodb-oidc.rst | 73 +++- 3 files changed, 243 insertions(+), 272 deletions(-) diff --git a/source/auth/auth.rst b/source/auth/auth.rst index 6b16be5bc2..b6b9dc6e0f 100644 --- a/source/auth/auth.rst +++ b/source/auth/auth.rst @@ -39,7 +39,7 @@ Definitions Credential The pieces of information used to establish the authenticity of a user. This is composed of an identity and some form of evidence such as a password or a certificate. -FQDN +FQDN Fully Qualified Domain Name Mechanism @@ -247,8 +247,8 @@ Caching credentials in SCRAM In the implementation of SCRAM authentication mechanisms (e.g. SCRAM-SHA-1 and SCRAM-SHA-256), drivers MUST maintain a cache of computed SCRAM credentials. -The cache entries SHOULD be identified by the password, salt, iteration count, -and a value that uniquely identifies the authentication mechanism (e.g. "SHA1" +The cache entries SHOULD be identified by the password, salt, iteration count, +and a value that uniquely identifies the authentication mechanism (e.g. "SHA1" or "SCRAM-SHA-256"). The cache entry value MUST be either the ``saltedPassword`` parameter or the @@ -443,7 +443,7 @@ Many languages will have the ability to utilize 3rd party libraries. The server GSSAPI ~~~~~~ -:since: +:since: 2.4 Enterprise 2.6 Enterprise on Windows @@ -663,7 +663,7 @@ source MUST be specified. Defaults to the database name if supplied on the connection string or ``admin``. password - MUST be specified. + MUST be specified. mechanism MUST be "SCRAM-SHA-1" @@ -743,33 +743,33 @@ MONGODB-AWS :since: 4.4 -MONGODB-AWS authenticates using AWS IAM credentials (an access key ID and a secret access key), `temporary AWS IAM credentials `_ obtained from an -`AWS Security Token Service (STS) `_ +MONGODB-AWS authenticates using AWS IAM credentials (an access key ID and a secret access key), `temporary AWS IAM credentials `_ obtained from an +`AWS Security Token Service (STS) `_ `Assume Role `_ request, an OpenID Connect ID token that supports `AssumeRoleWithWebIdentity `_, or temporary AWS IAM credentials assigned to an `EC2 instance `_ or ECS task. Temporary credentials, in addition to an access key ID and a secret access key, includes a security (or session) token. -MONGODB-AWS requires that a client create a randomly generated nonce. It is -imperative, for security sake, that this be as secure and truly random as possible. Additionally, the secret access key and only the secret access key is sensitive. Drivers MUST take proper precautions to ensure we do not leak this info. +MONGODB-AWS requires that a client create a randomly generated nonce. It is +imperative, for security sake, that this be as secure and truly random as possible. Additionally, the secret access key and only the secret access key is sensitive. Drivers MUST take proper precautions to ensure we do not leak this info. All messages between MongoDB clients and servers are sent as BSON V1.1 Objects in the payload field of saslStart and saslContinue. -All fields in these messages have a "short name" which is used in the serialized +All fields in these messages have a "short name" which is used in the serialized BSON representation and a human-readable "friendly name" which is used in this specification. They are as follows: -==== ==================== ================= ============================================================================================================================================== +==== ==================== ================= ============================================================================================================================================== Name Friendly Name Type Description ==== ==================== ================= ============================================================================================================================================== -r client nonce BinData Subtype 0 32 byte cryptographically secure random number +r client nonce BinData Subtype 0 32 byte cryptographically secure random number p gs2-cb-flag int32 The integer representation of the ASCII charater 'n' or 'y', i.e., ``110`` or ``121`` s server nonce BinData Subtype 0 64 bytes total, 32 bytes from the client first message and a 32 byte cryptographically secure random number generated by the server -h sts host string FQDN of the STS service +h sts host string FQDN of the STS service a authorization header string Authorization header for `AWS Signature Version 4 `_ d X-AMZ-Date string Current date in UTC. See `AWS Signature Version 4 `_ t X-AMZ-Security-Token string Optional AWS security token -==== ==================== ================= ============================================================================================================================================== +==== ==================== ================= ============================================================================================================================================== Drivers MUST NOT advertise support for channel binding, as the server does not support it and legacy servers may fail authentication if drivers advertise -support. The client-first-message MUST set the gs2-cb-flag to the integer representation +support. The client-first-message MUST set the gs2-cb-flag to the integer representation of the ASCII character ``n``, i.e., ``110``. Conversation @@ -785,7 +785,7 @@ Client First .. code:: javascript - { + { "r" : new BinData(0, "dzw1U2IwSEtgaWI0IUxZMVJqc2xuQzNCcUxBc05wZjI="), "p" : 110 } @@ -794,7 +794,7 @@ Server First .. code:: javascript - { + { "s" : new BinData(0, "dzw1U2IwSEtgaWI0IUxZMVJqc2xuQzNCcUxBc05wZjIGS0J9EgLwzEZ9dIzr/hnnK2mgd4D7F52t8g9yTC5cIA=="), "h" : "sts.amazonaws.com" } @@ -817,9 +817,9 @@ Client First .. code:: javascript - { - "saslStart" : 1, - "mechanism" : "MONGODB-AWS" + { + "saslStart" : 1, + "mechanism" : "MONGODB-AWS" "payload" : new BinData(0, "NAAAAAVyACAAAAAAWj0lSjp8M0BMKGU+QVAzRSpWfk0hJigqO1V+b0FaVz4QcABuAAAAAA==") } @@ -828,8 +828,8 @@ Server First .. code:: javascript { - "conversationId" : 1, - "done" : false, + "conversationId" : 1, + "done" : false, "payload" : new BinData(0, "ZgAAAAVzAEAAAAAAWj0lSjp8M0BMKGU+QVAzRSpWfk0hJigqO1V+b0FaVz5Rj7x9UOBHJLvPgvgPS9sSzZUWgAPTy8HBbI1cG1WJ9gJoABIAAABzdHMuYW1hem9uYXdzLmNvbQAA"), "ok" : 1.0 } @@ -859,16 +859,16 @@ The following example shows a finished Authorization header. .. code:: javascript - Authorization: AWS4-HMAC-SHA256 Credential=AKIDEXAMPLE/20150830/us-east-1/iam/aws4_request, SignedHeaders=content-type;host;x-amz-date, Signature=5d672d79c15b13162d9279b0855cfba6789a8edb4c82c400e06b5924a6f2b5d7 + Authorization: AWS4-HMAC-SHA256 Credential=AKIDEXAMPLE/20150830/us-east-1/iam/aws4_request, SignedHeaders=content-type;host;x-amz-date, Signature=5d672d79c15b13162d9279b0855cfba6789a8edb4c82c400e06b5924a6f2b5d7 The following diagram is a summary of the steps drivers MUST follow to calculate the signature. .. image:: includes/calculating_a_signature.png ======================== ====================================================================================================== -Name Value +Name Value ======================== ====================================================================================================== -HTTP Request Method POST +HTTP Request Method POST URI / Content-Type* application/x-www-form-urlencoded Content-Length* 43 @@ -888,7 +888,7 @@ Body Action=GetCallerIdentity&Version=2011-06-15 Region Calculation `````````````````` -To get the region from the host, the driver MUST follow the algorithm expressed in psuedocode below. :: +To get the region from the host, the driver MUST follow the algorithm expressed in psuedocode below. :: if the host is invalid according to the rules described earlier the region is undefined and the driver must raise an error. @@ -899,21 +899,21 @@ To get the region from the host, the driver MUST follow the algorithm expressed else // the valid host string contains no periods and is not "aws.amazonaws.com" the region is "us-east-1" -Examples are provided below. +Examples are provided below. ============================== ========= ====================================================== -Host Region Notes +Host Region Notes ============================== ========= ====================================================== -sts.amazonaws.com us-east-1 the host is "sts.amazonaws.com"; use `us-east-1` -sts.us-west-2.amazonaws.com us-west-2 use the second label +sts.amazonaws.com us-east-1 the host is "sts.amazonaws.com"; use `us-east-1` +sts.us-west-2.amazonaws.com us-west-2 use the second label sts.us-west-2.amazonaws.com.ch us-west-2 use the second label -example.com com use the second label +example.com com use the second label localhost us-east-1 no "``.``" character; use the default region -sts..com second label is empty -.amazonaws.com starts with a period -sts.amazonaws. ends with a period -"" empty string -"string longer than 255" string longer than 255 bytes +sts..com second label is empty +.amazonaws.com starts with a period +sts.amazonaws. ends with a period +"" empty string +"string longer than 255" string longer than 255 bytes ============================== ========= ====================================================== `MongoCredential`_ Properties @@ -940,8 +940,8 @@ mechanism_properties Obtaining Credentials ````````````````````` -Drivers will need AWS IAM credentials (an access key, a secret access key and optionally a session token) to complete the steps in the `Signature Version 4 Signing Process -`_. If a username and password are provided drivers +Drivers will need AWS IAM credentials (an access key, a secret access key and optionally a session token) to complete the steps in the `Signature Version 4 Signing Process +`_. If a username and password are provided drivers MUST use these for the AWS IAM access key and AWS IAM secret key, respectively. If, additionally, a session token is provided Drivers MUST use it as well. If a username is provided without a password (or vice-versa) or if *only* a session token is provided Drivers MUST raise an error. In other words, regardless of how Drivers obtain credentials the only valid combination of credentials is an access key ID and a secret access key or an access key ID, a secret access key and a session token. AWS recommends using an SDK to "take care of some of the heavy lifting @@ -1083,7 +1083,7 @@ credentials. Querying the URI will return the JSON response: "SecretAccessKey": , "Token": } - + EC2 endpoint ____________ If the environment variable ``AWS_CONTAINER_CREDENTIALS_RELATIVE_URI`` is unset, drivers MUST use the EC2 endpoint, @@ -1132,7 +1132,7 @@ To re-direct queries from the EC2 endpoint to the mock server, replace the link- $ TOKEN=`curl -X PUT "http://localhost:8000/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 30"` $ ROLE_NAME=`curl http://localhost:8000/latest/meta-data/iam/security-credentials/ -H "X-aws-ec2-metadata-token: $TOKEN"` $ curl http://localhost:8000/latest/meta-data/iam/security-credentials/$ROLE_NAME -H "X-aws-ec2-metadata-token: $TOKEN" - + The JSON response from both the actual and mock EC2 endpoint will be in this format: .. code:: javascript @@ -1147,8 +1147,8 @@ The JSON response from both the actual and mock EC2 endpoint will be in this for "Expiration": } -From the JSON response drivers -MUST obtain the ``access_key``, ``secret_key`` and ``security_token`` which will be used during the `Signature Version 4 Signing Process +From the JSON response drivers +MUST obtain the ``access_key``, ``secret_key`` and ``security_token`` which will be used during the `Signature Version 4 Signing Process `_. Caching Credentials @@ -1180,65 +1180,16 @@ MONGODB-OIDC :since: 7.0 Enterprise -MONGODB-OIDC authenticates using an `OIDC `_ access tokens. Drivers MUST support -both Callback-driven OIDC and Automatic OIDC Authentication for AWS. - - -Conversation -```````````` - -Authenticating using the MONGODB-OIDC mechanism will require 1 or 2 round trips between the MongoDB driver and server. The requests from the driver and the replies from the server are described by the following IDL structs which are encoded in the payload as octet sequences defining BSON objects: - -.. code:: idl - - PrincipalStepRequest: - description: Driver’s opening request in saslStart - fields: - n: - description: "Name of the OIDC user Principal" - type: string - optional: true - -Note that the principal name is optional as it may be provided by the IDP in environments where only one IDP is used. The username provided by the user MUST be used as the principalName. - -.. code:: idl - - IdPServerInfo: - description: "The information used by callbacks to authenticate with the Identity Provider." - fields: - issuer: - description: >- - URL which describes the Authentication Server. This identifier should be - the iss of provided access tokens, and be viable for RFC8414 - metadata discovery and RFC9207 identification. - type:string - clientId: - description: "Unique client ID for this OIDC client" - type: string - requestScopes: - description: "Additional scopes to request from IDP" - type: array - optional: true - -Server will use principalName (n) if provided in the driver’s PrincipalStepRequest to select an appropriate IDP. This IDP's configuration will be returned in the server’s response that will be used by the end-user to acquire an Access Token. - -This Access Token will be used as the JWT in the driver’s JwtStepRequest to complete authentication. - -.. code:: idl - - JwtStepRequest: - description: "Client's request with signed token" - fields: - jwt: - description: "Compact serialized JWT with signature" - cpp_name: JWT - type: string +MONGODB-OIDC authenticates using an `OIDC `_ access tokens. +Drivers MAY implement the human authentication workflow with ``REQUEST_TOKEN_CALLBACK``, but it is only currently required in the Node driver, +to support usage in the MongoDB Shell. The machine authentication workflow is currently not supported. `MongoCredential`_ Properties ````````````````````````````` username - MUST NOT be specified in automatic authentication. Drivers MUST allow the user to specify this in the callback-driven authentication. If a user omits this when multiple OIDC providers are configured, the server will produce an error during authentication. + MUST NOT be specified in machine authentication. Drivers MUST allow the user to specify this in the human authentication. + If a user omits this when multiple OIDC providers are configured, the server will produce an error during authentication. source MUST be "$external". Defaults to ``$external``. @@ -1250,20 +1201,13 @@ mechanism MUST be "MONGODB-OIDC" mechanism_properties - PROVIDER_NAME - Drivers MUST allow the user to specify a name for using a service - to obtain credentials that is one of ["aws"]. REQUEST_TOKEN_CALLBACK - Drivers MUST allow the user to specify a callback of the form - "onRequest" (defined below), if the driver supports - providing objects as mechanism property values. Otherwise the driver MUST allow it as a MongoClientOption. - REFRESH_TOKEN_CALLBACK - Drivers MUST allow the user to specify a callback of the form - "onRefresh" (defined below), if the driver supports + Drivers MUST allow the user to specify a callback of the form "onRequest" (defined below), if the driver supports providing objects as mechanism property values. Otherwise the driver MUST allow it as a MongoClientOption. ALLOWED_HOSTS The list of allowed hostnames or ip-addresses (ignoring ports) for - MongoDB connections. The hostnames may include a leading "*." wildcard, which allows for matching (potentially nested) subdomains. ALLOWED_HOSTS is a + MongoDB connections. The hostnames may include a leading "*." wildcard, which allows for matching + (potentially nested) subdomains. ALLOWED_HOSTS is a security feature and MUST default to ``["*.mongodb.net", '*.mongodb-dev.net", "*.mongodbgov.net", "localhost", "127.0.0.1", "::1"]``. When ``MONGODB-OIDC`` authentication is attempted against a hostname @@ -1272,151 +1216,164 @@ mechanism_properties callbacks. This value MUST not be allowed in the URI connection string. The hostname check MUST be performed after SRV record resolution, if applicable. + This property is only applicable when ``REQUEST_TOKEN_CALLBACK`` is given. -Drivers MUST NOT send a PrincipalStepRequest when performing automatic authentication -or when there is a cached IdPServerResponse. Drivers must instead use ``saslStart`` with a JwtStepRequest. - -Speculative Authentication -``````````````````````````````````` -Drivers MUST implement speculative authentication for MONGODB-OIDC during the ``hello`` handshake. If there is an unexpired access token, the JwtStepRequest SASL command will be used as the speculation command. If there is no cache value, the PrincipalStepRequest will be used as the speculation command. The driver MUST NOT call any callbacks during speculative authentication. +Interfaces +`````````` -User Provided Callbacks -``````````````````````` +Authenticating using the MONGODB-OIDC mechanism will require 1 or 2 round trips between the MongoDB driver and server. +The requests from the driver and the replies from the server are described by the following interfaces which are encoded +in the payload as octet sequences defining BSON objects: -Drivers MUST allow the user to provide callbacks for token request and -token refresh. The driver MUST provide a way for the both callbacks to be either automatically -canceled, or to cancel itself. This can be as a timeout argument to the -callback, a cancellation context passed to the callback, or some other -language-appropriate mechanism. The timeout duration MUST be 5 minutes, -to account for the fact that there may be human interaction involved. +.. code:: typescript -Callbacks can be synchronous and/or asynchronous, depending on the driver -and/or language. Asynchronous callbacks should be preferred when other -operations in the driver use asynchronous functions. + // Driver’s opening request in saslStart. + interface PrincipalStepRequest { + // Name of the OIDC user Principal. + n?: str; + } -The driver MUST pass the following information to the request callback: ``IdpServerInfo``, and either a ``timeoutSeconds`` or ``timeoutContext`` object for the callback. The signature of the callback is up to the driver's discretion, but the driver MUST ensure that, in the future, callbacks may have additional optional parameters passed to them. An example might look like: +Note that the principal name is optional as it may be provided by the IdP in environments where only one IdP is used. +If given, then ``username`` provided by the user MUST be used as the Principal ``(n)``. -.. code: typescript +.. code:: typescript - function onRequest(info: IdpServerInfo, params: RequestParameters): IdpServerResponse + // The information used by a REQUEST_TOKEN_CALLBACK to authenticate with the Identity Provider. + interface IdpInfo { + // URL which describes the Authentication Server. This identifier should be + // the iss of provided access tokens, and be viable for RFC8414 + // metadata discovery and RFC9207 identification. + issuer: str; -In this example, one of the timeout values would then need to be present on ``RequestParameters``. ``IdpServerResponse`` is defined as: + // Unique client ID for this OIDC client. + clientId: str; -.. code:: idl + // Additional scopes to request from IdP. + requestScopes?: Array; + } - IdPServerResponse: - description: "The result of a token request" - strict: false - fields: - accessToken: - description: "The OIDC access token" - type: string - expiresInSeconds: - description: "The expiration time in seconds from the current time" - type: int - optional: true - refreshToken: - description: "The OIDC refresh token" - type: str - optional: true +The server will use Principal ``(n)`` if provided in the driver’s ``PrincipalStepRequest`` to select an appropriate IdP. +This IdP's configuration will be returned in the server’s response that will be used by the end-user to acquire an Access Token. -The token refresh callback must take the same arguments as the request callback, as well as the ``refreshToken`` string given by the ``IdpServerResponse``, and might look like the following:: +This Access Token will be used as the JWT in the driver’s ``JwtStepRequest`` to complete authentication. .. code:: typescript - function onRefresh(info: IdpServerInfo, params: RefreshParameters): IdpServerResponse + // Client's request with signed token. + interface JwtStepRequest: + // Compact serialized JWT with signature. + jwt: str; + } -Before calling a callback, the driver MUST acquire a lock unique to the cache key. The driver MUST ensure that credentials have not changed between when the lock was requested and when it was acquired. The lock MUST be released -when the callback call as finished or errored. -This is because request callbacks may involve human interaction, and refresh -callbacks could use refresh tokens that can only be used once. +The IdP response that is expected to be returned by the ``REQUEST_TOKEN_CALLBACK`` is as follows: -If either callback does not return an object in the correct form of ``IdpServerResponse``, the driver MUST raise an error either using the type system or by raising an error when non-optional properties are missing . The driver MUST NOT attempt to validate -the token(s) directly. It is expected that if the server changes the expected fields, the SASL exchange will be updated with a version parameter. Drivers do not need to attempt to provide old-driver-new-server compatibility. + .. code:: typescript -If the refresh callback is given and the request callback is not given, -the driver MUST raise an error. If PROVIDER_NAME is given and one or more -callbacks are given, the driver MUST raise an error. + // The result of a token request. + interface IdPResponse { + // The oidc access token. + accessToken: str; -If no callbacks are given, the driver MUST enforce that a PROVIDER_NAME -mechanism_properties is set and one of ("aws",). -The callback mechanism can be used to support both callback-based or automatic workflows that are not explicitly implemented -by drivers. If there is no callback and no PROVIDER_NAME, or the -PROVIDER_NAME is set but credentials cannot be automatically obtained, -the driver MUST raise an error. + // The OIDC refresh token. + refreshToken?: str; + // The expiration time in seconds from the current time (ignored). + expiresInSeconds?: str; + } -Supported Service Providers -``````````````````````````` +Conversation +```````````` +If using the ``PrincipalStepRequest`` first, the conversation will look like: -Drivers MUST support obtaining credentials for a service for "aws", given -by the PROVIDER_NAME mechanism property. In all cases the acquired token -will be given as the ``jwt`` argument and the JwtStepRequest MUST be made immediately, as part of speculative authentication if appropriate, skipping the PrincipalStepRequest. -Drivers MUST raise an error if both a PROVIDER_NAME and username are -given, since using a service will not use the username. +| C: :javascript:`{saslStart: 1, mechanism: "MONGODB-OIDC", payload: BinData(0, "...")}` +| S: :javascript:`{conversationId : 1, payload: BinData(0,"..."), done: false, ok: 1}` +| C: :javascript:`{saslContinue: 1, conversationId: 1, payload: BinData(0, "...")}` +| S: :javascript:`{conversationId: 1, payload: BinData(0,".."), done: true, ok: 1}` -AWS -___ +If using the ``JwtStepRequest`` directly, the conversation will look like: -When the PROVIDER_NAME mechanism property is set to "aws", the driver MUST -attempt to read the value given by the ``AWS_WEB_IDENTITY_TOKEN_FILE`` and -interpret it as a file path. The contents of the file are read as the -access token. If the path does not exist or cannot be read, or the environment variable does not exist, the driver MUST raise an error. +| C: :javascript:`{saslStart: 1, mechanism: "MONGODB-OIDC", payload: BinData(0, "...")}` +| S: :javascript:`{conversationId : 1, payload: BinData(0,"..."), done: true, ok: 1}` +Drivers MUST NOT send a ``PrincipalStepRequest`` when performing machine authentication +or when there is a cached Access Token. Drivers must instead use ``saslStart`` with a ``JwtStepRequest``. -Caching Credentials -``````````````````` +Caching +``````` +Drivers MUST cache the ``IdPInfo``, Access Token, and Refresh Token associated with each +``MongoClient``. If any operation fails the driver MUST clear the Access Token. The Refresh Token is handled differently, +see the ``Reauthentication`` section below. + +Drivers MUST also use a token generation id that is incremented whenever a new Access Token is retrieved. +When a connection succeeds, the token generation id MUST be associated with or stored on the connection object. +This value is used in the ``Reauthentication`` section below. + +Speculative Authentication +`````````````````````````` +Drivers MUST implement speculative authentication for MONGODB-OIDC during the ``hello`` handshake. +If there is a cached Access Token the ``JwtStepRequest`` SASL command will be used as the speculation command. +If there is no cached Access Token, the ``PrincipalStepRequest`` will be used as the speculation command. +The driver MUST NOT call the callback during speculative authentication. + +REQUEST_TOKEN_CALLBACK +`````````````````````` +The driver MUST provide a way for the ``REQUEST_TOKEN_CALLBACK`` to be either automatically +canceled, or to cancel itself. This can be as a timeout argument to the +callback, a cancellation context passed to the callback, or some other +language-appropriate mechanism. The timeout duration MUST be 5 minutes, +to account for the fact that there may be human interaction involved. +This callback is not subject to CSOT. -Drivers MUST use caching for callback-based authentication.. -When an authentication request is made and there is an available cached response, -the driver MUST use the cached Access Token from that response, if it has not expired. +Callbacks can be synchronous and/or asynchronous, depending on the driver +and/or language. Asynchronous callbacks should be preferred when other +operations in the driver use asynchronous functions. -A cache MUST be able to store the IDPServerInfo, the IdPServerResponse tokens, and the known expiration time of the access token. -The cache is kept alive even if the Access Token is expired to preserve the IdPServerInfo response, as well as -account for the Refresh Token, which typically has an (unknown) lifetime that -is longer than the access token lifetime. Drivers MUST ensure that the cache does not leak memory, by an appropriate time or space-based cache and auditing the cache at a regular interval. +The driver MUST pass the following information to the callback: +``IdpInfo``, and either a ``timeoutSeconds`` or ``timeoutContext`` object for the callback. +The signature of the callback is up to the driver's discretion, but the driver MUST ensure that, +in the future, callbacks may have additional optional parameters passed to them. +An example might look like: -If a global cache is used, the cache keys MUST include the username (or empty string) and the -actually used socket address and port for the current server. The cache key -MUST also include hashes of the callback function(s), if hash comparisons are possible in the driver language. In the case of a global cache, using the socket address and port accounts for the case when two different servers use the same username but could be configured differently. -There is an edge case where if the same username is used and two aliases -to the same local host address are given, there will be duplicate user/service -interactions, unless the driver can resolve the local host address as well. -Note that because we use the server socket address, there will different cache -keys for each member of a replica set. +.. code: typescript -The driver MUST cache the IdPServerInfo as part of the cache value, -to enable skipping the PrincipalStepRequest on subsequent authentications of the same -cache key. + interface RequestParameters { + // Timeout in seconds for the callback. Optionally, timeoutContext instead if applicable to language. + timeoutSeconds: int; -A global cache SHOULD be preferred, to prevent multiple browser interactions -in the case of an Authentication Code workflow. However, drivers or dev tools -can choose to use their own caching scheme if appropriate for their language/ -environment. + // The refresh token, if applicable, to be used by the callback to request a new token from the issuer. + refreshToken?: str; + } -A cached Access Token will expire 5 minutes before the ``expiresInSeconds`` -time, if given. If there is no ``expiresInSeconds``, the token must be considered expired as soon as the ensuing JwtStepRequest is started. If a cached value is found but its -Access Token is rejected by the server with a ``ReauthenticationRequired`` error, the Access Token must be marked expired and the Refresh callback MUST be called (if given) with the IdPServerInfo and Refresh Token, and it will return a new IdpServerResponse. If the Refresh Callback fails, the error is raised to the user. If the Refresh Callback succeeds, the new Access Token MUST be sent using a JwtStepRequest. If the request fails with a ReauthenticationRequired error, the cache should be cleared, and a PrincipalStepRequest MUST be sent. Next, the Request Callback should be called. If the callback fails, the error is raised to the user. -If the callback succeeds, the new Access Token MUST be sent using a JwtStepRequest. If the request fails with a ReauthenticationRequired error, that error MUST be propagated to the user. -The driver MUST have a guard or a flag in place to differentiate between a JwtStepRequest ReauthenticationRequired failure that takes place after a PrincipalStepRequest has been made to prevent an infinite loop. +.. code: typescript + + function onRequest(info: IdpInfo, params: RequestParameters): IdpResponse -If there is no refresh callback and no unexpired Access Token, the request callback will be called. Multithreaded drivers MUST ensure that there is at most one concurrent invocation of the above fallback logic for a given cache key. +Before calling the callback, the driver MUST acquire a lock unique to the ``MongoClient``. +The driver MUST ensure that credentials have not changed between when the lock was requested and when it was acquired. The lock MUST be released +when the callback call as finished or errored. +This is because the ``REQUEST_TOKEN_CALLBACK`` may involve human interaction, and refresh tokens migh only be able to be used used once. -If a cached value is used and the authentication step fails or times out, the driver MUST clear the -cached value. +If the callback does not return an object in the correct form of ``IdpResponse``, the driver MUST raise an error either using the type +system or by raising an error when non-optional properties are missing . +The driver MUST NOT attempt to validate the token(s) directly. +It is expected that if the server changes the expected fields, the SASL exchange will be updated with a version parameter. +Drivers do not need to attempt to provide old-driver-new-server compatibility. +If no callback is given, the driver MUST must raise a validation error. Reauthentication ```````````````` -When reauthentication is requested by the server (as a 391 error code) and MONGODB-OIDC is in use, the driver MUST -ensure that the Access Token that was most-recently used to authenticate this connection is not used for subsequent authentication, by marking it as expired. If non-expired Access Token is available in the cache, it should be used as usual. If a refresh -callback is given, it will be called as usual. Otherwise the IdPServerResponse will be cleared if present and authentication will proceed from the request callback. - -If the ``sasl`` step(s) fail with a 391 error code and the payload of the command contained ``jwt`` , the driver MUST clear the IdPServerResponse and -attempt to authenticate one more time starting from -``PrincipalStepRequest``. An initial reauthentication may fail for various reasons, such as token expiration or identity provider reconfiguration, so a second reauthentication might be needed. - -The driver MUST account for the case of multiple connections hitting a reauthentication error at different times, to prevent unnecessary callback calls. If another connection has already reauthenticated, then the Access Token should not be expired. The driver can either cache a token generation id per connection as well as in the main cache, or some other equivalent method to track whether a reauthentication has already occurred. +When reauthentication is requested by the server (as a 391 error code) and MONGODB-OIDC is in use, the driver MUST perform a reauthentication. +The driver MUST account for the case of multiple connections hitting a reauthentication error at different times. +The driver MUST also account for a reauthenication that results from an IdP configuration change on the server. +To accomplish these goal, the following algorithm is used to handle a reauthenication error: + +- First, see if the Access Token on the MongoClient is different than the Access Token on the connection. + - If they are different, optimisitically try to authenticate. On error, continue. +- Next, perform a ``PrincipalNameRequest`` to determine if the ``IdpInfo`` has changed. + - If it has changed, clear the current Access Token and Refresh Token, and continue. +- If we have a Refresh Token, attempt to authenticate. On error, clear the Refresh Token and continue. +- Attempt to authenticate. Raise any errors to the user. ------------------------- Connection String Options @@ -1441,7 +1398,7 @@ authSource For MONGODB-CR, SCRAM-SHA-1 and SCRAM-SHA-256 authMechanisms, the authSource defaults to the database name if supplied on the connection string or ``admin``. authMechanismProperties=PROPERTY_NAME:PROPERTY_VALUE,PROPERTY_NAME2:PROPERTY_VALUE2 - A generic method to set mechanism properties in the connection string. + A generic method to set mechanism properties in the connection string. For example, to set REALM and CANONICALIZE_HOST_NAME, the option would be ``authMechanismProperties=CANONICALIZE_HOST_NAME:forward,SERVICE_REALM:AWESOME``. @@ -1553,8 +1510,8 @@ As a URI, those have to be UTF-8 encoded and URL-escaped, e.g.: - mongodb://%E2%85%A8:IV@mongodb.example.com/admin - mongodb://%E2%85%A8:I%C2%ADV@mongodb.example.com/admin --------------------------- -Speculative Authentication +-------------------------- +Speculative Authentication -------------------------- See the speculative authentication section in the `MongoDB Handshake spec `_. @@ -1604,22 +1561,22 @@ Q: It's possible to continue using authenticated sockets even if new sockets fai Yes, that's technically true. The issue with doing that is for drivers using connection pooling. An application would function normally until an operation needed an additional connection(s) during a spike. Each new connection would fail to authenticate causing intermittent failures that would be very difficult to understand for a user. Q: Should a driver support multiple credentials? - No. + No. - Historically, the MongoDB server and drivers have supported multiple credentials, one per authSource, on a single connection. It was necessary because early versions of MongoDB allowed a user to be granted privileges - to access the database in which the user was defined (or all databases in the special case of the "admin" database). But with the introduction of role-based access control in MongoDB 2.6, that restriction was + Historically, the MongoDB server and drivers have supported multiple credentials, one per authSource, on a single connection. It was necessary because early versions of MongoDB allowed a user to be granted privileges + to access the database in which the user was defined (or all databases in the special case of the "admin" database). But with the introduction of role-based access control in MongoDB 2.6, that restriction was removed and it became possible to create applications that access multiple databases with a single authenticated user. - Role-based access control also introduces the potential for accidental privilege escalation. An application may, for example, authenticate user A from authSource X, and user B from authSource Y, thinking that + Role-based access control also introduces the potential for accidental privilege escalation. An application may, for example, authenticate user A from authSource X, and user B from authSource Y, thinking that user A has privileges only on collections in X and user B has privileges only on collections in Y. But with role-based access control that restriction no longer exists, and it's possible that user B has, for example, more privileges on collections in X than user A does. Due to this risk it's generally safer to create a single user with only the privileges required for a given application, and authenticate only that one user in the application. - In addition, since only a single credential is supported per authSource, certain mechanisms are restricted to a single credential and some credentials cannot be used in conjunction (GSSAPI and X509 both use the "$external" database). + In addition, since only a single credential is supported per authSource, certain mechanisms are restricted to a single credential and some credentials cannot be used in conjunction (GSSAPI and X509 both use the "$external" database). + + Finally, MongoDB 3.6 introduces sessions, and allows at most a single authenticated user on any connection which makes use of one. Therefore any application that requires multiple authenticated users will not be able to make use of any feature that builds on sessions (e.g. retryable writes). - Finally, MongoDB 3.6 introduces sessions, and allows at most a single authenticated user on any connection which makes use of one. Therefore any application that requires multiple authenticated users will not be able to make use of any feature that builds on sessions (e.g. retryable writes). - - Drivers should therefore guide application creators in the right direction by supporting the association of at most one credential with a MongoClient instance. + Drivers should therefore guide application creators in the right direction by supporting the association of at most one credential with a MongoClient instance. Q: Should a driver support lazy authentication? No, for the same reasons as given in the previous section, as lazy authentication is another mechanism for allowing multiple credentials to be associated with a single MongoClient instance. @@ -1647,7 +1604,7 @@ Q: Why does SCRAM sometimes SASLprep and sometimes not? problem, MongoDB decided that the best user experience on upgrade and lowest technical risk of implementation is to require drivers to continue to not SASLprep usernames in SCRAM-SHA-256. - + Q: Should drivers support accessing Amazon EC2 instance metadata in Amazon ECS? No. While it's possible to allow access to EC2 instance metadata in ECS, for security reasons, Amazon states it's best practice to avoid this. (See `accessing EC2 metadata in ECS `_ and `IAM Roles for Tasks `_) diff --git a/source/auth/tests/legacy/connection-string.yml b/source/auth/tests/legacy/connection-string.yml index 9f8aab4a72..d4fee90d4c 100644 --- a/source/auth/tests/legacy/connection-string.yml +++ b/source/auth/tests/legacy/connection-string.yml @@ -401,44 +401,13 @@ tests: mechanism: MONGODB-OIDC mechanism_properties: REQUEST_TOKEN_CALLBACK: true -- description: should recognise the mechanism with aws device (MONGODB-OIDC) - uri: mongodb://localhost/?authMechanism=MONGODB-OIDC&authMechanismProperties=PROVIDER_NAME:aws - valid: true - credential: - username: - password: - source: "$external" - mechanism: MONGODB-OIDC - mechanism_properties: - PROVIDER_NAME: aws -- description: should recognise the mechanism when auth source is explicitly specified - and with aws device (MONGODB-OIDC) - uri: mongodb://localhost/?authMechanism=MONGODB-OIDC&authSource=$external&authMechanismProperties=PROVIDER_NAME:aws - valid: true - credential: - username: - password: - source: "$external" - mechanism: MONGODB-OIDC - mechanism_properties: - PROVIDER_NAME: aws - description: should throw an exception if username and password are specified (MONGODB-OIDC) uri: mongodb://user:pass@localhost/?authMechanism=MONGODB-OIDC callback: - oidcRequest valid: false credential: -- description: should throw an exception if username and deviceName are specified - (MONGODB-OIDC) - uri: mongodb://principalName@localhost/?authMechanism=MONGODB-OIDC&PROVIDER_NAME:gcp - valid: false - credential: -- description: should throw an exception if specified deviceName is not supported - (MONGODB-OIDC) - uri: mongodb://localhost/?authMechanism=MONGODB-OIDC&authMechanismProperties=PROVIDER_NAME:unexisted - valid: false - credential: -- description: should throw an exception if neither deviceName nor callbacks specified +- description: should throw an exception if no callbacks specified (MONGODB-OIDC) uri: mongodb://localhost/?authMechanism=MONGODB-OIDC valid: false diff --git a/source/auth/tests/mongodb-oidc.rst b/source/auth/tests/mongodb-oidc.rst index f6496210c3..ffa3a151dd 100644 --- a/source/auth/tests/mongodb-oidc.rst +++ b/source/auth/tests/mongodb-oidc.rst @@ -15,28 +15,20 @@ Drivers MUST test the following scenarios: Drivers MUST be able to authenticate against a server configured with either one or two configured identity providers. -Note that typically the preconfigured Atlas Dev clusters are used for testing. The URIs can be fetched -from the ``drivers/oidc`` Secrets vault, see vault instructions . Use ``OIDC_ATLAS_URI_SINGLE`` for ``MONGODB_URI_SINGLE`` and +Note that typically the preconfigured Atlas Dev clusters are used for testing, in Evergreen and localy. The URIs can be fetched +from the ``drivers/oidc`` Secrets vault, see `vault instructions`_. Use ``OIDC_ATLAS_URI_SINGLE`` for ``MONGODB_URI_SINGLE`` and ``OIDC_ATLAS_URI_MULTI`` for ``OIDC_ATLAS_URI_MULTI``. -If using local servers is preferred, using the ``drivers-evergreen-tools`` Local Testing method , +If using local servers is preferred, using the `Local Testing`_ method, use ``mongodb://localhost/?authMechanism=MONGODB-OIDC`` for ``MONGODB_URI_SINGLE`` and ``mongodb://localhost:27018/?authMechanism=MONGODB-OIDC&directConnection=true&readPreference=secondaryPreferred`` for ``MONGODB_URI_MULTI`` because the other server is a secondary on a replica set, on port ``27018``. -The driver MUST generate valid local tokens at the location given by ``OIDC_TOKEN_DIR``, see ``drivers-evergreen-tools`` for example . -. - -Drivers will need to be able to internally query and clear the cached -credentials to verify usage for testing purposes. Clearing the cache -means removing all data from the cache, including ``OIDCMechanismServerStep1`` -information. - The default OIDC client used in the tests will be configured with ``MONGODB_URI_SINGLE`` and a valid request callback handler -that returns the ``test_user1`` local token. +that returns the ``test_user1`` local token in ``OIDC_TOKEN_DIR`` as the "access_token", and a dummy "refresh_token". -https://wiki.corp.mongodb.com/display/DRIVERS/Using+AWS+Secrets+Manager+to+Store+Testing+Secrets -https://github.com/mongodb-labs/drivers-evergreen-tools/blob/master/.evergreen/auth_oidc/README.md +.. _Local Testing: https://github.com/mongodb-labs/drivers-evergreen-tools/blob/master/.evergreen/auth_oidc/README.md#local-testing +.. _vault instructions: https://wiki.corp.mongodb.com/display/DRIVERS/Using+AWS+Secrets+Manager+to+Store+Testing+Secrets Callback-Driven Auth ==================== @@ -191,6 +183,59 @@ Succeeds - Assert that a ``find`` operation failed once during the command execution. - Close the client. +Succeeds no refresh +~~~~~~~~~~~~~~~~~~~ +- Create a default OIDC client with a request callback that does not return + a refresh token. +- Perform a ``find`` operation that succeeds. +- Assert that the request callback has been called once. +- Force a reauthenication using a ``failCommand`` of the form: + +.. code:: javascript + + { + "configureFailPoint": "failCommand", + "mode": { + "times": 1 + }, + "data": { + "failCommands": [ + "find" + ], + "errorCode": 391 + } + } + +- Perform a ``find`` operation that succeeds. +- Assert that the request callback has been called twice. +- Close the client. + +Succeeds after refresh fails +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +- Create a default OIDC client. +- Perform a ``find`` operation that succeeds. +- Assert that the request callback has been called once. +- Force a reauthenication using a ``failCommand`` of the form: + +.. code:: javascript + + { + "configureFailPoint": "failCommand", + "mode": { + "times": 2 + }, + "data": { + "failCommands": [ + "find", "saslContinue" + ], + "errorCode": 391 + } + } + +- Perform a ``find`` operation that succeeds. +- Assert that the request callback has been called three times. +- Close the client. + Fails ~~~~~ - Create a default OIDC client. From 62130cf33ca0f34daddc7e8c14b4b9e0bb5f56da Mon Sep 17 00:00:00 2001 From: Steven Silvester Date: Tue, 5 Sep 2023 08:21:36 -0500 Subject: [PATCH 03/44] syntax --- source/auth/tests/mongodb-oidc.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/source/auth/tests/mongodb-oidc.rst b/source/auth/tests/mongodb-oidc.rst index ffa3a151dd..9eb5cada24 100644 --- a/source/auth/tests/mongodb-oidc.rst +++ b/source/auth/tests/mongodb-oidc.rst @@ -244,6 +244,7 @@ Fails - Force a reauthenication using a failCommand of the form: .. code:: javascript + { "configureFailPoint": "failCommand", "mode": { From d09ae6c453ed7d19dbbfba6394422d75317c4895 Mon Sep 17 00:00:00 2001 From: Steven Silvester Date: Tue, 5 Sep 2023 12:08:14 -0500 Subject: [PATCH 04/44] Update source/auth/auth.rst Co-authored-by: Anna Henningsen --- source/auth/auth.rst | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/source/auth/auth.rst b/source/auth/auth.rst index b6b9dc6e0f..82420f711b 100644 --- a/source/auth/auth.rst +++ b/source/auth/auth.rst @@ -1181,8 +1181,7 @@ MONGODB-OIDC :since: 7.0 Enterprise MONGODB-OIDC authenticates using an `OIDC `_ access tokens. -Drivers MAY implement the human authentication workflow with ``REQUEST_TOKEN_CALLBACK``, but it is only currently required in the Node driver, -to support usage in the MongoDB Shell. The machine authentication workflow is currently not supported. +Drivers MAY implement the human authentication workflow with ``REQUEST_TOKEN_CALLBACK``. `MongoCredential`_ Properties ````````````````````````````` From 67c9794536b1b8137a7f607775bdedbf8c8e69d6 Mon Sep 17 00:00:00 2001 From: Steven Silvester Date: Tue, 5 Sep 2023 12:08:26 -0500 Subject: [PATCH 05/44] Update source/auth/auth.rst Co-authored-by: Anna Henningsen --- source/auth/auth.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/auth/auth.rst b/source/auth/auth.rst index 82420f711b..6177c41fc2 100644 --- a/source/auth/auth.rst +++ b/source/auth/auth.rst @@ -1295,7 +1295,7 @@ If using the ``JwtStepRequest`` directly, the conversation will look like: | S: :javascript:`{conversationId : 1, payload: BinData(0,"..."), done: true, ok: 1}` Drivers MUST NOT send a ``PrincipalStepRequest`` when performing machine authentication -or when there is a cached Access Token. Drivers must instead use ``saslStart`` with a ``JwtStepRequest``. +or when there is a cached Access Token. Drivers MUST instead use ``saslStart`` with a ``JwtStepRequest``. Caching ``````` From b7f750c36537674e02a4370addffcd44ae81109b Mon Sep 17 00:00:00 2001 From: Steven Silvester Date: Tue, 5 Sep 2023 12:11:21 -0500 Subject: [PATCH 06/44] address review --- source/auth/auth.rst | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/source/auth/auth.rst b/source/auth/auth.rst index 6177c41fc2..dc5aacb826 100644 --- a/source/auth/auth.rst +++ b/source/auth/auth.rst @@ -1205,7 +1205,7 @@ mechanism_properties providing objects as mechanism property values. Otherwise the driver MUST allow it as a MongoClientOption. ALLOWED_HOSTS The list of allowed hostnames or ip-addresses (ignoring ports) for - MongoDB connections. The hostnames may include a leading "*." wildcard, which allows for matching + MongoDB connections. The hostnames may include a leading "\*." wildcard, which allows for matching (potentially nested) subdomains. ALLOWED_HOSTS is a security feature and MUST default to ``["*.mongodb.net", '*.mongodb-dev.net", "*.mongodbgov.net", "localhost", "127.0.0.1", "::1"]``. @@ -1339,6 +1339,9 @@ An example might look like: // Timeout in seconds for the callback. Optionally, timeoutContext instead if applicable to language. timeoutSeconds: int; + // The version of the callback parameter interface. + version: int; + // The refresh token, if applicable, to be used by the callback to request a new token from the issuer. refreshToken?: str; } From a3333535d8c86ae65ba53c6d25a26f27a8daf75d Mon Sep 17 00:00:00 2001 From: Steven Silvester Date: Tue, 19 Sep 2023 21:58:35 -0500 Subject: [PATCH 07/44] Update source/auth/auth.rst Co-authored-by: Matt Dale <9760375+matthewdale@users.noreply.github.com> --- source/auth/auth.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/auth/auth.rst b/source/auth/auth.rst index dc5aacb826..ed9fc5819e 100644 --- a/source/auth/auth.rst +++ b/source/auth/auth.rst @@ -1353,7 +1353,7 @@ An example might look like: Before calling the callback, the driver MUST acquire a lock unique to the ``MongoClient``. The driver MUST ensure that credentials have not changed between when the lock was requested and when it was acquired. The lock MUST be released when the callback call as finished or errored. -This is because the ``REQUEST_TOKEN_CALLBACK`` may involve human interaction, and refresh tokens migh only be able to be used used once. +This is because the ``REQUEST_TOKEN_CALLBACK`` may involve human interaction, and refresh tokens might only be able to be used used once. If the callback does not return an object in the correct form of ``IdpResponse``, the driver MUST raise an error either using the type system or by raising an error when non-optional properties are missing . From aa1ada7814b3d3d34007d4a9f1fb88ebe6f857d9 Mon Sep 17 00:00:00 2001 From: Steven Silvester Date: Tue, 19 Sep 2023 22:05:20 -0500 Subject: [PATCH 08/44] address review --- source/auth/auth.rst | 20 ++++++++++---------- source/auth/tests/mongodb-oidc.rst | 3 ++- 2 files changed, 12 insertions(+), 11 deletions(-) diff --git a/source/auth/auth.rst b/source/auth/auth.rst index ed9fc5819e..c4e0b6be00 100644 --- a/source/auth/auth.rst +++ b/source/auth/auth.rst @@ -1187,7 +1187,7 @@ Drivers MAY implement the human authentication workflow with ``REQUEST_TOKEN_CAL ````````````````````````````` username - MUST NOT be specified in machine authentication. Drivers MUST allow the user to specify this in the human authentication. + MUST NOT be specified in machine authentication. Drivers MUST allow the user to specify this in the human authentication workflow. If a user omits this when multiple OIDC providers are configured, the server will produce an error during authentication. source @@ -1229,7 +1229,7 @@ in the payload as octet sequences defining BSON objects: // Driver’s opening request in saslStart. interface PrincipalStepRequest { // Name of the OIDC user Principal. - n?: str; + n: Optional; } Note that the principal name is optional as it may be provided by the IdP in environments where only one IdP is used. @@ -1242,13 +1242,13 @@ If given, then ``username`` provided by the user MUST be used as the Principal ` // URL which describes the Authentication Server. This identifier should be // the iss of provided access tokens, and be viable for RFC8414 // metadata discovery and RFC9207 identification. - issuer: str; + issuer: string; // Unique client ID for this OIDC client. - clientId: str; + clientId: string; // Additional scopes to request from IdP. - requestScopes?: Array; + requestScopes: Optional>; } The server will use Principal ``(n)`` if provided in the driver’s ``PrincipalStepRequest`` to select an appropriate IdP. @@ -1261,7 +1261,7 @@ This Access Token will be used as the JWT in the driver’s ``JwtStepRequest`` t // Client's request with signed token. interface JwtStepRequest: // Compact serialized JWT with signature. - jwt: str; + jwt: string; } The IdP response that is expected to be returned by the ``REQUEST_TOKEN_CALLBACK`` is as follows: @@ -1271,13 +1271,13 @@ The IdP response that is expected to be returned by the ``REQUEST_TOKEN_CALLBACK // The result of a token request. interface IdPResponse { // The oidc access token. - accessToken: str; + accessToken: string; // The OIDC refresh token. - refreshToken?: str; + refreshToken: Optional; // The expiration time in seconds from the current time (ignored). - expiresInSeconds?: str; + expiresInSeconds: Optional; } Conversation @@ -1343,7 +1343,7 @@ An example might look like: version: int; // The refresh token, if applicable, to be used by the callback to request a new token from the issuer. - refreshToken?: str; + refreshToken: Optional; } .. code: typescript diff --git a/source/auth/tests/mongodb-oidc.rst b/source/auth/tests/mongodb-oidc.rst index 9eb5cada24..50cd4e500c 100644 --- a/source/auth/tests/mongodb-oidc.rst +++ b/source/auth/tests/mongodb-oidc.rst @@ -266,7 +266,8 @@ Separate Connections Avoid Extra Callback Calls ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The following test assumes that the driver will be able to share a cache between two MongoClient objects, or ensure that the same MongoClient is used with two -different connections. If that is not possible, the test may be skipped. +different connections. Otherwise, the test would have a race condition. +If neither is possible, the test may be skipped. - Create a request callback that returns valid, and ensure that we can record the number of times the callback is called. From 93d9b27530f9b62d6277e82f2edaccaf44c60cfc Mon Sep 17 00:00:00 2001 From: Steven Silvester Date: Tue, 19 Sep 2023 22:06:17 -0500 Subject: [PATCH 09/44] add qa --- source/auth/auth.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/auth/auth.rst b/source/auth/auth.rst index c4e0b6be00..46ea9db614 100644 --- a/source/auth/auth.rst +++ b/source/auth/auth.rst @@ -1208,7 +1208,7 @@ mechanism_properties MongoDB connections. The hostnames may include a leading "\*." wildcard, which allows for matching (potentially nested) subdomains. ALLOWED_HOSTS is a security feature and MUST default to - ``["*.mongodb.net", '*.mongodb-dev.net", "*.mongodbgov.net", "localhost", "127.0.0.1", "::1"]``. + ``["*.mongodb.net", '*.mongodb-qa.net", '*.mongodb-dev.net", "*.mongodbgov.net", "localhost", "127.0.0.1", "::1"]``. When ``MONGODB-OIDC`` authentication is attempted against a hostname that does not match any of list of allowed hosts, the driver MUST raise a client-side error without invoking any user-provided From 46afc8da518cb0244b145b2c666c617cb5f68b5e Mon Sep 17 00:00:00 2001 From: Matt Dale <9760375+matthewdale@users.noreply.github.com> Date: Sat, 21 Oct 2023 21:06:42 -0700 Subject: [PATCH 10/44] DRIVERS-2672 Add OIDC machine workflow spec. --- source/auth/auth.rst | 84 ++++++++++++++++++++++++++++++++++++++------ 1 file changed, 74 insertions(+), 10 deletions(-) diff --git a/source/auth/auth.rst b/source/auth/auth.rst index 46ea9db614..fc90cb20ff 100644 --- a/source/auth/auth.rst +++ b/source/auth/auth.rst @@ -1180,15 +1180,26 @@ MONGODB-OIDC :since: 7.0 Enterprise -MONGODB-OIDC authenticates using an `OIDC `_ access tokens. -Drivers MAY implement the human authentication workflow with ``REQUEST_TOKEN_CALLBACK``. +MONGODB-OIDC authenticates using an `OIDC +`_ access tokens. There +are two workflows: + +- Machine authentication workflow +- Human authentication workflow + +Drivers MUST implement the machine authentication workflow with +``CUSTOM_TOKEN_CALLBACK``. Drivers MAY implement the human authentication +workflow with ``REQUEST_TOKEN_CALLBACK``. + `MongoCredential`_ Properties ````````````````````````````` username - MUST NOT be specified in machine authentication. Drivers MUST allow the user to specify this in the human authentication workflow. - If a user omits this when multiple OIDC providers are configured, the server will produce an error during authentication. + MUST NOT be specified in the machine authentication workflow. Drivers MUST + allow the user to specify this in the human authentication workflow. If a + user omits this when multiple OIDC providers are configured, the server will + produce an error during authentication. source MUST be "$external". Defaults to ``$external``. @@ -1200,9 +1211,20 @@ mechanism MUST be "MONGODB-OIDC" mechanism_properties + PROVIDER_NAME + Drivers MUST allow the user to specify the name of the OIDC service to + use to obtain credentials. MUST be one of ["aws", "custom"]. + + CUSTOM_TOKEN_CALLBACK + Drivers MUST allow the user to specify a callback of the form + "getToken" (defined below), if the driver supports providing objects as + mechanism property values. Otherwise the driver MUST allow it as a + MongoClientOption. + REQUEST_TOKEN_CALLBACK Drivers MUST allow the user to specify a callback of the form "onRequest" (defined below), if the driver supports providing objects as mechanism property values. Otherwise the driver MUST allow it as a MongoClientOption. + ALLOWED_HOSTS The list of allowed hostnames or ip-addresses (ignoring ports) for MongoDB connections. The hostnames may include a leading "\*." wildcard, which allows for matching @@ -1314,14 +1336,56 @@ If there is a cached Access Token the ``JwtStepRequest`` SASL command will be us If there is no cached Access Token, the ``PrincipalStepRequest`` will be used as the speculation command. The driver MUST NOT call the callback during speculative authentication. + +CUSTOM_TOKEN_CALLBACK +````````````````````` +Drivers MUST provide a way for the ``CUSTOM_TOKEN_CALLBACK`` to be either +automatically canceled, or to cancel itself. This can be as a timeout argument +to the callback, a cancellation context passed to the callback, or some other +language-appropriate mechanism. The timeout deadline MUST be the same as the +connection establishment deadline (TODO: Does that make sense?). + +Callbacks can be synchronous and/or asynchronous, depending on the driver +and/or language. Asynchronous callbacks should be preferred when other +operations in the driver use asynchronous functions. + +The driver MUST pass the following information to the callback: + +- ``timeoutMS``: A timeout, in milliseconds, deadline, or ``timeoutContext``. +- ``mechanismProperties``: All key-value pairs from the + AUTH_MECHANISM_PROPERTIES parameter. +- ``invalidatedToken``: Optional. The previous token that was invalidated after + receiving a ``ReauthenticationRequired`` error. + +The callback MUST return an OIDC JWT. + +The signature of the callback is up to the driver's discretion, but the driver +MUST ensure that, in the future, callbacks may have additional optional +parameters passed to them. An example might look like: + +.. code: typescript + + interface CallbackParameters { + // All key-value pairs from the AUTH_MECHANISM_PROPERTIES parameter. + mechanismProperties Object; + + // The refresh token, if applicable, to be used by the callback to request a new token from the issuer. + invalidatedToken: Optional; + } + +.. code: typescript + + function getToken(timeoutMS: int, params CallbackParameters): string + REQUEST_TOKEN_CALLBACK `````````````````````` -The driver MUST provide a way for the ``REQUEST_TOKEN_CALLBACK`` to be either automatically -canceled, or to cancel itself. This can be as a timeout argument to the -callback, a cancellation context passed to the callback, or some other -language-appropriate mechanism. The timeout duration MUST be 5 minutes, -to account for the fact that there may be human interaction involved. -This callback is not subject to CSOT. +Drivers that implement ``REQUEST_TOKEN_CALLBACK`` (i.e. the human authentication +workflow) MUST provide a way for the ``REQUEST_TOKEN_CALLBACK`` to be either +automatically canceled, or to cancel itself. This can be as a timeout argument +to the callback, a cancellation context passed to the callback, or some other +language-appropriate mechanism. The timeout duration MUST be 5 minutes, to +account for the fact that there may be human interaction involved. This callback +is not subject to CSOT. Callbacks can be synchronous and/or asynchronous, depending on the driver and/or language. Asynchronous callbacks should be preferred when other From 2c0e07a1f3226e90f4ab992f3af64f199414e2ac Mon Sep 17 00:00:00 2001 From: Matt Dale <9760375+matthewdale@users.noreply.github.com> Date: Thu, 2 Nov 2023 22:37:02 -0700 Subject: [PATCH 11/44] Separate OIDC machine and human auth flow specs. --- source/auth/auth.rst | 200 ++++++++++++++++++++++++++++--------------- 1 file changed, 133 insertions(+), 67 deletions(-) diff --git a/source/auth/auth.rst b/source/auth/auth.rst index fc90cb20ff..5e9f101d15 100644 --- a/source/auth/auth.rst +++ b/source/auth/auth.rst @@ -1175,31 +1175,39 @@ If AWS authentication fails for any reason, the cache MUST be cleared. .. note:: Five minutes was chosen based on the AWS documentation for `IAM roles for EC2 `_ : "We make new credentials available at least five minutes before the expiration of the old credentials". The intent is to have some buffer between when the driver fetches the credentials and when the server verifies them. + MONGODB-OIDC ~~~~~~~~~~~~ :since: 7.0 Enterprise -MONGODB-OIDC authenticates using an `OIDC -`_ access tokens. There -are two workflows: +MONGODB-OIDC authenticates using an `OpenID Connect (OIDC) +`_ access token. Most +OIDC Identity Providers (*IdP*) support two identity federation flows: + +- Workload Identity Federation - Also called the "machine authentication flow", + this flow is intended to authorize access for software and services without + requiring human interaction. + + TODO: Is there a 1:1 corresponding OIDC spec flow? + +- Workforce Identity Federation - Also called the "human authentication flow", + this flow is intended to authorize access for end users and typically requires + human interaction to complete (e.g. clicking a button in a browser). -- Machine authentication workflow -- Human authentication workflow + TODO: Is there a 1:1 corresponding OIDC spec flow? -Drivers MUST implement the machine authentication workflow with -``CUSTOM_TOKEN_CALLBACK``. Drivers MAY implement the human authentication -workflow with ``REQUEST_TOKEN_CALLBACK``. +Drivers MUST implement the Workload Identity Federation flow. Drivers MAY +implement the Workforce Identity Federation flow. +Workload Identity Federation +---------------------------- + `MongoCredential`_ Properties ````````````````````````````` - username - MUST NOT be specified in the machine authentication workflow. Drivers MUST - allow the user to specify this in the human authentication workflow. If a - user omits this when multiple OIDC providers are configured, the server will - produce an error during authentication. + MUST NOT be specified. source MUST be "$external". Defaults to ``$external``. @@ -1213,14 +1221,113 @@ mechanism mechanism_properties PROVIDER_NAME Drivers MUST allow the user to specify the name of the OIDC service to - use to obtain credentials. MUST be one of ["aws", "custom"]. + use to obtain credentials. MUST be one of ["aws"]. + +Interfaces +`````````` + +Custom Callback +``````````````` +Drivers MUST provide a way for the callback to be either +automatically canceled, or to cancel itself. This can be as a timeout argument +to the callback, a cancellation context passed to the callback, or some other +language-appropriate mechanism. The timeout deadline MUST be the same as the +connection establishment deadline (TODO: Does that make sense?). + +Callbacks can be synchronous and/or asynchronous, depending on the driver +and/or language. Asynchronous callbacks should be preferred when other +operations in the driver use asynchronous functions. + +The driver MUST pass the following information to the callback: + +- ``timeoutMS``: A timeout, in milliseconds, deadline, or ``timeoutContext``. +- ``mechanismProperties``: All key-value pairs from the + AUTH_MECHANISM_PROPERTIES parameter. +- ``invalidatedToken``: Optional. The previous token that was invalidated after + receiving a ``ReauthenticationRequired`` error. + +The callback MUST return an OIDC access token, formatted as a base64-encoded +JWT. +TODO: Is that correct? + +The signature of the callback is up to the driver's discretion, but the driver +MUST ensure that additional optional parameters can be added to the callback +signature in the future. An example might look like: + +.. code:: typescript + + interface CallbackParameters { + // All key-value pairs from the AUTH_MECHANISM_PROPERTIES parameter. + mechanismProperties Object; + + // The refresh token, if applicable, to be used by the callback to request a new token from the issuer. + invalidatedToken: Optional; + } + +.. code:: typescript + + function workloadToken(timeoutMS: int, params CallbackParameters): string + +Conversation +```````````` +The SASL conversation uses a ``JwtStepRequest``. An example looks like: + +| C: :javascript:`{saslStart: 1, mechanism: "MONGODB-OIDC", payload: BinData(0, BSON({"jwt": ""}))}` +| S: :javascript:`{conversationId : 1, payload: BinData(0, "..."), done: true, ok: 1}` + +Access Token Caching +``````` +Drivers MUST cache the access token returned either by a built-in or custom +workload callback on the connection object (TODO: on the ``MongoClient``?). If +any operation fails with ``ReauthenticationRequired`` (391), the driver MUST +clear the cached access token and reauthenticate the connection. + +TODO: Is token generation ID required? What is it used for? + +Speculative Authentication +`````````````````````````` +Drivers MUST implement speculative authentication for MONGODB-OIDC during the +``hello`` handshake. Drivers MUST call the configured workload callback to +retrieve a new access token for the new connection and send that access token +with the ``JwtStepRequest`` SASL command. + +TODO: What if we cache on the ``MongoClient`` instead? + +Reauthentication +```````````````` +When reauthentication is requested by the server (as a 391 error code) and +MONGODB-OIDC is in use, the driver MUST perform a reauthentication. The +following algorithm is used to handle a reauthenication error: + +#. Clear the cached JWT from the connection. +#. Call the configured workload callback to retrieve a new JWT. +#. Cache the new JWT on the connection. +#. Attempt to authenticate the connection. + + +Workforce Identity Federation +----------------------------- +See more information about workforce identity federation on Atlas here: +https://www.mongodb.com/docs/atlas/security-oidc/ + +`MongoCredential`_ Properties +````````````````````````````` + +username + Drivers MUST allow the user to specify this in the workforce identity + federation flow. If a user omits this when multiple OIDC providers are + configured, the server will produce an error during authentication. + +source + MUST be "$external". Defaults to ``$external``. + +password + MUST NOT be specified. - CUSTOM_TOKEN_CALLBACK - Drivers MUST allow the user to specify a callback of the form - "getToken" (defined below), if the driver supports providing objects as - mechanism property values. Otherwise the driver MUST allow it as a - MongoClientOption. +mechanism + MUST be "MONGODB-OIDC" +mechanism_properties REQUEST_TOKEN_CALLBACK Drivers MUST allow the user to specify a callback of the form "onRequest" (defined below), if the driver supports providing objects as mechanism property values. Otherwise the driver MUST allow it as a MongoClientOption. @@ -1241,14 +1348,13 @@ mechanism_properties Interfaces `````````` - Authenticating using the MONGODB-OIDC mechanism will require 1 or 2 round trips between the MongoDB driver and server. The requests from the driver and the replies from the server are described by the following interfaces which are encoded in the payload as octet sequences defining BSON objects: .. code:: typescript - // Driver’s opening request in saslStart. + // Driver's opening request in saslStart. interface PrincipalStepRequest { // Name of the OIDC user Principal. n: Optional; @@ -1273,10 +1379,10 @@ If given, then ``username`` provided by the user MUST be used as the Principal ` requestScopes: Optional>; } -The server will use Principal ``(n)`` if provided in the driver’s ``PrincipalStepRequest`` to select an appropriate IdP. -This IdP's configuration will be returned in the server’s response that will be used by the end-user to acquire an Access Token. +The server will use Principal ``(n)`` if provided in the driver's ``PrincipalStepRequest`` to select an appropriate IdP. +This IdP's configuration will be returned in the server's response that will be used by the end-user to acquire an Access Token. -This Access Token will be used as the JWT in the driver’s ``JwtStepRequest`` to complete authentication. +This Access Token will be used as the JWT in the driver's ``JwtStepRequest`` to complete authentication. .. code:: typescript @@ -1322,7 +1428,7 @@ or when there is a cached Access Token. Drivers MUST instead use ``saslStart`` Caching ``````` Drivers MUST cache the ``IdPInfo``, Access Token, and Refresh Token associated with each -``MongoClient``. If any operation fails the driver MUST clear the Access Token. The Refresh Token is handled differently, +``MongoClient``. If any operation fails the driver MUST clear the Access Token. The Refresh Token is handled differently, see the ``Reauthentication`` section below. Drivers MUST also use a token generation id that is incremented whenever a new Access Token is retrieved. @@ -1336,51 +1442,10 @@ If there is a cached Access Token the ``JwtStepRequest`` SASL command will be us If there is no cached Access Token, the ``PrincipalStepRequest`` will be used as the speculation command. The driver MUST NOT call the callback during speculative authentication. - -CUSTOM_TOKEN_CALLBACK -````````````````````` -Drivers MUST provide a way for the ``CUSTOM_TOKEN_CALLBACK`` to be either -automatically canceled, or to cancel itself. This can be as a timeout argument -to the callback, a cancellation context passed to the callback, or some other -language-appropriate mechanism. The timeout deadline MUST be the same as the -connection establishment deadline (TODO: Does that make sense?). - -Callbacks can be synchronous and/or asynchronous, depending on the driver -and/or language. Asynchronous callbacks should be preferred when other -operations in the driver use asynchronous functions. - -The driver MUST pass the following information to the callback: - -- ``timeoutMS``: A timeout, in milliseconds, deadline, or ``timeoutContext``. -- ``mechanismProperties``: All key-value pairs from the - AUTH_MECHANISM_PROPERTIES parameter. -- ``invalidatedToken``: Optional. The previous token that was invalidated after - receiving a ``ReauthenticationRequired`` error. - -The callback MUST return an OIDC JWT. - -The signature of the callback is up to the driver's discretion, but the driver -MUST ensure that, in the future, callbacks may have additional optional -parameters passed to them. An example might look like: - -.. code: typescript - - interface CallbackParameters { - // All key-value pairs from the AUTH_MECHANISM_PROPERTIES parameter. - mechanismProperties Object; - - // The refresh token, if applicable, to be used by the callback to request a new token from the issuer. - invalidatedToken: Optional; - } - -.. code: typescript - - function getToken(timeoutMS: int, params CallbackParameters): string - REQUEST_TOKEN_CALLBACK `````````````````````` Drivers that implement ``REQUEST_TOKEN_CALLBACK`` (i.e. the human authentication -workflow) MUST provide a way for the ``REQUEST_TOKEN_CALLBACK`` to be either +flow) MUST provide a way for the ``REQUEST_TOKEN_CALLBACK`` to be either automatically canceled, or to cancel itself. This can be as a timeout argument to the callback, a cancellation context passed to the callback, or some other language-appropriate mechanism. The timeout duration MUST be 5 minutes, to @@ -1677,6 +1742,7 @@ Q: Should drivers support accessing Amazon EC2 instance metadata in Amazon ECS? Changelog ========= +:2023-11-01: Separated MONGODB-OIDC machine and human authentication flow specs. :2023-04-28: Added MONGODB-OIDC auth mechanism :2022-11-02: Require environment variables to be read dynamically. :2022-10-28: Recommend the use of AWS SDKs where available. From becadd503b6a658260a85d6e89e74479949e52e3 Mon Sep 17 00:00:00 2001 From: Matt Dale <9760375+matthewdale@users.noreply.github.com> Date: Fri, 3 Nov 2023 12:31:07 -0700 Subject: [PATCH 12/44] Apply suggestions from code review Co-authored-by: Anna Henningsen --- source/auth/auth.rst | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/source/auth/auth.rst b/source/auth/auth.rst index 5e9f101d15..ce9dfe4531 100644 --- a/source/auth/auth.rst +++ b/source/auth/auth.rst @@ -1246,9 +1246,7 @@ The driver MUST pass the following information to the callback: - ``invalidatedToken``: Optional. The previous token that was invalidated after receiving a ``ReauthenticationRequired`` error. -The callback MUST return an OIDC access token, formatted as a base64-encoded -JWT. -TODO: Is that correct? +The callback MUST return an OIDC access token in JWT format. The signature of the callback is up to the driver's discretion, but the driver MUST ensure that additional optional parameters can be added to the callback @@ -1258,7 +1256,7 @@ signature in the future. An example might look like: interface CallbackParameters { // All key-value pairs from the AUTH_MECHANISM_PROPERTIES parameter. - mechanismProperties Object; + mechanismProperties: Object; // The refresh token, if applicable, to be used by the callback to request a new token from the issuer. invalidatedToken: Optional; From 5b0665e4734da398d317d555f543d12b33abe93b Mon Sep 17 00:00:00 2001 From: Matt Dale <9760375+matthewdale@users.noreply.github.com> Date: Sun, 5 Nov 2023 20:53:08 -0800 Subject: [PATCH 13/44] Remove references to GCP OIDC provider flows, fix heading formatting, correct cache behavior. --- source/auth/auth.rst | 115 +++++++++++++++++++++++-------------------- 1 file changed, 62 insertions(+), 53 deletions(-) diff --git a/source/auth/auth.rst b/source/auth/auth.rst index ce9dfe4531..1a20ee5f6c 100644 --- a/source/auth/auth.rst +++ b/source/auth/auth.rst @@ -1182,30 +1182,13 @@ MONGODB-OIDC :since: 7.0 Enterprise MONGODB-OIDC authenticates using an `OpenID Connect (OIDC) -`_ access token. Most -OIDC Identity Providers (*IdP*) support two identity federation flows: +`_ access token. -- Workload Identity Federation - Also called the "machine authentication flow", - this flow is intended to authorize access for software and services without - requiring human interaction. - - TODO: Is there a 1:1 corresponding OIDC spec flow? - -- Workforce Identity Federation - Also called the "human authentication flow", - this flow is intended to authorize access for end users and typically requires - human interaction to complete (e.g. clicking a button in a browser). - - TODO: Is there a 1:1 corresponding OIDC spec flow? - -Drivers MUST implement the Workload Identity Federation flow. Drivers MAY -implement the Workforce Identity Federation flow. - - -Workload Identity Federation ----------------------------- +TODO: Add background on what part of the auth flow drivers play specifically. `MongoCredential`_ Properties ````````````````````````````` + username MUST NOT be specified. @@ -1221,18 +1204,27 @@ mechanism mechanism_properties PROVIDER_NAME Drivers MUST allow the user to specify the name of the OIDC service to - use to obtain credentials. MUST be one of ["aws"]. + use to obtain credentials. MUST be one of ["aws"]. If PROVIDER_NAME is + given and a Custom provider callback is configured, the driver MUST + raise an error. -Interfaces -`````````` +Supported Service Providers +``````````````````````````` -Custom Callback -``````````````` -Drivers MUST provide a way for the callback to be either -automatically canceled, or to cancel itself. This can be as a timeout argument -to the callback, a cancellation context passed to the callback, or some other -language-appropriate mechanism. The timeout deadline MUST be the same as the -connection establishment deadline (TODO: Does that make sense?). +AWS +___ + +TODO: Add. + +Custom +______ + +Drivers MUST allow the user to integrate with other OIDC providers by +implementing a custom callback that returns an OIDC token. Drivers MUST provide +a way for the callback to be either automatically canceled, or to cancel itself. +This can be as a timeout argument to the callback, a cancellation context passed +to the callback, or some other language-appropriate mechanism. The timeout +deadline MUST be the same as the connection establishment deadline. Callbacks can be synchronous and/or asynchronous, depending on the driver and/or language. Asynchronous callbacks should be preferred when other @@ -1264,52 +1256,60 @@ signature in the future. An example might look like: .. code:: typescript - function workloadToken(timeoutMS: int, params CallbackParameters): string + function oidcToken(timeoutMS: int, params CallbackParameters): string Conversation ```````````` -The SASL conversation uses a ``JwtStepRequest``. An example looks like: -| C: :javascript:`{saslStart: 1, mechanism: "MONGODB-OIDC", payload: BinData(0, BSON({"jwt": ""}))}` +As an example, given the OIDC token JWT string "abcd1234", the SASL conversation looks like: + +| C: :javascript:`{saslStart: 1, mechanism: "MONGODB-OIDC", payload: BinData(0, BSON({"jwt": "abcd1234"}))}` | S: :javascript:`{conversationId : 1, payload: BinData(0, "..."), done: true, ok: 1}` +Where the sent ``payload`` is a generic binary blob containing a BSON-encoded document ``{"jwt": "abcd1234"}``. + Access Token Caching -``````` -Drivers MUST cache the access token returned either by a built-in or custom -workload callback on the connection object (TODO: on the ``MongoClient``?). If -any operation fails with ``ReauthenticationRequired`` (391), the driver MUST -clear the cached access token and reauthenticate the connection. +```````````````````` -TODO: Is token generation ID required? What is it used for? +Drivers MUST cache the Access Token used to authenticate a connection on the +connection object. Additionally, drivers MUST cache the most recent access Token +on the ``MongoClient``. If any operation fails with ``ReauthenticationRequired`` +(391), the driver MUST clear the access Token cached on the connection object +and and reauthenticate the connection as described in ``Reauthentication`` +section below. Speculative Authentication `````````````````````````` -Drivers MUST implement speculative authentication for MONGODB-OIDC during the -``hello`` handshake. Drivers MUST call the configured workload callback to -retrieve a new access token for the new connection and send that access token -with the ``JwtStepRequest`` SASL command. -TODO: What if we cache on the ``MongoClient`` instead? +Drivers MUST implement speculative authentication for MONGODB-OIDC during the +``hello`` handshake. If there is a cached Access Token on the ``MongoClient``, +use it for authentication. Otherwise, call the configured workload callback to +retrieve a new Access Token for the new connection and send that Access Token +with the ``saslStart`` SASL command. Reauthentication ```````````````` + When reauthentication is requested by the server (as a 391 error code) and MONGODB-OIDC is in use, the driver MUST perform a reauthentication. The following algorithm is used to handle a reauthenication error: -#. Clear the cached JWT from the connection. -#. Call the configured workload callback to retrieve a new JWT. -#. Cache the new JWT on the connection. -#. Attempt to authenticate the connection. +- First, see if the Access Token on the ``MongoClient`` is different than the Access Token on the connection object. + - If they are different, optimisitically try to authenticate using the Access Token from the ``MongoClient``. On error, continue. +- Call the Access Token function for the configured provider (or the custom provider callback). +- Cache the returned Access Token on the current connection object and on the ``MongoClient``. +- Attempt to authenticate. Raise any errors to the user. +TODO: Attempt authentication first or cache new token first? + +Human Authentication Flow +````````````````````````` + +TODO: Why is this section separate? -Workforce Identity Federation ------------------------------ -See more information about workforce identity federation on Atlas here: -https://www.mongodb.com/docs/atlas/security-oidc/ `MongoCredential`_ Properties -````````````````````````````` +_____________________________ username Drivers MUST allow the user to specify this in the workforce identity @@ -1326,6 +1326,9 @@ mechanism MUST be "MONGODB-OIDC" mechanism_properties + PROVIDER_NAME + MUST NOT be specified. + REQUEST_TOKEN_CALLBACK Drivers MUST allow the user to specify a callback of the form "onRequest" (defined below), if the driver supports providing objects as mechanism property values. Otherwise the driver MUST allow it as a MongoClientOption. @@ -1346,6 +1349,7 @@ mechanism_properties Interfaces `````````` + Authenticating using the MONGODB-OIDC mechanism will require 1 or 2 round trips between the MongoDB driver and server. The requests from the driver and the replies from the server are described by the following interfaces which are encoded in the payload as octet sequences defining BSON objects: @@ -1408,6 +1412,7 @@ The IdP response that is expected to be returned by the ``REQUEST_TOKEN_CALLBACK Conversation ```````````` + If using the ``PrincipalStepRequest`` first, the conversation will look like: | C: :javascript:`{saslStart: 1, mechanism: "MONGODB-OIDC", payload: BinData(0, "...")}` @@ -1425,6 +1430,7 @@ or when there is a cached Access Token. Drivers MUST instead use ``saslStart`` Caching ``````` + Drivers MUST cache the ``IdPInfo``, Access Token, and Refresh Token associated with each ``MongoClient``. If any operation fails the driver MUST clear the Access Token. The Refresh Token is handled differently, see the ``Reauthentication`` section below. @@ -1435,6 +1441,7 @@ This value is used in the ``Reauthentication`` section below. Speculative Authentication `````````````````````````` + Drivers MUST implement speculative authentication for MONGODB-OIDC during the ``hello`` handshake. If there is a cached Access Token the ``JwtStepRequest`` SASL command will be used as the speculation command. If there is no cached Access Token, the ``PrincipalStepRequest`` will be used as the speculation command. @@ -1442,6 +1449,7 @@ The driver MUST NOT call the callback during speculative authentication. REQUEST_TOKEN_CALLBACK `````````````````````` + Drivers that implement ``REQUEST_TOKEN_CALLBACK`` (i.e. the human authentication flow) MUST provide a way for the ``REQUEST_TOKEN_CALLBACK`` to be either automatically canceled, or to cancel itself. This can be as a timeout argument @@ -1492,6 +1500,7 @@ If no callback is given, the driver MUST must raise a validation error. Reauthentication ```````````````` + When reauthentication is requested by the server (as a 391 error code) and MONGODB-OIDC is in use, the driver MUST perform a reauthentication. The driver MUST account for the case of multiple connections hitting a reauthentication error at different times. The driver MUST also account for a reauthenication that results from an IdP configuration change on the server. From 88de5b9009bf0b88435bacfb2dcfc2b2a08e6840 Mon Sep 17 00:00:00 2001 From: Matt Dale <9760375+matthewdale@users.noreply.github.com> Date: Thu, 9 Nov 2023 12:59:47 -0800 Subject: [PATCH 14/44] Update OIDC token callback signature and make the OIDC conversation more realistic. --- source/auth/auth.rst | 30 +++++++++++++++--------------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/source/auth/auth.rst b/source/auth/auth.rst index 1a20ee5f6c..dfcd1ed73f 100644 --- a/source/auth/auth.rst +++ b/source/auth/auth.rst @@ -1233,12 +1233,9 @@ operations in the driver use asynchronous functions. The driver MUST pass the following information to the callback: - ``timeoutMS``: A timeout, in milliseconds, deadline, or ``timeoutContext``. -- ``mechanismProperties``: All key-value pairs from the - AUTH_MECHANISM_PROPERTIES parameter. -- ``invalidatedToken``: Optional. The previous token that was invalidated after - receiving a ``ReauthenticationRequired`` error. -The callback MUST return an OIDC access token in JWT format. +The callback MUST return an OIDC access token in JWT format and, if available, +the expiry timestamp for that access token. The signature of the callback is up to the driver's discretion, but the driver MUST ensure that additional optional parameters can be added to the callback @@ -1246,27 +1243,30 @@ signature in the future. An example might look like: .. code:: typescript - interface CallbackParameters { - // All key-value pairs from the AUTH_MECHANISM_PROPERTIES parameter. - mechanismProperties: Object; + interface TokenParameters { + timeoutMS: int; + } - // The refresh token, if applicable, to be used by the callback to request a new token from the issuer. - invalidatedToken: Optional; + interface TokenResult { + accessToken: string; + expiresInSeconds: Optional; } .. code:: typescript - function oidcToken(timeoutMS: int, params CallbackParameters): string + function token(params: TokenParameters): TokenResult Conversation ```````````` -As an example, given the OIDC token JWT string "abcd1234", the SASL conversation looks like: +As an example, given the OIDC token JWT string "abcd1234", the SASL conversation +looks like: -| C: :javascript:`{saslStart: 1, mechanism: "MONGODB-OIDC", payload: BinData(0, BSON({"jwt": "abcd1234"}))}` -| S: :javascript:`{conversationId : 1, payload: BinData(0, "..."), done: true, ok: 1}` +| C: :javascript:`{saslStart: 1, mechanism: "MONGODB-OIDC", payload: BinData(0, "FwAAAAJqd3QACQAAAGFiY2QxMjM0AAA=")}` +| S: :javascript:`{conversationId : 1, payload: BinData(0, ""), done: true, ok: 1}` -Where the sent ``payload`` is a generic binary blob containing a BSON-encoded document ``{"jwt": "abcd1234"}``. +Where the sent ``payload`` is a generic binary blob containing a BSON document +in the form ``{"jwt": "abcd1234"}``. Access Token Caching ```````````````````` From e9972de4e24b5b39cffdf59792d400e3de265619 Mon Sep 17 00:00:00 2001 From: Matt Dale <9760375+matthewdale@users.noreply.github.com> Date: Mon, 16 Oct 2023 16:17:30 -0700 Subject: [PATCH 15/44] DRIVERS-2672 Update unified test format to support OIDC tests. --- source/unified-test-format/schema-1.18.json | 713 ++++++++++++++++++ .../unified-test-format.rst | 26 +- 2 files changed, 736 insertions(+), 3 deletions(-) create mode 100644 source/unified-test-format/schema-1.18.json diff --git a/source/unified-test-format/schema-1.18.json b/source/unified-test-format/schema-1.18.json new file mode 100644 index 0000000000..c2e7cca337 --- /dev/null +++ b/source/unified-test-format/schema-1.18.json @@ -0,0 +1,713 @@ +{ + "$schema": "http://json-schema.org/draft-07/schema#", + + "title": "Unified Test Format", + "type": "object", + "additionalProperties": false, + "required": ["description", "schemaVersion", "tests"], + "properties": { + "description": { "type": "string" }, + "schemaVersion": { "$ref": "#/definitions/version" }, + "runOnRequirements": { + "type": "array", + "minItems": 1, + "items": { "$ref": "#/definitions/runOnRequirement" } + }, + "createEntities": { + "type": "array", + "minItems": 1, + "items": { "$ref": "#/definitions/entity" } + }, + "initialData": { + "type": "array", + "minItems": 1, + "items": { "$ref": "#/definitions/collectionData" } + }, + "tests": { + "type": "array", + "minItems": 1, + "items": { "$ref": "#/definitions/test" } + }, + "_yamlAnchors": { + "type": "object", + "additionalProperties": true + } + }, + + "definitions": { + "version": { + "type": "string", + "pattern": "^[0-9]+(\\.[0-9]+){1,2}$" + }, + + "runOnRequirement": { + "type": "object", + "additionalProperties": false, + "minProperties": 1, + "properties": { + "maxServerVersion": { "$ref": "#/definitions/version" }, + "minServerVersion": { "$ref": "#/definitions/version" }, + "topologies": { + "type": "array", + "minItems": 1, + "items": { + "type": "string", + "enum": ["single", "replicaset", "sharded", "sharded-replicaset", "load-balanced"] + } + }, + "serverless": { + "type": "string", + "enum": ["require", "forbid", "allow"] + }, + "serverParameters": { + "type": "object", + "minProperties": 1 + }, + "auth": { "type": "boolean" }, + "authMechanism": { "type": "string" }, + "csfle": { "type": "boolean" } + } + }, + + "entity": { + "type": "object", + "additionalProperties": false, + "maxProperties": 1, + "minProperties": 1, + "properties": { + "client": { + "type": "object", + "additionalProperties": false, + "required": ["id"], + "properties": { + "id": { "type": "string" }, + "uriOptions": { "type": "object" }, + "useMultipleMongoses": { "type": "boolean" }, + "observeEvents": { + "type": "array", + "minItems": 1, + "items": { + "type": "string", + "enum": [ + "commandStartedEvent", + "commandSucceededEvent", + "commandFailedEvent", + "poolCreatedEvent", + "poolReadyEvent", + "poolClearedEvent", + "poolClosedEvent", + "connectionCreatedEvent", + "connectionReadyEvent", + "connectionClosedEvent", + "connectionCheckOutStartedEvent", + "connectionCheckOutFailedEvent", + "connectionCheckedOutEvent", + "connectionCheckedInEvent", + "serverDescriptionChangedEvent", + "topologyDescriptionChangedEvent" + ] + } + }, + "ignoreCommandMonitoringEvents": { + "type": "array", + "minItems": 1, + "items": { "type": "string" } + }, + "storeEventsAsEntities": { + "type": "array", + "minItems": 1, + "items": { "$ref": "#/definitions/storeEventsAsEntity" } + }, + "observeLogMessages": { + "type": "object", + "minProperties": 1, + "additionalProperties": false, + "properties": { + "command": { "$ref": "#/definitions/logSeverityLevel" }, + "topology": { "$ref": "#/definitions/logSeverityLevel" }, + "serverSelection": { "$ref": "#/definitions/logSeverityLevel" }, + "connection": { "$ref": "#/definitions/logSeverityLevel" } + } + }, + "serverApi": { "$ref": "#/definitions/serverApi" }, + "observeSensitiveCommands": { "type": "boolean" } + } + }, + "clientEncryption": { + "type": "object", + "additionalProperties": false, + "required": ["id", "clientEncryptionOpts"], + "properties": { + "id": { "type": "string" }, + "clientEncryptionOpts": { "$ref": "#/definitions/clientEncryptionOpts" } + } + }, + "database": { + "type": "object", + "additionalProperties": false, + "required": ["id", "client", "databaseName"], + "properties": { + "id": { "type": "string" }, + "client": { "type": "string" }, + "databaseName": { "type": "string" }, + "databaseOptions": { "$ref": "#/definitions/collectionOrDatabaseOptions" } + } + }, + "collection": { + "type": "object", + "additionalProperties": false, + "required": ["id", "database", "collectionName"], + "properties": { + "id": { "type": "string" }, + "database": { "type": "string" }, + "collectionName": { "type": "string" }, + "collectionOptions": { "$ref": "#/definitions/collectionOrDatabaseOptions" } + } + }, + "session": { + "type": "object", + "additionalProperties": false, + "required": ["id", "client"], + "properties": { + "id": { "type": "string" }, + "client": { "type": "string" }, + "sessionOptions": { "type": "object" } + } + }, + "bucket": { + "type": "object", + "additionalProperties": false, + "required": ["id", "database"], + "properties": { + "id": { "type": "string" }, + "database": { "type": "string" }, + "bucketOptions": { "type": "object" } + } + }, + "thread": { + "type": "object", + "additionalProperties": false, + "required": ["id"], + "properties": { + "id": { "type": "string" } + } + } + } + }, + + "logComponent": { + "type": "string", + "enum": ["command", "topology", "serverSelection", "connection"] + }, + + "logSeverityLevel": { + "type": "string", + "enum": ["emergency", "alert", "critical", "error", "warning", "notice", "info", "debug", "trace"] + }, + + "clientEncryptionOpts": { + "type": "object", + "additionalProperties": false, + "required": ["keyVaultClient", "keyVaultNamespace", "kmsProviders"], + "properties": { + "keyVaultClient": { "type": "string" }, + "keyVaultNamespace": { "type": "string" }, + "kmsProviders": { "$ref": "#/definitions/kmsProviders" } + } + }, + + "kmsProviders": { + "$defs": { + "stringOrPlaceholder": { + "oneOf": [ + { + "type": "string" + }, + { + "type": "object", + "additionalProperties": false, + "required": ["$$placeholder"], + "properties": { + "$$placeholder": {} + } + } + ] + } + }, + "type": "object", + "additionalProperties": false, + "properties": { + "aws": { + "type": "object", + "additionalProperties": false, + "properties": { + "accessKeyId": { "$ref": "#/definitions/kmsProviders/$defs/stringOrPlaceholder" }, + "secretAccessKey": { "$ref": "#/definitions/kmsProviders/$defs/stringOrPlaceholder" }, + "sessionToken": { "$ref": "#/definitions/kmsProviders/$defs/stringOrPlaceholder" } + } + }, + "azure": { + "type": "object", + "additionalProperties": false, + "properties": { + "tenantId": { "$ref": "#/definitions/kmsProviders/$defs/stringOrPlaceholder" }, + "clientId": { "$ref": "#/definitions/kmsProviders/$defs/stringOrPlaceholder" }, + "clientSecret": { "$ref": "#/definitions/kmsProviders/$defs/stringOrPlaceholder" }, + "identityPlatformEndpoint": { "$ref": "#/definitions/kmsProviders/$defs/stringOrPlaceholder" } + } + }, + "gcp": { + "type": "object", + "additionalProperties": false, + "properties": { + "email": { "$ref": "#/definitions/kmsProviders/$defs/stringOrPlaceholder" }, + "privateKey": { "$ref": "#/definitions/kmsProviders/$defs/stringOrPlaceholder" }, + "endpoint": { "$ref": "#/definitions/kmsProviders/$defs/stringOrPlaceholder" } + } + }, + "kmip": { + "type": "object", + "additionalProperties": false, + "properties": { + "endpoint": { "$ref": "#/definitions/kmsProviders/$defs/stringOrPlaceholder" } + } + }, + "local": { + "type": "object", + "additionalProperties": false, + "properties": { + "key": { "$ref": "#/definitions/kmsProviders/$defs/stringOrPlaceholder" } + } + } + } + }, + + "storeEventsAsEntity": { + "type": "object", + "additionalProperties": false, + "required": ["id", "events"], + "properties": { + "id": { "type": "string" }, + "events": { + "type": "array", + "minItems": 1, + "items": { + "type": "string", + "enum": [ + "PoolCreatedEvent", + "PoolReadyEvent", + "PoolClearedEvent", + "PoolClosedEvent", + "ConnectionCreatedEvent", + "ConnectionReadyEvent", + "ConnectionClosedEvent", + "ConnectionCheckOutStartedEvent", + "ConnectionCheckOutFailedEvent", + "ConnectionCheckedOutEvent", + "ConnectionCheckedInEvent", + "CommandStartedEvent", + "CommandSucceededEvent", + "CommandFailedEvent", + "ServerDescriptionChangedEvent", + "TopologyDescriptionChangedEvent" + ] + } + } + } + }, + + "collectionData": { + "type": "object", + "additionalProperties": false, + "required": ["collectionName", "databaseName", "documents"], + "properties": { + "collectionName": { "type": "string" }, + "databaseName": { "type": "string" }, + "createOptions": { + "type": "object", + "properties": { + "writeConcern": false + } + }, + "documents": { + "type": "array", + "items": { "type": "object" } + } + } + }, + + "expectedEventsForClient": { + "type": "object", + "additionalProperties": false, + "required": ["client", "events"], + "properties": { + "client": { "type": "string" }, + "eventType": { + "type": "string", + "enum": ["command", "cmap", "sdam"] + }, + "events": { "type": "array" }, + "ignoreExtraEvents": { "type": "boolean" } + }, + "oneOf": [ + { + "required": ["eventType"], + "properties": { + "eventType": { "const": "command" }, + "events": { + "type": "array", + "items": { "$ref": "#/definitions/expectedCommandEvent" } + } + } + }, + { + "required": ["eventType"], + "properties": { + "eventType": { "const": "cmap" }, + "events": { + "type": "array", + "items": { "$ref": "#/definitions/expectedCmapEvent" } + } + } + }, + { + "required": ["eventType"], + "properties": { + "eventType": { "const": "sdam" }, + "events": { + "type": "array", + "items": { "$ref": "#/definitions/expectedSdamEvent" } + } + } + }, + { + "additionalProperties": false, + "properties": { + "client": { "type": "string" }, + "events": { + "type": "array", + "items": { "$ref": "#/definitions/expectedCommandEvent" } + }, + "ignoreExtraEvents": { "type": "boolean" } + } + } + ] + }, + + "expectedCommandEvent": { + "type": "object", + "additionalProperties": false, + "maxProperties": 1, + "minProperties": 1, + "properties": { + "commandStartedEvent": { + "type": "object", + "additionalProperties": false, + "properties": { + "command": { "type": "object" }, + "commandName": { "type": "string" }, + "databaseName": { "type": "string" }, + "hasServiceId": { "type": "boolean" }, + "hasServerConnectionId": { "type": "boolean" } + } + }, + "commandSucceededEvent": { + "type": "object", + "additionalProperties": false, + "properties": { + "reply": { "type": "object" }, + "commandName": { "type": "string" }, + "databaseName": { "type": "string" }, + "hasServiceId": { "type": "boolean" }, + "hasServerConnectionId": { "type": "boolean" } + } + }, + "commandFailedEvent": { + "type": "object", + "additionalProperties": false, + "properties": { + "commandName": { "type": "string" }, + "databaseName": { "type": "string" }, + "hasServiceId": { "type": "boolean" }, + "hasServerConnectionId": { "type": "boolean" } + } + } + } + }, + + "expectedCmapEvent": { + "type": "object", + "additionalProperties": false, + "maxProperties": 1, + "minProperties": 1, + "properties": { + "poolCreatedEvent": { + "type": "object", + "additionalProperties": false, + "properties": {} + }, + "poolReadyEvent": { + "type": "object", + "additionalProperties": false, + "properties": {} + }, + "poolClearedEvent": { + "type": "object", + "additionalProperties": false, + "properties": { + "hasServiceId": { "type": "boolean" }, + "interruptInUseConnections": { "type": "boolean" } + } + }, + "poolClosedEvent": { + "type": "object", + "additionalProperties": false, + "properties": {} + }, + "connectionCreatedEvent": { + "type": "object", + "additionalProperties": false, + "properties": {} + }, + "connectionReadyEvent": { + "type": "object", + "additionalProperties": false, + "properties": {} + }, + "connectionClosedEvent": { + "type": "object", + "additionalProperties": false, + "properties": { + "reason": { "type": "string" } + } + }, + "connectionCheckOutStartedEvent": { + "type": "object", + "additionalProperties": false, + "properties": {} + }, + "connectionCheckOutFailedEvent": { + "type": "object", + "additionalProperties": false, + "properties": { + "reason": { "type": "string" } + } + }, + "connectionCheckedOutEvent": { + "type": "object", + "additionalProperties": false, + "properties": {} + }, + "connectionCheckedInEvent": { + "type": "object", + "additionalProperties": false, + "properties": {} + } + } + }, + + "expectedSdamEvent": { + "type": "object", + "additionalProperties": false, + "maxProperties": 1, + "minProperties": 1, + "properties": { + "serverDescriptionChangedEvent": { + "type": "object", + "additionalProperties": false, + "properties": { + "previousDescription": { "$ref": "#/definitions/serverDescription" }, + "newDescription": { "$ref": "#/definitions/serverDescription" } + } + }, + "topologyDescriptionChangedEvent": { + "type": "object", + "additionalProperties": false, + "properties": {} + }, + "serverHeartbeatStartedEvent": { + "type": "object", + "additionalProperties": false, + "properties": { + "awaited": { "type": "boolean" } + } + }, + "serverHeartbeatSucceededEvent": { + "type": "object", + "additionalProperties": false, + "properties": { + "awaited": { "type": "boolean" } + } + }, + "serverHeartbeatFailedEvent": { + "type": "object", + "additionalProperties": false, + "properties": { + "awaited": { "type": "boolean" } + } + } + } + }, + + "serverDescription": { + "type": "object", + "additionalProperties": false, + "properties": { + "type": { + "type": "string", + "enum": [ + "Standalone", + "Mongos", + "PossiblePrimary", + "RSPrimary", + "RSSecondary", + "RSOther", + "RSArbiter", + "RSGhost", + "LoadBalancer", + "Unknown" + ] + } + } + }, + + "expectedLogMessagesForClient": { + "type": "object", + "additionalProperties": false, + "required": ["client", "messages"], + "properties": { + "client": { "type": "string" }, + "messages": { + "type": "array", + "items": { "$ref": "#/definitions/expectedLogMessage" } + }, + "ignoreExtraMessages": { "type": "boolean" }, + "ignoreMessages": { + "type": "array", + "items": { "$ref": "#/definitions/expectedLogMessage" } + } + } + }, + + "expectedLogMessage": { + "type": "object", + "additionalProperties": false, + "required": ["level", "component", "data"], + "properties": { + "level": { "$ref": "#/definitions/logSeverityLevel" }, + "component": { "$ref": "#/definitions/logComponent" }, + "data": { "type": "object" }, + "failureIsRedacted": { "type": "boolean" } + } + }, + + "collectionOrDatabaseOptions": { + "type": "object", + "additionalProperties": false, + "properties": { + "readConcern": { "type": "object" }, + "readPreference": { "type": "object" }, + "writeConcern": { "type": "object" }, + "timeoutMS": { "type": "integer" } + } + }, + + "serverApi": { + "type": "object", + "additionalProperties": false, + "required": ["version"], + "properties": { + "version": { "type": "string" }, + "strict": { "type": "boolean" }, + "deprecationErrors": { "type": "boolean" } + } + }, + + "operation": { + "type": "object", + "additionalProperties": false, + "required": ["name", "object"], + "properties": { + "name": { "type": "string" }, + "object": { "type": "string" }, + "arguments": { "type": "object" }, + "ignoreResultAndError": { "type": "boolean" }, + "expectError": { "$ref": "#/definitions/expectedError" }, + "expectResult": {}, + "saveResultAsEntity": { "type": "string" } + }, + "allOf": [ + { "not": { "required": ["expectError", "expectResult"] } }, + { "not": { "required": ["expectError", "saveResultAsEntity"] } }, + { "not": { "required": ["ignoreResultAndError", "expectResult"] } }, + { "not": { "required": ["ignoreResultAndError", "expectError"] } }, + { "not": { "required": ["ignoreResultAndError", "saveResultAsEntity"] } } + ] + }, + + "expectedError": { + "type": "object", + "additionalProperties": false, + "minProperties": 1, + "properties": { + "isError": { + "type": "boolean", + "const": true + }, + "isClientError": { "type": "boolean" }, + "isTimeoutError": { "type": "boolean" }, + "errorContains": { "type": "string" }, + "errorCode": { "type": "integer" }, + "errorCodeName": { "type": "string" }, + "errorLabelsContain": { + "type": "array", + "minItems": 1, + "items": { "type": "string" } + }, + "errorLabelsOmit": { + "type": "array", + "minItems": 1, + "items": { "type": "string" } + }, + "errorResponse": { + "type": "object" + }, + "expectResult": {} + } + }, + + "test": { + "type": "object", + "additionalProperties": false, + "required": ["description", "operations"], + "properties": { + "description": { "type": "string" }, + "runOnRequirements": { + "type": "array", + "minItems": 1, + "items": { "$ref": "#/definitions/runOnRequirement" } + }, + "skipReason": { "type": "string" }, + "operations": { + "type": "array", + "items": { "$ref": "#/definitions/operation" } + }, + "expectEvents": { + "type": "array", + "minItems": 1, + "items": { "$ref": "#/definitions/expectedEventsForClient" } + }, + "expectLogMessages": { + "type": "array", + "minItems": 1, + "items": { "$ref": "#/definitions/expectedLogMessagesForClient" } + }, + "outcome": { + "type": "array", + "minItems": 1, + "items": { "$ref": "#/definitions/collectionData" } + } + } + } + } + } diff --git a/source/unified-test-format/unified-test-format.rst b/source/unified-test-format/unified-test-format.rst index 72a9a691e8..23d5c0a0fb 100644 --- a/source/unified-test-format/unified-test-format.rst +++ b/source/unified-test-format/unified-test-format.rst @@ -459,6 +459,14 @@ The structure of this object is as follows: is enabled. If false, tests MUST NOT run if authentication is enabled. If this field is omitted, there is no authentication requirement. +- ``authMechanism``: Optional string. Specifies an authentication mechanism that + the database needs to support for the test. If set, tests MUST only run if the + given string matches (case-insensitive) one of the strings in the + `authenticationMechanisms + https://www.mongodb.com/docs/manual/reference/parameters/#mongodb-parameter-param.authenticationMechanisms`__ + server parameter. If this field is omitted, there is no authentication + mechanism requirement. + - ``csfle``: Optional boolean. If true, the tests MUST only run if the driver and server support Client-Side Field Level Encryption. CSFLE is supported when all of the following are true: @@ -514,9 +522,9 @@ The structure of this object is as follows: - ``id``: Required string. Unique name for this entity. The YAML file SHOULD define a `node anchor`_ for this field (e.g. ``id: &client0 client0``). - - ``uriOptions``: Optional object. Additional URI options to apply to the - test suite's connection string that is used to create this client. Any keys - in this object MUST override conflicting keys in the connection string. + - ``uriOptions``: Optional object. Additional URI options to apply to the test + suite's connection string that is used to create this client. Any keys in + this object MUST override conflicting keys in the connection string. Documentation for supported options may be found in the `URI Options <../uri-options/uri-options.rst>`__ spec, with one notable @@ -524,6 +532,15 @@ The structure of this object is as follows: will map to an array of strings, each representing a tag set, since it is not feasible to define multiple ``readPreferenceTags`` keys in the object. + Any field in ``uriOptions`` may be a `$$placeholder`_ document and the test + runner MUST support replacing the placeholder document with values loaded + from the test environment. For example:: + + uriOptions: + authMechanism: "MONGODB-OIDC" + authMechanismProperties: + PROVIDER_NAME: { $$placeholder: 1 } + .. _entity_client_useMultipleMongoses: - ``useMultipleMongoses``: Optional boolean. If true and the topology is a @@ -4061,6 +4078,9 @@ Changelog .. Please note schema version bumps in changelog entries where applicable. +:2023-10-16: **Schema version 1.18.** + Add ``authMechanism`` to ``runOnRequirement`` and require that + ``uriOptions`` supports placeholder documents. :2023-10-04: **Schema version 1.17.** Add ``serverHeartbeatStartedEvent``, ``serverHeartbeatSucceededEvent``, and ``serverHeartbeatFailedEvent`` for asserting on SDAM server heartbeat events. From 17fd8a5c4523e3f47303c203ee1e4236d494567b Mon Sep 17 00:00:00 2001 From: Matt Dale <9760375+matthewdale@users.noreply.github.com> Date: Thu, 9 Nov 2023 14:07:55 -0800 Subject: [PATCH 16/44] Add spec test for OIDC auth with built-in providers. --- source/auth/tests/unified/oidc-auth.json | 166 +++++++++++++++++++++++ source/auth/tests/unified/oidc-auth.yml | 88 ++++++++++++ 2 files changed, 254 insertions(+) create mode 100644 source/auth/tests/unified/oidc-auth.json create mode 100644 source/auth/tests/unified/oidc-auth.yml diff --git a/source/auth/tests/unified/oidc-auth.json b/source/auth/tests/unified/oidc-auth.json new file mode 100644 index 0000000000..39e351d82e --- /dev/null +++ b/source/auth/tests/unified/oidc-auth.json @@ -0,0 +1,166 @@ +{ + "description": "reauthenticate_with_retry", + "schemaVersion": "1.12", + "runOnRequirements": [ + { + "minServerVersion": "6.3", + "auth": true, + "authMechanism": "MONGODB-OIDC" + } + ], + "createEntities": [ + { + "client": { + "id": "client0", + "uriOptions": { + "authMechanism": "MONGODB-OIDC", + "authMechanismProperties": { + "$$placeholder": 1 + }, + "retryReads": true, + "retryWrites": true + }, + "observeEvents": [ + "commandStartedEvent", + "commandSucceededEvent", + "commandFailedEvent" + ] + } + }, + { + "database": { + "id": "database0", + "client": "client0", + "databaseName": "db" + } + }, + { + "collection": { + "id": "collection0", + "database": "database0", + "collectionName": "collName" + } + } + ], + "initialData": [ + { + "collectionName": "collName", + "databaseName": "db", + "documents": [] + } + ], + "tests": [ + { + "description": "A simple find operation should succeed", + "operations": [ + { + "name": "find", + "arguments": { + "filter": { + } + }, + "object": "collection0", + "expectResult": [] + } + ], + "expectEvents": [ + { + "client": "client0", + "events": [ + { + "commandStartedEvent": { + "command": { + "find": "collName", + "filter": { + } + } + } + }, + { + "commandSucceededEvent": { + "commandName": "find" + } + } + ] + } + ] + }, + { + "description": "Write command should reauthenticate when receive ReauthenticationRequired error code and retryWrites=true", + "operations": [ + { + "name": "failPoint", + "object": "testRunner", + "arguments": { + "client": "client0", + "failPoint": { + "configureFailPoint": "failCommand", + "mode": { + "times": 1 + }, + "data": { + "failCommands": [ + "insert" + ], + "errorCode": 391 + } + } + } + }, + { + "name": "insertOne", + "object": "collection0", + "arguments": { + "document": { + "_id": 1, + "x": 1 + } + } + } + ], + "expectEvents": [ + { + "client": "client0", + "events": [ + { + "commandStartedEvent": { + "command": { + "insert": "collName", + "documents": [ + { + "_id": 1, + "x": 1 + } + ] + } + } + }, + { + "commandFailedEvent": { + "commandName": "insert" + } + }, + { + "commandStartedEvent": { + "command": { + "insert": "collName", + "documents": [ + { + "_id": 1, + "x": 1 + } + ] + } + } + }, + { + "commandSucceededEvent": { + "commandName": "insert" + } + } + ] + } + ] + } + ] +} diff --git a/source/auth/tests/unified/oidc-auth.yml b/source/auth/tests/unified/oidc-auth.yml new file mode 100644 index 0000000000..af9a7179fa --- /dev/null +++ b/source/auth/tests/unified/oidc-auth.yml @@ -0,0 +1,88 @@ +--- +description: reauthenticate_with_retry +schemaVersion: '1.12' +runOnRequirements: +- minServerVersion: '6.3' + auth: true + authMechanism: "MONGODB-OIDC" +createEntities: +- client: + id: client0 + uriOptions: + authMechanism: "MONGODB-OIDC" + authMechanismProperties: { $$placeholder: 1 } + retryReads: true + retryWrites: true + observeEvents: + - commandStartedEvent + - commandSucceededEvent + - commandFailedEvent +- database: + id: database0 + client: client0 + databaseName: db +- collection: + id: collection0 + database: database0 + collectionName: collName +initialData: +- collectionName: collName + databaseName: db + documents: [] +tests: +- description: A simple find operation should succeed + operations: + - name: find + arguments: + filter: {} + object: collection0 + expectResult: [] + expectEvents: + - client: client0 + events: + - commandStartedEvent: + command: + find: collName + filter: {} + - commandSucceededEvent: + commandName: find +- description: Write command should reauthenticate when receive ReauthenticationRequired + error code and retryWrites=true + operations: + - name: failPoint + object: testRunner + arguments: + client: client0 + failPoint: + configureFailPoint: failCommand + mode: + times: 1 + data: + failCommands: + - insert + errorCode: 391 + - name: insertOne + object: collection0 + arguments: + document: + _id: 1 + x: 1 + expectEvents: + - client: client0 + events: + - commandStartedEvent: + command: + insert: collName + documents: + - _id: 1 + x: 1 + - commandFailedEvent: + commandName: insert + - commandStartedEvent: + command: + insert: collName + documents: + - _id: 1 + x: 1 + - commandSucceededEvent: + commandName: insert From cf5455e1a398efa69ff286db0841675e375462b8 Mon Sep 17 00:00:00 2001 From: Matt Dale <9760375+matthewdale@users.noreply.github.com> Date: Thu, 9 Nov 2023 14:11:14 -0800 Subject: [PATCH 17/44] Fix quotes in ALLOWED_HOSTS Co-authored-by: Durran Jordan --- source/auth/auth.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/auth/auth.rst b/source/auth/auth.rst index dfcd1ed73f..6dc331d44a 100644 --- a/source/auth/auth.rst +++ b/source/auth/auth.rst @@ -1338,7 +1338,7 @@ mechanism_properties MongoDB connections. The hostnames may include a leading "\*." wildcard, which allows for matching (potentially nested) subdomains. ALLOWED_HOSTS is a security feature and MUST default to - ``["*.mongodb.net", '*.mongodb-qa.net", '*.mongodb-dev.net", "*.mongodbgov.net", "localhost", "127.0.0.1", "::1"]``. + ``["*.mongodb.net", "*.mongodb-qa.net", "*.mongodb-dev.net", "*.mongodbgov.net", "localhost", "127.0.0.1", "::1"]``. When ``MONGODB-OIDC`` authentication is attempted against a hostname that does not match any of list of allowed hosts, the driver MUST raise a client-side error without invoking any user-provided From b349eb231340c10c6853688bac25b0b22c0c2b6b Mon Sep 17 00:00:00 2001 From: Matt Dale <9760375+matthewdale@users.noreply.github.com> Date: Mon, 13 Nov 2023 21:44:00 -0800 Subject: [PATCH 18/44] Clarify timeout documentation and callback API. Respond to PR feedback. --- source/auth/auth.rst | 61 ++++++++++++++----------- source/auth/tests/unified/oidc-auth.yml | 4 ++ 2 files changed, 38 insertions(+), 27 deletions(-) diff --git a/source/auth/auth.rst b/source/auth/auth.rst index 6dc331d44a..9753d15962 100644 --- a/source/auth/auth.rst +++ b/source/auth/auth.rst @@ -1213,38 +1213,56 @@ Supported Service Providers AWS ___ - TODO: Add. Custom ______ - Drivers MUST allow the user to integrate with other OIDC providers by -implementing a custom callback that returns an OIDC token. Drivers MUST provide -a way for the callback to be either automatically canceled, or to cancel itself. -This can be as a timeout argument to the callback, a cancellation context passed -to the callback, or some other language-appropriate mechanism. The timeout -deadline MUST be the same as the connection establishment deadline. +implementing a custom callback that returns an OIDC token. Callbacks can be +synchronous and/or asynchronous, depending on the driver and/or language. +Asynchronous callbacks should be preferred when other operations in the driver +use asynchronous functions. -Callbacks can be synchronous and/or asynchronous, depending on the driver -and/or language. Asynchronous callbacks should be preferred when other -operations in the driver use asynchronous functions. +Drivers MUST provide a way for the callback to be either automatically canceled, +or to cancel itself. This can be as a timeout argument to the callback, a +cancellation context passed to the callback, or some other language-appropriate +mechanism. The timeout value MUST be ``min(remaining connectTimeoutMS, remaining +timeoutMS)`` as described in the Server Selection section of the CSOT spec. The driver MUST pass the following information to the callback: -- ``timeoutMS``: A timeout, in milliseconds, deadline, or ``timeoutContext``. +- ``timeout``: A timeout, in milliseconds, deadline, or ``timeoutContext``. +- ``version``: The callback API version number. The version number is used to + communicate callback API changes that are not breaking but that users may want + to know about and review their implementation. Drivers MUST pass ``1`` for the + current callback API version number. Maintainers MUST increment the callback + API version whenever the callback signature is changed. + + For example, users may add the following check in their callback: -The callback MUST return an OIDC access token in JWT format and, if available, -the expiry timestamp for that access token. + .. code:: typescript + + if(params.version > 1) { + throw new Error("OIDC callback API has changed!"); + } + +The callback MUST allow implementers to return the following information: + +- ``accessToken``: An OIDC access token string. The driver MUST NOT attempt to + validate ``accessToken`` directly. +- ``expiresIn``: The expiry time for the access token. Drivers MUST support and + document values for both an expiry time and a value that indicates there is no + expiry time, like 0 or ``null``. The signature of the callback is up to the driver's discretion, but the driver -MUST ensure that additional optional parameters can be added to the callback -signature in the future. An example might look like: +MUST ensure that additional optional input parameters and return values can be +added to the callback signature in the future. An example might look like: .. code:: typescript interface TokenParameters { - timeoutMS: int; + callbackTimeoutMS: int; + version: int; } interface TokenResult { @@ -1258,7 +1276,6 @@ signature in the future. An example might look like: Conversation ```````````` - As an example, given the OIDC token JWT string "abcd1234", the SASL conversation looks like: @@ -1270,7 +1287,6 @@ in the form ``{"jwt": "abcd1234"}``. Access Token Caching ```````````````````` - Drivers MUST cache the Access Token used to authenticate a connection on the connection object. Additionally, drivers MUST cache the most recent access Token on the ``MongoClient``. If any operation fails with ``ReauthenticationRequired`` @@ -1280,7 +1296,6 @@ section below. Speculative Authentication `````````````````````````` - Drivers MUST implement speculative authentication for MONGODB-OIDC during the ``hello`` handshake. If there is a cached Access Token on the ``MongoClient``, use it for authentication. Otherwise, call the configured workload callback to @@ -1289,7 +1304,6 @@ with the ``saslStart`` SASL command. Reauthentication ```````````````` - When reauthentication is requested by the server (as a 391 error code) and MONGODB-OIDC is in use, the driver MUST perform a reauthentication. The following algorithm is used to handle a reauthenication error: @@ -1304,7 +1318,6 @@ TODO: Attempt authentication first or cache new token first? Human Authentication Flow ````````````````````````` - TODO: Why is this section separate? @@ -1349,7 +1362,6 @@ mechanism_properties Interfaces `````````` - Authenticating using the MONGODB-OIDC mechanism will require 1 or 2 round trips between the MongoDB driver and server. The requests from the driver and the replies from the server are described by the following interfaces which are encoded in the payload as octet sequences defining BSON objects: @@ -1412,7 +1424,6 @@ The IdP response that is expected to be returned by the ``REQUEST_TOKEN_CALLBACK Conversation ```````````` - If using the ``PrincipalStepRequest`` first, the conversation will look like: | C: :javascript:`{saslStart: 1, mechanism: "MONGODB-OIDC", payload: BinData(0, "...")}` @@ -1430,7 +1441,6 @@ or when there is a cached Access Token. Drivers MUST instead use ``saslStart`` Caching ``````` - Drivers MUST cache the ``IdPInfo``, Access Token, and Refresh Token associated with each ``MongoClient``. If any operation fails the driver MUST clear the Access Token. The Refresh Token is handled differently, see the ``Reauthentication`` section below. @@ -1441,7 +1451,6 @@ This value is used in the ``Reauthentication`` section below. Speculative Authentication `````````````````````````` - Drivers MUST implement speculative authentication for MONGODB-OIDC during the ``hello`` handshake. If there is a cached Access Token the ``JwtStepRequest`` SASL command will be used as the speculation command. If there is no cached Access Token, the ``PrincipalStepRequest`` will be used as the speculation command. @@ -1449,7 +1458,6 @@ The driver MUST NOT call the callback during speculative authentication. REQUEST_TOKEN_CALLBACK `````````````````````` - Drivers that implement ``REQUEST_TOKEN_CALLBACK`` (i.e. the human authentication flow) MUST provide a way for the ``REQUEST_TOKEN_CALLBACK`` to be either automatically canceled, or to cancel itself. This can be as a timeout argument @@ -1500,7 +1508,6 @@ If no callback is given, the driver MUST must raise a validation error. Reauthentication ```````````````` - When reauthentication is requested by the server (as a 391 error code) and MONGODB-OIDC is in use, the driver MUST perform a reauthentication. The driver MUST account for the case of multiple connections hitting a reauthentication error at different times. The driver MUST also account for a reauthenication that results from an IdP configuration change on the server. diff --git a/source/auth/tests/unified/oidc-auth.yml b/source/auth/tests/unified/oidc-auth.yml index af9a7179fa..65850cc5c1 100644 --- a/source/auth/tests/unified/oidc-auth.yml +++ b/source/auth/tests/unified/oidc-auth.yml @@ -10,6 +10,10 @@ createEntities: id: client0 uriOptions: authMechanism: "MONGODB-OIDC" + # The $$placeholder document should be replaced by auth mechanism + # properties that enable OIDC auth on the target cloud platform. For + # example, when running the test on AWS, replace the $$placeholder + # document with {"PROVIDER_NAME": "aws"}. authMechanismProperties: { $$placeholder: 1 } retryReads: true retryWrites: true From 1fb61b6c477371d6d57482b37d397994670a95d2 Mon Sep 17 00:00:00 2001 From: Matt Dale <9760375+matthewdale@users.noreply.github.com> Date: Thu, 16 Nov 2023 09:05:08 -0800 Subject: [PATCH 19/44] Update spec tests to use the expected database name. --- source/auth/tests/unified/oidc-auth.json | 4 +-- source/auth/tests/unified/oidc-auth.yml | 4 +-- source/unified-test-format/schema-1.18.json | 36 ++++++++++----------- 3 files changed, 22 insertions(+), 22 deletions(-) diff --git a/source/auth/tests/unified/oidc-auth.json b/source/auth/tests/unified/oidc-auth.json index 39e351d82e..3dd8e1fa85 100644 --- a/source/auth/tests/unified/oidc-auth.json +++ b/source/auth/tests/unified/oidc-auth.json @@ -31,7 +31,7 @@ "database": { "id": "database0", "client": "client0", - "databaseName": "db" + "databaseName": "test" } }, { @@ -45,7 +45,7 @@ "initialData": [ { "collectionName": "collName", - "databaseName": "db", + "databaseName": "test", "documents": [] } ], diff --git a/source/auth/tests/unified/oidc-auth.yml b/source/auth/tests/unified/oidc-auth.yml index 65850cc5c1..9593515bab 100644 --- a/source/auth/tests/unified/oidc-auth.yml +++ b/source/auth/tests/unified/oidc-auth.yml @@ -24,14 +24,14 @@ createEntities: - database: id: database0 client: client0 - databaseName: db + databaseName: test - collection: id: collection0 database: database0 collectionName: collName initialData: - collectionName: collName - databaseName: db + databaseName: test documents: [] tests: - description: A simple find operation should succeed diff --git a/source/unified-test-format/schema-1.18.json b/source/unified-test-format/schema-1.18.json index c2e7cca337..9ed28819e1 100644 --- a/source/unified-test-format/schema-1.18.json +++ b/source/unified-test-format/schema-1.18.json @@ -1,6 +1,6 @@ { "$schema": "http://json-schema.org/draft-07/schema#", - + "title": "Unified Test Format", "type": "object", "additionalProperties": false, @@ -33,13 +33,13 @@ "additionalProperties": true } }, - + "definitions": { "version": { "type": "string", "pattern": "^[0-9]+(\\.[0-9]+){1,2}$" }, - + "runOnRequirement": { "type": "object", "additionalProperties": false, @@ -68,7 +68,7 @@ "csfle": { "type": "boolean" } } }, - + "entity": { "type": "object", "additionalProperties": false, @@ -194,7 +194,7 @@ } } }, - + "logComponent": { "type": "string", "enum": ["command", "topology", "serverSelection", "connection"] @@ -215,7 +215,7 @@ "kmsProviders": { "$ref": "#/definitions/kmsProviders" } } }, - + "kmsProviders": { "$defs": { "stringOrPlaceholder": { @@ -281,7 +281,7 @@ } } }, - + "storeEventsAsEntity": { "type": "object", "additionalProperties": false, @@ -315,7 +315,7 @@ } } }, - + "collectionData": { "type": "object", "additionalProperties": false, @@ -335,7 +335,7 @@ } } }, - + "expectedEventsForClient": { "type": "object", "additionalProperties": false, @@ -393,7 +393,7 @@ } ] }, - + "expectedCommandEvent": { "type": "object", "additionalProperties": false, @@ -434,7 +434,7 @@ } } }, - + "expectedCmapEvent": { "type": "object", "additionalProperties": false, @@ -505,7 +505,7 @@ } } }, - + "expectedSdamEvent": { "type": "object", "additionalProperties": false, @@ -548,12 +548,12 @@ } } }, - + "serverDescription": { "type": "object", "additionalProperties": false, "properties": { - "type": { + "type": { "type": "string", "enum": [ "Standalone", @@ -611,7 +611,7 @@ "timeoutMS": { "type": "integer" } } }, - + "serverApi": { "type": "object", "additionalProperties": false, @@ -622,7 +622,7 @@ "deprecationErrors": { "type": "boolean" } } }, - + "operation": { "type": "object", "additionalProperties": false, @@ -644,7 +644,7 @@ { "not": { "required": ["ignoreResultAndError", "saveResultAsEntity"] } } ] }, - + "expectedError": { "type": "object", "additionalProperties": false, @@ -675,7 +675,7 @@ "expectResult": {} } }, - + "test": { "type": "object", "additionalProperties": false, From be7f515f96e7cf6eb67d0f263e19716379559f77 Mon Sep 17 00:00:00 2001 From: Matt Dale <9760375+matthewdale@users.noreply.github.com> Date: Fri, 17 Nov 2023 21:00:11 -0800 Subject: [PATCH 20/44] Add OIDC spec and prose tests. Refine OIDC spec. --- source/auth/auth.rst | 16 +- source/auth/tests/README.rst | 16 +- source/auth/tests/mongodb-oidc.rst | 197 +++++++++++++----- ...dc-auth.json => oidc-auth-with-retry.json} | 14 +- ...oidc-auth.yml => oidc-auth-with-retry.yml} | 6 +- .../unified/oidc-auth-without-retry.json | 175 ++++++++++++++++ .../tests/unified/oidc-auth-without-retry.yml | 94 +++++++++ 7 files changed, 456 insertions(+), 62 deletions(-) rename source/auth/tests/unified/{oidc-auth.json => oidc-auth-with-retry.json} (95%) rename source/auth/tests/unified/{oidc-auth.yml => oidc-auth-with-retry.yml} (95%) create mode 100644 source/auth/tests/unified/oidc-auth-without-retry.json create mode 100644 source/auth/tests/unified/oidc-auth-without-retry.yml diff --git a/source/auth/auth.rst b/source/auth/auth.rst index 9753d15962..0337d34504 100644 --- a/source/auth/auth.rst +++ b/source/auth/auth.rst @@ -1184,8 +1184,20 @@ MONGODB-OIDC MONGODB-OIDC authenticates using an `OpenID Connect (OIDC) `_ access token. +Drivers MUST support the machine-to-machine authentication flow, which is +described in this section. The machine-to-machine authentication flow is +intended to be used in cases where human interaction is not practical or +necessary, such as for web services. sometimes called "Workforce Identity +Federation" in OIDC Identity Provider documentation. + +Drivers MAY support the human-in-the-loop authentication flow described in the +`Human Authentication Flow`_ section. + TODO: Add background on what part of the auth flow drivers play specifically. +The machine-to-machine OIDC authentication flow is called "Workforce Identity +Federation" by many Identity Providers. + `MongoCredential`_ Properties ````````````````````````````` @@ -1294,6 +1306,8 @@ on the ``MongoClient``. If any operation fails with ``ReauthenticationRequired`` and and reauthenticate the connection as described in ``Reauthentication`` section below. +TODO: Examples of how to evict cache, either comapring token values or by using token generation. + Speculative Authentication `````````````````````````` Drivers MUST implement speculative authentication for MONGODB-OIDC during the @@ -1318,7 +1332,7 @@ TODO: Attempt authentication first or cache new token first? Human Authentication Flow ````````````````````````` -TODO: Why is this section separate? +The human-in-the-loop authentication flow is `MongoCredential`_ Properties diff --git a/source/auth/tests/README.rst b/source/auth/tests/README.rst index 3bf86f4fb1..8671f42816 100644 --- a/source/auth/tests/README.rst +++ b/source/auth/tests/README.rst @@ -2,9 +2,17 @@ Auth Tests ========== -The YAML and JSON files in this directory tree are platform-independent tests -that drivers can use to prove their conformance to the Auth Spec at least with -respect to connection string URI input. +Introduction +============ + +This document describes the format of the driver spec tests included in the +JSON and YAML files included in the ``legacy`` sub-directory. Tests in the +``unified`` directory are written using the `Unified Test Format +<../../unified-test-format/unified-test-format.rst>`_. + +The YAML and JSON files in the ``legacy`` directory tree are +platform-independent tests that drivers can use to prove their conformance to +the Auth Spec at least with respect to connection string URI input. Drivers should do additional unit testing if there are alternate ways of configuring credentials on a client. @@ -12,7 +20,7 @@ configuring credentials on a client. Driver must also conduct the prose tests in the Auth Spec test plan section. Format ------- +====== Each YAML file contains an object with a single ``tests`` key. This key is an array of test case objects, each of which have the following keys: diff --git a/source/auth/tests/mongodb-oidc.rst b/source/auth/tests/mongodb-oidc.rst index 50cd4e500c..e49142b9b2 100644 --- a/source/auth/tests/mongodb-oidc.rst +++ b/source/auth/tests/mongodb-oidc.rst @@ -2,7 +2,102 @@ MongoDB OIDC ============ -Drivers MUST test the following scenarios: +TODO + +Local Testing +~~~~~~~~~~~~~ + +To test locally, use the `oidc_get_tokens.sh`_ script from +drivers-evergreen-tools_ to download a set of OIDC tokens, including +`test_user1` and `test_user1_expires`. You first have to install the AWS CLI and +login using the SSO flow. + +For example, if the selected AWS profile ID is "drivers-test", run: + +.. code:: shell + aws configure sso + AWS_PROFILE="drivers-test" ./oidc_get_tokens.sh + AWS_WEB_IDENTITY_TOKEN_FILE="/tmp/tokens/test_user1" /my/test/command + +.. _oidc_get_tokens.sh: https://github.com/mongodb-labs/drivers-evergreen-tools/blob/master/.evergreen/auth_oidc/oidc_get_tokens.sh +.. _drivers-evergreen-tools: https://github.com/mongodb-labs/drivers-evergreen-tools/ + +Prose Tests +=========== + + +1. Custom Callback +~~~~~~~~~~~~~~~~~~ + +- Create a ``MongoClient`` configured with an ``OIDCCallback`` set to the + built-in AWS provider callback. +- Perform a ``find`` operation that succeeds. +- Close the client. + +2. Callback is called during reauthentication +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +- Create a ``MongoClient`` configured with an ``OIDCCallback`` set to the + built-in AWS provider callback. +- Set a fail point for ``find`` commands of the form: + +.. code:: javascript + + { + "configureFailPoint": "failCommand", + "mode": { + "times": 1 + }, + "data": { + "failCommands": [ + "find" + ], + "errorCode": 391 + } + } + +- Perform a ``find`` operation that succeeds. +- Verify that the callback was called 2 times (once during the connection + handshake, and again during reauthentication). +- Close the client. + +3. Callback is called twice on handshake authentication failure +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +- Create a ``MongoClient`` configured with an ``OIDCCallback`` set to the + built-in AWS provider callback. +- Set a fail point for ``saslStart`` commands of the form: + +.. code:: javascript + + { + "configureFailPoint": "failCommand", + "mode": { + "times": 1 + }, + "data": { + "failCommands": [ + "saslStart" + ], + "errorCode": 18 + } + } + +- Perform a ``find`` operation that succeeds. +- Verify that the callback was called 2 times during connection handshake (once + to get the initial token, and once to refresh the token after the + authentication failure). +- Close the client. + +4. Reauthentication messages are sent. +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +TODO + +========================= +Human Authentication Flow +========================= + +Drivers that implement the Human Authentication Flow MUST test the following scenarios: - ``Callback-Driven Auth`` - ``Callback Validation`` @@ -10,65 +105,69 @@ Drivers MUST test the following scenarios: - ``Reauthentication`` - ``Separate Connections Avoid Extra Callback Calls`` +Drivers MUST be able to authenticate against a server configured with either one +or two configured identity providers. -.. sectnum:: +Note that typically the preconfigured Atlas Dev clusters are used for testing, +in Evergreen and localy. The URIs can be fetched from the ``drivers/oidc`` +Secrets vault, see `vault instructions`_. Use ``OIDC_ATLAS_URI_SINGLE`` for +``MONGODB_URI_SINGLE`` and ``OIDC_ATLAS_URI_MULTI`` for +``OIDC_ATLAS_URI_MULTI``. -Drivers MUST be able to authenticate against a server configured with either one or two configured identity providers. - -Note that typically the preconfigured Atlas Dev clusters are used for testing, in Evergreen and localy. The URIs can be fetched -from the ``drivers/oidc`` Secrets vault, see `vault instructions`_. Use ``OIDC_ATLAS_URI_SINGLE`` for ``MONGODB_URI_SINGLE`` and -``OIDC_ATLAS_URI_MULTI`` for ``OIDC_ATLAS_URI_MULTI``. - -If using local servers is preferred, using the `Local Testing`_ method, -use ``mongodb://localhost/?authMechanism=MONGODB-OIDC`` for ``MONGODB_URI_SINGLE`` and +If using local servers is preferred, using the `Local Testing`_ method, use +``mongodb://localhost/?authMechanism=MONGODB-OIDC`` for ``MONGODB_URI_SINGLE`` +and ``mongodb://localhost:27018/?authMechanism=MONGODB-OIDC&directConnection=true&readPreference=secondaryPreferred`` -for ``MONGODB_URI_MULTI`` because the other server is a secondary on a replica set, on port ``27018``. +for ``MONGODB_URI_MULTI`` because the other server is a secondary on a replica +set, on port ``27018``. -The default OIDC client used in the tests will be configured with ``MONGODB_URI_SINGLE`` and a valid request callback handler -that returns the ``test_user1`` local token in ``OIDC_TOKEN_DIR`` as the "access_token", and a dummy "refresh_token". +The default OIDC client used in the tests will be configured with +``MONGODB_URI_SINGLE`` and a valid request callback handler that returns the +``test_user1`` local token in ``OIDC_TOKEN_DIR`` as the "access_token", and a +dummy "refresh_token". .. _Local Testing: https://github.com/mongodb-labs/drivers-evergreen-tools/blob/master/.evergreen/auth_oidc/README.md#local-testing .. _vault instructions: https://wiki.corp.mongodb.com/display/DRIVERS/Using+AWS+Secrets+Manager+to+Store+Testing+Secrets -Callback-Driven Auth -==================== +1. Callback-Driven Auth +======================= Drivers MUST be able to authenticate using OIDC callback(s) when there is one principal configured. -Single Principal Implicit Username -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +1.1 Single Principal Implicit Username +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Create default OIDC client. - Perform a ``find`` operation. that succeeds. - Close the client. -Single Principal Explicit Username -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +1.2 Single Principal Explicit Username +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Create a client with ``MONGODB_URI_SINGLE``, a username of ``test_user1``, and the OIDC request callback. - Perform a ``find`` operation that succeeds. - Close the client. -Multiple Principal User 1 -~~~~~~~~~~~~~~~~~~~~~~~~~ +1.3 Multiple Principal User 1 +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Create a client with ``MONGODB_URI_MULTI``, a username of ``test_user1``, and the OIDC request callback. - Perform a ``find`` operation that succeeds. - Close the client. -Multiple Principal User 2 -~~~~~~~~~~~~~~~~~~~~~~~~~ +1.4 Multiple Principal User 2 +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Create a request callback that reads in the generated ``test_user2`` token file. - Create a client with ``MONGODB_URI_MULTI``, a username of ``test_user2``, and the OIDC request callback. - Perform a ``find`` operation that succeeds. - Close the client. -Multiple Principal No User -~~~~~~~~~~~~~~~~~~~~~~~~~~ +1.5 Multiple Principal No User +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Create a client with ``MONGODB_URI_MULTI``, no username, and the OIDC request callback. - Assert that a ``find`` operation fails. - Close the client. -Allowed Hosts Blocked -~~~~~~~~~~~~~~~~~~~~~ +1.6 Allowed Hosts Blocked +~~~~~~~~~~~~~~~~~~~~~~~~~ - Create a default OIDC client, with an ``ALLOWED_HOSTS`` that is an empty list. - Assert that a ``find`` operation fails with a client-side error. - Close the client. @@ -77,11 +176,11 @@ Allowed Hosts Blocked - Assert that a ``find`` operation fails with a client-side error. - Close the client. -Callback Validation -=================== +2. Callback Validation +====================== -Valid Callbacks -~~~~~~~~~~~~~~~ +2.1 Valid Callbacks +~~~~~~~~~~~~~~~~~~~ - Create request callback that validates its inputs and returns a valid token. - Create a client that uses the above callbacks. - Perform a ``find`` operation that succeeds. Verify that the request @@ -89,14 +188,14 @@ Valid Callbacks parameter if possible. Ensure that there are no unexpected fields. - Close the client. -Request Callback Returns Null -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +2.2 Request Callback Returns Null +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Create a client with a request callback that returns ``null``. - Perform a ``find`` operation that fails. - Close the client. -Request Callback Returns Invalid Data -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +2.3 Request Callback Returns Invalid Data +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Create a client with a request callback that returns data not conforming to the ``OIDCRequestTokenResult`` with missing field(s). - Perform a ``find`` operation that fails. @@ -106,8 +205,8 @@ Request Callback Returns Invalid Data - Perform a ``find`` operation that fails. - Close the client. -Speculative Authentication -========================== +3. Speculative Authentication +============================= We can only test the successful case, by verifying that ``saslStart`` is not called. @@ -137,14 +236,14 @@ is not called. - Perform a ``find`` operation that succeeds. - Close the client. -Reauthentication -================ +4. Reauthentication +=================== The driver MUST test reauthentication with MONGODB-OIDC for a read operation. -Succeeds -~~~~~~~~ +4.1 Succeeds +~~~~~~~~~~~~ - Create a default OIDC client and add an event listener. The following assumes that the driver does not emit ``saslStart`` or ``saslContinue`` events. If the driver does emit those events, ignore/filter them for the @@ -183,8 +282,8 @@ Succeeds - Assert that a ``find`` operation failed once during the command execution. - Close the client. -Succeeds no refresh -~~~~~~~~~~~~~~~~~~~ +4.2 Succeeds no refresh +~~~~~~~~~~~~~~~~~~~~~~~ - Create a default OIDC client with a request callback that does not return a refresh token. - Perform a ``find`` operation that succeeds. @@ -210,8 +309,8 @@ Succeeds no refresh - Assert that the request callback has been called twice. - Close the client. -Succeeds after refresh fails -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +4.3 Succeeds after refresh fails +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Create a default OIDC client. - Perform a ``find`` operation that succeeds. - Assert that the request callback has been called once. @@ -236,8 +335,8 @@ Succeeds after refresh fails - Assert that the request callback has been called three times. - Close the client. -Fails -~~~~~ +4.4 Fails +~~~~~~~~~ - Create a default OIDC client. - Perform a find operation that succeeds (to force a speculative auth). - Assert that the request callback has been called once. @@ -262,8 +361,8 @@ Fails - Assert that the request callback has been called twice. - Close the client. -Separate Connections Avoid Extra Callback Calls -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +4.5 Separate Connections Avoid Extra Callback Calls +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The following test assumes that the driver will be able to share a cache between two MongoClient objects, or ensure that the same MongoClient is used with two different connections. Otherwise, the test would have a race condition. diff --git a/source/auth/tests/unified/oidc-auth.json b/source/auth/tests/unified/oidc-auth-with-retry.json similarity index 95% rename from source/auth/tests/unified/oidc-auth.json rename to source/auth/tests/unified/oidc-auth-with-retry.json index 3dd8e1fa85..aeae3288c9 100644 --- a/source/auth/tests/unified/oidc-auth.json +++ b/source/auth/tests/unified/oidc-auth-with-retry.json @@ -1,9 +1,9 @@ { - "description": "reauthenticate_with_retry", - "schemaVersion": "1.12", + "description": "OIDC authentication with retry", + "schemaVersion": "1.18", "runOnRequirements": [ { - "minServerVersion": "6.3", + "minServerVersion": "7.0", "auth": true, "authMechanism": "MONGODB-OIDC" } @@ -46,7 +46,9 @@ { "collectionName": "collName", "databaseName": "test", - "documents": [] + "documents": [ + + ] } ], "tests": [ @@ -60,7 +62,9 @@ } }, "object": "collection0", - "expectResult": [] + "expectResult": [ + + ] } ], "expectEvents": [ diff --git a/source/auth/tests/unified/oidc-auth.yml b/source/auth/tests/unified/oidc-auth-with-retry.yml similarity index 95% rename from source/auth/tests/unified/oidc-auth.yml rename to source/auth/tests/unified/oidc-auth-with-retry.yml index 9593515bab..47481d963d 100644 --- a/source/auth/tests/unified/oidc-auth.yml +++ b/source/auth/tests/unified/oidc-auth-with-retry.yml @@ -1,8 +1,8 @@ --- -description: reauthenticate_with_retry -schemaVersion: '1.12' +description: "OIDC authentication with retry" +schemaVersion: "1.18" runOnRequirements: -- minServerVersion: '6.3' +- minServerVersion: "7.0" auth: true authMechanism: "MONGODB-OIDC" createEntities: diff --git a/source/auth/tests/unified/oidc-auth-without-retry.json b/source/auth/tests/unified/oidc-auth-without-retry.json new file mode 100644 index 0000000000..ad8c93c03f --- /dev/null +++ b/source/auth/tests/unified/oidc-auth-without-retry.json @@ -0,0 +1,175 @@ +{ + "description": "OIDC authentication without retry", + "schemaVersion": "1.18", + "runOnRequirements": [ + { + "minServerVersion": "7.0", + "auth": true, + "authMechanism": "MONGODB-OIDC" + } + ], + "createEntities": [ + { + "client": { + "id": "authClient" + } + }, + { + "client": { + "id": "client0", + "uriOptions": { + "authMechanism": "MONGODB-OIDC", + "authMechanismProperties": { + "$$placeholder": 1 + }, + "retryReads": true, + "retryWrites": true + }, + "observeEvents": [ + "commandStartedEvent", + "commandSucceededEvent", + "commandFailedEvent" + ] + } + }, + { + "database": { + "id": "database0", + "client": "client0", + "databaseName": "test" + } + }, + { + "collection": { + "id": "collection0", + "database": "database0", + "collectionName": "collName" + } + } + ], + "initialData": [ + { + "collectionName": "collName", + "databaseName": "test", + "documents": [ + + ] + } + ], + "tests": [ + { + "description": "A simple find operation should succeed", + "operations": [ + { + "name": "find", + "arguments": { + "filter": { + } + }, + "object": "collection0", + "expectResult": [ + + ] + } + ], + "expectEvents": [ + { + "client": "client0", + "events": [ + { + "commandStartedEvent": { + "command": { + "find": "collName", + "filter": { + } + } + } + }, + { + "commandSucceededEvent": { + "commandName": "find" + } + } + ] + } + ] + }, + { + "description": "Write command should reauthenticate when receive ReauthenticationRequired error code and retryWrites=true", + "operations": [ + { + "name": "failPoint", + "object": "testRunner", + "arguments": { + "client": "client0", + "failPoint": { + "configureFailPoint": "failCommand", + "mode": { + "times": 1 + }, + "data": { + "failCommands": [ + "insert" + ], + "errorCode": 391 + } + } + } + }, + { + "name": "insertOne", + "object": "collection0", + "arguments": { + "document": { + "_id": 1, + "x": 1 + } + } + } + ], + "expectEvents": [ + { + "client": "client0", + "events": [ + { + "commandStartedEvent": { + "command": { + "insert": "collName", + "documents": [ + { + "_id": 1, + "x": 1 + } + ] + } + } + }, + { + "commandFailedEvent": { + "commandName": "insert" + } + }, + { + "commandStartedEvent": { + "command": { + "insert": "collName", + "documents": [ + { + "_id": 1, + "x": 1 + } + ] + } + } + }, + { + "commandSucceededEvent": { + "commandName": "insert" + } + } + ] + } + ] + } + ] +} diff --git a/source/auth/tests/unified/oidc-auth-without-retry.yml b/source/auth/tests/unified/oidc-auth-without-retry.yml new file mode 100644 index 0000000000..16b4978d1d --- /dev/null +++ b/source/auth/tests/unified/oidc-auth-without-retry.yml @@ -0,0 +1,94 @@ +--- +description: "OIDC authentication without retry" +schemaVersion: "1.18" +runOnRequirements: +- minServerVersion: "7.0" + auth: true + authMechanism: "MONGODB-OIDC" +createEntities: +- client: + id: authClient +- client: + id: client0 + uriOptions: + authMechanism: "MONGODB-OIDC" + # The $$placeholder document should be replaced by auth mechanism + # properties that enable OIDC auth on the target cloud platform. For + # example, when running the test on AWS, replace the $$placeholder + # document with {"PROVIDER_NAME": "aws"}. + authMechanismProperties: { $$placeholder: 1 } + retryReads: true + retryWrites: true + observeEvents: + - commandStartedEvent + - commandSucceededEvent + - commandFailedEvent +- database: + id: database0 + client: client0 + databaseName: test +- collection: + id: collection0 + database: database0 + collectionName: collName +initialData: +- collectionName: collName + databaseName: test + documents: [] +tests: +- description: A simple find operation should succeed + operations: + - name: find + arguments: + filter: {} + object: collection0 + expectResult: [] + expectEvents: + - client: client0 + events: + - commandStartedEvent: + command: + find: collName + filter: {} + - commandSucceededEvent: + commandName: find +- description: Write command should reauthenticate when receive ReauthenticationRequired + error code and retryWrites=true + operations: + - name: failPoint + object: testRunner + arguments: + client: client0 + failPoint: + configureFailPoint: failCommand + mode: + times: 1 + data: + failCommands: + - insert + errorCode: 391 + - name: insertOne + object: collection0 + arguments: + document: + _id: 1 + x: 1 + expectEvents: + - client: client0 + events: + - commandStartedEvent: + command: + insert: collName + documents: + - _id: 1 + x: 1 + - commandFailedEvent: + commandName: insert + - commandStartedEvent: + command: + insert: collName + documents: + - _id: 1 + x: 1 + - commandSucceededEvent: + commandName: insert From 48f9fd46ef506374c5234a986bdb01bee2ce3a44 Mon Sep 17 00:00:00 2001 From: Matt Dale <9760375+matthewdale@users.noreply.github.com> Date: Mon, 27 Nov 2023 16:53:32 -0800 Subject: [PATCH 21/44] Clarify caching section and add examples. --- source/auth/auth.rst | 112 +++++++++++++++++++++++++++++++++++-------- 1 file changed, 93 insertions(+), 19 deletions(-) diff --git a/source/auth/auth.rst b/source/auth/auth.rst index 0337d34504..670433a926 100644 --- a/source/auth/auth.rst +++ b/source/auth/auth.rst @@ -1217,23 +1217,28 @@ mechanism_properties PROVIDER_NAME Drivers MUST allow the user to specify the name of the OIDC service to use to obtain credentials. MUST be one of ["aws"]. If PROVIDER_NAME is - given and a Custom provider callback is configured, the driver MUST + given and a custom callback is configured (either via , the driver MUST raise an error. + OIDC_TOKEN_CALLBACK + Drivers MAY allow the user to specify a custom OIDC callback using a + mechanism property. Drivers MUST only support OIDCToken + Supported Service Providers ``````````````````````````` AWS ___ -TODO: Add. +If ``PROVIDER_NAME:aws`` is specified, drivers should read the file ``AWS_WEB_IDENTITY_TOKEN_FILE`` +TODO -Custom -______ -Drivers MUST allow the user to integrate with other OIDC providers by -implementing a custom callback that returns an OIDC token. Callbacks can be -synchronous and/or asynchronous, depending on the driver and/or language. -Asynchronous callbacks should be preferred when other operations in the driver -use asynchronous functions. +Custom Callback +_______________ +Drivers MUST allow the user to integrate with OIDC providers not supported by +built-in logic by implementing a custom callback that returns an OIDC token. +Callbacks can be synchronous or asynchronous, depending on the driver and/or +language. Asynchronous callbacks should be preferred when other operations in +the driver use asynchronous functions. Drivers MUST provide a way for the callback to be either automatically canceled, or to cancel itself. This can be as a timeout argument to the callback, a @@ -1272,19 +1277,19 @@ added to the callback signature in the future. An example might look like: .. code:: typescript - interface TokenParameters { + interface OIDCTokenParams { callbackTimeoutMS: int; version: int; } - interface TokenResult { + interface OIDCToken { accessToken: string; expiresInSeconds: Optional; } .. code:: typescript - function token(params: TokenParameters): TokenResult + function oidcToken(params: OIDCTokenParams): OIDCToken Conversation ```````````` @@ -1299,14 +1304,82 @@ in the form ``{"jwt": "abcd1234"}``. Access Token Caching ```````````````````` -Drivers MUST cache the Access Token used to authenticate a connection on the -connection object. Additionally, drivers MUST cache the most recent access Token -on the ``MongoClient``. If any operation fails with ``ReauthenticationRequired`` -(391), the driver MUST clear the access Token cached on the connection object -and and reauthenticate the connection as described in ``Reauthentication`` -section below. +Drivers MUST cache the most recent Access Token per ``MongoClient`` (henceforth +referred to as the Client Cache). Drivers MAY store the Client Cache on the +``MongoClient`` object or any object that guarantees exactly 1 cached Access +Token per ``MongoClient``. Additionally, drivers MUST cache the Access Token +used to authenticate a connection on the connection object (henceforth referred +to as the Connection Cache). + +If any operation fails with ``ReauthenticationRequired`` (error code 391), the +driver MUST consider the Access Token from the Connection Cache expired and +refresh the Connection Cache using the following algorithm before performing +`reauthentication`_: + +#. Acquire the lock for the Client Cache. +#. Check if the Access Token in the Client Cache is the same as the expired + Access Token. + #. If the Client Cache is empty or has the same Access Token as the expired + Access Token, fetch a new Access Token from the Token Provider or custom callback + and store it in the Client Cache. +#. Release the lock for the Client Cache. +#. Store the Access Token from the Client Cache in the Connection Cache. + +If the MongoDB authentication handshake for a new connection fails with +``AuthenticationFailed`` (error code 18), the driver MUST check if the Access +Token used for the authentication handshake was from the Client Cache or was a +new Access Token from the Token Provider. If the Access Token was from the +Client Cache, it's possible the cached Access Token has expired, so the driver +MUST fetch a new Access Token from the Token Provider and attempt authentication +handshake a second time. + +Example code for Access Token caching during authentication and +reauthentication: + +.. code:: python + + def auth(conn): + token, is_cache = client_cache(None) + try: + sasl_conversation(conn, {"jwt": token}) + except AuthenticationFailed as err: + # If the Access Token is cached, it's possible that the token has expired, + # so fetch a new token and attempt another SASL conversation. If the + # second SASL conversation fails, raise an exception. + if is_cache: + invalid_token = token + token = client_cache(invalid_token) + sasl_conversation(conn, {"jwt": token}) + else: + raise err + conn.oidc_cache.token = token + + def reauth(conn): + invalid_token = conn.oidc_cache.token + token, is_cache = client_cache(invalid_token) + sasl_conversation(conn, {"jwt": token}) + conn.oidc_cache.token = token + + def client_cache(invalid_token): + # Lock the Client Cache so that only one caller can modify the cache and + # call the provider or custom callback at a time. + client.oidc_cache.lock() + token = client.oidc_cache.token + # Check if we need to fetch and cache a new token from the OIDC provider or + # custom callback. + is_cache = True + if token is None or token == invalid_token: + token = oidc_provider() + is_cache = False + client.oidc_cache.token = token + client.cached_token.unlock() + + return token, is_cache + +To avoid adding a bottleneck that would override the ``maxConnecting`` setting, +the driver MUST not hold an exclusive a lock while performing the authentication +handshake. -TODO: Examples of how to evict cache, either comapring token values or by using token generation. Speculative Authentication `````````````````````````` @@ -1316,6 +1389,7 @@ use it for authentication. Otherwise, call the configured workload callback to retrieve a new Access Token for the new connection and send that Access Token with the ``saslStart`` SASL command. +.. _reauthentication: Reauthentication ```````````````` When reauthentication is requested by the server (as a 391 error code) and From f841d1394f805d24ee1121f548aa2c24eba3ed93 Mon Sep 17 00:00:00 2001 From: Matt Dale <9760375+matthewdale@users.noreply.github.com> Date: Tue, 28 Nov 2023 21:02:04 -0800 Subject: [PATCH 22/44] Align caching, authentication, and reauthentication logic. Complete more TODOs. --- source/auth/auth.rst | 140 +++++++++++++++++++++++++------------------ 1 file changed, 83 insertions(+), 57 deletions(-) diff --git a/source/auth/auth.rst b/source/auth/auth.rst index 670433a926..b74d6e1421 100644 --- a/source/auth/auth.rst +++ b/source/auth/auth.rst @@ -1187,17 +1187,12 @@ MONGODB-OIDC authenticates using an `OpenID Connect (OIDC) Drivers MUST support the machine-to-machine authentication flow, which is described in this section. The machine-to-machine authentication flow is intended to be used in cases where human interaction is not practical or -necessary, such as for web services. sometimes called "Workforce Identity -Federation" in OIDC Identity Provider documentation. +necessary, such as for web services. Some OIDC documentation refers to +authentication for services as "workload" authentication. Drivers MAY support the human-in-the-loop authentication flow described in the `Human Authentication Flow`_ section. -TODO: Add background on what part of the auth flow drivers play specifically. - -The machine-to-machine OIDC authentication flow is called "Workforce Identity -Federation" by many Identity Providers. - `MongoCredential`_ Properties ````````````````````````````` @@ -1222,15 +1217,22 @@ mechanism_properties OIDC_TOKEN_CALLBACK Drivers MAY allow the user to specify a custom OIDC callback using a - mechanism property. Drivers MUST only support OIDCToken + mechanism property. Drivers MUST support either the ``MongoClient`` + configuration method or the mechanism property, but MUST NOT support + both. -Supported Service Providers -``````````````````````````` +Supported Providers +``````````````````` + +Drivers MUST support all of the following OIDC authentication providers. AWS ___ -If ``PROVIDER_NAME:aws`` is specified, drivers should read the file ``AWS_WEB_IDENTITY_TOKEN_FILE`` -TODO +The AWS provider is enabled by setting auth mechanism property +``PROVIDER_NAME:aws``. If enabled, drivers MUST read the file path from +environment variable ``AWS_WEB_IDENTITY_TOKEN_FILE`` and then read the OIDC +Access Token from that file. The driver MUST use the contents of that file as +value in the ``jwt`` field of the ``saslStart`` payload. Custom Callback _______________ @@ -1265,9 +1267,9 @@ The driver MUST pass the following information to the callback: The callback MUST allow implementers to return the following information: -- ``accessToken``: An OIDC access token string. The driver MUST NOT attempt to +- ``accessToken``: An OIDC access Token string. The driver MUST NOT attempt to validate ``accessToken`` directly. -- ``expiresIn``: The expiry time for the access token. Drivers MUST support and +- ``expiresIn``: The expiry time for the access Token. Drivers MUST support and document values for both an expiry time and a value that indicates there is no expiry time, like 0 or ``null``. @@ -1304,37 +1306,42 @@ in the form ``{"jwt": "abcd1234"}``. Access Token Caching ```````````````````` -Drivers MUST cache the most recent Access Token per ``MongoClient`` (henceforth -referred to as the Client Cache). Drivers MAY store the Client Cache on the -``MongoClient`` object or any object that guarantees exactly 1 cached Access -Token per ``MongoClient``. Additionally, drivers MUST cache the Access Token +Drivers MUST cache the most recent access token per ``MongoClient`` (henceforth +referred to as the *Client Cache*). Drivers MAY store the *Client Cache* on the +``MongoClient`` object or any object that guarantees exactly 1 cached access +token per ``MongoClient``. Additionally, drivers MUST cache the access token used to authenticate a connection on the connection object (henceforth referred -to as the Connection Cache). +to as the *Connection Cache*). If any operation fails with ``ReauthenticationRequired`` (error code 391), the -driver MUST consider the Access Token from the Connection Cache expired and -refresh the Connection Cache using the following algorithm before performing +driver MUST consider the access token from the *Connection Cache* expired and +refresh the *Connection Cache* using the following algorithm before performing `reauthentication`_: -#. Acquire the lock for the Client Cache. -#. Check if the Access Token in the Client Cache is the same as the expired - Access Token. - #. If the Client Cache is empty or has the same Access Token as the expired - Access Token, fetch a new Access Token from the Token Provider or custom callback - and store it in the Client Cache. -#. Release the lock for the Client Cache. -#. Store the Access Token from the Client Cache in the Connection Cache. +- Check if the access token in the *Client Cache* is different than the access + token in the *Connection Cache*. + - If they are different, cache the returned access token in *Connection + Cache* and optimisitically try to authenticate using the access token. On + error, continue. +- Call the access token function for the configured provider (or the custom + provider callback). +- Cache the returned access token in the *Client Cache* and *Connection Cache*. +- Attempt to authenticate. Raise any errors to the user. If the MongoDB authentication handshake for a new connection fails with -``AuthenticationFailed`` (error code 18), the driver MUST check if the Access -Token used for the authentication handshake was from the Client Cache or was a -new Access Token from the Token Provider. If the Access Token was from the -Client Cache, it's possible the cached Access Token has expired, so the driver -MUST fetch a new Access Token from the Token Provider and attempt authentication +``AuthenticationFailed`` (error code 18), the driver MUST check if the access +token used for the authentication handshake was from the *Client Cache* or was a +new access token from the Token Provider. If the access token was from the +*Client Cache*, it's possible the cached access token has expired, so the driver +MUST fetch a new access token from the Token Provider and attempt authentication handshake a second time. -Example code for Access Token caching during authentication and -reauthentication: +The driver MUST ensure that only one call to the configured provider or custom +provider callback can happen at a time. To avoid adding a bottleneck that would +override the ``maxConnecting`` setting, the driver MUST NOT hold an exclusive +lock while performing the authentication handshake. + +Example code for access token caching using the read-through cache pattern: .. code:: python @@ -1343,7 +1350,7 @@ reauthentication: try: sasl_conversation(conn, {"jwt": token}) except AuthenticationFailed as err: - # If the Access Token is cached, it's possible that the token has expired, + # If the access token is cached, it's possible that the token has expired, # so fetch a new token and attempt another SASL conversation. If the # second SASL conversation fails, raise an exception. if is_cache: @@ -1354,12 +1361,6 @@ reauthentication: raise err conn.oidc_cache.token = token - def reauth(conn): - invalid_token = conn.oidc_cache.token - token, is_cache = client_cache(invalid_token) - sasl_conversation(conn, {"jwt": token}) - conn.oidc_cache.token = token - def client_cache(invalid_token): # Lock the Client Cache so that only one caller can modify the cache and # call the provider or custom callback at a time. @@ -1376,38 +1377,63 @@ reauthentication: return token, is_cache -To avoid adding a bottleneck that would override the ``maxConnecting`` setting, -the driver MUST not hold an exclusive a lock while performing the authentication -handshake. - Speculative Authentication `````````````````````````` Drivers MUST implement speculative authentication for MONGODB-OIDC during the -``hello`` handshake. If there is a cached Access Token on the ``MongoClient``, +``hello`` handshake. If there is a cached access token on the ``MongoClient``, use it for authentication. Otherwise, call the configured workload callback to -retrieve a new Access Token for the new connection and send that Access Token +retrieve a new access token for the new connection and send that access token with the ``saslStart`` SASL command. + .. _reauthentication: + Reauthentication ```````````````` When reauthentication is requested by the server (as a 391 error code) and MONGODB-OIDC is in use, the driver MUST perform a reauthentication. The following algorithm is used to handle a reauthenication error: -- First, see if the Access Token on the ``MongoClient`` is different than the Access Token on the connection object. - - If they are different, optimisitically try to authenticate using the Access Token from the ``MongoClient``. On error, continue. -- Call the Access Token function for the configured provider (or the custom provider callback). -- Cache the returned Access Token on the current connection object and on the ``MongoClient``. -- Attempt to authenticate. Raise any errors to the user. +- Check if the access token in the *Client Cache* is different than the + access token on the connection object. + - If they are different, cache the returned access token in *Connection + Cache* and optimisitically try to authenticate using the access token. On + error, continue. +- Call the access token function for the configured provider (or the custom + provider callback). +- Cache the returned access token in the *Client Cache* and *Connection Cache*. +- Attempt to authenticate. Raise any errors to the user. + -TODO: Attempt authentication first or cache new token first? +Example code for reauthentication using the ``client_cache`` function from +`Access Token Caching`_: + +.. code:: python + + def reauth(conn): + invalid_token = conn.oidc_cache.token + token, is_cache = client_cache(invalid_token) + + if is_cache: + try: + conn.oidc_cache.token = token + sasl_conversation(conn, {"jwt": token}) + return + except AuthenticationFailed: + invalid_token = token + token = client_cache(invalid_token) + + conn.oidc_cache.token = token + sasl_conversation(conn, {"jwt": token}) Human Authentication Flow ````````````````````````` -The human-in-the-loop authentication flow is - +Drivers MAY support the human-in-the-loop authentication flow, which is +described in this section. The human-in-the-loop authentication flow is intended +to be used for applications that require direct human interaction, such as +database tools or CLIs. Some OIDC documentation refers to authentication for +humans as "workforce" authentication. `MongoCredential`_ Properties _____________________________ From 9523a849c200ae422975c5a756a6c12df37d4946 Mon Sep 17 00:00:00 2001 From: Matt Dale <9760375+matthewdale@users.noreply.github.com> Date: Wed, 29 Nov 2023 16:37:19 -0800 Subject: [PATCH 23/44] Specify cache algorithms for auth and reauth. Reorganize caching info. --- source/auth/auth.rst | 142 +++++++++++++++++++++++-------------------- 1 file changed, 77 insertions(+), 65 deletions(-) diff --git a/source/auth/auth.rst b/source/auth/auth.rst index b74d6e1421..c2803206b6 100644 --- a/source/auth/auth.rst +++ b/source/auth/auth.rst @@ -1187,8 +1187,8 @@ MONGODB-OIDC authenticates using an `OpenID Connect (OIDC) Drivers MUST support the machine-to-machine authentication flow, which is described in this section. The machine-to-machine authentication flow is intended to be used in cases where human interaction is not practical or -necessary, such as for web services. Some OIDC documentation refers to -authentication for services as "workload" authentication. +necessary, such as for web services. Some OIDC documentation refers to this type +of OIDC authentication as "workload" authentication. Drivers MAY support the human-in-the-loop authentication flow described in the `Human Authentication Flow`_ section. @@ -1223,7 +1223,6 @@ mechanism_properties Supported Providers ``````````````````` - Drivers MUST support all of the following OIDC authentication providers. AWS @@ -1304,8 +1303,29 @@ looks like: Where the sent ``payload`` is a generic binary blob containing a BSON document in the form ``{"jwt": "abcd1234"}``. +Speculative Authentication +`````````````````````````` +Drivers MUST implement speculative authentication for MONGODB-OIDC during the +``hello`` handshake. If there is a cached access token on the ``MongoClient``, +use it for authentication. Otherwise, call the configured workload callback to +retrieve a new access token for the new connection and send that access token +with the ``saslStart`` SASL command. + +.. _reauthentication: + +Reauthentication +```````````````` +When an operation fails with ``ReauthenticationRequired`` (error code 391) and +MONGODB-OIDC is in use, the driver MUST reauthenticate the connection. +Reauthenticating a connection requires fetching a new access token and +re-running the SASL conversation. + Access Token Caching ```````````````````` +Some OIDC access token providers may impose rate limits, incur per-request +costs, or be slow to return. To minimize those issues, drivers MUST cache and +reuse access tokens returned by OIDC access token providers. + Drivers MUST cache the most recent access token per ``MongoClient`` (henceforth referred to as the *Client Cache*). Drivers MAY store the *Client Cache* on the ``MongoClient`` object or any object that guarantees exactly 1 cached access @@ -1313,30 +1333,7 @@ token per ``MongoClient``. Additionally, drivers MUST cache the access token used to authenticate a connection on the connection object (henceforth referred to as the *Connection Cache*). -If any operation fails with ``ReauthenticationRequired`` (error code 391), the -driver MUST consider the access token from the *Connection Cache* expired and -refresh the *Connection Cache* using the following algorithm before performing -`reauthentication`_: - -- Check if the access token in the *Client Cache* is different than the access - token in the *Connection Cache*. - - If they are different, cache the returned access token in *Connection - Cache* and optimisitically try to authenticate using the access token. On - error, continue. -- Call the access token function for the configured provider (or the custom - provider callback). -- Cache the returned access token in the *Client Cache* and *Connection Cache*. -- Attempt to authenticate. Raise any errors to the user. - -If the MongoDB authentication handshake for a new connection fails with -``AuthenticationFailed`` (error code 18), the driver MUST check if the access -token used for the authentication handshake was from the *Client Cache* or was a -new access token from the Token Provider. If the access token was from the -*Client Cache*, it's possible the cached access token has expired, so the driver -MUST fetch a new access token from the Token Provider and attempt authentication -handshake a second time. - -The driver MUST ensure that only one call to the configured provider or custom +Driver MUST ensure that only one call to the configured provider or custom provider callback can happen at a time. To avoid adding a bottleneck that would override the ``maxConnecting`` setting, the driver MUST NOT hold an exclusive lock while performing the authentication handshake. @@ -1345,27 +1342,12 @@ Example code for access token caching using the read-through cache pattern: .. code:: python - def auth(conn): - token, is_cache = client_cache(None) - try: - sasl_conversation(conn, {"jwt": token}) - except AuthenticationFailed as err: - # If the access token is cached, it's possible that the token has expired, - # so fetch a new token and attempt another SASL conversation. If the - # second SASL conversation fails, raise an exception. - if is_cache: - invalid_token = token - token = client_cache(invalid_token) - sasl_conversation(conn, {"jwt": token}) - else: - raise err - conn.oidc_cache.token = token - def client_cache(invalid_token): # Lock the Client Cache so that only one caller can modify the cache and - # call the provider or custom callback at a time. + # call the configured provider or custom callback at a time. client.oidc_cache.lock() token = client.oidc_cache.token + # Check if we need to fetch and cache a new token from the OIDC provider or # custom callback. is_cache = True @@ -1377,37 +1359,62 @@ Example code for access token caching using the read-through cache pattern: return token, is_cache +Handshake Authentication +________________________ +Use the following algorithm to manage the caches during handshake +authentication: + +- Check if the the *Client Cache* has an access token. + - If it does, cache the returned access token in *Connection Cache* and + optimisitically try to authenticate using the access token. If the server + returns ``AuthenticationFailed`` (error code 18), continue. +- Call the access token function for the configured provider or the custom + provider callback. +- Cache the returned access token in the *Client Cache* and *Connection Cache*. +- Attempt to authenticate. Raise any errors to the user. -Speculative Authentication -`````````````````````````` -Drivers MUST implement speculative authentication for MONGODB-OIDC during the -``hello`` handshake. If there is a cached access token on the ``MongoClient``, -use it for authentication. Otherwise, call the configured workload callback to -retrieve a new access token for the new connection and send that access token -with the ``saslStart`` SASL command. +Example code for handshake authentication using the ``client_cache`` function +described above: +.. code:: python -.. _reauthentication: + def auth(conn): + token, is_cache = client_cache(None) + + # If there is a cached access token, try to authenticate with it. If + # authentication fails, it's possible the cached token is expired. In that + # case, invalidate the token, fetch a new token, and try to authenticate + # again. + if is_cache: + try: + conn.oidc_cache.token = token + sasl_conversation(conn, {"jwt": token}) + return + except AuthenticationFailed: + invalid_token = token + token = client_cache(invalid_token) + + conn.oidc_cache.token = token + sasl_conversation(conn, {"jwt": token}) Reauthentication -```````````````` -When reauthentication is requested by the server (as a 391 error code) and -MONGODB-OIDC is in use, the driver MUST perform a reauthentication. The -following algorithm is used to handle a reauthenication error: +________________ +If any operation fails with ``ReauthenticationRequired`` (error code 391), the +driver MUST consider the access token from the *Connection Cache* expired. Use +the following algorithm to manage the caches during `reauthentication`_: -- Check if the access token in the *Client Cache* is different than the - access token on the connection object. +- Check if the access token in the *Client Cache* is different than the access + token in the *Connection Cache*. - If they are different, cache the returned access token in *Connection - Cache* and optimisitically try to authenticate using the access token. On - error, continue. -- Call the access token function for the configured provider (or the custom - provider callback). + Cache* and optimisitically try to authenticate using the access token. If + the server returns ``AuthenticationFailed`` (error code 18), continue. +- Call the access token function for the configured provider or the custom + provider callback. - Cache the returned access token in the *Client Cache* and *Connection Cache*. - Attempt to authenticate. Raise any errors to the user. - -Example code for reauthentication using the ``client_cache`` function from -`Access Token Caching`_: +Example code for reauthentication using the ``client_cache`` function described +above: .. code:: python @@ -1415,6 +1422,10 @@ Example code for reauthentication using the ``client_cache`` function from invalid_token = conn.oidc_cache.token token, is_cache = client_cache(invalid_token) + # If there is a cached access token, try to authenticate with it. If + # authentication fails, it's possible the cached token is expired. In that + # case, invalidate the token, fetch a new token, and try to authenticate + # again. if is_cache: try: conn.oidc_cache.token = token @@ -1427,6 +1438,7 @@ Example code for reauthentication using the ``client_cache`` function from conn.oidc_cache.token = token sasl_conversation(conn, {"jwt": token}) + Human Authentication Flow ````````````````````````` Drivers MAY support the human-in-the-loop authentication flow, which is From 98eec9351cd703a69ce440feb75c7382907aa165 Mon Sep 17 00:00:00 2001 From: Matt Dale <9760375+matthewdale@users.noreply.github.com> Date: Thu, 7 Dec 2023 03:05:03 -0800 Subject: [PATCH 24/44] Wrap up spec and prose tests. --- source/auth/auth.rst | 34 ++++++---- source/auth/tests/mongodb-oidc.rst | 105 +++++++++++++++++++++-------- 2 files changed, 100 insertions(+), 39 deletions(-) diff --git a/source/auth/auth.rst b/source/auth/auth.rst index c2803206b6..bc59b146b1 100644 --- a/source/auth/auth.rst +++ b/source/auth/auth.rst @@ -1211,15 +1211,15 @@ mechanism mechanism_properties PROVIDER_NAME Drivers MUST allow the user to specify the name of the OIDC service to - use to obtain credentials. MUST be one of ["aws"]. If PROVIDER_NAME is - given and a custom callback is configured (either via , the driver MUST - raise an error. + use to obtain credentials. The value MUST be one of ["aws"]. If both + ``PROVIDER_NAME`` and a `custom callback`_ are configured for the same + ``MongoClient``, the driver MUST raise an error. OIDC_TOKEN_CALLBACK Drivers MAY allow the user to specify a custom OIDC callback using a - mechanism property. Drivers MUST support either the ``MongoClient`` - configuration method or the mechanism property, but MUST NOT support - both. + mechanism property. Drivers MUST support specifying a callback either as + a ``MongoClient`` configuration or a mechanism property, but MUST NOT + support both. Supported Providers ``````````````````` @@ -1233,6 +1233,12 @@ environment variable ``AWS_WEB_IDENTITY_TOKEN_FILE`` and then read the OIDC Access Token from that file. The driver MUST use the contents of that file as value in the ``jwt`` field of the ``saslStart`` payload. +Drivers MAY implement the AWS provider so that it conforms to the function +signature of the `custom callback`_ to prevent having to re-implement the AWS +provider logic in the OIDC prose tests. + +.. _`custom callback`: + Custom Callback _______________ Drivers MUST allow the user to integrate with OIDC providers not supported by @@ -1278,7 +1284,7 @@ added to the callback signature in the future. An example might look like: .. code:: typescript - interface OIDCTokenParams { + interface OIDCCallbackParams { callbackTimeoutMS: int; version: int; } @@ -1290,7 +1296,7 @@ added to the callback signature in the future. An example might look like: .. code:: typescript - function oidcToken(params: OIDCTokenParams): OIDCToken + function oidcCallback(params: OIDCCallbackParams): OIDCToken Conversation ```````````` @@ -1308,8 +1314,8 @@ Speculative Authentication Drivers MUST implement speculative authentication for MONGODB-OIDC during the ``hello`` handshake. If there is a cached access token on the ``MongoClient``, use it for authentication. Otherwise, call the configured workload callback to -retrieve a new access token for the new connection and send that access token -with the ``saslStart`` SASL command. +retrieve a new access token and send that access token with the speculative +authentication document. .. _reauthentication: @@ -1367,7 +1373,8 @@ authentication: - Check if the the *Client Cache* has an access token. - If it does, cache the returned access token in *Connection Cache* and optimisitically try to authenticate using the access token. If the server - returns ``AuthenticationFailed`` (error code 18), continue. + returns ``AuthenticationFailed`` (error code 18), slep 100ms then + continue. - Call the access token function for the configured provider or the custom provider callback. - Cache the returned access token in the *Client Cache* and *Connection Cache*. @@ -1391,6 +1398,7 @@ described above: sasl_conversation(conn, {"jwt": token}) return except AuthenticationFailed: + sleep(0.1) invalid_token = token token = client_cache(invalid_token) @@ -1407,7 +1415,8 @@ the following algorithm to manage the caches during `reauthentication`_: token in the *Connection Cache*. - If they are different, cache the returned access token in *Connection Cache* and optimisitically try to authenticate using the access token. If - the server returns ``AuthenticationFailed`` (error code 18), continue. + the server returns ``AuthenticationFailed`` (error code 18), sleep 100ms + then continue. - Call the access token function for the configured provider or the custom provider callback. - Cache the returned access token in the *Client Cache* and *Connection Cache*. @@ -1432,6 +1441,7 @@ above: sasl_conversation(conn, {"jwt": token}) return except AuthenticationFailed: + sleep(0.1) invalid_token = token token = client_cache(invalid_token) diff --git a/source/auth/tests/mongodb-oidc.rst b/source/auth/tests/mongodb-oidc.rst index e49142b9b2..32ce89d960 100644 --- a/source/auth/tests/mongodb-oidc.rst +++ b/source/auth/tests/mongodb-oidc.rst @@ -2,8 +2,6 @@ MongoDB OIDC ============ -TODO - Local Testing ~~~~~~~~~~~~~ @@ -25,34 +23,33 @@ For example, if the selected AWS profile ID is "drivers-test", run: Prose Tests =========== - 1. Custom Callback ~~~~~~~~~~~~~~~~~~ -- Create a ``MongoClient`` configured with an ``OIDCCallback`` set to the - built-in AWS provider callback. +- Create a ``MongoClient`` configured with a custom OIDC callback that + implements the AWS provider logic. - Perform a ``find`` operation that succeeds. - Close the client. 2. Callback is called during reauthentication ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -- Create a ``MongoClient`` configured with an ``OIDCCallback`` set to the - built-in AWS provider callback. +- Create a ``MongoClient`` configured with a custom OIDC callback that + implements the AWS provider logic. - Set a fail point for ``find`` commands of the form: .. code:: javascript { - "configureFailPoint": "failCommand", - "mode": { - "times": 1 + configureFailPoint: "failCommand", + mode: { + times: 1 }, - "data": { - "failCommands": [ + data: { + failCommands: [ "find" ], - "errorCode": 391 + errorCode: 391 } } @@ -61,37 +58,91 @@ Prose Tests handshake, and again during reauthentication). - Close the client. -3. Callback is called twice on handshake authentication failure -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +3. Authentication failures with cached tokens retry with a new token +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +- Create a ``MongoClient`` configured with ``retryReads=false`` and a custom + OIDC callback that implements the AWS provider logic. +- Set a fail point for ``find`` commands of the form: + +.. code:: javascript -- Create a ``MongoClient`` configured with an ``OIDCCallback`` set to the - built-in AWS provider callback. + { + configureFailPoint: "failCommand", + mode: { + times: 1 + }, + data: { + failCommands: [ + "find" + ], + closeConnection: true + } + } + +- Perform a ``find`` operation that fails. This is to force the ``MongoClient`` + to cache an access token. - Set a fail point for ``saslStart`` commands of the form: .. code:: javascript { - "configureFailPoint": "failCommand", - "mode": { - "times": 1 + configureFailPoint: "failCommand", + mode: { + times: 2 }, - "data": { - "failCommands": [ + data: { + failCommands: [ "saslStart" ], - "errorCode": 18 + errorCode: 18 } } -- Perform a ``find`` operation that succeeds. +- Perform a ``find`` operation that fails. - Verify that the callback was called 2 times during connection handshake (once to get the initial token, and once to refresh the token after the authentication failure). - Close the client. -4. Reauthentication messages are sent. -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -TODO +4. Reauthentication messages are sent +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +- Create a ``MongoClient`` configured with a custom OIDC callback that + implements the AWS provider logic. +- Perform a ``find`` operation that succeeds. +- Set fail points for ``find`` and ``saslStart`` of the form: + +.. code:: javascript + + { + configureFailPoint: "failCommand", + mode: { + times: 1 + }, + data: { + failCommands: [ + "find" + ], + errorCode: 391 + } + } + + { + configureFailPoint: "failCommand", + mode: { + times: 2 + }, + data: { + failCommands: [ + "saslStart" + ], + errorCode: 18 + } + } + +- Perform a ``find`` operation that fails. +- Close the client. ========================= Human Authentication Flow From f8c3b4ba78644ca9974fc870144df5bbc41c4eb7 Mon Sep 17 00:00:00 2001 From: Matt Dale <9760375+matthewdale@users.noreply.github.com> Date: Thu, 7 Dec 2023 03:07:43 -0800 Subject: [PATCH 25/44] Fix indentation spaces. --- source/auth/auth.rst | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/source/auth/auth.rst b/source/auth/auth.rst index bc59b146b1..6e3619a1e7 100644 --- a/source/auth/auth.rst +++ b/source/auth/auth.rst @@ -1371,10 +1371,12 @@ Use the following algorithm to manage the caches during handshake authentication: - Check if the the *Client Cache* has an access token. + - If it does, cache the returned access token in *Connection Cache* and optimisitically try to authenticate using the access token. If the server returns ``AuthenticationFailed`` (error code 18), slep 100ms then continue. + - Call the access token function for the configured provider or the custom provider callback. - Cache the returned access token in the *Client Cache* and *Connection Cache*. @@ -1413,10 +1415,12 @@ the following algorithm to manage the caches during `reauthentication`_: - Check if the access token in the *Client Cache* is different than the access token in the *Connection Cache*. + - If they are different, cache the returned access token in *Connection Cache* and optimisitically try to authenticate using the access token. If the server returns ``AuthenticationFailed`` (error code 18), sleep 100ms then continue. + - Call the access token function for the configured provider or the custom provider callback. - Cache the returned access token in the *Client Cache* and *Connection Cache*. From 1ec045ca43c09663f517d307894d8f8fe6851d1e Mon Sep 17 00:00:00 2001 From: Matt Dale <9760375+matthewdale@users.noreply.github.com> Date: Wed, 3 Jan 2024 21:40:16 -0800 Subject: [PATCH 26/44] Remove the undocumented 'callback' param from auth spec tests and combine OIDC MongoCredential properties sections. --- source/auth/auth.rst | 96 ++++++++----------- .../auth/tests/legacy/connection-string.json | 95 +++--------------- .../auth/tests/legacy/connection-string.yml | 57 ++++------- 3 files changed, 74 insertions(+), 174 deletions(-) diff --git a/source/auth/auth.rst b/source/auth/auth.rst index 6e3619a1e7..6397fb4fd7 100644 --- a/source/auth/auth.rst +++ b/source/auth/auth.rst @@ -254,21 +254,26 @@ or "SCRAM-SHA-256"). The cache entry value MUST be either the ``saltedPassword`` parameter or the combination of the ``clientKey`` and ``serverKey`` parameters. +.. _reauthentication: + Reauthentication ~~~~~~~~~~~~~~~~ -On any operation that requires authentication, the server may raise the -error ``ReauthenticationRequired`` (391), typically if the user's credential -has expired. Drivers MUST immediately attempt a reauthentication on -the connection using suitable credentials, as specified by the particular authentication mechanism when this error is raised, and then re-attempt the operation. -This attempt MUST be irrespective of whether the operation is considered -retryable. Drivers MUST NOT resend a hello message during reauthentication, instead using SASL messages directly. Any errors that could not be recovered from during reauthentication, or that were encountered during the -subsequent re-attempt of the operation MUST be raised to the user. Currently -the only authentication mechanism on the server that supports reauthentication is OIDC. -See the OIDC documentation on reauthentication for more details. -Note that in order to implement the unified spec tests for reauthentication, -it may be necessary to add reauthentication support for whichever auth -mechanism is used when running the authentication spec tests. +On any operation that requires authentication, the server may raise the error +``ReauthenticationRequired`` (391), typically if the user's credential has +expired. Drivers MUST immediately attempt a reauthentication on the connection +using suitable credentials, as specified by the particular authentication +mechanism when this error is raised, and then re-attempt the operation. This +attempt MUST be irrespective of whether the operation is considered retryable. +Drivers MUST NOT resend a hello message during reauthentication, instead using +SASL messages directly. Any errors that could not be recovered from during +reauthentication, or that were encountered during the subsequent re-attempt of +the operation MUST be raised to the user. Currently the only authentication +mechanism on the server that supports reauthentication is OIDC. See the OIDC +documentation on reauthentication for more details. Note that in order to +implement the unified spec tests for reauthentication, it may be necessary to +add reauthentication support for whichever auth mechanism is used when running +the authentication spec tests. -------------------------------- Supported Authentication Methods @@ -1197,7 +1202,7 @@ Drivers MAY support the human-in-the-loop authentication flow described in the ````````````````````````````` username - MUST NOT be specified. + MAY be specified. Its meaning depends on the OIDC provider used. source MUST be "$external". Defaults to ``$external``. @@ -1221,6 +1226,28 @@ mechanism_properties a ``MongoClient`` configuration or a mechanism property, but MUST NOT support both. + REQUEST_TOKEN_CALLBACK + This property is only required for drivers that support the `Human + Authentication Flow`_. Drivers MUST allow the user to specify a callback + of the form "onRequest" (defined below), if the driver supports + providing objects as mechanism property values. Otherwise the driver + MUST allow it as a MongoClientOption. + + ALLOWED_HOSTS + This property is only required for drivers that support the `Human + Authentication Flow`_. The list of allowed hostnames or ip-addresses + (ignoring ports) for MongoDB connections. The hostnames may include a + leading "\*." wildcard, which allows for matching (potentially nested) + subdomains. ALLOWED_HOSTS is a security feature and MUST default to + ``["*.mongodb.net", "*.mongodb-qa.net", "*.mongodb-dev.net", + "*.mongodbgov.net", "localhost", "127.0.0.1", "::1"]``. When + ``MONGODB-OIDC`` authentication is attempted against a hostname that + does not match any of list of allowed hosts, the driver MUST raise a + client-side error without invoking any user-provided callbacks. This + value MUST not be allowed in the URI connection string. The hostname + check MUST be performed after SRV record resolution, if applicable. This + property is only applicable when ``REQUEST_TOKEN_CALLBACK`` is given. + Supported Providers ``````````````````` Drivers MUST support all of the following OIDC authentication providers. @@ -1317,8 +1344,6 @@ use it for authentication. Otherwise, call the configured workload callback to retrieve a new access token and send that access token with the speculative authentication document. -.. _reauthentication: - Reauthentication ```````````````` When an operation fails with ``ReauthenticationRequired`` (error code 391) and @@ -1374,7 +1399,7 @@ authentication: - If it does, cache the returned access token in *Connection Cache* and optimisitically try to authenticate using the access token. If the server - returns ``AuthenticationFailed`` (error code 18), slep 100ms then + returns ``AuthenticationFailed`` (error code 18), sleep 100ms then continue. - Call the access token function for the configured provider or the custom @@ -1461,45 +1486,6 @@ to be used for applications that require direct human interaction, such as database tools or CLIs. Some OIDC documentation refers to authentication for humans as "workforce" authentication. -`MongoCredential`_ Properties -_____________________________ - -username - Drivers MUST allow the user to specify this in the workforce identity - federation flow. If a user omits this when multiple OIDC providers are - configured, the server will produce an error during authentication. - -source - MUST be "$external". Defaults to ``$external``. - -password - MUST NOT be specified. - -mechanism - MUST be "MONGODB-OIDC" - -mechanism_properties - PROVIDER_NAME - MUST NOT be specified. - - REQUEST_TOKEN_CALLBACK - Drivers MUST allow the user to specify a callback of the form "onRequest" (defined below), if the driver supports - providing objects as mechanism property values. Otherwise the driver MUST allow it as a MongoClientOption. - - ALLOWED_HOSTS - The list of allowed hostnames or ip-addresses (ignoring ports) for - MongoDB connections. The hostnames may include a leading "\*." wildcard, which allows for matching - (potentially nested) subdomains. ALLOWED_HOSTS is a - security feature and MUST default to - ``["*.mongodb.net", "*.mongodb-qa.net", "*.mongodb-dev.net", "*.mongodbgov.net", "localhost", "127.0.0.1", "::1"]``. - When ``MONGODB-OIDC`` authentication is attempted against a hostname - that does not match any of list of allowed hosts, the driver MUST - raise a client-side error without invoking any user-provided - callbacks. This value MUST not be allowed in the URI connection - string. The hostname check MUST be performed after SRV record - resolution, if applicable. - This property is only applicable when ``REQUEST_TOKEN_CALLBACK`` is given. - Interfaces `````````` Authenticating using the MONGODB-OIDC mechanism will require 1 or 2 round trips between the MongoDB driver and server. diff --git a/source/auth/tests/legacy/connection-string.json b/source/auth/tests/legacy/connection-string.json index 40ef630ca3..c7bb483689 100644 --- a/source/auth/tests/legacy/connection-string.json +++ b/source/auth/tests/legacy/connection-string.json @@ -482,28 +482,8 @@ } }, { - "description": "should recognise the mechanism and request callback (MONGODB-OIDC)", - "uri": "mongodb://localhost/?authMechanism=MONGODB-OIDC", - "callback": [ - "oidcRequest" - ], - "valid": true, - "credential": { - "username": null, - "password": null, - "source": "$external", - "mechanism": "MONGODB-OIDC", - "mechanism_properties": { - "REQUEST_TOKEN_CALLBACK": true - } - } - }, - { - "description": "should recognise the mechanism when auth source is explicitly specified and with request callback (MONGODB-OIDC)", - "uri": "mongodb://localhost/?authMechanism=MONGODB-OIDC&authSource=$external", - "callback": [ - "oidcRequest" - ], + "description": "should recognise the mechanism with aws provider (MONGODB-OIDC)", + "uri": "mongodb://localhost/?authMechanism=MONGODB-OIDC&authMechanismProperties=PROVIDER_NAME:aws", "valid": true, "credential": { "username": null, @@ -511,17 +491,13 @@ "source": "$external", "mechanism": "MONGODB-OIDC", "mechanism_properties": { - "REQUEST_TOKEN_CALLBACK": true + "PROVIDER_NAME": "aws" } } }, { - "description": "should recognise the mechanism with request and refresh callback (MONGODB-OIDC)", - "uri": "mongodb://localhost/?authMechanism=MONGODB-OIDC", - "callback": [ - "oidcRequest", - "oidcRefresh" - ], + "description": "should recognise the mechanism when auth source is explicitly specified and with provider (MONGODB-OIDC)", + "uri": "mongodb://localhost/?authMechanism=MONGODB-OIDC&authSource=$external&authMechanismProperties=PROVIDER_NAME:aws", "valid": true, "credential": { "username": null, @@ -529,32 +505,17 @@ "source": "$external", "mechanism": "MONGODB-OIDC", "mechanism_properties": { - "REQUEST_TOKEN_CALLBACK": true, - "REFRESH_TOKEN_CALLBACK": true + "PROVIDER_NAME": "aws" } } }, { - "description": "should recognise the mechanism and username with request callback (MONGODB-OIDC)", - "uri": "mongodb://principalName@localhost/?authMechanism=MONGODB-OIDC", + "description": "should ignore username and password if specified for aws provider (MONGODB-OIDC)", + "uri": "mongodb://user:pass@localhost/?authMechanism=MONGODB-OIDC&authMechanismProperties=PROVIDER_NAME:aws", "callback": [ "oidcRequest" ], "valid": true, - "credential": { - "username": "principalName", - "password": null, - "source": "$external", - "mechanism": "MONGODB-OIDC", - "mechanism_properties": { - "REQUEST_TOKEN_CALLBACK": true - } - } - }, - { - "description": "should recognise the mechanism with aws device (MONGODB-OIDC)", - "uri": "mongodb://localhost/?authMechanism=MONGODB-OIDC&authMechanismProperties=PROVIDER_NAME:aws", - "valid": true, "credential": { "username": null, "password": null, @@ -566,55 +527,29 @@ } }, { - "description": "should recognise the mechanism when auth source is explicitly specified and with aws device (MONGODB-OIDC)", - "uri": "mongodb://localhost/?authMechanism=MONGODB-OIDC&authSource=$external&authMechanismProperties=PROVIDER_NAME:aws", - "valid": true, - "credential": { - "username": null, - "password": null, - "source": "$external", - "mechanism": "MONGODB-OIDC", - "mechanism_properties": { - "PROVIDER_NAME": "aws" - } - } - }, - { - "description": "should throw an exception if username and password are specified (MONGODB-OIDC)", - "uri": "mongodb://user:pass@localhost/?authMechanism=MONGODB-OIDC", - "callback": [ - "oidcRequest" - ], + "description": "should throw an exception if username is specified for aws (MONGODB-OIDC)", + "uri": "mongodb://principalName@localhost/?authMechanism=MONGODB-OIDC&PROVIDER_NAME:aws", "valid": false, "credential": null }, { - "description": "should throw an exception if username and deviceName are specified (MONGODB-OIDC)", - "uri": "mongodb://principalName@localhost/?authMechanism=MONGODB-OIDC&PROVIDER_NAME:gcp", + "description": "should throw an exception if specified provider is not supported (MONGODB-OIDC)", + "uri": "mongodb://localhost/?authMechanism=MONGODB-OIDC&authMechanismProperties=PROVIDER_NAME:invalid", "valid": false, "credential": null }, { - "description": "should throw an exception if specified deviceName is not supported (MONGODB-OIDC)", - "uri": "mongodb://localhost/?authMechanism=MONGODB-OIDC&authMechanismProperties=PROVIDER_NAME:unexisted", + "description": "should throw an exception custom callback is chosen but no callback is provided (MONGODB-OIDC)", + "uri": "mongodb://localhost/?authMechanism=MONGODB-OIDC&authMechanismProperties=PROVIDER_NAME:custom", "valid": false, "credential": null }, { - "description": "should throw an exception if neither deviceName nor callbacks specified (MONGODB-OIDC)", + "description": "should throw an exception if neither provider nor callbacks specified (MONGODB-OIDC)", "uri": "mongodb://localhost/?authMechanism=MONGODB-OIDC", "valid": false, "credential": null }, - { - "description": "should throw an exception when only refresh callback is specified (MONGODB-OIDC)", - "uri": "mongodb://localhost/?authMechanism=MONGODB-OIDC", - "callback": [ - "oidcRefresh" - ], - "valid": false, - "credential": null - }, { "description": "should throw an exception when unsupported auth property is specified (MONGODB-OIDC)", "uri": "mongodb://localhost/?authMechanism=MONGODB-OIDC&authMechanismProperties=UnsupportedProperty:unexisted", diff --git a/source/auth/tests/legacy/connection-string.yml b/source/auth/tests/legacy/connection-string.yml index d4fee90d4c..0d0a352b87 100644 --- a/source/auth/tests/legacy/connection-string.yml +++ b/source/auth/tests/legacy/connection-string.yml @@ -350,10 +350,8 @@ tests: mechanism: MONGODB-AWS mechanism_properties: AWS_SESSION_TOKEN: token!@#$%^&*()_+ -- description: should recognise the mechanism and request callback (MONGODB-OIDC) - uri: mongodb://localhost/?authMechanism=MONGODB-OIDC - callback: - - oidcRequest +- description: should recognise the mechanism with aws provider (MONGODB-OIDC) + uri: mongodb://localhost/?authMechanism=MONGODB-OIDC&authMechanismProperties=PROVIDER_NAME:aws valid: true credential: username: @@ -361,12 +359,9 @@ tests: source: "$external" mechanism: MONGODB-OIDC mechanism_properties: - REQUEST_TOKEN_CALLBACK: true -- description: should recognise the mechanism when auth source is explicitly specified - and with request callback (MONGODB-OIDC) - uri: mongodb://localhost/?authMechanism=MONGODB-OIDC&authSource=$external - callback: - - oidcRequest + PROVIDER_NAME: aws +- description: should recognise the mechanism when auth source is explicitly specified and with provider (MONGODB-OIDC) + uri: mongodb://localhost/?authMechanism=MONGODB-OIDC&authSource=$external&authMechanismProperties=PROVIDER_NAME:aws valid: true credential: username: @@ -374,12 +369,11 @@ tests: source: "$external" mechanism: MONGODB-OIDC mechanism_properties: - REQUEST_TOKEN_CALLBACK: true -- description: should recognise the mechanism with request and refresh callback (MONGODB-OIDC) - uri: mongodb://localhost/?authMechanism=MONGODB-OIDC + PROVIDER_NAME: aws +- description: should ignore username and password if specified for aws provider (MONGODB-OIDC) + uri: mongodb://user:pass@localhost/?authMechanism=MONGODB-OIDC&authMechanismProperties=PROVIDER_NAME:aws callback: - oidcRequest - - oidcRefresh valid: true credential: username: @@ -387,39 +381,24 @@ tests: source: "$external" mechanism: MONGODB-OIDC mechanism_properties: - REQUEST_TOKEN_CALLBACK: true - REFRESH_TOKEN_CALLBACK: true -- description: should recognise the mechanism and username with request callback (MONGODB-OIDC) - uri: mongodb://principalName@localhost/?authMechanism=MONGODB-OIDC - callback: - - oidcRequest - valid: true + PROVIDER_NAME: aws +- description: should throw an exception if username is specified for aws (MONGODB-OIDC) + uri: mongodb://principalName@localhost/?authMechanism=MONGODB-OIDC&PROVIDER_NAME:aws + valid: false credential: - username: principalName - password: - source: "$external" - mechanism: MONGODB-OIDC - mechanism_properties: - REQUEST_TOKEN_CALLBACK: true -- description: should throw an exception if username and password are specified (MONGODB-OIDC) - uri: mongodb://user:pass@localhost/?authMechanism=MONGODB-OIDC - callback: - - oidcRequest +- description: should throw an exception if specified provider is not supported (MONGODB-OIDC) + uri: mongodb://localhost/?authMechanism=MONGODB-OIDC&authMechanismProperties=PROVIDER_NAME:invalid valid: false credential: -- description: should throw an exception if no callbacks specified - (MONGODB-OIDC) - uri: mongodb://localhost/?authMechanism=MONGODB-OIDC +- description: should throw an exception custom callback is chosen but no callback is provided (MONGODB-OIDC) + uri: mongodb://localhost/?authMechanism=MONGODB-OIDC&authMechanismProperties=PROVIDER_NAME:custom valid: false credential: -- description: should throw an exception when only refresh callback is specified (MONGODB-OIDC) +- description: should throw an exception if neither provider nor callbacks specified (MONGODB-OIDC) uri: mongodb://localhost/?authMechanism=MONGODB-OIDC - callback: - - oidcRefresh valid: false credential: -- description: should throw an exception when unsupported auth property is specified - (MONGODB-OIDC) +- description: should throw an exception when unsupported auth property is specified (MONGODB-OIDC) uri: mongodb://localhost/?authMechanism=MONGODB-OIDC&authMechanismProperties=UnsupportedProperty:unexisted valid: false credential: From 82a20d41ddfb821010ab0a675c5d29d3ce11ea5a Mon Sep 17 00:00:00 2001 From: Matt Dale <9760375+matthewdale@users.noreply.github.com> Date: Wed, 3 Jan 2024 21:42:06 -0800 Subject: [PATCH 27/44] Apply suggestions from code review Co-authored-by: Durran Jordan --- source/auth/tests/mongodb-oidc.rst | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/source/auth/tests/mongodb-oidc.rst b/source/auth/tests/mongodb-oidc.rst index 32ce89d960..afc4b3750b 100644 --- a/source/auth/tests/mongodb-oidc.rst +++ b/source/auth/tests/mongodb-oidc.rst @@ -188,32 +188,32 @@ is one principal configured. 1.1 Single Principal Implicit Username ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -- Create default OIDC client. +- Create default OIDC client with `authMechanism=MONGODB-OIDC`. - Perform a ``find`` operation. that succeeds. - Close the client. 1.2 Single Principal Explicit Username ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -- Create a client with ``MONGODB_URI_SINGLE``, a username of ``test_user1``, and the OIDC request callback. +- Create a client with ``MONGODB_URI_SINGLE``, a username of ``test_user1``, `authMechanism=MONGODB-OIDC`, and the OIDC request callback. - Perform a ``find`` operation that succeeds. - Close the client. 1.3 Multiple Principal User 1 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -- Create a client with ``MONGODB_URI_MULTI``, a username of ``test_user1``, and the OIDC request callback. +- Create a client with ``MONGODB_URI_MULTI``, a username of ``test_user1``, `authMechanism=MONGODB-OIDC`, and the OIDC request callback. - Perform a ``find`` operation that succeeds. - Close the client. 1.4 Multiple Principal User 2 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Create a request callback that reads in the generated ``test_user2`` token file. -- Create a client with ``MONGODB_URI_MULTI``, a username of ``test_user2``, and the OIDC request callback. +- Create a client with ``MONGODB_URI_MULTI``, a username of ``test_user2``, `authMechanism=MONGODB-OIDC`, and the OIDC request callback. - Perform a ``find`` operation that succeeds. - Close the client. 1.5 Multiple Principal No User ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -- Create a client with ``MONGODB_URI_MULTI``, no username, and the OIDC request callback. +- Create a client with ``MONGODB_URI_MULTI``, no username, `authMechanism=MONGODB-OIDC`, and the OIDC request callback. - Assert that a ``find`` operation fails. - Close the client. From 9ff52c0962262ae9690378b8c7a8ed8174c2c8f3 Mon Sep 17 00:00:00 2001 From: Matt Dale <9760375+matthewdale@users.noreply.github.com> Date: Wed, 3 Jan 2024 21:44:44 -0800 Subject: [PATCH 28/44] Fix RST formatting. --- source/auth/tests/mongodb-oidc.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/source/auth/tests/mongodb-oidc.rst b/source/auth/tests/mongodb-oidc.rst index afc4b3750b..880a390bb8 100644 --- a/source/auth/tests/mongodb-oidc.rst +++ b/source/auth/tests/mongodb-oidc.rst @@ -13,6 +13,7 @@ login using the SSO flow. For example, if the selected AWS profile ID is "drivers-test", run: .. code:: shell + aws configure sso AWS_PROFILE="drivers-test" ./oidc_get_tokens.sh AWS_WEB_IDENTITY_TOKEN_FILE="/tmp/tokens/test_user1" /my/test/command From 4e1a5f5426d76d14bfc6ef474154d59a22b9d4e8 Mon Sep 17 00:00:00 2001 From: Matt Dale <9760375+matthewdale@users.noreply.github.com> Date: Wed, 3 Jan 2024 22:11:24 -0800 Subject: [PATCH 29/44] Fix Sphinx lint errors. --- source/auth/tests/mongodb-oidc.rst | 97 ++++++++++--------- .../unified-test-format.rst | 2 +- 2 files changed, 50 insertions(+), 49 deletions(-) diff --git a/source/auth/tests/mongodb-oidc.rst b/source/auth/tests/mongodb-oidc.rst index 880a390bb8..e2142621ee 100644 --- a/source/auth/tests/mongodb-oidc.rst +++ b/source/auth/tests/mongodb-oidc.rst @@ -3,7 +3,7 @@ MongoDB OIDC ============ Local Testing -~~~~~~~~~~~~~ +============= To test locally, use the `oidc_get_tokens.sh`_ script from drivers-evergreen-tools_ to download a set of OIDC tokens, including @@ -24,16 +24,16 @@ For example, if the selected AWS profile ID is "drivers-test", run: Prose Tests =========== -1. Custom Callback -~~~~~~~~~~~~~~~~~~ +(1) Custom Callback +~~~~~~~~~~~~~~~~~~~ - Create a ``MongoClient`` configured with a custom OIDC callback that implements the AWS provider logic. - Perform a ``find`` operation that succeeds. - Close the client. -2. Callback is called during reauthentication -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +(2) Callback is called during reauthentication +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Create a ``MongoClient`` configured with a custom OIDC callback that implements the AWS provider logic. @@ -59,8 +59,8 @@ Prose Tests handshake, and again during reauthentication). - Close the client. -3. Authentication failures with cached tokens retry with a new token -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +(3) Authentication failures with cached tokens retry with a new token +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Create a ``MongoClient`` configured with ``retryReads=false`` and a custom OIDC callback that implements the AWS provider logic. @@ -106,8 +106,8 @@ Prose Tests authentication failure). - Close the client. -4. Reauthentication messages are sent -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +(4) Reauthentication messages are sent +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Create a ``MongoClient`` configured with a custom OIDC callback that implements the AWS provider logic. @@ -145,9 +145,10 @@ Prose Tests - Perform a ``find`` operation that fails. - Close the client. -========================= -Human Authentication Flow -========================= +---------- + +Human Authentication Flow Prose Tests +===================================== Drivers that implement the Human Authentication Flow MUST test the following scenarios: @@ -181,45 +182,45 @@ dummy "refresh_token". .. _Local Testing: https://github.com/mongodb-labs/drivers-evergreen-tools/blob/master/.evergreen/auth_oidc/README.md#local-testing .. _vault instructions: https://wiki.corp.mongodb.com/display/DRIVERS/Using+AWS+Secrets+Manager+to+Store+Testing+Secrets -1. Callback-Driven Auth -======================= +(1) Callback-Driven Auth +~~~~~~~~~~~~~~~~~~~~~~~~ Drivers MUST be able to authenticate using OIDC callback(s) when there is one principal configured. -1.1 Single Principal Implicit Username -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +**1.1 Single Principal Implicit Username** + - Create default OIDC client with `authMechanism=MONGODB-OIDC`. - Perform a ``find`` operation. that succeeds. - Close the client. -1.2 Single Principal Explicit Username -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +**1.2 Single Principal Explicit Username** + - Create a client with ``MONGODB_URI_SINGLE``, a username of ``test_user1``, `authMechanism=MONGODB-OIDC`, and the OIDC request callback. - Perform a ``find`` operation that succeeds. - Close the client. -1.3 Multiple Principal User 1 -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +**1.3 Multiple Principal User 1** + - Create a client with ``MONGODB_URI_MULTI``, a username of ``test_user1``, `authMechanism=MONGODB-OIDC`, and the OIDC request callback. - Perform a ``find`` operation that succeeds. - Close the client. -1.4 Multiple Principal User 2 -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +**1.4 Multiple Principal User 2** + - Create a request callback that reads in the generated ``test_user2`` token file. - Create a client with ``MONGODB_URI_MULTI``, a username of ``test_user2``, `authMechanism=MONGODB-OIDC`, and the OIDC request callback. - Perform a ``find`` operation that succeeds. - Close the client. -1.5 Multiple Principal No User -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +**1.5 Multiple Principal No User** + - Create a client with ``MONGODB_URI_MULTI``, no username, `authMechanism=MONGODB-OIDC`, and the OIDC request callback. - Assert that a ``find`` operation fails. - Close the client. -1.6 Allowed Hosts Blocked -~~~~~~~~~~~~~~~~~~~~~~~~~ +**1.6 Allowed Hosts Blocked** + - Create a default OIDC client, with an ``ALLOWED_HOSTS`` that is an empty list. - Assert that a ``find`` operation fails with a client-side error. - Close the client. @@ -228,11 +229,11 @@ is one principal configured. - Assert that a ``find`` operation fails with a client-side error. - Close the client. -2. Callback Validation -====================== +(2) Callback Validation +~~~~~~~~~~~~~~~~~~~~~~~ + +**2.1 Valid Callbacks** -2.1 Valid Callbacks -~~~~~~~~~~~~~~~~~~~ - Create request callback that validates its inputs and returns a valid token. - Create a client that uses the above callbacks. - Perform a ``find`` operation that succeeds. Verify that the request @@ -240,14 +241,14 @@ is one principal configured. parameter if possible. Ensure that there are no unexpected fields. - Close the client. -2.2 Request Callback Returns Null -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +**2.2 Request Callback Returns Null** + - Create a client with a request callback that returns ``null``. - Perform a ``find`` operation that fails. - Close the client. -2.3 Request Callback Returns Invalid Data -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +**2.3 Request Callback Returns Invalid Data** + - Create a client with a request callback that returns data not conforming to the ``OIDCRequestTokenResult`` with missing field(s). - Perform a ``find`` operation that fails. @@ -257,8 +258,8 @@ is one principal configured. - Perform a ``find`` operation that fails. - Close the client. -3. Speculative Authentication -============================= +(3) Speculative Authentication +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ We can only test the successful case, by verifying that ``saslStart`` is not called. @@ -288,14 +289,14 @@ is not called. - Perform a ``find`` operation that succeeds. - Close the client. -4. Reauthentication -=================== +(4) Reauthentication +~~~~~~~~~~~~~~~~~~~~ The driver MUST test reauthentication with MONGODB-OIDC for a read operation. -4.1 Succeeds -~~~~~~~~~~~~ +**4.1 Succeeds** + - Create a default OIDC client and add an event listener. The following assumes that the driver does not emit ``saslStart`` or ``saslContinue`` events. If the driver does emit those events, ignore/filter them for the @@ -334,8 +335,8 @@ operation. - Assert that a ``find`` operation failed once during the command execution. - Close the client. -4.2 Succeeds no refresh -~~~~~~~~~~~~~~~~~~~~~~~ +**4.2 Succeeds no refresh** + - Create a default OIDC client with a request callback that does not return a refresh token. - Perform a ``find`` operation that succeeds. @@ -361,8 +362,8 @@ operation. - Assert that the request callback has been called twice. - Close the client. -4.3 Succeeds after refresh fails -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +**4.3 Succeeds after refresh fails** + - Create a default OIDC client. - Perform a ``find`` operation that succeeds. - Assert that the request callback has been called once. @@ -387,8 +388,8 @@ operation. - Assert that the request callback has been called three times. - Close the client. -4.4 Fails -~~~~~~~~~ +**4.4 Fails** + - Create a default OIDC client. - Perform a find operation that succeeds (to force a speculative auth). - Assert that the request callback has been called once. @@ -413,8 +414,8 @@ operation. - Assert that the request callback has been called twice. - Close the client. -4.5 Separate Connections Avoid Extra Callback Calls -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +**4.5 Separate Connections Avoid Extra Callback Calls** + The following test assumes that the driver will be able to share a cache between two MongoClient objects, or ensure that the same MongoClient is used with two different connections. Otherwise, the test would have a race condition. diff --git a/source/unified-test-format/unified-test-format.rst b/source/unified-test-format/unified-test-format.rst index 23d5c0a0fb..d55e791676 100644 --- a/source/unified-test-format/unified-test-format.rst +++ b/source/unified-test-format/unified-test-format.rst @@ -463,7 +463,7 @@ The structure of this object is as follows: the database needs to support for the test. If set, tests MUST only run if the given string matches (case-insensitive) one of the strings in the `authenticationMechanisms - https://www.mongodb.com/docs/manual/reference/parameters/#mongodb-parameter-param.authenticationMechanisms`__ + `__ server parameter. If this field is omitted, there is no authentication mechanism requirement. From 7f79ecdb85979811553338b15ccde5e5deb534bf Mon Sep 17 00:00:00 2001 From: Matt Dale <9760375+matthewdale@users.noreply.github.com> Date: Thu, 4 Jan 2024 12:54:04 -0800 Subject: [PATCH 30/44] Fix OIDC prose tests and update spec auth description in handshake spec. --- source/auth/tests/mongodb-oidc.rst | 89 ++++---------------------- source/mongodb-handshake/handshake.rst | 8 ++- 2 files changed, 17 insertions(+), 80 deletions(-) diff --git a/source/auth/tests/mongodb-oidc.rst b/source/auth/tests/mongodb-oidc.rst index e2142621ee..56a3de7074 100644 --- a/source/auth/tests/mongodb-oidc.rst +++ b/source/auth/tests/mongodb-oidc.rst @@ -21,6 +21,8 @@ For example, if the selected AWS profile ID is "drivers-test", run: .. _oidc_get_tokens.sh: https://github.com/mongodb-labs/drivers-evergreen-tools/blob/master/.evergreen/auth_oidc/oidc_get_tokens.sh .. _drivers-evergreen-tools: https://github.com/mongodb-labs/drivers-evergreen-tools/ +---------- + Prose Tests =========== @@ -59,90 +61,23 @@ Prose Tests handshake, and again during reauthentication). - Close the client. -(3) Authentication failures with cached tokens retry with a new token -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +(3) Authentication failures with cached tokens fetch a new token and retry +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Create a ``MongoClient`` configured with ``retryReads=false`` and a custom OIDC callback that implements the AWS provider logic. -- Set a fail point for ``find`` commands of the form: - -.. code:: javascript - - { - configureFailPoint: "failCommand", - mode: { - times: 1 - }, - data: { - failCommands: [ - "find" - ], - closeConnection: true - } - } - -- Perform a ``find`` operation that fails. This is to force the ``MongoClient`` - to cache an access token. -- Set a fail point for ``saslStart`` commands of the form: - -.. code:: javascript - - { - configureFailPoint: "failCommand", - mode: { - times: 2 - }, - data: { - failCommands: [ - "saslStart" - ], - errorCode: 18 - } - } - -- Perform a ``find`` operation that fails. -- Verify that the callback was called 2 times during connection handshake (once - to get the initial token, and once to refresh the token after the - authentication failure). -- Close the client. - -(4) Reauthentication messages are sent -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -- Create a ``MongoClient`` configured with a custom OIDC callback that - implements the AWS provider logic. +- Poison the cache with an invalid access token. - Perform a ``find`` operation that succeeds. -- Set fail points for ``find`` and ``saslStart`` of the form: - -.. code:: javascript - - { - configureFailPoint: "failCommand", - mode: { - times: 1 - }, - data: { - failCommands: [ - "find" - ], - errorCode: 391 - } - } +- Verify that the callback was called 1 time. +- Close the client. - { - configureFailPoint: "failCommand", - mode: { - times: 2 - }, - data: { - failCommands: [ - "saslStart" - ], - errorCode: 18 - } - } +(4) Authentication failures without cached tokens return an error +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +- Create a ``MongoClient`` configured with ``retryReads=false`` and a custom + OIDC callback that always returns invalid access tokens. - Perform a ``find`` operation that fails. +- Verify that the callback was called 1 time. - Close the client. ---------- diff --git a/source/mongodb-handshake/handshake.rst b/source/mongodb-handshake/handshake.rst index 90e6156800..a2fb4002c0 100644 --- a/source/mongodb-handshake/handshake.rst +++ b/source/mongodb-handshake/handshake.rst @@ -410,9 +410,11 @@ SCRAM-SHA-256 sections in the `Driver Authentication spec `_. However, -the driver MUST not call a callback as part of ``speculativeAuthenticate``. +structure as seen in the MONGODB-OIDC conversation section in the `Driver +Authentication spec +`_. +However, the driver MUST not call a callback as part of +``speculativeAuthenticate`` during the `OIDC Human Authentication Flow <../auth/auth.rst#human-authentication-flow>`_. If the initial handshake command with a ``speculativeAuthenticate`` argument succeeds, the client should proceed with the next step of the exchange. If the initial handshake From 697bed6d8c37a616132f515e2f8d4cef9b857eba Mon Sep 17 00:00:00 2001 From: Matt Dale <9760375+matthewdale@users.noreply.github.com> Date: Wed, 17 Jan 2024 14:18:15 -0800 Subject: [PATCH 31/44] Rewrite the OIDC spec to make human auth an extension to machine auth. --- source/auth/auth.rst | 705 ++++++++++++++----------- source/mongodb-handshake/handshake.rst | 2 +- 2 files changed, 405 insertions(+), 302 deletions(-) diff --git a/source/auth/auth.rst b/source/auth/auth.rst index 6397fb4fd7..4d12169d6f 100644 --- a/source/auth/auth.rst +++ b/source/auth/auth.rst @@ -1189,20 +1189,48 @@ MONGODB-OIDC MONGODB-OIDC authenticates using an `OpenID Connect (OIDC) `_ access token. -Drivers MUST support the machine-to-machine authentication flow, which is -described in this section. The machine-to-machine authentication flow is -intended to be used in cases where human interaction is not practical or -necessary, such as for web services. Some OIDC documentation refers to this type -of OIDC authentication as "workload" authentication. +There are two OIDC authentication flows that drivers can support: +machine-to-machine ("machine") and human-in-the-loop ("human"). Drivers MUST +support the machine authentication flow. Drivers MAY support the human +authentication flow. + +Machine Authentication Flow +``````````````````````````` +The machine authentication flow is intended to be used in cases where human +interaction is not necessary or practical, such as to authenticate database +access for a web service. Some OIDC documentation refers to the machine +authentication flow as "workload authentication". + +Drivers MUST implement all behaviors described in the ``MONGODB-OIDC`` +specification, unless the section or block specifically says that it only +applies to the `Human Authentication Flow`_. -Drivers MAY support the human-in-the-loop authentication flow described in the -`Human Authentication Flow`_ section. +Human Authentication Flow +````````````````````````` +The human authentication flow is intended to be used for applications that +involve direct human interaction, such as database tools or CLIs. Some OIDC +documentation refers to the human authentication flow as "workforce +authentication". + +In addition to the **access token**, the human authentication flow introduces +some additional concepts: + +- **Identity Provider (IdP)**: A service that manages user accounts and + authenticates users, such as Okta or OneLogin. +- **Refresh token**: Some OIDC providers may return a refresh token in addition + to an access token. A refresh token can be used to retrieve new access tokens + without requiring a human to re-authorize the application. + +Drivers that support the `Human Authentication Flow`_ MUST implement all +behaviors described in the ``MONGODB-OIDC`` specification, including sections or +blocks that specifically say that it only applies the `Human Authentication +Flow`_. `MongoCredential`_ Properties ````````````````````````````` username - MAY be specified. Its meaning depends on the OIDC provider used. + MAY be specified. Its meaning varies depending on the OIDC provider used. source MUST be "$external". Defaults to ``$external``. @@ -1215,64 +1243,67 @@ mechanism mechanism_properties PROVIDER_NAME - Drivers MUST allow the user to specify the name of the OIDC service to - use to obtain credentials. The value MUST be one of ["aws"]. If both - ``PROVIDER_NAME`` and a `custom callback`_ are configured for the same - ``MongoClient``, the driver MUST raise an error. - - OIDC_TOKEN_CALLBACK - Drivers MAY allow the user to specify a custom OIDC callback using a - mechanism property. Drivers MUST support specifying a callback either as - a ``MongoClient`` configuration or a mechanism property, but MUST NOT - support both. - - REQUEST_TOKEN_CALLBACK - This property is only required for drivers that support the `Human - Authentication Flow`_. Drivers MUST allow the user to specify a callback - of the form "onRequest" (defined below), if the driver supports - providing objects as mechanism property values. Otherwise the driver - MUST allow it as a MongoClientOption. + The name of a built-in OIDC provider integration to use to obtain + credentials. The value MUST be one of ["aws"]. If both ``PROVIDER_NAME`` + and an `OIDC Callback`_ are provided for the same ``MongoClient`` + (either via the ``CALLBACK`` mechanism property or a ``MongoClient`` + configuration), the driver MUST raise an error. + + CALLBACK + An `OIDC Callback` that returns credentials for OIDC providers that do + not have a built-in integraiton. Drivers MAY allow the user to specify + an `OIDC Callback`_ using a ``MongoClient`` configuration instead of a + mechanism property, depending on what is conventional for the driver. + Drivers MUST NOT support both the ``CALLBACK`` mechanism property and + the ``MongoClient`` configuration. + + CALLBACK_TYPE + The type of `OIDC Callback`. The value MUST be one of ["machine", + "human"]. If an `OIDC Callback`_ is configured and ``CALLBACK_TYPE`` is + not specified, the driver MUST raise an error. This property is only + required for drivers that support the `Human Authentication Flow`_. ALLOWED_HOSTS - This property is only required for drivers that support the `Human - Authentication Flow`_. The list of allowed hostnames or ip-addresses - (ignoring ports) for MongoDB connections. The hostnames may include a - leading "\*." wildcard, which allows for matching (potentially nested) - subdomains. ALLOWED_HOSTS is a security feature and MUST default to + The list of allowed hostnames or ip-addresses (ignoring ports) for + MongoDB connections. The hostnames may include a leading "\*." wildcard, + which allows for matching (potentially nested) subdomains. + ``ALLOWED_HOSTS`` is a security feature and MUST default to ``["*.mongodb.net", "*.mongodb-qa.net", "*.mongodb-dev.net", "*.mongodbgov.net", "localhost", "127.0.0.1", "::1"]``. When - ``MONGODB-OIDC`` authentication is attempted against a hostname that - does not match any of list of allowed hosts, the driver MUST raise a - client-side error without invoking any user-provided callbacks. This - value MUST not be allowed in the URI connection string. The hostname - check MUST be performed after SRV record resolution, if applicable. This - property is only applicable when ``REQUEST_TOKEN_CALLBACK`` is given. - -Supported Providers -``````````````````` -Drivers MUST support all of the following OIDC authentication providers. + ``MONGODB-OIDC`` authentication using a `Human Callback`_ is attempted + against a hostname that does not match any of list of allowed hosts, the + driver MUST raise a client-side error without invoking any user-provided + callbacks. This value MUST NOT be allowed in the URI connection string. + The hostname check MUST be performed after SRV record resolution, if + applicable. This property is only required for drivers that support the + `Human Authentication Flow`_. + +Built-in Provider Integrations +`````````````````````````````` +Drivers MUST support all of the following built-in OIDC providers. AWS ___ The AWS provider is enabled by setting auth mechanism property -``PROVIDER_NAME:aws``. If enabled, drivers MUST read the file path from +``PROVIDER_NAME:aws``. + +If enabled, drivers MUST read the file path from environment variable ``AWS_WEB_IDENTITY_TOKEN_FILE`` and then read the OIDC Access Token from that file. The driver MUST use the contents of that file as value in the ``jwt`` field of the ``saslStart`` payload. Drivers MAY implement the AWS provider so that it conforms to the function -signature of the `custom callback`_ to prevent having to re-implement the AWS +signature of the `OIDC Callback`_ to prevent having to re-implement the AWS provider logic in the OIDC prose tests. -.. _`custom callback`: - -Custom Callback -_______________ -Drivers MUST allow the user to integrate with OIDC providers not supported by -built-in logic by implementing a custom callback that returns an OIDC token. -Callbacks can be synchronous or asynchronous, depending on the driver and/or -language. Asynchronous callbacks should be preferred when other operations in -the driver use asynchronous functions. +OIDC Callback +````````````` +Drivers MUST allow users to provide a callback that returns an OIDC access +token. The purpose of the callback is to allow users to integrate with OIDC +providers not supported by the `Built-in Provider Integrations`_. Callbacks can +be synchronous or asynchronous, depending on the driver and/or language. +Asynchronous callbacks should be preferred when other operations in the driver +use asynchronous functions. Drivers MUST provide a way for the callback to be either automatically canceled, or to cancel itself. This can be as a timeout argument to the callback, a @@ -1282,12 +1313,11 @@ timeoutMS)`` as described in the Server Selection section of the CSOT spec. The driver MUST pass the following information to the callback: -- ``timeout``: A timeout, in milliseconds, deadline, or ``timeoutContext``. +- ``timeout``: A timeout, in milliseconds, a deadline, or a ``timeoutContext``. - ``version``: The callback API version number. The version number is used to communicate callback API changes that are not breaking but that users may want to know about and review their implementation. Drivers MUST pass ``1`` for the - current callback API version number. Maintainers MUST increment the callback - API version whenever the callback signature is changed. + current callback API version number. For example, users may add the following check in their callback: @@ -1297,17 +1327,18 @@ The driver MUST pass the following information to the callback: throw new Error("OIDC callback API has changed!"); } -The callback MUST allow implementers to return the following information: +The callback MUST be able to return the following information: - ``accessToken``: An OIDC access Token string. The driver MUST NOT attempt to validate ``accessToken`` directly. -- ``expiresIn``: The expiry time for the access Token. Drivers MUST support and +- ``expiresIn``: An optional expiry time for the access Token. Drivers MUST support and document values for both an expiry time and a value that indicates there is no expiry time, like 0 or ``null``. The signature of the callback is up to the driver's discretion, but the driver MUST ensure that additional optional input parameters and return values can be -added to the callback signature in the future. An example might look like: +added to the callback signature in the future. An example callback API might +look like: .. code:: typescript @@ -1316,335 +1347,407 @@ added to the callback signature in the future. An example might look like: version: int; } - interface OIDCToken { + interface OIDCCredential { accessToken: string; expiresInSeconds: Optional; } -.. code:: typescript + function oidcCallback(params: OIDCCallbackParams): OIDCCredential - function oidcCallback(params: OIDCCallbackParams): OIDCToken +Human Callback +______________ +Drivers that support the `Human Authentication Flow`_ MUST implement the human +callback version of the `OIDC Callback`. -Conversation -```````````` -As an example, given the OIDC token JWT string "abcd1234", the SASL conversation -looks like: +In addition to the information described in the `OIDC Callback` section, +drivers MUST be able to pass the following information to the callback: -| C: :javascript:`{saslStart: 1, mechanism: "MONGODB-OIDC", payload: BinData(0, "FwAAAAJqd3QACQAAAGFiY2QxMjM0AAA=")}` -| S: :javascript:`{conversationId : 1, payload: BinData(0, ""), done: true, ok: 1}` +- ``idpInfo``: Information used to authenticate with the IdP. -Where the sent ``payload`` is a generic binary blob containing a BSON document -in the form ``{"jwt": "abcd1234"}``. + - ``issuer``: A URL which describes the Authentication Server. This identifier + should be the iss of provided access tokens, and be viable for RFC8414 + metadata discovery and RFC9207 identification. + - ``clientId``: A unique client ID for this OIDC client. + - ``requestScopes``: A list of additional scopes to request from IdP. -Speculative Authentication -`````````````````````````` -Drivers MUST implement speculative authentication for MONGODB-OIDC during the -``hello`` handshake. If there is a cached access token on the ``MongoClient``, -use it for authentication. Otherwise, call the configured workload callback to -retrieve a new access token and send that access token with the speculative -authentication document. +- ``refreshToken``: The refresh token, if applicable, to be used by the callback + to request a new token from the issuer. -Reauthentication -```````````````` -When an operation fails with ``ReauthenticationRequired`` (error code 391) and -MONGODB-OIDC is in use, the driver MUST reauthenticate the connection. -Reauthenticating a connection requires fetching a new access token and -re-running the SASL conversation. +In addition to the information described in the `OIDC Callback` section, the +callback MUST be able to return the following information: -Access Token Caching -```````````````````` -Some OIDC access token providers may impose rate limits, incur per-request -costs, or be slow to return. To minimize those issues, drivers MUST cache and -reuse access tokens returned by OIDC access token providers. +- ``refreshToken``: An optional refresh token that can be used to fetch new + access tokens. -Drivers MUST cache the most recent access token per ``MongoClient`` (henceforth -referred to as the *Client Cache*). Drivers MAY store the *Client Cache* on the -``MongoClient`` object or any object that guarantees exactly 1 cached access -token per ``MongoClient``. Additionally, drivers MUST cache the access token -used to authenticate a connection on the connection object (henceforth referred -to as the *Connection Cache*). - -Driver MUST ensure that only one call to the configured provider or custom -provider callback can happen at a time. To avoid adding a bottleneck that would -override the ``maxConnecting`` setting, the driver MUST NOT hold an exclusive -lock while performing the authentication handshake. - -Example code for access token caching using the read-through cache pattern: - -.. code:: python - - def client_cache(invalid_token): - # Lock the Client Cache so that only one caller can modify the cache and - # call the configured provider or custom callback at a time. - client.oidc_cache.lock() - token = client.oidc_cache.token +An example callback API that supports the human callback might look like: - # Check if we need to fetch and cache a new token from the OIDC provider or - # custom callback. - is_cache = True - if token is None or token == invalid_token: - token = oidc_provider() - is_cache = False - client.oidc_cache.token = token - client.cached_token.unlock() - - return token, is_cache +.. code:: typescript -Handshake Authentication -________________________ -Use the following algorithm to manage the caches during handshake -authentication: + interface IdpInfo { + issuer: string; + clientId: string; + requestScopes: Optional>; + } -- Check if the the *Client Cache* has an access token. + interface OIDCCallbackParams { + callbackTimeoutMS: int; + version: int; + idpInfo: Optional; + refreshToken: Optional; + } - - If it does, cache the returned access token in *Connection Cache* and - optimisitically try to authenticate using the access token. If the server - returns ``AuthenticationFailed`` (error code 18), sleep 100ms then - continue. + interface OIDCCredential { + accessToken: string; + refreshToken: Optional; + expiresInSeconds: Optional; + } -- Call the access token function for the configured provider or the custom - provider callback. -- Cache the returned access token in the *Client Cache* and *Connection Cache*. -- Attempt to authenticate. Raise any errors to the user. + function oidcCallback(params: OIDCCallbackParams): OIDCCredential -Example code for handshake authentication using the ``client_cache`` function -described above: +Users enable the human callback behavior by setting mechanism property +``CALLBACK_TYPE:human``. When the human callback behavior is enabled, drivers +MUST use the following behaviors when calling the callback: -.. code:: python +- The driver MUST pass the ``IdpInfo`` and the ``refreshToken`` (if available) + to the callback. +- The timeout duration MUST be 5 minutes. This is to account for the human + interaction required to complete the callback. In this case, the callback is + not subject to CSOT. - def auth(conn): - token, is_cache = client_cache(None) +If ``CALLBACK_TYPE:machine`` drivers MUST use the callback behavior described in +the `OIDC Callback`_ section. - # If there is a cached access token, try to authenticate with it. If - # authentication fails, it's possible the cached token is expired. In that - # case, invalidate the token, fetch a new token, and try to authenticate - # again. - if is_cache: - try: - conn.oidc_cache.token = token - sasl_conversation(conn, {"jwt": token}) - return - except AuthenticationFailed: - sleep(0.1) - invalid_token = token - token = client_cache(invalid_token) +Conversation +```````````` +OIDC supports two conversation styles: one-step and two-step. The server detects +whether the driver is using a one-step or two-step conversation based on the +structure of the ``saslStart`` payload. - conn.oidc_cache.token = token - sasl_conversation(conn, {"jwt": token}) +One-Step +________ +A one-step conversation is used for OIDC providers that allow direct access to +an access token. For example, an OIDC provider configured for machine-to-machine +authentication may provide an access token via a local file pre-loaded on an +application host. -Reauthentication -________________ -If any operation fails with ``ReauthenticationRequired`` (error code 391), the -driver MUST consider the access token from the *Connection Cache* expired. Use -the following algorithm to manage the caches during `reauthentication`_: +Drivers MUST use a one-step conversation when using a cached access token, one +of the `Built-in Provider Integrations`_, an `OIDC Callback` (not a `Human +Callback`_). -- Check if the access token in the *Client Cache* is different than the access - token in the *Connection Cache*. +The one-step conversation starts with a ``saslStart`` containing a +``JwtStepRequest`` payload. The value of ``jwt`` is the OIDC access token +string. - - If they are different, cache the returned access token in *Connection - Cache* and optimisitically try to authenticate using the access token. If - the server returns ``AuthenticationFailed`` (error code 18), sleep 100ms - then continue. +.. code:: typescript -- Call the access token function for the configured provider or the custom - provider callback. -- Cache the returned access token in the *Client Cache* and *Connection Cache*. -- Attempt to authenticate. Raise any errors to the user. + interface JwtStepRequest: + // Compact serialized JWT with signature. + jwt: string; + } -Example code for reauthentication using the ``client_cache`` function described -above: +An example OIDC one-step SASL conversation with access token string "abcd1234" +looks like: -.. code:: python +.. code:: javascript - def reauth(conn): - invalid_token = conn.oidc_cache.token - token, is_cache = client_cache(invalid_token) + // Client: + { + saslStart: 1, + mechanism: "MONGODB-OIDC", + // payload is a BSON generic binary field containing a JwtStepRequest BSON + // document: {"jwt": "abcd1234"} + payload: BinData(0, "FwAAAAJqd3QACQAAAGFiY2QxMjM0AAA=") + } - # If there is a cached access token, try to authenticate with it. If - # authentication fails, it's possible the cached token is expired. In that - # case, invalidate the token, fetch a new token, and try to authenticate - # again. - if is_cache: - try: - conn.oidc_cache.token = token - sasl_conversation(conn, {"jwt": token}) - return - except AuthenticationFailed: - sleep(0.1) - invalid_token = token - token = client_cache(invalid_token) + // Server: + { + conversationId : 1, + payload: BinData(0, ""), + done: true, + ok: 1 + } - conn.oidc_cache.token = token - sasl_conversation(conn, {"jwt": token}) +Two-Step +________ +A two-step conversation is used for OIDC providers that require an extra +authorization step before issuing a credential. For example, an OIDC provider +configured for end-user authentication may require redirecting the user to a +webpage so they can authorize the request. +Drivers that support the `Human Authentication Flow`_ MUST implement the +two-step conversation. Drivers MUST use a two-step conversation when using a +`Human Callback`_ and when there is no cached access token. -Human Authentication Flow -````````````````````````` -Drivers MAY support the human-in-the-loop authentication flow, which is -described in this section. The human-in-the-loop authentication flow is intended -to be used for applications that require direct human interaction, such as -database tools or CLIs. Some OIDC documentation refers to authentication for -humans as "workforce" authentication. - -Interfaces -`````````` -Authenticating using the MONGODB-OIDC mechanism will require 1 or 2 round trips between the MongoDB driver and server. -The requests from the driver and the replies from the server are described by the following interfaces which are encoded -in the payload as octet sequences defining BSON objects: +The two-step conversation starts with a ``saslStart`` containing a +``PrincipalStepRequest`` payload. The value of ``n`` is the ``username`` from +the connection string. If a ``username`` is not provided, field ``n`` should be +omitted. .. code:: typescript - // Driver's opening request in saslStart. interface PrincipalStepRequest { - // Name of the OIDC user Principal. + // Name of the OIDC user principal. n: Optional; } -Note that the principal name is optional as it may be provided by the IdP in environments where only one IdP is used. -If given, then ``username`` provided by the user MUST be used as the Principal ``(n)``. +The server uses ``n`` (if provided) to select an appropriate IdP. Note that the +principal name is optional as it may be provided by the IdP in environments +where only one IdP is used. + +The server responds to the ``PrincipalStepRequest`` with ``IdpInfo`` for the +selected IdP: .. code:: typescript - // The information used by a REQUEST_TOKEN_CALLBACK to authenticate with the Identity Provider. interface IdpInfo { - // URL which describes the Authentication Server. This identifier should be - // the iss of provided access tokens, and be viable for RFC8414 - // metadata discovery and RFC9207 identification. + // A URL which describes the Authentication Server. This identifier should + // be the iss of provided access tokens, and be viable for RFC8414 metadata + // discovery and RFC9207 identification. issuer: string; - // Unique client ID for this OIDC client. + // A unique client ID for this OIDC client. clientId: string; - // Additional scopes to request from IdP. + // A list of additional scopes to request from IdP. requestScopes: Optional>; } -The server will use Principal ``(n)`` if provided in the driver's ``PrincipalStepRequest`` to select an appropriate IdP. -This IdP's configuration will be returned in the server's response that will be used by the end-user to acquire an Access Token. +The driver passes the IdP information to the `Human Callback`_, which should +return an OIDC credential containing an access token and, optionally, a refresh +token. -This Access Token will be used as the JWT in the driver's ``JwtStepRequest`` to complete authentication. +The driver then sends a ``saslContinue`` with a ``JwtStepRequest`` payload to +complete authentication. The value of ``jwt`` is the OIDC access token string. .. code:: typescript - // Client's request with signed token. interface JwtStepRequest: // Compact serialized JWT with signature. jwt: string; } -The IdP response that is expected to be returned by the ``REQUEST_TOKEN_CALLBACK`` is as follows: +An example OIDC two-step SASL conversation with username "myidp" and access +token string "abcd1234" looks like: - .. code:: typescript +.. code:: javascript - // The result of a token request. - interface IdPResponse { - // The oidc access token. - accessToken: string; + // Client: + { + saslStart: 1, + mechanism: "MONGODB-OIDC", + // payload is a BSON generic binary field containing a PrincipalStepRequest + // BSON document: {"n": "myidp"} + payload: BinData(0, "EgAAAAJuAAYAAABteWlkcAAA") + } - // The OIDC refresh token. - refreshToken: Optional; + // Server: + { + conversationId : 1, + // payload is a BSON generic binary field containing an IdpInfo BSON document: + // {"issuer": "https://issuer", "clientId": "abcd", "requestScopes": ["a","b"]} + payload: BinData(0, "WQAAAAJpc3N1ZXIADwAAAGh0dHBzOi8vaXNzdWVyAAJjbGllbnRJZAAFAAAAYWJjZAAEcmVxdWVzdFNjb3BlcwAXAAAAAjAAAgAAAGEAAjEAAgAAAGIAAAA="), + done: false, + ok: 1 + } - // The expiration time in seconds from the current time (ignored). - expiresInSeconds: Optional; - } + // Client: + { + saslContinue: 1, + conversationId: 1, + // payload is a BSON generic binary field containing a JwtStepRequest BSON + // document: {"jwt": "abcd1234"} + payload: BinData(0, "FwAAAAJqd3QACQAAAGFiY2QxMjM0AAA=") + } -Conversation -```````````` -If using the ``PrincipalStepRequest`` first, the conversation will look like: + // Server: + { + conversationId: 1, + payload: BinData(0, ""), + done: true, + ok: 1 + } -| C: :javascript:`{saslStart: 1, mechanism: "MONGODB-OIDC", payload: BinData(0, "...")}` -| S: :javascript:`{conversationId : 1, payload: BinData(0,"..."), done: false, ok: 1}` -| C: :javascript:`{saslContinue: 1, conversationId: 1, payload: BinData(0, "...")}` -| S: :javascript:`{conversationId: 1, payload: BinData(0,".."), done: true, ok: 1}` +Credential Caching +`````````````````` +Some OIDC providers may impose rate limits, incur per-request costs, or be slow +to return. To minimize those issues, drivers MUST cache and reuse access tokens +returned by OIDC providers. -If using the ``JwtStepRequest`` directly, the conversation will look like: +Drivers MUST cache the most recent access token per ``MongoClient`` (henceforth +referred to as the *Client Cache*). Drivers MAY store the *Client Cache* on the +``MongoClient`` object or any object that guarantees exactly 1 cached access +token per ``MongoClient``. Additionally, drivers MUST cache the access token +used to authenticate a connection on the connection object (henceforth referred +to as the *Connection Cache*). -| C: :javascript:`{saslStart: 1, mechanism: "MONGODB-OIDC", payload: BinData(0, "...")}` -| S: :javascript:`{conversationId : 1, payload: BinData(0,"..."), done: true, ok: 1}` +Drivers MUST ensure that only one call to the configured provider or OIDC +callback can happen at a time. To avoid adding a bottleneck that would override +the ``maxConnecting`` setting, the driver MUST NOT hold an exclusive lock while +running ``saslStart`` or ``saslContinue``. -Drivers MUST NOT send a ``PrincipalStepRequest`` when performing machine authentication -or when there is a cached Access Token. Drivers MUST instead use ``saslStart`` with a ``JwtStepRequest``. +Example code for credential caching using the read-through cache pattern: -Caching -``````` -Drivers MUST cache the ``IdPInfo``, Access Token, and Refresh Token associated with each -``MongoClient``. If any operation fails the driver MUST clear the Access Token. The Refresh Token is handled differently, -see the ``Reauthentication`` section below. +.. code:: python -Drivers MUST also use a token generation id that is incremented whenever a new Access Token is retrieved. -When a connection succeeds, the token generation id MUST be associated with or stored on the connection object. -This value is used in the ``Reauthentication`` section below. + def get_access_token(): + # Lock the OIDC authenticator so that only one caller can modify the cache + # and call the configured OIDC provider at a time. + client.oidc_cache.lock() -Speculative Authentication -`````````````````````````` -Drivers MUST implement speculative authentication for MONGODB-OIDC during the ``hello`` handshake. -If there is a cached Access Token the ``JwtStepRequest`` SASL command will be used as the speculation command. -If there is no cached Access Token, the ``PrincipalStepRequest`` will be used as the speculation command. -The driver MUST NOT call the callback during speculative authentication. - -REQUEST_TOKEN_CALLBACK -`````````````````````` -Drivers that implement ``REQUEST_TOKEN_CALLBACK`` (i.e. the human authentication -flow) MUST provide a way for the ``REQUEST_TOKEN_CALLBACK`` to be either -automatically canceled, or to cancel itself. This can be as a timeout argument -to the callback, a cancellation context passed to the callback, or some other -language-appropriate mechanism. The timeout duration MUST be 5 minutes, to -account for the fact that there may be human interaction involved. This callback -is not subject to CSOT. - -Callbacks can be synchronous and/or asynchronous, depending on the driver -and/or language. Asynchronous callbacks should be preferred when other -operations in the driver use asynchronous functions. + # Check if we can use the access token from the Client Cache or if we need + # to fetch and cache a new access token from the OIDC provider. + access_token = client.oidc_cache.access_token + is_cache = True + if access_token is None + credential = oidc_provider() + is_cache = False + client.oidc_cache.access_token = credential.access_token -The driver MUST pass the following information to the callback: -``IdpInfo``, and either a ``timeoutSeconds`` or ``timeoutContext`` object for the callback. -The signature of the callback is up to the driver's discretion, but the driver MUST ensure that, -in the future, callbacks may have additional optional parameters passed to them. -An example might look like: + client.oidc_cache.unlock() -.. code: typescript + return access_token, is_cache - interface RequestParameters { - // Timeout in seconds for the callback. Optionally, timeoutContext instead if applicable to language. - timeoutSeconds: int; +Drivers MUST have a way to invalidate a specific access token from the *Client +Cache*. Invalidation MUST only clear the cached access token if it is the same +as the invalid access token and MUST be an atomic operation (e.g. using a mutex +or a compare-and-swap operation). - // The version of the callback parameter interface. - version: int; +Example code for invalidation: - // The refresh token, if applicable, to be used by the callback to request a new token from the issuer. - refreshToken: Optional; - } +.. code:: python -.. code: typescript + def invalidate(access_token): + client.oidc_cache.lock() - function onRequest(info: IdpInfo, params: RequestParameters): IdpResponse + if client.oidc_cache.access_token == access_token: + client.oidc_cache.access_token = None -Before calling the callback, the driver MUST acquire a lock unique to the ``MongoClient``. -The driver MUST ensure that credentials have not changed between when the lock was requested and when it was acquired. The lock MUST be released -when the callback call as finished or errored. -This is because the ``REQUEST_TOKEN_CALLBACK`` may involve human interaction, and refresh tokens might only be able to be used used once. + client.oidc_cache.unlock() -If the callback does not return an object in the correct form of ``IdpResponse``, the driver MUST raise an error either using the type -system or by raising an error when non-optional properties are missing . -The driver MUST NOT attempt to validate the token(s) directly. -It is expected that if the server changes the expected fields, the SASL exchange will be updated with a version parameter. -Drivers do not need to attempt to provide old-driver-new-server compatibility. +Drivers that support the `Human Authentication Flow`_ MUST also cache the +``IdPInfo`` and refresh token in the *Client Cache* when a `Human +Callback`_ is configured. + +Authentication +______________ +Use the following algorithm to authenticate a new connection: + +- Check if the the *Client Cache* has an access token. -If no callback is given, the driver MUST must raise a validation error. + - If it does, cache the access token in the *Connection Cache* and perform a + `One-Step` SASL conversation using the access token in the *Client Cache*. + If the server returns an error, invalidate that access token, sleep 100ms + then continue. + +- Call the configured built-in provider integration or the OIDC callback to + retrieve a new access token. +- Cache the new access token in the *Client Cache* and *Connection Cache*. +- Perform a `One-Step` SASL conversation using the new access token. + Raise any errors to the user. + +Example code to authenticate a connection using the ``get_access_token`` and +``invalidate`` functions described above: + +.. code:: python + + def auth(connection): + access_token, is_cache = get_access_token() + + # If there is a cached access token, try to authenticate with it. If + # authentication fails, it's possible the cached access token is expired. In + # that case, invalidate the access token, fetch a new access token, and try + # to authenticate again. + if is_cache: + try: + connection.oidc_cache.access_token = access_token + sasl_conversation(connection, {"jwt": access_token}) + return + except AuthenticationFailed: + invalidate(access_token) + sleep(0.1) + access_token, _ = get_access_token() + + connection.oidc_cache.access_token = access_token + sasl_conversation(connection, {"jwt": access_token}) + +For drivers that support the `Human Authentication Flow`_, use the following +algorithm to authenticate a new connection when a `Human Callback`_ is +configured: + +- Check if the *Client Cache* has an access token. + + - If it does, cache the access token in the *Connection Cache* and perform a + `One-Step`_ SASL conversation using that access token. If the server returns + an error, invalidate that access token token from the *Client Cache* and + *Connection Cache* and continue. + +- Check if the *Client Cache* has a refresh token. + + - If it does, call the `Human Callback`_ with the cached refresh token and + ``IdpInfo`` to get a new access token. Cache the new access token in the + *Client Cache* and *Connection Cache*. Perform a `One-Step`_ SASL + conversation using the new access token. If any errors occur, invalidate the + refresh token from the *Client Cache* and the access token from the *Client + Cache* and *Connection Cache* and continue. + +- Start a new `Two-Step`_ SASL conversation. +- Run a ``PrincipalStepRequest`` to get the ``IdpInfo``. +- Call the `Human Callback`_ with the new ``IdpInfo`` to get a new access token + and optional refresh token. +- Cache the new ``IdpInfo`` and refresh token in the *Client Cache* and the new + access token in the *Client Cache* and *Connection Cache*. +- Attempt to authenticate using a ``JwtStepRequest`` with the new access token. + Raise any errors to the user. + +Speculative Authentication +`````````````````````````` +Drivers MUST implement speculative authentication for MONGODB-OIDC during the +``hello`` handshake. Use the following algorithm to build the speculative +authentication document: + +- Check if the *Client Cache* has an access token. + + - If it does, send a ``JwtStepRequest`` with the cached access token in the + speculative authentication document. + +- Call the configured built-in provider integration or the OIDC callback to + retrieve a new access token. +- Cache the new access token in the *Client Cache* and *Connection Cache*. +- Send a ``JwtStepRequest`` with the new access token in the speculative + authentication document. + +For drivers that support the `Human Authentication Flow`_, use the following +algorithm to build the speculative authentication document when a `Human +Callback`_ is configured: + +- Check if the *Client Cache* has an access token. + + - If it does, send a ``JwtStepRequest`` with the cached access token in the + speculative authentication document. + +- Send a ``PrincipalStepRequest`` (with the ``username`` from the connection + string, if provided) in the speculative authentication document. + +Drivers MUST NOT call a `Human Callback`_ callback during speculative +authentication. Reauthentication ```````````````` -When reauthentication is requested by the server (as a 391 error code) and MONGODB-OIDC is in use, the driver MUST perform a reauthentication. -The driver MUST account for the case of multiple connections hitting a reauthentication error at different times. -The driver MUST also account for a reauthenication that results from an IdP configuration change on the server. -To accomplish these goal, the following algorithm is used to handle a reauthenication error: - -- First, see if the Access Token on the MongoClient is different than the Access Token on the connection. - - If they are different, optimisitically try to authenticate. On error, continue. -- Next, perform a ``PrincipalNameRequest`` to determine if the ``IdpInfo`` has changed. - - If it has changed, clear the current Access Token and Refresh Token, and continue. -- If we have a Refresh Token, attempt to authenticate. On error, clear the Refresh Token and continue. -- Attempt to authenticate. Raise any errors to the user. +If any operation fails with ``ReauthenticationRequired`` (error code 391) and +MONGODB-OIDC is in use, the driver MUST reauthenticate the connection. To +reauthenticate a connection, invalidate the access token stored on the +connection (i.e. the *Connection Cache*) from the *Client Cache*, fetch a new +access token, and re-run the SASL conversation. Drivers MUST NOT send a +``hello`` when reauthenticating a connection. + +Example code for reauthentication using the ``auth`` function described above: + +.. code:: python + + def reauth(conn): + invalidate(conn.oidc_cache.access_token) + conn.oidc_cache.access_token = None + auth(conn) ------------------------- Connection String Options diff --git a/source/mongodb-handshake/handshake.rst b/source/mongodb-handshake/handshake.rst index a2fb4002c0..0d99966374 100644 --- a/source/mongodb-handshake/handshake.rst +++ b/source/mongodb-handshake/handshake.rst @@ -414,7 +414,7 @@ structure as seen in the MONGODB-OIDC conversation section in the `Driver Authentication spec `_. However, the driver MUST not call a callback as part of -``speculativeAuthenticate`` during the `OIDC Human Authentication Flow <../auth/auth.rst#human-authentication-flow>`_. +``speculativeAuthenticate`` during the `Human Authentication Flow <../auth/auth.rst#human-authentication-flow>`_. If the initial handshake command with a ``speculativeAuthenticate`` argument succeeds, the client should proceed with the next step of the exchange. If the initial handshake From cd1470d7cd0ca28a3e94920b1b50d7dd7556b46a Mon Sep 17 00:00:00 2001 From: Matt Dale <9760375+matthewdale@users.noreply.github.com> Date: Wed, 17 Jan 2024 22:35:14 -0800 Subject: [PATCH 32/44] Combine OIDC and reauthentication spec tests. Add OIDC prose tests. --- source/auth/auth.rst | 43 ++- source/auth/tests/mongodb-oidc.rst | 248 +++++++----- .../tests/unified/mongodb-oidc-no-retry.json | 360 ++++++++++++++++++ .../tests/unified/mongodb-oidc-no-retry.yml | 184 +++++++++ ...ith_retry.json => mongodb-oidc-retry.json} | 62 ++- ...thout-retry.yml => mongodb-oidc-retry.yml} | 41 +- .../tests/unified/oidc-auth-with-retry.json | 170 --------- .../tests/unified/oidc-auth-with-retry.yml | 92 ----- .../unified/oidc-auth-without-retry.json | 175 --------- .../unified/reauthenticate_with_retry.yml | 104 ----- .../unified/reauthenticate_without_retry.json | 191 ---------- .../unified/reauthenticate_without_retry.yml | 104 ----- 12 files changed, 787 insertions(+), 987 deletions(-) create mode 100644 source/auth/tests/unified/mongodb-oidc-no-retry.json create mode 100644 source/auth/tests/unified/mongodb-oidc-no-retry.yml rename source/auth/tests/unified/{reauthenticate_with_retry.json => mongodb-oidc-retry.json} (75%) rename source/auth/tests/unified/{oidc-auth-without-retry.yml => mongodb-oidc-retry.yml} (69%) delete mode 100644 source/auth/tests/unified/oidc-auth-with-retry.json delete mode 100644 source/auth/tests/unified/oidc-auth-with-retry.yml delete mode 100644 source/auth/tests/unified/oidc-auth-without-retry.json delete mode 100644 source/auth/tests/unified/reauthenticate_with_retry.yml delete mode 100644 source/auth/tests/unified/reauthenticate_without_retry.json delete mode 100644 source/auth/tests/unified/reauthenticate_without_retry.yml diff --git a/source/auth/auth.rst b/source/auth/auth.rst index 4d12169d6f..0687503ffa 100644 --- a/source/auth/auth.rst +++ b/source/auth/auth.rst @@ -265,15 +265,17 @@ expired. Drivers MUST immediately attempt a reauthentication on the connection using suitable credentials, as specified by the particular authentication mechanism when this error is raised, and then re-attempt the operation. This attempt MUST be irrespective of whether the operation is considered retryable. -Drivers MUST NOT resend a hello message during reauthentication, instead using -SASL messages directly. Any errors that could not be recovered from during +Drivers MUST NOT resend a ``hello`` message during reauthentication, instead +using SASL messages directly. Any errors that could not be recovered from during reauthentication, or that were encountered during the subsequent re-attempt of -the operation MUST be raised to the user. Currently the only authentication -mechanism on the server that supports reauthentication is OIDC. See the OIDC -documentation on reauthentication for more details. Note that in order to -implement the unified spec tests for reauthentication, it may be necessary to -add reauthentication support for whichever auth mechanism is used when running -the authentication spec tests. +the operation MUST be raised to the user. + +Currently the only authentication mechanism on the server that supports +reauthentication is ``MONGODB-OIDC``. See the `MONGODB-OIDC`_ section on +reauthentication for more details. Note that in order to implement the unified +spec tests for reauthentication, it may be necessary to add reauthentication +support for whichever auth mechanism is used when running the authentication +spec tests. -------------------------------- Supported Authentication Methods @@ -1230,7 +1232,8 @@ Flow`_. ````````````````````````````` username - MAY be specified. Its meaning varies depending on the OIDC provider used. + MAY be specified. Its meaning varies depending on the OIDC provider + integration used. source MUST be "$external". Defaults to ``$external``. @@ -1734,20 +1737,23 @@ authentication. Reauthentication ```````````````` If any operation fails with ``ReauthenticationRequired`` (error code 391) and -MONGODB-OIDC is in use, the driver MUST reauthenticate the connection. To -reauthenticate a connection, invalidate the access token stored on the +MONGODB-OIDC is in use, the driver MUST reauthenticate the connection. Drivers +MUST NOT resend a ``hello`` message during reauthentication, instead using SASL +messages directly. See the main `reauthentication`_ section for more +information. + +To reauthenticate a connection, invalidate the access token stored on the connection (i.e. the *Connection Cache*) from the *Client Cache*, fetch a new -access token, and re-run the SASL conversation. Drivers MUST NOT send a -``hello`` when reauthenticating a connection. +access token, and re-run the SASL conversation. Example code for reauthentication using the ``auth`` function described above: .. code:: python - def reauth(conn): - invalidate(conn.oidc_cache.access_token) - conn.oidc_cache.access_token = None - auth(conn) + def reauth(connection): + invalidate(connection.oidc_cache.access_token) + connection.oidc_cache.access_token = None + auth(connection) ------------------------- Connection String Options @@ -1985,7 +1991,8 @@ Q: Should drivers support accessing Amazon EC2 instance metadata in Amazon ECS? Changelog ========= -:2023-11-01: Separated MONGODB-OIDC machine and human authentication flow specs. +:2024-01-17: Added MONGODB-OIDC machine auth flow spec and combine with human + auth flow specs. :2023-04-28: Added MONGODB-OIDC auth mechanism :2022-11-02: Require environment variables to be read dynamically. :2022-10-28: Recommend the use of AWS SDKs where available. diff --git a/source/auth/tests/mongodb-oidc.rst b/source/auth/tests/mongodb-oidc.rst index 56a3de7074..3938d1b20a 100644 --- a/source/auth/tests/mongodb-oidc.rst +++ b/source/auth/tests/mongodb-oidc.rst @@ -26,19 +26,23 @@ For example, if the selected AWS profile ID is "drivers-test", run: Prose Tests =========== -(1) Custom Callback -~~~~~~~~~~~~~~~~~~~ +Drivers MUST implement all prose tests in this section. -- Create a ``MongoClient`` configured with a custom OIDC callback that - implements the AWS provider logic. +(1) OIDC Callback Authentication +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +**1.1 Callback is called during authentication** + +- Create a ``MongoClient`` configured with an OIDC callback that implements the + AWS provider logic. - Perform a ``find`` operation that succeeds. +- Verify that the callback was called 1 time. - Close the client. -(2) Callback is called during reauthentication -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +**1.2 Callback is called during reauthentication** -- Create a ``MongoClient`` configured with a custom OIDC callback that - implements the AWS provider logic. +- Create a ``MongoClient`` configured with an OIDC callback that implements the + AWS provider logic. - Set a fail point for ``find`` commands of the form: .. code:: javascript @@ -61,44 +65,82 @@ Prose Tests handshake, and again during reauthentication). - Close the client. -(3) Authentication failures with cached tokens fetch a new token and retry -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +(2) OIDC Callback Validation +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +**2.1 Valid Callback Inputs** + +- Create a ``MongoClient`` with an OIDC callback that validates its inputs and + returns a valid access token. +- Perform a ``find`` operation that succeeds. +- Verify that the OIDC callback was called with the appropriate inputs, + including the timeout parameter if possible. Ensure that there are no + unexpected fields. +- Close the client. + +**2.2 OIDC Callback Returns Null** + +- Create a ``MongoClient`` with an OIDC callback that returns ``null``. +- Perform a ``find`` operation that fails. +- Close the client. + +**2.3 OIDC Callback Returns Missing Data** + +- Create a ``MongoClient`` with an OIDC callback that returns data not + conforming to the ``OIDCCredential`` with missing fields. +- Perform a ``find`` operation that fails. +- Close the client. + +**2.4 OIDC Callback Returns Invalid Data** + +- Create a ``MongoClient`` with an OIDC callback that returns data not + conforming to the ``OIDCCredential`` with extra fields. +- Perform a ``find`` operation that fails. +- Close the client. + +(3) Authentication Failure +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +**3.1 Authentication failure with cached tokens fetch a new token and retry** -- Create a ``MongoClient`` configured with ``retryReads=false`` and a custom - OIDC callback that implements the AWS provider logic. +- Create a ``MongoClient`` configured with ``retryReads=false`` and an OIDC + callback that implements the AWS provider logic. - Poison the cache with an invalid access token. - Perform a ``find`` operation that succeeds. - Verify that the callback was called 1 time. - Close the client. -(4) Authentication failures without cached tokens return an error -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +**3.2 Authentication failures without cached tokens return an error** -- Create a ``MongoClient`` configured with ``retryReads=false`` and a custom - OIDC callback that always returns invalid access tokens. +- Create a ``MongoClient`` configured with ``retryReads=false`` and an OIDC + callback that always returns invalid access tokens. - Perform a ``find`` operation that fails. - Verify that the callback was called 1 time. - Close the client. +(3) Reauthentication +~~~~~~~~~~~~~~~~~~~~ + +TODO: +- Custom callback and provider_name returns error +- Reauthentication succeeds on multiple connections + + ---------- Human Authentication Flow Prose Tests ===================================== -Drivers that implement the Human Authentication Flow MUST test the following scenarios: - -- ``Callback-Driven Auth`` -- ``Callback Validation`` -- ``Speculative Authentication`` -- ``Reauthentication`` -- ``Separate Connections Avoid Extra Callback Calls`` +Drivers that support the `Human Authentication Flow +<../auth/auth.rst#human-authentication-flow>`_ MUST implement all prose tests in +this section. Drivers MUST be able to authenticate against a server configured with either one or two configured identity providers. Note that typically the preconfigured Atlas Dev clusters are used for testing, -in Evergreen and localy. The URIs can be fetched from the ``drivers/oidc`` -Secrets vault, see `vault instructions`_. Use ``OIDC_ATLAS_URI_SINGLE`` for +in Evergreen and localy. The URIs can be fetched from the ``drivers/oidc`` +Secrets vault, see `vault instructions`_. Use ``OIDC_ATLAS_URI_SINGLE`` for ``MONGODB_URI_SINGLE`` and ``OIDC_ATLAS_URI_MULTI`` for ``OIDC_ATLAS_URI_MULTI``. @@ -110,15 +152,15 @@ for ``MONGODB_URI_MULTI`` because the other server is a secondary on a replica set, on port ``27018``. The default OIDC client used in the tests will be configured with -``MONGODB_URI_SINGLE`` and a valid request callback handler that returns the +``MONGODB_URI_SINGLE`` and a valid human callback handler that returns the ``test_user1`` local token in ``OIDC_TOKEN_DIR`` as the "access_token", and a dummy "refresh_token". .. _Local Testing: https://github.com/mongodb-labs/drivers-evergreen-tools/blob/master/.evergreen/auth_oidc/README.md#local-testing .. _vault instructions: https://wiki.corp.mongodb.com/display/DRIVERS/Using+AWS+Secrets+Manager+to+Store+Testing+Secrets -(1) Callback-Driven Auth -~~~~~~~~~~~~~~~~~~~~~~~~ +(1) Human Callback Authentication +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Drivers MUST be able to authenticate using OIDC callback(s) when there is one principal configured. @@ -126,31 +168,35 @@ is one principal configured. **1.1 Single Principal Implicit Username** - Create default OIDC client with `authMechanism=MONGODB-OIDC`. -- Perform a ``find`` operation. that succeeds. +- Perform a ``find`` operation that succeeds. - Close the client. **1.2 Single Principal Explicit Username** -- Create a client with ``MONGODB_URI_SINGLE``, a username of ``test_user1``, `authMechanism=MONGODB-OIDC`, and the OIDC request callback. +- Create a client with ``MONGODB_URI_SINGLE``, a username of ``test_user1``, + `authMechanism=MONGODB-OIDC`, and the OIDC human callback. - Perform a ``find`` operation that succeeds. - Close the client. **1.3 Multiple Principal User 1** -- Create a client with ``MONGODB_URI_MULTI``, a username of ``test_user1``, `authMechanism=MONGODB-OIDC`, and the OIDC request callback. +- Create a client with ``MONGODB_URI_MULTI``, a username of ``test_user1``, + `authMechanism=MONGODB-OIDC`, and the OIDC human callback. - Perform a ``find`` operation that succeeds. - Close the client. **1.4 Multiple Principal User 2** -- Create a request callback that reads in the generated ``test_user2`` token file. -- Create a client with ``MONGODB_URI_MULTI``, a username of ``test_user2``, `authMechanism=MONGODB-OIDC`, and the OIDC request callback. +- Create a human callback that reads in the generated ``test_user2`` token file. +- Create a client with ``MONGODB_URI_MULTI``, a username of ``test_user2``, + `authMechanism=MONGODB-OIDC`, and the OIDC human callback. - Perform a ``find`` operation that succeeds. - Close the client. **1.5 Multiple Principal No User** -- Create a client with ``MONGODB_URI_MULTI``, no username, `authMechanism=MONGODB-OIDC`, and the OIDC request callback. +- Create a client with ``MONGODB_URI_MULTI``, no username, + `authMechanism=MONGODB-OIDC`, and the OIDC human callback. - Assert that a ``find`` operation fails. - Close the client. @@ -159,37 +205,35 @@ is one principal configured. - Create a default OIDC client, with an ``ALLOWED_HOSTS`` that is an empty list. - Assert that a ``find`` operation fails with a client-side error. - Close the client. -- Create a client that uses the url ``mongodb://localhost/?authMechanism=MONGODB-OIDC&ignored=example.com`` a request callback, and an - ``ALLOWED_HOSTS`` that contains ``["example.com"]``. +- Create a client that uses the URL + ``mongodb://localhost/?authMechanism=MONGODB-OIDC&ignored=example.com``, a + human callback, and an ``ALLOWED_HOSTS`` that contains ``["example.com"]``. - Assert that a ``find`` operation fails with a client-side error. - Close the client. -(2) Callback Validation -~~~~~~~~~~~~~~~~~~~~~~~ +(2) Human Callback Validation +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -**2.1 Valid Callbacks** +**2.1 Valid Callback Inputs** -- Create request callback that validates its inputs and returns a valid token. -- Create a client that uses the above callbacks. -- Perform a ``find`` operation that succeeds. Verify that the request +- Create a ``MongoClient`` with a human callback that validates its inputs and + returns a valid access token. +- Perform a ``find`` operation that succeeds. Verify that the human callback was called with the appropriate inputs, including the timeout - parameter if possible. Ensure that there are no unexpected fields. + parameter if possible. Ensure that there are no unexpected fields. - Close the client. -**2.2 Request Callback Returns Null** +**2.3 Human Callback Returns Missing Data** -- Create a client with a request callback that returns ``null``. +- Create a ``MongoClient`` with a human callback that returns data not conforming to + the ``OIDCCredential`` with missing fields. - Perform a ``find`` operation that fails. - Close the client. -**2.3 Request Callback Returns Invalid Data** +**2.4 OIDC Callback Returns Invalid Data** -- Create a client with a request callback that returns data not conforming to - the ``OIDCRequestTokenResult`` with missing field(s). -- Perform a ``find`` operation that fails. -- Close the client. -- Create a client with a request callback that returns data not conforming to - the ``OIDCRequestTokenResult`` with extra field(s). +- Create a ``MongoClient`` with a human callback that returns data not + conforming to the ``OIDCCredential`` with extra fields. - Perform a ``find`` operation that fails. - Close the client. @@ -198,21 +242,19 @@ is one principal configured. We can only test the successful case, by verifying that ``saslStart`` is not called. -- Create a client with a request callback that returns a valid token. +- Create a ``MongoClient`` with a human callback that returns a valid token. - Set a fail point for ``saslStart`` commands of the form: .. code:: javascript { - "configureFailPoint": "failCommand", - "mode": { - "times": 2 - }, - "data": { - "failCommands": [ + configureFailPoint: "failCommand", + mode: "alwaysOn", + data: { + failCommands: [ "saslStart" ], - "errorCode": 18 + errorCode: 20 } } @@ -237,22 +279,22 @@ operation. events. If the driver does emit those events, ignore/filter them for the purposes of this test. - Perform a ``find`` operation that succeeds. -- Assert that the request callback has been called once. +- Assert that the human callback has been called once. - Clear the listener state if possible. - Force a reauthenication using a ``failCommand`` of the form: .. code:: javascript { - "configureFailPoint": "failCommand", - "mode": { - "times": 1 + configureFailPoint: "failCommand", + mode: { + times: 1 }, - "data": { - "failCommands": [ + data: { + failCommands: [ "find" ], - "errorCode": 391 + errorCode: 391 } } @@ -262,7 +304,7 @@ operation. remove the ``failCommand`` after the test to prevent leakage. - Perform another find operation that succeeds. -- Assert that the request callback has been called twice. +- Assert that the human callback has been called twice. - Assert that the ordering of list started events is [``find``], , ``find``. Note that if the listener stat could not be cleared then there will and be extra ``find`` command. @@ -272,62 +314,62 @@ operation. **4.2 Succeeds no refresh** -- Create a default OIDC client with a request callback that does not return +- Create a default OIDC client with a human callback that does not return a refresh token. - Perform a ``find`` operation that succeeds. -- Assert that the request callback has been called once. +- Assert that the human callback has been called once. - Force a reauthenication using a ``failCommand`` of the form: .. code:: javascript { - "configureFailPoint": "failCommand", - "mode": { - "times": 1 + configureFailPoint: "failCommand", + mode: { + times: 1 }, - "data": { - "failCommands": [ + data: { + failCommands: [ "find" ], - "errorCode": 391 + errorCode: 391 } } - Perform a ``find`` operation that succeeds. -- Assert that the request callback has been called twice. +- Assert that the human callback has been called twice. - Close the client. **4.3 Succeeds after refresh fails** - Create a default OIDC client. - Perform a ``find`` operation that succeeds. -- Assert that the request callback has been called once. +- Assert that the human callback has been called once. - Force a reauthenication using a ``failCommand`` of the form: .. code:: javascript { - "configureFailPoint": "failCommand", - "mode": { - "times": 2 + configureFailPoint: "failCommand", + mode: { + times: 2 }, - "data": { - "failCommands": [ + data: { + failCommands: [ "find", "saslContinue" ], - "errorCode": 391 + errorCode: 391 } } - Perform a ``find`` operation that succeeds. -- Assert that the request callback has been called three times. +- Assert that the human callback has been called three times. - Close the client. **4.4 Fails** - Create a default OIDC client. - Perform a find operation that succeeds (to force a speculative auth). -- Assert that the request callback has been called once. +- Assert that the human callback has been called once. - Force a reauthenication using a failCommand of the form: .. code:: javascript @@ -346,7 +388,7 @@ operation. } - Perform a find operation that fails. -- Assert that the request callback has been called twice. +- Assert that the human callback has been called twice. - Close the client. **4.5 Separate Connections Avoid Extra Callback Calls** @@ -356,32 +398,34 @@ two MongoClient objects, or ensure that the same MongoClient is used with two different connections. Otherwise, the test would have a race condition. If neither is possible, the test may be skipped. -- Create a request callback that returns valid, and ensure that we can record the number - of times the callback is called. -- Create two clients using the callbacks, or a single client and two connection objects. +- Create a human callback that returns valid, and ensure that we can record the + number of times the callback is called. +- Create two clients using the callbacks, or a single client and two connection + objects. - Peform a find operation on each client/connection that succeeds. - If using a single client, share the underlying cache between clients. -- Ensure that the request callback has been called twice. -- Force a reauthenication on the first client/connection using a ``failCommand`` of the - form: +- Ensure that the human callback has been called twice. +- Force a reauthenication on the first client/connection using a ``failCommand`` + of the form: .. code:: javascript { - "configureFailPoint": "failCommand", - "mode": { - "times": 1 + configureFailPoint: "failCommand", + mode: { + times: 1 }, - "data": { - "failCommands": [ + data: { + failCommands: [ "find" ], - "errorCode": 391 + errorCode: 391 } } - Perform a ``find`` operation that succeds. -- Ensure that the request callback has been called three times. -- Repeat the ``failCommand`` and ``find`` operation on the second client/connection. -- Ensure that the request callback has been called three times. +- Ensure that the human callback has been called three times. +- Repeat the ``failCommand`` and ``find`` operation on the second + client/connection. +- Ensure that the human callback has been called three times. - Close all clients/connections. diff --git a/source/auth/tests/unified/mongodb-oidc-no-retry.json b/source/auth/tests/unified/mongodb-oidc-no-retry.json new file mode 100644 index 0000000000..7b79a1c2a3 --- /dev/null +++ b/source/auth/tests/unified/mongodb-oidc-no-retry.json @@ -0,0 +1,360 @@ +{ + "description": "MONGODB-OIDC authentication with retry disabled", + "schemaVersion": "1.18", + "runOnRequirements": [ + { + "minServerVersion": "7.0", + "auth": true, + "authMechanism": "MONGODB-OIDC" + } + ], + "createEntities": [ + { + "client": { + "id": "authClient" + } + }, + { + "client": { + "id": "client0", + "uriOptions": { + "authMechanism": "MONGODB-OIDC", + "authMechanismProperties": { + "$$placeholder": 1 + }, + "retryReads": false, + "retryWrites": false + }, + "observeEvents": [ + "commandStartedEvent", + "commandSucceededEvent", + "commandFailedEvent" + ] + } + }, + { + "database": { + "id": "database0", + "client": "client0", + "databaseName": "test" + } + }, + { + "collection": { + "id": "collection0", + "database": "database0", + "collectionName": "collName" + } + } + ], + "initialData": [ + { + "collectionName": "collName", + "databaseName": "test", + "documents": [ + + ] + } + ], + "tests": [ + { + "description": "A read operation should succeed", + "operations": [ + { + "name": "find", + "arguments": { + "filter": { + } + }, + "object": "collection0", + "expectResult": [ + + ] + } + ], + "expectEvents": [ + { + "client": "client0", + "events": [ + { + "commandStartedEvent": { + "command": { + "find": "collName", + "filter": { + } + } + } + }, + { + "commandSucceededEvent": { + "commandName": "find" + } + } + ] + } + ] + }, + { + "description": "A write operation should succeed", + "operations": [ + { + "name": "insertOne", + "object": "collection0", + "arguments": { + "document": { + "_id": 1, + "x": 1 + } + } + } + ], + "expectEvents": [ + { + "client": "client0", + "events": [ + { + "commandStartedEvent": { + "command": { + "insert": "collName", + "documents": [ + { + "_id": 1, + "x": 1 + } + ] + } + } + }, + { + "commandSucceededEvent": { + "commandName": "insert" + } + } + ] + } + ] + }, + { + "description": "Read commands should reauthenticate and retry when a ReauthenticationRequired error happens", + "operations": [ + { + "name": "failPoint", + "object": "testRunner", + "arguments": { + "client": "client0", + "failPoint": { + "configureFailPoint": "failCommand", + "mode": { + "times": 1 + }, + "data": { + "failCommands": [ + "find" + ], + "errorCode": 391 + } + } + } + }, + { + "name": "find", + "arguments": { + "filter": { + } + }, + "object": "collection0", + "expectResult": [ + + ] + } + ], + "expectEvents": [ + { + "client": "client0", + "events": [ + { + "commandStartedEvent": { + "command": { + "find": "collName", + "filter": { + } + } + } + }, + { + "commandFailedEvent": { + "commandName": "find" + } + }, + { + "commandStartedEvent": { + "command": { + "find": "collName", + "filter": { + } + } + } + }, + { + "commandSucceededEvent": { + "commandName": "find" + } + } + ] + } + ] + }, + { + "description": "Write commands should reauthenticate and retry when a ReauthenticationRequired error happens", + "operations": [ + { + "name": "failPoint", + "object": "testRunner", + "arguments": { + "client": "client0", + "failPoint": { + "configureFailPoint": "failCommand", + "mode": { + "times": 1 + }, + "data": { + "failCommands": [ + "insert" + ], + "errorCode": 391 + } + } + } + }, + { + "name": "insertOne", + "object": "collection0", + "arguments": { + "document": { + "_id": 1, + "x": 1 + } + } + } + ], + "expectEvents": [ + { + "client": "client0", + "events": [ + { + "commandStartedEvent": { + "command": { + "insert": "collName", + "documents": [ + { + "_id": 1, + "x": 1 + } + ] + } + } + }, + { + "commandFailedEvent": { + "commandName": "insert" + } + }, + { + "commandStartedEvent": { + "command": { + "insert": "collName", + "documents": [ + { + "_id": 1, + "x": 1 + } + ] + } + } + }, + { + "commandSucceededEvent": { + "commandName": "insert" + } + } + ] + } + ] + }, + { + "description": "Handshake should use speculative authentication", + "operations": [ + { + "name": "failPoint", + "object": "testRunner", + "arguments": { + "client": "client0", + "failPoint": { + "configureFailPoint": "failCommand", + "mode": "alwaysOn", + "data": { + "failCommands": [ + "saslStart" + ], + "errorCode": 20 + } + } + } + }, + { + "name": "insertOne", + "object": "collection0", + "arguments": { + "document": { + "_id": 1, + "x": 1 + } + } + } + ], + "expectEvents": [ + { + "client": "client0", + "events": [ + { + "commandStartedEvent": { + "command": { + "insert": "collName", + "documents": [ + { + "_id": 1, + "x": 1 + } + ] + } + } + }, + { + "commandFailedEvent": { + "commandName": "insert" + } + }, + { + "commandStartedEvent": { + "command": { + "insert": "collName", + "documents": [ + { + "_id": 1, + "x": 1 + } + ] + } + } + }, + { + "commandSucceededEvent": { + "commandName": "insert" + } + } + ] + } + ] + } + ] +} diff --git a/source/auth/tests/unified/mongodb-oidc-no-retry.yml b/source/auth/tests/unified/mongodb-oidc-no-retry.yml new file mode 100644 index 0000000000..b3276a450d --- /dev/null +++ b/source/auth/tests/unified/mongodb-oidc-no-retry.yml @@ -0,0 +1,184 @@ +--- +description: "MONGODB-OIDC authentication with retry disabled" +schemaVersion: "1.18" +runOnRequirements: +- minServerVersion: "7.0" + auth: true + authMechanism: "MONGODB-OIDC" +createEntities: +- client: + id: authClient +- client: + id: client0 + uriOptions: + authMechanism: "MONGODB-OIDC" + # The $$placeholder document should be replaced by auth mechanism + # properties that enable OIDC auth on the target cloud platform. For + # example, when running the test on AWS, replace the $$placeholder + # document with {"PROVIDER_NAME": "aws"}. + authMechanismProperties: { $$placeholder: 1 } + retryReads: false + retryWrites: false + observeEvents: + - commandStartedEvent + - commandSucceededEvent + - commandFailedEvent +- database: + id: database0 + client: client0 + databaseName: test +- collection: + id: collection0 + database: database0 + collectionName: collName +initialData: +- collectionName: collName + databaseName: test + documents: [] +tests: +- description: A read operation should succeed + operations: + - name: find + arguments: + filter: {} + object: collection0 + expectResult: [] + expectEvents: + - client: client0 + events: + - commandStartedEvent: + command: + find: collName + filter: {} + - commandSucceededEvent: + commandName: find +- description: A write operation should succeed + operations: + - name: insertOne + object: collection0 + arguments: + document: + _id: 1 + x: 1 + expectEvents: + - client: client0 + events: + - commandStartedEvent: + command: + insert: collName + documents: + - _id: 1 + x: 1 + - commandSucceededEvent: + commandName: insert +- description: Read commands should reauthenticate and retry when a ReauthenticationRequired error happens + operations: + - name: failPoint + object: testRunner + arguments: + client: client0 + failPoint: + configureFailPoint: failCommand + mode: + times: 1 + data: + failCommands: + - find + errorCode: 391 # ReauthenticationRequired + - name: find + arguments: + filter: {} + object: collection0 + expectResult: [] + expectEvents: + - client: client0 + events: + - commandStartedEvent: + command: + find: collName + filter: {} + - commandFailedEvent: + commandName: find + - commandStartedEvent: + command: + find: collName + filter: {} + - commandSucceededEvent: + commandName: find +- description: Write commands should reauthenticate and retry when a ReauthenticationRequired error happens + operations: + - name: failPoint + object: testRunner + arguments: + client: client0 + failPoint: + configureFailPoint: failCommand + mode: + times: 1 + data: + failCommands: + - insert + errorCode: 391 # ReauthenticationRequired + - name: insertOne + object: collection0 + arguments: + document: + _id: 1 + x: 1 + expectEvents: + - client: client0 + events: + - commandStartedEvent: + command: + insert: collName + documents: + - _id: 1 + x: 1 + - commandFailedEvent: + commandName: insert + - commandStartedEvent: + command: + insert: collName + documents: + - _id: 1 + x: 1 + - commandSucceededEvent: + commandName: insert +- description: Handshake should use speculative authentication + operations: + - name: failPoint + object: testRunner + arguments: + client: client0 + failPoint: + configureFailPoint: failCommand + mode: "alwaysOn" + data: + failCommands: + - saslStart + errorCode: 20 # IllegalOperation + - name: insertOne + object: collection0 + arguments: + document: + _id: 1 + x: 1 + expectEvents: + - client: client0 + events: + - commandStartedEvent: + command: + insert: collName + documents: + - _id: 1 + x: 1 + - commandFailedEvent: + commandName: insert + - commandStartedEvent: + command: + insert: collName + documents: + - _id: 1 + x: 1 + - commandSucceededEvent: + commandName: insert diff --git a/source/auth/tests/unified/reauthenticate_with_retry.json b/source/auth/tests/unified/mongodb-oidc-retry.json similarity index 75% rename from source/auth/tests/unified/reauthenticate_with_retry.json rename to source/auth/tests/unified/mongodb-oidc-retry.json index e094a8b58b..1d7aed787c 100644 --- a/source/auth/tests/unified/reauthenticate_with_retry.json +++ b/source/auth/tests/unified/mongodb-oidc-retry.json @@ -1,17 +1,27 @@ { - "description": "reauthenticate_with_retry", - "schemaVersion": "1.3", + "description": "MONGODB-OIDC authentication with retry enabled", + "schemaVersion": "1.18", "runOnRequirements": [ { - "minServerVersion": "6.3", - "auth": true + "minServerVersion": "7.0", + "auth": true, + "authMechanism": "MONGODB-OIDC" } ], "createEntities": [ + { + "client": { + "id": "authClient" + } + }, { "client": { "id": "client0", "uriOptions": { + "authMechanism": "MONGODB-OIDC", + "authMechanismProperties": { + "$$placeholder": 1 + }, "retryReads": true, "retryWrites": true }, @@ -26,7 +36,7 @@ "database": { "id": "database0", "client": "client0", - "databaseName": "db" + "databaseName": "test" } }, { @@ -40,13 +50,15 @@ "initialData": [ { "collectionName": "collName", - "databaseName": "db", - "documents": [] + "databaseName": "test", + "documents": [ + + ] } ], "tests": [ { - "description": "Read command should reauthenticate when receive ReauthenticationRequired error code and retryReads=true", + "description": "Read commands should fail if reauthentication fails", "operations": [ { "name": "failPoint", @@ -56,11 +68,12 @@ "failPoint": { "configureFailPoint": "failCommand", "mode": { - "times": 1 + "times": 2 }, "data": { "failCommands": [ - "find" + "find", + "saslStart" ], "errorCode": 391 } @@ -69,11 +82,14 @@ }, { "name": "find", + "object": "collection0", "arguments": { - "filter": {} + "filter": { + } }, - "object": "collection0", - "expectResult": [] + "expectError": { + "errorCode": 391 + } } ], "expectEvents": [ @@ -84,7 +100,8 @@ "commandStartedEvent": { "command": { "find": "collName", - "filter": {} + "filter": { + } } } }, @@ -97,12 +114,13 @@ "commandStartedEvent": { "command": { "find": "collName", - "filter": {} + "filter": { + } } } }, { - "commandSucceededEvent": { + "commandFailedEvent": { "commandName": "find" } } @@ -111,7 +129,7 @@ ] }, { - "description": "Write command should reauthenticate when receive ReauthenticationRequired error code and retryWrites=true", + "description": "Write commands should fail if reauthentication fails", "operations": [ { "name": "failPoint", @@ -121,11 +139,12 @@ "failPoint": { "configureFailPoint": "failCommand", "mode": { - "times": 1 + "times": 2 }, "data": { "failCommands": [ - "insert" + "insert", + "saslStart" ], "errorCode": 391 } @@ -140,6 +159,9 @@ "_id": 1, "x": 1 } + }, + "expectError": { + "errorCode": 391 } } ], @@ -179,7 +201,7 @@ } }, { - "commandSucceededEvent": { + "commandFailedEvent": { "commandName": "insert" } } diff --git a/source/auth/tests/unified/oidc-auth-without-retry.yml b/source/auth/tests/unified/mongodb-oidc-retry.yml similarity index 69% rename from source/auth/tests/unified/oidc-auth-without-retry.yml rename to source/auth/tests/unified/mongodb-oidc-retry.yml index 16b4978d1d..bd62120098 100644 --- a/source/auth/tests/unified/oidc-auth-without-retry.yml +++ b/source/auth/tests/unified/mongodb-oidc-retry.yml @@ -1,5 +1,4 @@ ---- -description: "OIDC authentication without retry" +description: "MONGODB-OIDC authentication with retry enabled" schemaVersion: "1.18" runOnRequirements: - minServerVersion: "7.0" @@ -36,13 +35,26 @@ initialData: databaseName: test documents: [] tests: -- description: A simple find operation should succeed +- description: Read commands should fail if reauthentication fails operations: + - name: failPoint + object: testRunner + arguments: + client: client0 + failPoint: + configureFailPoint: failCommand + mode: + times: 2 + data: + failCommands: + - find + - saslStart + errorCode: 391 # ReauthenticationRequired - name: find + object: collection0 arguments: filter: {} - object: collection0 - expectResult: [] + expectError: { errorCode: 391 } expectEvents: - client: client0 events: @@ -50,10 +62,15 @@ tests: command: find: collName filter: {} - - commandSucceededEvent: + - commandFailedEvent: commandName: find -- description: Write command should reauthenticate when receive ReauthenticationRequired - error code and retryWrites=true + - commandStartedEvent: + command: + find: collName + filter: {} + - commandFailedEvent: + commandName: find +- description: Write commands should fail if reauthentication fails operations: - name: failPoint object: testRunner @@ -62,17 +79,19 @@ tests: failPoint: configureFailPoint: failCommand mode: - times: 1 + times: 2 data: failCommands: - insert - errorCode: 391 + - saslStart + errorCode: 391 # ReauthenticationRequired - name: insertOne object: collection0 arguments: document: _id: 1 x: 1 + expectError: { errorCode: 391 } expectEvents: - client: client0 events: @@ -90,5 +109,5 @@ tests: documents: - _id: 1 x: 1 - - commandSucceededEvent: + - commandFailedEvent: commandName: insert diff --git a/source/auth/tests/unified/oidc-auth-with-retry.json b/source/auth/tests/unified/oidc-auth-with-retry.json deleted file mode 100644 index aeae3288c9..0000000000 --- a/source/auth/tests/unified/oidc-auth-with-retry.json +++ /dev/null @@ -1,170 +0,0 @@ -{ - "description": "OIDC authentication with retry", - "schemaVersion": "1.18", - "runOnRequirements": [ - { - "minServerVersion": "7.0", - "auth": true, - "authMechanism": "MONGODB-OIDC" - } - ], - "createEntities": [ - { - "client": { - "id": "client0", - "uriOptions": { - "authMechanism": "MONGODB-OIDC", - "authMechanismProperties": { - "$$placeholder": 1 - }, - "retryReads": true, - "retryWrites": true - }, - "observeEvents": [ - "commandStartedEvent", - "commandSucceededEvent", - "commandFailedEvent" - ] - } - }, - { - "database": { - "id": "database0", - "client": "client0", - "databaseName": "test" - } - }, - { - "collection": { - "id": "collection0", - "database": "database0", - "collectionName": "collName" - } - } - ], - "initialData": [ - { - "collectionName": "collName", - "databaseName": "test", - "documents": [ - - ] - } - ], - "tests": [ - { - "description": "A simple find operation should succeed", - "operations": [ - { - "name": "find", - "arguments": { - "filter": { - } - }, - "object": "collection0", - "expectResult": [ - - ] - } - ], - "expectEvents": [ - { - "client": "client0", - "events": [ - { - "commandStartedEvent": { - "command": { - "find": "collName", - "filter": { - } - } - } - }, - { - "commandSucceededEvent": { - "commandName": "find" - } - } - ] - } - ] - }, - { - "description": "Write command should reauthenticate when receive ReauthenticationRequired error code and retryWrites=true", - "operations": [ - { - "name": "failPoint", - "object": "testRunner", - "arguments": { - "client": "client0", - "failPoint": { - "configureFailPoint": "failCommand", - "mode": { - "times": 1 - }, - "data": { - "failCommands": [ - "insert" - ], - "errorCode": 391 - } - } - } - }, - { - "name": "insertOne", - "object": "collection0", - "arguments": { - "document": { - "_id": 1, - "x": 1 - } - } - } - ], - "expectEvents": [ - { - "client": "client0", - "events": [ - { - "commandStartedEvent": { - "command": { - "insert": "collName", - "documents": [ - { - "_id": 1, - "x": 1 - } - ] - } - } - }, - { - "commandFailedEvent": { - "commandName": "insert" - } - }, - { - "commandStartedEvent": { - "command": { - "insert": "collName", - "documents": [ - { - "_id": 1, - "x": 1 - } - ] - } - } - }, - { - "commandSucceededEvent": { - "commandName": "insert" - } - } - ] - } - ] - } - ] -} diff --git a/source/auth/tests/unified/oidc-auth-with-retry.yml b/source/auth/tests/unified/oidc-auth-with-retry.yml deleted file mode 100644 index 47481d963d..0000000000 --- a/source/auth/tests/unified/oidc-auth-with-retry.yml +++ /dev/null @@ -1,92 +0,0 @@ ---- -description: "OIDC authentication with retry" -schemaVersion: "1.18" -runOnRequirements: -- minServerVersion: "7.0" - auth: true - authMechanism: "MONGODB-OIDC" -createEntities: -- client: - id: client0 - uriOptions: - authMechanism: "MONGODB-OIDC" - # The $$placeholder document should be replaced by auth mechanism - # properties that enable OIDC auth on the target cloud platform. For - # example, when running the test on AWS, replace the $$placeholder - # document with {"PROVIDER_NAME": "aws"}. - authMechanismProperties: { $$placeholder: 1 } - retryReads: true - retryWrites: true - observeEvents: - - commandStartedEvent - - commandSucceededEvent - - commandFailedEvent -- database: - id: database0 - client: client0 - databaseName: test -- collection: - id: collection0 - database: database0 - collectionName: collName -initialData: -- collectionName: collName - databaseName: test - documents: [] -tests: -- description: A simple find operation should succeed - operations: - - name: find - arguments: - filter: {} - object: collection0 - expectResult: [] - expectEvents: - - client: client0 - events: - - commandStartedEvent: - command: - find: collName - filter: {} - - commandSucceededEvent: - commandName: find -- description: Write command should reauthenticate when receive ReauthenticationRequired - error code and retryWrites=true - operations: - - name: failPoint - object: testRunner - arguments: - client: client0 - failPoint: - configureFailPoint: failCommand - mode: - times: 1 - data: - failCommands: - - insert - errorCode: 391 - - name: insertOne - object: collection0 - arguments: - document: - _id: 1 - x: 1 - expectEvents: - - client: client0 - events: - - commandStartedEvent: - command: - insert: collName - documents: - - _id: 1 - x: 1 - - commandFailedEvent: - commandName: insert - - commandStartedEvent: - command: - insert: collName - documents: - - _id: 1 - x: 1 - - commandSucceededEvent: - commandName: insert diff --git a/source/auth/tests/unified/oidc-auth-without-retry.json b/source/auth/tests/unified/oidc-auth-without-retry.json deleted file mode 100644 index ad8c93c03f..0000000000 --- a/source/auth/tests/unified/oidc-auth-without-retry.json +++ /dev/null @@ -1,175 +0,0 @@ -{ - "description": "OIDC authentication without retry", - "schemaVersion": "1.18", - "runOnRequirements": [ - { - "minServerVersion": "7.0", - "auth": true, - "authMechanism": "MONGODB-OIDC" - } - ], - "createEntities": [ - { - "client": { - "id": "authClient" - } - }, - { - "client": { - "id": "client0", - "uriOptions": { - "authMechanism": "MONGODB-OIDC", - "authMechanismProperties": { - "$$placeholder": 1 - }, - "retryReads": true, - "retryWrites": true - }, - "observeEvents": [ - "commandStartedEvent", - "commandSucceededEvent", - "commandFailedEvent" - ] - } - }, - { - "database": { - "id": "database0", - "client": "client0", - "databaseName": "test" - } - }, - { - "collection": { - "id": "collection0", - "database": "database0", - "collectionName": "collName" - } - } - ], - "initialData": [ - { - "collectionName": "collName", - "databaseName": "test", - "documents": [ - - ] - } - ], - "tests": [ - { - "description": "A simple find operation should succeed", - "operations": [ - { - "name": "find", - "arguments": { - "filter": { - } - }, - "object": "collection0", - "expectResult": [ - - ] - } - ], - "expectEvents": [ - { - "client": "client0", - "events": [ - { - "commandStartedEvent": { - "command": { - "find": "collName", - "filter": { - } - } - } - }, - { - "commandSucceededEvent": { - "commandName": "find" - } - } - ] - } - ] - }, - { - "description": "Write command should reauthenticate when receive ReauthenticationRequired error code and retryWrites=true", - "operations": [ - { - "name": "failPoint", - "object": "testRunner", - "arguments": { - "client": "client0", - "failPoint": { - "configureFailPoint": "failCommand", - "mode": { - "times": 1 - }, - "data": { - "failCommands": [ - "insert" - ], - "errorCode": 391 - } - } - } - }, - { - "name": "insertOne", - "object": "collection0", - "arguments": { - "document": { - "_id": 1, - "x": 1 - } - } - } - ], - "expectEvents": [ - { - "client": "client0", - "events": [ - { - "commandStartedEvent": { - "command": { - "insert": "collName", - "documents": [ - { - "_id": 1, - "x": 1 - } - ] - } - } - }, - { - "commandFailedEvent": { - "commandName": "insert" - } - }, - { - "commandStartedEvent": { - "command": { - "insert": "collName", - "documents": [ - { - "_id": 1, - "x": 1 - } - ] - } - } - }, - { - "commandSucceededEvent": { - "commandName": "insert" - } - } - ] - } - ] - } - ] -} diff --git a/source/auth/tests/unified/reauthenticate_with_retry.yml b/source/auth/tests/unified/reauthenticate_with_retry.yml deleted file mode 100644 index 7cbae968f5..0000000000 --- a/source/auth/tests/unified/reauthenticate_with_retry.yml +++ /dev/null @@ -1,104 +0,0 @@ ---- -description: reauthenticate_with_retry -schemaVersion: '1.3' -runOnRequirements: -- minServerVersion: '6.3' - auth: true -createEntities: -- client: - id: client0 - uriOptions: - retryReads: true - retryWrites: true - observeEvents: - - commandStartedEvent - - commandSucceededEvent - - commandFailedEvent -- database: - id: database0 - client: client0 - databaseName: db -- collection: - id: collection0 - database: database0 - collectionName: collName -initialData: -- collectionName: collName - databaseName: db - documents: [] -tests: -- description: Read command should reauthenticate when receive ReauthenticationRequired - error code and retryReads=true - operations: - - name: failPoint - object: testRunner - arguments: - client: client0 - failPoint: - configureFailPoint: failCommand - mode: - times: 1 - data: - failCommands: - - find - errorCode: 391 - - name: find - arguments: - filter: {} - object: collection0 - expectResult: [] - expectEvents: - - client: client0 - events: - - commandStartedEvent: - command: - find: collName - filter: {} - - commandFailedEvent: - commandName: find - - commandStartedEvent: - command: - find: collName - filter: {} - - commandSucceededEvent: - commandName: find -- description: Write command should reauthenticate when receive ReauthenticationRequired - error code and retryWrites=true - operations: - - name: failPoint - object: testRunner - arguments: - client: client0 - failPoint: - configureFailPoint: failCommand - mode: - times: 1 - data: - failCommands: - - insert - errorCode: 391 - - name: insertOne - object: collection0 - arguments: - document: - _id: 1 - x: 1 - expectEvents: - - client: client0 - events: - - commandStartedEvent: - command: - insert: collName - documents: - - _id: 1 - x: 1 - - commandFailedEvent: - commandName: insert - - commandStartedEvent: - command: - insert: collName - documents: - - _id: 1 - x: 1 - - commandSucceededEvent: - commandName: insert diff --git a/source/auth/tests/unified/reauthenticate_without_retry.json b/source/auth/tests/unified/reauthenticate_without_retry.json deleted file mode 100644 index 8bbc5cc64d..0000000000 --- a/source/auth/tests/unified/reauthenticate_without_retry.json +++ /dev/null @@ -1,191 +0,0 @@ -{ - "description": "reauthenticate_without_retry", - "schemaVersion": "1.3", - "runOnRequirements": [ - { - "minServerVersion": "6.3", - "auth": true - } - ], - "createEntities": [ - { - "client": { - "id": "client0", - "uriOptions": { - "retryReads": false, - "retryWrites": false - }, - "observeEvents": [ - "commandStartedEvent", - "commandSucceededEvent", - "commandFailedEvent" - ] - } - }, - { - "database": { - "id": "database0", - "client": "client0", - "databaseName": "db" - } - }, - { - "collection": { - "id": "collection0", - "database": "database0", - "collectionName": "collName" - } - } - ], - "initialData": [ - { - "collectionName": "collName", - "databaseName": "db", - "documents": [] - } - ], - "tests": [ - { - "description": "Read command should reauthenticate when receive ReauthenticationRequired error code and retryReads=false", - "operations": [ - { - "name": "failPoint", - "object": "testRunner", - "arguments": { - "client": "client0", - "failPoint": { - "configureFailPoint": "failCommand", - "mode": { - "times": 1 - }, - "data": { - "failCommands": [ - "find" - ], - "errorCode": 391 - } - } - } - }, - { - "name": "find", - "arguments": { - "filter": {} - }, - "object": "collection0", - "expectResult": [] - } - ], - "expectEvents": [ - { - "client": "client0", - "events": [ - { - "commandStartedEvent": { - "command": { - "find": "collName", - "filter": {} - } - } - }, - { - "commandFailedEvent": { - "commandName": "find" - } - }, - { - "commandStartedEvent": { - "command": { - "find": "collName", - "filter": {} - } - } - }, - { - "commandSucceededEvent": { - "commandName": "find" - } - } - ] - } - ] - }, - { - "description": "Write command should reauthenticate when receive ReauthenticationRequired error code and retryWrites=false", - "operations": [ - { - "name": "failPoint", - "object": "testRunner", - "arguments": { - "client": "client0", - "failPoint": { - "configureFailPoint": "failCommand", - "mode": { - "times": 1 - }, - "data": { - "failCommands": [ - "insert" - ], - "errorCode": 391 - } - } - } - }, - { - "name": "insertOne", - "object": "collection0", - "arguments": { - "document": { - "_id": 1, - "x": 1 - } - } - } - ], - "expectEvents": [ - { - "client": "client0", - "events": [ - { - "commandStartedEvent": { - "command": { - "insert": "collName", - "documents": [ - { - "_id": 1, - "x": 1 - } - ] - } - } - }, - { - "commandFailedEvent": { - "commandName": "insert" - } - }, - { - "commandStartedEvent": { - "command": { - "insert": "collName", - "documents": [ - { - "_id": 1, - "x": 1 - } - ] - } - } - }, - { - "commandSucceededEvent": { - "commandName": "insert" - } - } - ] - } - ] - } - ] -} diff --git a/source/auth/tests/unified/reauthenticate_without_retry.yml b/source/auth/tests/unified/reauthenticate_without_retry.yml deleted file mode 100644 index 7b53f29135..0000000000 --- a/source/auth/tests/unified/reauthenticate_without_retry.yml +++ /dev/null @@ -1,104 +0,0 @@ ---- -description: reauthenticate_without_retry -schemaVersion: '1.3' -runOnRequirements: -- minServerVersion: '6.3' - auth: true -createEntities: -- client: - id: client0 - uriOptions: - retryReads: false - retryWrites: false - observeEvents: - - commandStartedEvent - - commandSucceededEvent - - commandFailedEvent -- database: - id: database0 - client: client0 - databaseName: db -- collection: - id: collection0 - database: database0 - collectionName: collName -initialData: -- collectionName: collName - databaseName: db - documents: [] -tests: -- description: Read command should reauthenticate when receive ReauthenticationRequired - error code and retryReads=false - operations: - - name: failPoint - object: testRunner - arguments: - client: client0 - failPoint: - configureFailPoint: failCommand - mode: - times: 1 - data: - failCommands: - - find - errorCode: 391 - - name: find - arguments: - filter: {} - object: collection0 - expectResult: [] - expectEvents: - - client: client0 - events: - - commandStartedEvent: - command: - find: collName - filter: {} - - commandFailedEvent: - commandName: find - - commandStartedEvent: - command: - find: collName - filter: {} - - commandSucceededEvent: - commandName: find -- description: Write command should reauthenticate when receive ReauthenticationRequired - error code and retryWrites=false - operations: - - name: failPoint - object: testRunner - arguments: - client: client0 - failPoint: - configureFailPoint: failCommand - mode: - times: 1 - data: - failCommands: - - insert - errorCode: 391 - - name: insertOne - object: collection0 - arguments: - document: - _id: 1 - x: 1 - expectEvents: - - client: client0 - events: - - commandStartedEvent: - command: - insert: collName - documents: - - _id: 1 - x: 1 - - commandFailedEvent: - commandName: insert - - commandStartedEvent: - command: - insert: collName - documents: - - _id: 1 - x: 1 - - commandSucceededEvent: - commandName: insert From bb0946956340f379bda208b5108cdd7ffd0efc77 Mon Sep 17 00:00:00 2001 From: Matt Dale <9760375+matthewdale@users.noreply.github.com> Date: Thu, 18 Jan 2024 09:20:27 -0800 Subject: [PATCH 33/44] Fix master merge. --- source/auth/tests/mongodb-oidc.rst | 66 +----------------------------- 1 file changed, 2 insertions(+), 64 deletions(-) diff --git a/source/auth/tests/mongodb-oidc.rst b/source/auth/tests/mongodb-oidc.rst index 9dcb0693d1..4854062fee 100644 --- a/source/auth/tests/mongodb-oidc.rst +++ b/source/auth/tests/mongodb-oidc.rst @@ -327,70 +327,8 @@ operation. mode: { times: 1 }, -<<<<<<< HEAD data: { failCommands: [ -======= - "data": { - "failCommands": [ - "find", "saslStart" - ], - "errorCode": 391 - } - } - -- Perform a ``find`` operation that succeeds. -- Close the client. - -Retries and Fails with no Cache -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -- Clear the cache. -- Create request and refresh callbacks that return valid credentials - that will not expire soon. -- Perform a ``find`` operation that succeeds (to force a speculative auth). -- Clear the cache. -- Force a reauthenication using a ``failCommand`` of the form: - -.. code:: javascript - - { - "configureFailPoint": "failCommand", - "mode": { - "times": 2 - }, - "data": { - "failCommands": [ - "find", "saslStart" - ], - "errorCode": 391 - } - } - -- Perform a ``find`` operation that fails. -- Close the client. - -Separate Connections Avoid Extra Callback Calls -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -- Clear the cache. -- Create request and refresh callbacks that return tokens that will not expire - soon. Ensure that we can record the number of times each callback is called. -- Create two clients using the callbacks -- Perform a find operation on each client that succeeds. -- Ensure that the request callback has been called once and the refresh - callback has not been called. -- Force a reauthenication on the first client using a ``failCommand`` of the - form: - -.. code:: javascript - - { - "configureFailPoint": "failCommand", - "mode": { - "times": 1 - }, - "data": { - "failCommands": [ ->>>>>>> master "find" ], errorCode: 391 @@ -464,7 +402,7 @@ If neither is possible, the test may be skipped. number of times the callback is called. - Create two clients using the callbacks, or a single client and two connection objects. -- Peform a find operation on each client/connection that succeeds. +- Perform a find operation on each client/connection that succeeds. - If using a single client, share the underlying cache between clients. - Ensure that the human callback has been called twice. - Force a reauthenication on the first client/connection using a ``failCommand`` @@ -485,7 +423,7 @@ If neither is possible, the test may be skipped. } } -- Perform a ``find`` operation that succeds. +- Perform a ``find`` operation that succeeds. - Ensure that the human callback has been called three times. - Repeat the ``failCommand`` and ``find`` operation on the second client/connection. From f2135be18445909d9ccf1564330f4611f8652b6e Mon Sep 17 00:00:00 2001 From: Matt Dale <9760375+matthewdale@users.noreply.github.com> Date: Fri, 19 Jan 2024 17:47:22 -0800 Subject: [PATCH 34/44] Complete OIDC prose tests. PR feedback. --- source/auth/auth.rst | 100 +++++++++------- source/auth/tests/mongodb-oidc.rst | 179 ++++++++++++----------------- 2 files changed, 133 insertions(+), 146 deletions(-) diff --git a/source/auth/auth.rst b/source/auth/auth.rst index 38366bbc0d..02d12fa7ff 100644 --- a/source/auth/auth.rst +++ b/source/auth/auth.rst @@ -271,7 +271,7 @@ reauthentication, or that were encountered during the subsequent re-attempt of the operation MUST be raised to the user. Currently the only authentication mechanism on the server that supports -reauthentication is ``MONGODB-OIDC``. See the `MONGODB-OIDC`_ section on +reauthentication is MONGODB-OIDC. See the `MONGODB-OIDC`_ section on reauthentication for more details. Note that in order to implement the unified spec tests for reauthentication, it may be necessary to add reauthentication support for whichever auth mechanism is used when running the authentication @@ -1196,6 +1196,20 @@ machine-to-machine ("machine") and human-in-the-loop ("human"). Drivers MUST support the machine authentication flow. Drivers MAY support the human authentication flow. +The MONGODB-OIDC specification refers to the following OIDC concepts: + +- **Identity Provider (IdP)**: A service that manages user accounts and + authenticates users or applications, such as Okta or OneLogin. In the `Human + Authentication Flow`_, the `Human Callback`_ interacts directly the IdP. In + the `Machine Authentication Flow`_, only the MongoDB server interacts directly + the IdP. +- **Access token**: Used to authenticate requests to protected resources. OIDC + access tokens are signed JWT strings. +- **Refresh token**: Some OIDC providers may return a refresh token in addition + to an access token. A refresh token can be used to retrieve new access tokens + without requiring a human to re-authorize the application. Refresh tokens are + typically only supported by the `Human Authentication Flow`_. + Machine Authentication Flow ``````````````````````````` The machine authentication flow is intended to be used in cases where human @@ -1203,7 +1217,7 @@ interaction is not necessary or practical, such as to authenticate database access for a web service. Some OIDC documentation refers to the machine authentication flow as "workload authentication". -Drivers MUST implement all behaviors described in the ``MONGODB-OIDC`` +Drivers MUST implement all behaviors described in the MONGODB-OIDC specification, unless the section or block specifically says that it only applies to the `Human Authentication Flow`_. @@ -1214,17 +1228,8 @@ involve direct human interaction, such as database tools or CLIs. Some OIDC documentation refers to the human authentication flow as "workforce authentication". -In addition to the **access token**, the human authentication flow introduces -some additional concepts: - -- **Identity Provider (IdP)**: A service that manages user accounts and - authenticates users, such as Okta or OneLogin. -- **Refresh token**: Some OIDC providers may return a refresh token in addition - to an access token. A refresh token can be used to retrieve new access tokens - without requiring a human to re-authorize the application. - Drivers that support the `Human Authentication Flow`_ MUST implement all -behaviors described in the ``MONGODB-OIDC`` specification, including sections or +behaviors described in the MONGODB-OIDC specification, including sections or blocks that specifically say that it only applies the `Human Authentication Flow`_. @@ -1253,7 +1258,7 @@ mechanism_properties configuration), the driver MUST raise an error. CALLBACK - An `OIDC Callback` that returns credentials for OIDC providers that do + An `OIDC Callback`_ that returns credentials for OIDC providers that do not have a built-in integraiton. Drivers MAY allow the user to specify an `OIDC Callback`_ using a ``MongoClient`` configuration instead of a mechanism property, depending on what is conventional for the driver. @@ -1261,9 +1266,13 @@ mechanism_properties the ``MongoClient`` configuration. CALLBACK_TYPE - The type of `OIDC Callback`. The value MUST be one of ["machine", - "human"]. If an `OIDC Callback`_ is configured and ``CALLBACK_TYPE`` is - not specified, the driver MUST raise an error. This property is only + The type of `OIDC Callback`_. The value MUST be one of ["machine", + "human"]. Drivers MUST NOT allow the user to specify ``CALLBACK_TYPE`` + in the connection string. Drivers MAY allow the user to specify the + callback type using a ``MongoClient`` configuration instead of a + mechanism property to be consistent with how the `OIDC Callback`_ is + configured. If an `OIDC Callback`_ is configured and ``CALLBACK_TYPE`` + is not specified, the driver MUST raise an error. This property is only required for drivers that support the `Human Authentication Flow`_. ALLOWED_HOSTS @@ -1273,7 +1282,7 @@ mechanism_properties ``ALLOWED_HOSTS`` is a security feature and MUST default to ``["*.mongodb.net", "*.mongodb-qa.net", "*.mongodb-dev.net", "*.mongodbgov.net", "localhost", "127.0.0.1", "::1"]``. When - ``MONGODB-OIDC`` authentication using a `Human Callback`_ is attempted + MONGODB-OIDC authentication using a `Human Callback`_ is attempted against a hostname that does not match any of list of allowed hosts, the driver MUST raise a client-side error without invoking any user-provided callbacks. This value MUST NOT be allowed in the URI connection string. @@ -1290,10 +1299,10 @@ ___ The AWS provider is enabled by setting auth mechanism property ``PROVIDER_NAME:aws``. -If enabled, drivers MUST read the file path from -environment variable ``AWS_WEB_IDENTITY_TOKEN_FILE`` and then read the OIDC -Access Token from that file. The driver MUST use the contents of that file as -value in the ``jwt`` field of the ``saslStart`` payload. +If enabled, drivers MUST read the file path from environment variable +``AWS_WEB_IDENTITY_TOKEN_FILE`` and then read the OIDC access token from that +file. The driver MUST use the contents of that file as value in the ``jwt`` +field of the ``saslStart`` payload. Drivers MAY implement the AWS provider so that it conforms to the function signature of the `OIDC Callback`_ to prevent having to re-implement the AWS @@ -1320,7 +1329,9 @@ The driver MUST pass the following information to the callback: - ``version``: The callback API version number. The version number is used to communicate callback API changes that are not breaking but that users may want to know about and review their implementation. Drivers MUST pass ``1`` for the - current callback API version number. + initial callback API version number and increment the version number anytime + the API changes. Note that this may eventually lead to some drivers having + different callback version numbers. For example, users may add the following check in their callback: @@ -1332,11 +1343,12 @@ The driver MUST pass the following information to the callback: The callback MUST be able to return the following information: -- ``accessToken``: An OIDC access Token string. The driver MUST NOT attempt to +- ``accessToken``: An OIDC access token string. The driver MUST NOT attempt to validate ``accessToken`` directly. -- ``expiresIn``: An optional expiry time for the access Token. Drivers MUST support and - document values for both an expiry time and a value that indicates there is no - expiry time, like 0 or ``null``. +- ``expiresInSeconds``: An optional expiry duration for the access token. + Drivers MUST support and document values for both an expiry duration and a + value that indicates the expiry duration is unknown or infinite, like 0 or + ``null``. The signature of the callback is up to the driver's discretion, but the driver MUST ensure that additional optional input parameters and return values can be @@ -1360,9 +1372,9 @@ look like: Human Callback ______________ Drivers that support the `Human Authentication Flow`_ MUST implement the human -callback version of the `OIDC Callback`. +callback version of the `OIDC Callback`_. -In addition to the information described in the `OIDC Callback` section, +In addition to the information described in the `OIDC Callback`_ section, drivers MUST be able to pass the following information to the callback: - ``idpInfo``: Information used to authenticate with the IdP. @@ -1376,7 +1388,7 @@ drivers MUST be able to pass the following information to the callback: - ``refreshToken``: The refresh token, if applicable, to be used by the callback to request a new token from the issuer. -In addition to the information described in the `OIDC Callback` section, the +In addition to the information described in the `OIDC Callback`_ section, the callback MUST be able to return the following information: - ``refreshToken``: An optional refresh token that can be used to fetch new @@ -1396,13 +1408,13 @@ An example callback API that supports the human callback might look like: callbackTimeoutMS: int; version: int; idpInfo: Optional; - refreshToken: Optional; + refreshToken: Optional; } interface OIDCCredential { accessToken: string; - refreshToken: Optional; expiresInSeconds: Optional; + refreshToken: Optional; } function oidcCallback(params: OIDCCallbackParams): OIDCCredential @@ -1411,8 +1423,13 @@ Users enable the human callback behavior by setting mechanism property ``CALLBACK_TYPE:human``. When the human callback behavior is enabled, drivers MUST use the following behaviors when calling the callback: -- The driver MUST pass the ``IdpInfo`` and the ``refreshToken`` (if available) +- The driver MUST pass the ``IdpInfo`` and the refresh token (if available) to the callback. + + - If there is no cached ``IdpInfo``, drivers MUST start a `Two-Step`_ + conversation before calling the human callback. See the Conversation and + `Credential Caching`_ sections for more details. + - The timeout duration MUST be 5 minutes. This is to account for the human interaction required to complete the callback. In this case, the callback is not subject to CSOT. @@ -1434,7 +1451,7 @@ authentication may provide an access token via a local file pre-loaded on an application host. Drivers MUST use a one-step conversation when using a cached access token, one -of the `Built-in Provider Integrations`_, an `OIDC Callback` (not a `Human +of the `Built-in Provider Integrations`_, an `OIDC Callback`_ (not a `Human Callback`_). The one-step conversation starts with a ``saslStart`` containing a @@ -1663,7 +1680,7 @@ Example code to authenticate a connection using the ``get_access_token`` and if is_cache: try: connection.oidc_cache.access_token = access_token - sasl_conversation(connection, {"jwt": access_token}) + sasl_start(connection, {"jwt": access_token}) return except AuthenticationFailed: invalidate(access_token) @@ -1671,7 +1688,7 @@ Example code to authenticate a connection using the ``get_access_token`` and access_token, _ = get_access_token() connection.oidc_cache.access_token = access_token - sasl_conversation(connection, {"jwt": access_token}) + sasl_start(connection, {"jwt": access_token}) For drivers that support the `Human Authentication Flow`_, use the following algorithm to authenticate a new connection when a `Human Callback`_ is @@ -1680,9 +1697,9 @@ configured: - Check if the *Client Cache* has an access token. - If it does, cache the access token in the *Connection Cache* and perform a - `One-Step`_ SASL conversation using that access token. If the server returns - an error, invalidate that access token token from the *Client Cache* and - *Connection Cache* and continue. + `One-Step`_ SASL conversation using the access token. If the server returns + an error, invalidate the access token token from the *Client Cache*, clear + the *Connection Cache*, and continue. - Check if the *Client Cache* has a refresh token. @@ -1690,13 +1707,14 @@ configured: ``IdpInfo`` to get a new access token. Cache the new access token in the *Client Cache* and *Connection Cache*. Perform a `One-Step`_ SASL conversation using the new access token. If any errors occur, invalidate the - refresh token from the *Client Cache* and the access token from the *Client - Cache* and *Connection Cache* and continue. + access token from the *Client Cache*, clear the *Connection Cache*, and + continue. - Start a new `Two-Step`_ SASL conversation. - Run a ``PrincipalStepRequest`` to get the ``IdpInfo``. - Call the `Human Callback`_ with the new ``IdpInfo`` to get a new access token - and optional refresh token. + and optional refresh token. Drivers MUST NOT pass a cached refresh token to + the callback when performing a new `Two-Step`_ conversation. - Cache the new ``IdpInfo`` and refresh token in the *Client Cache* and the new access token in the *Client Cache* and *Connection Cache*. - Attempt to authenticate using a ``JwtStepRequest`` with the new access token. diff --git a/source/auth/tests/mongodb-oidc.rst b/source/auth/tests/mongodb-oidc.rst index 4854062fee..a7b289ad5b 100644 --- a/source/auth/tests/mongodb-oidc.rst +++ b/source/auth/tests/mongodb-oidc.rst @@ -6,7 +6,7 @@ Local Testing ============= To test locally, use the `oidc_get_tokens.sh`_ script from -drivers-evergreen-tools_ to download a set of OIDC tokens, including +`drivers-evergreen-tools`_ to download a set of OIDC tokens, including `test_user1` and `test_user1_expires`. You first have to install the AWS CLI and login using the SSO flow. @@ -28,6 +28,12 @@ Prose Tests Drivers MUST implement all prose tests in this section. +.. note:: + + For test cases that create fail points, drivers MUST either use a unique + ``appName`` or explicitly remove the fail point after the test to prevent + interaction between test cases. + (1) OIDC Callback Authentication ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -36,33 +42,16 @@ Drivers MUST implement all prose tests in this section. - Create a ``MongoClient`` configured with an OIDC callback that implements the AWS provider logic. - Perform a ``find`` operation that succeeds. -- Verify that the callback was called 1 time. +- Assert that the callback was called 1 time. - Close the client. -**1.2 Callback is called during reauthentication** +**1.2 Callback is called once for multiple connections** - Create a ``MongoClient`` configured with an OIDC callback that implements the AWS provider logic. -- Set a fail point for ``find`` commands of the form: - -.. code:: javascript - - { - configureFailPoint: "failCommand", - mode: { - times: 1 - }, - data: { - failCommands: [ - "find" - ], - errorCode: 391 - } - } - -- Perform a ``find`` operation that succeeds. -- Verify that the callback was called 2 times (once during the connection - handshake, and again during reauthentication). +- Start 10 threads and run 100 ``find`` operations in each thread that all + succeed. +- Assert that the callback was called 1 time. - Close the client. (2) OIDC Callback Validation @@ -70,34 +59,41 @@ Drivers MUST implement all prose tests in this section. **2.1 Valid Callback Inputs** -- Create a ``MongoClient`` with an OIDC callback that validates its inputs and - returns a valid access token. +- Create a ``MongoClient`` configured with an OIDC callback that validates its + inputs and returns a valid access token. - Perform a ``find`` operation that succeeds. -- Verify that the OIDC callback was called with the appropriate inputs, +- Assert that the OIDC callback was called with the appropriate inputs, including the timeout parameter if possible. Ensure that there are no unexpected fields. - Close the client. **2.2 OIDC Callback Returns Null** -- Create a ``MongoClient`` with an OIDC callback that returns ``null``. +- Create a ``MongoClient`` configured with an OIDC callback that returns + ``null``. - Perform a ``find`` operation that fails. - Close the client. **2.3 OIDC Callback Returns Missing Data** -- Create a ``MongoClient`` with an OIDC callback that returns data not - conforming to the ``OIDCCredential`` with missing fields. +- Create a ``MongoClient`` configured with an OIDC callback that returns data + not conforming to the ``OIDCCredential`` with missing fields. - Perform a ``find`` operation that fails. - Close the client. **2.4 OIDC Callback Returns Invalid Data** -- Create a ``MongoClient`` with an OIDC callback that returns data not - conforming to the ``OIDCCredential`` with extra fields. +- Create a ``MongoClient`` configured with an OIDC callback that returns data + not conforming to the ``OIDCCredential`` with extra fields. - Perform a ``find`` operation that fails. - Close the client. +**2.5 Invalid Client Configuration with Callback** + +- Create a ``MongoClient`` configured with an OIDC callback and auth mechanism + property ``PROVIDER_NAME:aws``. +- Assert it returns a client configuration error. + (3) Authentication Failure ~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -107,7 +103,7 @@ Drivers MUST implement all prose tests in this section. callback that implements the AWS provider logic. - Poison the cache with an invalid access token. - Perform a ``find`` operation that succeeds. -- Verify that the callback was called 1 time. +- Assert that the callback was called 1 time. - Close the client. **3.2 Authentication failures without cached tokens return an error** @@ -115,16 +111,35 @@ Drivers MUST implement all prose tests in this section. - Create a ``MongoClient`` configured with ``retryReads=false`` and an OIDC callback that always returns invalid access tokens. - Perform a ``find`` operation that fails. -- Verify that the callback was called 1 time. +- Assert that the callback was called 1 time. - Close the client. -(3) Reauthentication +(4) Reauthentication ~~~~~~~~~~~~~~~~~~~~ -TODO: -- Custom callback and provider_name returns error -- Reauthentication succeeds on multiple connections +- Create a ``MongoClient`` configured with an OIDC callback that implements the + AWS provider logic. +- Set a fail point for ``find`` commands of the form: + +.. code:: javascript + + { + configureFailPoint: "failCommand", + mode: { + times: 1 + }, + data: { + failCommands: [ + "find" + ], + errorCode: 391 + } + } +- Perform a ``find`` operation that succeeds. +- Assert that the callback was called 2 times (once during the connection + handshake, and again during reauthentication). +- Close the client. ---------- @@ -135,11 +150,17 @@ Drivers that support the `Human Authentication Flow <../auth/auth.rst#human-authentication-flow>`_ MUST implement all prose tests in this section. +.. note:: + + For test cases that create fail points, drivers MUST either use a unique + ``appName`` or explicitly remove the fail point after the test to prevent + interaction between test cases. + Drivers MUST be able to authenticate against a server configured with either one or two configured identity providers. Note that typically the preconfigured Atlas Dev clusters are used for testing, -in Evergreen and localy. The URIs can be fetched from the ``drivers/oidc`` +in Evergreen and locally. The URIs can be fetched from the ``drivers/oidc`` Secrets vault, see `vault instructions`_. Use ``OIDC_ATLAS_URI_SINGLE`` for ``MONGODB_URI_SINGLE`` and ``OIDC_ATLAS_URI_MULTI`` for ``OIDC_ATLAS_URI_MULTI``. @@ -167,21 +188,21 @@ is one principal configured. **1.1 Single Principal Implicit Username** -- Create default OIDC client with `authMechanism=MONGODB-OIDC`. +- Create default OIDC client with ``authMechanism=MONGODB-OIDC``. - Perform a ``find`` operation that succeeds. - Close the client. **1.2 Single Principal Explicit Username** - Create a client with ``MONGODB_URI_SINGLE``, a username of ``test_user1``, - `authMechanism=MONGODB-OIDC`, and the OIDC human callback. + ``authMechanism=MONGODB-OIDC``, and the OIDC human callback. - Perform a ``find`` operation that succeeds. - Close the client. **1.3 Multiple Principal User 1** - Create a client with ``MONGODB_URI_MULTI``, a username of ``test_user1``, - `authMechanism=MONGODB-OIDC`, and the OIDC human callback. + ``authMechanism=MONGODB-OIDC``, and the OIDC human callback. - Perform a ``find`` operation that succeeds. - Close the client. @@ -189,14 +210,14 @@ is one principal configured. - Create a human callback that reads in the generated ``test_user2`` token file. - Create a client with ``MONGODB_URI_MULTI``, a username of ``test_user2``, - `authMechanism=MONGODB-OIDC`, and the OIDC human callback. + ``authMechanism=MONGODB-OIDC``, and the OIDC human callback. - Perform a ``find`` operation that succeeds. - Close the client. **1.5 Multiple Principal No User** - Create a client with ``MONGODB_URI_MULTI``, no username, - `authMechanism=MONGODB-OIDC`, and the OIDC human callback. + ``authMechanism=MONGODB-OIDC``, and the OIDC human callback. - Assert that a ``find`` operation fails. - Close the client. @@ -225,12 +246,12 @@ is one principal configured. **2.3 Human Callback Returns Missing Data** -- Create a ``MongoClient`` with a human callback that returns data not conforming to - the ``OIDCCredential`` with missing fields. +- Create a ``MongoClient`` with a human callback that returns data not + conforming to the ``OIDCCredential`` with missing fields. - Perform a ``find`` operation that fails. - Close the client. -**2.4 OIDC Callback Returns Invalid Data** +**2.4 Human Callback Returns Invalid Data** - Create a ``MongoClient`` with a human callback that returns data not conforming to the ``OIDCCredential`` with extra fields. @@ -258,30 +279,22 @@ is not called. } } -.. note:: - - The driver MUST either use a unique ``appName`` or explicitly - remove the ``failCommand`` after the test to prevent leakage. - - Perform a ``find`` operation that succeeds. - Close the client. (4) Reauthentication ~~~~~~~~~~~~~~~~~~~~ -The driver MUST test reauthentication with MONGODB-OIDC for a read -operation. - **4.1 Succeeds** -- Create a default OIDC client and add an event listener. The following +- Create a default OIDC client and add an event listener. The following assumes that the driver does not emit ``saslStart`` or ``saslContinue`` - events. If the driver does emit those events, ignore/filter them for the + events. If the driver does emit those events, ignore/filter them for the purposes of this test. - Perform a ``find`` operation that succeeds. - Assert that the human callback has been called once. - Clear the listener state if possible. -- Force a reauthenication using a ``failCommand`` of the form: +- Force a reauthenication using a fail point of the form: .. code:: javascript @@ -298,15 +311,10 @@ operation. } } -.. note:: - - the driver MUST either use a unique ``appName`` or explicitly - remove the ``failCommand`` after the test to prevent leakage. - - Perform another find operation that succeeds. - Assert that the human callback has been called twice. - Assert that the ordering of list started events is [``find``], - , ``find``. Note that if the listener stat could not be cleared then there + , ``find``. Note that if the listener stat could not be cleared then there will and be extra ``find`` command. - Assert that the list of command succeeded events is [``find``]. - Assert that a ``find`` operation failed once during the command execution. @@ -318,7 +326,7 @@ operation. a refresh token. - Perform a ``find`` operation that succeeds. - Assert that the human callback has been called once. -- Force a reauthenication using a ``failCommand`` of the form: +- Force a reauthenication using a fail point of the form: .. code:: javascript @@ -344,7 +352,7 @@ operation. - Create a default OIDC client. - Perform a ``find`` operation that succeeds. - Assert that the human callback has been called once. -- Force a reauthenication using a ``failCommand`` of the form: +- Force a reauthenication using a fail point of the form: .. code:: javascript @@ -362,7 +370,7 @@ operation. } - Perform a ``find`` operation that succeeds. -- Assert that the human callback has been called three times. +- Assert that the human callback has been called 3 times. - Close the client. **4.4 Fails** @@ -390,42 +398,3 @@ operation. - Perform a find operation that fails. - Assert that the human callback has been called twice. - Close the client. - -**4.5 Separate Connections Avoid Extra Callback Calls** - -The following test assumes that the driver will be able to share a cache between -two MongoClient objects, or ensure that the same MongoClient is used with two -different connections. Otherwise, the test would have a race condition. -If neither is possible, the test may be skipped. - -- Create a human callback that returns valid, and ensure that we can record the - number of times the callback is called. -- Create two clients using the callbacks, or a single client and two connection - objects. -- Perform a find operation on each client/connection that succeeds. -- If using a single client, share the underlying cache between clients. -- Ensure that the human callback has been called twice. -- Force a reauthenication on the first client/connection using a ``failCommand`` - of the form: - -.. code:: javascript - - { - configureFailPoint: "failCommand", - mode: { - times: 1 - }, - data: { - failCommands: [ - "find" - ], - errorCode: 391 - } - } - -- Perform a ``find`` operation that succeeds. -- Ensure that the human callback has been called three times. -- Repeat the ``failCommand`` and ``find`` operation on the second - client/connection. -- Ensure that the human callback has been called three times. -- Close all clients/connections. From 9b6bd5f36b6444a71b482c148e065c07741ebc3b Mon Sep 17 00:00:00 2001 From: Matt Dale <9760375+matthewdale@users.noreply.github.com> Date: Mon, 22 Jan 2024 11:21:40 -0800 Subject: [PATCH 35/44] Add type test for authMechanism runOnRequirement. --- .../runOnRequirement-authMechanism-type.json | 17 +++++++++++++++++ .../runOnRequirement-authMechanism-type.yml | 10 ++++++++++ 2 files changed, 27 insertions(+) create mode 100644 source/unified-test-format/tests/invalid/runOnRequirement-authMechanism-type.json create mode 100644 source/unified-test-format/tests/invalid/runOnRequirement-authMechanism-type.yml diff --git a/source/unified-test-format/tests/invalid/runOnRequirement-authMechanism-type.json b/source/unified-test-format/tests/invalid/runOnRequirement-authMechanism-type.json new file mode 100644 index 0000000000..35b7a9d821 --- /dev/null +++ b/source/unified-test-format/tests/invalid/runOnRequirement-authMechanism-type.json @@ -0,0 +1,17 @@ +{ + "description": "runOnRequirement-authMechanism-type", + "schemaVersion": "1.18", + "runOnRequirements": [ + { + "authMechanism": 0 + } + ], + "tests": [ + { + "description": "foo", + "operations": [ + + ] + } + ] +} diff --git a/source/unified-test-format/tests/invalid/runOnRequirement-authMechanism-type.yml b/source/unified-test-format/tests/invalid/runOnRequirement-authMechanism-type.yml new file mode 100644 index 0000000000..481b666da6 --- /dev/null +++ b/source/unified-test-format/tests/invalid/runOnRequirement-authMechanism-type.yml @@ -0,0 +1,10 @@ +description: runOnRequirement-authMechanism-type + +schemaVersion: '1.18' + +runOnRequirements: + - authMechanism: 0 + +tests: + - description: foo + operations: [] From 11357dbac39fac35b2e1b87c65c0116a2c4d6504 Mon Sep 17 00:00:00 2001 From: Matt Dale <9760375+matthewdale@users.noreply.github.com> Date: Tue, 23 Jan 2024 16:53:31 -0800 Subject: [PATCH 36/44] Update source/auth/auth.rst Co-authored-by: Jeff Yemin --- source/auth/auth.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/auth/auth.rst b/source/auth/auth.rst index 02d12fa7ff..029edea83c 100644 --- a/source/auth/auth.rst +++ b/source/auth/auth.rst @@ -1259,7 +1259,7 @@ mechanism_properties CALLBACK An `OIDC Callback`_ that returns credentials for OIDC providers that do - not have a built-in integraiton. Drivers MAY allow the user to specify + not have a built-in integration. Drivers MAY allow the user to specify an `OIDC Callback`_ using a ``MongoClient`` configuration instead of a mechanism property, depending on what is conventional for the driver. Drivers MUST NOT support both the ``CALLBACK`` mechanism property and From 357cad99fd650a43c20c63f681825b9de72a23e5 Mon Sep 17 00:00:00 2001 From: Matt Dale <9760375+matthewdale@users.noreply.github.com> Date: Wed, 24 Jan 2024 10:39:30 -0800 Subject: [PATCH 37/44] Allow drivers to use either a unified or separate callback API. Additional PR feedback. --- source/auth/auth.rst | 107 +++++++++++++++-------------- source/auth/tests/mongodb-oidc.rst | 16 ++--- 2 files changed, 63 insertions(+), 60 deletions(-) diff --git a/source/auth/auth.rst b/source/auth/auth.rst index 029edea83c..fda43446af 100644 --- a/source/auth/auth.rst +++ b/source/auth/auth.rst @@ -1250,45 +1250,44 @@ mechanism MUST be "MONGODB-OIDC" mechanism_properties - PROVIDER_NAME - The name of a built-in OIDC provider integration to use to obtain - credentials. The value MUST be one of ["aws"]. If both ``PROVIDER_NAME`` - and an `OIDC Callback`_ are provided for the same ``MongoClient`` - (either via the ``CALLBACK`` mechanism property or a ``MongoClient`` - configuration), the driver MUST raise an error. - - CALLBACK - An `OIDC Callback`_ that returns credentials for OIDC providers that do - not have a built-in integration. Drivers MAY allow the user to specify - an `OIDC Callback`_ using a ``MongoClient`` configuration instead of a - mechanism property, depending on what is conventional for the driver. - Drivers MUST NOT support both the ``CALLBACK`` mechanism property and - the ``MongoClient`` configuration. - - CALLBACK_TYPE - The type of `OIDC Callback`_. The value MUST be one of ["machine", - "human"]. Drivers MUST NOT allow the user to specify ``CALLBACK_TYPE`` - in the connection string. Drivers MAY allow the user to specify the - callback type using a ``MongoClient`` configuration instead of a - mechanism property to be consistent with how the `OIDC Callback`_ is - configured. If an `OIDC Callback`_ is configured and ``CALLBACK_TYPE`` - is not specified, the driver MUST raise an error. This property is only - required for drivers that support the `Human Authentication Flow`_. - - ALLOWED_HOSTS - The list of allowed hostnames or ip-addresses (ignoring ports) for - MongoDB connections. The hostnames may include a leading "\*." wildcard, - which allows for matching (potentially nested) subdomains. - ``ALLOWED_HOSTS`` is a security feature and MUST default to - ``["*.mongodb.net", "*.mongodb-qa.net", "*.mongodb-dev.net", - "*.mongodbgov.net", "localhost", "127.0.0.1", "::1"]``. When - MONGODB-OIDC authentication using a `Human Callback`_ is attempted - against a hostname that does not match any of list of allowed hosts, the - driver MUST raise a client-side error without invoking any user-provided - callbacks. This value MUST NOT be allowed in the URI connection string. - The hostname check MUST be performed after SRV record resolution, if - applicable. This property is only required for drivers that support the - `Human Authentication Flow`_. + PROVIDER_NAME + Drivers MUST allow the user to specify the name of a built-in OIDC provider + integration to use to obtain credentials. If provided, the value MUST be one + of ["aws"]. If both ``PROVIDER_NAME`` and an `OIDC Callback`_ or `Human + Callback`_ are provided for the same ``MongoClient``, the driver MUST raise + an error. + + CALLBACK + An `OIDC Callback`_ that returns OIDC credentials. Drivers MAY allow the + user to specify an `OIDC Callback`_ using a ``MongoClient`` configuration + instead of a mechanism property, depending on what is idiomatic for the + driver. Drivers MUST NOT support both the ``CALLBACK`` mechanism property + and the ``MongoClient`` configuration. + + HUMAN_CALLBACK + A `Human Callback`_ that returns OIDC credentials. Drivers MAY allow the + user to specify a `Human Callback`_ using a ``MongoClient`` configuration + instead of a mechanism property, depending on what is idiomatic for the + driver. Drivers MUST NOT support both the ``HUMAN_CALLBACK`` mechanism + property and the ``MongoClient`` configuration. Drivers MUST return an error + if both an `OIDC Callback`_ and `Human Callback` are provided for the same + ``MongoClient``. This property is only required for drivers that support the + `Human Authentication Flow`_. + + ALLOWED_HOSTS + The list of allowed hostnames or ip-addresses (ignoring ports) for + MongoDB connections. The hostnames may include a leading "\*." wildcard, + which allows for matching (potentially nested) subdomains. + ``ALLOWED_HOSTS`` is a security feature and MUST default to + ``["*.mongodb.net", "*.mongodb-qa.net", "*.mongodb-dev.net", + "*.mongodbgov.net", "localhost", "127.0.0.1", "::1"]``. When + MONGODB-OIDC authentication using a `Human Callback`_ is attempted + against a hostname that does not match any of list of allowed hosts, the + driver MUST raise a client-side error without invoking any user-provided + callbacks. This value MUST NOT be allowed in the URI connection string. + The hostname check MUST be performed after SRV record resolution, if + applicable. This property is only required for drivers that support the + `Human Authentication Flow`_. Built-in Provider Integrations `````````````````````````````` @@ -1350,10 +1349,11 @@ The callback MUST be able to return the following information: value that indicates the expiry duration is unknown or infinite, like 0 or ``null``. -The signature of the callback is up to the driver's discretion, but the driver -MUST ensure that additional optional input parameters and return values can be -added to the callback signature in the future. An example callback API might -look like: +The signature of the callback is up to the driver's discretion. Drivers MUST +ensure that additional optional input parameters and return values can be added +to the callback signature in the future without breaking backward compatibility. + +An example callback API might look like: .. code:: typescript @@ -1371,8 +1371,9 @@ look like: Human Callback ______________ -Drivers that support the `Human Authentication Flow`_ MUST implement the human -callback version of the `OIDC Callback`_. +The human callback is an OIDC callback that includes additional information that +is required when using the `Human Authentication Flow`_. Drivers that support +the `Human Authentication Flow`_ MUST implement the human callback. In addition to the information described in the `OIDC Callback`_ section, drivers MUST be able to pass the following information to the callback: @@ -1394,7 +1395,13 @@ callback MUST be able to return the following information: - ``refreshToken``: An optional refresh token that can be used to fetch new access tokens. -An example callback API that supports the human callback might look like: +The signature of the callback is up to the driver's discretion. Drivers MAY use +a single callback API for both callback types or separate callback APIs for each +callback type. Drivers MUST ensure that additional optional input parameters and +return values can be added to the callback signature in the future without +breaking backward compatibility. + +An example human callback API might look like: .. code:: typescript @@ -1419,9 +1426,8 @@ An example callback API that supports the human callback might look like: function oidcCallback(params: OIDCCallbackParams): OIDCCredential -Users enable the human callback behavior by setting mechanism property -``CALLBACK_TYPE:human``. When the human callback behavior is enabled, drivers -MUST use the following behaviors when calling the callback: +When a human callback is provided, drivers MUST use the following behaviors when +calling the callback: - The driver MUST pass the ``IdpInfo`` and the refresh token (if available) to the callback. @@ -1434,9 +1440,6 @@ MUST use the following behaviors when calling the callback: interaction required to complete the callback. In this case, the callback is not subject to CSOT. -If ``CALLBACK_TYPE:machine`` drivers MUST use the callback behavior described in -the `OIDC Callback`_ section. - Conversation ```````````` OIDC supports two conversation styles: one-step and two-step. The server detects diff --git a/source/auth/tests/mongodb-oidc.rst b/source/auth/tests/mongodb-oidc.rst index a7b289ad5b..f6e99f4a14 100644 --- a/source/auth/tests/mongodb-oidc.rst +++ b/source/auth/tests/mongodb-oidc.rst @@ -163,7 +163,7 @@ Note that typically the preconfigured Atlas Dev clusters are used for testing, in Evergreen and locally. The URIs can be fetched from the ``drivers/oidc`` Secrets vault, see `vault instructions`_. Use ``OIDC_ATLAS_URI_SINGLE`` for ``MONGODB_URI_SINGLE`` and ``OIDC_ATLAS_URI_MULTI`` for -``OIDC_ATLAS_URI_MULTI``. +``MONGODB_URI_MULTI``. If using local servers is preferred, using the `Local Testing`_ method, use ``mongodb://localhost/?authMechanism=MONGODB-OIDC`` for ``MONGODB_URI_SINGLE`` @@ -363,7 +363,7 @@ is not called. }, data: { failCommands: [ - "find", "saslContinue" + "find", "saslStart" ], errorCode: 391 } @@ -383,15 +383,15 @@ is not called. .. code:: javascript { - "configureFailPoint": "failCommand", - "mode": { - "times": 2 + configureFailPoint: "failCommand", + mode: { + times: 2 }, - "data": { - "failCommands": [ + data: { + failCommands: [ "find", "saslStart" ], - "errorCode": 391 + errorCode: 391 } } From 09554a6381472581b8b7070a21ce4ef2d2e5aded Mon Sep 17 00:00:00 2001 From: Matt Dale <9760375+matthewdale@users.noreply.github.com> Date: Wed, 24 Jan 2024 18:55:44 -0800 Subject: [PATCH 38/44] Update callback naming. Fix auth connection string tests. --- source/auth/auth.rst | 10 ++++---- .../auth/tests/legacy/connection-string.json | 23 +++---------------- .../auth/tests/legacy/connection-string.yml | 16 ++----------- 3 files changed, 10 insertions(+), 39 deletions(-) diff --git a/source/auth/auth.rst b/source/auth/auth.rst index fda43446af..f834b6a103 100644 --- a/source/auth/auth.rst +++ b/source/auth/auth.rst @@ -1257,18 +1257,18 @@ mechanism_properties Callback`_ are provided for the same ``MongoClient``, the driver MUST raise an error. - CALLBACK + OIDC_CALLBACK An `OIDC Callback`_ that returns OIDC credentials. Drivers MAY allow the user to specify an `OIDC Callback`_ using a ``MongoClient`` configuration instead of a mechanism property, depending on what is idiomatic for the - driver. Drivers MUST NOT support both the ``CALLBACK`` mechanism property - and the ``MongoClient`` configuration. + driver. Drivers MUST NOT support both the ``OIDC_CALLBACK`` mechanism + property and the ``MongoClient`` configuration. - HUMAN_CALLBACK + OIDC_HUMAN_CALLBACK A `Human Callback`_ that returns OIDC credentials. Drivers MAY allow the user to specify a `Human Callback`_ using a ``MongoClient`` configuration instead of a mechanism property, depending on what is idiomatic for the - driver. Drivers MUST NOT support both the ``HUMAN_CALLBACK`` mechanism + driver. Drivers MUST NOT support both the ``OIDC_HUMAN_CALLBACK`` mechanism property and the ``MongoClient`` configuration. Drivers MUST return an error if both an `OIDC Callback`_ and `Human Callback` are provided for the same ``MongoClient``. This property is only required for drivers that support the diff --git a/source/auth/tests/legacy/connection-string.json b/source/auth/tests/legacy/connection-string.json index c7bb483689..982edb8b36 100644 --- a/source/auth/tests/legacy/connection-string.json +++ b/source/auth/tests/legacy/connection-string.json @@ -510,21 +510,10 @@ } }, { - "description": "should ignore username and password if specified for aws provider (MONGODB-OIDC)", + "description": "should throw an exception if supplied a password (MONGODB-OIDC)", "uri": "mongodb://user:pass@localhost/?authMechanism=MONGODB-OIDC&authMechanismProperties=PROVIDER_NAME:aws", - "callback": [ - "oidcRequest" - ], - "valid": true, - "credential": { - "username": null, - "password": null, - "source": "$external", - "mechanism": "MONGODB-OIDC", - "mechanism_properties": { - "PROVIDER_NAME": "aws" - } - } + "valid": false, + "credential": null }, { "description": "should throw an exception if username is specified for aws (MONGODB-OIDC)", @@ -538,12 +527,6 @@ "valid": false, "credential": null }, - { - "description": "should throw an exception custom callback is chosen but no callback is provided (MONGODB-OIDC)", - "uri": "mongodb://localhost/?authMechanism=MONGODB-OIDC&authMechanismProperties=PROVIDER_NAME:custom", - "valid": false, - "credential": null - }, { "description": "should throw an exception if neither provider nor callbacks specified (MONGODB-OIDC)", "uri": "mongodb://localhost/?authMechanism=MONGODB-OIDC", diff --git a/source/auth/tests/legacy/connection-string.yml b/source/auth/tests/legacy/connection-string.yml index 0d0a352b87..d2658e0309 100644 --- a/source/auth/tests/legacy/connection-string.yml +++ b/source/auth/tests/legacy/connection-string.yml @@ -370,18 +370,10 @@ tests: mechanism: MONGODB-OIDC mechanism_properties: PROVIDER_NAME: aws -- description: should ignore username and password if specified for aws provider (MONGODB-OIDC) +- description: should throw an exception if supplied a password (MONGODB-OIDC) uri: mongodb://user:pass@localhost/?authMechanism=MONGODB-OIDC&authMechanismProperties=PROVIDER_NAME:aws - callback: - - oidcRequest - valid: true + valid: false credential: - username: - password: - source: "$external" - mechanism: MONGODB-OIDC - mechanism_properties: - PROVIDER_NAME: aws - description: should throw an exception if username is specified for aws (MONGODB-OIDC) uri: mongodb://principalName@localhost/?authMechanism=MONGODB-OIDC&PROVIDER_NAME:aws valid: false @@ -390,10 +382,6 @@ tests: uri: mongodb://localhost/?authMechanism=MONGODB-OIDC&authMechanismProperties=PROVIDER_NAME:invalid valid: false credential: -- description: should throw an exception custom callback is chosen but no callback is provided (MONGODB-OIDC) - uri: mongodb://localhost/?authMechanism=MONGODB-OIDC&authMechanismProperties=PROVIDER_NAME:custom - valid: false - credential: - description: should throw an exception if neither provider nor callbacks specified (MONGODB-OIDC) uri: mongodb://localhost/?authMechanism=MONGODB-OIDC valid: false From 37102c5d6d7a53fa259ac89aa6a1e6a2dae411f6 Mon Sep 17 00:00:00 2001 From: Matt Dale <9760375+matthewdale@users.noreply.github.com> Date: Fri, 26 Jan 2024 00:51:17 -0800 Subject: [PATCH 39/44] Simplify speculative auth. Clarify cache behaviors. PR feedback. --- source/auth/auth.rst | 164 +++++++++++++------------ source/auth/tests/mongodb-oidc.rst | 24 +--- source/mongodb-handshake/handshake.rst | 2 - 3 files changed, 91 insertions(+), 99 deletions(-) diff --git a/source/auth/auth.rst b/source/auth/auth.rst index f834b6a103..983e19d248 100644 --- a/source/auth/auth.rst +++ b/source/auth/auth.rst @@ -1200,9 +1200,9 @@ The MONGODB-OIDC specification refers to the following OIDC concepts: - **Identity Provider (IdP)**: A service that manages user accounts and authenticates users or applications, such as Okta or OneLogin. In the `Human - Authentication Flow`_, the `Human Callback`_ interacts directly the IdP. In - the `Machine Authentication Flow`_, only the MongoDB server interacts directly - the IdP. + Authentication Flow`_, the `OIDC Human Callback`_ interacts directly the IdP. + In the `Machine Authentication Flow`_, only the MongoDB server interacts + directly the IdP. - **Access token**: Used to authenticate requests to protected resources. OIDC access tokens are signed JWT strings. - **Refresh token**: Some OIDC providers may return a refresh token in addition @@ -1253,7 +1253,7 @@ mechanism_properties PROVIDER_NAME Drivers MUST allow the user to specify the name of a built-in OIDC provider integration to use to obtain credentials. If provided, the value MUST be one - of ["aws"]. If both ``PROVIDER_NAME`` and an `OIDC Callback`_ or `Human + of ["aws"]. If both ``PROVIDER_NAME`` and an `OIDC Callback`_ or `OIDC Human Callback`_ are provided for the same ``MongoClient``, the driver MUST raise an error. @@ -1262,32 +1262,32 @@ mechanism_properties user to specify an `OIDC Callback`_ using a ``MongoClient`` configuration instead of a mechanism property, depending on what is idiomatic for the driver. Drivers MUST NOT support both the ``OIDC_CALLBACK`` mechanism - property and the ``MongoClient`` configuration. + property and a ``MongoClient`` configuration. OIDC_HUMAN_CALLBACK - A `Human Callback`_ that returns OIDC credentials. Drivers MAY allow the - user to specify a `Human Callback`_ using a ``MongoClient`` configuration - instead of a mechanism property, depending on what is idiomatic for the - driver. Drivers MUST NOT support both the ``OIDC_HUMAN_CALLBACK`` mechanism - property and the ``MongoClient`` configuration. Drivers MUST return an error - if both an `OIDC Callback`_ and `Human Callback` are provided for the same - ``MongoClient``. This property is only required for drivers that support the - `Human Authentication Flow`_. + An `OIDC Human Callback`_ that returns OIDC credentials. Drivers MAY allow + the user to specify a `OIDC Human Callback`_ using a ``MongoClient`` + configuration instead of a mechanism property, depending on what is + idiomatic for the driver. Drivers MUST NOT support both the + ``OIDC_HUMAN_CALLBACK`` mechanism property and a ``MongoClient`` + configuration. Drivers MUST return an error if both an `OIDC Callback`_ and + `OIDC Human Callback` are provided for the same ``MongoClient``. This + property is only required for drivers that support the `Human Authentication + Flow`_. ALLOWED_HOSTS - The list of allowed hostnames or ip-addresses (ignoring ports) for - MongoDB connections. The hostnames may include a leading "\*." wildcard, - which allows for matching (potentially nested) subdomains. - ``ALLOWED_HOSTS`` is a security feature and MUST default to - ``["*.mongodb.net", "*.mongodb-qa.net", "*.mongodb-dev.net", - "*.mongodbgov.net", "localhost", "127.0.0.1", "::1"]``. When - MONGODB-OIDC authentication using a `Human Callback`_ is attempted + The list of allowed hostnames or ip-addresses (ignoring ports) for MongoDB + connections. The hostnames may include a leading "\*." wildcard, which + allows for matching (potentially nested) subdomains. ``ALLOWED_HOSTS`` is a + security feature and MUST default to ``["*.mongodb.net", "*.mongodb-qa.net", + "*.mongodb-dev.net", "*.mongodbgov.net", "localhost", "127.0.0.1", "::1"]``. + When MONGODB-OIDC authentication using a `OIDC Human Callback`_ is attempted against a hostname that does not match any of list of allowed hosts, the driver MUST raise a client-side error without invoking any user-provided - callbacks. This value MUST NOT be allowed in the URI connection string. - The hostname check MUST be performed after SRV record resolution, if - applicable. This property is only required for drivers that support the - `Human Authentication Flow`_. + callbacks. This value MUST NOT be allowed in the URI connection string. The + hostname check MUST be performed after SRV record resolution, if applicable. + This property is only required for drivers that support the `Human + Authentication Flow`_. Built-in Provider Integrations `````````````````````````````` @@ -1344,14 +1344,17 @@ The callback MUST be able to return the following information: - ``accessToken``: An OIDC access token string. The driver MUST NOT attempt to validate ``accessToken`` directly. -- ``expiresInSeconds``: An optional expiry duration for the access token. - Drivers MUST support and document values for both an expiry duration and a - value that indicates the expiry duration is unknown or infinite, like 0 or - ``null``. - -The signature of the callback is up to the driver's discretion. Drivers MUST -ensure that additional optional input parameters and return values can be added -to the callback signature in the future without breaking backward compatibility. +- ``expiresIn``: An optional expiry duration for the access token. Drivers MUST + interpret the value 0 as an infinite duration and error if a negative value is + returned. Drivers SHOULD use the most idiomatic type for representing a + duration in the driver's language. Note that the access token expiry value is + currently not used in `Credential Caching`_, but is intended to support future + caching optimizations. + +The signature and naming of the callback API is up to the driver's discretion. +Drivers MUST ensure that additional optional input parameters and return values +can be added to the callback signature in the future without breaking backward +compatibility. An example callback API might look like: @@ -1369,8 +1372,8 @@ An example callback API might look like: function oidcCallback(params: OIDCCallbackParams): OIDCCredential -Human Callback -______________ +OIDC Human Callback +___________________ The human callback is an OIDC callback that includes additional information that is required when using the `Human Authentication Flow`_. Drivers that support the `Human Authentication Flow`_ MUST implement the human callback. @@ -1395,11 +1398,11 @@ callback MUST be able to return the following information: - ``refreshToken``: An optional refresh token that can be used to fetch new access tokens. -The signature of the callback is up to the driver's discretion. Drivers MAY use -a single callback API for both callback types or separate callback APIs for each -callback type. Drivers MUST ensure that additional optional input parameters and -return values can be added to the callback signature in the future without -breaking backward compatibility. +The signature and naming of the callback API is up to the driver's discretion. +Drivers MAY use a single callback API for both callback types or separate +callback APIs for each callback type. Drivers MUST ensure that additional +optional input parameters and return values can be added to the callback +signature in the future without breaking backward compatibility. An example human callback API might look like: @@ -1454,8 +1457,8 @@ authentication may provide an access token via a local file pre-loaded on an application host. Drivers MUST use a one-step conversation when using a cached access token, one -of the `Built-in Provider Integrations`_, an `OIDC Callback`_ (not a `Human -Callback`_). +of the `Built-in Provider Integrations`_, or an `OIDC Callback`_ (not an `OIDC +Human Callback`_). The one-step conversation starts with a ``saslStart`` containing a ``JwtStepRequest`` payload. The value of ``jwt`` is the OIDC access token @@ -1499,7 +1502,7 @@ webpage so they can authorize the request. Drivers that support the `Human Authentication Flow`_ MUST implement the two-step conversation. Drivers MUST use a two-step conversation when using a -`Human Callback`_ and when there is no cached access token. +`OIDC Human Callback`_ and when there is no cached access token. The two-step conversation starts with a ``saslStart`` containing a ``PrincipalStepRequest`` payload. The value of ``n`` is the ``username`` from @@ -1535,9 +1538,9 @@ selected IdP: requestScopes: Optional>; } -The driver passes the IdP information to the `Human Callback`_, which should -return an OIDC credential containing an access token and, optionally, a refresh -token. +The driver passes the IdP information to the `OIDC Human Callback`_, which +should return an OIDC credential containing an access token and, optionally, a +refresh token. The driver then sends a ``saslContinue`` with a ``JwtStepRequest`` payload to complete authentication. The value of ``jwt`` is the OIDC access token string. @@ -1648,7 +1651,7 @@ Example code for invalidation: client.oidc_cache.unlock() Drivers that support the `Human Authentication Flow`_ MUST also cache the -``IdPInfo`` and refresh token in the *Client Cache* when a `Human +``IdPInfo`` and refresh token in the *Client Cache* when a `OIDC Human Callback`_ is configured. Authentication @@ -1683,18 +1686,18 @@ Example code to authenticate a connection using the ``get_access_token`` and if is_cache: try: connection.oidc_cache.access_token = access_token - sasl_start(connection, {"jwt": access_token}) + sasl_start(connection, payload={"jwt": access_token}) return - except AuthenticationFailed: + except ServerError: invalidate(access_token) sleep(0.1) access_token, _ = get_access_token() connection.oidc_cache.access_token = access_token - sasl_start(connection, {"jwt": access_token}) + sasl_start(connection, payload={"jwt": access_token}) For drivers that support the `Human Authentication Flow`_, use the following -algorithm to authenticate a new connection when a `Human Callback`_ is +algorithm to authenticate a new connection when a `OIDC Human Callback`_ is configured: - Check if the *Client Cache* has an access token. @@ -1706,18 +1709,18 @@ configured: - Check if the *Client Cache* has a refresh token. - - If it does, call the `Human Callback`_ with the cached refresh token and - ``IdpInfo`` to get a new access token. Cache the new access token in the + - If it does, call the `OIDC Human Callback`_ with the cached refresh token + and ``IdpInfo`` to get a new access token. Cache the new access token in the *Client Cache* and *Connection Cache*. Perform a `One-Step`_ SASL - conversation using the new access token. If any errors occur, invalidate the - access token from the *Client Cache*, clear the *Connection Cache*, and - continue. + conversation using the new access token. If the `OIDC Human Callback`_ or + the server return an error, invalidate the access token from the *Client + Cache*, clear the *Connection Cache*, and continue. - Start a new `Two-Step`_ SASL conversation. - Run a ``PrincipalStepRequest`` to get the ``IdpInfo``. -- Call the `Human Callback`_ with the new ``IdpInfo`` to get a new access token - and optional refresh token. Drivers MUST NOT pass a cached refresh token to - the callback when performing a new `Two-Step`_ conversation. +- Call the `OIDC Human Callback`_ with the new ``IdpInfo`` to get a new access + token and optional refresh token. Drivers MUST NOT pass a cached refresh token + to the callback when performing a new `Two-Step`_ conversation. - Cache the new ``IdpInfo`` and refresh token in the *Client Cache* and the new access token in the *Client Cache* and *Connection Cache*. - Attempt to authenticate using a ``JwtStepRequest`` with the new access token. @@ -1726,34 +1729,38 @@ configured: Speculative Authentication `````````````````````````` Drivers MUST implement speculative authentication for MONGODB-OIDC during the -``hello`` handshake. Use the following algorithm to build the speculative -authentication document: +``hello`` handshake. Drivers MUST NOT attempt speculative authentication if the +*Client Cache* does not have a cached access token. Drivers MUST NOT invalidate +tokens from the *Client Cache* if speculative authentication does not succeed. -- Check if the *Client Cache* has an access token. +Use the following algorithm to perform speculative authentication: - - If it does, send a ``JwtStepRequest`` with the cached access token in the - speculative authentication document. +- Check if the *Client Cache* has an access token. -- Call the configured built-in provider integration or the OIDC callback to - retrieve a new access token. -- Cache the new access token in the *Client Cache* and *Connection Cache*. -- Send a ``JwtStepRequest`` with the new access token in the speculative - authentication document. + - If it does, cache the access token in the *Connection Cache* and send a + ``JwtStepRequest`` with the cached access token in the speculative + authentication SASL payload. If the response is missing a speculative + authentication document or the speculative authentication document indicates + authentication was not successful, clear the the *Connection Cache* and + continue. -For drivers that support the `Human Authentication Flow`_, use the following -algorithm to build the speculative authentication document when a `Human -Callback`_ is configured: +- Authenticate with the standard authentication handshake. -- Check if the *Client Cache* has an access token. +Example code for speculative authentication using the ``auth`` function +described above: - - If it does, send a ``JwtStepRequest`` with the cached access token in the - speculative authentication document. +.. code:: python -- Send a ``PrincipalStepRequest`` (with the ``username`` from the connection - string, if provided) in the speculative authentication document. + def speculative_auth(connection): + access_token = client.oidc_cache.access_token + if access_token != None: + connection.oidc_cache.access_token = access_token + res = hello(connection, payload={"jwt": access_token}) + if res.speculative_authenticate.done: + return -Drivers MUST NOT call a `Human Callback`_ callback during speculative -authentication. + connection.oidc_cache.access_token = None + auth(connection) Reauthentication ```````````````` @@ -2009,6 +2016,7 @@ Q: Why does SCRAM sometimes SASLprep and sometimes not? Q: Should drivers support accessing Amazon EC2 instance metadata in Amazon ECS? No. While it's possible to allow access to EC2 instance metadata in ECS, for security reasons, Amazon states it's best practice to avoid this. (See `accessing EC2 metadata in ECS `_ and `IAM Roles for Tasks `_) + Changelog ========= diff --git a/source/auth/tests/mongodb-oidc.rst b/source/auth/tests/mongodb-oidc.rst index f6e99f4a14..8d4f53f21d 100644 --- a/source/auth/tests/mongodb-oidc.rst +++ b/source/auth/tests/mongodb-oidc.rst @@ -81,14 +81,7 @@ Drivers MUST implement all prose tests in this section. - Perform a ``find`` operation that fails. - Close the client. -**2.4 OIDC Callback Returns Invalid Data** - -- Create a ``MongoClient`` configured with an OIDC callback that returns data - not conforming to the ``OIDCCredential`` with extra fields. -- Perform a ``find`` operation that fails. -- Close the client. - -**2.5 Invalid Client Configuration with Callback** +**2.4 Invalid Client Configuration with Callback** - Create a ``MongoClient`` configured with an OIDC callback and auth mechanism property ``PROVIDER_NAME:aws``. @@ -180,8 +173,8 @@ dummy "refresh_token". .. _Local Testing: https://github.com/mongodb-labs/drivers-evergreen-tools/blob/master/.evergreen/auth_oidc/README.md#local-testing .. _vault instructions: https://wiki.corp.mongodb.com/display/DRIVERS/Using+AWS+Secrets+Manager+to+Store+Testing+Secrets -(1) Human Callback Authentication -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +(1) OIDC Human Callback Authentication +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Drivers MUST be able to authenticate using OIDC callback(s) when there is one principal configured. @@ -232,8 +225,8 @@ is one principal configured. - Assert that a ``find`` operation fails with a client-side error. - Close the client. -(2) Human Callback Validation -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +(2) OIDC Human Callback Validation +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ **2.1 Valid Callback Inputs** @@ -251,13 +244,6 @@ is one principal configured. - Perform a ``find`` operation that fails. - Close the client. -**2.4 Human Callback Returns Invalid Data** - -- Create a ``MongoClient`` with a human callback that returns data not - conforming to the ``OIDCCredential`` with extra fields. -- Perform a ``find`` operation that fails. -- Close the client. - (3) Speculative Authentication ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ We can only test the successful case, by verifying that ``saslStart`` diff --git a/source/mongodb-handshake/handshake.rst b/source/mongodb-handshake/handshake.rst index 22d4712a39..6c012a6259 100644 --- a/source/mongodb-handshake/handshake.rst +++ b/source/mongodb-handshake/handshake.rst @@ -413,8 +413,6 @@ When the mechanism is ``MONGODB-OIDC``, ``speculativeAuthenticate`` has the same structure as seen in the MONGODB-OIDC conversation section in the `Driver Authentication spec `_. -However, the driver MUST not call a callback as part of -``speculativeAuthenticate`` during the `Human Authentication Flow <../auth/auth.rst#human-authentication-flow>`_. If the initial handshake command with a ``speculativeAuthenticate`` argument succeeds, the client should proceed with the next step of the exchange. If the initial handshake From 03a0f8b69ca342b3079e1faa0947484b6fd6d0f9 Mon Sep 17 00:00:00 2001 From: Matt Dale <9760375+matthewdale@users.noreply.github.com> Date: Mon, 29 Jan 2024 16:07:26 -0800 Subject: [PATCH 40/44] Fix prose and spec tests. Add to prose test local testing guide. --- source/auth/tests/mongodb-oidc.rst | 70 ++++- .../tests/unified/mongodb-oidc-no-retry.json | 255 +++++++++++++++++- .../tests/unified/mongodb-oidc-no-retry.yml | 143 +++++++++- .../tests/unified/mongodb-oidc-retry.json | 213 --------------- .../auth/tests/unified/mongodb-oidc-retry.yml | 113 -------- 5 files changed, 443 insertions(+), 351 deletions(-) delete mode 100644 source/auth/tests/unified/mongodb-oidc-retry.json delete mode 100644 source/auth/tests/unified/mongodb-oidc-retry.yml diff --git a/source/auth/tests/mongodb-oidc.rst b/source/auth/tests/mongodb-oidc.rst index 8d4f53f21d..d2432e6fdc 100644 --- a/source/auth/tests/mongodb-oidc.rst +++ b/source/auth/tests/mongodb-oidc.rst @@ -34,6 +34,13 @@ Drivers MUST implement all prose tests in this section. ``appName`` or explicitly remove the fail point after the test to prevent interaction between test cases. +Note that typically the preconfigured Atlas Dev clusters are used for testing, +in Evergreen and locally. The URIs can be fetched from the ``drivers/oidc`` +Secrets vault, see `vault instructions`_. Use ``OIDC_ATLAS_URI_SINGLE`` for the +``MONGODB_URI``. If using local servers is preferred, using the `Local Testing`_ +method, use ``mongodb://localhost/?authMechanism=MONGODB-OIDC`` for +``MONGODB_URI``. + (1) OIDC Callback Authentication ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -94,7 +101,7 @@ Drivers MUST implement all prose tests in this section. - Create a ``MongoClient`` configured with ``retryReads=false`` and an OIDC callback that implements the AWS provider logic. -- Poison the cache with an invalid access token. +- Poison the *Client Cache* with an invalid access token. - Perform a ``find`` operation that succeeds. - Assert that the callback was called 1 time. - Close the client. @@ -125,7 +132,7 @@ Drivers MUST implement all prose tests in this section. failCommands: [ "find" ], - errorCode: 391 + errorCode: 391 // ReauthenticationRequired } } @@ -155,8 +162,10 @@ or two configured identity providers. Note that typically the preconfigured Atlas Dev clusters are used for testing, in Evergreen and locally. The URIs can be fetched from the ``drivers/oidc`` Secrets vault, see `vault instructions`_. Use ``OIDC_ATLAS_URI_SINGLE`` for -``MONGODB_URI_SINGLE`` and ``OIDC_ATLAS_URI_MULTI`` for -``MONGODB_URI_MULTI``. +``MONGODB_URI_SINGLE`` and ``OIDC_ATLAS_URI_MULTI`` for ``MONGODB_URI_MULTI``. +Currently the ``OIDC_ATLAS_URI_MULTI`` cluster does not work correctly with fail +points, so all prose tests that use fail points SHOULD use +``OIDC_ATLAS_URI_SINGLE``. If using local servers is preferred, using the `Local Testing`_ method, use ``mongodb://localhost/?authMechanism=MONGODB-OIDC`` for ``MONGODB_URI_SINGLE`` @@ -246,10 +255,28 @@ is one principal configured. (3) Speculative Authentication ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -We can only test the successful case, by verifying that ``saslStart`` -is not called. + +**3.1 Uses speculative authentication if there is a cached token** - Create a ``MongoClient`` with a human callback that returns a valid token. +- Set a fail point for ``find`` commands of the form: + +.. code:: javascript + + { + configureFailPoint: "failCommand", + mode: { + times: 1 + }, + data: { + failCommands: [ + "find" + ], + closeConnection: true + } + } + +- Perform a ``find`` operation that fails. - Set a fail point for ``saslStart`` commands of the form: .. code:: javascript @@ -268,6 +295,27 @@ is not called. - Perform a ``find`` operation that succeeds. - Close the client. +**3.2 Does not use speculative authentication if there is no cached token** + +- Create a ``MongoClient`` with a human callback that returns a valid token. +- Set a fail point for ``saslStart`` commands of the form: + +.. code:: javascript + + { + configureFailPoint: "failCommand", + mode: "alwaysOn", + data: { + failCommands: [ + "saslStart" + ], + errorCode: 20 // IllegalOperation + } + } + +- Perform a ``find`` operation that fails. +- Close the client. + (4) Reauthentication ~~~~~~~~~~~~~~~~~~~~ @@ -293,7 +341,7 @@ is not called. failCommands: [ "find" ], - errorCode: 391 + errorCode: 391 // ReauthenticationRequired } } @@ -325,7 +373,7 @@ is not called. failCommands: [ "find" ], - errorCode: 391 + errorCode: 391 // ReauthenticationRequired } } @@ -351,7 +399,7 @@ is not called. failCommands: [ "find", "saslStart" ], - errorCode: 391 + errorCode: 391 // ReauthenticationRequired } } @@ -371,13 +419,13 @@ is not called. { configureFailPoint: "failCommand", mode: { - times: 2 + times: 3 }, data: { failCommands: [ "find", "saslStart" ], - errorCode: 391 + errorCode: 391 // ReauthenticationRequired } } diff --git a/source/auth/tests/unified/mongodb-oidc-no-retry.json b/source/auth/tests/unified/mongodb-oidc-no-retry.json index 7b79a1c2a3..e0aa156eb0 100644 --- a/source/auth/tests/unified/mongodb-oidc-no-retry.json +++ b/source/auth/tests/unified/mongodb-oidc-no-retry.json @@ -11,7 +11,8 @@ "createEntities": [ { "client": { - "id": "authClient" + "id": "failPointClient", + "useMultipleMongoses": false } }, { @@ -62,11 +63,11 @@ "operations": [ { "name": "find", + "object": "collection0", "arguments": { "filter": { } }, - "object": "collection0", "expectResult": [ ] @@ -141,7 +142,7 @@ "name": "failPoint", "object": "testRunner", "arguments": { - "client": "client0", + "client": "failPointClient", "failPoint": { "configureFailPoint": "failCommand", "mode": { @@ -158,11 +159,11 @@ }, { "name": "find", + "object": "collection0", "arguments": { "filter": { } }, - "object": "collection0", "expectResult": [ ] @@ -211,7 +212,7 @@ "name": "failPoint", "object": "testRunner", "arguments": { - "client": "client0", + "client": "failPointClient", "failPoint": { "configureFailPoint": "failCommand", "mode": { @@ -282,13 +283,45 @@ ] }, { - "description": "Handshake should use speculative authentication", + "description": "Handshake with cached token should use speculative authentication", "operations": [ { "name": "failPoint", "object": "testRunner", "arguments": { - "client": "client0", + "client": "failPointClient", + "failPoint": { + "configureFailPoint": "failCommand", + "mode": { + "times": 1 + }, + "data": { + "failCommands": [ + "insert" + ], + "closeConnection": true + } + } + } + }, + { + "name": "insertOne", + "object": "collection0", + "arguments": { + "document": { + "_id": 1, + "x": 1 + } + }, + "expectError": { + "isClientError": true + } + }, + { + "name": "failPoint", + "object": "testRunner", + "arguments": { + "client": "failPointClient", "failPoint": { "configureFailPoint": "failCommand", "mode": "alwaysOn", @@ -355,6 +388,214 @@ ] } ] + }, + { + "description": "Handshake without cached token should not use speculative authentication", + "operations": [ + { + "name": "failPoint", + "object": "testRunner", + "arguments": { + "client": "failPointClient", + "failPoint": { + "configureFailPoint": "failCommand", + "mode": "alwaysOn", + "data": { + "failCommands": [ + "saslStart" + ], + "errorCode": 20 + } + } + } + }, + { + "name": "insertOne", + "object": "collection0", + "arguments": { + "document": { + "_id": 1, + "x": 1 + } + }, + "expectError": { + "errorCode": 20 + } + } + ] + }, + { + "description": "Read commands should fail if reauthentication fails", + "operations": [ + { + "name": "find", + "object": "collection0", + "arguments": { + "filter": { + } + }, + "expectResult": [ + + ] + }, + { + "name": "failPoint", + "object": "testRunner", + "arguments": { + "client": "failPointClient", + "failPoint": { + "configureFailPoint": "failCommand", + "mode": { + "times": 2 + }, + "data": { + "failCommands": [ + "find", + "saslStart" + ], + "errorCode": 391 + } + } + } + }, + { + "name": "find", + "object": "collection0", + "arguments": { + "filter": { + } + }, + "expectError": { + "errorCode": 391 + } + } + ], + "expectEvents": [ + { + "client": "client0", + "events": [ + { + "commandStartedEvent": { + "command": { + "find": "collName", + "filter": { + } + } + } + }, + { + "commandSucceededEvent": { + "commandName": "find" + } + }, + { + "commandStartedEvent": { + "command": { + "find": "collName", + "filter": { + } + } + } + }, + { + "commandFailedEvent": { + "commandName": "find" + } + } + ] + } + ] + }, + { + "description": "Write commands should fail if reauthentication fails", + "operations": [ + { + "name": "insertOne", + "object": "collection0", + "arguments": { + "document": { + "_id": 1, + "x": 1 + } + } + }, + { + "name": "failPoint", + "object": "testRunner", + "arguments": { + "client": "failPointClient", + "failPoint": { + "configureFailPoint": "failCommand", + "mode": { + "times": 2 + }, + "data": { + "failCommands": [ + "insert", + "saslStart" + ], + "errorCode": 391 + } + } + } + }, + { + "name": "insertOne", + "object": "collection0", + "arguments": { + "document": { + "_id": 2, + "x": 2 + } + }, + "expectError": { + "errorCode": 391 + } + } + ], + "expectEvents": [ + { + "client": "client0", + "events": [ + { + "commandStartedEvent": { + "command": { + "insert": "collName", + "documents": [ + { + "_id": 1, + "x": 1 + } + ] + } + } + }, + { + "commandSucceededEvent": { + "commandName": "insert" + } + }, + { + "commandStartedEvent": { + "command": { + "insert": "collName", + "documents": [ + { + "_id": 2, + "x": 2 + } + ] + } + } + }, + { + "commandFailedEvent": { + "commandName": "insert" + } + } + ] + } + ] } ] } diff --git a/source/auth/tests/unified/mongodb-oidc-no-retry.yml b/source/auth/tests/unified/mongodb-oidc-no-retry.yml index b3276a450d..f4e596f8f3 100644 --- a/source/auth/tests/unified/mongodb-oidc-no-retry.yml +++ b/source/auth/tests/unified/mongodb-oidc-no-retry.yml @@ -7,7 +7,8 @@ runOnRequirements: authMechanism: "MONGODB-OIDC" createEntities: - client: - id: authClient + id: &failPointClient failPointClient + useMultipleMongoses: false - client: id: client0 uriOptions: @@ -39,9 +40,9 @@ tests: - description: A read operation should succeed operations: - name: find + object: collection0 arguments: filter: {} - object: collection0 expectResult: [] expectEvents: - client: client0 @@ -76,7 +77,7 @@ tests: - name: failPoint object: testRunner arguments: - client: client0 + client: failPointClient failPoint: configureFailPoint: failCommand mode: @@ -86,9 +87,9 @@ tests: - find errorCode: 391 # ReauthenticationRequired - name: find + object: collection0 arguments: filter: {} - object: collection0 expectResult: [] expectEvents: - client: client0 @@ -110,7 +111,7 @@ tests: - name: failPoint object: testRunner arguments: - client: client0 + client: failPointClient failPoint: configureFailPoint: failCommand mode: @@ -144,12 +145,32 @@ tests: x: 1 - commandSucceededEvent: commandName: insert -- description: Handshake should use speculative authentication +- description: Handshake with cached token should use speculative authentication operations: - name: failPoint object: testRunner arguments: - client: client0 + client: failPointClient + failPoint: + configureFailPoint: failCommand + mode: + times: 1 + data: + failCommands: + - insert + closeConnection: true + - name: insertOne + object: collection0 + arguments: + document: + _id: 1 + x: 1 + expectError: + isClientError: true + - name: failPoint + object: testRunner + arguments: + client: failPointClient failPoint: configureFailPoint: failCommand mode: "alwaysOn" @@ -182,3 +203,111 @@ tests: x: 1 - commandSucceededEvent: commandName: insert +- description: Handshake without cached token should not use speculative authentication + operations: + - name: failPoint + object: testRunner + arguments: + client: failPointClient + failPoint: + configureFailPoint: failCommand + mode: "alwaysOn" + data: + failCommands: + - saslStart + errorCode: 20 # IllegalOperation + - name: insertOne + object: collection0 + arguments: + document: + _id: 1 + x: 1 + expectError: + errorCode: 20 # IllegalOperation +- description: Read commands should fail if reauthentication fails + operations: + - name: find + object: collection0 + arguments: + filter: {} + expectResult: [] + - name: failPoint + object: testRunner + arguments: + client: failPointClient + failPoint: + configureFailPoint: failCommand + mode: + times: 2 + data: + failCommands: + - find + - saslStart + errorCode: 391 # ReauthenticationRequired + - name: find + object: collection0 + arguments: + filter: {} + expectError: { errorCode: 391 } + expectEvents: + - client: client0 + events: + - commandStartedEvent: + command: + find: collName + filter: {} + - commandSucceededEvent: + commandName: find + - commandStartedEvent: + command: + find: collName + filter: {} + - commandFailedEvent: + commandName: find +- description: Write commands should fail if reauthentication fails + operations: + - name: insertOne + object: collection0 + arguments: + document: + _id: 1 + x: 1 + - name: failPoint + object: testRunner + arguments: + client: failPointClient + failPoint: + configureFailPoint: failCommand + mode: + times: 2 + data: + failCommands: + - insert + - saslStart + errorCode: 391 # ReauthenticationRequired + - name: insertOne + object: collection0 + arguments: + document: + _id: 2 + x: 2 + expectError: { errorCode: 391 } + expectEvents: + - client: client0 + events: + - commandStartedEvent: + command: + insert: collName + documents: + - _id: 1 + x: 1 + - commandSucceededEvent: + commandName: insert + - commandStartedEvent: + command: + insert: collName + documents: + - _id: 2 + x: 2 + - commandFailedEvent: + commandName: insert diff --git a/source/auth/tests/unified/mongodb-oidc-retry.json b/source/auth/tests/unified/mongodb-oidc-retry.json deleted file mode 100644 index 1d7aed787c..0000000000 --- a/source/auth/tests/unified/mongodb-oidc-retry.json +++ /dev/null @@ -1,213 +0,0 @@ -{ - "description": "MONGODB-OIDC authentication with retry enabled", - "schemaVersion": "1.18", - "runOnRequirements": [ - { - "minServerVersion": "7.0", - "auth": true, - "authMechanism": "MONGODB-OIDC" - } - ], - "createEntities": [ - { - "client": { - "id": "authClient" - } - }, - { - "client": { - "id": "client0", - "uriOptions": { - "authMechanism": "MONGODB-OIDC", - "authMechanismProperties": { - "$$placeholder": 1 - }, - "retryReads": true, - "retryWrites": true - }, - "observeEvents": [ - "commandStartedEvent", - "commandSucceededEvent", - "commandFailedEvent" - ] - } - }, - { - "database": { - "id": "database0", - "client": "client0", - "databaseName": "test" - } - }, - { - "collection": { - "id": "collection0", - "database": "database0", - "collectionName": "collName" - } - } - ], - "initialData": [ - { - "collectionName": "collName", - "databaseName": "test", - "documents": [ - - ] - } - ], - "tests": [ - { - "description": "Read commands should fail if reauthentication fails", - "operations": [ - { - "name": "failPoint", - "object": "testRunner", - "arguments": { - "client": "client0", - "failPoint": { - "configureFailPoint": "failCommand", - "mode": { - "times": 2 - }, - "data": { - "failCommands": [ - "find", - "saslStart" - ], - "errorCode": 391 - } - } - } - }, - { - "name": "find", - "object": "collection0", - "arguments": { - "filter": { - } - }, - "expectError": { - "errorCode": 391 - } - } - ], - "expectEvents": [ - { - "client": "client0", - "events": [ - { - "commandStartedEvent": { - "command": { - "find": "collName", - "filter": { - } - } - } - }, - { - "commandFailedEvent": { - "commandName": "find" - } - }, - { - "commandStartedEvent": { - "command": { - "find": "collName", - "filter": { - } - } - } - }, - { - "commandFailedEvent": { - "commandName": "find" - } - } - ] - } - ] - }, - { - "description": "Write commands should fail if reauthentication fails", - "operations": [ - { - "name": "failPoint", - "object": "testRunner", - "arguments": { - "client": "client0", - "failPoint": { - "configureFailPoint": "failCommand", - "mode": { - "times": 2 - }, - "data": { - "failCommands": [ - "insert", - "saslStart" - ], - "errorCode": 391 - } - } - } - }, - { - "name": "insertOne", - "object": "collection0", - "arguments": { - "document": { - "_id": 1, - "x": 1 - } - }, - "expectError": { - "errorCode": 391 - } - } - ], - "expectEvents": [ - { - "client": "client0", - "events": [ - { - "commandStartedEvent": { - "command": { - "insert": "collName", - "documents": [ - { - "_id": 1, - "x": 1 - } - ] - } - } - }, - { - "commandFailedEvent": { - "commandName": "insert" - } - }, - { - "commandStartedEvent": { - "command": { - "insert": "collName", - "documents": [ - { - "_id": 1, - "x": 1 - } - ] - } - } - }, - { - "commandFailedEvent": { - "commandName": "insert" - } - } - ] - } - ] - } - ] -} diff --git a/source/auth/tests/unified/mongodb-oidc-retry.yml b/source/auth/tests/unified/mongodb-oidc-retry.yml deleted file mode 100644 index bd62120098..0000000000 --- a/source/auth/tests/unified/mongodb-oidc-retry.yml +++ /dev/null @@ -1,113 +0,0 @@ -description: "MONGODB-OIDC authentication with retry enabled" -schemaVersion: "1.18" -runOnRequirements: -- minServerVersion: "7.0" - auth: true - authMechanism: "MONGODB-OIDC" -createEntities: -- client: - id: authClient -- client: - id: client0 - uriOptions: - authMechanism: "MONGODB-OIDC" - # The $$placeholder document should be replaced by auth mechanism - # properties that enable OIDC auth on the target cloud platform. For - # example, when running the test on AWS, replace the $$placeholder - # document with {"PROVIDER_NAME": "aws"}. - authMechanismProperties: { $$placeholder: 1 } - retryReads: true - retryWrites: true - observeEvents: - - commandStartedEvent - - commandSucceededEvent - - commandFailedEvent -- database: - id: database0 - client: client0 - databaseName: test -- collection: - id: collection0 - database: database0 - collectionName: collName -initialData: -- collectionName: collName - databaseName: test - documents: [] -tests: -- description: Read commands should fail if reauthentication fails - operations: - - name: failPoint - object: testRunner - arguments: - client: client0 - failPoint: - configureFailPoint: failCommand - mode: - times: 2 - data: - failCommands: - - find - - saslStart - errorCode: 391 # ReauthenticationRequired - - name: find - object: collection0 - arguments: - filter: {} - expectError: { errorCode: 391 } - expectEvents: - - client: client0 - events: - - commandStartedEvent: - command: - find: collName - filter: {} - - commandFailedEvent: - commandName: find - - commandStartedEvent: - command: - find: collName - filter: {} - - commandFailedEvent: - commandName: find -- description: Write commands should fail if reauthentication fails - operations: - - name: failPoint - object: testRunner - arguments: - client: client0 - failPoint: - configureFailPoint: failCommand - mode: - times: 2 - data: - failCommands: - - insert - - saslStart - errorCode: 391 # ReauthenticationRequired - - name: insertOne - object: collection0 - arguments: - document: - _id: 1 - x: 1 - expectError: { errorCode: 391 } - expectEvents: - - client: client0 - events: - - commandStartedEvent: - command: - insert: collName - documents: - - _id: 1 - x: 1 - - commandFailedEvent: - commandName: insert - - commandStartedEvent: - command: - insert: collName - documents: - - _id: 1 - x: 1 - - commandFailedEvent: - commandName: insert From 95395e25ccb859d7b87e4a7ccd4188153e2d959c Mon Sep 17 00:00:00 2001 From: Matt Dale <9760375+matthewdale@users.noreply.github.com> Date: Tue, 30 Jan 2024 16:54:40 -0800 Subject: [PATCH 41/44] Clarify config of retryReads in prose tests. Remove checking for extra fields in prose tests. --- source/auth/tests/mongodb-oidc.rst | 28 +++++++++++++++------------- 1 file changed, 15 insertions(+), 13 deletions(-) diff --git a/source/auth/tests/mongodb-oidc.rst b/source/auth/tests/mongodb-oidc.rst index d2432e6fdc..6c345ec8cf 100644 --- a/source/auth/tests/mongodb-oidc.rst +++ b/source/auth/tests/mongodb-oidc.rst @@ -15,8 +15,9 @@ For example, if the selected AWS profile ID is "drivers-test", run: .. code:: shell aws configure sso - AWS_PROFILE="drivers-test" ./oidc_get_tokens.sh - AWS_WEB_IDENTITY_TOKEN_FILE="/tmp/tokens/test_user1" /my/test/command + export OIDC_TOKEN_DIR=/tmp/tokens + AWS_PROFILE="drivers-test" oidc_get_tokens.sh + AWS_WEB_IDENTITY_TOKEN_FILE="$OIDC_TOKEN_DIR/test_user1" /my/test/command .. _oidc_get_tokens.sh: https://github.com/mongodb-labs/drivers-evergreen-tools/blob/master/.evergreen/auth_oidc/oidc_get_tokens.sh .. _drivers-evergreen-tools: https://github.com/mongodb-labs/drivers-evergreen-tools/ @@ -26,7 +27,8 @@ For example, if the selected AWS profile ID is "drivers-test", run: Prose Tests =========== -Drivers MUST implement all prose tests in this section. +Drivers MUST implement all prose tests in this section. Unless otherwise noted, +all ``MongoClient`` instances MUST be configured with ``retryReads=false``. .. note:: @@ -70,8 +72,7 @@ method, use ``mongodb://localhost/?authMechanism=MONGODB-OIDC`` for inputs and returns a valid access token. - Perform a ``find`` operation that succeeds. - Assert that the OIDC callback was called with the appropriate inputs, - including the timeout parameter if possible. Ensure that there are no - unexpected fields. + including the timeout parameter if possible. - Close the client. **2.2 OIDC Callback Returns Null** @@ -97,10 +98,10 @@ method, use ``mongodb://localhost/?authMechanism=MONGODB-OIDC`` for (3) Authentication Failure ~~~~~~~~~~~~~~~~~~~~~~~~~~ -**3.1 Authentication failure with cached tokens fetch a new token and retry** +**3.1 Authentication failure with cached tokens fetch a new token and retry auth** -- Create a ``MongoClient`` configured with ``retryReads=false`` and an OIDC - callback that implements the AWS provider logic. +- Create a ``MongoClient`` configured with an OIDC callback that implements the + AWS provider logic. - Poison the *Client Cache* with an invalid access token. - Perform a ``find`` operation that succeeds. - Assert that the callback was called 1 time. @@ -108,8 +109,8 @@ method, use ``mongodb://localhost/?authMechanism=MONGODB-OIDC`` for **3.2 Authentication failures without cached tokens return an error** -- Create a ``MongoClient`` configured with ``retryReads=false`` and an OIDC - callback that always returns invalid access tokens. +- Create a ``MongoClient`` configured with an OIDC callback that always returns + invalid access tokens. - Perform a ``find`` operation that fails. - Assert that the callback was called 1 time. - Close the client. @@ -148,7 +149,8 @@ Human Authentication Flow Prose Tests Drivers that support the `Human Authentication Flow <../auth/auth.rst#human-authentication-flow>`_ MUST implement all prose tests in -this section. +this section. Unless otherwise noted, all ``MongoClient`` instances MUST be +configured with ``retryReads=false``. .. note:: @@ -174,7 +176,7 @@ and for ``MONGODB_URI_MULTI`` because the other server is a secondary on a replica set, on port ``27018``. -The default OIDC client used in the tests will be configured with +The default OIDC client used in the tests is configured with ``MONGODB_URI_SINGLE`` and a valid human callback handler that returns the ``test_user1`` local token in ``OIDC_TOKEN_DIR`` as the "access_token", and a dummy "refresh_token". @@ -243,7 +245,7 @@ is one principal configured. returns a valid access token. - Perform a ``find`` operation that succeeds. Verify that the human callback was called with the appropriate inputs, including the timeout - parameter if possible. Ensure that there are no unexpected fields. + parameter if possible. - Close the client. **2.3 Human Callback Returns Missing Data** From f41bb37cb3b72d0d0f629e0f4b02ec818eb50604 Mon Sep 17 00:00:00 2001 From: Matt Dale <9760375+matthewdale@users.noreply.github.com> Date: Tue, 30 Jan 2024 17:19:56 -0800 Subject: [PATCH 42/44] Bump unified test format to 1.19 --- source/auth/tests/unified/mongodb-oidc-no-retry.json | 2 +- source/auth/tests/unified/mongodb-oidc-no-retry.yml | 2 +- .../unified-test-format/{schema-1.18.json => schema-1.19.json} | 0 .../tests/invalid/runOnRequirement-authMechanism-type.json | 2 +- .../tests/invalid/runOnRequirement-authMechanism-type.yml | 2 +- source/unified-test-format/unified-test-format.rst | 2 +- 6 files changed, 5 insertions(+), 5 deletions(-) rename source/unified-test-format/{schema-1.18.json => schema-1.19.json} (100%) diff --git a/source/auth/tests/unified/mongodb-oidc-no-retry.json b/source/auth/tests/unified/mongodb-oidc-no-retry.json index e0aa156eb0..83d73e4e50 100644 --- a/source/auth/tests/unified/mongodb-oidc-no-retry.json +++ b/source/auth/tests/unified/mongodb-oidc-no-retry.json @@ -1,6 +1,6 @@ { "description": "MONGODB-OIDC authentication with retry disabled", - "schemaVersion": "1.18", + "schemaVersion": "1.19", "runOnRequirements": [ { "minServerVersion": "7.0", diff --git a/source/auth/tests/unified/mongodb-oidc-no-retry.yml b/source/auth/tests/unified/mongodb-oidc-no-retry.yml index f4e596f8f3..b500fb7db6 100644 --- a/source/auth/tests/unified/mongodb-oidc-no-retry.yml +++ b/source/auth/tests/unified/mongodb-oidc-no-retry.yml @@ -1,6 +1,6 @@ --- description: "MONGODB-OIDC authentication with retry disabled" -schemaVersion: "1.18" +schemaVersion: "1.19" runOnRequirements: - minServerVersion: "7.0" auth: true diff --git a/source/unified-test-format/schema-1.18.json b/source/unified-test-format/schema-1.19.json similarity index 100% rename from source/unified-test-format/schema-1.18.json rename to source/unified-test-format/schema-1.19.json diff --git a/source/unified-test-format/tests/invalid/runOnRequirement-authMechanism-type.json b/source/unified-test-format/tests/invalid/runOnRequirement-authMechanism-type.json index 35b7a9d821..b97654a743 100644 --- a/source/unified-test-format/tests/invalid/runOnRequirement-authMechanism-type.json +++ b/source/unified-test-format/tests/invalid/runOnRequirement-authMechanism-type.json @@ -1,6 +1,6 @@ { "description": "runOnRequirement-authMechanism-type", - "schemaVersion": "1.18", + "schemaVersion": "1.19", "runOnRequirements": [ { "authMechanism": 0 diff --git a/source/unified-test-format/tests/invalid/runOnRequirement-authMechanism-type.yml b/source/unified-test-format/tests/invalid/runOnRequirement-authMechanism-type.yml index 481b666da6..a0e835612b 100644 --- a/source/unified-test-format/tests/invalid/runOnRequirement-authMechanism-type.yml +++ b/source/unified-test-format/tests/invalid/runOnRequirement-authMechanism-type.yml @@ -1,6 +1,6 @@ description: runOnRequirement-authMechanism-type -schemaVersion: '1.18' +schemaVersion: '1.19' runOnRequirements: - authMechanism: 0 diff --git a/source/unified-test-format/unified-test-format.rst b/source/unified-test-format/unified-test-format.rst index 8d51639554..bca6067d20 100644 --- a/source/unified-test-format/unified-test-format.rst +++ b/source/unified-test-format/unified-test-format.rst @@ -4089,7 +4089,7 @@ Changelog .. Please note schema version bumps in changelog entries where applicable. -:2024-01-17: **Schema version 1.18.** +:2024-01-31: **Schema version 1.19.** Add ``authMechanism`` to ``runOnRequirement`` and require that ``uriOptions`` supports placeholder documents. :2024-01-03: Document server version requirements for ``errorLabels`` and From 353fa83307f8454af642d2297acc8590b6ad78cd Mon Sep 17 00:00:00 2001 From: Matt Dale <9760375+matthewdale@users.noreply.github.com> Date: Tue, 30 Jan 2024 17:22:26 -0800 Subject: [PATCH 43/44] Correct Unified Test Format spec current schema version. --- source/unified-test-format/unified-test-format.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/unified-test-format/unified-test-format.rst b/source/unified-test-format/unified-test-format.rst index e56e43bdba..8145a74662 100644 --- a/source/unified-test-format/unified-test-format.rst +++ b/source/unified-test-format/unified-test-format.rst @@ -4,7 +4,7 @@ Unified Test Format :Status: Accepted :Minimum Server Version: N/A -:Current Schema Version: 1.18.0 +:Current Schema Version: 1.19.0 .. contents:: From 4203a6de155f25b8093fd2903d5e4f33545d5ef5 Mon Sep 17 00:00:00 2001 From: Matt Dale <9760375+matthewdale@users.noreply.github.com> Date: Tue, 30 Jan 2024 17:29:42 -0800 Subject: [PATCH 44/44] Include all changes from unified test spec 1.18 in 1.19. --- source/unified-test-format/schema-1.19.json | 1608 ++++++++++------- source/unified-test-format/tests/Makefile | 2 +- .../unified-test-format.rst | 2 +- 3 files changed, 986 insertions(+), 626 deletions(-) diff --git a/source/unified-test-format/schema-1.19.json b/source/unified-test-format/schema-1.19.json index 9ed28819e1..79609fe539 100644 --- a/source/unified-test-format/schema-1.19.json +++ b/source/unified-test-format/schema-1.19.json @@ -1,713 +1,1073 @@ { "$schema": "http://json-schema.org/draft-07/schema#", - "title": "Unified Test Format", "type": "object", "additionalProperties": false, - "required": ["description", "schemaVersion", "tests"], + "required": [ + "description", + "schemaVersion", + "tests" + ], "properties": { - "description": { "type": "string" }, - "schemaVersion": { "$ref": "#/definitions/version" }, - "runOnRequirements": { - "type": "array", - "minItems": 1, - "items": { "$ref": "#/definitions/runOnRequirement" } - }, - "createEntities": { - "type": "array", - "minItems": 1, - "items": { "$ref": "#/definitions/entity" } - }, - "initialData": { - "type": "array", - "minItems": 1, - "items": { "$ref": "#/definitions/collectionData" } - }, - "tests": { - "type": "array", - "minItems": 1, - "items": { "$ref": "#/definitions/test" } - }, - "_yamlAnchors": { - "type": "object", - "additionalProperties": true - } - }, - - "definitions": { - "version": { - "type": "string", - "pattern": "^[0-9]+(\\.[0-9]+){1,2}$" - }, - - "runOnRequirement": { - "type": "object", - "additionalProperties": false, - "minProperties": 1, - "properties": { - "maxServerVersion": { "$ref": "#/definitions/version" }, - "minServerVersion": { "$ref": "#/definitions/version" }, - "topologies": { + "description": { + "type": "string" + }, + "schemaVersion": { + "$ref": "#/definitions/version" + }, + "runOnRequirements": { "type": "array", "minItems": 1, "items": { - "type": "string", - "enum": ["single", "replicaset", "sharded", "sharded-replicaset", "load-balanced"] + "$ref": "#/definitions/runOnRequirement" } - }, - "serverless": { - "type": "string", - "enum": ["require", "forbid", "allow"] - }, - "serverParameters": { + }, + "createEntities": { + "type": "array", + "minItems": 1, + "items": { + "$ref": "#/definitions/entity" + } + }, + "initialData": { + "type": "array", + "minItems": 1, + "items": { + "$ref": "#/definitions/collectionData" + } + }, + "tests": { + "type": "array", + "minItems": 1, + "items": { + "$ref": "#/definitions/test" + } + }, + "_yamlAnchors": { "type": "object", - "minProperties": 1 - }, - "auth": { "type": "boolean" }, - "authMechanism": { "type": "string" }, - "csfle": { "type": "boolean" } + "additionalProperties": true } - }, - - "entity": { - "type": "object", - "additionalProperties": false, - "maxProperties": 1, - "minProperties": 1, - "properties": { - "client": { + }, + "definitions": { + "version": { + "type": "string", + "pattern": "^[0-9]+(\\.[0-9]+){1,2}$" + }, + "runOnRequirement": { "type": "object", "additionalProperties": false, - "required": ["id"], + "minProperties": 1, "properties": { - "id": { "type": "string" }, - "uriOptions": { "type": "object" }, - "useMultipleMongoses": { "type": "boolean" }, - "observeEvents": { - "type": "array", - "minItems": 1, - "items": { - "type": "string", - "enum": [ - "commandStartedEvent", - "commandSucceededEvent", - "commandFailedEvent", - "poolCreatedEvent", - "poolReadyEvent", - "poolClearedEvent", - "poolClosedEvent", - "connectionCreatedEvent", - "connectionReadyEvent", - "connectionClosedEvent", - "connectionCheckOutStartedEvent", - "connectionCheckOutFailedEvent", - "connectionCheckedOutEvent", - "connectionCheckedInEvent", - "serverDescriptionChangedEvent", - "topologyDescriptionChangedEvent" - ] - } - }, - "ignoreCommandMonitoringEvents": { - "type": "array", - "minItems": 1, - "items": { "type": "string" } - }, - "storeEventsAsEntities": { - "type": "array", - "minItems": 1, - "items": { "$ref": "#/definitions/storeEventsAsEntity" } - }, - "observeLogMessages": { - "type": "object", - "minProperties": 1, - "additionalProperties": false, - "properties": { - "command": { "$ref": "#/definitions/logSeverityLevel" }, - "topology": { "$ref": "#/definitions/logSeverityLevel" }, - "serverSelection": { "$ref": "#/definitions/logSeverityLevel" }, - "connection": { "$ref": "#/definitions/logSeverityLevel" } + "maxServerVersion": { + "$ref": "#/definitions/version" + }, + "minServerVersion": { + "$ref": "#/definitions/version" + }, + "topologies": { + "type": "array", + "minItems": 1, + "items": { + "type": "string", + "enum": [ + "single", + "replicaset", + "sharded", + "sharded-replicaset", + "load-balanced" + ] + } + }, + "serverless": { + "type": "string", + "enum": [ + "require", + "forbid", + "allow" + ] + }, + "serverParameters": { + "type": "object", + "minProperties": 1 + }, + "auth": { + "type": "boolean" + }, + "authMechanism": { + "type": "string" + }, + "csfle": { + "type": "boolean" } - }, - "serverApi": { "$ref": "#/definitions/serverApi" }, - "observeSensitiveCommands": { "type": "boolean" } } - }, - "clientEncryption": { + }, + "entity": { "type": "object", "additionalProperties": false, - "required": ["id", "clientEncryptionOpts"], + "maxProperties": 1, + "minProperties": 1, "properties": { - "id": { "type": "string" }, - "clientEncryptionOpts": { "$ref": "#/definitions/clientEncryptionOpts" } + "client": { + "type": "object", + "additionalProperties": false, + "required": [ + "id" + ], + "properties": { + "id": { + "type": "string" + }, + "uriOptions": { + "type": "object" + }, + "useMultipleMongoses": { + "type": "boolean" + }, + "observeEvents": { + "type": "array", + "minItems": 1, + "items": { + "type": "string", + "enum": [ + "commandStartedEvent", + "commandSucceededEvent", + "commandFailedEvent", + "poolCreatedEvent", + "poolReadyEvent", + "poolClearedEvent", + "poolClosedEvent", + "connectionCreatedEvent", + "connectionReadyEvent", + "connectionClosedEvent", + "connectionCheckOutStartedEvent", + "connectionCheckOutFailedEvent", + "connectionCheckedOutEvent", + "connectionCheckedInEvent", + "serverDescriptionChangedEvent", + "topologyDescriptionChangedEvent" + ] + } + }, + "ignoreCommandMonitoringEvents": { + "type": "array", + "minItems": 1, + "items": { + "type": "string" + } + }, + "storeEventsAsEntities": { + "type": "array", + "minItems": 1, + "items": { + "$ref": "#/definitions/storeEventsAsEntity" + } + }, + "observeLogMessages": { + "type": "object", + "minProperties": 1, + "additionalProperties": false, + "properties": { + "command": { + "$ref": "#/definitions/logSeverityLevel" + }, + "topology": { + "$ref": "#/definitions/logSeverityLevel" + }, + "serverSelection": { + "$ref": "#/definitions/logSeverityLevel" + }, + "connection": { + "$ref": "#/definitions/logSeverityLevel" + } + } + }, + "serverApi": { + "$ref": "#/definitions/serverApi" + }, + "observeSensitiveCommands": { + "type": "boolean" + } + } + }, + "clientEncryption": { + "type": "object", + "additionalProperties": false, + "required": [ + "id", + "clientEncryptionOpts" + ], + "properties": { + "id": { + "type": "string" + }, + "clientEncryptionOpts": { + "$ref": "#/definitions/clientEncryptionOpts" + } + } + }, + "database": { + "type": "object", + "additionalProperties": false, + "required": [ + "id", + "client", + "databaseName" + ], + "properties": { + "id": { + "type": "string" + }, + "client": { + "type": "string" + }, + "databaseName": { + "type": "string" + }, + "databaseOptions": { + "$ref": "#/definitions/collectionOrDatabaseOptions" + } + } + }, + "collection": { + "type": "object", + "additionalProperties": false, + "required": [ + "id", + "database", + "collectionName" + ], + "properties": { + "id": { + "type": "string" + }, + "database": { + "type": "string" + }, + "collectionName": { + "type": "string" + }, + "collectionOptions": { + "$ref": "#/definitions/collectionOrDatabaseOptions" + } + } + }, + "session": { + "type": "object", + "additionalProperties": false, + "required": [ + "id", + "client" + ], + "properties": { + "id": { + "type": "string" + }, + "client": { + "type": "string" + }, + "sessionOptions": { + "type": "object" + } + } + }, + "bucket": { + "type": "object", + "additionalProperties": false, + "required": [ + "id", + "database" + ], + "properties": { + "id": { + "type": "string" + }, + "database": { + "type": "string" + }, + "bucketOptions": { + "type": "object" + } + } + }, + "thread": { + "type": "object", + "additionalProperties": false, + "required": [ + "id" + ], + "properties": { + "id": { + "type": "string" + } + } + } } - }, - "database": { + }, + "logComponent": { + "type": "string", + "enum": [ + "command", + "topology", + "serverSelection", + "connection" + ] + }, + "logSeverityLevel": { + "type": "string", + "enum": [ + "emergency", + "alert", + "critical", + "error", + "warning", + "notice", + "info", + "debug", + "trace" + ] + }, + "clientEncryptionOpts": { "type": "object", "additionalProperties": false, - "required": ["id", "client", "databaseName"], + "required": [ + "keyVaultClient", + "keyVaultNamespace", + "kmsProviders" + ], "properties": { - "id": { "type": "string" }, - "client": { "type": "string" }, - "databaseName": { "type": "string" }, - "databaseOptions": { "$ref": "#/definitions/collectionOrDatabaseOptions" } + "keyVaultClient": { + "type": "string" + }, + "keyVaultNamespace": { + "type": "string" + }, + "kmsProviders": { + "$ref": "#/definitions/kmsProviders" + } } - }, - "collection": { + }, + "kmsProviders": { + "$defs": { + "stringOrPlaceholder": { + "oneOf": [ + { + "type": "string" + }, + { + "type": "object", + "additionalProperties": false, + "required": [ + "$$placeholder" + ], + "properties": { + "$$placeholder": {} + } + } + ] + } + }, "type": "object", "additionalProperties": false, - "required": ["id", "database", "collectionName"], - "properties": { - "id": { "type": "string" }, - "database": { "type": "string" }, - "collectionName": { "type": "string" }, - "collectionOptions": { "$ref": "#/definitions/collectionOrDatabaseOptions" } + "patternProperties": { + "^aws(:[a-zA-Z0-9_]+)?$": { + "type": "object", + "additionalProperties": false, + "properties": { + "accessKeyId": { + "$ref": "#/definitions/kmsProviders/$defs/stringOrPlaceholder" + }, + "secretAccessKey": { + "$ref": "#/definitions/kmsProviders/$defs/stringOrPlaceholder" + }, + "sessionToken": { + "$ref": "#/definitions/kmsProviders/$defs/stringOrPlaceholder" + } + } + }, + "^azure(:[a-zA-Z0-9_]+)?$": { + "type": "object", + "additionalProperties": false, + "properties": { + "tenantId": { + "$ref": "#/definitions/kmsProviders/$defs/stringOrPlaceholder" + }, + "clientId": { + "$ref": "#/definitions/kmsProviders/$defs/stringOrPlaceholder" + }, + "clientSecret": { + "$ref": "#/definitions/kmsProviders/$defs/stringOrPlaceholder" + }, + "identityPlatformEndpoint": { + "$ref": "#/definitions/kmsProviders/$defs/stringOrPlaceholder" + } + } + }, + "^gcp(:[a-zA-Z0-9_]+)?$": { + "type": "object", + "additionalProperties": false, + "properties": { + "email": { + "$ref": "#/definitions/kmsProviders/$defs/stringOrPlaceholder" + }, + "privateKey": { + "$ref": "#/definitions/kmsProviders/$defs/stringOrPlaceholder" + }, + "endpoint": { + "$ref": "#/definitions/kmsProviders/$defs/stringOrPlaceholder" + } + } + }, + "^kmip(:[a-zA-Z0-9_]+)?$": { + "type": "object", + "additionalProperties": false, + "properties": { + "endpoint": { + "$ref": "#/definitions/kmsProviders/$defs/stringOrPlaceholder" + } + } + }, + "^local(:[a-zA-Z0-9_]+)?$": { + "type": "object", + "additionalProperties": false, + "properties": { + "key": { + "$ref": "#/definitions/kmsProviders/$defs/stringOrPlaceholder" + } + } + } } - }, - "session": { + }, + "storeEventsAsEntity": { "type": "object", "additionalProperties": false, - "required": ["id", "client"], + "required": [ + "id", + "events" + ], "properties": { - "id": { "type": "string" }, - "client": { "type": "string" }, - "sessionOptions": { "type": "object" } + "id": { + "type": "string" + }, + "events": { + "type": "array", + "minItems": 1, + "items": { + "type": "string", + "enum": [ + "PoolCreatedEvent", + "PoolReadyEvent", + "PoolClearedEvent", + "PoolClosedEvent", + "ConnectionCreatedEvent", + "ConnectionReadyEvent", + "ConnectionClosedEvent", + "ConnectionCheckOutStartedEvent", + "ConnectionCheckOutFailedEvent", + "ConnectionCheckedOutEvent", + "ConnectionCheckedInEvent", + "CommandStartedEvent", + "CommandSucceededEvent", + "CommandFailedEvent", + "ServerDescriptionChangedEvent", + "TopologyDescriptionChangedEvent" + ] + } + } } - }, - "bucket": { + }, + "collectionData": { "type": "object", "additionalProperties": false, - "required": ["id", "database"], + "required": [ + "collectionName", + "databaseName", + "documents" + ], "properties": { - "id": { "type": "string" }, - "database": { "type": "string" }, - "bucketOptions": { "type": "object" } + "collectionName": { + "type": "string" + }, + "databaseName": { + "type": "string" + }, + "createOptions": { + "type": "object", + "properties": { + "writeConcern": false + } + }, + "documents": { + "type": "array", + "items": { + "type": "object" + } + } } - }, - "thread": { + }, + "expectedEventsForClient": { "type": "object", "additionalProperties": false, - "required": ["id"], + "required": [ + "client", + "events" + ], "properties": { - "id": { "type": "string" } - } - } - } - }, - - "logComponent": { - "type": "string", - "enum": ["command", "topology", "serverSelection", "connection"] - }, - - "logSeverityLevel": { - "type": "string", - "enum": ["emergency", "alert", "critical", "error", "warning", "notice", "info", "debug", "trace"] - }, - - "clientEncryptionOpts": { - "type": "object", - "additionalProperties": false, - "required": ["keyVaultClient", "keyVaultNamespace", "kmsProviders"], - "properties": { - "keyVaultClient": { "type": "string" }, - "keyVaultNamespace": { "type": "string" }, - "kmsProviders": { "$ref": "#/definitions/kmsProviders" } - } - }, - - "kmsProviders": { - "$defs": { - "stringOrPlaceholder": { + "client": { + "type": "string" + }, + "eventType": { + "type": "string", + "enum": [ + "command", + "cmap", + "sdam" + ] + }, + "events": { + "type": "array" + }, + "ignoreExtraEvents": { + "type": "boolean" + } + }, "oneOf": [ - { - "type": "string" - }, - { - "type": "object", - "additionalProperties": false, - "required": ["$$placeholder"], - "properties": { - "$$placeholder": {} + { + "required": [ + "eventType" + ], + "properties": { + "eventType": { + "const": "command" + }, + "events": { + "type": "array", + "items": { + "$ref": "#/definitions/expectedCommandEvent" + } + } + } + }, + { + "required": [ + "eventType" + ], + "properties": { + "eventType": { + "const": "cmap" + }, + "events": { + "type": "array", + "items": { + "$ref": "#/definitions/expectedCmapEvent" + } + } + } + }, + { + "required": [ + "eventType" + ], + "properties": { + "eventType": { + "const": "sdam" + }, + "events": { + "type": "array", + "items": { + "$ref": "#/definitions/expectedSdamEvent" + } + } + } + }, + { + "additionalProperties": false, + "properties": { + "client": { + "type": "string" + }, + "events": { + "type": "array", + "items": { + "$ref": "#/definitions/expectedCommandEvent" + } + }, + "ignoreExtraEvents": { + "type": "boolean" + } + } } - } ] - } }, - "type": "object", - "additionalProperties": false, - "properties": { - "aws": { - "type": "object", - "additionalProperties": false, - "properties": { - "accessKeyId": { "$ref": "#/definitions/kmsProviders/$defs/stringOrPlaceholder" }, - "secretAccessKey": { "$ref": "#/definitions/kmsProviders/$defs/stringOrPlaceholder" }, - "sessionToken": { "$ref": "#/definitions/kmsProviders/$defs/stringOrPlaceholder" } - } - }, - "azure": { - "type": "object", - "additionalProperties": false, - "properties": { - "tenantId": { "$ref": "#/definitions/kmsProviders/$defs/stringOrPlaceholder" }, - "clientId": { "$ref": "#/definitions/kmsProviders/$defs/stringOrPlaceholder" }, - "clientSecret": { "$ref": "#/definitions/kmsProviders/$defs/stringOrPlaceholder" }, - "identityPlatformEndpoint": { "$ref": "#/definitions/kmsProviders/$defs/stringOrPlaceholder" } - } - }, - "gcp": { + "expectedCommandEvent": { "type": "object", "additionalProperties": false, + "maxProperties": 1, + "minProperties": 1, "properties": { - "email": { "$ref": "#/definitions/kmsProviders/$defs/stringOrPlaceholder" }, - "privateKey": { "$ref": "#/definitions/kmsProviders/$defs/stringOrPlaceholder" }, - "endpoint": { "$ref": "#/definitions/kmsProviders/$defs/stringOrPlaceholder" } - } - }, - "kmip": { - "type": "object", - "additionalProperties": false, - "properties": { - "endpoint": { "$ref": "#/definitions/kmsProviders/$defs/stringOrPlaceholder" } - } - }, - "local": { - "type": "object", - "additionalProperties": false, - "properties": { - "key": { "$ref": "#/definitions/kmsProviders/$defs/stringOrPlaceholder" } - } - } - } - }, - - "storeEventsAsEntity": { - "type": "object", - "additionalProperties": false, - "required": ["id", "events"], - "properties": { - "id": { "type": "string" }, - "events": { - "type": "array", - "minItems": 1, - "items": { - "type": "string", - "enum": [ - "PoolCreatedEvent", - "PoolReadyEvent", - "PoolClearedEvent", - "PoolClosedEvent", - "ConnectionCreatedEvent", - "ConnectionReadyEvent", - "ConnectionClosedEvent", - "ConnectionCheckOutStartedEvent", - "ConnectionCheckOutFailedEvent", - "ConnectionCheckedOutEvent", - "ConnectionCheckedInEvent", - "CommandStartedEvent", - "CommandSucceededEvent", - "CommandFailedEvent", - "ServerDescriptionChangedEvent", - "TopologyDescriptionChangedEvent" - ] - } - } - } - }, - - "collectionData": { - "type": "object", - "additionalProperties": false, - "required": ["collectionName", "databaseName", "documents"], - "properties": { - "collectionName": { "type": "string" }, - "databaseName": { "type": "string" }, - "createOptions": { - "type": "object", - "properties": { - "writeConcern": false + "commandStartedEvent": { + "type": "object", + "additionalProperties": false, + "properties": { + "command": { + "type": "object" + }, + "commandName": { + "type": "string" + }, + "databaseName": { + "type": "string" + }, + "hasServiceId": { + "type": "boolean" + }, + "hasServerConnectionId": { + "type": "boolean" + } + } + }, + "commandSucceededEvent": { + "type": "object", + "additionalProperties": false, + "properties": { + "reply": { + "type": "object" + }, + "commandName": { + "type": "string" + }, + "databaseName": { + "type": "string" + }, + "hasServiceId": { + "type": "boolean" + }, + "hasServerConnectionId": { + "type": "boolean" + } + } + }, + "commandFailedEvent": { + "type": "object", + "additionalProperties": false, + "properties": { + "commandName": { + "type": "string" + }, + "databaseName": { + "type": "string" + }, + "hasServiceId": { + "type": "boolean" + }, + "hasServerConnectionId": { + "type": "boolean" + } + } + } } - }, - "documents": { - "type": "array", - "items": { "type": "object" } - } - } - }, - - "expectedEventsForClient": { - "type": "object", - "additionalProperties": false, - "required": ["client", "events"], - "properties": { - "client": { "type": "string" }, - "eventType": { - "type": "string", - "enum": ["command", "cmap", "sdam"] - }, - "events": { "type": "array" }, - "ignoreExtraEvents": { "type": "boolean" } }, - "oneOf": [ - { - "required": ["eventType"], - "properties": { - "eventType": { "const": "command" }, - "events": { - "type": "array", - "items": { "$ref": "#/definitions/expectedCommandEvent" } - } - } - }, - { - "required": ["eventType"], - "properties": { - "eventType": { "const": "cmap" }, - "events": { - "type": "array", - "items": { "$ref": "#/definitions/expectedCmapEvent" } - } - } - }, - { - "required": ["eventType"], - "properties": { - "eventType": { "const": "sdam" }, - "events": { - "type": "array", - "items": { "$ref": "#/definitions/expectedSdamEvent" } - } - } - }, - { - "additionalProperties": false, - "properties": { - "client": { "type": "string" }, - "events": { - "type": "array", - "items": { "$ref": "#/definitions/expectedCommandEvent" } - }, - "ignoreExtraEvents": { "type": "boolean" } - } - } - ] - }, - - "expectedCommandEvent": { - "type": "object", - "additionalProperties": false, - "maxProperties": 1, - "minProperties": 1, - "properties": { - "commandStartedEvent": { + "expectedCmapEvent": { "type": "object", "additionalProperties": false, + "maxProperties": 1, + "minProperties": 1, "properties": { - "command": { "type": "object" }, - "commandName": { "type": "string" }, - "databaseName": { "type": "string" }, - "hasServiceId": { "type": "boolean" }, - "hasServerConnectionId": { "type": "boolean" } + "poolCreatedEvent": { + "type": "object", + "additionalProperties": false, + "properties": {} + }, + "poolReadyEvent": { + "type": "object", + "additionalProperties": false, + "properties": {} + }, + "poolClearedEvent": { + "type": "object", + "additionalProperties": false, + "properties": { + "hasServiceId": { + "type": "boolean" + }, + "interruptInUseConnections": { + "type": "boolean" + } + } + }, + "poolClosedEvent": { + "type": "object", + "additionalProperties": false, + "properties": {} + }, + "connectionCreatedEvent": { + "type": "object", + "additionalProperties": false, + "properties": {} + }, + "connectionReadyEvent": { + "type": "object", + "additionalProperties": false, + "properties": {} + }, + "connectionClosedEvent": { + "type": "object", + "additionalProperties": false, + "properties": { + "reason": { + "type": "string" + } + } + }, + "connectionCheckOutStartedEvent": { + "type": "object", + "additionalProperties": false, + "properties": {} + }, + "connectionCheckOutFailedEvent": { + "type": "object", + "additionalProperties": false, + "properties": { + "reason": { + "type": "string" + } + } + }, + "connectionCheckedOutEvent": { + "type": "object", + "additionalProperties": false, + "properties": {} + }, + "connectionCheckedInEvent": { + "type": "object", + "additionalProperties": false, + "properties": {} + } } - }, - "commandSucceededEvent": { + }, + "expectedSdamEvent": { "type": "object", "additionalProperties": false, + "maxProperties": 1, + "minProperties": 1, "properties": { - "reply": { "type": "object" }, - "commandName": { "type": "string" }, - "databaseName": { "type": "string" }, - "hasServiceId": { "type": "boolean" }, - "hasServerConnectionId": { "type": "boolean" } + "serverDescriptionChangedEvent": { + "type": "object", + "additionalProperties": false, + "properties": { + "previousDescription": { + "$ref": "#/definitions/serverDescription" + }, + "newDescription": { + "$ref": "#/definitions/serverDescription" + } + } + }, + "topologyDescriptionChangedEvent": { + "type": "object", + "additionalProperties": false, + "properties": {} + }, + "serverHeartbeatStartedEvent": { + "type": "object", + "additionalProperties": false, + "properties": { + "awaited": { + "type": "boolean" + } + } + }, + "serverHeartbeatSucceededEvent": { + "type": "object", + "additionalProperties": false, + "properties": { + "awaited": { + "type": "boolean" + } + } + }, + "serverHeartbeatFailedEvent": { + "type": "object", + "additionalProperties": false, + "properties": { + "awaited": { + "type": "boolean" + } + } + } } - }, - "commandFailedEvent": { + }, + "serverDescription": { "type": "object", "additionalProperties": false, "properties": { - "commandName": { "type": "string" }, - "databaseName": { "type": "string" }, - "hasServiceId": { "type": "boolean" }, - "hasServerConnectionId": { "type": "boolean" } + "type": { + "type": "string", + "enum": [ + "Standalone", + "Mongos", + "PossiblePrimary", + "RSPrimary", + "RSSecondary", + "RSOther", + "RSArbiter", + "RSGhost", + "LoadBalancer", + "Unknown" + ] + } } - } - } - }, - - "expectedCmapEvent": { - "type": "object", - "additionalProperties": false, - "maxProperties": 1, - "minProperties": 1, - "properties": { - "poolCreatedEvent": { - "type": "object", - "additionalProperties": false, - "properties": {} - }, - "poolReadyEvent": { - "type": "object", - "additionalProperties": false, - "properties": {} - }, - "poolClearedEvent": { + }, + "expectedLogMessagesForClient": { "type": "object", "additionalProperties": false, + "required": [ + "client", + "messages" + ], "properties": { - "hasServiceId": { "type": "boolean" }, - "interruptInUseConnections": { "type": "boolean" } + "client": { + "type": "string" + }, + "messages": { + "type": "array", + "items": { + "$ref": "#/definitions/expectedLogMessage" + } + }, + "ignoreExtraMessages": { + "type": "boolean" + }, + "ignoreMessages": { + "type": "array", + "items": { + "$ref": "#/definitions/expectedLogMessage" + } + } } - }, - "poolClosedEvent": { - "type": "object", - "additionalProperties": false, - "properties": {} - }, - "connectionCreatedEvent": { - "type": "object", - "additionalProperties": false, - "properties": {} - }, - "connectionReadyEvent": { - "type": "object", - "additionalProperties": false, - "properties": {} - }, - "connectionClosedEvent": { + }, + "expectedLogMessage": { "type": "object", "additionalProperties": false, + "required": [ + "level", + "component", + "data" + ], "properties": { - "reason": { "type": "string" } + "level": { + "$ref": "#/definitions/logSeverityLevel" + }, + "component": { + "$ref": "#/definitions/logComponent" + }, + "data": { + "type": "object" + }, + "failureIsRedacted": { + "type": "boolean" + } } - }, - "connectionCheckOutStartedEvent": { - "type": "object", - "additionalProperties": false, - "properties": {} - }, - "connectionCheckOutFailedEvent": { + }, + "collectionOrDatabaseOptions": { "type": "object", "additionalProperties": false, "properties": { - "reason": { "type": "string" } + "readConcern": { + "type": "object" + }, + "readPreference": { + "type": "object" + }, + "writeConcern": { + "type": "object" + }, + "timeoutMS": { + "type": "integer" + } } - }, - "connectionCheckedOutEvent": { - "type": "object", - "additionalProperties": false, - "properties": {} - }, - "connectionCheckedInEvent": { - "type": "object", - "additionalProperties": false, - "properties": {} - } - } - }, - - "expectedSdamEvent": { - "type": "object", - "additionalProperties": false, - "maxProperties": 1, - "minProperties": 1, - "properties": { - "serverDescriptionChangedEvent": { + }, + "serverApi": { "type": "object", "additionalProperties": false, + "required": [ + "version" + ], "properties": { - "previousDescription": { "$ref": "#/definitions/serverDescription" }, - "newDescription": { "$ref": "#/definitions/serverDescription" } + "version": { + "type": "string" + }, + "strict": { + "type": "boolean" + }, + "deprecationErrors": { + "type": "boolean" + } } - }, - "topologyDescriptionChangedEvent": { - "type": "object", - "additionalProperties": false, - "properties": {} - }, - "serverHeartbeatStartedEvent": { + }, + "operation": { "type": "object", "additionalProperties": false, + "required": [ + "name", + "object" + ], "properties": { - "awaited": { "type": "boolean" } - } - }, - "serverHeartbeatSucceededEvent": { + "name": { + "type": "string" + }, + "object": { + "type": "string" + }, + "arguments": { + "type": "object" + }, + "ignoreResultAndError": { + "type": "boolean" + }, + "expectError": { + "$ref": "#/definitions/expectedError" + }, + "expectResult": {}, + "saveResultAsEntity": { + "type": "string" + } + }, + "allOf": [ + { + "not": { + "required": [ + "expectError", + "expectResult" + ] + } + }, + { + "not": { + "required": [ + "expectError", + "saveResultAsEntity" + ] + } + }, + { + "not": { + "required": [ + "ignoreResultAndError", + "expectResult" + ] + } + }, + { + "not": { + "required": [ + "ignoreResultAndError", + "expectError" + ] + } + }, + { + "not": { + "required": [ + "ignoreResultAndError", + "saveResultAsEntity" + ] + } + } + ] + }, + "expectedError": { "type": "object", "additionalProperties": false, + "minProperties": 1, "properties": { - "awaited": { "type": "boolean" } + "isError": { + "type": "boolean", + "const": true + }, + "isClientError": { + "type": "boolean" + }, + "isTimeoutError": { + "type": "boolean" + }, + "errorContains": { + "type": "string" + }, + "errorCode": { + "type": "integer" + }, + "errorCodeName": { + "type": "string" + }, + "errorLabelsContain": { + "type": "array", + "minItems": 1, + "items": { + "type": "string" + } + }, + "errorLabelsOmit": { + "type": "array", + "minItems": 1, + "items": { + "type": "string" + } + }, + "errorResponse": { + "type": "object" + }, + "expectResult": {} } - }, - "serverHeartbeatFailedEvent": { + }, + "test": { "type": "object", "additionalProperties": false, + "required": [ + "description", + "operations" + ], "properties": { - "awaited": { "type": "boolean" } - } - } - } - }, - - "serverDescription": { - "type": "object", - "additionalProperties": false, - "properties": { - "type": { - "type": "string", - "enum": [ - "Standalone", - "Mongos", - "PossiblePrimary", - "RSPrimary", - "RSSecondary", - "RSOther", - "RSArbiter", - "RSGhost", - "LoadBalancer", - "Unknown" - ] - } - } - }, - - "expectedLogMessagesForClient": { - "type": "object", - "additionalProperties": false, - "required": ["client", "messages"], - "properties": { - "client": { "type": "string" }, - "messages": { - "type": "array", - "items": { "$ref": "#/definitions/expectedLogMessage" } - }, - "ignoreExtraMessages": { "type": "boolean" }, - "ignoreMessages": { - "type": "array", - "items": { "$ref": "#/definitions/expectedLogMessage" } + "description": { + "type": "string" + }, + "runOnRequirements": { + "type": "array", + "minItems": 1, + "items": { + "$ref": "#/definitions/runOnRequirement" + } + }, + "skipReason": { + "type": "string" + }, + "operations": { + "type": "array", + "items": { + "$ref": "#/definitions/operation" + } + }, + "expectEvents": { + "type": "array", + "minItems": 1, + "items": { + "$ref": "#/definitions/expectedEventsForClient" + } + }, + "expectLogMessages": { + "type": "array", + "minItems": 1, + "items": { + "$ref": "#/definitions/expectedLogMessagesForClient" + } + }, + "outcome": { + "type": "array", + "minItems": 1, + "items": { + "$ref": "#/definitions/collectionData" + } + } } } - }, - - "expectedLogMessage": { - "type": "object", - "additionalProperties": false, - "required": ["level", "component", "data"], - "properties": { - "level": { "$ref": "#/definitions/logSeverityLevel" }, - "component": { "$ref": "#/definitions/logComponent" }, - "data": { "type": "object" }, - "failureIsRedacted": { "type": "boolean" } - } - }, - - "collectionOrDatabaseOptions": { - "type": "object", - "additionalProperties": false, - "properties": { - "readConcern": { "type": "object" }, - "readPreference": { "type": "object" }, - "writeConcern": { "type": "object" }, - "timeoutMS": { "type": "integer" } - } - }, - - "serverApi": { - "type": "object", - "additionalProperties": false, - "required": ["version"], - "properties": { - "version": { "type": "string" }, - "strict": { "type": "boolean" }, - "deprecationErrors": { "type": "boolean" } - } - }, - - "operation": { - "type": "object", - "additionalProperties": false, - "required": ["name", "object"], - "properties": { - "name": { "type": "string" }, - "object": { "type": "string" }, - "arguments": { "type": "object" }, - "ignoreResultAndError": { "type": "boolean" }, - "expectError": { "$ref": "#/definitions/expectedError" }, - "expectResult": {}, - "saveResultAsEntity": { "type": "string" } - }, - "allOf": [ - { "not": { "required": ["expectError", "expectResult"] } }, - { "not": { "required": ["expectError", "saveResultAsEntity"] } }, - { "not": { "required": ["ignoreResultAndError", "expectResult"] } }, - { "not": { "required": ["ignoreResultAndError", "expectError"] } }, - { "not": { "required": ["ignoreResultAndError", "saveResultAsEntity"] } } - ] - }, - - "expectedError": { - "type": "object", - "additionalProperties": false, - "minProperties": 1, - "properties": { - "isError": { - "type": "boolean", - "const": true - }, - "isClientError": { "type": "boolean" }, - "isTimeoutError": { "type": "boolean" }, - "errorContains": { "type": "string" }, - "errorCode": { "type": "integer" }, - "errorCodeName": { "type": "string" }, - "errorLabelsContain": { - "type": "array", - "minItems": 1, - "items": { "type": "string" } - }, - "errorLabelsOmit": { - "type": "array", - "minItems": 1, - "items": { "type": "string" } - }, - "errorResponse": { - "type": "object" - }, - "expectResult": {} - } - }, - - "test": { - "type": "object", - "additionalProperties": false, - "required": ["description", "operations"], - "properties": { - "description": { "type": "string" }, - "runOnRequirements": { - "type": "array", - "minItems": 1, - "items": { "$ref": "#/definitions/runOnRequirement" } - }, - "skipReason": { "type": "string" }, - "operations": { - "type": "array", - "items": { "$ref": "#/definitions/operation" } - }, - "expectEvents": { - "type": "array", - "minItems": 1, - "items": { "$ref": "#/definitions/expectedEventsForClient" } - }, - "expectLogMessages": { - "type": "array", - "minItems": 1, - "items": { "$ref": "#/definitions/expectedLogMessagesForClient" } - }, - "outcome": { - "type": "array", - "minItems": 1, - "items": { "$ref": "#/definitions/collectionData" } - } - } - } } - } +} diff --git a/source/unified-test-format/tests/Makefile b/source/unified-test-format/tests/Makefile index a434cd65a5..80de5bf329 100644 --- a/source/unified-test-format/tests/Makefile +++ b/source/unified-test-format/tests/Makefile @@ -1,4 +1,4 @@ -SCHEMA=../schema-1.18.json +SCHEMA=../schema-1.19.json .PHONY: all invalid valid-fail valid-pass versioned-api load-balancers gridfs transactions crud collection-management sessions command-logging-and-monitoring client-side-operations-timeout HAS_AJV diff --git a/source/unified-test-format/unified-test-format.rst b/source/unified-test-format/unified-test-format.rst index 8145a74662..0e01d396ec 100644 --- a/source/unified-test-format/unified-test-format.rst +++ b/source/unified-test-format/unified-test-format.rst @@ -460,7 +460,7 @@ The structure of this object is as follows: field is omitted, there is no authentication requirement. - ``authMechanism``: Optional string. Specifies an authentication mechanism that - the database needs to support for the test. If set, tests MUST only run if the + the server needs to support for the test. If set, tests MUST only run if the given string matches (case-insensitive) one of the strings in the `authenticationMechanisms `__