Skip to content

Add support for escaped . in authz validation requests#161

Draft
sdewhitt wants to merge 6 commits into
amazon-mq:mainfrom
sdewhitt:wip/oauth-authz-escaped-dot-16947
Draft

Add support for escaped . in authz validation requests#161
sdewhitt wants to merge 6 commits into
amazon-mq:mainfrom
sdewhitt:wip/oauth-authz-escaped-dot-16947

Conversation

@sdewhitt

@sdewhitt sdewhitt commented Jul 21, 2026

Copy link
Copy Markdown
Member

This PR lands after both #159, #160, and rabbitmq-server 16985 merge.

rabbitmq-server 16947 brought to light issues with how "." splitting is handled, and rabbitmq-server 16985 adds support for escaping dots (like \.). Since said PR introduces such a change to the upstream OAuth backend, there needs to be proper support on the validation endpoint to maintain parity.

Background

The upstream OAuth backend resolves additional_scopes_key by splitting the key on . into a nested-map path (rabbit_auth_backend_oauth2:split_path/1). A flat claim key that itself contains dots — e.g. an STS/OIDC key like https://sts.amazonaws.com/tags — is therefore split into ["https://sts", "amazonaws", "com/tags"] and never found. rabbitmq-server#16985 fixes this by:

  • adding an escaped form \. (a literal dot in a segment) via rabbit_oauth2_schema:tokenize_additional_scopes_key/1, which produces a pre-tokenized [[binary()]] (list of paths, each a list of segments); and
  • teaching the record's additional_scopes_key to accept that list form directly (a new is_list/1 clause on extract_token_value/4 / extract_scopes_from_additional_scopes_key/2), so a pre-split key is consumed without being re-split.

Because the validation endpoint replays the broker's real scope-resolution code, it must interpret additional_scopes_key identically — otherwise an escaped key that the live broker would resolve would be reported as authz_unverified, breaking parity in the safe-but-misleading direction.

This PR adds said support by:

  • Delegating tokenization to the broker's own rabbit_oauth2_schema:tokenize_additional_scopes_key/1 rather than reimplementing the escape logic. This keeps the same "execute the broker's code, don't mirror it" guarantee the authz layer already relies on: the endpoint splits additional_scopes_key byte-for-byte the way auth_oauth2.additional_scopes_key does in rabbitmq.conf, so an escaped \. stays a literal dot in a flat claim name and an unescaped . remains a nesting separator.
  • Confining the change to aws_auth_validate_oauth_authz:evaluate/3 (a new tokenize_additional_scopes_key/1 helper that the record overlay routes through). The pure parse phase in aws_auth_validate_oauth is untouched — it still validates additional_scopes_key as a plain binary and never depends on the OAuth backend being loaded.
  • Keeping the request contract unchanged. Callers still send additional_scopes_key as a plain string; allowed_fields/0, the shape validation, and the response categories are all unchanged. This is purely additive — no new fields, no new statuses.
  • Falling back gracefully on pre-fix broker series. When rabbit_oauth2_schema:tokenize_additional_scopes_key/1 is not exported (a broker without rabbitmq-server#16985), the helper passes the raw binary through unchanged, so normalize_token_scope/2 takes its existing binary/split_path/1 clause and the endpoint behaves exactly as it does today. The feature stays available on old brokers; it simply lacks escaped-dot support there, matching what those brokers themselves can do. No change to the HAVE_OAUTH2_RESOURCE_SERVER compile gate or available/0.
  • Guarding against load order. The helper calls code:ensure_loaded/1 before erlang:function_exported/3 (a bare function_exported returns false for a beam that is on the path but not yet loaded, which would make escaped-dot support depend on load order — the same trap available/0 already documents).
  • Preserving parity in both directions, verified by tests:
    • authz_escaped_dotted_additional_scopes_key_test_ (new) — an escaped key https://sts\.amazonaws\.com/tags resolves the flat claim and grants (ok). Double-gated on maybe_skip_authz/1 (the arity-4 scope API) and tokenizer presence, so it skips cleanly on a pre-fix broker rather than failing.
    • authz_dotted_additional_scopes_key_parity_pin_test_ (from OAuth Authz validation parity test enhancements #160) — an unescaped key still splits and does not resolve a flat dotted claim (authz_unverified). Its comment is corrected to state that the fix adds an escaped form; it does not make unescaped dots resolve a flat key.

Verification

  • gmake eunit t=aws_auth_validate_oauth_tests — all tests pass; escaped→ok and unescaped→authz_unverified confirmed executing (not skipped) with the fixed OAuth backend on the code path.
  • Manually exercised end-to-end through PUT /api/aws/auth/validate/oauth against a broker built with rabbitmq-server#16985: same token carrying a flat dotted claim, escaped additional_scopes_key204, unescaped → 422 authz_unverified.
  • erlfmt -c clean.

Notes for reviewers

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

sdewhitt and others added 6 commits July 20, 2026 14:33
…back-only test flag

  Follow-ups from a post-merge review, stacked without individual issues.

  - aws_auth_validate_ldap: express the LDAP SSRF range policy as
    ?LDAP_DENIED_V4_CIDRS/?LDAP_DENIED_V6_CIDRS consumed by the shared
    aws_auth_validate_net:in_any_cidr/2, replacing the hand-coded guard
    clauses. LDAP keeps its own broader list (all RFC1918/CGNAT/reserved,
    not just broker infra) but both backends now enumerate ranges in ONE
    vocabulary, so a future range or segment-math fix cannot silently apply
    to only one classifier. Mirrors the amazon-mq#153 embedded_v4 unification, one
    layer up (F3).
  - aws_auth_validate_ldap: default an omitted port to 389, matching
    rabbit_auth_backend_ldap's application-env default (the broker does not
    bump to 636 for TLS). A config the live broker would accept no longer
    fails input_invalid for parity's sake (F5).
  - aws_auth_validate_ldap: the test-only auth_validation_allow_private_networks
    flag now relaxes ONLY loopback (via is_denied_ip/1, mirroring
    aws_auth_validate_net:classify_denied/1) instead of fully bypassing both
    SSRF checks. IMDS/link-local/RFC1918/CGNAT stay denied even under the
    flag, so the LDAP backend no longer gets a broader test-mode bypass than
    http/oauth. Both the pre-connect and post-connect peer checks always run
    (F7).
  - tests: cover the loopback-only relaxation (loopback reachable; IMDS/
    RFC1918/CGNAT still denied on both the pre-connect and post-connect
    checks) and the port default (absent -> 389, supplied honoured,
    out-of-range/non-integer still rejected). Export parse_port/2 under
    -ifdef(TEST). All 205 eunit tests pass.
…ity (amazon-mq#157, amazon-mq#158)

  - aws_auth_validate.js: add body_too_large to AWS_AUTH_VALIDATE_CATEGORY.
    The backend returns {error:"body_too_large"} at HTTP 400 and 4ee6ff7
    added it to status_for_category/1, but the UI status map never got the
    matching key, so an oversized config fell through to the generic
    "Unexpected response (400)" banner instead of an actionable message. It
    was the sole backend category missing from the map (amazon-mq#157).
  - aws_auth_validate_oauth: allow a pre-existing query string on jwks_uri
    (both the directly-supplied one and the one discovered from an OIDC
    document). jwks_uri is fetched VERBATIM -- the live broker's
    uaa_jwks:get/2 fetches it as-is -- so rejecting a query string
    input_invalid diverged too-strict from the broker, which fetches it
    fine. issuer keeps rejecting a query (OIDC discovery appends the
    well-known path, so a query there is ambiguous). Threaded via a
    QueryPolicy (allow_query | reject_query) arg on parse_url/2; parse_url/1
    keeps the reject_query default (amazon-mq#158).
  - tests: jwks_uri-with-query is now accepted in the pure phase (asserted
    via parse_input/1, no network) and issuer-with-query is still rejected,
    replacing the old url_with_query_rejected_test. All 475 auth-validate
    eunit tests pass.
  The CI production build (make app, warnings-as-errors, no -ifdef(TEST)
  exports) failed with "function parse_url/1 is unused". The prior commit
  moved both production call sites to parse_url/2 but left the parse_url/1
  wrapper, which only survived the local eunit build because parse_url/1 was
  in the -ifdef(TEST) export list (so it counted as exported there, never
  unused). Remove the dead wrapper and update the test export to parse_url/2.
  Both call sites already pass an explicit query policy (allow_query for
  jwks_uri, reject_query for issuer), so no behavior change.

  Verified: clean `make app` (production) build passes; 304 auth-validate
  eunit tests pass.
  - Add authz_dotted_additional_scopes_key_parity_pin_test_ exercising an
    STS-style dotted claim key (https://sts.amazonaws.com/tags) as
    additional_scopes_key. Runs the real upstream normalize_token_scope ->
    get_expanded_scopes -> resource_access path (not mocks), asserting the
    current authz_unverified/"no effective scopes" behavior: split_path/1
    mis-splits the dotted key on '.', so the flat claim is never found.
  - Flips to a grant when upstream resolves rabbitmq/rabbitmq-server
    discussion #16947; the pin surfaces that drift as a test failure and
    forces a coordinated broker-dependency bump.
  - Update the aws_auth_validate_oauth over-trust caveat comment: scope
    authorization is now checkable via the optional authz_check block
    (delegated to aws_auth_validate_oauth_authz), noting the remaining
    resource_server-field overlay parity gap.
  - Route the request's additional_scopes_key through the broker's own
    rabbit_oauth2_schema:tokenize_additional_scopes_key/1 when available, so
    the endpoint splits the key IDENTICALLY to auth_oauth2.additional_scopes_key
    in rabbitmq.conf: an escaped "\." stays a literal dot in a flat claim name
    (the STS/OIDC case behind rabbitmq/rabbitmq-server #16947), while an
    unescaped "." remains a nesting separator. The record's list-form clause
    consumes the tokenized [[binary()]] without re-splitting.
  - Probe with code:ensure_loaded before function_exported so escaped-dot
    support does not depend on load order (the trap available/0 documents).
  - Fallback: on a pre-#16947 broker series the tokenizer is absent, so we pass
    the raw binary through unchanged and normalize_token_scope/2's own
    split_path/1 applies -- the endpoint behaves exactly as before. The feature
    stays available on old brokers; it just lacks escaped-dot support there,
    matching what those brokers themselves can do. Request contract unchanged
    (still a plain string).
  - Add authz_escaped_dotted_additional_scopes_key_test_ (escaped key -> grant),
    double-gated on maybe_skip_authz + tokenizer presence so it skips cleanly on
    a pre-fix broker rather than failing. Correct the parity-pin comment: the fix
    adds an escaped form, it does not make unescaped dots resolve a flat key.
@sdewhitt sdewhitt added this to the 0.4.0 milestone Jul 21, 2026
@sdewhitt sdewhitt self-assigned this Jul 21, 2026
@sdewhitt sdewhitt added C-enhancement Category: Improvements A-auth-validation Area: The auth-validation endpoint (aws_auth_validate_*). E-small Effort: Easy. Little experience required. P-low Priority: Low. Minor; can wait. labels Jul 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-auth-validation Area: The auth-validation endpoint (aws_auth_validate_*). C-enhancement Category: Improvements E-small Effort: Easy. Little experience required. P-low Priority: Low. Minor; can wait.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant