ConnectionDetails for external providers in Spring Security modules

[ParkerM]

Part of my dev workflow involves using Testcontainers with Keycloak for integration testing against a local, preconfigured IdP/authorization server. It would be awesome if I could use the very same configuration with a running instance through spring-boot-testcontainers, but from what I can tell this isn't currently possible unless the respective configurators implement or expose a ConnectionDetails in some way. (I assume I could achieve something like it if I replace the Spring Security autoconfigurations with my own, but I'd prefer to stick with conventions as much as possible.)

The use cases that come to mind are:

• OAuth2 client provider(s) - URLs defined under spring.security.oauth2.client.provider
• SAML asserting party - Metadata URI defined under spring.security.saml2.relyingparty.registration.<id>.assertingparty
• Probably a few other properties/modules I'm missing

Does this seem like a sensible use case for the ConnectionDetail abstraction? I've noticed the current implementations seem concerned with strictly persistent connections, so perhaps this does not fit the intent. But, since the provider auto-configurations can lead to startup failure when the server is unreachable, the connections at least have some sort of persistent essence.

[philwebb]

@jgrandja @jzheaux Do you have any thoughts on this?

[philwebb]

@ParkerM Can you share a bit more information about how you use Testcontainers with Keycloak? Are you using https://testcontainers.com/modules/keycloak/ ?

[ParkerM]

@philwebb yep, that's the one. Right now it looks a bit like the TestConfiguration + Bean method + injected DynamicPropertyRegistry approach described in the docs here (plus a few startup tasks that would ideally be rolled into the image or exist as opinionated defaults). This class is imported for int tests, and applied for the test application as from(MyApp::main).with(ContainerTestConfig.class).

I did manage to get the test application to start up with this configuration after some finagling — I was experiencing some sort of lifecycle issue with eager connections failing startup before the container was ready. Removing @RestartScope from the bean method caused startup to wait as expected, so I assume it was the cause or a catalyst if not PEBCAK. Unsure if this is in scope for this issue, but wanted to mention it as another reason I yearn for a prescribed convention and to override as a last resort.

---

In any case, I'm curious whether the goals of the ConnectionDetails abstraction fundamentally align here (even if it's for my own selfish understanding of the API and its goals). If my understanding is correct then the Spring Security configs could potentially be a good fit, since most contain a representation of remote service(s) and the credentials used to connect. This would also provide more sensible means for programmatically accessing validated connection properties. My current approach involves injecting Boot's ConfigurationProperties beans and clumsily accessing deeply nested properties. Separating that concern into ConnectionDetails would make for more obvious injections and provide more convenient options for test stubs and such.

Apologies if the info above isn't what you were looking for. If so let me know and I can expand or perhaps create an example project on my own time.

[mhalbritter]

We talked about this yesterday and we think the ConnectionDetails abstraction fits for something like this, too.

The approach would be the following:

1. Implement ConnectionDetails in the OAuth2 client auto-configuration
2. Implement adapters from the Keycloak testcontainer to the newly created ConnectionDetails
3. Optional, but would be very nice to have: Implement adapters from a Keycloak docker compose service to ConnectionDetails.

Step 2 and 3 is the tricky bit.

I've not looked at the Keycloak testcontainer in detail, but I remember from the standalone Keycloak that one has to do some realm / client setup before it's usable.

[ParkerM]

I've not looked at the Keycloak testcontainer in detail, but I remember from the standalone Keycloak that one has to do some realm / client setup before it's usable.

Yeah, I currently have realm import do most of the work then use the Keycloak client to adjust some ports before the tests run (maybe the future will see something like a configurer DSL that simplifies test setup).

My original plan was to just implement my own ContainerConnectionDetails and such, which led me to realize ConnectionDetails would be the blocker unless I forego Spring Security AutoConfigurations.

[Kehrlann]

Would definitely like to see this happen as well, for any generic OpenID or OAuth2 provider. I'm specifically thinking about Dex and custom-built authorization servers.

I'd want client_id and client_secret (optionally) available on the ConnectionDetails.

[jgrandja]

I'm just seeing this now @philwebb. Apologies for the delay.

@rwinch and I have been talking about introducing integration testing support (spring-authorization-server#258) for Spring Authorization Server.

I'm not familiar with the ConnectionDetails abstraction so I would need to look into this and see if this is an entry point for the support.

[wilkinsona]

Moving into 3.4.x to consider alongside #39391 which doesn't tackle Testcontainers or Spring Authorization Server.

[wilkinsona]

I've been looking at #39391 which tackles part of this, and I'm struggling to figure out exactly how broad the connection details should be. It's not clear to me where we should draw the line in terms of what the connection details should override and what should come from the normal configuration.

Just focussing on the OAuth2 client side of things for now, @ParkerM, @Kehrlann, @PhilKes, could you please provide some more details about how you're configuring things at the moment (the application.properties or application.yaml) with spring.security.oauth2.client.registration.* and spring.security.oauth2.provider.* properties and then what you'd typically do with Keycloak to test things that are set up using these properties. I'm also curious if you'd only ever use Keycloak managed by Docker Compose or Testcontainers when the app would also use Keycloak in production, or if you try to use Keycloak as an easy-to-run-in-a-container substitute for something else.

[Kehrlann]

Hey @wilkinsona,

The one thing you absolutely need is the issuer-uri because the port of your testcontainer changes every time. When @rwinch started Spring Boot testjars, the first use-case was an auth-server. The only thing that we extract from the running authserver is the issuer-uri. You could then imagine that the rest is hardcoded in your test config to match whatever you configured in your IDP.

I have a lightweight Testcontainers module for the Dex IDP. I did my own starter with ConnectionDetails. Let's say you register your OAuth2 clients dynamically, then you need:

• issuer-uri
• client-secret, as this likely IDP-generated
• Maybe client-id, as this may be IDP-generated too (online IDPs do that they give you a random ID)

For the rest I'd imagine it comes from configuration:

• client-name is app specific
• scope is what your application wants
• redirectUri I usually use the defaults
• authorizationGrantType is usually configured to match what the IDP uses, but you know that in advance
• clientAuthenticationMethod same as grant types
• provider.* is usually derived from provider.issuer-uri

Keycloak in a container, why?

I'm also curious if you'd only ever use Keycloak managed by Docker Compose or Testcontainers when the app would also use Keycloak in production, or if you try to use Keycloak as an easy-to-run-in-a-container substitute for something else.

In my experience, Keycloak in a container is a pain - slow to start and the config is huge. I'm using Dex, and custom apps built with Spring-Auth-Server, as a substitute for online IDPs (Okta, Azure).

Learnings from using Dex in Testcontainers

The problem I had with Dex is that there is a lifecycle issue, at least in tests:

• The Boot app needs to known where the IDP is (issuer-uri)
• The IDP needs to know where the Boot app runs (to know its redirect_uris)

So when you run in tests and both have dynamic ports, this is harder to know. Current flow:

• Start the container, grab the issuer-uri
• Start the boot app, configured with the issuer-uri
• In the Boot app (through the starter), on WebServerInitializedEvent, register the client with Dex

Any provider that does strict redirect uri checking would have the same problem.

[ParkerM]

could you please provide some more details about how you're configuring things at the moment (the application.properties or application.yaml) with spring.security.oauth2.client.registration.* and spring.security.oauth2.provider.* properties and then what you'd typically do with Keycloak to test things that are set up using these properties.

In my case I was working on a federated login sort of thing that used spring authorization server as a primary OAuth2 provider and Keycloak as an "external" IdP. So, while I did need to set some registration properties at runtime to configure the complicated test, I think that's more specific to the peculiar nature of the project.

I didn't really foresee registration properties being in scope for ConnectionDetails because they either require configuration at the IdP (e.g., establishing redirect_uri for auth code grant), or are options that the app itself selects (e.g., requested scopes, grant type, client authentication method).

On the other hand, provider properties seem like a good fit because they are merely properties of the IdP itself, rather than properties of an established relationship between the application and the IdP. ConnectionDetails would be a convenient critical path from which IdP properties are consumed, be it physical properties like URL's or declared properties like supported algorithms — basically anything that the application must deal with or die. I realize this distinction neglects the fact that credentials are a registration property, so take what I say with a grain of salt.

In my particular case I was using Keycloak as a SAML provider, so I was interested in configuring the assertingparty.metadata-uri property. There are also many other provider-esque properties under the saml2 assertingparty namespace that can be used in lieu of a metadata-uri, but they lack the clear boundaries we get with oauth2's registration and provider so I'll skip them to avoid being distracting.

I'm also curious if you'd only ever use Keycloak managed by Docker Compose or Testcontainers when the app would also use Keycloak in production, or if you try to use Keycloak as an easy-to-run-in-a-container substitute for something else.

We support Keycloak as a provider and occasionally use it in production-like environments, but in this case I selected it for easy-to-run-in-a-container reasons. Namely for the convenience of realm export/import as a way to "serialize" a production instance so it can be reproduced in an ephemeral container with little fuss and minor additional configuration (and it's FOSS and supports SAML). The additional configuration not handled by the realm file boils down to (re-)configuring callback URI's via the Keycloak API.