[🐛 Bug]: Error while creating session with the driver service

[vmassol]

What happened?

The XWiki build broke recently on Mac M1 only, due to our TestContainer-based tests all failing. We're using:

• Latest testcontainers version: 1.18.3
• Latest Selenium java: 4.10.0
• Latest seleniarm image: seleniarm/standalone-firefox:latest (currently pointing to 114.0-geckodriver-0.33-20230615)

I've tried various combinations of different versions of TestContainers/Selenium/seleniarm image to no avail.

Not that it works fine on other oses (other than Mac M1) so the issue is mostly likely an incompatibility between the geckdriver version used with seleniarm latest (114.0).

Any idea of what I could do?

Command used to start Selenium Grid with Docker

Done by TestContainers.

Relevant log output

The logs show:


...
14:40:39.990 [docker-java-stream-1970431651] INFO  o.x.t.d.i.j.b.BrowserContainerExecutor - STDOUT: 14:40:39.988 INFO [LocalDistributor.newSession] - Session request received by the Distributor: 
14:40:39.990 [docker-java-stream-1970431651] INFO  o.x.t.d.i.j.b.BrowserContainerExecutor - STDOUT:  [Capabilities {acceptInsecureCerts: true, browserName: firefox, moz:debuggerAddress: true, moz:firefoxOptions: {prefs: {dom.disable_beforeunload: false}}, unhandledPromptBehavior: ignore}]
14:40:40.001 [docker-java-stream-1970431651] INFO  o.x.t.d.i.j.b.BrowserContainerExecutor - STDOUT: 14:40:40.000 INFO [DriverService$Builder.getLogOutput] - Driver logs no longer sent to console by default; https://www.selenium.dev/documentation/webdriver/drivers/service/#setting-log-output
14:40:40.061 [docker-java-stream-1970431651] INFO  o.x.t.d.i.j.b.BrowserContainerExecutor - STDOUT: 14:40:40.059 WARN [DriverServiceSessionFactory.apply] - Error while creating session with the driver service. Stopping driver service: Driver server process died prematurely.
14:40:40.061 [docker-java-stream-1970431651] INFO  o.x.t.d.i.j.b.BrowserContainerExecutor - STDOUT: Build info: version: '4.10.0', revision: 'c14d967899'
14:40:40.061 [docker-java-stream-1970431651] INFO  o.x.t.d.i.j.b.BrowserContainerExecutor - STDOUT: System info: os.name: 'Linux', os.arch: 'aarch64', os.version: '5.15.49-linuxkit-pr', java.version: '11.0.18'
14:40:40.061 [docker-java-stream-1970431651] INFO  o.x.t.d.i.j.b.BrowserContainerExecutor - STDOUT: Driver info: driver.version: unknown
14:40:45.065 [docker-java-stream-1970431651] INFO  o.x.t.d.i.j.b.BrowserContainerExecutor - STDOUT: 14:40:45.064 WARN [ExecutorServices.shutdownGracefully] - Failed to shutdown Driver Service Executor
14:40:45.067 [docker-java-stream-1970431651] INFO  o.x.t.d.i.j.b.BrowserContainerExecutor - STDOUT: 14:40:45.067 WARN [SeleniumSpanExporter$1.lambda$export$1] - Error while creating session with the driver service. Stopping driver service: Driver server process died prematurely.
14:40:45.068 [docker-java-stream-1970431651] INFO  o.x.t.d.i.j.b.BrowserContainerExecutor - STDOUT: Build info: version: '4.10.0', revision: 'c14d967899'
14:40:45.068 [docker-java-stream-1970431651] INFO  o.x.t.d.i.j.b.BrowserContainerExecutor - STDOUT: System info: os.name: 'Linux', os.arch: 'aarch64', os.version: '5.15.49-linuxkit-pr', java.version: '11.0.18'
14:40:45.068 [docker-java-stream-1970431651] INFO  o.x.t.d.i.j.b.BrowserContainerExecutor - STDOUT: Driver info: driver.version: unknown
14:40:45.068 [docker-java-stream-1970431651] INFO  o.x.t.d.i.j.b.BrowserContainerExecutor - STDOUT: 14:40:45.067 WARN [SeleniumSpanExporter$1.lambda$export$1] - org.openqa.selenium.WebDriverException: Driver server process died prematurely.
14:40:45.068 [docker-java-stream-1970431651] INFO  o.x.t.d.i.j.b.BrowserContainerExecutor - STDOUT: Build info: version: '4.10.0', revision: 'c14d967899'
14:40:45.068 [docker-java-stream-1970431651] INFO  o.x.t.d.i.j.b.BrowserContainerExecutor - STDOUT: System info: os.name: 'Linux', os.arch: 'aarch64', os.version: '5.15.49-linuxkit-pr', java.version: '11.0.18'
14:40:45.068 [docker-java-stream-1970431651] INFO  o.x.t.d.i.j.b.BrowserContainerExecutor - STDOUT: Driver info: driver.version: unknown
14:40:45.068 [docker-java-stream-1970431651] INFO  o.x.t.d.i.j.b.BrowserContainerExecutor - STDOUT: 	at org.openqa.selenium.remote.service.DriverService.start(DriverService.java:249)
14:40:45.068 [docker-java-stream-1970431651] INFO  o.x.t.d.i.j.b.BrowserContainerExecutor - STDOUT: 	at org.openqa.selenium.grid.node.config.DriverServiceSessionFactory.apply(DriverServiceSessionFactory.java:138)
14:40:45.068 [docker-java-stream-1970431651] INFO  o.x.t.d.i.j.b.BrowserContainerExecutor - STDOUT: 	at org.openqa.selenium.grid.node.config.DriverServiceSessionFactory.apply(DriverServiceSessionFactory.java:69)
14:40:45.068 [docker-java-stream-1970431651] INFO  o.x.t.d.i.j.b.BrowserContainerExecutor - STDOUT: 	at org.openqa.selenium.grid.node.local.SessionSlot.apply(SessionSlot.java:147)
14:40:45.068 [docker-java-stream-1970431651] INFO  o.x.t.d.i.j.b.BrowserContainerExecutor - STDOUT: 	at org.openqa.selenium.grid.node.local.LocalNode.newSession(LocalNode.java:467)
14:40:45.068 [docker-java-stream-1970431651] INFO  o.x.t.d.i.j.b.BrowserContainerExecutor - STDOUT: 	at org.openqa.selenium.grid.distributor.local.LocalDistributor.startSession(LocalDistributor.java:643)
14:40:45.068 [docker-java-stream-1970431651] INFO  o.x.t.d.i.j.b.BrowserContainerExecutor - STDOUT: 	at org.openqa.selenium.grid.distributor.local.LocalDistributor.newSession(LocalDistributor.java:560)
14:40:45.068 [docker-java-stream-1970431651] INFO  o.x.t.d.i.j.b.BrowserContainerExecutor - STDOUT: 	at org.openqa.selenium.grid.distributor.local.LocalDistributor$NewSessionRunnable.handleNewSessionRequest(LocalDistributor.java:814)
14:40:45.068 [docker-java-stream-1970431651] INFO  o.x.t.d.i.j.b.BrowserContainerExecutor - STDOUT: 	at org.openqa.selenium.grid.distributor.local.LocalDistributor$NewSessionRunnable.lambda$run$1(LocalDistributor.java:773)
14:40:45.068 [docker-java-stream-1970431651] INFO  o.x.t.d.i.j.b.BrowserContainerExecutor - STDOUT: 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
14:40:45.068 [docker-java-stream-1970431651] INFO  o.x.t.d.i.j.b.BrowserContainerExecutor - STDOUT: 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
14:40:45.068 [docker-java-stream-1970431651] INFO  o.x.t.d.i.j.b.BrowserContainerExecutor - STDOUT: 	at java.base/java.lang.Thread.run(Thread.java:829)
14:40:45.068 [docker-java-stream-1970431651] INFO  o.x.t.d.i.j.b.BrowserContainerExecutor - STDOUT: 
14:40:45.068 [docker-java-stream-1970431651] INFO  o.x.t.d.i.j.b.BrowserContainerExecutor - STDOUT: 14:40:45.068 WARN [SeleniumSpanExporter$1.lambda$export$3] - {"traceId": "e6eae8750348513a7b833e65c83b2fbb","eventTime": 1686919240059817666,"eventName": "exception","attributes": {"exception.message": "Error while creating session with the driver service. Stopping driver service: Driver server process died prematurely.\nBuild info: version: '4.10.0', revision: 'c14d967899'\nSystem info: os.name: 'Linux', os.arch: 'aarch64', os.version: '5.15.49-linuxkit-pr', java.version: '11.0.18'\nDriver info: driver.version: unknown","exception.stacktrace": "org.openqa.selenium.WebDriverException: Driver server process died prematurely.\nBuild info: version: '4.10.0', revision: 'c14d967899'\nSystem info: os.name: 'Linux', os.arch: 'aarch64', os.version: '5.15.49-linuxkit-pr', java.version: '11.0.18'\nDriver info: driver.version: unknown\n\tat org.openqa.selenium.remote.service.DriverService.start(DriverService.java:249)\n\tat org.openqa.selenium.grid.node.config.DriverServiceSessionFactory.apply(DriverServiceSessionFactory.java:138)\n\tat org.openqa.selenium.grid.node.config.DriverServiceSessionFactory.apply(DriverServiceSessionFactory.java:69)\n\tat org.openqa.selenium.grid.node.local.SessionSlot.apply(SessionSlot.java:147)\n\tat org.openqa.selenium.grid.node.local.LocalNode.newSession(LocalNode.java:467)\n\tat org.openqa.selenium.grid.distributor.local.LocalDistributor.startSession(LocalDistributor.java:643)\n\tat org.openqa.selenium.grid.distributor.local.LocalDistributor.newSession(LocalDistributor.java:560)\n\tat org.openqa.selenium.grid.distributor.local.LocalDistributor$NewSessionRunnable.handleNewSessionRequest(LocalDistributor.java:814)\n\tat org.openqa.selenium.grid.distributor.local.LocalDistributor$NewSessionRunnable.lambda$run$1(LocalDistributor.java:773)\n\tat java.base\u002fjava.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)\n\tat java.base\u002fjava.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)\n\tat java.base\u002fjava.lang.Thread.run(Thread.java:829)\n","exception.type": "org.openqa.selenium.WebDriverException","logger": "org.openqa.selenium.grid.node.config.DriverServiceSessionFactory","session.capabilities": "{\"moz:debuggerAddress\": true,\"acceptInsecureCerts\": true,\"browserVersion\": \"114.0\",\"se:noVncPort\": 7900,\"browserName\": \"firefox\",\"unhandledPromptBehavior\": \"ignore\",\"moz:firefoxOptions\": {\"prefs\": {\"dom.disable_beforeunload\": false}},\"se:vncEnabled\": true}\n"}}
14:40:45.068 [docker-java-stream-1970431651] INFO  o.x.t.d.i.j.b.BrowserContainerExecutor - STDOUT: 
14:40:45.070 [docker-java-stream-1970431651] INFO  o.x.t.d.i.j.b.BrowserContainerExecutor - STDOUT: 14:40:45.069 WARN [SeleniumSpanExporter$1.lambda$export$3] - {"traceId": "e6eae8750348513a7b833e65c83b2fbb","eventTime": 1686919245068368044,"eventName": "Unable to create session with the driver","attributes": {"current.session.count": 0,"logger": "org.openqa.selenium.grid.node.local.LocalNode","session.request.capabilities": "Capabilities {acceptInsecureCerts: true, browserName: firefox, moz:debuggerAddress: true, moz:firefoxOptions: {prefs: {dom.disable_beforeunload: false}}, unhandledPromptBehavior: ignore}","session.request.downstreamdialect": "[W3C]"}}
14:40:45.070 [docker-java-stream-1970431651] INFO  o.x.t.d.i.j.b.BrowserContainerExecutor - STDOUT: 
14:40:45.071 [docker-java-stream-1970431651] INFO  o.x.t.d.i.j.b.BrowserContainerExecutor - STDOUT: 14:40:45.070 WARN [SeleniumSpanExporter$1.lambda$export$3] - {"traceId": "e6eae8750348513a7b833e65c83b2fbb","eventTime": 1686919239987627416,"eventName": "Session request received by the Distributor","attributes": {"logger": "org.openqa.selenium.grid.distributor.local.LocalDistributor","request.payload": "[Capabilities {acceptInsecureCerts: true, browserName: firefox, moz:debuggerAddress: true, moz:firefoxOptions: {prefs: {dom.disable_beforeunload: false}}, unhandledPromptBehavior: ignore}]"}}
14:40:45.071 [docker-java-stream-1970431651] INFO  o.x.t.d.i.j.b.BrowserContainerExecutor - STDOUT: 
14:40:45.071 [docker-java-stream-1970431651] INFO  o.x.t.d.i.j.b.BrowserContainerExecutor - STDOUT: 14:40:45.070 WARN [SeleniumSpanExporter$1.lambda$export$1] - Unable to create session: Could not start a new session. Error while creating session with the driver service. Stopping driver service: Driver server process died prematurely.
...

### Operating System

MacOS M1

### Docker Selenium version (tag)

4.10.0

[jamesmortensen]

Hi @vmassol looks like you were able to get this figured out. What was the problem and resolution?

[vmassol]

@jamesmortensen Unfortunately no, I'm still blocked on this. I had opened a ticket on the TC issue tracker but they rightly told me that I should open an issue on the Selenium tracker which is what I've done here. I've closed the other issue since there's this one now.

I have tried many combinations of the seleniarm tags and selenium versions to no avail so far. If someone has any idea how I could debug this further, I would be much appreciated.

Thanks

[diemol]

I checked the TC docs but could not find it. How do you configure the used image?

[vmassol]

@diemol With TC, use it like this:

        BrowserWebDriverContainer<?> webDriverContainer =
            new BrowserWebDriverContainer<>("seleniarm/standalone-firefox:latest")
            // We set the width and height to one of the most used resolution by users so that we can reproduce issues
            // for the larger use case and also we use a relatively large resolution to display the maximum number of
            // elements on screen and reduce the risk of false positives in tests that could be due to elements not
            // visible or missing elements (on small screens we don't display all elements).
            .withEnv("SCREEN_WIDTH", DEFAULT_WIDTH_RESOLUTION)
            .withEnv("SCREEN_HEIGHT", DEFAULT_HEIGHT_RESOLUTION)
            // TODO: The default session timeout is 300 seconds (i.e. 5mn). We think this is what could cause the
            // "Unable to find session with ID" error message we see sometimes on the CI. We think that it's possible
            // that a session timeout of 300s means that the whole test suite of a docker test module must take less
            // than 5mn or we can get the error. Thus, as a test, we increase the value to 10 times the default value.
            // If there are still errors happening after this change then it'll mean the problem is elsewhere and we'll
            // revert this change.
            .withEnv("SE_NODE_SESSION_TIMEOUT", String.valueOf(10 * 300L))
            .withCapabilities(browser.getCapabilities())
            .withNetwork(Network.SHARED)
            .withNetworkAliases("vnchost")
            .withRecordingMode(BrowserWebDriverContainer.VncRecordingMode.SKIP, null);

Note that the source code for the TC BrowserWebDriverContainer class is at https://github.com/testcontainers/testcontainers-java/blob/main/modules/selenium/src/main/java/org/testcontainers/containers/BrowserWebDriverContainer.java

Thanks

[jamesmortensen]

@vmassol did you say this worked with the previous release of docker-seleniarm images? What makes you think it's a Firefox browser and geckodriver incompatibility? Reason I ask is that we did update to geckodriver v0.33.0.

[jamesmortensen]

@vmassol I created a standalone-firefox with selenium 4.10.0, Firefox 114 and geckodriver 0.32.2 for debugging purposes. Do you want to try this one and see if there's any differences. It will rule out or confirm your theory on geckodriver being a problem:

docker pull jamesmortensen1/standalone-firefox/4.1.0-20230627

NOTE:

• The tag is wrong. It says 4.1.0 but rest assured it is indeed Selenium 4.10.0.
• This is an arm64 image only, so it will only work on Mac M1, not Intel or Raspberry Pi.

Please let me know what happens and if previous versions of Firefox/geckodriver/selenium work for you.

[vmassol]

@jamesmortensen thanks a lot, much appreciated. I'm on holidays and will be back in 10 days but I'll try to find some time to check it out. Mentioning this so that you're aware that I may be slow to answer, but I'm on it! Thx

[jamesmortensen]

@vmassol no worries. Enjoy your holidays!