Skip to content
This repository was archived by the owner on Jun 22, 2024. It is now read-only.
This repository was archived by the owner on Jun 22, 2024. It is now read-only.

[🐛 Bug]: Session crashed #67

Open
@Turkmen1Mehmet

Description

@Turkmen1Mehmet

What happened?

Screenshot 2024-03-14 at 12 30 44

I am doing web scraping by running the Python code on the selenium grid. While it works fast and well, the session suddenly crashes and gives an error before it can pull the whole thing. What is the reason for this?
I get errors with both startup commands

Command used to start Selenium Grid with Docker (or Kubernetes)

docker run --rm -it -e VNC_PASSWORD=1234 -p 4444:4444 --shm-size 8g seleniarm/standalone-chromium:latest

docker run -it -e VNC_PASSWORD=1234 -p 4444:4444 --shm-size 8g --restart on-failure seleniarm/standalone-chromium:latest

Relevant log output

2024-03-14 14:58:28 2024-03-14 11:58:28,703 INFO Included extra file "/etc/supervisor/conf.d/selenium.conf" during parsing
2024-03-14 14:58:28 2024-03-14 11:58:28,705 INFO RPC interface 'supervisor' initialized
2024-03-14 14:58:28 2024-03-14 11:58:28,705 CRIT Server 'unix_http_server' running without any HTTP authentication checking
2024-03-14 14:58:28 2024-03-14 11:58:28,705 INFO supervisord started with pid 8
2024-03-14 14:58:29 2024-03-14 11:58:29,717 INFO spawned: 'xvfb' with pid 9
2024-03-14 14:58:29 2024-03-14 11:58:29,724 INFO spawned: 'vnc' with pid 10
2024-03-14 14:58:29 2024-03-14 11:58:29,727 INFO spawned: 'novnc' with pid 11
2024-03-14 14:58:29 2024-03-14 11:58:29,732 INFO spawned: 'selenium-standalone' with pid 12
2024-03-14 14:58:29 2024-03-14 11:58:29,740 INFO success: xvfb entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
2024-03-14 14:58:29 2024-03-14 11:58:29,740 INFO success: vnc entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
2024-03-14 14:58:29 2024-03-14 11:58:29,741 INFO success: novnc entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
2024-03-14 14:58:29 2024-03-14 11:58:29,741 INFO success: selenium-standalone entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
2024-03-14 14:58:29 Appending Selenium options: --heartbeat-period 30
2024-03-14 14:58:29 Appending Selenium options: --reject-unsupported-caps false
2024-03-14 14:58:29 Setting up SE_NODE_GRID_URL...
2024-03-14 14:58:29 Selenium Grid Standalone configuration: 
2024-03-14 14:58:29 [network]
2024-03-14 14:58:29 relax-checks = true
2024-03-14 14:58:29 
2024-03-14 14:58:29 [node]
2024-03-14 14:58:29 session-timeout = "300"
2024-03-14 14:58:29 override-max-sessions = false
2024-03-14 14:58:29 detect-drivers = false
2024-03-14 14:58:29 drain-after-session-count = 0
2024-03-14 14:58:29 max-sessions = 1
2024-03-14 14:58:29 
2024-03-14 14:58:29 [[node.driver-configuration]]
2024-03-14 14:58:29 display-name = "chrome"
2024-03-14 14:58:29 stereotype = '{"browserName": "chrome", "browserVersion": "121.0", "platformName": "Linux", "goog:chromeOptions": {"binary": "/usr/bin/chromium"}}'
2024-03-14 14:58:29 max-sessions = 1
2024-03-14 14:58:29 
2024-03-14 14:58:29 Starting Selenium Grid Standalone...
2024-03-14 14:58:29 Tracing is disabled
2024-03-14 14:58:29 11:58:29.971 INFO [LoggingOptions.configureLogEncoding] - Using the system default encoding
2024-03-14 14:58:29 11:58:29.974 INFO [OpenTelemetryTracer.createTracer] - Using OpenTelemetry for tracing
2024-03-14 14:58:30 11:58:30.251 INFO [NodeOptions.getSessionFactories] - Detected 10 available processors
2024-03-14 14:58:30 11:58:30.274 INFO [NodeOptions.report] - Adding chrome for {"browserName": "chrome","browserVersion": "121.0","goog:chromeOptions": {"binary": "\u002fusr\u002fbin\u002fchromium"},"platformName": "linux","se:noVncPort": 7900,"se:vncEnabled": true} 1 times
2024-03-14 14:58:30 11:58:30.285 INFO [Node.<init>] - Binding additional locator mechanisms: relative
2024-03-14 14:58:30 11:58:30.293 INFO [GridModel.setAvailability] - Switching Node b1f8e0c5-124d-4b86-82ce-3757af06cac6 (uri: http://172.17.0.2:4444) from DOWN to UP
2024-03-14 14:58:30 11:58:30.293 INFO [LocalDistributor.add] - Added node b1f8e0c5-124d-4b86-82ce-3757af06cac6 at http://172.17.0.2:4444. Health check every 120s
2024-03-14 14:58:30 11:58:30.352 INFO [Standalone.execute] - Started Selenium Standalone 4.18.0 (revision b6bf9de7cc): http://172.17.0.2:4444
2024-03-14 14:58:42 11:58:42.524 INFO [LocalDistributor.newSession] - Session request received by the Distributor: 
2024-03-14 14:58:42  [Capabilities {browserName: chrome, goog:chromeOptions: {args: [], extensions: []}, pageLoadStrategy: normal}]
2024-03-14 14:58:42 11:58:42.870 INFO [LocalNode.newSession] - Session created by the Node. Id: a61ac5ea98200d4482c564da3d1cb67f, Caps: Capabilities {acceptInsecureCerts: false, browserName: chrome, browserVersion: 121.0.6167.160, chrome: {chromedriverVersion: 121.0.6167.160 (8e23a63b5de..., userDataDir: /tmp/.org.chromium.Chromium...}, fedcm:accounts: true, goog:chromeOptions: {debuggerAddress: localhost:46455}, networkConnectionEnabled: false, pageLoadStrategy: normal, platformName: linux, proxy: Proxy(), se:bidiEnabled: false, se:cdp: ws://172.17.0.2:4444/sessio..., se:cdpVersion: 121.0.6167.160, se:vnc: ws://172.17.0.2:4444/sessio..., se:vncEnabled: true, se:vncLocalAddress: ws://172.17.0.2:7900, setWindowRect: true, strictFileInteractability: false, timeouts: {implicit: 0, pageLoad: 300000, script: 30000}, unhandledPromptBehavior: dismiss and notify, webauthn:extension:credBlob: true, webauthn:extension:largeBlob: true, webauthn:extension:minPinLength: true, webauthn:extension:prf: true, webauthn:virtualAuthenticators: true}
2024-03-14 14:58:42 11:58:42.873 INFO [LocalDistributor.newSession] - Session created by the Distributor. Id: a61ac5ea98200d4482c564da3d1cb67f 
2024-03-14 14:58:42  Caps: Capabilities {acceptInsecureCerts: false, browserName: chrome, browserVersion: 121.0.6167.160, chrome: {chromedriverVersion: 121.0.6167.160 (8e23a63b5de..., userDataDir: /tmp/.org.chromium.Chromium...}, fedcm:accounts: true, goog:chromeOptions: {debuggerAddress: localhost:46455}, networkConnectionEnabled: false, pageLoadStrategy: normal, platformName: linux, proxy: Proxy(), se:bidiEnabled: false, se:cdp: ws://172.17.0.2:4444/sessio..., se:cdpVersion: 121.0.6167.160, se:vnc: ws://172.17.0.2:4444/sessio..., se:vncEnabled: true, se:vncLocalAddress: ws://172.17.0.2:7900, setWindowRect: true, strictFileInteractability: false, timeouts: {implicit: 0, pageLoad: 300000, script: 30000}, unhandledPromptBehavior: dismiss and notify, webauthn:extension:credBlob: true, webauthn:extension:largeBlob: true, webauthn:extension:minPinLength: true, webauthn:extension:prf: true, webauthn:virtualAuthenticators: true}
2024-03-14 14:58:48 11:58:48.623 INFO [ProxyNodeWebsockets.createWsEndPoint] - Establishing connection to ws://172.17.0.2:7900
2024-03-14 15:06:56 12:06:56.170 WARN [SeleniumSpanExporter$1.lambda$export$3] - {"traceId": "1a03916f6f48290bcb81e28efc94e792","eventTime": 1710418016167987750,"eventName": "HTTP request execution complete","attributes": {"http.flavor": 1,"http.handler_class": "org.openqa.selenium.remote.http.Route$PredicatedRoute","http.host": "localhost:4444","http.method": "POST","http.request_content_length": "44683","http.scheme": "HTTP","http.status_code": 500,"http.target": "\u002fsession\u002fa61ac5ea98200d4482c564da3d1cb67f\u002fexecute\u002fsync","http.user_agent": "selenium\u002f4.3.0 (python mac)"}}
2024-03-14 15:06:56 
2024-03-14 15:06:56 12:06:56.241 WARN [SeleniumSpanExporter$1.lambda$export$3] - {"traceId": "13b0d6e775412d4c9984c59512f15b9a","eventTime": 1710418016240619792,"eventName": "HTTP request execution complete","attributes": {"http.flavor": 1,"http.handler_class": "org.openqa.selenium.remote.http.Route$PredicatedRoute","http.host": "localhost:4444","http.method": "POST","http.request_content_length": "80","http.scheme": "HTTP","http.status_code": 404,"http.target": "\u002fsession\u002fa61ac5ea98200d4482c564da3d1cb67f\u002furl","http.user_agent": "selenium\u002f4.3.0 (python mac)"}}
2024-03-14 15:06:56 
2024-03-14 15:06:56 12:06:56.246 WARN [SeleniumSpanExporter$1.lambda$export$3] - {"traceId": "4e7d361b3b8538f868aa428059f882ca","eventTime": 1710418016245501041,"eventName": "HTTP request execution complete","attributes": {"http.flavor": 1,"http.handler_class": "org.openqa.selenium.remote.http.Route$PredicatedRoute","http.host": "localhost:4444","http.method": "POST","http.request_content_length": "81","http.scheme": "HTTP","http.status_code": 404,"http.target": "\u002fsession\u002fa61ac5ea98200d4482c564da3d1cb67f\u002furl","http.user_agent": "selenium\u002f4.3.0 (python mac)"}}
2024-03-14 15:06:56 
2024-03-14 15:06:56 12:06:56.249 WARN [SeleniumSpanExporter$1.lambda$export$3] - {"traceId": "5b1fd3e1879d811f9e1847f9782b8ee8","eventTime": 1710418016249299708,"eventName": "HTTP request execution complete","attributes": {"http.flavor": 1,"http.handler_class": "org.openqa.selenium.remote.http.Route$PredicatedRoute","http.host": "localhost:4444","http.method": "POST","http.request_content_length": "81","http.scheme": "HTTP","http.status_code": 404,"http.target": "\u002fsession\u002fa61ac5ea98200d4482c564da3d1cb67f\u002furl","http.user_agent": "selenium\u002f4.3.0 (python mac)"}}
2024-03-14 15:06:56 
2024-03-14 15:06:56 12:06:56.253 WARN [SeleniumSpanExporter$1.lambda$export$3] - {"traceId": "81e76917cf1392335bf667ef2ea6f771","eventTime": 1710418016252397000,"eventName": "HTTP request execution complete","attributes": {"http.flavor": 1,"http.handler_class": "org.openqa.selenium.remote.http.Route$PredicatedRoute","http.host": "localhost:4444","http.method": "POST","http.request_content_length": "76","http.scheme": "HTTP","http.status_code": 404,"http.target": "\u002fsession\u002fa61ac5ea98200d4482c564da3d1cb67f\u002furl","http.user_agent": "selenium\u002f4.3.0 (python mac)"}}
2024-03-14 15:06:56 
2024-03-14 15:06:56 12:06:56.256 WARN [SeleniumSpanExporter$1.lambda$export$3] - {"traceId": "cfd11b9fc298ecd1a86af1e094e53bd2","eventTime": 1710418016256262625,"eventName": "HTTP request execution complete","attributes": {"http.flavor": 1,"http.handler_class": "org.openqa.selenium.remote.http.Route$PredicatedRoute","http.host": "localhost:4444","http.method": "POST","http.request_content_length": "84","http.scheme": "HTTP","http.status_code": 404,"http.target": "\u002fsession\u002fa61ac5ea98200d4482c564da3d1cb67f\u002furl","http.user_agent": "selenium\u002f4.3.0 (python mac)"}}
2024-03-14 15:06:56 
2024-03-14 15:06:56 12:06:56.260 WARN [SeleniumSpanExporter$1.lambda$export$3] - {"traceId": "884dfe90c5f9e38044eec739ea6988f3","eventTime": 1710418016259334500,"eventName": "HTTP request execution complete","attributes": {"http.flavor": 1,"http.handler_class": "org.openqa.selenium.remote.http.Route$PredicatedRoute","http.host": "localhost:4444","http.method": "POST","http.request_content_length": "69","http.scheme": "HTTP","http.status_code": 404,"http.target": "\u002fsession\u002fa61ac5ea98200d4482c564da3d1cb67f\u002furl","http.user_agent": "selenium\u002f4.3.0 (python mac)"}}
2024-03-14 15:06:56 
2024-03-14 15:06:56 12:06:56.263 WARN [SeleniumSpanExporter$1.lambda$export$3] - {"traceId": "91407eb157574f17dc8a06277870e80a","eventTime": 1710418016262375833,"eventName": "HTTP request execution complete","attributes": {"http.flavor": 1,"http.handler_class": "org.openqa.selenium.remote.http.Route$PredicatedRoute","http.host": "localhost:4444","http.method": "POST","http.request_content_length": "77","http.scheme": "HTTP","http.status_code": 404,"http.target": "\u002fsession\u002fa61ac5ea98200d4482c564da3d1cb67f\u002furl","http.user_agent": "selenium\u002f4.3.0 (python mac)"}}
2024-03-14 15:06:56 
2024-03-14 15:06:56 12:06:56.266 WARN [SeleniumSpanExporter$1.lambda$export$3] - {"traceId": "5ece989b3f0acb76f181db11201daf80","eventTime": 1710418016265305208,"eventName": "HTTP request execution complete","attributes": {"http.flavor": 1,"http.handler_class": "org.openqa.selenium.remote.http.Route$PredicatedRoute","http.host": "localhost:4444","http.method": "POST","http.request_content_length": "101","http.scheme": "HTTP","http.status_code": 404,"http.target": "\u002fsession\u002fa61ac5ea98200d4482c564da3d1cb67f\u002furl","http.user_agent": "selenium\u002f4.3.0 (python mac)"}}
2024-03-14 15:06:56 
2024-03-14 15:06:56 12:06:56.270 WARN [SeleniumSpanExporter$1.lambda$export$3] - {"traceId": "fac775e219b7d599c730e4aef2abd391","eventTime": 1710418016269290667,"eventName": "HTTP request execution complete","attributes": {"http.flavor": 1,"http.handler_class": "org.openqa.selenium.remote.http.Route$PredicatedRoute","http.host": "localhost:4444","http.method": "POST","http.request_content_length": "74","http.scheme": "HTTP","http.status_code": 404,"http.target": "\u002fsession\u002fa61ac5ea98200d4482c564da3d1cb67f\u002furl","http.user_agent": "selenium\u002f4.3.0 (python mac)"}}
2024-03-14 15:06:56 
2024-03-14 15:06:56 12:06:56.274 WARN [SeleniumSpanExporter$1.lambda$export$3] - {"traceId": "0077bb0753ad02ea99662beb0fe38c7c","eventTime": 1710418016273329667,"eventName": "HTTP request execution complete","attributes": {"http.flavor": 1,"http.handler_class": "org.openqa.selenium.remote.http.Route$PredicatedRoute","http.host": "localhost:4444","http.method": "POST","http.request_content_length": "80","http.scheme": "HTTP","http.status_code": 404,"http.target": "\u002fsession\u002fa61ac5ea98200d4482c564da3d1cb67f\u002furl","http.user_agent": "selenium\u002f4.3.0 (python mac)"}}
2024-03-14 15:06:56 
2024-03-14 15:08:52 ^CTrapped SIGTERM/SIGINT/x so shutting down supervisord...
2024-03-14 15:08:52 2024-03-14 12:08:52,495 WARN received SIGINT indicating exit request
2024-03-14 15:08:52 2024-03-14 12:08:52,496 INFO waiting for xvfb, vnc, novnc, selenium-standalone to die
2024-03-14 15:08:52 12:08:52.518 INFO [LocalNode.stopAllSessions] - Trying to stop all running sessions before shutting down...
2024-03-14 15:08:52 2024-03-14 12:08:52,521 WARN stopped: selenium-standalone (terminated by SIGTERM)
2024-03-14 15:08:52 2024-03-14 12:08:52,521 WARN received SIGTERM indicating exit request
2024-03-14 15:08:52 2024-03-14 12:08:52,957 WARN stopped: novnc (terminated by SIGTERM)
2024-03-14 15:08:53 2024-03-14 12:08:53,964 WARN stopped: vnc (terminated by SIGTERM)
2024-03-14 15:08:54 2024-03-14 12:08:54,972 WARN stopped: xvfb (terminated by SIGTERM)
2024-03-14 15:08:55 Shutdown complete
2024-03-14 15:08:55 2024-03-14 12:08:55,301 INFO Included extra file "/etc/supervisor/conf.d/selenium.conf" during parsing
2024-03-14 15:08:55 2024-03-14 12:08:55,303 INFO RPC interface 'supervisor' initialized
2024-03-14 15:08:55 2024-03-14 12:08:55,303 CRIT Server 'unix_http_server' running without any HTTP authentication checking
2024-03-14 15:08:55 2024-03-14 12:08:55,303 INFO supervisord started with pid 7
2024-03-14 15:08:56 2024-03-14 12:08:56,315 INFO spawned: 'xvfb' with pid 8
2024-03-14 15:08:56 2024-03-14 12:08:56,319 INFO spawned: 'vnc' with pid 9
2024-03-14 15:08:56 2024-03-14 12:08:56,322 INFO spawned: 'novnc' with pid 10
2024-03-14 15:08:56 2024-03-14 12:08:56,325 INFO spawned: 'selenium-standalone' with pid 16
2024-03-14 15:08:56 2024-03-14 12:08:56,329 INFO success: xvfb entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
2024-03-14 15:08:56 2024-03-14 12:08:56,329 INFO success: vnc entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
2024-03-14 15:08:56 2024-03-14 12:08:56,329 INFO success: novnc entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
2024-03-14 15:08:56 2024-03-14 12:08:56,329 INFO success: selenium-standalone entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
2024-03-14 15:08:56 Appending Selenium options: --heartbeat-period 30
2024-03-14 15:08:56 Appending Selenium options: --reject-unsupported-caps false
2024-03-14 15:08:56 Setting up SE_NODE_GRID_URL...
2024-03-14 15:08:56 Selenium Grid Standalone configuration: 
2024-03-14 15:08:56 [network]
2024-03-14 15:08:56 relax-checks = true
2024-03-14 15:08:56 
2024-03-14 15:08:56 [node]
2024-03-14 15:08:56 session-timeout = "300"
2024-03-14 15:08:56 override-max-sessions = false
2024-03-14 15:08:56 detect-drivers = false
2024-03-14 15:08:56 drain-after-session-count = 0
2024-03-14 15:08:56 max-sessions = 1
2024-03-14 15:08:56 
2024-03-14 15:08:56 [[node.driver-configuration]]
2024-03-14 15:08:56 display-name = "chrome"
2024-03-14 15:08:56 stereotype = '{"browserName": "chrome", "browserVersion": "121.0", "platformName": "Linux", "goog:chromeOptions": {"binary": "/usr/bin/chromium"}}'
2024-03-14 15:08:56 max-sessions = 1
2024-03-14 15:08:56 
2024-03-14 15:08:56 Starting Selenium Grid Standalone...
2024-03-14 15:08:56 Tracing is disabled
2024-03-14 15:08:56 12:08:56.549 INFO [LoggingOptions.configureLogEncoding] - Using the system default encoding
2024-03-14 15:08:56 12:08:56.553 INFO [OpenTelemetryTracer.createTracer] - Using OpenTelemetry for tracing
2024-03-14 15:08:56 12:08:56.827 INFO [NodeOptions.getSessionFactories] - Detected 10 available processors
2024-03-14 15:08:56 12:08:56.849 INFO [NodeOptions.report] - Adding chrome for {"browserName": "chrome","browserVersion": "121.0","goog:chromeOptions": {"binary": "\u002fusr\u002fbin\u002fchromium"},"platformName": "linux","se:noVncPort": 7900,"se:vncEnabled": true} 1 times
2024-03-14 15:08:56 12:08:56.859 INFO [Node.<init>] - Binding additional locator mechanisms: relative
2024-03-14 15:08:56 12:08:56.868 INFO [GridModel.setAvailability] - Switching Node a706d0a8-208d-47fd-a90b-bb71a174690e (uri: http://172.17.0.2:4444) from DOWN to UP
2024-03-14 15:08:56 12:08:56.868 INFO [LocalDistributor.add] - Added node a706d0a8-208d-47fd-a90b-bb71a174690e at http://172.17.0.2:4444. Health check every 120s
2024-03-14 15:08:56 12:08:56.924 INFO [Standalone.execute] - Started Selenium Standalone 4.18.0 (revision b6bf9de7cc): http://172.17.0.2:4444

Operating System

macos M1 Pro

Docker Selenium version (image tag)

seleniarm/standalone-chromium:latest

Selenium Grid chart version (chart version)

4.18.0

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions