Skip to content

Conversation

@markbrockhoff
Copy link
Collaborator

@markbrockhoff markbrockhoff commented Dec 8, 2025

TaskWPB-21987 [WEB/QA] POC - Use sharding to run e2e tests in parallel in CI

Pull Request

Summary

  • Instead of building the kalium testservice in every CI run, use the now available prebuilt image of it

Security Checklist (required)

  • External inputs are validated & sanitized on client and/or server where applicable.
  • API responses are validated; unexpected shapes are handled safely (fallbacks or errors).
  • No unsafe HTML is rendered; if unavoidable, sanitization is applied and documented where it happens.
  • Injection risks (XSS/SQL/command) are prevented via safe APIs and/or escaping.

Standards Acknowledgement (required)


Notes for reviewers

  • I checked the test case @TC-3009 to verify the testservice works as it requires it. The test run where it was successfully executed can be seen here

@github-actions
Copy link
Contributor

github-actions bot commented Dec 8, 2025

🔗 Download Full Report Artifact

🧪 Playwright Test Summary

  • Passed: 53
  • Failed: 47
  • Skipped: 5
  • 🔁 Flaky: 4
  • 📊 Total: 109
  • Total Runtime: 475.1s (~ 7 min 55 sec)
specs/2FA-for-TeamsSpecs/2fa-for-teams.spec.ts (❌ 2 failed, ⚠️ 0 flaky)
  • ❌ f2a for teams > 2FA Code (tags: TC-8749, regression)
  • ❌ f2a for teams > I want to verify that verification code is not required after login if 2FA has been disabled (tags: TC-8749, regression)
specs/Accessibility/Accessibility.spec.ts (❌ 3 failed, ⚠️ 0 flaky)
  • ❌ Accessibility > I want to see typing indicator in group conversation (tags: TC-46, regression)
  • ❌ Accessibility > I want to see collapsed view when app is narrow (tags: TC-48, regression)
  • ❌ Accessibility > I should not lose a drafted message when switching between conversations in collapsed view (tags: TC-51, regression)
specs/AccountSettingsSpecs/accountSettings.spec.ts (❌ 7 failed, ⚠️ 0 flaky)
  • ❌ account settings > Edit Profile (tags: TC-8770, regression)
  • ❌ account settings > I should not be able to change my email to already taken email (tags: TC-58, regression)
  • ❌ account settings > I should not be able to change my email to an invalid email address (tags: TC-59, regression)
  • ❌ account settings > I should not be able to change email of user managed by SCIM (tags: TC-60, regression)
  • ❌ account settings > Verify links to manage and create teams are shown when logged in as team owner (tags: TC-1723, regression)
  • ❌ account settings > Verify I can retrieve calling logs (tags: TC-1725, regression)
  • ❌ account settings > I want to see the Full Name wherever my name gets displayed (tags: TC-1948, regression)
specs/AppLock/AppLock.spec.ts (❌ 3 failed, ⚠️ 0 flaky)
  • ❌ AppLock > Web: App should not lock if I switch back to webapp tab in time (during inactivity timeout) (tags: TC-2752, TC-2753, regression)
  • ❌ AppLock > I should not be able to switch off app lock if it is enforced for the team (tags: TC-2770, TC-2767, regression)
  • ❌ AppLock > I want to switch off app lock (tags: TC-2771, TC-2772, regression)
specs/ArchiveSpecs/archive.spec.ts (❌ 4 failed, ⚠️ 0 flaky)
  • ❌ Accessibility > I want to archive and unarchive conversation via conversation list (tags: TC-97, regression)
  • ❌ Accessibility > Verify the conversation is not unarchived when there are new messages in this conversation (tags: TC-99, regression)
  • ❌ Accessibility > I want to archive the group conversation from conversation details (tags: TC-104, regression)
  • ❌ Accessibility > I want to archive the 1on1 conversation from conversation details (tags: TC-105, regression)
specs/Authentication/authentication.spec.ts (❌ 1 failed, ⚠️ 0 flaky)
  • ❌ Authentication > I want to be asked to share telemetry data when I log in (tags: TC-8780, regression)
specs/Block/block.spec.ts (❌ 6 failed, ⚠️ 0 flaky)
  • ❌ Block: User A and User B are NOT in the same team > I want to cancel blocking a 1:1 conversation from conversation list (tags: TC-137, regression)
  • ❌ Block: User A and User B are NOT in the same team > Verify you can block a user who is not in your team (tags: TC-140, regression)
  • ❌ Block: User A and User B are NOT in the same team > Verify you still receive messages from blocked person in a group chat (tags: TC-141, regression)
  • ❌ Block: User A and User B are NOT in the same team > Verify you can unblock user from 1:1 conversation details (tags: TC-142, regression)
  • ❌ Block: User A and User B are NOT in the same team > Verify you cannot add a person who blocked you to a group chat (tags: TC-143, regression)
  • ❌ Block: User A and User B are NOT in the same team > Verify you can unblock someone from conversation list options (tags: TC-148, regression)
specs/Connections/Connections.spec.ts (❌ 1 failed, ⚠️ 0 flaky)
  • ❌ Connections > Verify 1on1 conversation is not created on the second end after you ignore connection request (tags: TC-365, TC-369, TC-370, TC-371, regression)
specs/CriticalFlow/accountManagement-TC-8639.spec.ts (❌ 1 failed, ⚠️ 0 flaky)
  • ❌ Account Management (tags: TC-8639, crit-flow-web)
specs/CriticalFlow/addMembersToChat-TC-8631.spec.ts (❌ 1 failed, ⚠️ 0 flaky)
  • ❌ Team owner adds whole team to an all team chat (tags: TC-8631, crit-flow-web)
specs/CriticalFlow/backupRestoration-TC-8634.spec.ts (❌ 1 failed, ⚠️ 0 flaky)
  • ❌ Setting up new device with a backup (tags: TC-8634, crit-flow-web)
specs/CriticalFlow/Cells/uploadingFileInGroupConversation.spec.ts (❌ 1 failed, ⚠️ 0 flaky)
  • ❌ Uploading an file in a group conversation (tags: crit-flow-cells, regression)
specs/CriticalFlow/channelsCall-TC-8755.spec.ts (❌ 1 failed, ⚠️ 0 flaky)
  • ❌ Calls in channels with device switch and screenshare (tags: TC-8754, crit-flow-web)
specs/CriticalFlow/channelsManagement-TC-8752.spec.ts (❌ 1 failed, ⚠️ 0 flaky)
  • ❌ Channels Management (tags: TC-8752, crit-flow-web)
specs/CriticalFlow/conversationManagement-TC-8636.spec.ts (❌ 1 failed, ⚠️ 0 flaky)
  • ❌ Conversation Management (tags: TC-8636, crit-flow-web)
specs/CriticalFlow/groupCalls-TC-8632.spec.ts (❌ 1 failed, ⚠️ 0 flaky)
  • ❌ Planning group call with sending various messages during call (tags: TC-8632, crit-flow-web)
specs/CriticalFlow/groupVideoCall-TC-8637.spec.ts (❌ 1 failed, ⚠️ 0 flaky)
  • ❌ Group Video call (tags: TC-8637, crit-flow-web)
specs/CriticalFlow/joinTeam-TC-8635.spec.ts (❌ 1 failed, ⚠️ 0 flaky)
  • ❌ New person joins team and setups up device (tags: TC-8635, crit-flow-web)
specs/CriticalFlow/messagesIn1On1-TC-8750.spec.ts (❌ 1 failed, ⚠️ 0 flaky)
  • ❌ Messages in 1:1 (tags: TC-8750, crit-flow-web)
specs/CriticalFlow/messagesInChannels-TC-8753.spec.ts (❌ 1 failed, ⚠️ 0 flaky)
  • ❌ Messages in Channels (tags: TC-8753, crit-flow-web)
specs/CriticalFlow/messagesInGroups-TC-8751.spec.ts (❌ 1 failed, ⚠️ 0 flaky)
  • ❌ Messages in Groups (tags: TC-8751, crit-flow-web)
specs/CriticalFlow/oneOnOneCall-TC-8754.spec.ts (❌ 1 failed, ⚠️ 0 flaky)
  • ❌ 1:1 Video call with device switch and screenshare (tags: TC-8754, crit-flow-web)
specs/CriticalFlow/personalAccountLifecycle-TC-8638.spec.ts (❌ 1 failed, ⚠️ 0 flaky)
  • ❌ Personal Account Lifecycle (tags: TC-8638, crit-flow-web)
specs/Edit/edit.spec.ts (❌ 1 failed, ⚠️ 2 flaky)
  • ❌ Edit > I cannot edit another users message (tags: TC-683, regression)
  • ⚠️ Edit > Editing a message does not create unread dot on receiver side (tags: TC-690, regression)
  • ⚠️ Edit > I can see the changed message was edited from another user (tags: TC-692, regression)
specs/LoginSpecs/login.spec.ts (❌ 1 failed, ⚠️ 0 flaky)
  • ❌ Verify you can sign in by email (tags: TC-3461, regression)
specs/RegressionSpecs/archive.spec.ts (❌ 1 failed, ⚠️ 0 flaky)
  • ❌ Archive status of conversation should only change by user explicitly changing it (tags: TC-97, TC-99, TC-104, TC-105, regression)
specs/RegressionSpecs/block-messages.spec.ts (❌ 1 failed, ⚠️ 0 flaky)
  • ❌ Block specs (tags: TC-141, regression)
specs/RegressionSpecs/block-search.spec.ts (❌ 1 failed, ⚠️ 0 flaky)
  • ❌ Block from search specs (tags: TC-144, regression)
specs/SelfDeletingMessages/selfDeletingMessages.spec.ts (❌ 0 failed, ⚠️ 2 flaky)
  • ⚠️ set globally in group conversation > I want to see timed message disable in an input bar when global settings conversation options are set (tags: TC-3718, regression)
  • ⚠️ in search results > I want to see ephemeral messages in the search results (tags: TC-3717, regression)

@markbrockhoff markbrockhoff force-pushed the ci/use-prebuild-testservice branch from 87ae928 to 3530484 Compare December 10, 2025 16:35
@markbrockhoff markbrockhoff force-pushed the ci/use-prebuild-testservice branch from 3530484 to c35ad19 Compare December 10, 2025 16:36
@markbrockhoff markbrockhoff marked this pull request as ready for review December 10, 2025 16:37
@markbrockhoff markbrockhoff requested review from a team and otto-the-bot as code owners December 10, 2025 16:37
@sonarqubecloud
Copy link

@codecov
Copy link

codecov bot commented Dec 10, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 43.99%. Comparing base (6c7afb2) to head (c35ad19).
⚠️ Report is 1 commits behind head on dev.

Additional details and impacted files
@@           Coverage Diff           @@
##              dev   #19867   +/-   ##
=======================================
  Coverage   43.99%   43.99%           
=======================================
  Files        1309     1309           
  Lines       32928    32928           
  Branches     7298     7298           
=======================================
  Hits        14487    14487           
  Misses      16715    16715           
  Partials     1726     1726           
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@markbrockhoff markbrockhoff enabled auto-merge (squash) December 10, 2025 17:03
@markbrockhoff markbrockhoff merged commit b593d17 into dev Dec 10, 2025
31 of 50 checks passed
@markbrockhoff markbrockhoff deleted the ci/use-prebuild-testservice branch December 10, 2025 17:06
zskhan pushed a commit that referenced this pull request Dec 12, 2025
* ci(WPB-21987): use prebuilt testservice in CI
* docs: document usage of testservice for running tests locally without VPN
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants