ci(WPB-21987): group test in playwright comment by file

[markbrockhoff]

WPB-21987 [WEB/QA] POC - Use sharding to run e2e tests in parallel in CI

Pull Request

Summary

• Group tests in playwright report comment on PRs by file
• Migrate report script to TS (I upgraded the projects node version from 22.16 to 22.18 so I can use node to run the ts file)

---

Security Checklist (required)

• External inputs are validated & sanitized on client and/or server where applicable.
• API responses are validated; unexpected shapes are handled safely (fallbacks or errors).
• No unsafe HTML is rendered; if unavoidable, sanitization is applied and documented where it happens.
• Injection risks (XSS/SQL/command) are prevented via safe APIs and/or escaping.

Standards Acknowledgement (required)

• I have read and this PR upholds our Coding Standards and Tech Radar Choices.

---

Screenshots or demo (if the user interface changed)

Notes for reviewers

The new comment is a bit less detailed than the previous one, skipping the error message. I noticed that the plain playwright error message usually isn't helpful and in almost all cases the trace needs to be downloaded for further investigation. So I decided to keep the script as simple as possible.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 43.71%. Comparing base (4577eed) to head (295fc73).
⚠️ Report is 1 commits behind head on dev.

Additional details and impacted files

@@           Coverage Diff           @@
##              dev   #19830   +/-   ##
=======================================
  Coverage   43.71%   43.71%           
=======================================
  Files        1298     1298           
  Lines       32707    32707           
  Branches     7259     7259           
=======================================
  Hits        14297    14297           
  Misses      16693    16693           
  Partials     1717     1717           

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[github-actions]

🔗 Download Full Report Artifact

🧪 Playwright Test Summary

• ✅ Passed: 83
• ❌ Failed: 17
• ⏭ Skipped: 5
• 🔁 Flaky: 3
• 📊 Total: 108
• ⏱ Total Runtime: 480.3s (~ 8 min 0 sec)

specs/Accessibility/Accessibility.spec.ts (❌ 2 failed, ⚠️ 0 flaky)

• ❌ Accessibility > I want to see collapsed view when app is narrow (tags: TC-48, regression)
• ❌ Accessibility > I should not lose a drafted message when switching between conversations in collapsed view (tags: TC-51, regression)

specs/AccountSettingsSpecs/accountSettings.spec.ts (❌ 2 failed, ⚠️ 1 flaky)

• ❌ account settings > I should not be able to change email of user managed by SCIM (tags: TC-60, regression)
• ❌ account settings > Verify links to manage and create teams are shown when logged in as team owner (tags: TC-1723, regression)
• ⚠️ account settings > Verify link to manage a team is not shown when logged in as team member or normal use (tags: TC-1723, regression)

specs/AppLock/AppLock.spec.ts (❌ 1 failed, ⚠️ 0 flaky)

• ❌ AppLock > Web: App should not lock if I switch back to webapp tab in time (during inactivity timeout) (tags: TC-2752, TC-2753, regression)

specs/Authentication/authentication.spec.ts (❌ 2 failed, ⚠️ 0 flaky)

• ❌ Authentication > I want to be asked to share telemetry data when I log in (tags: TC-8780, regression)
• ❌ Authentication > Make sure user does not see data of user of previous sessions on same browser (tags: TC-1311, regression)

specs/Connections/Connections.spec.ts (❌ 1 failed, ⚠️ 0 flaky)

• ❌ Connections > Verify 1on1 conversation is not created on the second end after you ignore connection request (tags: TC-365, TC-369, TC-370, TC-371, regression)

specs/CriticalFlow/accountManagement-TC-8639.spec.ts (❌ 1 failed, ⚠️ 0 flaky)

• ❌ Account Management (tags: TC-8639, crit-flow-web)

specs/CriticalFlow/addMembersToChat-TC-8631.spec.ts (❌ 1 failed, ⚠️ 0 flaky)

• ❌ Team owner adds whole team to an all team chat (tags: TC-8631, crit-flow-web)

specs/CriticalFlow/channelsManagement-TC-8752.spec.ts (❌ 1 failed, ⚠️ 0 flaky)

• ❌ Channels Management (tags: TC-8752, crit-flow-web)

specs/CriticalFlow/conversationManagement-TC-8636.spec.ts (❌ 0 failed, ⚠️ 1 flaky)

• ⚠️ Conversation Management (tags: TC-8636, crit-flow-web)

specs/CriticalFlow/groupCalls-TC-8632.spec.ts (❌ 1 failed, ⚠️ 0 flaky)

• ❌ Planning group call with sending various messages during call (tags: TC-8632, crit-flow-web)

specs/CriticalFlow/groupVideoCall-TC-8637.spec.ts (❌ 1 failed, ⚠️ 0 flaky)

• ❌ Group Video call (tags: TC-8637, crit-flow-web)

specs/CriticalFlow/joinTeam-TC-8635.spec.ts (❌ 1 failed, ⚠️ 0 flaky)

• ❌ New person joins team and setups up device (tags: TC-8635, crit-flow-web)

specs/CriticalFlow/messagesIn1On1-TC-8750.spec.ts (❌ 1 failed, ⚠️ 0 flaky)

• ❌ Messages in 1:1 (tags: TC-8750, crit-flow-web)

specs/Edit/edit.spec.ts (❌ 0 failed, ⚠️ 1 flaky)

• ⚠️ Edit > Editing a message does not create unread dot on receiver side (tags: TC-690, regression)

specs/Reply/reply.spec.ts (❌ 2 failed, ⚠️ 0 flaky)

• ❌ Reply > I want to see a placeholder text as quote when original message is not available anymore (tags: TC-2994, regression)
• ❌ Reply > I should not be able to send a reply after I got removed from the conversation (tags: TC-3014, regression)

[sonarqubecloud]

Quality Gate passed

Issues
4 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud