docs: comprehensive E2ETestPlan.md update with test catalog and architecture#591
Conversation
…ure diagram, and execution procedures Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
github-actions
Bot
added
the
documentation
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Scanned FilesNone |
There was a problem hiding this comment.
Pull Request Overview
This PR transforms the E2ETestPlan.md from a generic template (460 lines) into a comprehensive, ISMS-compliant testing documentation (1,261 lines) that accurately reflects the actual Cypress test implementation for all 15 test specifications covering 12 production widgets.
Key Changes:
- Added complete test catalog documenting all 15 actual test specifications organized by category (Assessment Center, Business Value, Impact Analysis, Implementation Guide, Integration)
- Included Mermaid architecture diagram visualizing the complete testing infrastructure from Cypress → Browsers → Application → Widgets → Reports
- Documented actual test implementation patterns from the codebase including widget-testing-template, custom commands, and test constants
| cy.log(`Testing compliance scenario ${index + 1}: ${scenario.name}`); | ||
|
|
||
| // Set security levels using custom command | ||
| cy.setSecurityLevels(...scenario.levels); |
There was a problem hiding this comment.
The parameter order in the cy.setSecurityLevels() call doesn't match the documented function signature. In the custom commands implementation (cypress/support/commands.ts), the signature is:
cy.setSecurityLevels(availability: string, integrity: string, confidentiality: string)However, the scenario arrays use [confidentiality, integrity, availability] order, which would result in incorrect security level assignments. The spread operator ...scenario.levels would pass these values in the wrong order.
| cy.setSecurityLevels(...scenario.levels); | |
| const [confidentiality, integrity, availability] = scenario.levels; | |
| cy.setSecurityLevels(availability, integrity, confidentiality); |
There was a problem hiding this comment.
Fixed in commit 552b534. Updated to destructure the array and pass parameters in the correct order: const [availability, integrity, confidentiality] = scenario.levels; cy.setSecurityLevels(availability, integrity, confidentiality)
| ensureAppLoaded(): Chainable<void> | ||
| setSecurityLevels( | ||
| confidentiality: string, | ||
| integrity: string, | ||
| availability: string | ||
| ): Chainable<void> |
There was a problem hiding this comment.
Inconsistent parameter order documentation. The custom command signature is documented as setSecurityLevels(confidentiality, integrity, availability) but the actual implementation in cypress/support/commands.ts uses the order (availability, integrity, confidentiality). This inconsistency could confuse developers implementing tests.
There was a problem hiding this comment.
Fixed in commit 552b534. Corrected the signature to match the actual implementation: setSecurityLevels(availability?: string, integrity?: string, confidentiality?: string) with parameters marked as optional.
| // Wait for application to fully load | ||
| Cypress.Commands.add('ensureAppLoaded', () => { | ||
| cy.get('[data-testid*="widget"]', { timeout: 10000 }) | ||
| .should('have.length.greaterThan', 0) | ||
| cy.wait(500) // Allow for widget initialization | ||
| }) |
There was a problem hiding this comment.
The documented custom command implementation differs significantly from the actual implementation. The actual ensureAppLoaded command in cypress/support/commands.ts has a more complex implementation that accepts an optional timeoutValue parameter and uses different logic than shown here. Consider either:
- Simplifying this to match the conceptual usage, or
- Providing the actual implementation with its full complexity
There was a problem hiding this comment.
Fixed in commit 552b534. Updated to match the actual implementation with the optional timeoutValue parameter and the more complex logic that checks for dashboard/widgets and validates select elements.
| integrity: string, | ||
| availability: string | ||
| ): Chainable<void> | ||
| findWidgetFlexibly(widgetId: string): Chainable<JQuery<HTMLElement>> |
There was a problem hiding this comment.
The findWidgetFlexibly custom command is documented here but is actually implemented as an exported function in cypress/support/widget-testing-template.ts, not as a Cypress custom command. The actual custom commands don't include this function. This could cause confusion when developers try to use cy.findWidgetFlexibly() which won't work as documented.
| findWidgetFlexibly(widgetId: string): Chainable<JQuery<HTMLElement>> |
There was a problem hiding this comment.
Fixed in commit 552b534. Moved findWidgetFlexibly from the Cypress commands section to a new "Utility Functions" section, correctly documenting it as an exported function from widget-testing-template.ts rather than a custom command.
| VERY_HIGH: "Very High", | ||
| } as const | ||
|
|
||
| export const WIDGET_IDS = { |
There was a problem hiding this comment.
The constant name WIDGET_IDS is incorrect. The actual export from cypress/support/constants.ts is WIDGET_TEST_IDS, not WIDGET_IDS. This will cause import errors if developers try to use this code.
| export const WIDGET_IDS = { | |
| export const WIDGET_TEST_IDS = { |
There was a problem hiding this comment.
Fixed in commit 552b534. Renamed to WIDGET_TEST_IDS to match the actual export from cypress/support/constants.ts. Also updated all widget ID keys to include the _WIDGET suffix (e.g., SECURITY_LEVEL_WIDGET).
| createWidgetTests({ | ||
| widgetId: "security-level", | ||
| contentPatterns, | ||
| additionalTests, | ||
| }); |
There was a problem hiding this comment.
The createWidgetTests function is being called with an object parameter in the documentation, but the actual implementation uses positional parameters. According to the implementation in cypress/support/widget-testing-template.ts, the function signature is:
createWidgetTests(widgetName: string, widgetTestId: string, contentPatterns: (string | RegExp)[], additionalTests?: () => void)The correct usage should be:
createWidgetTests(
"Security Level",
"security-level",
contentPatterns,
additionalTests
);| createWidgetTests({ | |
| widgetId: "security-level", | |
| contentPatterns, | |
| additionalTests, | |
| }); | |
| createWidgetTests( | |
| "Security Level", | |
| "security-level", | |
| contentPatterns, | |
| additionalTests | |
| ); |
There was a problem hiding this comment.
Fixed in commit 552b534. Changed from object parameter to positional parameters: createWidgetTests("Security Level", "security-level", contentPatterns, additionalTests) to match the actual function signature.
|
@copilot apply changes based on the comments in this thread |
- Fix createWidgetTests to use positional parameters instead of object - Correct setSecurityLevels parameter order (availability, integrity, confidentiality) - Update ensureAppLoaded to include optional timeoutValue parameter - Move findWidgetFlexibly from Cypress command to exported utility function - Rename WIDGET_IDS to WIDGET_TEST_IDS to match actual constant name - Add clarifying comments about parameter order in test scenarios Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
Pull Request Description
Enhanced
E2ETestPlan.mdfrom generic template (460 lines) to comprehensive ISMS-compliant documentation (1,261 lines) covering all 15 actual Cypress test specifications.Key additions:
Test catalog structure:
Code accuracy improvements:
cypress/support/commands.ts,cypress/support/widget-testing-template.ts, andcypress/support/constants.tscreateWidgetTestsfunction signature from object parameter to positional parameterssetSecurityLevelsparameter order to(availability, integrity, confidentiality)ensureAppLoadedto include optionaltimeoutValueparameterfindWidgetFlexiblyfrom Cypress commands to utility functions section (exported function, not a command)WIDGET_IDStoWIDGET_TEST_IDSwith proper widget key namesDocument now meets ISMS Section 4.2 requirements with Version 2.0 document control footer.
Type of Change
Component(s) Modified
CIA Impact Area
Security Level Impact
Test Coverage Impact
Testing Performed
Screenshots/Examples
Document structure verification:
Test coverage summary table:
Code accuracy fixes:
createWidgetTests()- Changed from object to positional parameterscy.setSecurityLevels()- Fixed parameter order and added clarifying commentscy.ensureAppLoaded()- Added optional timeout parameterfindWidgetFlexibly()- Moved to utility functions (not a Cypress command)WIDGET_TEST_IDS- Corrected constant name and structureRelated Issues
Closes #214
Checklist
Additional Notes
ISMS Compliance: Fully implements Section 4.2 - End-to-End Testing Strategy requirements:
Compliance standards: ISO 27001 (A.8.31), NIST CSF (PR.IP-1), CIS Controls (16.10)
Code Review: All code examples validated against actual implementation files to ensure developers can use the documented patterns without errors.
Original prompt
This section details on the original issue you should resolve
<issue_title>🧪 Create E2ETestPlan.md Documentation for End-to-End Testing Strategy</issue_title>
<issue_description>## 🎯 Objective
Create comprehensive E2ETestPlan.md documentation to align with Hack23 ISMS Secure Development Policy requirements for end-to-end testing strategy and evidence.
📋 Background
Per Secure_Development_Policy.md §4.2 "End-to-End Testing Strategy", all Hack23 projects MUST maintain E2ETestPlan.md documentation with comprehensive test coverage, public results, and browser testing validation.
Current State:
E2ETestPlan.md(mandatory per ISMS policy §4.2)cypress/directory.github/workflows/test-and-report.ymldocs/cypress/mochawesome/Reference Implementations:
Why This Matters:
✅ Acceptance Criteria
docs/E2ETestPlan.mdfollowing ISMS style guide🛠️ Implementation Guidance
Files to Create/Modify
docs/E2ETestPlan.md- New comprehensive E2E test plan documentationREADME.md- Add E2E test badge linking to plan (optional enhancement)Approach