Skip to content

Prevent CNG CBC size-query stack overflow for payloads > 2 MB#65521

Closed
Copilot wants to merge 4 commits intomainfrom
copilot/fix-cngcbc-authenticated-encryptor
Closed

Prevent CNG CBC size-query stack overflow for payloads > 2 MB#65521
Copilot wants to merge 4 commits intomainfrom
copilot/fix-cngcbc-authenticated-encryptor

Conversation

Copy link
Contributor

Copilot AI commented Feb 24, 2026

Prevent CNG CBC size-query stack overflow for payloads > 2 MB

  • You've read the Contributor Guide and Code of Conduct.
  • You've included unit or integration tests for your change, where applicable.
  • You've included inline docs for your change, where applicable.
  • There's an open issue for the PR that you are making. If you'd like to propose a new feature or change, please open an issue to discuss the change or find an existing issue.

Summary of the changes (Less than 80 chars)

Avoid large stackalloc in CNG CBC encrypted-size query

Description

Encrypting buffers above ~2 MB regressed into stack overflow in the CNG CBC path.
The size-query helper allocated stackalloc byte[cbInput], so large plaintext length directly consumed thread stack.

  • Root-cause correction

    • In CbcAuthenticatedEncryptor.GetCbcEncryptedOutputSizeWithPadding(uint cbInput), replaced input-sized stack allocation with a single-byte dummy pointer for BCrypt size queries.
    • Kept cbInput length unchanged for output-size computation; only removed stack growth proportional to payload size.
  • Regression coverage

    • Added Encrypt_OverTwoMB_RoundTrips (Windows-conditional) to validate encrypt/decrypt roundtrip for (2 * 1024 * 1024) + 1 byte payloads.
byte dummyByte = 0;
byte* pbDummyInput = &dummyByte; // size query uses length, not input contents

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Copilot AI and others added 3 commits February 24, 2026 07:40
Co-authored-by: halter73 <54385+halter73@users.noreply.github.com>
Co-authored-by: halter73 <54385+halter73@users.noreply.github.com>
Co-authored-by: halter73 <54385+halter73@users.noreply.github.com>
@halter73 halter73 closed this Feb 24, 2026
@halter73 halter73 deleted the copilot/fix-cngcbc-authenticated-encryptor branch February 24, 2026 07:43
@dotnet-policy-service dotnet-policy-service bot added this to the 11.0-preview2 milestone Feb 24, 2026
Copilot AI changed the title [WIP] Fix overflow issue with large array encryption Prevent CNG CBC size-query stack overflow for payloads > 2 MB Feb 24, 2026
Copilot AI requested a review from halter73 February 24, 2026 07:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants