bpo-43794: OpenSSL 3.0.0: set OP_IGNORE_UNEXPECTED_EOF by default (GH-25309) #25309

tiran · 2021-04-09T14:44:55Z

Signed-off-by: Christian Heimes [email protected]

https://bugs.python.org/issue43794

Signed-off-by: Christian Heimes <[email protected]>

miss-islington · 2021-04-09T15:59:23Z

Thanks @tiran for the PR 🌮🎉.. I'm working now to backport this PR to: 3.8, 3.9.
🐍🍒⛏🤖

bedevere-bot · 2021-04-09T15:59:32Z

GH-25313 is a backport of this pull request to the 3.9 branch.

…thonGH-25309) Signed-off-by: Christian Heimes <[email protected]> (cherry picked from commit 6f37ebc) Co-authored-by: Christian Heimes <[email protected]>

bedevere-bot · 2021-04-09T15:59:40Z

GH-25314 is a backport of this pull request to the 3.8 branch.

…-25309) Signed-off-by: Christian Heimes <[email protected]> (cherry picked from commit 6f37ebc) Co-authored-by: Christian Heimes <[email protected]>

…thonGH-25309)

pythonGH-25309 enabled SSL_OP_IGNORE_UNEXPECTED_EOF by default, with a comment that it restores OpenSSL 1.1.1 behavior, but this wasn't quite right. That option causes OpenSSL to treat transport EOF as the same as close_notify (i.e. SSL_ERROR_ZERO_RETURN), whereas Python actually has distinct SSLEOFError and SSLZeroReturnError exceptions. (The latter is usually mapped to a zero return from read.) In OpenSSL 1.1.1, the ssl module would raise them for transport EOF and close_notify, respectively. In OpenSSL 3.0, both act like close_notify. Fix this by, instead, just detecting SSL_R_UNEXPECTED_EOF_WHILE_READING and mapping that to the other exception type. There doesn't seem to have been any unit test of this error, so fill in the missing one. This had to be done with the BIO path because it's actually slightly tricky to simulate a transport EOF with Python's fd based APIs. (If you instruct the server to close the socket, it gets confused, probably because the server's SSL object is still referencing the now dead fd?)

GH-25309 enabled SSL_OP_IGNORE_UNEXPECTED_EOF by default, with a comment that it restores OpenSSL 1.1.1 behavior, but this wasn't quite right. That option causes OpenSSL to treat transport EOF as the same as close_notify (i.e. SSL_ERROR_ZERO_RETURN), whereas Python actually has distinct SSLEOFError and SSLZeroReturnError exceptions. (The latter is usually mapped to a zero return from read.) In OpenSSL 1.1.1, the ssl module would raise them for transport EOF and close_notify, respectively. In OpenSSL 3.0, both act like close_notify. Fix this by, instead, just detecting SSL_R_UNEXPECTED_EOF_WHILE_READING and mapping that to the other exception type. There doesn't seem to have been any unit test of this error, so fill in the missing one. This had to be done with the BIO path because it's actually slightly tricky to simulate a transport EOF with Python's fd based APIs. (If you instruct the server to close the socket, it gets confused, probably because the server's SSL object is still referencing the now dead fd?)

…5495) pythonGH-25309 enabled SSL_OP_IGNORE_UNEXPECTED_EOF by default, with a comment that it restores OpenSSL 1.1.1 behavior, but this wasn't quite right. That option causes OpenSSL to treat transport EOF as the same as close_notify (i.e. SSL_ERROR_ZERO_RETURN), whereas Python actually has distinct SSLEOFError and SSLZeroReturnError exceptions. (The latter is usually mapped to a zero return from read.) In OpenSSL 1.1.1, the ssl module would raise them for transport EOF and close_notify, respectively. In OpenSSL 3.0, both act like close_notify. Fix this by, instead, just detecting SSL_R_UNEXPECTED_EOF_WHILE_READING and mapping that to the other exception type. There doesn't seem to have been any unit test of this error, so fill in the missing one. This had to be done with the BIO path because it's actually slightly tricky to simulate a transport EOF with Python's fd based APIs. (If you instruct the server to close the socket, it gets confused, probably because the server's SSL object is still referencing the now dead fd?) (cherry picked from commit 420bbb7) Co-authored-by: David Benjamin <[email protected]>

…5495) pythonGH-25309 enabled SSL_OP_IGNORE_UNEXPECTED_EOF by default, with a comment that it restores OpenSSL 1.1.1 behavior, but this wasn't quite right. That option causes OpenSSL to treat transport EOF as the same as close_notify (i.e. SSL_ERROR_ZERO_RETURN), whereas Python actually has distinct SSLEOFError and SSLZeroReturnError exceptions. (The latter is usually mapped to a zero return from read.) In OpenSSL 1.1.1, the ssl module would raise them for transport EOF and close_notify, respectively. In OpenSSL 3.0, both act like close_notify. Fix this by, instead, just detecting SSL_R_UNEXPECTED_EOF_WHILE_READING and mapping that to the other exception type. There doesn't seem to have been any unit test of this error, so fill in the missing one. This had to be done with the BIO path because it's actually slightly tricky to simulate a transport EOF with Python's fd based APIs. (If you instruct the server to close the socket, it gets confused, probably because the server's SSL object is still referencing the now dead fd?)

…#103006) GH-25309 enabled SSL_OP_IGNORE_UNEXPECTED_EOF by default, with a comment that it restores OpenSSL 1.1.1 behavior, but this wasn't quite right. That option causes OpenSSL to treat transport EOF as the same as close_notify (i.e. SSL_ERROR_ZERO_RETURN), whereas Python actually has distinct SSLEOFError and SSLZeroReturnError exceptions. (The latter is usually mapped to a zero return from read.) In OpenSSL 1.1.1, the ssl module would raise them for transport EOF and close_notify, respectively. In OpenSSL 3.0, both act like close_notify. Fix this by, instead, just detecting SSL_R_UNEXPECTED_EOF_WHILE_READING and mapping that to the other exception type. There doesn't seem to have been any unit test of this error, so fill in the missing one. This had to be done with the BIO path because it's actually slightly tricky to simulate a transport EOF with Python's fd based APIs. (If you instruct the server to close the socket, it gets confused, probably because the server's SSL object is still referencing the now dead fd?) (cherry picked from commit 420bbb7) Co-authored-by: David Benjamin <[email protected]>

…#103007) GH-25309 enabled SSL_OP_IGNORE_UNEXPECTED_EOF by default, with a comment that it restores OpenSSL 1.1.1 behavior, but this wasn't quite right. That option causes OpenSSL to treat transport EOF as the same as close_notify (i.e. SSL_ERROR_ZERO_RETURN), whereas Python actually has distinct SSLEOFError and SSLZeroReturnError exceptions. (The latter is usually mapped to a zero return from read.) In OpenSSL 1.1.1, the ssl module would raise them for transport EOF and close_notify, respectively. In OpenSSL 3.0, both act like close_notify. Fix this by, instead, just detecting SSL_R_UNEXPECTED_EOF_WHILE_READING and mapping that to the other exception type. There doesn't seem to have been any unit test of this error, so fill in the missing one. This had to be done with the BIO path because it's actually slightly tricky to simulate a transport EOF with Python's fd based APIs. (If you instruct the server to close the socket, it gets confused, probably because the server's SSL object is still referencing the now dead fd?) (cherry picked from commit 420bbb7) Co-authored-by: David Benjamin <[email protected]>

…5495) pythonGH-25309 enabled SSL_OP_IGNORE_UNEXPECTED_EOF by default, with a comment that it restores OpenSSL 1.1.1 behavior, but this wasn't quite right. That option causes OpenSSL to treat transport EOF as the same as close_notify (i.e. SSL_ERROR_ZERO_RETURN), whereas Python actually has distinct SSLEOFError and SSLZeroReturnError exceptions. (The latter is usually mapped to a zero return from read.) In OpenSSL 1.1.1, the ssl module would raise them for transport EOF and close_notify, respectively. In OpenSSL 3.0, both act like close_notify. Fix this by, instead, just detecting SSL_R_UNEXPECTED_EOF_WHILE_READING and mapping that to the other exception type. There doesn't seem to have been any unit test of this error, so fill in the missing one. This had to be done with the BIO path because it's actually slightly tricky to simulate a transport EOF with Python's fd based APIs. (If you instruct the server to close the socket, it gets confused, probably because the server's SSL object is still referencing the now dead fd?)

…thonGH-25309) Signed-off-by: Christian Heimes <[email protected]> (cherry picked from commit 6f37ebc) Co-authored-by: Christian Heimes <[email protected]>

bpo-43794: OpenSSL 3.0.0: set OP_IGNORE_UNEXPECTED_EOF by default

ae65096

Signed-off-by: Christian Heimes <[email protected]>

tiran added needs backport to 3.8 needs backport to 3.9 only security fixes labels Apr 9, 2021

the-knights-who-say-ni added the CLA signed label Apr 9, 2021

bedevere-bot added the awaiting core review label Apr 9, 2021

tiran changed the title ~~bpo-43794: OpenSSL 3.0.0: set OP_IGNORE_UNEXPECTED_EOF by default~~ bpo-43794: OpenSSL 3.0.0: set OP_IGNORE_UNEXPECTED_EOF by default (GH-25309) Apr 9, 2021

tiran merged commit 6f37ebc into python:master Apr 9, 2021

tiran deleted the bpo-43794-ssl-eof branch April 9, 2021 15:59

bedevere-bot removed the awaiting core review label Apr 9, 2021

bedevere-bot removed the needs backport to 3.9 only security fixes label Apr 9, 2021

bedevere-bot removed the needs backport to 3.8 label Apr 9, 2021

This was referenced Jun 29, 2022

[Sidekiq 6.5.1] SSL_read: unexpected eof while reading sidekiq/sidekiq#5402

Closed

OpenSSL::SSL::SSLError: SSL_read: unexpected eof while reading redis/redis-rb#1106

Closed

tiran added a commit to tiran/cpython that referenced this pull request Jul 1, 2022

bpo-43794: OpenSSL 3.0.0: set OP_IGNORE_UNEXPECTED_EOF by default (py…

9c16ffe

…thonGH-25309)

edmorley mentioned this pull request Jul 6, 2022

SSL_read: unexpected eof while reading using redis-rb on Heroku-22 heroku/base-images#220

Closed

This was referenced Jul 31, 2022

ssl module with OpenSSL 3.0 does not throw SSLEOFError on unclean shutdown #95494

Closed

gh-95494: Fix transport EOF handling in OpenSSL 3.0 #95495

Merged

This was referenced Mar 24, 2023

[3.11] GH-95494: Fix transport EOF handling in OpenSSL 3.0 (GH-95495) #103006

Merged

[3.10] GH-95494: Fix transport EOF handling in OpenSSL 3.0 (GH-95495) #103007

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

bpo-43794: OpenSSL 3.0.0: set OP_IGNORE_UNEXPECTED_EOF by default (GH-25309) #25309

bpo-43794: OpenSSL 3.0.0: set OP_IGNORE_UNEXPECTED_EOF by default (GH-25309) #25309

tiran commented Apr 9, 2021 •

edited by bedevere-bot

Loading

miss-islington commented Apr 9, 2021

bedevere-bot commented Apr 9, 2021

bedevere-bot commented Apr 9, 2021

bpo-43794: OpenSSL 3.0.0: set OP_IGNORE_UNEXPECTED_EOF by default (GH-25309) #25309

bpo-43794: OpenSSL 3.0.0: set OP_IGNORE_UNEXPECTED_EOF by default (GH-25309) #25309

Conversation

tiran commented Apr 9, 2021 • edited by bedevere-bot Loading

miss-islington commented Apr 9, 2021

bedevere-bot commented Apr 9, 2021

bedevere-bot commented Apr 9, 2021

tiran commented Apr 9, 2021 •

edited by bedevere-bot

Loading