-
Notifications
You must be signed in to change notification settings - Fork 167
Closed
Labels
api: storageIssues related to the googleapis/python-storage API.Issues related to the googleapis/python-storage API.
Description
The latest https://github.com/urllib3/urllib3/releases/tag/2.6.0 has changed the ContentDecoder class, such that the existing Gzip/Brotli decoders are incompatible and throw an exception. From the release notes: If you use custom decompressors, please make sure to update them to respect the changed API of urllib3.response.ContentDecoder.
Problematic code:
python-storage/google/cloud/storage/_media/requests/download.py
Lines 699 to 766 in 6ed12b9
| class _GzipDecoder(urllib3.response.GzipDecoder): | |
| """Custom subclass of ``urllib3`` decoder for ``gzip``-ed bytes. | |
| Allows a checksum function to see the compressed bytes before they are | |
| decoded. This way the checksum of the compressed value can be computed. | |
| Args: | |
| checksum (object): | |
| A checksum which will be updated with compressed bytes. | |
| """ | |
| def __init__(self, checksum): | |
| super().__init__() | |
| self._checksum = checksum | |
| def decompress(self, data): | |
| """Decompress the bytes. | |
| Args: | |
| data (bytes): The compressed bytes to be decompressed. | |
| Returns: | |
| bytes: The decompressed bytes from ``data``. | |
| """ | |
| self._checksum.update(data) | |
| return super().decompress(data) | |
| # urllib3.response.BrotliDecoder might not exist depending on whether brotli is | |
| # installed. | |
| if hasattr(urllib3.response, "BrotliDecoder"): | |
| class _BrotliDecoder: | |
| """Handler for ``brotli`` encoded bytes. | |
| Allows a checksum function to see the compressed bytes before they are | |
| decoded. This way the checksum of the compressed value can be computed. | |
| Because BrotliDecoder's decompress method is dynamically created in | |
| urllib3, a subclass is not practical. Instead, this class creates a | |
| captive urllib3.requests.BrotliDecoder instance and acts as a proxy. | |
| Args: | |
| checksum (object): | |
| A checksum which will be updated with compressed bytes. | |
| """ | |
| def __init__(self, checksum): | |
| self._decoder = urllib3.response.BrotliDecoder() | |
| self._checksum = checksum | |
| def decompress(self, data): | |
| """Decompress the bytes. | |
| Args: | |
| data (bytes): The compressed bytes to be decompressed. | |
| Returns: | |
| bytes: The decompressed bytes from ``data``. | |
| """ | |
| self._checksum.update(data) | |
| return self._decoder.decompress(data) | |
| def flush(self): | |
| return self._decoder.flush() | |
| else: # pragma: NO COVER | |
| _BrotliDecoder = None # type: ignore # pragma: NO COVER |
Metadata
Metadata
Assignees
Labels
api: storageIssues related to the googleapis/python-storage API.Issues related to the googleapis/python-storage API.