Supporting other compression methods (BZip2)

[andormarkus]

Is your feature request related to a problem? Please describe.
7z has open architecture, so it can support any new compression methods: LZMA, LZMA2, PPMD, BCJ, BCJ2, BZip2. (source: https://www.7-zip.org/7z.html)

I have a file which was compressed with BZip2 compression method and has the following properties:

Path = FILE_WITH_BZip2.7z
Type = 7z
Physical Size = 716637328
Headers Size = 146
Method = BZip2
Solid = -
Blocks = 1

I tried to decompress this file with the latest v0.6a1 release and I got the following error:

Traceback (most recent call last):
File "test.py", line 17, in
print(archive.test())
File "/home/amarkus/.pyenv/versions/3.7.3/lib/python3.7/site-packages/py7zr/py7zr.py", line 688, in test
return self._test_digests()
File "/home/amarkus/.pyenv/versions/3.7.3/lib/python3.7/site-packages/py7zr/py7zr.py", line 528, in _test_digests
if self._test_unpack_digest():
File "/home/amarkus/.pyenv/versions/3.7.3/lib/python3.7/site-packages/py7zr/py7zr.py", line 520, in _test_unpack_digest
self.worker.extract(self.fp) # TODO: print progress
File "/home/amarkus/.pyenv/versions/3.7.3/lib/python3.7/site-packages/py7zr/compression.py", line 277, in extract
self.src_start + positions[i + 1])
File "/home/amarkus/.pyenv/versions/3.7.3/lib/python3.7/site-packages/py7zr/compression.py", line 307, in extract_single
self.decompress(fp, f.folder, fileish, f.uncompressed[-1], f.compressed, src_end)
File "/home/amarkus/.pyenv/versions/3.7.3/lib/python3.7/site-packages/py7zr/compression.py", line 335, in decompress
raise DecompressionError
py7zr.exceptions.DecompressionError

Describe the solution you'd like
It would be great if py7zr package would support BZip2 compression method for file decompression.

Describe alternatives you've considered
I have tried to decompress this file in python with bz2 package but it failed. Im open to any other suggestion.

import bz2

filepath = '/home/amarkus/Downloads/FILE_WITH_BZip2.7z'
zipfile = bz2.BZ2File(filepath) # open the file
data = zipfile.read() # get the decompressed data

I got the following error:

Traceback (most recent call last):
File "test.py", line 5, in
data = zipfile.read() # get the decompressed data
File "/home/amarkus/.pyenv/versions/3.7.3/lib/python3.7/bz2.py", line 178, in read
return self._buffer.read(size)
File "/home/amarkus/.pyenv/versions/3.7.3/lib/python3.7/_compression.py", line 103, in read
data = self._decompressor.decompress(rawblock, size)
OSError: Invalid data stream

Additional context
I dont have any

[miurahr]

py7zr intend to support bzip2 compression method for 7zip format.
From your log, py7zr happily decode header information and start decompression.

An error you report is happend in

py7zr/py7zr/compression.py

Lines 327 to 333 in 1eae95a

	if read_size == 0:
	tmp = decompressor.decompress(b'', max_length)
	else:
	inp = fp.read(read_size)
	tmp = decompressor.decompress(inp, max_length)
	if len(tmp) == 0:
	raise DecompressionError

This means output data of bzip2 decompressor returns a zero size. I'm not sure a condition when it happened, and I don't have data to reproduce it.

If you can prepare minimum and small test archive to reproduce it, please upload to this report.

[andormarkus]

Hi,

Here is the sample file: 10000SalesRecords.zip
I have zipped the 7z archive so I can able to upload here.

How did I created the sample .7z file and tested it:

1. Downloaded and decompressed sample .csv file http://eforexcel.com/wp/wp-content/uploads/2017/07/10000-Sales-Records.zip

2. On Win10 using 7-Zip software I compressed the file using BZip2 compression method
   

3. Tested the decompression on Fedora 31 and py7zr v0.6a1 release

Used code for testing:

import py7zr

archive = py7zr.SevenZipFile('10000SalesRecords.7z', mode='r')
print(archive.getnames())
print(archive.test())

Results:

['10000SalesRecords.csv']
Traceback (most recent call last):
File "decompress.py", line 7, in
print(archive.test())
File "/home/amarkus/.pyenv/versions/3.7.3/lib/python3.7/site-packages/py7zr/py7zr.py", line 688, in test
return self._test_digests()
File "/home/amarkus/.pyenv/versions/3.7.3/lib/python3.7/site-packages/py7zr/py7zr.py", line 528, in _test_digests
if self._test_unpack_digest():
File "/home/amarkus/.pyenv/versions/3.7.3/lib/python3.7/site-packages/py7zr/py7zr.py", line 520, in _test_unpack_digest
self.worker.extract(self.fp) # TODO: print progress
File "/home/amarkus/.pyenv/versions/3.7.3/lib/python3.7/site-packages/py7zr/compression.py", line 277, in extract
self.src_start + positions[i + 1])
File "/home/amarkus/.pyenv/versions/3.7.3/lib/python3.7/site-packages/py7zr/compression.py", line 307, in extract_single
self.decompress(fp, f.folder, fileish, f.uncompressed[-1], f.compressed, src_end)
File "/home/amarkus/.pyenv/versions/3.7.3/lib/python3.7/site-packages/py7zr/compression.py", line 335, in decompress
raise DecompressionError
py7zr.exceptions.DecompressionError

[miurahr]

Thank you for sharing test data! I can reproduce a problem, and I'd like to add this data as test case.

[andormarkus]

Sure, feel free to add it to the test cases

[miurahr]

The case is solved and released.