Can't use Password with Unicode

[christianwejwoda]

Steps to reproduce

Create zipFile using Password "DatenFürDich" which uses the german ü == u with dots.

Expected behavior

the Password should be usefull when extracting files

Actual behavior

When trying zu extract again with Windows or 7Zip --> Wrong Password

Version of SharpZipLib

1.2.0

Obtained from (only keep the relevant lines)

• Package installed using NuGet

[Numpsy]

It looks like 7-zip won't allow you to create a .zip file with that password, not sure offhand if that's something decided by 7-zip, or is in the spec somewhere.

Looking at the code, I wonder if there is an issue with the way it handles passwords for ZipCrypto - it currently converts the supplied password to a byte array using the codepage specified in ZipStrings (which currently defaults to UTF-8), and i'm not certain if that is something that should be configurable?
Changing it to use Windows 1252 for the conversion seems to allow the Windows built in zip support to open the file, but i'm not certain if that is the right general approach either.

[piksel]

I'm not even sure we're talking about ZipCrypto (we should probably start using this name since 7zip calls it that) or about AES encryption passwords.
For the ZipCrypto ones we should actually not depend on the Unicode flag (which we do right now):

D.5 General purpose bit 11 will not imply any encoding of file content or
password.  Values defining character encoding for file content or 
password MUST be stored within the 0x0008 Extended Language Encoding 
Extra Field.

I have not looked into using the extra fields for this though. But if nothing else, we should at least only use CP437/1252 right now.

[Numpsy]

I took the comment about Windows giving you a wrong password error to mean ZipCrypto, as I don't think Explorer will extract AES encrypted files created by 7zip etc anyway (you get an error rather than a password request trying to read such entries I think?)

Perhaps I misunderstood though)

[christianwejwoda]

That i know! I uses 7-ZIP for my attempt to unzip the File.

…

I took the comment about Windows giving you a wrong password error to mean ZipCrypto, as I don't think Explorer will extract > AES encrypted files created by 7zip etc anyway? (perhaps I misunderstood though)

[Numpsy]

D.5 General purpose bit 11 will not imply any encoding of file content or
password.  Values defining character encoding for file content or 
password MUST be stored within the 0x0008 Extended Language Encoding 
Extra Field.

I like how it uses MUST there, while also saying that the format of the 0x0008 field is undefined :-(

[Numpsy]

I have not looked into using the extra fields for this though. But if nothing else, we should at least only use CP437/1252 right now.

What's the situation with doing that in .NET Standard and/or .NET Core? (you might need to use the System.Text.Encoding.CodePages package to ensure the code pages are available in Core I think?)

[piksel]

.NET Core supports Unicode and ISO 8859-1 (CP 28591), so we can at least fall back to using that if 437 is not available. The differences can be seen here: https://en.wikipedia.org/wiki/Windows-1252#Character_set
Ideally we would throw an exception if a character that falls outside of the shared ones and we are in ISO 8859-1 fallback mode...

[Numpsy]

throw an exception if a character that falls outside of the shared ones

The shared ones being ASCII, from the looks of it?

That's the approach that 7-Zip seems to take anyway:

[piksel]

The shared ones are all but some parts of 0x80-0x9f, they are indicated by the thick border in the linked table.

[Numpsy]

I might be confusing myself reading the different tables, but the

ü

character in the ops password looks to map to 0x81 in CP437 and 0xFC in 1252, which is what I meant by not-shared (it isn't in the bordered area in the 1252 table).

(Edit: I took your previous comment to mean throw if a character is one that would be different between CP437 and ISO 8859-1, which looks like a larger range than 0x80-0x9f)