Skip to content

Update encoder.py to work on windows #28

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Feb 15, 2019
Merged

Conversation

imgntn
Copy link
Contributor

@imgntn imgntn commented Feb 15, 2019

This fixes #26

    bpe_data = f.read()
  File "C:\Anaconda\envs\gpt-2\lib\encodings\cp1252.py", line 23, in decode
    return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 66951: character maps to <undefined>```

This fixes openai#26

```  File "C:\Users\James Pollack\Desktop\gpt-2\src\encoder.py", line 112, in get_encoder
    bpe_data = f.read()
  File "C:\Anaconda\envs\gpt-2\lib\encodings\cp1252.py", line 23, in decode
    return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 66951: character maps to <undefined>```
@WuTheFWasThat WuTheFWasThat merged commit 2d0b622 into openai:master Feb 15, 2019
@WuTheFWasThat
Copy link
Contributor

thanks!

@WuTheFWasThat
Copy link
Contributor

@imgntn same question as the other PR. I just added a LICENSE (#10) - do you agree to license your contribution under the MIT license? Thanks!

@WuTheFWasThat WuTheFWasThat mentioned this pull request Feb 20, 2019
@imgntn
Copy link
Contributor Author

imgntn commented Feb 20, 2019

Yes, I agree. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

bug in encoder.py
2 participants