[BUG] Dockerfile with nvidia/cuda:10.2-cudnn7-runtime-ubuntu18.04 gives ascii codec error

[rishabh1212]

Context

• When executing python script while opening file in read mode containing non-ascii chars give

Traceback (most recent call last):
  File "src/data_exploration/data_explore.py", line 39, in <module>
    query = f.read()
  File "/usr/lib/python3.6/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe4 in position 310: ordinal not in range(128)

• This was fixed by UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 text#77, this happens due to python 3.6

RUN apt-get update --fix-missing && apt-get install locales
RUN locale-gen en_US.UTF-8
ENV LANG en_US.UTF-8
ENV LC_ALL en_US.UTF-8

• torch version: 1.7.0+cpu
• torchvision version [if any]: 0.8.1+cpu
• torchtext version [if any]: 0.8.0
• java version:
• Operating System and version:

Your Environment

• Installed using source? [yes/no]: no (pip install)
• Are you planning to deploy it using docker container? [yes/no]: yes
• Is it a CPU or GPU environment?: GPU
• Using a default/custom handler? [If possible upload/share custom handler/model]: No
• What kind of model is it e.g. vision, text, audio?: text

Expected Behavior

While reading a file, it should not fail due to ASCII decoding. It runs successfully in Python 3.8.5

Current Behavior

• When executing python script while opening file in read mode containing non-ascii chars give

Traceback (most recent call last):
  File "src/data_exploration/data_explore.py", line 39, in <module>
    query = f.read()
  File "/usr/lib/python3.6/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe4 in position 310: ordinal not in range(128)

Possible Solution

RUN apt-get update --fix-missing && apt-get install locales
RUN locale-gen en_US.UTF-8
ENV LANG en_US.UTF-8
ENV LC_ALL en_US.UTF-8

Steps to Reproduce

DOCKER_BUILDKIT=1 docker build --file $dockerfile_name --build-arg BASE_IMAGE=nvidia/cuda:10.2-cudnn7-runtime-ubuntu18.04 -t ${image}:gpu .

docker run --net=host -e AWS_DEFAULT_REGION=ap-northeast-1 -it torchserve_recommend:gpu python

3. Try to read a file containing Japanese text without specifying the encoding type and it will fail with python 3.6. But this does not happen in 3.8.5

Failure Logs [if any]

Traceback (most recent call last):
  File "src/data_exploration/data_explore.py", line 39, in <module>
    query = f.read()
  File "/usr/lib/python3.6/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe4 in position 310: ordinal not in range(128)

[rishabh1212]

Also, I want to be a contributor, but I get access denied when I try to create a pull request.

[dhaniram-kshirsagar]

Also, I want to be a contributor, but I get access denied when I try to create a pull request.

You will have to fork this repo and make changes to that. Once it's ready you can rebase it to base repo's master.

[rishabh1212]

@harshbafna is it okay if you allow me to take up this issue?

[msaroufim]

Hi @rishabh1212, we welcome contributions so feel free to take this on and I'll review. No need for permission!