CRNN-CTC cannot download file data.tar.gz

[sfraczek]

Problem description

I cannot download CRNN-CTC dataset https://github.com/PaddlePaddle/models/blob/develop/fluid/ocr_recognition/ctc_reader.py#L14.
Sever resets connection. I already retried on different networks and machines with the same result. Can you upload the file somewhere else? What is this file? Can I find it elsewhere?

Command run

python ctc_train.py

Output

λ 2b5c263aa3a5 /dataset/sfraczek/paddle-models/fluid/ocr_recognition {develop} python ctc_train.py
-----------  Configuration Arguments -----------
average_window: 0.15
batch_size: 32
eval_period: 15000
init_model: None
log_period: 1000
max_average_window: 15625
min_average_window: 10000
parallel: False
pass_num: 100
save_model_dir: ./models
save_model_period: 15000
use_gpu: True
------------------------------------------------
/usr/local/lib/python2.7/dist-packages/paddle/fluid/evaluator.py:64: Warning: The EditDistance is deprecated, because maintain a modified program inside evaluator cause bug easily, please use fluid.metrics.EditDistance instead.
  % (self.__class__.__name__, self.__class__.__name__), Warning)
file md5 b5df92d1c0ab0d585f4771e062a065e1 1de60d54d19632022144e4e58c2637b5
Cache file /root/.cache/paddle/dataset/ctc_data/data.tar.gz not found, downloading http://cloud.dlnel.org/filepub/?uuid=df937251-3c0b-480d-9a7b-0080dfeee65c
Traceback (most recent call last):
  File "ctc_train.py", line 150, in <module>
    main()
  File "ctc_train.py", line 146, in main
    train(args, data_reader=ctc_reader)
  File "ctc_train.py", line 52, in train
    train_list_file=train_list)
  File "/dataset/sfraczek/paddle-models/fluid/ocr_recognition/ctc_reader.py", line 166, in train
    data_dir = download_data()
  File "/dataset/sfraczek/paddle-models/fluid/ocr_recognition/ctc_reader.py", line 194, in download_data
    DATA_URL, CACHE_DIR_NAME, DATA_MD5, save_name=SAVED_FILE_NAME)
  File "/usr/local/lib/python2.7/dist-packages/paddle/v2/dataset/common.py", line 90, in download
    shutil.copyfileobj(r.raw, f)
  File "/usr/lib/python2.7/shutil.py", line 49, in copyfileobj
    buf = fsrc.read(length)
  File "/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/response.py", line 311, in read
    flush_decoder = True
  File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__
    self.gen.throw(type, value, traceback)
  File "/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/response.py", line 244, in _error_catcher
    raise ProtocolError('Connection broken: %r' % e, e)
requests.packages.urllib3.exceptions.ProtocolError: ('Connection broken: IncompleteRead(15973 bytes read, 411 more expected)', IncompleteRead(15973 bytes read, 411 more expected))

@wanghaoshuang maybe you could help me?

[wanghaoshuang]

@sfraczek The Chinese dataset cannot be accessed because of copyright. We are preparing a new English dataset.

[sfraczek]

Thanks for quick reply.
When can we expect it?

[wanghaoshuang]

@sfraczek It will be ready today or tomorrow. (Time in Warsaw)

[sfraczek]

Thank you 👍

[sfraczek]

I have launched the script now but it has failed unfortunately. I will try to work around this but maybe you want to know about it. Maybe you could help?

LD_LIBRARY_PATH=/usr/local/lib python infer.py --use_gpu False --model_path models/model_00000_1 --input_images_list /root/.cache/paddle/dataset/ctc_data/d
ata/test.list --input_images_dir /root/.cache/paddle/dataset/ctc_data/data/test_images/
-----------  Configuration Arguments -----------
dict: None
input_images_dir: /root/.cache/paddle/dataset/ctc_data/data/test_images/
input_images_list: /root/.cache/paddle/dataset/ctc_data/data/test.list
model_path: models/model_00000_1
use_gpu: 0
------------------------------------------------
Init model from: models/model_00000_1.
Traceback (most recent call last):
  File "infer.py", line 79, in <module>
    main()
  File "infer.py", line 75, in main
    inference(args, data_reader=ctc_reader)
  File "infer.py", line 59, in inference
    for data in infer_reader():
  File "/usr/local/lib/python2.7/dist-packages/paddle/v2/minibatch.py", line 33, in batch_reader
    for instance in r:
  File "/dataset/sfraczek/paddle-models/fluid/ocr_recognition/ctc_reader.py", line 139, in reader
    yield img, label
NameError: global name 'label' is not defined

I found that something like below might be working:

diff --git a/fluid/ocr_recognition/ctc_reader.py b/fluid/ocr_recognition/ctc_reader.py
index aa7c4ed..f469cb0 100644
--- a/fluid/ocr_recognition/ctc_reader.py
+++ b/fluid/ocr_recognition/ctc_reader.py
@@ -136,6 +136,7 @@ class DataGenerator(object):
                     img = Image.open(img_path).convert('L')
                     img = np.array(img) - 127.5
                     img = img[np.newaxis, ...]
+                    label = [int(c) for c in line.split(' ')[-1].split(',')]
                     yield img, label
             else:
                 while True:

[sfraczek]

Hey @wanghaoshuang, is this correct? I have created a pull request #912

[wanghaoshuang]

@sfraczek I'm very grateful to you for your help.