Skip to content

parsing/decoding video source failed, check GPU memory is available #3770

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
SangerY opened this issue Jul 18, 2024 · 17 comments
Closed

parsing/decoding video source failed, check GPU memory is available #3770

SangerY opened this issue Jul 18, 2024 · 17 comments

Comments

@SangerY
Copy link

SangerY commented Jul 18, 2024

System information (version)
  • OpenCV => 4.10.0-pre
  • Operating System / Platform => ubuntu
  • Compiler => python3.10
  • GPU => RTX4090
Detailed description

error when decode video exceeding 1920*1080.
video_reader.cpp:174:error:(-2:Unspecified error) Parsing/Decoding video source failed, check GPU memory is available and GPU supports requested functionality. in function 'waitForDecoderInit'

Steps to reproduce
```.py
video_decoder=cv2.cudacodec.createVideoReader(path)
```
@cudawarped
Copy link
Contributor

Can you provide the video source?

@SangerY
Copy link
Author

SangerY commented Aug 1, 2024

resolved

@SangerY SangerY closed this as completed Aug 1, 2024
@swo026
Copy link

swo026 commented Oct 31, 2024

Hello, I met the same problem, how did you solve it?

@cudawarped
Copy link
Contributor

Hello, I met the same problem, how did you solve it?

There can be many sources to this error, OpenCV version, driver, unsuported video source, unsupported GPU. Can you share your video source?

@swo026
Copy link

swo026 commented Nov 1, 2024

Hello, I met the same problem, how did you solve it?

There can be many sources to this error, OpenCV version, driver, unsuported video source, unsupported GPU. Can you share your video source?

The video source is a .mp4 file

@cudawarped
Copy link
Contributor

The video source is a .mp4 file

I need the actual file to see if it is supported.

Can you also provide

  1. OpenCV version
  2. CUDA version
  3. Nvidia Video Codec SDK version
  4. OS
  5. GPU
  6. Driver version

@swo026
Copy link

swo026 commented Nov 1, 2024

I need the actual file to see if it is supported.

Can you also provide

  1. OpenCV version
  2. CUDA version
  3. Nvidia Video Codec SDK version
  4. OS
  5. GPU
  6. Driver version

Thanks for your response

1 opencv 4.10.0
2 cuda 11.8
3 Nvidia Video Codec SDK version 12.2.72
4 OS wsl2_ubuntu22.04
5 GPU 3090
6 Driver version 560.31.01
This is my video file : https://drive.google.com/file/d/168SxKktnrgmVNlZrg6l9E-F5xe2dv6ZD/view?usp=sharing

@cudawarped
Copy link
Contributor

I don't have any issues with the following on Windows

string src = "D:\\dloads\\fire1.mp4";
Ptr<cv::cudacodec::VideoReader> reader = cudacodec::createVideoReader(src);
GpuMat frame;
while (reader->nextFrame(frame)) {
    Mat frameHost; frame.download(frameHost);
    cv::imshow("Video", frameHost);
    cv::waitKey(1);
}

What is your exact error?

@swo026
Copy link

swo026 commented Nov 4, 2024

I don't have any issues with the following on Windows

string src = "D:\\dloads\\fire1.mp4";
Ptr<cv::cudacodec::VideoReader> reader = cudacodec::createVideoReader(src);
GpuMat frame;
while (reader->nextFrame(frame)) {
    Mat frameHost; frame.download(frameHost);
    cv::imshow("Video", frameHost);
    cv::waitKey(1);
}

What is your exact error?

The code is:

camera_path = './src/video/fire/fire1.mp4'
cap = cv2.cudacodec.createVideoReader(camera_path)

The specific error output is:

/opt/miniconda3/envs/cv/lib/python3.10/site-packages/numpy/core/getlimits.py:549: UserWarning: The value of the smallest subnormal for <class 'numpy.float64'> type is zero.
  setattr(self, word, getattr(machar, word).flat[0])
/opt/miniconda3/envs/cv/lib/python3.10/site-packages/numpy/core/getlimits.py:89: UserWarning: The value of the smallest subnormal for <class 'numpy.float64'> type is zero.
  return self._float_to_str(self.smallest_subnormal)
/opt/miniconda3/envs/cv/lib/python3.10/site-packages/numpy/core/getlimits.py:549: UserWarning: The value of the smallest subnormal for <class 'numpy.float32'> type is zero.
  setattr(self, word, getattr(machar, word).flat[0])
/opt/miniconda3/envs/cv/lib/python3.10/site-packages/numpy/core/getlimits.py:89: UserWarning: The value of the smallest subnormal for <class 'numpy.float32'> type is zero.
  return self._float_to_str(self.smallest_subnormal)
[ERROR:[email protected]] global video_parser.cpp:86 parseVideoData OpenCV(4.10.0-dev) /mnt/e/project/src/opencv/opencv_contrib/modules/cudacodec/src/video_decoder.cpp:124: error: (-217:Gpu API call) CUDA_ERROR_UNKNOWN [Code = 999] in function 'create'

Traceback (most recent call last):
  File "/mnt/e/project/code/dev/cloud_platform/cloud-platform/sia-compute-engine/video-analysis-engine/with_ffmpeg/server_test.py", line 14, in <module>
    cv2.cudacodec.createVideoReader(camera_path)
cv2.error: OpenCV(4.10.0-dev) /mnt/e/project/src/opencv/opencv_contrib/modules/cudacodec/src/video_reader.cpp:174: error: (-2:Unspecified error) Parsing/Decoding video source failed, check GPU memory is available and GPU supports requested functionality. in function 'waitForDecoderInit'

@cudawarped
Copy link
Contributor

It looks like your first call to anything which uses the Nvidia decode library fails. This points to an installation or dependancy issue.

Can you check that you didn't copy the Nvidia Video Codec SDK stub libraries into a folder on your path? i.e. /usr/local/cuda/lib64

If you did its possible that OpenCV is trying to use a stub .so instead of the one provided by the driver.

A quick check would be to look at the output from ldd on your cv2.abi3.so (the exact name may differ depending on how you installed OpenCV)

ldd /home/b/miniforge3/lib/python3.10/site-packages/cv2/cv2.abi3.so

    linux-vdso.so.1 (0x00007ffe6e99f000)
    libcuda.so.1 => /usr/lib/wsl/lib/libcuda.so.1 (0x00007efdb3a00000)
    libnvcuvid.so.1 => /usr/lib/wsl/lib/libnvcuvid.so.1 (0x00007efdb2d89000)
    libnvidia-encode.so.1 => /usr/lib/wsl/lib/libnvidia-encode.so.1 (0x00007efdb3d14000)
    libavcodec.so.58 => /lib/x86_64-linux-gnu/libavcodec.so.58 (0x00007efdb1981000)
    libavformat.so.58 => /lib/x86_64-linux-gnu/libavformat.so.58 (0x00007efdb16e8000)
    libavutil.so.56 => /lib/x86_64-linux-gnu/libavutil.so.56 (0x00007efdb142f000)
    libswscale.so.5 => /lib/x86_64-linux-gnu/libswscale.so.5 (0x00007efdb3c68000)
    libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007efdb3c4c000)
    libnppc.so.12 => /usr/local/cuda/lib64/libnppc.so.12 (0x00007efdb1000000)
    libnppial.so.12 => /usr/local/cuda/lib64/libnppial.so.12 (0x00007efdafc00000)
    libnppicc.so.12 => /usr/local/cuda/lib64/libnppicc.so.12 (0x00007efdaf200000)
    libnppidei.so.12 => /usr/local/cuda/lib64/libnppidei.so.12 (0x00007efdae400000)
    libnppif.so.12 => /usr/local/cuda/lib64/libnppif.so.12 (0x00007efda8000000)
    libnppig.so.12 => /usr/local/cuda/lib64/libnppig.so.12 (0x00007efda5600000)
    libnppim.so.12 => /usr/local/cuda/lib64/libnppim.so.12 (0x00007efda4800000)
    libnppist.so.12 => /usr/local/cuda/lib64/libnppist.so.12 (0x00007efda2000000)
    libnppitc.so.12 => /usr/local/cuda/lib64/libnppitc.so.12 (0x00007efda1800000)
    libcublas.so.12 => /usr/local/cuda/lib64/libcublas.so.12 (0x00007efd9b200000)
    libcudnn.so.9 => /usr/local/cuda/lib64/libcudnn.so.9 (0x00007efd9ae00000)
    libcufft.so.11 => /usr/local/cuda/lib64/libcufft.so.11 (0x00007efd8a000000)

...

Notice that libnvcuvid.so.1 => /usr/lib/wsl/lib/libnvcuvid.so.1 (0x00007efdb2d89000) is part of the driver and not located inside /usr/local/cuda/lib64/ like the other CUDA libraries (libnppc.so.12 etc.).

@hepingtao
Copy link

hepingtao commented Dec 5, 2024

@cudawarped
I am an another one facing the exactly same error. And my error occurs when the video resolution is exceeded over 4K(3840 * 2160). The information is below, any advices will be appreciated.

  • Version
  1. OpenCV: 4.10.0
  2. CUDA: 11.8
  3. Nvidia Video Codec SDK: 12.2.72
  4. OS: CentOS 8.8
  5. GPU: Quadro RTX 4000
  6. GPU Driver Version: 550.135, note: previous version 525 does not work, so I upgrade to the latest version, still the same.
  • ldd ~/micromamba/envs/py310_opencv_cuda/lib/python3.10/site-packages/cv2/cv2.abi3.so
        linux-vdso.so.1 (0x00007ffe4070f000)
        libcuda.so.1 => /lib64/libcuda.so.1 (0x00007f8486e34000)
        libnvcuvid.so.1 => /lib64/libnvcuvid.so.1 (0x00007f8486200000)
        libnvidia-encode.so.1 => /lib64/libnvidia-encode.so.1 (0x00007f8485e00000)
        libgstbase-1.0.so.0 => /lib64/libgstbase-1.0.so.0 (0x00007f8485a00000)
        libgstreamer-1.0.so.0 => /lib64/libgstreamer-1.0.so.0 (0x00007f8485600000)
        libgobject-2.0.so.0 => /lib64/libgobject-2.0.so.0 (0x00007f8485200000)
        libglib-2.0.so.0 => /lib64/libglib-2.0.so.0 (0x00007f8484e00000)
        libgstapp-1.0.so.0 => /lib64/libgstapp-1.0.so.0 (0x00007f8484a00000)
        libgstriff-1.0.so.0 => /lib64/libgstriff-1.0.so.0 (0x00007f8484600000)
        libgstpbutils-1.0.so.0 => /lib64/libgstpbutils-1.0.so.0 (0x00007f8484200000)
        libgstvideo-1.0.so.0 => /lib64/libgstvideo-1.0.so.0 (0x00007f8483e00000)
        libgstaudio-1.0.so.0 => /lib64/libgstaudio-1.0.so.0 (0x00007f8483a00000)
        libavcodec.so.58 => /home/itouchtv/soft/ffmpeg/lib/libavcodec.so.58 (0x00007f8482400000)
        libavformat.so.58 => /home/itouchtv/soft/ffmpeg/lib/libavformat.so.58 (0x00007f8481e00000)
        libavutil.so.56 => /home/itouchtv/soft/ffmpeg/lib/libavutil.so.56 (0x00007f8481800000)
        libswscale.so.5 => /home/itouchtv/soft/ffmpeg/lib/libswscale.so.5 (0x00007f8481400000)
        libpng16.so.16 => /lib64/libpng16.so.16 (0x00007f8481000000)
        libz.so.1 => /lib64/libz.so.1 (0x00007f8480c00000)
        libopenblas.so.0 => /lib64/libopenblas.so.0 (0x00007f847e950000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f847e600000)
        libnppc.so.11 => /home/itouchtv/cuda/cuda_root/lib64/libnppc.so.11 (0x00007f847e200000)
        libnppial.so.11 => /home/itouchtv/cuda/cuda_root/lib64/libnppial.so.11 (0x00007f847d000000)
        libnppicc.so.11 => /home/itouchtv/cuda/cuda_root/lib64/libnppicc.so.11 (0x00007f847c600000)
        libnppidei.so.11 => /home/itouchtv/cuda/cuda_root/lib64/libnppidei.so.11 (0x00007f847b800000)
        libnppif.so.11 => /home/itouchtv/cuda/cuda_root/lib64/libnppif.so.11 (0x00007f8475200000)
        libnppig.so.11 => /home/itouchtv/cuda/cuda_root/lib64/libnppig.so.11 (0x00007f8472a00000)
        libnppim.so.11 => /home/itouchtv/cuda/cuda_root/lib64/libnppim.so.11 (0x00007f8471e00000)
        libnppist.so.11 => /home/itouchtv/cuda/cuda_root/lib64/libnppist.so.11 (0x00007f846f400000)
        libnppitc.so.11 => /home/itouchtv/cuda/cuda_root/lib64/libnppitc.so.11 (0x00007f846ec00000)
        libcublas.so.11 => /home/itouchtv/cuda/cuda_root/lib64/libcublas.so.11 (0x00007f8468e00000)
        libcudnn.so.8 => /home/itouchtv/cuda/cuda_root/lib64/libcudnn.so.8 (0x00007f8468a00000)
        libcufft.so.10 => /home/itouchtv/cuda/cuda_root/lib64/libcufft.so.10 (0x00007f8457a00000)
        librt.so.1 => /lib64/librt.so.1 (0x00007f8457600000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007f8457200000)
        libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f8456e00000)
        libm.so.6 => /lib64/libm.so.6 (0x00007f8456a00000)
        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f8456600000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f8456200000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f8495ba5000)
        libgmodule-2.0.so.0 => /lib64/libgmodule-2.0.so.0 (0x00007f8455e00000)
        libffi.so.6 => /lib64/libffi.so.6 (0x00007f8455a00000)
        libpcre.so.1 => /lib64/libpcre.so.1 (0x00007f8455600000)
        libgsttag-1.0.so.0 => /lib64/libgsttag-1.0.so.0 (0x00007f8455200000)
        liborc-0.4.so.0 => /lib64/liborc-0.4.so.0 (0x00007f8454e00000)
        libswresample.so.3 => /home/itouchtv/soft/ffmpeg/lib/libswresample.so.3 (0x00007f8454a00000)
        liblzma.so.5 => /lib64/liblzma.so.5 (0x00007f8454600000)
        libbz2.so.1 => /lib64/libbz2.so.1 (0x00007f8454200000)
        libgfortran.so.5 => /lib64/libgfortran.so.5 (0x00007f8453c00000)
        libcublasLt.so.11 => /home/itouchtv/cuda/cuda_root/lib64/libcublasLt.so.11 (0x00007f842f600000)
        libquadmath.so.0 => /lib64/libquadmath.so.0 (0x00007f842f200000)

@cudawarped
Copy link
Contributor

Did you copy /lib64/libnvidia-encode.so.1 and /lib64/libnvcuvid.so.1 from the Lib/linux/stubs directory or are they part of the driver?

If you copied them then they need to be removed as they are stubs provided for building on systems without the Nvidia driver installed. i.e. they don't contain any implementation code for accessing the hardware encoder/decoder.

@hepingtao
Copy link

Did you copy /lib64/libnvidia-encode.so.1 and /lib64/libnvcuvid.so.1 from the Lib/linux/stubs directory or are they part of the driver?

No, I did not copy them from Video_Codec_SDK_12.2.72/Lib/linux/stubs/x86_64 to /lib64, they existed likely with the driver installed together, as following:

lrwxrwxrwx 1 root root 21 Dec  5 17:42 /lib64/libnvcuvid.so.1 -> libnvcuvid.so.550.135
lrwxrwxrwx 1 root root 27 Dec  5 17:42 /lib64/libnvidia-encode.so.1 -> libnvidia-encode.so.550.135

But the other thing which I have done was that I surely copied the .so files from Video_Codec_SDK_12.2.72/Lib/linux/stubs/x86_64 to <CUDA_PATH>/lib64/stubs, like this:

-rw-rw-r-- 1 itouchtv itouchtv 3528 Sep  9 15:58 /home/itouchtv/cuda/cuda_root/lib64/stubs/libnvcuvid.so
-rw-rw-r-- 1 itouchtv itouchtv 1480 Sep  9 15:58 /home/itouchtv/cuda/cuda_root/lib64/stubs/libnvidia-encode.so

If you copied them then they need to be removed as they are stubs provided for building on systems without the Nvidia driver installed. i.e. they don't contain any implementation code for accessing the hardware encoder/decoder.

So what is the next step for me, do I need to try to remove the symbolic links?

@cudawarped
Copy link
Contributor

So what is the next step for me, do I need to try to remove the symbolic links?

Copying the stubs to the stubs directory should be fine as you can see from the output in ldd your python bindings should not be using them. That said to be 100% you could just remove them and see if it runs in case for some reason they are being added to a path at runtime by micromamba.

Are you getting the exact same error as described above?

@hepingtao
Copy link

Actually my first try is to rename the cuda_root/lib64/stubs/libnvcuvid.so to another one, but does not work.

All the content of error I encountered is shown below:

$ ipython
Python 3.10.14 | packaged by conda-forge | (main, Mar 20 2024, 12:45:18) [GCC 12.3.0]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.27.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import cv2

In [2]: video_path = '~/hpt/output.mp4'

In [3]: video_capture_cuda = cv2.cudacodec.createVideoReader(video_path)
[ERROR:0@39.220] global video_parser.cpp:86 parseVideoData OpenCV(4.10.0) /home/itouchtv/hpt/opencv_src/opencv-python/opencv_contrib/modules/cudacodec/src/video_decoder.cpp:149: error: (-215:Assertion failed) videoFormat.ulWidth >= decodeCaps.nMinWidth && videoFormat.ulHeight >= decodeCaps.nMinHeight && videoFormat.ulWidth <= decodeCaps.nMaxWidth && videoFormat.ulHeight <= decodeCaps.nMaxHeight in function 'create'

---------------------------------------------------------------------------
error                                     Traceback (most recent call last)
Cell In[3], line 1
----> 1 video_capture_cuda = cv2.cudacodec.createVideoReader(video_path)

error: OpenCV(4.10.0) /home/itouchtv/hpt/opencv_src/opencv-python/opencv_contrib/modules/cudacodec/src/video_reader.cpp:174: error: (-2:Unspecified error) Parsing/Decoding video source failed, check GPU memory is available and GPU supports requested functionality. in function 'waitForDecoderInit'


In [4]: video_capture = cv2.VideoCapture(video_path)

In [6]: video_capture.get(cv2.CAP_PROP_FRAME_WIDTH)
Out[6]: 4724.0

In [7]: video_capture.get(cv2.CAP_PROP_FRAME_HEIGHT)
Out[7]: 2656.0

In [8]: _, frame = video_capture.read()

In [10]: cv2.imwrite('frame.jpg', frame)
Out[10]: True

In other words, when the video resolution exceeds 3840 * 2160, my OpenCV using GPU for frame extraction encounters this error, while using the CPU works normally; if the video resolution is equal to or less than 3840 * 2160, both work normally. However, generally speaking, when the video resolution reaches above 3840 * 2160, GPU processing speed is much faster than CPU, so I want to resolve this issue.

@cudawarped
Copy link
Contributor

cudawarped commented Dec 6, 2024

The error is not the same but is explicit in the cause

[ERROR:[email protected]] global video_parser.cpp:86 parseVideoData OpenCV(4.10.0) /home/itouchtv/hpt/opencv_src/opencv-python/opencv_contrib/modules/cudacodec/src/video_decoder.cpp:149: error: (-215:Assertion failed) videoFormat.ulWidth >= decodeCaps.nMinWidth && videoFormat.ulHeight >= decodeCaps.nMinHeight && videoFormat.ulWidth <= decodeCaps.nMaxWidth && videoFormat.ulHeight <= decodeCaps.nMaxHeight in function 'create'

The message could be a little more explicit but its saying that your hardware decoder does not support the width or height of the encoded video. Refer to the Video Decoder Capabilities table for a list of supported resolutions by codec. What codec are you using?

If the table indicates that the video should be supported can you share the video so I can test it?

@hepingtao
Copy link

@cudawarped
Sorry to keep you waiting, I followed the guidelines you provided and checked all the information covered in the table as detailed below:

GPU Architecture: (There are two machines)

  1. GA102GL is part of the Ampere architecture.
  2. TU104GL is a variant of the Turing architecture
lspci | grep -i NVIDIA

Output 1:

00:08.0 3D controller: NVIDIA Corporation GA102GL [A10] (rev a1)

Output 2:

01:00.0 VGA compatible controller: NVIDIA Corporation TU104GL [Quadro RTX 4000] (rev a1)

Video codec: h264

ffmpeg -i output.mp4

Output :

  Stream #0:0(eng): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv, bt709), 4724x2656, 9998 kb/s, 25 fps, 25 tbr, 25k tbn, 50 tbc (default)

Comparing the table requirements, my GPU architecture is in the last two rows, find the column corresponding to h264 (column name is H.264/AVCHD)

20241210113316

The result is: Maximum Resolution: 4096 x 4096

So the conclusion should be: The maximum resolution of the H264 codec video supported by my GPU architecture is 4096 x 4096, and a width or height greater than this value will not work.

Thank you very much for your help and have fun.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants