I'm trying to get GStreamer + OpenCV RTSP video capture working in a Docker container based on a NVIDIA PyTorch image. I had to end up building OpenCV from source to enable GStreamer integration, which I do in my Dockerfile like so:
FROM nvcr.io/nvidia/pytorch:19.12-py3
# OpenCV custom build instructions from:
# https://medium.com/@galaktyk01/how-to-build-opencv-with-gstreamer-b11668fa09c
# https://github.com/junjuew/Docker-OpenCV-GStreamer/blob/master/opencv3-gstreamer1.0-Dockerfile
# Install base dependencies + gstreamer
RUN pip uninstall -y opencv-python
RUN apt-get update
RUN apt-get -y install build-essential
RUN apt-get -y install pkg-config
RUN apt-get install -y libgstreamer1.0-0 \
gstreamer1.0-plugins-base \
gstreamer1.0-plugins-good \
gstreamer1.0-plugins-bad \
gstreamer1.0-plugins-ugly \
gstreamer1.0-libav \
gstreamer1.0-doc \
gstreamer1.0-tools \
libgstreamer1.0-dev \
libgstreamer-plugins-base1.0-dev \
cmake \
protobuf-compiler \
libgtk2.0-dev \
ocl-icd-opencl-dev
# Clone OpenCV repo
WORKDIR /
RUN git clone https://github.com/opencv/opencv.git
WORKDIR /opencv
RUN git checkout 4.2.0
# Build OpenCV
RUN mkdir /opencv/build
WORKDIR /opencv/build
RUN ln -s /opt/conda/lib/python3.6/site-packages/numpy/core/include/numpy /usr/include/numpy
RUN cmake -D CMAKE_BUILD_TYPE=RELEASE \
-D INSTALL_PYTHON_EXAMPLES=ON \
-D INSTALL_C_EXAMPLES=OFF \
-D PYTHON_EXECUTABLE=$(which python) \
-D BUILD_opencv_python2=OFF \
-D CMAKE_INSTALL_PREFIX=$(python -c "import sys; print(sys.prefix)") \
-D PYTHON3_EXECUTABLE=$(which python3) \
-D PYTHON3_INCLUDE_DIR=$(python -c "from distutils.sysconfig import get_python_inc; print(get_python_inc())") \
-D PYTHON3_PACKAGES_PATH=$(python -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())") \
-D WITH_GSTREAMER=ON \
-D BUILD_EXAMPLES=ON ..
RUN make -j$(nproc)
# Install OpenCV
RUN make install
RUN ldconfig
This builds successfully, and if I print OpenCV's build information from within the Docker container, GStreamer shows as available:
python -c 'import cv2; print(cv2.getBuildInformation());'
/* snip */
Video I/O:
DC1394: NO
FFMPEG: NO
avcodec: NO
avformat: NO
avutil: NO
swscale: NO
avresample: NO
GStreamer: YES (1.14.5)
v4l/v4l2: YES (linux/videodev2.h)
However, as soon as I try to use a GStreamer pipeline with cv2.VideoCapture()
within the Docker container, it immediately fails:
import cv2
video = cv2.VideoCapture('gst-launch-1.0 rtspsrc location=<<rtsp URL>> latency=0 ! queue ! rtph264depay ! h264parse ! avdec_h264 ! videoconvert ! appsink', cv2.CAP_GSTREAMER)
I get this "warning" (e.g., error). I'm not able to pull frames from the RTSP feed.
[ WARN:0] global /opencv/modules/videoio/src/cap_gstreamer.cpp (713) open OpenCV | GStreamer warning: Error opening bin: unexpected reference "gst-launch-1" - ignoring
[ WARN:0] global /opencv/modules/videoio/src/cap_gstreamer.cpp (480) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created
If I do this outside of the Docker container, it works like a charm. Also note, if I run the GStreamer pipeline from the command line within the Docker container, I get reasonable output that's identical to running the same command outside of the Docker container:
root:/# gst-launch-1.0 rtspsrc location=<<rtsp URL>> latency=0 ! queue ! rtph264depay ! h264parse ! avdec_h264 ! videoconvert ! appsink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Progress: (open) Opening Stream
Progress: (connect) Connecting to <<rtsp URL>>
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info
Progress: (request) SETUP stream 0
Progress: (request) SETUP stream 1
Progress: (open) Opened Stream
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Progress: (request) Sending PLAY request
Progress: (request) Sending PLAY request
Progress: (request) Sent PLAY request
Redistribute latency...
Redistribute latency...
I'm not sure what to do next in terms of debugging the issue with GStreamer not working with OpenCV's VideoCapture
- any suggestions?
Florian Zwoch :
I don't think you are supposed to include the gst-launch-1.0 command line tool into the cv2 pipeline description.\nInstead of a console command it wants the sole GStreamer pipeline only. E.g.:\n gst_str = ('v4l2src device=/dev/video{} ! '\n 'video/x-raw, width=(int){}, height=(int){} ! '\n 'videoconvert ! appsink').format(dev, width, height)\n return cv2.VideoCapture(gst_str, cv2.CAP_GSTREAMER)\n\nSo in your case try:\nimport cv2\nvideo = cv2.VideoCapture('rtspsrc location=<<rtsp URL>> latency=0 ! queue ! rtph264depay ! h264parse ! avdec_h264 ! videoconvert ! appsink', cv2.CAP_GSTREAMER)\n",
2021-05-04T08:15:45