Work subtotal-GPU coding and dependent libraries
This is the third record about encoding and decoding. GPU encoding has been used in the project for a long time. Because yuv is too large, encoding is very important. The problem encountered this time was the construction of the environment. It is necessary to put the environment on the development machine into docker to ensure that GPU encoding can also be performed in docker.
1 Positioning problem
Docker is provided by the algorithm department, and naturally comes with cuda, gpu driver and other environments. But when the code calls the decoder, the corresponding hard decoder is not found.
To locate the problem, first determine whether the encoder is really not supported.
Check whether the library supports GPU
strings libavcodec.so | grep -i cuda
I saw a lot of cuda output, and the most important thing is this compilation option. You can see that the corresponding cuda, nvenc, and cuvid are all enabled.
--prefix=/opt/ffmpeg --enable-shared --enable-ffplay --enable-cuda --enable-cuvid --enable-nvenc --enable-nonfree --enable-libnpp --enable- gpl --extra-cflags='-I/usr/local/cuda/include /usr/local/cuda-11.1/targets/x86_64-linux/include' --extra-ldflags='-L/usr/ local/cuda/lib64 -L/usr/local/cuda-11.1/targets/x86_64-linux/lib' --disable-x86asm --extra-cflags=-fPIC --extra-cxxflags=-fPIC --enable- libmfx --enable-nonfree --enable-encoder=h264_qsv --enable-decoder=h264_qsv --enable-encoder=hevc_qsv --enable-decoder=hevc_qsv --prefix=/opt/ffmpeg --libdir=/opt/ffmpeg /lib --extra-cflags=-I/opt/intel/mediasdk/include --extra-ldflags=-L/opt/intel/mediasdk/lib64
Check whether hardware decoding is supported at runtime
I have a ready-made h265 file on hand.
# Get yuv file ffmpeg -i input.h265 -c:v rawvideo -pix_fmt yuv420p output.yuv # Get MP4 files ffmpeg -i input.h265 -c:v libx264 -crf 23 -c:a aac -strict experimental output.mp4 # Perform h264/hevc(h265) hardware encoding on yuv ffmpeg -f rawvideo -pix_fmt yuv420p -s 3840x2160 -r 30 -i output.yuv -c:v hevc_nvenc output.mp4 ffmpeg -f rawvideo -pix_fmt yuv420p -s 3840x2160 -r 30 -i output.yuv -c:v h264_nvenc output.mp4
Sure enough, the hard-coded error was reported.
It can be seen that the hard coding of h264 and h265 both reported errors.
[h264_nvenc @ 0x258a880] Cannot load libnvidia-encode.so.1 [hevc_nvenc @ 0x258a880] The minimum required Nvidia driver for nvenc is (unknown) or newer
2 Solve the problem
Directly search the host environment
i386-linux-gnu is for 32-bit environment and is ignored directly. Go to the corresponding folder to find the library corresponding to nvidia
Compare with docker
Sure enough, I checked a lot, because the environment in docker is a deep learning environment, and it will definitely be different from our hardware coding library.
I copied it and still got an error. Finally located the cuvid problem. The trick is that when ffmpeg lacks hardware encoding and cuvid, it will also report the problem of missing libnvidia-encode.so.
You can see that the two libraries nvenc and cuvid respectively correspond to the hardware encoding and decoding. Previous projects used more hard-coded solutions, but here there are more hard-coded ones.
libnvcuvid.sois a library file in the NVIDIA Video Codec SDK, which provides functions for decoding and processing videos. It allows applications to use NVIDIA GPUs to accelerate video decoding, thereby improving video processing performance.
libnvidia-encode.sois another library file in the NVIDIA Video Codec SDK that provides functions for encoding and processing videos. It allows applications to use NVIDIA GPUs to accelerate video encoding, thereby improving video processing performance.
Both library files are tools provided by NVIDIA for video processing and can be used on systems that support NVIDIA GPUs. They provide developers with interfaces and capabilities for video decoding and encoding using GPUs for more efficient video processing and acceleration.
At this point, the problem is solved.
3 docker related
Additional records of some docker-related understandings.
If docker wants to call the GPU, it must communicate with the GPU in the host machine (a superficial understanding can be that nvidia-container-toolkit is installed separately) to complete a remote call/transit call. This call was previously done by nvidia-docker. Higher versions of docker integrate nvidia-docker, so just enter the –gpu parameter. As long as nvidia-smi in the container is normal, it will be almost the same, because there is an interaction. However, the driver refers to the instructions for calling the GPU, *.so, etc., which still need to be installed in docker. Otherwise, there will be no way to communicate with the host and no way to be called by upper-layer applications.
nvidia-container-cli --version # Check whether the corresponding version is installed
Using GPUs in containers usually requires installing the NVIDIA Container Toolkit on both the host and the container. Installing the NVIDIA Container Toolkit in the host is used to manage the GPU resources on the host, while installing the NVIDIA Container Toolkit in the container is used to access these GPU resources within the container.
Host
https://github.com/NVIDIA/k8s-device-plugin#preparing-your-gpu-nodes
distribution=$(./etc/os-release;echo $ID$VERSION_ID) curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add - curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list sudo apt-get update & amp; & amp; sudo apt-get install -y nvidia-docker2
container
# Build from a base image with NVIDIA driver FROM nvidia/cuda:11.0-base #Install NVIDIA Container Toolkit related software packages RUN apt-get update & amp; & amp; apt-get install -y nvidia-container-toolkit # Set NVIDIA runtime environment variables ENV NVIDIA_VISIBLE_DEVICES all ENV NVIDIA_DRIVER_CAPABILITIES compute,utility # Verify that NVIDIA GPU configuration is correct RUN nvidia-smi # Run your application or service CMD ["/your/app/command"]
4 gpu driver related
There are still problems after going to the cloud. This error is because so is a backward compatible driver.
The project eventually involved upgrading the GPU driver locally, because the driver of the GPU cloud environment was too high.
Upgrade the GPU driver, easy to understand. The upgrades are cuda*.so and libnvidia*.so (as well as the *.ko files related to GPU software). Intuitively, it seems that the driver version of nvidia-smi has become higher.
The corresponding official website instructions are made into instructions.
https://developer.nvidia.com/cuda-downloads?target_os=Linux & amp;target_arch=x86_64 & amp;Distribution=Ubuntu & amp;target_version=22.04
To uninstall the driver, you can use apt-get purge. After the upgrade is completed.
sudo updatedb //Update the cache of the database first and then query locate libnvidia-encode.so
You can also see the upgrade from 86 to 98
The specific functions of these libraries are as follows, and they will be upgraded together.
libnvidia-encode.so: Related to NVENC, provides hardware-accelerated video encoding functionality. libnvidia-glcore.so: related to OpenGL, providing 3D rendering functions. libnvidia-eglcore.so: Related to EGL, providing an interface for rendering windows and images. libnvidia-glsi.so: Related to OpenGL scene graph. libnvidia-nvvm.so: Related to NVIDIA's virtual machine, used for CUDA. libnvidia-ml.so: Provides query function for GPU hardware status, such as temperature, voltage, etc. libnvidia-cfg.so: Provides functions related to GPU configuration. libnvidia-opticalflow.so: Provides optical flow calculation functions, commonly used in video analysis and computer vision. libnvidia-rtcore.so: related to ray tracing. libnvidia-gtk2/3.so: Provides integration with GTK2 and GTK3 interface toolkits. libnvidia-container.so: Related to the NVIDIA Container Toolkit, supporting the use of GPUs in Docker containers.
In addition, we can know that as long as there is a corresponding nvso library in docker, even if the host machine does not have a corresponding so library, as long as there is a GPU driver, the program can be successfully run, because so interacts with the driver, and nv-docker is < strong>Let the so in the container interact with the host’s gpu driver.
libcuda.so and libnvida.so
Within NVIDIA’s software stack, libcuda.soXXX (also known as the CUDA driver API) and libnvidia-encode.soXXX (NVENC, for hardware-accelerated video encoding) are lower-level libraries. Comparing their “levels” can be described as follows:
libcuda.soXXX (CUDA driver API): This is the basis of the CUDA architecture. It allows applications to interact directly with NVIDIA GPUs. CUDA-based applications and libraries, such as cuDNN, cuBLAS, etc., rely on this library to interact with the GPU.
libnvidia-encode.soXXX (NVENC): This is a library dedicated to video encoding. It provides hardware-accelerated video encoding capabilities and interacts directly with the NVIDIA GPU’s video encoding hardware.
Comparison between the two
libcuda is more “basic” or “low-level” because it is the cornerstone of the entire CUDA architecture. And libnvidia-encode is a specialized library for specific functions (video encoding).
libnvidia-encode.so is NVIDIA’s encoding (NVENC) library, which provides hardware-accelerated video encoding capabilities. libcuda.so is the main library of the CUDA runtime, which allows applications to access the parallel computing capabilities of NVIDIA GPUs.
Although both libraries are related to the functionality of NVIDIA GPUs, they serve different purposes. libnvidia-encode.so is mainly used for video encoding, while libcuda.so is used for parallel computing.
However, from an implementation perspective, libnvidia-encode.so may internally use libcuda.so or other GPU driver-related libraries, which can interact with the GPU more directly. Whether libnvidia-encode.so directly calls libcuda.so depends on NVIDIA’s internal implementation. Usually these details are not public.