Use Docker to deploy CPU + GPU
- 1.CPU
- 2.GPU
- 3.cuDNN installation
-
- 3.1 Prerequisites
- 3.2 Download Linux version cuDNN
- 3.3 Installation
1.CPU
This description is based on DeepFace’s Docker image file deepface_image.tar.
# 1. Import image docker load -i deepface_image.tar # 2. Create a model folder [and upload the downloaded model file] mkdir -p /root/.deepface/weights/ # 3. Start the container # Network isolation is affected but performance is good docker run --name deepface --privileged=true --restart=always --net="host" -v /root/.deepface/weights/:/root/.deepface/weights/ -d deepface_image # General use docker run --name deepface --privileged=true --restart=always -p 5000:5000 -v /root/.deepface/weights/:/root/.deepface/weights/ -d deepface_image # Use the latest code for container startup docker run --name deepface_src --privileged=true --restart=always --net="host" \ -v /root/.deepface/weights/:/root/.deepface/weights/ \ -v /opt/test-facesearch/deepfacesrc/:/app/deepface/ \ -d deepface_image
Warning message:
# Execute command docker run --name deepface --privileged=true --restart=always --net="host" -p 5000:5000 -v /root/.deepface/weights/:/root/.deepface/weights/ - d deepface_image # warn WARNING: Published ports are discarded when using host network mode
This warning usually occurs when using Docker’s host network mode, because in this mode, the container and the host share the same network namespace, so the ports in the container will be mapped directly to the host without the need for port forwarding. Therefore, using the -p option to publish a container port has no effect and will result in a warning message. To resolve this issue, you can try the following:
- If you don’t need to map the container port to the host, remove the -p option.
- If you need to map container ports to the host, use Docker’s other networking modes (such as bridge mode).
- If you do need to use host networking mode, consider using the host IP address to access services in the container rather than using port forwarding.
2.GPU
First, start the container installation tensorrt:
pip install tensorrt -i https://pypi.tuna.tsinghua.edu.cn/simple
Startup command after installation:
docker run --name deepface --privileged=true --restart=always --net="host" \ -e PATH=/usr/local/cuda-11.2/bin:$PATH -e LD_LIBRARY_PATH=/usr/local/cuda-11.2/lib64:$LD_LIBRARY_PATH \ -v /root/.deepface/weights/:/root/.deepface/weights/ \ -v /usr/local/cuda-11.2/:/usr/local/cuda-11.2/ \ -v /opt/xinan-facesearch-service-public/deepface/api/app.py:/app/app.py \ -d deepface_image
Test fastmtcnn to mount the latest code into the directory:
docker run --name deepface_gpu_src --privileged=true --restart=always --net="host" \ -e PATH=/usr/local/cuda-11.2/bin:$PATH -e LD_LIBRARY_PATH=/usr/local/cuda-11.2/lib64:$LD_LIBRARY_PATH \ -v /root/.deepface/weights/:/root/.deepface/weights/ \ -v /usr/local/cuda-11.2/:/usr/local/cuda-11.2/ \ -v /opt/test-facesearch/deepfacesrc/:/app/deepface/ \ -v /opt/xinan-facesearch-service-public/deepface/api/app.py:/app/app.py \ -d deepface_image
Differences from CPU deployment:
- Two environment variables are set
-e PATH=/usr/local/cuda-11.2/bin:$PATH -e LD_LIBRARY_PATH=/usr/local/cuda-11.2/lib64:$LD_LIBRARY_PATH - Added a mounting directory
-v /usr/local/cuda-11.2/:/usr/local/cuda-11.2/ - Added a mounting file
-v /deepface/api/app.py:/app/app.py
The content of the file /deepface/api/app.py is as follows:
import tensorrt as tr
import tensorflow astf
from flask import Flask
from routes import blueprint
def create_app():
available = tf.config.list_physical_devices('GPU')
print(f"available：{<!-- -->available}")
app = Flask(__name__)
app.register_blueprint(blueprint)
return app
Before calling tensorflow, you need to introduce tensorrt.
3.cuDNN installation
Official website installation documentation: https://docs.nvidia.com/deeplearning/cudnn/install-guide/index.html
cuDNN support matrix: https://docs.nvidia.com/deeplearning/cudnn/support-matrix/index.html
The NVIDIA CUDA? Deep Neural Network library (cuDNN) is a GPU-accelerated library of primitives for deep neural networks. cuDNN provides highly tuned implementations for standard routines such as forward and backward convolution, attention, matmul, pooling, and normalization.
Installation Environment:
[root@localhost ~]# cat /etc/centos-release CentOS Linux release 7.7.1908 (Core)
3.1 Prerequisites
You need to install 1.GPU Driver and 2.CUDAToolkit first
nvidia-smi # search result +------------------------------------------------- ---------------------------- + | NVIDIA-SMI 460.27.04 Driver Version: 460.27.04 CUDA Version: 11.2 | |---------------------------------- + ----------------- ----- + ---------------------- +
and 3.zlib
yum list installed | grep zlib # search result zlib.x86_64 1.2.7-18.el7 @anaconda zlib-devel.x86_64 1.2.7-18.el7 @base
3.2 Download Linux version cuDNN
To download cuDNN, you need to first register for the NVIDIA Developer Program: https://developer.nvidia.com/developer-program. Download page: https://developer.nvidia.com/cudnn. Select the platform and corresponding version to download. This time The downloaded file is cudnn-11.2-linux-x64-v8.1.1.33.tgz and the size is 1.2G. Browser downloads tend to fail. You can copy the browser download link and download on the Linux server [Tencent Cloud Server Speed 12MB/s]:
wget https://developer.download.nvidia.cn/compute/machine-learning/cudnn/secure/8.1.1.33/11.2_20210301/cudnn-11.2-linux-x64-v8.1.1.33.tgz?G2wTHq8E --2jJ9iEfgtFbqfMGX0I1XD6BIksPkVIiU9F3ttrupv_oYvURaZX1dV71EIqEI767WbG5svvSMBElcaVrqZl15UEOUORNWbYwKZDyxidGmwHmG44XiEo6yyM1Rt7ct6NGlVXnxx0etcI9pNJ1 PiaHYddY86Lc_yaBLdJwy9hqku4TW6NSNr7XfuCYXvGOPvOmraR4EOfg6Q= & amp;t=eyJscyI6IndlYnNpdGUiLCJsc2QiOiJkZXZlbG9wZXIubnZpZGlhLmNvbS9jdWRhLTEwLjItZG93bmxvYWQt YXJjaGl2ZT90YXJnZXRfb3M9TGludXgifQ==
3.3 Installation
The following steps describe how to build a cuDNN dependent program. Choose the installation method that meets your environment needs. For example, the tar file installation applies to all Linux platforms. The Debian package installation applies to Debian 11, Ubuntu 18.04, Ubuntu 20.04, and 22.04. The RPM package installation applies to RHEL7, RHEL8, and RHEL9. In the following sections:
- your CUDA directory path is referred to as /usr/local/cuda/
- your cuDNN download path is referred to as
You can choose the appropriate installation method according to different platforms. The tar file is suitable for all Linux platforms. The installation steps are:
- Unzip the installation package
tar -xvf cudnn-linux-$arch-8.x.x.x_cudaX.Y-archive.tar.xz
- Copy the following files into the CUDA toolkit directory
$ sudo cp cudnn-*-archive/include/cudnn*.h /usr/local/cuda/include $ sudo cp -P cudnn-*-archive/lib/libcudnn* /usr/local/cuda/lib64 $ sudo chmod a + r /usr/local/cuda/include/cudnn*.h /usr/local/cuda/lib64/libcudnn*
The installation file is cudnn-11.2-linux-x64-v8.1.1.33.tgz. The actual steps are:
# 1. Unzip tar -zxvf cudnn-11.2-linux-x64-v8.1.1.33.tgz # 2. Copy and empower #The unzipped folder name is cuda # inculde【18 files】 cp ./cuda/include/cudnn*.h /usr/local/cuda/include # lib64 [8 files 15 soft links] -P option means to retain the attributes of the source file or directory cp -P ./cuda/lib64/libcudnn* /usr/local/cuda/lib64 # Grant read permission to all users chmod a + r /usr/local/cuda/include/cudnn*.h /usr/local/cuda/lib64/libcudnn*
The installation file of another version is cudnn-linux-x86_64-8.6.0.163_cuda11-archive.tar.xz. The steps are:
# 1. Unzip tar -xvf cudnn-linux-x86_64-8.6.0.163_cuda11-archive.tar.xz # 2. Copy and empower inculde【18 files】lib【13 files 20 soft links】 cp ./cudnn-linux-x86_64-8.6.0.163_cuda11-archive/include/cudnn*.h /usr/local/cuda/include cp -P ./cudnn-linux-x86_64-8.6.0.163_cuda11-archive/lib/libcudnn* /usr/local/cuda/lib64 chmod a + r /usr/local/cuda/include/cudnn*.h /usr/local/cuda/lib64/libcudnn*