Face related algorithms (Retinaface, yolov5face, yolov7face, Arcface, Face_Angle, Silent_Face_Anti_Spoofing, TensorRT+CUDA)

If you find it useful, you might as well give a Star to support it~ Thank you!

Project address

(github link)

FaceAlgorithm

Features

  1. Including face detection (Retinaface, yolov5face, yolov7face), face detection and tracking (ByteTracker), face angle calculation (Face_Angle), face correction (Face_Aligner), face recognition (Arcface), mask detection (MaskRecognitiion), age and gender detection (Face_Angle), silent living detection (Silent_Face_Anti_Spoofing);

  2. All models use C++ and TensorRT to accelerate reasoning, and the pre- and post-processing of yolov7face uses cuda to accelerate, (other model acceleration optimization can also be referred to);

  3. All models use C++ and OnnxRuntime.OpenVINO, NCNN accelerated reasoning (TO DO);

  4. The structure is similar to NV Deepstream, supporting multiple reasoning frameworks (TensorRT, OnnxRuntime, OpenVINO, NCNN), used for multi-channel RTSP pull stream + hard decoding + Pipeline + push stream (TO DO);

  5. Automatically generate the corresponding engine according to different graphics card models (if there are other graphics card adaptation engines under the folder, delete the engine to regenerate the engine corresponding to the graphics card in use);

  6. Provide C/C++ interface, which can be directly transplanted in the project;

  7. General process of face recognition:

    1) Face detection (image, video stream)

    2) According to the angle returned by each face, select a face with a suitable angle for face correction and face recognition

    3) Face correction (based on 5 face key points)

    4) Extraction of facial features (512-dimensional features)

    5) Face feature comparison (face similarity calculation)

  8. Conditional Compilation Test Description

    Test type enable Description
    face_detect 1 Face detection
    yolov5face_detect 1 yolov5face face detection
    yolov5face_detect 1 yolov7face face detection
    face_recognition 1 Face recognition (face feature extraction) + similarity calculation
    face_detect_tracker 1 face detection and tracking
    face_detect_aligner_recognitiion 0 Face detection – correction – Recognition (face feature extraction)
    mask_recognition 1 mask recognition
    gender_age_recognition 1 Gender and age identification
    silnet_face_anti_spoofing 1 Silent liveness detection

Algorithm Description

1. Face detection

  1. retinaface (mobilenet0.25, R50 need to modify the code by yourself)
  2. yolov5face (yolov5sface, n, m, l, x need to convert the corresponding onnx)
  3. yolov7face (yolov7sface, in addition, models of different sizes need to be converted by themselves)
  4. yolov8facee (TO DO))

2. Face recognition

  1. arcface (R50)
  2. arcface (R101, you need to download the model and modify the code yourself)

3. Mask identification

  1. classification model

4. Age and gender

  1. Age and gender recognition in InsightFace;

5. Silent living body recognition

  1. Silent-Face-Anti-Spoofing

6. Track

  1. ByteTracker (plus face bbox and face key points as input for tracking, bug modification)

7. Algorithm interface

/**
 * @brief face initialization function
 * @param config module configuration parameter structure
 * @return HZFLAG
 */
HZFLAG Initialize(Config & config);


/**
 * @brief face detection
 * @param img opencv Mat format
 * @param FaceDets face detection result list, including face bbox, confidence, five key point coordinates
 * @return HZFLAG
 */
HZFLAG Face_Detect(std::vector<cv::Mat> & amp;img, std::vector<std::vector<FaceDet>> & amp;FaceDets);


/**
 * @brief face detection (yolov5_face)
 * @param img opencv Mat format
 * @param FaceDets face detection result list, including face bbox, confidence, five key point coordinates
 * @return HZFLAG
 */
HZFLAG Yolov5Face_Detect(std::vector<cv::Mat> & amp;img, std::vector<std::vector<FaceDet>> & amp;FaceDets);

/**
   * @brief face detection (yolov7_face)
   * @param img opencv Mat format
   * @param FaceDets face detection result list, including face bbox, confidence, five key point coordinates
   * @return HZFLAG
   */
HZFLAG Yolov7Face_Detect(std::vector<cv::Mat> & amp;img, std::vector<std::vector<FaceDet>> & amp;FaceDets);

/**
 * @brief face detection and tracking (video stream)
 * @param img opencv Mat format
 * @param FaceDets FaceDets face detection result list, including face bbox, id, confidence, yaw angle, pitch angle, five key point coordinates
 * @return HZFLAG
 */
HZFLAG Face_Detect_Tracker(std::vector<cv::Mat> & amp;img, std::vector<std::vector<FaceDet>> & amp;FaceDets);


/**
 * @brief face correction
 * @param Faceimg The face image that needs to be corrected (the face image obtained by expanding the rectangular box bbox by 1.2 times and then corrected!!!!)
 * @param KeyPoints face key points
 * @param Face_Aligener Corrected image
 * @return HZFLAG
 */
HZFLAG Face_Aligner(cv::Mat & amp;Face_image,cv::Point2f *KeyPoints,cv::Mat & amp;Face_Aligner);

/**
 * @brief face feature extraction
 * @param Face_Aligener face image after face correction
 * @param Face_Feature face feature (512-dimensional feature)
 * @return HZFLAG
 */
HZFLAG Face_Feature_Extraction(cv::Mat & amp;Face_Aligener,Feature & amp;Face_Feature);


/**
 * @brief Calculate the similarity of face features
 * @param Feature1 face image after face correction
 * @param Feature2 face feature (512-dimensional feature)
 * @return float similarity score
 */
float Cal_Score(Feature & amp;Feature1,Feature & amp;Feature2);

/**
 * @brief face mask recognition
 * @param img The image of the face wearing a mask that needs to be recognized
 * @param Result The face mask recognition result
 * @return HZFLAG
 */
HZFLAG Mask_Recognition(cv::Mat & amp; img, float & amp; pred);

/**
 * @brief gender age recognition
 * @param img face image to be recognized
 * @param Result gender and age identification result
 * @return HZFLAG
 */
HZFLAG Gender_Age_Recognition(cv::Mat & amp;img,attribute & amp;gender_age);

/**
 * @brief silent liveness detection
 * @param img The face image to be detected
 * @param Result Silent liveness detection recognition result
 * @return HZFLAG
 */
HZFLAG Silent_Face_Anti_Spoofing(cv::Mat & amp;img, SilentFace & amp;silentface);

/**
 * @brief deinitialization
 * @return HZFLAG
 */
HZFLAG Release(Config & config);

How to use

1. Model download

(Baidu Drive code: 5xaa)

Model Function Description
FaceDetect.wts Face Detection
FaceRecognition.wts Face recognition
GenderAge.onnx Age gender recognition
MaskRecognition.onnx Mask recognition
yolov5s-face_bs =1.onnx yolov5s face detection
yolov5s-face_bs=4.onnx yolov5s face detection
yolov7s-face_bs=1.onnx yolov7s face detection
yolov7s-face_bs=4.onnx yolov7s people Face Detection
2.7_80x80_MiniFASNetV2.onnx Silent Liveness Detection

2. Environment

  1. ubuntu20.04 + cuda11.1 + cudnn8.2.1 + TrnsorRT8.2.5.1 (test passed)
  2. ubuntu18.04 + cuda10.2 + cudnn8.2.1 + TrnsorRT8.2.5.1 (test passed)
  3. Win10 + cuda11.1 + cudnn8.2.1 + TrnsorRT8.2.5.1 (test passed)
  4. In other environments, please try by yourself or join the group to understand

3. Compile

  1. Change the CMakeLists.txt in the root directory to set the tensorrt installation directory
set(TensorRT_INCLUDE "/xxx/xxx/TensorRT-8.2.5.1/include" CACHE INTERNAL "TensorRT Library include location")
set(TensorRT_LIB "/xxx/xxx/TensorRT-8.2.5.1/lib" CACHE INTERNAL "TensorRT Library lib location")

  1. By default opencv is installed, cuda, cudnn are installed

  2. Compile the -g O0 version by default for Debug, if you want to speed up, please compile the Release version

  3. Compile with Visual Studio Code shortcut keys (choose one of 4 and 5):

 ctrl + shift + B
  1. Compile using the command line (choose one of 4 and 5):
 mkdir build
   cd build
   cmake..
   make -j6

References

  1. https://github.com/deepcam-cn/yolov5-face
  2. https://github.com/wang-xinyu/tensorrtx
  3. https://github.com/minivision-ai/Silent-Face-Anti-Spoofing
  4. https://github.com/linghu8812/tensorrt_inference
  5. https://github.com/derronqi/yolov7-face/tree/main
  6. https://github.com/we0091234/yolov7-face-tensorrt

Acknowledgments & Contact

1. WeChat ID: cbp931126

Add me on WeChat (remark: FaceAlgorithm), and pull you into the group

2.QQ Group: 517671804