JETSON face detection–Ultra-Light-Fast-Generic-Face-Detector-1MB
Article directory
- JETSON Face Detection–Ultra-Light-Fast-Generic-Face-Detector-1MB
-
- Ultra-Light-Fast-Generic-Face-Detector-1MB (hereinafter referred to as ultraface) introduction
- Source code
- Environment setup and related C++ deployment
-
-
- Download and compile NCNN
- Compile Ultra-Light-Fast-Generic-Face-Detector-1MB
- Modify main.cpp for video reading and face detection
- Obtaining data such as CPU resource usage
-
Ultra-Light-Fast-Generic-Face-Detector-1MB (hereinafter referred to as ultraface) introduction
You can refer to this blog: Ultra-Light-Fast-Generic-Face-Detector-1MB for face detection
It contains relevant principles, data comparison and network architecture.
Source code
Refer to this link: https://github.com/Linzaer/Ultra-Light-Fast-Generic-Face-Detector-1MB
This is the official code of Ultra-Light-Fast-Generic-Face-Detector-1MB.
Environment construction and related C++ deployment
Because this time the device JETSON-ORIN-NX performs face detection mainly to integrate with subsequent hardware.
Therefore, the C + + language is mainly used. Therefore, relevant C++ deployment of the source code is required.
** ultraface** can be found in the README.md file of the Github source code:
- Provide NCNN C++ inference code
- Provide MNN C++ inference code, M
- NN Python inference code
- FP32/INT8 quantized model
This article tried to use MNN for C++ reasoning, but there were too many errors and it was not continued, so it is not shown here. However, interested students can refer to the following links to try it themselves:
- C++ implementation of Ultra-Light-Fast-Generic-Face-Detector-1MB with MNN
- Install MNN deep learning framework on Jetson Nano
- Compile MNN and deploy mnist in Ubuntu20.04 environment
- Compile and install MNN on linux
However, this article attempts to use NCNN for C++ deployment and it is successful. Therefore, the following will show the process of NCNN deploying C++ and modifying the source code to detect videos and then obtain the results.
Download and compile NCNN
When downloading the ultraface source code, you can find it in this path: …/Ultra-Light-Fast-Generic-Face-Detector-1MB/ncnn/3rdparty/ncnn folder The bottom is empty, there is nothing. This is because the source code of NCNN needs to be downloaded to this folder.
Before downloading the source code and compiling, you need to download some things required for environment configuration. For example: opencv4, g++, cmake, protobuf
For detailed installation steps, please refer to this blog: Install ncnn in ubuntu environment
After the preliminary preparation and installation are completed, NCNN can be downloaded and configured.
NCNN Source code link: ncnn
git clone https://github.com/Tencent/ncnn
After downloading the source code package, copy the files in the source code to the folder in the path above.
Then just compile:
cd ./Ultra-Light-Fast-Generic-Face-Detector-1MB/ncnn/3rdparty/ncnn mkdir build & amp; & amp; cd build cmake.. make -j4
This compilation process is quite simple. Much simpler than that MNN.
Compile Ultra-Light-Fast-Generic-Face-Detector-1MB
Because NCNN has been built under the path: …/Ultra-Light-Fast-Generic-Face-Detector-1MB/ncnn/3rdparty/ncnn, and then it starts To compile ultraface:
// The official website of the compilation process is: https://github.com/Linzaer/Ultra-Light-Fast-Generic-Face-Detector-1MB/tree/master/ncnn cd Ultra-Light-Fast-Generic-Face-Detector-1MB/ncnn mkdir build & amp; & amp; cd build & amp; & amp; cmake .. make -j$(nproc)
After compilation is complete, you can run the test file:
./main ../data/version-RFB/RFB-320.bin ../data/version-RFB/RFB-320.param ../data/test.jpg
Among them, version-slim-320 and version-RFB-320 under the data folder are the officially converted NCNN models. Just use it directly.
The part after the official website link is about how to convert the pre-trained model into a NCNN model. If you need to train the network yourself, you can refer to it.
Modify main.cpp for video reading and face detection
main.cpp before modification:
// UltraFaceTest
#include "UltraFace.hpp"
#include <iostream>
#include <opencv2/opencv.hpp>
int main(int argc, char **argv) {<!-- -->
if (argc <= 3) {<!-- -->
fprintf(stderr, "Usage: %s <ncnn bin> <ncnn param> [image files...]\
", argv[0]);
return 1;
}
std::string bin_path = argv[1];
std::string param_path = argv[2];
UltraFace ultraface(bin_path, param_path, 320, 240, 1, 0.7); // config model input
for (int i = 3; i < argc; i + + ) {<!-- -->
std::string image_file = argv[i];
std::cout << "Processing " << image_file << std::endl;
cv::Mat frame = cv::imread(image_file);
ncnn::Mat inmat = ncnn::Mat::from_pixels(frame.data, ncnn::Mat::PIXEL_BGR2RGB, frame.cols, frame.rows);
std::vector<FaceInfo> face_info;
ultraface.detect(inmat, face_info);
for (int i = 0; i < face_info.size(); i + + ) {<!-- -->
auto face = face_info[i];
cv::Point pt1(face.x1, face.y1);
cv::Point pt2(face.x2, face.y2);
cv::rectangle(frame, pt1, pt2, cv::Scalar(0, 255, 0), 2);
}
cv::imshow("UltraFace", frame);
cv::waitKey();
cv::imwrite("result.jpg", frame);
}
return 0;
}
Modified main.cpp:
// UltraFaceTest
#include <chrono>
#include "UltraFace.hpp"
#include <iostream>
#include <opencv2/opencv.hpp>
#include <opencv2/core.hpp>
#include <opencv2/imgproc.hpp>
#include <opencv2/highgui.hpp>
int main(int argc, char **argv) {<!-- -->
if (argc <= 3) {<!-- -->
fprintf(stderr, "Usage: %s <ncnn bin> <ncnn param> [image files...]\
", argv[0]);
return 1;
}
\t\t
std::string bin_path = argv[1];
std::string param_path = argv[2];
std::string path = argv[3];
UltraFace ultraface(bin_path, param_path, 320, 240, 1, 0.7); // config model input
cv::VideoCapture cap(path);
cv::Mat image;
auto start1 = std::chrono::steady_clock::now(); // end1-start1 represents the total time of code running, end-start represents the time of single model derivation
// cap.isOpened(): Check whether the video is opened successfully
if (!cap.isOpened()) {<!-- -->
std::cout << "Cannot open video file" << std::endl;
return -1;
}
double desired_fps = 20.0; // Set the desired frame rate
double delay_pre = 1000.0 / desired_fps; // Calculate delay time
// cap.read(): Read the next frame of video and store it in the frame variable
while(cap.read(image)){<!-- -->
ncnn::Mat inputBlob = ncnn::Mat::from_pixels(image.data, ncnn::Mat::PIXEL_BGR2RGB, image.cols, image.rows);
std::vector<FaceInfo> face_info;
auto start = std::chrono::steady_clock::now();
ultraface.detect(inputBlob, face_info);
for (int i = 0; i < face_info.size(); i + + ) {<!-- -->
auto face = face_info[i];
cv::Point pt1(face.x1, face.y1);
cv::Point pt2(face.x2, face.y2);
cv::rectangle(image, pt1, pt2, cv::Scalar(0, 255, 0), 2);
}
auto end = std::chrono::steady_clock::now();
std::chrono::duration<double> elapsedSeconds = end - start;
double inferenceTime = elapsedSeconds.count();
std::cout << "Inference Time: " << inferenceTime << " seconds" << std::endl; // Output the time of a single model inference
cv::imshow("UltraFace", image);
double delay = delay_pre-inferenceTime*1000-5;
if (delay >=10 ){<!-- -->
cv::waitKey(delay);
}else {<!-- -->
cv::waitKey(10);
}
}
auto end1 = std::chrono::steady_clock::now();
std::chrono::duration<double> elapsedSeconds1 = end1 - start1;
double inferenceTime1 = elapsedSeconds1.count();
std::cout << "Inference Time: " << inferenceTime1 << " seconds" << std::endl; // Output the overall running time
cap.release(); // Release the video capture object
cv::destroyAllWindows(); // Close all windows
return 0;
}
Then enter in the terminal
cd Ultra-Light-Fast-Generic-Face-Detector-1MB/ncnn/build make -j4
After compiling the modified main.cpp, you can download the face detection video and perform detection:
Recommended video download website: Visual China’s website (https://www.vcg.com/creative-video/)
After downloading the video, copy it to the data folder of ncnn and rename it to test.mp4.
Run the following test command to test:
./main ../data/version-RFB/RFB-320.bin ../data/version-RFB/RFB-320.param ../data/test.mp4
The test results are shown in the figure:
Acquisition of data such as CPU resource usage
At this point, enter top in the terminal:
You can get data on the resources occupied during the running of the code.
Among them, the current face detection code consumes the most CPU.
If you want to reduce CPU usage, you can use the version-slim-320 model.
The command is:
./main ../data/version-slim/slim_320.bin ../data/version-slim/slim_320.param ../data/test.mp4