click on the blue word
Follow us to make development more interesting
The following article comes from Intel Internet of Things, authored by Intel Internet of Things Industry Innovation Ambassador Yang Xuefeng
This series of articles will use the OpenVINO? development kit on the AI Aix development board to deploy and evaluate YOLOv8’s
Classification models, object detection models, instance segmentation models, andhuman pose estimation models.
Next, this article will introduce the development environment for setting up the OpenVINO? reasoning program on the AIX development board, export and optimize the YOLOv8 classification model, complete the reasoning program of the YOLOv8 classification model, and finally give the measured performance.
Please download the code warehouse of this article first:
git clone https://gitee.com/ppov-nuc/yolov8_openvino.git
1
Introduction to YOLOv8
YOLOv8 is a SOTA model toolkit for object detection and tracking, instance segmentation, image classification and pose estimation tasks released by Ultralytics based on the YOLO framework.
Only a few lines of Python code or one line of commands are needed to complete training (Training a model from scratch) or fine-tune (Fine-tune) the YOLOv8 model on your own data set.
Train YOLOv8 model with Python code
Train the YOLOv8 model from the command line
It is also very convenient to export the trained YOLOv8 model and deploy it on the Intel hardware platform with OpenVINO?, which will be introduced in turn below.
2
Prepare YOLOv8’s
OpenVINO? Reasoning program development environment
Please complete the development environment installation with one line of commands based on the requirements.txt file provided in the sample code warehouse of this article.
# Usage: pip install -r requirements.txt Ultralytics # Base ---------------------------------------- matplotlib>=3.2.2 numpy>=1.21.6 opencv-python>=4.6.0 Pillow>=7.1.2 PyYAML>=5.3.1 requests>=2.23.0 scipy>=1.4.1 torch==1.13.1 torchvision==0.14.1 tqdm>=4.64.0 # Plotting ------------------------------------ pandas>=1.1.4 seaborn>=0.11.0 # Export -------------------------------------- onnx>=1.12.0 # ONNX export onnxruntime openvino-dev==2023.0.0.dev20230407 # modify the openvino-dev version to the latest one # Extras -------------------------------------- psutil # system utilization thop>=0.1.1 # FLOPs computation
Swipe right to view the full code
One line of command completes the development environment installation.
pip install -r requirements.txt
3
Export YOLOv8 OpenVINO?IR model
There are 5 classification models for YOLOv8, and the accuracy on the ImageNet dataset is shown in the table below.
First use the command:
yolo classify export model=yolov8n-cls.pt format=onnx imgsz=224
Complete the yolov8n-cls.onnx model export, as shown in the figure below.
Then use the command:
mo -m yolov8n-cls.onnx –compress_to_fp16,
Optimize and export the OpenVINO? IR format model with FP16 precision, as shown in the figure below.
4
Test the inference computing performance of YOLOv8 classification model with benchmark_app
benchmark_app is an AI model inference computing performance testing tool that comes with the OpenVINO tool suite. It can be specified on different computing devices in synchronous or asynchronous mode to test the pure AI model inference computing performance without pre- and post-processing.
Use the command: benchmark_app -m yolov8n-cls.xml -d GPU to obtain the asynchronous inference computing performance of the yolov8n-cls.xml model on the integrated graphics card of the AIX development board, as shown in the figure below.
5
Writing a YOLOv8 Classification Model Inference Program Using the OpenVINO? Python API
The core source code of the YOLOv8 classification model sample program yolov8_cls_ov_sync_infer.py based on OpenVINO? Python API is as follows:
# Instantiate the Core object
core = Core()
# load and compile the model
net = core.compile_model(f'{MODEL_NAME}-cls.xml', device_name="GPU")
# Get model input and output nodes
input_node = net.inputs[0] # yolov8n-cls has only one input node
N, C, H, W = input_node.shape # Get the shape of the input tensor
output_node = net.outputs[0] # yolov8n-cls has only one output node
ir = net.create_infer_request()
#############################################
# --- Define preprocessing and postprocessing functions according to the model -------
#############################################
# define preprocessing function
def preprocess(image, new_shape=(W,H)):
# Preprocess image data from OpenCV
image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB) # BGR->RGB
resized = cv2.resize(image, new_shape) # Resize Image
norm = (resized - IMAGENET_MEAN) / IMAGENET_STD # Normalization
blob = np.transpose(norm, (2, 0, 1)) # HWC->CHW
blob = np. expand_dims(blob, 0) # CHW->NCHW
return blob
# Define the post-processing function
def postprocess(outs):
score = np.max(outs)
id = np.argmax(outs)
return score, id, imagenet_labels[id]
#############################################
# ----- AI synchronous reasoning calculation ------------
#############################################
# capture image
image = cv2.imread("bus.jpg")
# Data preprocessing
blob = preprocess(image)
# Execute the inference calculation and get the result
outs = ir.infer(blob)[output_node]
# Post-processing the inference results
score, id, label = postprocess(outs)
# display processing results
msg = f"YOLOv5s-cls Result:{label}, Score:{score:4.2f}, FPS:{FPS:4.2f}"
cv2.putText(image, msg, (10, 50), cv2.FONT_HERSHEY_SIMPLEX, 0.8, (100, 100, 200), 2)
cv2.imshow("YOLOv5s-cls OpenVINO Sync Infer Demo",image)
cv2.waitKey()
cv2.destroyAllWindows()
Swipe right to view the full code
The running result is shown in the figure below:
6
Boost Inference Computing Performance Using OpenVINO? Preprocessing API
Refer to “Common Tips for Embedding Data Preprocessing into AI Models”, use OpenVINO? preprocessing API, and embed data preprocessing into YOLOv8 classification model, which can further improve the utilization rate of inference devices (for example, Intel? integrated graphics/discrete graphics cards), In turn, it improves the end-to-end AI inference computing performance including pre- and post-processing.
The core code of the example program export_yolov8_cls_ppp.py that uses the OpenVINO? preprocessing API to preprocess the embedded model is as follows:
# ======== Step 0: read original model =========
core = Core()
model = core.read_model(f"{MODEL_NAME}-cls.xml")
# Step 1: Add Preprocessing steps to a model ==
ppp = PrePostProcessor(model)
# Declare User’s Data Format
ppp.input().tensor() \
.set_element_type(Type.u8) \
.set_spatial_dynamic_shape() \
.set_layout(Layout('NHWC')) \
.set_color_format(ColorFormat.BGR)
# Declaring Model Layout
ppp.input().model().set_layout(Layout('NCHW'))
# Explicit preprocessing steps. Layout conversion will be done automatically as last step
ppp.input().preprocess() \
.convert_element_type() \
.convert_color(ColorFormat.RGB)\
.resize(ResizeAlgorithm.RESIZE_LINEAR) \
.mean([123.675, 116.28, 103.53]) \
.scale([58.624, 57.12, 57.375])
# Integrate preprocessing Steps into a Model
print(f'Dump preprocessor: {ppp}')
model_with_ppp = ppp. build()
# ========= Step 2: Save the model with preprocessor=================
serialize(model_with_ppp, f'{MODEL_NAME}-cls_ppp.xml', f'{MODEL_NAME}-cls_ppp.bin')
Swipe right to view the full code
The running result is shown in the figure below:
Inference program based on embedded preprocessing model
The core code of yolov8_cls_ppp_sync_infer.py is as follows
############################################
# ----- AI synchronous reasoning calculation ------------
#############################################
# capture image
image = cv2.imread("bus.jpg")
blob = np.expand_dims(image,0)
# Execute the inference calculation and get the result
outs = ir.infer(blob)[output_node]
# Post-processing the inference results
score, id, label = postprocess(outs)
#############################################
# ----- Statistical AI reasoning performance with pre- and post-processing ------
#############################################
start = time. time()
N = 1000
for i in range(N):
blob = np.expand_dims(image,0)
outs = ir.infer(blob)[output_node]
score, id, label = postprocess(outs)
FPS = N / (time. time() - start)
Swipe right to view the full code
The running result is as follows:
in conclusion
With the help of the integrated graphics card (24 execution units) of the N5105 processor and OpenVINO?, the AI Aix development board can achieve quite good performance on the classification model of YOLOv8.
Through asynchronous processing and AsyncInferQueue, the utilization rate of computing equipment can be further improved, and the throughput of AI reasoning programs can be improved. The next article will continue to introduce the “Accelerating YOLOv8 Object Detection Model with OpenVINO? on AI X Development Board”.
–END–
You may want to know (click the blue word to view) Draw with AI, wish her a happy holiday; three simple steps, OpenVINO? Help you easily experience AIGC Don't know how to draw with OpenVINO Click for tutorial. How to contribute to open source projects? | Developer holiday benefits A few lines of code can easily realize real-time reasoning for PaddleOCR, come get it! Use OpenVINO to quickly realize high-performance artificial intelligence reasoning in "device-edge-cloud" Extracting text from pictures is amazing? Try to achieve OCR in three steps! 【Notebook Series Issue 6】Based on the Pytorch pre-training model to achieve semantic segmentation tasks Use OpenVINO? preprocessing API to further improve YOLOv5 reasoning performance
Scan the QR code below to experience it immediately OpenVINO? Toolkit 2022.3
Click to read the original text and experience OpenVINO 2022.3 now
The article is so wonderful, are you “reading”?