Opening new doors for .NET: OpenVINO.NET open source project newly released

Preface

Today, as AI applications become more and more widespread, optimizing deep learning models and deploying them for inference has become a necessary technology.

The OpenVINO toolkit (Open Visual Inference and Neural network Optimization) developed by Intel is such a powerful tool.

As an open source toolkit, OpenVINO provides developers with powerful deep learning model optimization and inference capabilities, supporting deployment across different Intel hardware platforms, including CPUs, integrated GPUs, Intel Movidius VPUs, and FPGAs.

The original intention of this toolkit is to implement a machine learning inference solution that can be deployed anywhere after coding. However, in the process of dealing with deep learning model inference, I gradually discovered that the PaddleSharp project I originally developed based on Baidu Paddlepaddle has better CPU inference OCR performance for the same model, and OpenVINO has better performance. So I started paying attention to OpenVINO and found that its C API did not have a suitable and high-quality package for the .NET world.

Some packages on the market may only meet the needs of specific projects and have insufficient functions; some may have perfect functions but the naming convention may not comply with the .NET community standards; some may have problems with error handling and performance, or may not be cross-platform. This contradicts the cross-platform nature of OpenVINO.

The .NET world desperately needs a higher quality OpenVINO package, and I feel like I have the ability to work on it. Therefore, I set a flag before this year’s holiday – I worked hard during the National Day and started the open source journey of OpenVINO.NET.

Address: https://github.com/sdcb/openvino.net

407bc2fd87f63a1667ee21e09975f9c1.png

Package name Version number Introduction
Sdcb.OpenVINO outside_default.png .NET PInvoke
Sdcb.OpenVINO.runtime.centos.7-x64 outside_default.png CentOS 7 x64
Sdcb.OpenVINO.runtime.debian.9-arm outside_default.png Debian 9 ARM
Sdcb.OpenVINO.runtime.debian.9-arm64 outside_default.png Debian 9 ARM64
Sdcb.OpenVINO.runtime. rhel.8-x64 outside_default.png RHEL 8 x64
Sdcb.OpenVINO.runtime.ubuntu.18.04-x64 outside_default.png Ubuntu 18.04 x64
Sdcb.OpenVINO.runtime.ubuntu.20.04-x64 outside_default.png Ubuntu 20.04 x64
Sdcb.OpenVINO.runtime.ubuntu.22.04-x64 outside_default.png Ubuntu 22.04 x64
Sdcb.OpenVINO.runtime.win-x64 outside_default.png Windows x64

46ec8f4545c2b1d846c2636ba4dce8d9.png

  • Q: Why does OpenVINO.NET not directly reference OpenCvSharp4?

  • A: I personally like OpenCvSharp4. The open source protocol is very friendly. However, OpenCvSharp4 does not officially support many platforms, and some people may prefer Emgu.CV. or ImageSharp, try not to kidnap it.

  • Q: The C API has 158 function interfaces, 26 interface bodies, and detailed XML comments. How did you achieve it with high quality in a short time?

  • A: I generated it automatically, I used the CppSharp project, CppSharp converts the header file contents of the C API into abstract syntax trees (AST), and then I converted these ASTs into well-annotated XML C# code. In fact, this is not the first time that I have applied CppSharp to an open source project. Interested friends can check out my Github Sdcb.OpenVINO.AutoGen project for implementation details.

For friends who want to know how to use it, I also wrote inference examples based on detection and classification of yolov8, official face detection examples of OpenVINO and my native design and Migrated PaddleOCR project. In addition, I would also like to talk about the design ideas and future development direction of the project.

four examples

Face detection – based on the face-detection-0200 model provided by OpenVINO official website

In this example, I use the face-detection-0200 model provided by the OpenVINO official website. The official website provides an introduction page: https://docs.openvino.ai/2023.1/omz_models_model_face_detection_0200.html. The detailed sample code can be found from the mini-openvino-facedetection Github repository I created. When running, it will locate the face position in the camera and frame it. The rendering is as follows: 24e02316c6d2b592e619d4ec64dbf3bd.png

df727adb3764cd856c9641009cdbe0f6.png

9656dda543bf2027968e264aba6b3e2a.jpeg

As shown in the picture, three objects were detected, the person, mobile phone and water cup in the picture, and the total time took about 30ms.

Object classification – classification model based on yolov8

The yolov8 model provides 1,000 different predefined categories. Like the above model, it needs to be downloaded and converted from the yolov8 official website. Friends who just want to try it out quickly can directly open another Github example I wrote: sdcb-openvino-yolov8-cls When running, the code will read an image and then try to guess which of the 1000 categories the image most resembles. In my code example, the input image is hen.jpg:

36e479472bcf71fc41c85a3eddc09c67.jpeg

The output is as follows:

class id=hen, score=0.59
Preprocess time: 0.00ms
infer time: 1.65ms
Post process time: 0.49ms
Total time: 2.14ms

46316f27e5dacb570deb4ccbdf202978.png

using OpenCvSharp;
using Sdcb.OpenVINO.PaddleOCR.Models.Online;
using Sdcb.OpenVINO.PaddleOCR.Models;
using Sdcb.OpenVINO.PaddleOCR;
using System.Diagnostics;
using System;

FullOcrModel model = await OnlineFullModels.ChineseV3.DownloadAsync();

using Mat src = Cv2.ImDecode(await new HttpClient().GetByteArrayAsync("https://io.starworks.cc:88/paddlesharp/ocr/samples/xdr5450.webp"), ImreadModes.Color);

using (PaddleOcrAll all = new(model)
{
    AllowRotateDetection = true,
    Enable180Classification = true,
})
{
    // Load local file by following code:
    // using (Mat src2 = Cv2.ImRead(@"C:\test.jpg"))
    Stopwatch sw = Stopwatch.StartNew();
    PaddleOcrResult result = all.Run(src);
    Console.WriteLine($"elapsed={sw.ElapsedMilliseconds} ms");
    Console.WriteLine("Detected all texts: \\
" + result.Text);
    foreach (PaddleOcrResultRegion region in result.Regions)
    {
        Console.WriteLine($"Text: {region.Text}, Score: {region.Score}, RectCenter: {region.Rect.Center}, RectSize: {region.Rect.Size}, Angle: {region.Rect. Angle}");
    }
}

The running effect is as follows:

elapsed=246 ms
Detected all texts:
High-speed 4X4160MHz data stream
Double the number of streams in the 5GHz band
twice as fast 3
AX5400 wireless specification router,
The 5GHz band uses high-speed 4X4160MHz data stream,
Compared with the mainstream AX3000 router on the market (2X2 data stream)
The 5GHz band has twice the number of streams and twice the speed.
...

cfb35fa23d8c59fe705b8a1a65eadda3.png

If you find this project useful or are currently using it, I welcome you to give me a star on my project homepage. This will be a huge encouragement to me! Your experience and stars are the motivation for me to continue working on .NET open source projects!

Reprinted from: .NET Sao operation

Link: cnblogs.com/sdflysha/p/20231015-sdcb-openvino-net.html

-EOF-

Technology group: Add editor WeChat dotnet999

Public account: Dotnet Lecture Hall