AI model edge deployment whole process 2 (Tools)

1. How to use VSCODE for local debugging on edge deployment

Reference link: ARM-Linux GDB remote debugging
Embedded linux arm development board gdb debugging through vscode

Written before: Most people who work on AI models start from pycharm or other Python IDEs, so it is very convenient to use breakpoints to debug when running the program. Moreover, breakpoint debugging is helpful for tracking data flow and is also very suitable for problem analysis. However, when deployed at the edge, it is difficult to perform remote breakpoint debugging like using pycharm. The bottom line is that the edge cannot compile code and can only run applications. Therefore, this extends to how to perform breakpoint debugging of applications deployed on the edge on the remote side (I usually use Windows).

1.1. Preparation:

Before we begin, we must first clarify the several platforms involved:

  • Windows (local): This is mainly used to connect to remote or WSL or virtual machine compilation platforms. The following is replaced by PC
  • Linux (compilation platform): This is generally the Ubuntu system. Depending on the use of different people, it may be WSL, a virtual machine, or a remote compilation server. This is mainly used to compile code to generate binary programs that can be run by the board. Linux will be used instead
  • Linux (board platform): This is the operating system of the board. It is mainly used to run compiled debug programs and gdbserver. For me personally, I use NPU to run model programs. The following will be replaced by ARM

1.1.1, vscode software and extension preparation

Before you start debugging, you must first determine whether your board can perform GDB debugging. This is very important. If not, you need to search for how to install gdb and gdbserver offline. If your board does not support the GDB service at all, then there is no need to read the following. . ThereforeThis article is based on the premise that GDB and gdbserver exist on the board side. To check whether the board supports GDB debugging, you can check the documentation provided by the board or enter the following command on the command line after connecting to the board:

gdb --version ##View gdb version
gdbserver --version #View gdbserver version

There is also linux segment ssh support

About vscode software

  1. Download and install VSCODE, download address: VSCODE
  2. Open VSCODE on PC and download“remote-ssh”.


Figure 1. Remote SSH

After downloading remote ssh, connect to your Linux. For specific instructions on how to connect, you can search vscode online to connect to the remote server. There are many tutorials, so I won’t go into details here. After the PC is connected to your Linux, install “C/C++” and “GDB Debug” on your Linux side. The extension installed here is also in the extension: store search. The difference is that there is one installed in XXX. Since I am using WSL, the one shown is installed in WSL.


Figure 2. C/C + +

After you install these two plug-ins, you will see the following interface when you open the extension. The local extension is the VS code extension installed on your PC, and the following one is the VS code extension for your Linux. All you have to do is ensure that the linux installation extensions include “C/C++” and “GDB Debug”.


Figure 3. VS code extension interface

1.1.2. Preparation of compiler CMakelist.txt

In most cases, the compilation work is not performed on the ARM side, but on the Linux side, but the corresponding compilation must be compiled using the corresponding cross-compilation tool chain of ARM. Compilation is usually done using CMakelist.txt. Of course, there are also cases where you only need to compile a program file that does not require external dependencies. In this case, use the following command:

g + + -g XXX.cpp -o AAA
#XXX is your code name, AAA is the name of the binary application you generated

If you use the CMakelist.txt file for compilation, you need to add the following command at the beginning of the outermost CMakelist.txt (I have not tried adding it to other CMakelists, only the outermost one):

SET(CMAKE_BUILD_TYPE Debug)
set(CMAKE_C_FLAGS_DEBUG "$ENV{CXXFLAGS} -O0 -Wall -g")
set(CMAKE_CXX_FLAGS_DEBUG "$ENV{CXXFLAGS} -O0 -Wall -g")

At this point, the preparation work has been roughly completed.

1.2. Start debugging

The overall debugging flow chart is shown below:


Figure 4. Debugging flow chart

The PC side uses vscode to connect to the compilation server Linux, and then Linux uses the gdb cross-compilation tool chain provided by the board to debug the aaa program run by the board using gdbserver. The debug breakpoint is set in vscode on the PC side, and the variables are also displayed on vscode.

1.2.1, gdbserver

The gdbserver service is a gdb debugging service running on the arm side. Its command to run under the network port is as follows:

#Note that you need to first switch to the directory where the program aaa is located. If it is not there, you need to add the path before aaa.
gdbserver zz.zz.zz.zz:8888 aaa
#zz.zz.zz is the IP address of the Linux side. If wsl is used, the IP addresses of Linux and PC are the same. Port number 8888 can be set freely
#Just make sure that the port number is not occupied.

1.2.2, Linux side launch.json configuration

First check in the code you need to debug and put the corresponding breakpoints


Figure 4. Code corresponding to the program to be debugged

Then press F5 and the options shown below will pop up:


Figure 5. Options

There is another option after selecting GDB/LLDB. Just select the default option. After selecting, the launch.json file will be automatically generated, the content is as follows


Figure 6. launch,json

Where "progarm" is the program location compiled and generated by your Linux side to execute the debug program on ARM (not the code location, but the location of the executable program generated after cmake), "miDebuggerPath" is the location of the cross-compilation tool chain on the Linux side. You can find it through the which command. "miDebuggerServerAddress" is the IP address and port number monitored by gdbserver on arm.

After the configuration is completed, run F5 directly to get the results as shown in the figure:


Figure 7. debug

At this point the entire process is completed, enjoy your debugging process.