This article teaches you how to use SpireCV for pod control and target detection and tracking.

Function Overview

SpireCV-SDK is an edge real-time sensing SDK library specially built for intelligent unmanned systems. It can realize pod control functions and control the drone’s camera and pod, including taking pictures, recording, streaming and other functions, and can save videos. and push streaming, as well as complete target detection, identification and tracking functions. This article will describe in detail how to use SpireCV for pod control.

Note: For pod products that can control Amov, such as G1, Q10, etc., for specific function implementation details, please search for relevant articles in the official account or follow the official website of Amov Laboratory.

Instructions for use

1. Azimuth angle definition

The way gestures are represented varies between pods, so we unified them.

Euler angles are used to represent the attitude information of the pod. When the line of sight is in the same direction as the camera, it is defined as follows:

  • roll: roll sideways. Counterclockwise is positive.

  • pitch: pitch. Downward is positive.

  • yaw: Yaw. To the left is positive.

As shown in the figure below, the arrow direction is the positive direction:

2. How to initialize the pod

(1) Pod status reading function

The status of the pod is read through the callback function, and the user can choose the default function or specify a self-coded function. The paradigm of the status acquisition callback function is as follows (default function):

void gimableCallback(double & amp;frame_ang_r, double & amp;frame_ang_p, double & amp;frame_ang_y,
double & amp;imu_ang_r, double & amp;imu_ang_p, double & amp;imu_ang_y,
double & amp;fov_x, double & amp;fov_y)
{
}

Note:

imu_ang_r: IMU roll angle, value range: (-60, 60) ;

imu_ang_p: IMU pitch angle, value range: (-135, 135) ;

imu_ang_y: IMU yaw angle, value range: (-150, 150);

rotor_ang_r: motor roll angle, value range: (-60, 60) ;

rotor_ang_p: motor pitch angle, value range: (-135, 135);

rotor_ang_y: motor yaw angle, value range: (-150, 150) ;

fov_x: camera lateral field of view angle, changes according to different focal lengths of different cameras ;

fov_y: camera longitudinal field of view angle, changes according to different focal lengths of different cameras .

3. Constructor

The user specifies the model of the pod and the type of control port used in the constructor.

Provide a default call with default parameters:

sv::GimbalType::G1;
sv::GimbalLink::SERIAL;
Gimbal(GimbalType gtype, GimbalLink ltype).

Note:

gtype: The model of the pod. See type sv::GimbalType.

ltype: The control port type used by the pod. See type sv::GimbalLink.

Pod control port configuration

Considering that there are many different parameters for communication with the pod, a communication port configuration function is reserved. However, note that the configuration needs to be completed before the pod is opened, that is, before the open() function is called.

  • serial port

  • Set serial port name

void setSerialPort(const std::string & amp;port)

  • port: device serial port name. Default value dev/ttyUSB0

Note: The serial port device name in Windows is usually *`COMx`*; in Linux it is usually *`dev/ttyUSBx`*.

  • Set serial port baud rate

void setSerialPort(const int baud_rate)

  • baud_rate: serial communication baud rate. Default value is 115200.

  • Set common serial port configurations

void setSerialPort(GimablSerialByteSize byte_size,

  • byte_size: Data bit length. default value:

sv::GimablSerialByteSize::EIGHT_BYTES

  • parity: verification method. default value:

sv::GimablSerialParity::PARITY_NONE

  • stop_bits: stop bit length. default value:

sv::GimablSerialStopBits::STOPBITS_ONE

  • flowcontrol: flow control mode. default value:

sv::GimablSerialFlowControl::FLOWCONTROL_NONE

  • time_out: Serial port timeout time. . Default value is 500.

  • Network port

  • Set IP address

void setNetIp(const std::string & amp;ip)

  • ip: pod network control IP address. Default value “192.168.2.64”.

  • Set port number

void setNetPort(const int & amp;port)

  • port: pod network control port number. Default value 9090

Enable pod control

After the user completes the configuration, he can use this function to enable the control function of the pod.

Provides a default call, with the default parameter sv::emptyCallback.

bool open(PStateInvoke callback)

  • callback: Specified status acquisition callback function

Recommended pod instantiation process

Before starting to use the pod control function, the user needs to configure the pod parameters and then call the open() function to enable the pod control function. The recommended process is as follows, taking the G1 pod as an example:

#include <sv_gimbal.h>
sv::Gimbal *gimbal;
gimbal=newsv::Gimbal(sv::GimbalType::G1, sv::GimbalLink::SERIAL);//Use the serial port to initialize the pod control
gimbal->setSerialPort("/dev/ttyUSB0");//Use */dev/ttyUSB0* as the control serial port
if(gimbal->open(gimableCallback))//Use the function named gimableCallback as the status reading function
{
std::cout << "gimbal open failed" << std::endl;
exit(1);
};

When the pod supports network control, it can be configured into network configuration mode as follows (G1 does not support network control, this is only a demonstration example):

#include <sv_gimbal.h>
sv::Gimbal *gimbal;
gimbal = new sv::Gimbal(sv::GimbalType::G1, sv::GimbalLink::ETHERNET_TCP);//Use TCP protocol to initialize pod control
gimbal->setNetIp("192.168.1.1");//PTZ control IP address *192.168.1.1*
gimbal->setNetPort(8080);//Use port 8080
if(gimbal->open(gimableCallback))//Use the function named *gimableCallback* as the status reading function
{
std::cout << "gimbal open failed" << std::endl;
exit(1);
};

How to control the pod

  • Change status to get callback function

Users can use this function to modify the previously set status to obtain the callback function. This function can be used during a pod run, but is not recommended and may result in data corruption.

void setStateCallback(PStateInvoke callback)

  • callback: The specified status acquisition callback function.

  • Pod angle control

The user can use this function to control the movement of the pod to a specified attitude, represented by Euler angles.

void setAngleEuler(double roll,double pitch,double yaw,
double roll_rate = 0,double pitch_rate = 0,double yaw_rate = 0);

Note:

roll: pod target roll angle. Value range: (-60, 60)

pitch: pod target pitch angle. Value range: (-135, 135)

yaw: pod target yaw angle. Value range: (-150, 150)

roll_rate: The maximum absolute value of the angular velocity of the pod’s roll axis. This value is invalid when set to 0

pitch_rate: the maximum absolute value of the pod’s pitch axis angular velocity. This value is invalid when set to 0

yaw_rate: The maximum absolute value of the pod’s yaw angular velocity. This value is invalid when set to 0

The angle value that the pod can achieve cannot exceed the physical limits of the pod.

  • Pod angular speed control

The user can use this function to control the pod to move in one or more axes at a specified angular velocity, expressed as Euler angles.

void setAngleRateEuler(double roll_rate,double pitch_rate,double yaw_rate);

Note:

roll_rate: pod target roll axis angular velocity.

pitch_rate: pod target pitch axis angular velocity.

yaw_rate: pod target yaw angular speed.

Note: Different pods may respond differently to angular velocity commands. It is best to send this command periodically.

  • Centering the pod

The user can quickly return the pod to its initial posture by calling this function.

bool setHome();

  • Set the pod zoom ratio (specify the multiple)

Users can use this function to set the pod to zoom the screen by a specified multiple.

bool setZoom(double x);

  • x: Zoom factor relative to the original image.

Note: If only the pod uses the USB interface, the port number may change if other devices use the USB interface at the same time.

  • Set the pod zoom factor (specify direction)

Users can control the zoom direction of the pod screen through this function.

bool setAutoZoom(int state);

Note:

state: screen zoom direction. 0: Reduce; 1: Enlarge; 2: Stop.

This feature requires pod support.

  • Adjust the pod focal plane

The user can adjust the focal plane of the pod through this function. Normally, the pod adjusts the focal plane on its own to get the sharpest imaging, without user intervention.

bool setAutoFocus(int state);

Note:

state: focal plane adjustment direction. 0: Approach; 1: Far away; 2: Stop.

This feature requires pod support.

Pod image acquisition

  • Pod photo shoot

Users can use this function to call the pod to take photos with a preset resolution and store them inside the pod.

bool takePhoto();

  • Pod recording video

Users can use this function to call the pod to record video at the preset resolution and store it inside the pod.

bool takeVideo(int state);

Note: state: recording command. 0: Start recording; 1: Turn off recording

  • Get pod recording status

Users can obtain the status of the pod’s video recording through this function.

int getVideoState();

Note: return: recording status. 0: Recording; 1: Recording not started

  • Routine compilation reference

(1) Routine file location: ~/SpireCV/samples/demo. For pod control, you need to create a new cpp file yourself.

(2) If you create a new cpp file or change the parameters in the routine, you need to recompile:

# Compile routine
cd <path to SpireCV/build>
sudo make install

-End-

Previous recommendations:

SpireCV: An edge real-time sensing SDK library specially built for intelligent unmanned systemsicon-default.png?t=N7T8http://mp.weixin.qq.com/s ?__biz=MzIwNzgxNzg0Mg== & amp;mid=2247544685 & amp;idx=1 & amp;sn=2352672ecae97b5d6ca69ca8f7dc3e54 & amp;chksm=970ef270a0797b660ad1f2cb5da77d660f67e94 fb19b2bd46d9f146727c300acdf86192212ea & amp;scene=21#wechat_redirect

How to quickly implement data annotation, model training, and efficient deployment without typing code? icon-default.png?t=N7T8https://mp.weixin.qq.com/s?__biz=MzIwNzgxNzg0Mg== & amp;mid=2247533308 & amp; idx=2 &sn=2cccc2fd7e0cb1c27c4d990bd680ce2b &chksm=970e9fe1a07916f7dfbaa155d989a44dc35679aafba552b1a70197413c8efcc8c1bed18cbbde &token=40 9620283 &lang=zh_CN &scene=21#wechat_redirect

How does SpireCV take advantage of TensorRT acceleration? icon-default.png?t=N7T8http://mp.weixin.qq.com/s?__biz=MzIwNzgxNzg0Mg== & amp;mid=2247546760 & amp; idx=1 &sn=8493ff75cb91f8babbea3d03c160952f &chksm=970eea95a0796383e6845600cc545602c6010918b6fbb4d2cc3f1fbd56cc4627eaf715eda0a8 &scene= 21#wechat_redirect