Technical background
The new national standard GB/T28181-2022 “Technical Requirements for Information Transmission, Exchange and Control of Networked Public Security Video Surveillance System” was released on December 30, 2022 and will be officially implemented on July 1, 2023.
The national standard GB/T28181-2022 “Technical Requirements for Information Transmission, Exchange and Control of Public Security Video Surveillance Networking System” stipulates the interconnection structure of the public security video surveillance network system (hereinafter referred to as “network system”), and the basic principles of transmission, exchange and control. Requirements and security requirements, as well as technical requirements such as controls, transmission processes, and protocol interfaces. It is suitable for the scheme design, system testing, acceptance and related equipment development and production of the public security video surveillance networking system. Other video surveillance networking systems can be implemented by reference.
The new version of the national standard GB/T28181-2022 “Technical Requirements for Information Transmission, Exchange and Control of Networked Public Security Video Surveillance System” replaces GB/T28181-2016 “Technical Requirements for Information Transmission, Exchange and Control of Networked Public Security Video Surveillance System”. Compared with /T28181-2016, except for structural adjustment and editorial changes, there are some changes in related main technologies. For example, the scope of the standard has been changed, and the terms and definitions of “networked system information”, “digital access”, “analog access”, “analog-digital hybrid monitoring system”, “digital monitoring system”, “monitoring point” and “monitoring center” have been deleted. Changed the terms and definitions of “SIP monitoring domain”, “non-SIP monitoring domain”, “cascading” and “interconnection”, changed the “SIP monitoring domain interconnection structure diagram”, changed the “networked system communication protocol structure diagram”, media stream channel Added H.265, G.722.1, AAC, etc.
GB/T28181-2022 specification description
PS encapsulation of video and audio data based on RTP
RTP-based PS encapsulation first encapsulates the video and audio streams into PS packets according to ISO/IEC 13818-1:2019, and then encapsulates the PS packets into RTP packets in the form of payload.
When performing PS encapsulation, you should put the team Map), the system header and the PSM after the PS header, the first ti (System Header) and PSM (Program Stream Map), the system header and the PSM after the PS header, first PES pack
Before.
A typical video key frame PS packet structure is shown in Figure C.1, where PESV is a video PES packet, PESA is an audio PES packet, and the PS packet structure of a video non-key frame generally does not include the system header and PSM. For the specific data structure of each part in the PS package, refer to the relevant description in ISO/IEC13818-1:2019.
The system header should contain a description of the type of stream in the PS package, where the stream ID (stream_id) values for video and audio are as follows:
a) Video stream ID: 0xEO;
b) Audio stream ID: 0xCO.
For several video and audio formats specified in this document, the values of the stream type (stream_type) in PSM are as follows:
a) MPEG-4 video stream: 0x10;
b) H.264 video stream: 0x1B;
c) SVAC video stream: 0x80;
d) H.265 video stream: 0x24;
e) G.711A law audio stream: 0x90;
f) G.711U-law audio stream: 0x91;
g) G.722.1 audio stream: 0x92;
h) G.723.1 audio stream: 0x93;
i) G.729 audio stream: 0x99;
j) sVAC audio stream: 0x9B;
k) AAC audio stream: 0xOF.
See ISO/IEC 13818-1:2019 for other specific technical specifications of PS package encapsulation.
The RTP encapsulation format of the PS packet refers to IETF RFC 2250, and the main parameters of RTP are set as follows: a) load type (payload type): 96;
b) encoding name (encoding name); PS; c) clock frequency (clock rate): 90kHz;
d) The “media” item in the “m” field in the SDP description: video.
RTP-based video and audio encapsulation
RTP encapsulation of H.265 video stream
The RTP payload format of H.265 shall conform to the relevant regulations of IETF RFC 7798.
Selection of the payload type (Payload Type) identification number of the H.265 video stream RTP packet: select from the dynamic range (96~~127) in Table 5 of the IETF RFC3551 protocol, it is recommended to set it as 100, and fill it according to actual needs.
The technical requirements of H.265 video encoding and decoding will not be repeated here. What needs to be noted here is: in order to ensure the efficiency of code stream parsing, the corresponding video parameter set (Video Param-eter Set , VPS), sequence parameter set (Sequence Parameter Set, SPS) and image parameter set (Picture Parameter Set, PPS).
Technical implementation
In fact, when we implemented GB28181-2016, we already supported H.265 encoding. It should be noted that due to the high complexity of H.265 encoding, the Android platform generally recommends hardcoding:
The encoding type is selected as follows:
//Video encoding type selection + + + + + + + + + +
videoEncodeTypeSelector = (Spinner)findViewById(R.id.videoEncodeTypeSelector);
final String[] videoEncodeTypes = new String[]{"Soft (H.264)", "Hard (H.264)", "Hard (H.265)"};
ArrayAdapter<String> adapterVideoEncodeType = new ArrayAdapter<String>(this,
android.R.layout.simple_spinner_item, videoEncodeTypes);
adapterVideoEncodeType.setDropDownViewResource(android.R.layout.simple_spinner_dropdown_item);
videoEncodeTypeSelector.setAdapter(adapterVideoEncodeType);
videoEncodeTypeSelector.setOnItemSelectedListener(new AdapterView.OnItemSelectedListener() {
@Override
public void onItemSelected(AdapterView<?> parent, View view,
int position, long id) {
if (isRTSPPublisherRunning || isPushingRtmp || isGB28181StreamRunning || isRecording) {
Log.e(TAG, "Could not switch video encoder type during publishing..");
return;
}
videoEncodeType = position;
Log.i(TAG, "[Video encoding type]Currently choosing: " + videoEncodeTypes[position] + ", videoEncodeType: " + videoEncodeType);
}
@Override
public void onNothingSelected(AdapterView<?> parent) {
}
});
Interface design:
/**
* Author: daniusdk.com
*
* Set Video H.265(hevc) hardware encoder, if support H.265(hevc) hardware encoder, it will return 0(set H.265 hardcoded)
*
* @param kbps: the kbps of different resolution.
*
* @return {0} if successful
*/
public native int SetSmartPublisherVideoHevcHWEncoder(long handle, int kbps);
If you need to use native mediacodec encoding, you can set the underlying ndk hard-coded mode:
/**
* Set whether to use Native Media NDK for video hardcoding, the default is not used, devices below Android 5.0 do not support
* @param handle
* @param is_native: 0 means not used, 1 means used, sdk defaults to 0.
* @return {0} if successful
*/
public native int SetNativeMediaNDK(long handle, int is_native);
In addition, we have made more refined processing for H.264 and H.265 hardcoding:
/*
* Set video hardcoded rate control mode
* @param hw_bitrate_mode: -1 means to use the default value, if not set, the default value will also be used, 0:CQ, 1:VBR, 2:CBR, 3:CBR_FD, please refer to: android.media.MediaCodecInfo.EncoderCapabilities
* Note that hard coding is related to mobile phone hardware. Most mobile phones only support some bit rate modes. In addition, hard coding devices are very different, and the effect of the same bit rate control mode on different devices may be different.
* @return {0} if successful
*/
public native int SetVideoHWEncoderBitrateMode(long handle, int hw_bitrate_mode);
/*
* Set video hard-coded complexity, Android 5.0 and above support
* @param hw_complexity: -1 means not set, please refer to: android.media.MediaCodecInfo.EncoderCapabilities.getComplexityRange() and android.media.MediaFormat.KEY_COMPLEXITY
* Note that hard coding is related to mobile phone hardware, some mobile phones may not support this setting
* @return {0} if successful
*/
public native int SetVideoHWEncoderComplexity(long handle, int hw_complexity);
/*
* Set video hardcoded quality, supported by Android 9 and above, only valid when the hardcoder bitrate mode (BitrateMode) is CQ (constant-quality mode)
* @param hw_quality: -1 means not set, please refer to: android.media.MediaCodecInfo.EncoderCapabilities.getQualityRange() and android.media.MediaFormat.KEY_QUALITY
* Note that hard coding is related to mobile phone hardware, some mobile phones may not support this setting
* @return {0} if successful
*/
public native int SetVideoHWEncoderQuality(long handle, int hw_quality);
/*
* Set H.264 hardcoded Profile, supported by Android 7 and above
* @param hw_avc_profile: 0 means use the default value, 0x01: Baseline, 0x02: Main, 0x08: High, 0x10000: ConstrainedBaseline, 0x80000: ConstrainedHigh;
* Note: ConstrainedBaseline and ConstrainedHigh may not be supported by most devices,
* H.264 is recommended to use High or ConstrainedHigh, if the hard decoding of your mobile phone cannot be solved, then set Baseline
* If the set Profile hardcoder does not support it, the encoder will use the default value
* Specific reference: android.media.MediaCodecInfo.CodecProfileLevel
* @return {0} if successful
*/
public native int SetAVCHWEncoderProfile(long handle, int hw_avc_profile);
/*
* Set the H.264 hard-coded Level, this is only valid when the Profile is set, supported by Android 7 and above
* @param hw_avc_level: 0 means use the default value, 0x100: Level3, 0x200: Level3.1, 0x400: Level3.2,
* 0x800: Level4, 0x1000: Level4.1, 0x2000: Level4.2,
* 0x4000: Level5, 0x8000: Level5.1, 0x10000: Level5.2,
* 0x20000: Level6, 0x40000: Level6.1, 0x80000: Level6.2,
* If the set level is too high and the hardcoder does not support it, the SDK will make corresponding adjustments
* Note: The minimum support for 640*480@25fps is Level3, the minimum support for 720p is Level3.1, and the minimum support for 1080p is Level4
* Specific reference: android.media.MediaCodecInfo.CodecProfileLevel
* @return {0} if successful
*/
public native int SetAVCHWEncoderLevel(long handle, int hw_avc_level);
/*
* Set the video hard-coded maximum bit rate, Android has no relevant documentation, so it is not recommended to set it,
* @param hw_max_bitrate: Maximum bit rate per second, unit bps
* @return {0} if successful
*/
public native int SetVideoHWEncoderMaxBitrate(long handle, long hw_max_bitrate);
External calls are as follows:
int hevcHWKbps = setHardwareEncoderKbps(false, video_width_, video_height_);
hevcHWKbps = hevcHWKbps*fps/25;
Log.i(TAG, "hevcHWKbps: " + hevcHWKbps);
int isSupportHevcHWEncoder = libPublisher
.SetSmartPublisherVideoHevcHWEncoder(publisherHandle, hevcHWKbps);
if (isSupportHevcHWEncoder == 0) {
libPublisher. SetNativeMediaNDK(publisherHandle, 0);
libPublisher.SetVideoHWEncoderBitrateMode(publisherHandle, 0); // 0:CQ, 1:VBR, 2:CBR
libPublisher. SetVideoHWEncoderQuality(publisherHandle, 39);
// libPublisher.SetVideoHWEncoderMaxBitrate(publisherHandle, ((long)hevcHWKbps)*1200);
Log.i(TAG, "Great, it supports hevc hardware encoder!");
}
Technical Summary
GB28181-2022 is aimed at H.265 encoding instructions, making up for the shortcomings of the 2016 specification. H.265 encoding, soft editing for mobile terminals, especially for high-resolution frame rates, has a large performance bottleneck. A more reasonable solution is to achieve high-efficiency H. The .265 hardcoded mode, GB28181-2022, makes H.265 support finally reflected in the specification level, which is justified and well-founded, and I believe it will have a wider application.