RTSP and RTMP use ffmpeg to push local cameras to the RTSP server

1. Streaming media: RTSP and RTMP

1. Working principles of RTSP and RTMP
1) RTSP working principle
The user device sends an RTSP request to the video streaming platform

The video streaming platform returns a list of requests that can be operated, such as play, pause, etc.

The user device sends specific requests to the video streaming platform, such as playback

The video streaming platform parses the request and calls the specified mechanism to start video stream processing

Because RTSP relies on dedicated servers and relies on RTP (underlying UDP), the protocol does not support encrypting video content or retransmitting lost packets.

Here is an explanation of how UDP and TCP are used in RTSP:

RTP protocol, full English name: Real-time Transport Protocol, Chinese is Real-time Transport Protocol, its bottom layer is actually UDP, so that low latency can be achieved.

In addition to the RTP protocol, to ensure smooth and consistent streaming, RTSP also uses two other network communication protocols:

TCP sends and receives control commands (such as play or stop requests): TCP reliable transmission. For example, when the user presses play or stops playing, this is an accurate request. This needs to ensure reliability. At this time, the role of TCP is reflected.
UDP transmits audio, video and data: UDP is a low-latency protocol, so it can achieve very efficient results when used to transmit audio, video and data.
This can be easily understood through the open source rtsp server: TCP listening port is 8554, UDP listening port is 8000

2) Working principle of RTMP

Camera capture video
Stream video via encoder to Video Platform Server
Video platform processes video streams
Distributed to the server closest to the user through CDN
Finally, the video stream can successfully reach the user device.

As the video travels from the camera to the server, RTMP splits large amounts of data into small pieces and transmits them across multiple virtual channels (content delivery network CDN), providing a stable and smooth video stream between the video source and the RTMP server.

2. Advantages and disadvantages of RTSP and RTMP
1) Advantages and disadvantages of RTSP
Advantages of RTSP:

1. Easily customize streaming: You can develop your own video streaming solution by combining different protocols.

2. Segmented Streaming: RTSP streaming enables viewers to access video content before the download is complete, rather than having to download the entire video to stream the content.

Disadvantages of RTSP:

1. Incompatibility with HTTP: There is no simple solution to play RTSP streams in a web browser, because RTSP is designed to stream video over a private network and additional software must be borrowed.

2. Low usage: Since video players and streaming services do not widely support RTSP streaming, usage is relatively low.

2) Advantages and disadvantages of RTMP
Advantages of RTMP:

1. Low latency: RTMP uses the exclusive 1935 port without buffering and can achieve low latency.

2. Highly adaptable: All RTMP servers can record live media streams, while also allowing viewers to skip part of the broadcast and join the live stream after it starts.

3. Flexibility: RTMP supports the integration of text, video and audio, supports MP3 and AAC audio streams, and also supports MP4, FLV and F4V videos.

Disadvantages of RTMP:

1. HTML5 not supported: The standard HTML5 player does not support RTMP streaming.

2. Vulnerable to bandwidth issues: RTMP streams often suffer from low bandwidth issues, causing video interruptions.

3. HTTP incompatibility: RTMP cannot be streamed over HTTP. You must implement a special server and use a third-party content delivery network or use a streaming video platform.

3) Comparison between RTSP and RTMP
RTMP and RTSP protocols are both streaming media protocols:

RTMP (Real Time Message Protocol) was proposed by Adobe to solve the problems of multiplexing and packetizing of multimedia data transmission streams. It has the advantages of low latency, high stability, and support for All camera formats can be played directly by loading the flash plug-in in the browser.
RTSP (Real-Time Stream Protocol) is a text-based multimedia playback control protocol jointly proposed by Real Networks and Netscape. RTSP defines the streaming format, and the streaming data is transmitted via RTP; RTSP has very good real-time effect and is suitable for video chat, video monitoring, etc.
The difference between RTMP and RTSP protocols:

Although RTSP has the best real-time performance, its implementation is complex and is suitable for video chat and video monitoring;

RTMP is strong in browser support. It can be played directly after loading the flash plug-in, so it is very popular. On the contrary, it is very difficult to play RTSP in the browser.

3. How to choose between RTSP and RTMP
IP camera selection RTSP: Almost all IP cameras support RTSP. This is because IP cameras existed long before the creation of the RTMP protocol. When used in conjunction with RTSP and IP cameras, the IP camera itself acts as an RTSP server, which means that the camera Connect to IP camera server and broadcast video.

Choose RTSP for IoT devices: RTSP is often built into drones or IoT software, allowing access to video sources. One of its benefits is low latency, ensuring there is no delay in the video, which is crucial for drones. .

Choose RTMP for streaming media applications: For example, various short video software, live video software, etc. have built-in RTMP. RTMP is designed to meet the needs of modern streaming media.

4. How to play RTSP on the browser
The live broadcast protocols include: rtmp, http, rtsp, etc. There are two most commonly used ones: http and rtmp. When using the http protocol, the video format needs to be m3u8 or flv. The advantages and disadvantages of each environment are explained in detail below. First of all, rtsp cannot be used in the web environment (including PC and mobile terminals), so the live broadcast can only choose rtmp or http.

The rtmp protocol only supports flashplayer, that is, it can only be used when flashplayer is installed on the PC (or the flashplayer component is installed in the Android environment, which is relatively rare). According to the current trend, flashplayer will be gradually eliminated. Of course, it will still exist in China for a relatively long time.

There are two formats for http protocol live broadcast, m3u8 and flv. flv is a live broadcast format that is about to be phased out. It has become too powerful to do live broadcasts. So comprehensively considered, m3u8 is relatively better. The advantage is that it supports mobile terminals and supports the environment where flashplayer is installed on the PC terminal. The disadvantages are the same as rtmp. flashplayer is not the future development trend. Another disadvantage is that m3u8 is delayed. It is not real-time, and the real-time transmission is not as good as the rtmp protocol. Because the live broadcast principle of m3u8 is to continuously compress the live broadcast source into ts files of specified length (such as 9 seconds, 10 seconds per ts file) and update the list in the m3u8 file in real time to achieve the live broadcast effect. This will have a time delay of at least 9,10 seconds. If the compression is too small, the video may become stuck due to client network issues.

To implement rtsp to http and use m3u8 format for live broadcast, please refer to RTSP Webcam to HLS Live Streaming using FFMPEG and XAMPP | PART 1

Specific process: Connect an external webcam that supports RTSP; use the ffplay command to play the RTSP stream, and the real-time video can be written to the specified folder according to the parameters (segmented writing); xampp opens apache (opens port 80), allowing the page to pass The saved m3u8 file can access the webcam monitoring interface in real time.

2. ffmpeg pushes the local camera to the RTSP server

2) RTMP working principle
Camera capture video

Stream video to video platform server via encoder

Video platform processes video streams

Distributed to the server closest to the user through CDN

Finally, the video stream can successfully reach the user device.

As the video travels from the camera to the server, RTMP splits large amounts of data into small pieces and transmits them across multiple virtual channels (content delivery network CDN), providing a stable and smooth video stream between the video source and the RTMP server.

2. Advantages and disadvantages of RTSP and RTMP
1) Advantages and disadvantages of RTSP
Advantages of RTSP:

1. Easily customize streaming: You can develop your own video streaming solution by combining different protocols.

2. Segmented Streaming: RTSP streaming enables viewers to access video content before the download is complete, rather than having to download the entire video to stream the content.

Disadvantages of RTSP:

1. Incompatibility with HTTP: There is no simple solution to play RTSP streams in a web browser, because RTSP is designed to stream video over a private network and additional software must be borrowed.

2. Low usage: Since video players and streaming services do not widely support RTSP streaming, usage is relatively low.

2) Advantages and disadvantages of RTMP
Advantages of RTMP:

1. Low latency: RTMP uses the exclusive 1935 port without buffering and can achieve low latency.

2. Highly adaptable: All RTMP servers can record live media streams, while also allowing viewers to skip part of the broadcast and join the live stream after it starts.

3. Flexibility: RTMP supports the integration of text, video and audio, supports MP3 and AAC audio streams, and also supports MP4, FLV and F4V videos.

Disadvantages of RTMP:

1. HTML5 not supported: The standard HTML5 player does not support RTMP streaming.

2. Vulnerable to bandwidth issues: RTMP streams often suffer from low bandwidth issues, causing video interruptions.

3. HTTP incompatibility: RTMP cannot be streamed over HTTP. You must implement a special server and use a third-party content delivery network or use a streaming video platform.

3) Comparison between RTSP and RTMP
RTMP and RTSP protocols are both streaming media protocols:

RTMP (Real Time Message Protocol) was proposed by Adobe to solve the problems of multiplexing and packetizing of multimedia data transmission streams. It has the advantages of low latency, high stability, and support for All camera formats can be played directly by loading the flash plug-in in the browser.
RTSP (Real-Time Stream Protocol) is a text-based multimedia playback control protocol jointly proposed by Real Networks and Netscape. RTSP defines the streaming format, and the streaming data is transmitted via RTP; RTSP has very good real-time effect and is suitable for video chat, video monitoring, etc.
The difference between RTMP and RTSP protocols:

Although RTSP has the best real-time performance, its implementation is complex and is suitable for video chat and video monitoring;

RTMP is strong in browser support. It can be played directly after loading the flash plug-in, so it is very popular. On the contrary, it is very difficult to play RTSP in the browser.

3. How to choose between RTSP and RTMP
IP camera selection RTSP: Almost all IP cameras support RTSP. This is because IP cameras existed long before the creation of the RTMP protocol. When used in conjunction with RTSP and IP cameras, the IP camera itself acts as an RTSP server, which means that the camera Connect to IP camera server and broadcast video.

Choose RTSP for IoT devices: RTSP is often built into drones or IoT software, allowing access to video sources. One of its benefits is low latency, ensuring there is no delay in the video, which is crucial for drones. .

Choose RTMP for streaming media applications: For example, various short video software, live video software, etc. have built-in RTMP. RTMP is designed to meet the needs of modern streaming media.

4. How to play RTSP on the browser
The live broadcast protocols include: rtmp, http, rtsp, etc. There are two most commonly used ones: http and rtmp. When using the http protocol, the video format needs to be m3u8 or flv. The advantages and disadvantages of each environment are explained in detail below. First of all, rtsp cannot be used in the web environment (including PC and mobile terminals), so the live broadcast can only choose rtmp or http.

The rtmp protocol only supports flashplayer, that is, it can only be used when flashplayer is installed on the PC (or the flashplayer component is installed in the Android environment, which is relatively rare). According to the current trend, flashplayer will be gradually eliminated. Of course, it will still exist in China for a relatively long time.

There are two formats for http protocol live broadcast, m3u8 and flv. flv is a live broadcast format that is about to be phased out. It has become too powerful to do live broadcasts. So comprehensively considered, m3u8 is relatively better. The advantage is that it supports mobile terminals and supports the environment where flashplayer is installed on the PC terminal. The disadvantages are the same as rtmp. flashplayer is not the future development trend. Another disadvantage is that m3u8 is delayed. It is not real-time, and the real-time transmission is not as good as the rtmp protocol. Because the live broadcast principle of m3u8 is to continuously compress the live broadcast source into ts files of specified length (such as 9 seconds, 10 seconds per ts file) and update the list in the m3u8 file in real time to achieve the live broadcast effect. This will have a time delay of at least 9,10 seconds. If the compression is too small, the video may become stuck due to client network issues.

To implement rtsp to http and use m3u8 format for live broadcast, please refer to RTSP Webcam to HLS Live Streaming using FFMPEG and XAMPP | PART 1

Specific process: Connect an external webcam that supports RTSP; use the ffplay command to play the RTSP stream, and the real-time video can be written to the specified folder according to the parameters (segmented writing); xampp opens apache (opens port 80), allowing the page to pass The saved m3u8 file can access the webcam monitoring interface in real time.

2. ffmpeg pushes the local camera to the RTSP server

Note: ffmpeg pushes the local camera to the 8554 port of rtsp (rtsp-simple-server listens to the 8554 port when processing rtsp. If you specify other ports, ffmpeg will fail to push the stream)

1. Install ffmpeg and rtsp-simple-server
Rough implementation process: Use rtsp-simple-server as a transfer server for ffmpeg (writing client) to push streams, and background service (reading clients) to pull streams

1) Install rtsp-simple-server and ffmpeg on windows
Refer to the Windows environment to build an RTSP video streaming server (remember to modify the IP address in the rtsp-simple-server.yml configuration file)

2) Install rtsp-simple-server and ffmpeg on linux
Install rtsp-simple-server_v0.20.2_linux_amd64.tar.gz (x86 CPU is used as an example here), decompress and modify the IP address in the rtsp-simple-server.yml configuration file (vim replacement command is %s:/127.0.0.1 /192.168.132.100/g), execute ./rtsp-simple-server to start the rtsp server.

If you want to start the rtsp server in the background, execute the following command

nohup ./rtsp-simple-server >> rtsp_server.log 2> & amp;1 & amp; #Non-hanging startup command

tail rtsp_server.log #View rtsp-simple-server startup log file

ps -aux | grep rtsp_simple_server #View rtsp-simple-server process
dpf 2116 0.0 0.0 13140 1016 pts/0 S + 04:54 0:00 grep –color=auto rtsp_simple_server

Install ffmpeg, decompress and execute ./ffmpeg to use ffmpeg. Please refer to the method of using ffmpeg under Linux.

Note: Please check online for the decompression operations of tar.gz, xz, and tar in Linux.

2. Push the local camera to the RTSP server
Rough implementation process: Use rtsp-simple-server as a transfer server for ffmpeg (writing client) to push streams, and background service (reading clients) to pull streams

Here, the windows system is used as a demonstration. First, decompress rtsp-simple-server_v0.19.1_windows_amd64.zip, open rtsp-simple-server.exe to monitor the TCP port 8554 under RTSP, and then send the image frames collected by the specified camera to the port through ffmpeg. Push streaming (i.e. socket communication between multiple clients and server)

1) Write client: ffmpeg
ffmpeg pushes the video file to the specified ip + port (-stream_loop -1):

ffmpeg -re -stream_loop -1 -i the file name of your video -c copy -f rtsp rtsp://127.0.0.1:8554/videoFile_test
1
ffmpeg pushes the video stream of the local camera to the specified ip + port, you need

//Get the local camera name
ffmpeg -list_devices true -f dshow -i dummy

//ffmpeg pushes the stream to the specified port (mine is Integrated Camera)
ffmpeg -f dshow -i video=”own camera driver name” -vcodec libx264 -preset:v ultrafast -tune:v zerolatency -rtsp_transport tcp -f rtsp rtsp://127.0.0.1:8554/camera_test
//libx264 encoding
ffmpeg -f dshow -i video=”Integrated Camera” -vcodec libx264 -preset:v ultrafast -tune:v zerolatency -rtsp_transport tcp -f rtsp rtsp://127.0.0.1:8554/camera_test
) Server side: RTSP server
The initial startup effect is as follows:

3) Reading client: Reading client can be achieved in two ways
Install VLC, select streaming data playback mode, enter rtsp://127.0.0.1:8554/camera_test, rtsp://127.0.0.1:8554/videoFile_test to play ;
Or use the following python code:

import cv2

def capture_video(rtsp_path):
    name = rtsp_path.split("/")[-1]
    capture = cv2.VideoCapture(rtsp_path)
    while capture.isOpened():
        ret, frame = capture.read()
        if not ret:
            break
        cv2.imshow(name, frame)
        if cv2.waitKey(50) == 27:
            break

if __name__ == '__main__':
    # rtsp_paths = ['rtsp://127.0.0.1:8554/videoFile_test','rtsp://127.0.0.1:8554/camera_test']
    rtsp_paths = ['rtsp://127.0.0.1:8554/videoFile_test']
    for rtsp_path in rtsp_paths:
        capture_video(rtsp_path)

    cv2.waitKey(0)
    cv2.destroyAllWindows()

At this time, there will be two createby and reading, that is, two processes will be opened to read the video stream

1. Streaming media: RTSP and RTMP

2) Working principle of RTMP

3) Reading client: Reading client can be achieved in two ways