Environment construction
start SRS
clone srs source code:
git clone https://github.com/ossrs/srs.git & amp; & amp; git checkout 5.0release
Configure the conf/rtmp2rtc.conf file, enable webrtc features and configure candidate:
mingo@localhost:~/Applications/workspace/multimedia/srs/trunk$cat conf/rtmp2rtc.conf
listen 1935;
max_connections 100;
daemon off;
srs_log_tank console;
http_server {
enabled on;
listen 8081;
dir ./objs/nginx/html;
}
http_api {
enabled on;
listen 1985;
}
stats {
network0;
}
rtc_server {
enabled on;
listen 8000; # UDP port
# @see https://ossrs.net/lts/zh-cn/docs/v4/doc/webrtc#config-candidate
candidate *;
}
vhost __defaultVhost__ {
rtc {
enabled on;
# @see https://ossrs.net/lts/zh-cn/docs/v4/doc/webrtc#rtmp-to-rtc
rtmp_to_rtc on;
# @see https://ossrs.net/lts/zh-cn/docs/v4/doc/webrtc#rtc-to-rtmp
rtc_to_rtmp on;
}
http_remux {
enabled on;
mount [vhost]/[app]/[stream].flv;
}
}
In particular, candidate configuration, you can also specify the local IP:
rtc_server {
enabled on;
listen 8000; # UDP port
# @see https://ossrs.net/lts/zh-cn/docs/v4/doc/webrtc#config-candidate
candidate *;
}
Start the srs program:
mingo@localhost:~/Applications/workspace/multimedia/srs/trunk$./objs/srs -c conf/rtmp2rtc.conf
As you can see, srs started successfully:
RTC play page
SRS implements RTC’s H5 player page. After successfully launching SRS, type in the browser address bar:
http://localhost:8081/players/rtc_player.html
As you can see, the RTC playback page is successfully opened. At this point, the environment configuration is complete.
Packet capture analysis
HTTP signaling
First use wireshark to capture packets and analyze the signaling process of srs’s webrtc:
It can be seen that the signaling is initiated by the Chrome webrtc client, and SRS responds passively as the webrtc server. The complete signaling is as follows:
{
"api": "http://localhost:1985/rtc/v1/play/",
"tid": "9328266",
"streamurl": "webrtc://localhost/live/rtc-test",
"clientip": null,
"sdp": "v=0\r\
o=- 4536557109234937850 2 IN IP4 127.0.0.1\r\
s=-\r\
t=0 0\r\
a=group:BUNDLE 0 1\r\
a=extmap -allow-mixed\r\
a=msid-semantic: WMS\r\
m=audio 9 UDP/TLS/RTP/SAVPF 111 63 103 104 9 0 8 106 105 13 110 112 113 126\r\
c=IN IP4 0.0 .0.0\r\
a=rtcp:9 IN IP4 0.0.0.0\r\
a=ice-ufrag:pmt4\r\
a=ice-pwd:zGRQaVd + 4OwEBJ8Wwg6iqo4x\r\
a=ice-options:trickle\r\ na=fingerprint:sha-256 0B:09:D5:22:2E:61:ED:05:D8:CF:7B:DB:5C:35:F1:7A:E5:1D:0B:94:11:E6 :E9:A9:46:AF:07:D4:4A:CD:40:93\r\
a=setup:actpass\r\
a=mid:0\r\
a=extmap:1 urn:ietf:params: rtp-hdrext:ssrc-audio-level\r\
a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time\r\
a=extmap:3 http:// www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01\r\
a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:mid\r\
a=recvonly \r\
a=rtcp-mux\r\
a=rtpmap:111 opus/48000/2\r\
a=rtcp-fb:111 transport-cc\r\
a=fmtp:111 minptime=10;useinbandfec=1\ r\
a=rtpmap:63 red/48000/2\r\
a=fmtp:63 111/111\r\
a=rtpmap:103 ISAC/16000\r\
a=rtpmap:104 ISAC/ 32000\r\
a=rtpmap:9 G722/8000\r\
a=rtpmap:0 PCMU/8000\r\
a=rtpmap:8 PCMA/8000\r\
a=rtpmap:106 CN/32000\r\
a= rtpmap:105 CN/16000\r\
a=rtpmap:13 CN/8000\r\
a=rtpmap:110 telephone-event/48000\r\
a=rtpmap:112 telephone-event/32000\r\
a=rtpmap: 113 telephone-event/16000\r\
a=rtpmap:126 telephone-event/8000\r\
m=video 9 UDP/TLS/RTP/SAVPF 96 97 98 99 100 101 35 36 37 38 102 122 127 121 125 107 108 109 124 120 39 40 41 42 43 44 45 46 47 48 123 119 114 115 116 49\r\
c=IN IP4 0.0.0.0\r\
a=rtcp:9 IN IP4 0.0.0.0\r\
a=ice-ufrag :pmt4\r\
a=ice-pwd:zGRQaVd + 4OwEBJ8Wwg6iqo4x\r\
a=ice-options:trickle\r\
a=fingerprint:sha-256 0B:09:D5:22:2E:61:ED:05: D8:CF:7B:DB:5C:35:F1:7A:E5:1D:0B:94:11:E6:E9:A9:46:AF:07:D4:4A:CD:40:93\r\ na=setup:actpass\r\
a=mid:1\r\
a=extmap:14 urn:ietf:params:rtp-hdrext:toffset\r\
a=extmap:2 http://www.webrtc.org/ experiments/rtp-hdrext/abs-send-time\r\
a=extmap:13 urn:3gpp:video-orientation\r\
a=extmap:3 http://www.ietf.org/id/draft-holmer- rmcat-transport-wide-cc-extensions-01\r\
a=e xtmap:5 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay\r\
a=extmap:6 http://www.webrtc.org/experiments/rtp-hdrext/video-content- type\r\
a=extmap:7 http://www.webrtc.org/experiments/rtp-hdrext/video-timing\r\
a=extmap:8 http://www.webrtc.org/experiments/rtp- hdrext/color-space\r\
a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:mid\r\
a=extmap:10 urn:ietf:params:rtp-hdrext:sdes:rtp-stream- id\r\
a=extmap:11 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id\r\
a=recvonly\r\
a=rtcp-mux\r\
a=rtcp-rsize \r\
a=rtpmap:96 VP8/90000\r\
a=rtcp-fb:96 goog-remb\r\
a=rtcp-fb:96 transport-cc\r\
a=rtcp-fb:96 ccm fir\ r\
a=rtcp-fb:96 nack\r\
a=rtcp-fb:96 nack pli\r\
a=rtpmap:97 rtx/90000\r\
a=fmtp:97 apt=96\r\
a=rtpmap :98 VP9/90000\r\
a=rtcp-fb:98 goog-remb\r\
a=rtcp-fb:98 transport-cc\r\
a=rtcp-fb:98 ccm fir\r\
a=rtcp- fb:98 nack\r\
a=rtcp-fb:98 nack pli\r\
a=fmtp:98 profile-id=0\r\
a=rtpmap:99 rtx/90000\r\
a=fmtp:99 apt= 98\r\
a=rtpmap:100 VP9/90000\r\
a=rtcp-fb:100 goog-remb\r\
a=rtcp-fb:100 t ransport-cc\r\
a=rtcp-fb:100 ccm fir\r\
a=rtcp-fb:100 nack\r\
a=rtcp-fb:100 nack pli\r\
a=fmtp:100 profile-id= 2\r\
a=rtpmap:101 rtx/90000\r\
a=fmtp:101 apt=100\r\
a=rtpmap:35 VP9/90000\r\
a=rtcp-fb:35 goog-remb\r\ na=rtcp-fb:35 transport-cc\r\
a=rtcp-fb:35 ccm fir\r\
a=rtcp-fb:35 nack\r\
a=rtcp-fb:35 nack pli\r\
a= fmtp:35 profile-id=1\r\
a=rtpmap:36 rtx/90000\r\
a=fmtp:36 apt=35\r\
a=rtpmap:37 VP9/90000\r\
a=rtcp-fb: 37 goog-remb\r\
a=rtcp-fb:37 transport-cc\r\
a=rtcp-fb:37 ccm fir\r\
a=rtcp-fb:37 nack\r\
a=rtcp-fb:37 nack pli\r\
a=fmtp:37 profile-id=3\r\
a=rtpmap:38 rtx/90000\r\
a=fmtp:38 apt=37\r\
a=rtpmap:102 H264/90000\r \
a=rtcp-fb:102 goog-remb\r\
a=rtcp-fb:102 transport-cc\r\
a=rtcp-fb:102 ccm fir\r\
a=rtcp-fb:102 nack\r\ na=rtcp-fb:102 nack pli\r\
a=fmtp:102 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42001f\r\
a=rtpmap:122 rtx/90000\ r\
a=fmtp:122 apt=102\r\
a=rtpmap:127 H264/90000\r\
a=rtcp-fb:127 goog-remb\r\
a=rtcp-fb:127 transport-cc\r\ na=rtcp-fb:127 ccm fir \r\
a=rtcp-fb:127 nack\r\
a=rtcp-fb:127 nack pli\r\
a=fmtp:127 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id =42001f\r\
a=rtpmap:121 rtx/90000\r\
a=fmtp:121 apt=127\r\
a=rtpmap:125 H264/90000\r\
a=rtcp-fb:125 goog-remb\r \
a=rtcp-fb:125 transport-cc\r\
a=rtcp-fb:125 ccm fir\r\
a=rtcp-fb:125 nack\r\
a=rtcp-fb:125 nack pli\r\
a =fmtp:125 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f\r\
a=rtpmap:107 rtx/90000\r\
a=fmtp:107 apt=125\r\ na=rtpmap:108 H264/90000\r\
a=rtcp-fb:108 goog-remb\r\
a=rtcp-fb:108 transport-cc\r\
a=rtcp-fb:108 ccm fir\r\
a =rtcp-fb:108 nack\r\
a=rtcp-fb:108 nack pli\r\
a=fmtp:108 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42e01f\r \
a=rtpmap:109 rtx/90000\r\
a=fmtp:109 apt=108\r\
a=rtpmap:124 H264/90000\r\
a=rtcp-fb:124 goog-remb\r\
a=rtcp -fb:124 transport-cc\r\
a=rtcp-fb:124 ccm fir\r\
a=rtcp-fb:124 nack\r\
a=rtcp-fb:124 nack pli\r\
a=fmtp:124 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=4d0 01f\r\
a=rtpmap:120 rtx/90000\r\
a=fmtp:120 apt=124\r\
a=rtpmap:39 H264/90000\r\
a=rtcp-fb:39 goog-remb\r\ na=rtcp-fb:39 transport-cc\r\
a=rtcp-fb:39 ccm fir\r\
a=rtcp-fb:39 nack\r\
a=rtcp-fb:39 nack pli\r\
a= fmtp:39 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=4d001f\r\
a=rtpmap:40 rtx/90000\r\
a=fmtp:40 apt=39\r\
a =rtpmap:41 H264/90000\r\
a=rtcp-fb:41 goog-remb\r\
a=rtcp-fb:41 transport-cc\r\
a=rtcp-fb:41 ccm fir\r\
a= rtcp-fb:41 nack\r\
a=rtcp-fb:41 nack pli\r\
a=fmtp:41 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=f4001f\r\ na=rtpmap:42 rtx/90000\r\
a=fmtp:42 apt=41\r\
a=rtpmap:43 H264/90000\r\
a=rtcp-fb:43 goog-remb\r\
a=rtcp- fb:43 transport-cc\r\
a=rtcp-fb:43 ccm fir\r\
a=rtcp-fb:43 nack\r\
a=rtcp-fb:43 nack pli\r\
a=fmtp:43 level -asymmetry-allowed=1;packetization-mode=0;profile-level-id=f4001f\r\
a=rtpmap:44 rtx/90000\r\
a=fmtp:44 apt=43\r\
a=rtpmap:45 AV1/90000\r\
a=rtcp-fb:45 goog-remb\r\
a=rtcp-fb:45 transport-cc\r\
a=rtcp-fb:45 ccm fir\r\
a =rtcp-fb:45 nack\r\
a=rtcp-fb:45 nack pli\r\
a=rtpmap:46 rtx/90000\r\
a=fmtp:46 apt=45\r\
a=rtpmap:47 AV1 /90000\r\
a=rtcp-fb:47 goog-remb\r\
a=rtcp-fb:47 transport-cc\r\
a=rtcp-fb:47 ccm fir\r\
a=rtcp-fb:47 nack\r\
a=rtcp-fb:47 nack pli\r\
a=fmtp:47 profile=1\r\
a=rtpmap:48 rtx/90000\r\
a=fmtp:48 apt=47\r\
a =rtpmap:123 H264/90000\r\
a=rtcp-fb:123 goog-remb\r\
a=rtcp-fb:123 transport-cc\r\
a=rtcp-fb:123 ccm fir\r\
a= rtcp-fb:123 nack\r\
a=rtcp-fb:123 nack pli\r\
a=fmtp:123 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=64001f\r\ na=rtpmap:119 rtx/90000\r\
a=fmtp:119 apt=123\r\
a=rtpmap:114 red/90000\r\
a=rtpmap:115 rtx/90000\r\
a=fmtp:115 apt =114\r\
a=rtpmap:116 ulpfec/90000\r\
a=rtpmap:49 flexfec-03/90000\r\
a=rtcp-fb:49 goog-remb\r\
a=rtcp-fb:49 transport -cc\r\
a=fmtp:49 repair-window=10000000\r\
"
}
The requested sdp is too long and will not be listed here.
srs as response from webrtc backend:
{
"code": 0,
"server": "vid-6714831",
"sdp": "v=0\r\
o=SRS/5.0.81(Bee) 4757130752 2 IN IP4 0.0.0.0\r\
s=SRSPlaySession\r\
t=0 0\r\
a=ice-lite\ r\
a=group:BUNDLE 0 1\r\
a=msid-semantic: WMS live/rtc-test\r\
m=audio 9 UDP/TLS/RTP/SAVPF 111\r\
c=IN IP4 0.0.0.0\ r\
a=ice-ufrag:36qb3jm5\r\
a=ice-pwd:4poj87d02hi4894s922p343n478684h6\r\
a=fingerprint:sha-256 D0:AD:CD:CE:DE:EA:49:01:BA:A3:D8 :40:C3:6E:7C:8B:84:AA:FC:61:4A:34:81:18:59:BD:57:EB:54:7F:23:00\r\
a=setup:passive \r\
a=mid:0\r\
a=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01\r\
a=sendonly\ r\
a=rtcp-mux\r\
a=rtcp-rsize\r\
a=rtpmap:111 opus/48000/2\r\
a=rtcp-fb:111 transport-cc\r\
a=ssrc:276242153 cname :902u91640g4b5a51\r\
a=ssrc:276242153 label:audio-4073vjgc\r\
a=candidate:0 1 udp 2130706431 192.168.31.242 8000 typ host generation 0\r\
m=video 9 UDP/SATLS/RTP r\
c=IN IP4 0.0.0.0\r\
a=ice-ufrag:36qb3jm5\r\
a=ice-pwd:4poj87d02hi4894s922p343n478684h6\r\
a=fingerprint:sha-256 D0:AD:CD:CE:DE:EA :49:01:BA:A3:D8:40:C3:6E:7C:8B:84:AA:FC:6 1:4A:34:81:18:59:BD:57:EB:54:7F:23:00\r\
a=setup:passive\r\
a=mid:1\r\
a=extmap:3 http ://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01\r\
a=sendonly\r\
a=rtcp-mux\r\
a=rtcp-rsize\r \
a=rtpmap:125 H264/90000\r\
a=rtcp-fb:125 transport-cc\r\
a=rtcp-fb:125 nack\r\
a=rtcp-fb:125 nack pli\r\
a= fmtp:125 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f\r\
a=ssrc:276242154 cname:902u91640g4b5a51\r\
a=ssrc:276242154 label:video-417lk9ha\r \
a=candidate:0 1 udp 2130706431 192.168.31.242 8000 typ host generation 0\r\
",
"sessionid": "36qb3jm5:pmt4"
}
"v=0\r\ o=SRS/5.0.81(Bee) 4757130752 2 IN IP4 0.0.0.0\r\ s=SRSPlaySession\r\ t=0 0\r\ a=ice-lite\r\ a=group:BUNDLE 0 1\r\ a=msid-semantic: WMS live/rtc-test\r\ m=audio 9 UDP/TLS/RTP/SAVPF 111\r\ c=IN IP4 0.0.0.0\r\ a=ice-ufrag:36qb3jm5\r\ a=ice-pwd:4poj87d02hi4894s922p343n478684h6\r\ a=fingerprint:sha-256 D0:AD:CD:CE:DE:EA:49:01:BA:A3:D8:40:C3:6E:7C:8B:84:AA:FC:61:4A:34 :81:18:59:BD:57:EB:54:7F:23:00\r\ a=setup:passive\r\ a=mid:0\r\ a=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01\r\ a=sendonly\r\ a=rtcp-mux\r\ a=rtcp-rsize\r\ a=rtpmap:111 opus/48000/2\r\ a=rtcp-fb:111 transport-cc\r\ a=ssrc:276242153 cname:902u91640g4b5a51\r\ a=ssrc:276242153 label:audio-4073vjgc\r\ a=candidate:0 1 udp 2130706431 192.168.31.242 8000 typ host generation 0\r\ m=video 9 UDP/TLS/RTP/SAVPF 125\r\ c=IN IP4 0.0.0.0\r\ a=ice-ufrag:36qb3jm5\r\ a=ice-pwd:4poj87d02hi4894s922p343n478684h6\r\ a=fingerprint:sha-256 D0:AD:CD:CE:DE:EA:49:01:BA:A3:D8:40:C3:6E:7C:8B:84:AA:FC:61:4A:34 :81:18:59:BD:57:EB:54:7F:23:00\r\ a=setup:passive\r\ a=mid:1\r\ a=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01\r\ a=sendonly\r\ a=rtcp-mux\r\ a=rtcp-rsize\r\ a=rtpmap:125 H264/90000\r\ a=rtcp-fb:125 transport-cc\r\ a=rtcp-fb:125 nack\r\ a=rtcp-fb:125 nack pli\r\ a=fmtp:125 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f\r\ a=ssrc:276242154 cname:902u91640g4b5a51\r\ a=ssrc:276242154 label:video-417lk9ha\r\ a=candidate:0 1 udp 2130706431 192.168.31.242 8000 typ host generation 0\r\ ",
ICE Signaling
Through packet analysis, after the HTTP signaling interaction is completed, the next step is the ICE signaling and DTLS process:
STUN process
STUN binding request format package, if you want to further analyze the package, you still need to refer to the STUN protocol format:
0000 00 01 00 50 21 12 a4 42 79 73 48 6e 2f 4c 41 62 ...P!..BysHn/LAb
0010 45 55 5a 6e 00 06 00 0d 33 36 71 62 33 6a 6d 35 EUZn....36qb3jm5
0020 3a 70 6d 74 34 00 00 00 c0 57 00 04 00 00 03 e7 :pmt4....W...
0030 80 2a 00 08 40 93 5b 78 d1 33 e6 f0 00 24 00 04 .*..@.[x.3...$..
0040 6e 00 1e ff 00 08 00 14 16 4b bf d5 d2 8a f6 e2 n...K...
0050 2f 95 ec 4c be 98 f3 25 b1 47 03 7b 80 28 00 04 /..L...%.G.{.(..
0060 67 d5 6c a0 g.l.
The Binding Request User here is:
36qb3jm5:pmt4
STUN binding response format package:
0000 01 01 00 40 21 12 a4 42 79 73 48 6e 2f 4c 41 62 ...@!..BysHn/LAb 0010 45 55 5a 6e 00 06 00 0d 33 36 71 62 33 6a 6d 35 EUZn....36qb3jm5 0020 3a 70 6d 74 34 00 00 00 00 20 00 08 00 01 eb ea :pmt4... ? 0030 e1 ba bb b0 00 08 00 14 74 24 c4 1a 65 e7 8e d8 .....t$..e... 0040 dc b9 af 22 ab 47 b6 9a 2b 2d 8a 79 80 28 00 04 ...".G.. + -.y.(.. 0050 46 66 4b 19 FfK.
The Binding Success Response User candidate here is:
XOR-MAPPED-ADDRESS 192.168.31.242:51960
DTLS process
The next step is the DTLS key exchange process, which will not be analyzed in detail here, but another analysis can be started.
audio and video data
Next, analyze the audio and video data flow of RTC.
Through packet analysis, after HTTP signaling & amp;STUN signaling & amp;DTLS key exchange is successful, the next step is RTC audio and video SRTP packets and their SRTCP control packets (UDP audio and video packets need to be decoded as SRTP via wireshark):
The payload type of vio is 125, and that of audio is 111. The ssrc of video is 0x10771eea, and the ssrc of audio is 0x10771ee9.
Except that the payload part is encrypted and cannot be viewed directly, the SRTP header part is easy to understand.
Summary
The WebRTC process of SRS is mainly HTTP signaling & amp;STUN signaling & amp;DTLS key exchange & amp; SRTP/SRTCP audio and video data and control packets.
Attach the complete message
Link: SRS5.0-WebRTC plays the complete message
Extraction code: 6x7r