The experimental environment of this article is based on: CentOS Linux release 7.6.1810 (Core)
Reference: Linux ipip tunnel and implementation – the road to cloud native
1. Introduction to IP tunnel technology
There are three main types of IP tunneling technologies implemented by the Linux system kernel (protocols or software such as PPP, PPTP, and L2TP are not based on kernel modules): ipip, gre, and sit. All three tunneling technologies require the support of the kernel module tunnel4.ko.
-
ipip
requires the kernel module ipip.ko. This method is the simplest! But you cannot forward broadcast or IPv6 packets through IP-in-IP tunnels. You are just connecting two IPv4 networks that normally cannot communicate directly. As for compatibility, this part of the code has a long history, and its compatibility can be traced back to version 1.3 of the kernel. According to information found on the Internet, Linux’s IP-in-IP tunnel cannot communicate with other operating systems or routers. It’s simple and effective. -
GRE
requires the kernel module ip_gre.ko. GRE is a tunnel protocol originally developed by CISCO. It can do things that IP-in-IP tunnels cannot. For example, you can use GRE tunnels to transmit multicast packets and IPv6 packets. -
sit
Its function is to connect ipv4 and ipv6 networks. Personally, I feel that it is not as widely used as gre.
2. Testing requirements
As shown above, two virtual machines are created.
The IPs of the two network cards of host A: 192.168.155.10, 192.168.154.11
The IPs of the two network cards of host B: 192.168.156.10, 192.168.154.12
Finally, I hope that host A can ping the 192.168.156.10 address of host B, and host B can ping the 192.168.155.10 address of host A.
3. Implementation 1: Using ip_gre
3.1 Execute the following commands on host A
# The gre0 and gretap0 network devices will be added automatically. The devices will disappear after restarting the machine. modprobe ip_gre # Manually add the tun0 tunnel device. The tunnel uses gre mode. The connected remote address is 192.168.154.12 and the local address is 192.168.154.11. ip tunnel add tun0 mode gre remote 192.168.154.12 local 192.168.154.11 ttl 64 # Turn on tun0 ip link set tun0 up # Set the routing configuration of 192.168.156.0/24 ip route add 192.168.156.0/24 dev tun0
Note: The ip route add 192.168.156.0/24 dev tun0
command can be replaced by the ip addr add 192.168.155.10 peer 192.168.156.10 dev tun0
command. This name sets the tun0 device The IP address is 192.168.155.10 (this is consistent with the IP of the ensp0s3 device, or it can be set to be inconsistent), and the IP address of the point-to-point connection is set to 192.168.156.10. At the same time, this step will automatically add a route 192.168.156.10 dev tun0 proto kernel scope link src 192.168.155.10
. The key is that the automatically generated route works.
3.2 Execute the following commands on host B
modprobe ip_gre ip tunnel add tun0 mode gre remote 192.168.154.11 local 192.168.154.12 ttl 64 ip link set tun0 up ip route add 192.168.155.0/24 dev tun0 iptables -F
Similarly: the ip route add 192.168.155.0/24 dev tun0
command can be replaced by the ip addr add 192.168.156.10 peer 192.168.155.10 dev tun0
command
3.3 Test
Execute tests on host A
ping 192.168.156.10
Execute tests on host B
ping 192.168.155.10
3.4 Data packet flow
Next, execute ping 192.168.156.10 on host A to analyze the flow of data packets, and capture and analyze the packets on hosts A and B respectively.
3.4.1 Packet Capture Analysis of Host A
Grab the data of enp0s8
tcpdump -i enp0s8 -ne 'host not 192.168.154.1'
Fetch the data results of enp0s8
17:30:12.465060 08:00:27:01:7f:b9 > 08:00:27:e4:97:96, ethertype IPv4 (0x0800), length 122: 192.168.154.11 > 192.168.154.12: GREv0 , proto IPv4 (0x0800), length 88: 192.168.155.10 > 192.168.156.10: ICMP echo request, id 3848, seq 1, length 64 17:30:12.465463 08:00:27:e4:97:96 > 08:00:27:01:7f:b9, ethertype IPv4 (0x0800), length 122: 192.168.154.12 > 192.168.154.11: GREv0, proto IPv4 (0x0800), length 88: 192.168.156.10 > 192.168.155.10: ICMP echo reply, id 3848, seq 1, length 64 17:30:17.472994 08:00:27:01:7f:b9 > 08:00:27:e4:97:96, ethertype ARP (0x0806), length 42: Request who-has 192.168.154.12 tell 192.168.154.11, length 28 17:30:17.473161 08:00:27:e4:97:96 > 08:00:27:01:7f:b9, ethertype ARP (0x0806), length 60: Reply 192.168.154.12 is-at 08:00:27 :e4:97:96, length 46
-
The data flow direction found in the data packet of the network card enp0s8 is
192.168.154.11 > 192.168.154.12
, which is the point-to-point IP address of the IP tunnel. The protocol isGREv0
and the length is88
. The content of the data is192.168.155.10 > 192.168.156.10: ICMP echo request, id 3848, seq 1, length 64
. From this data, we can see that the IP tunnel encapsulates an IP header in the entire packet of the original data (including the IP header). This confirms thatIP tunneling is a technology that encapsulates one IP packet in another IP packet
.
Grab the data of tun0
tcpdump -i tun0 -ne
Grab the data result of tun0
17:30:12.465050 Out ethertype IPv4 (0x0800), length 100: 192.168.155.10 > 192.168.156.10: ICMP echo request, id 3848, seq 1, length 64 17:30:12.465500 In ethertype IPv4 (0x0800), length 100: 192.168.156.10 > 192.168.155.10: ICMP echo reply, id 3848, seq 1, length 64
-
Here you can see that the data received and returned by tun0
does not have the IP tunnel header
, but is directly the original destination address and source address (192.168.156.10 and 192.168.155.10)
3.4.2 B host packet capture analysis
Grab the data of enp0s8
tcpdump -i enp0s8 -ne 'host not 192.168.154.1'
Fetch the data results of enp0s8
17:30:14.410721 08:00:27:01:7f:b9 > 08:00:27:e4:97:96, ethertype IPv4 (0x0800), length 122: 192.168.154.11 > 192.168.154.12: GREv0 , proto IPv4 (0x0800), length 88: 192.168.155.10 > 192.168.156.10: ICMP echo request, id 3848, seq 1, length 64 17:30:14.410840 08:00:27:e4:97:96 > 08:00:27:01:7f:b9, ethertype IPv4 (0x0800), length 122: 192.168.154.12 > 192.168.154.11: GREv0, proto IPv4 (0x0800), length 88: 192.168.156.10 > 192.168.155.10: ICMP echo reply, id 3848, seq 1, length 64 17:30:19.418512 08:00:27:01:7f:b9 > 08:00:27:e4:97:96, ethertype ARP (0x0806), length 60: Request who-has 192.168.154.12 tell 192.168.154.11, length 46 17:30:19.418523 08:00:27:e4:97:96 > 08:00:27:01:7f:b9, ethertype ARP (0x0806), length 42: Reply 192.168.154.12 is-at 08:00:27 :e4:97:96, length 28
Grab the data of tun0
tcpdump -i tun0 -ne
Grab the data result of tun0
17:30:14.410795 In ethertype IPv4 (0x0800), length 100: 192.168.155.10 > 192.168.156.10: ICMP echo request, id 3848, seq 1, length 64 17:30:14.410829 Out ethertype IPv4 (0x0800), length 100: 192.168.156.10 > 192.168.155.10: ICMP echo reply, id 3848, seq 1, length 64
-
Note that
the packet of the destination network card enp0s3 (192.168.156.10) is not captured here
. The possible reason is that GRE processes the data in the protocol stack, so tcpdump cannot capture the enp0s3 package. Another possible reason is that the logical outflow direction ofenp0s3 should be to the network cable, not the protocol stack
, so tcpdump cannot capture the enp0s3 packet.
4. Implementation 2: Using IPIP
4.1 Execute the following commands on host A
modprobeipip ip tunnel add tun0 mode ipip remote 192.168.154.12 local 192.168.154.11 ttl 64 ip link set tun0 up ip route add 192.168.156.0/24 dev tun0
4.2 Execute the following commands on host B
modprobeipip ip tunnel add tun0 mode ipip remote 192.168.154.11 local 192.168.154.12 ttl 64 ip link set tun0 up ip route add 192.168.155.0/24 dev tun0
4.3 Test, data packet flow
Similar to ip_gre mode.
5. Implementation 3: ipip one-to-many configuration
Reference: https://morven.life/notes/networking-3-ipip/
5.1 Execute the following commands on host A
modprobeipip # After ipip is enabled, tunl0 will be created by default, so we will not build the tun0 device here. It should be that a tunnel device with an IP address of 0.0.0.0 and a brd broadcast address of 0.0.0.0 can only be established once in a host. ip link set tunl0 up # Here the gateway to access the 192.168.156.0/24 network segment is set to 192.168.154.12. After setting, the tunl0 tunnel will forward the data to 192.168.154.12 through the routing table configuration. ip route add 192.168.156.0/24 via 192.168.154.12 dev tunl0 onlink
5.2 Execute the following commands on host B
modprobeipip ip link set tunl0 up ip route add 192.168.155.0/24 via 192.168.154.11 dev tunl0 onlink
5.3 Test
Consistent with previous test results
5.4 Data packet flow
The flow direction is consistent with the previous packet. Just need to replace the tunl0 device with the tun0 device.
5.4.1 Packet capture data from host A
# enp0s8 network card packet capture command tcpdump -i enp0s8 -ne 'host not 192.168.154.1' # enp0s8 network card packet capture data 11:00:47.369825 08:00:27:01:7f:b9 > 08:00:27:e4:97:96, ethertype IPv4 (0x0800), length 118: 192.168.154.11 > 192.168.154.12: 192.168.155.10 > 192.168.156.10: ICMP echo request, id 3890, seq 1, length 64 (ipip-proto-4) 11:00:47.370130 08:00:27:e4:97:96 > 08:00:27:01:7f:b9, ethertype IPv4 (0x0800), length 118: 192.168.154.12 > 192.168.154.11: 192.168.156.10 > 192.168.155.10: ICMP echo reply, id 3890, seq 1, length 64 (ipip-proto-4) # tunl0 network card packet capture command tcpdump -i tunl0 -n # tunl0 network card packet capture data 11:00:47.369813 ip: 192.168.155.10 > 192.168.156.10: ICMP echo request, id 3890, seq 1, length 64 11:00:47.370197 ip: 192.168.156.10 > 192.168.155.10: ICMP echo reply, id 3890, seq 1, length 64
5.4.2 Packet capture data from host B
# enp0s8 network card packet capture command tcpdump -i enp0s8 -ne 'host not 192.168.154.1' # enp0s8 network card packet capture data 11:00:47.460654 08:00:27:01:7f:b9 > 08:00:27:e4:97:96, ethertype IPv4 (0x0800), length 118: 192.168.154.11 > 192.168.154.12: 192.168.155.10 > 192.168.156.10: ICMP echo request, id 3890, seq 1, length 64 (ipip-proto-4) 11:00:47.460727 08:00:27:e4:97:96 > 08:00:27:01:7f:b9, ethertype IPv4 (0x0800), length 118: 192.168.154.12 > 192.168.154.11: 192.168.156.10 > 192.168.155.10: ICMP echo reply, id 3890, seq 1, length 64 (ipip-proto-4) 11:00:52.476786 08:00:27:01:7f:b9 > 08:00:27:e4:97:96, ethertype ARP (0x0806), length 60: Request who-has 192.168.154.12 tell 192.168.154.11, length 46 11:00:52.476801 08:00:27:e4:97:96 > 08:00:27:01:7f:b9, ethertype ARP (0x0806), length 42: Reply 192.168.154.12 is-at 08:00:27 :e4:97:96, length 28 # tunl0 network card packet capture command tcpdump -i tunl0 -ne # tunl0 network card packet capture data 11:00:47.460701 ip: 192.168.155.10 > 192.168.156.10: ICMP echo request, id 3890, seq 1, length 64 11:00:47.460719 ip: 192.168.156.10 > 192.168.155.10: ICMP echo reply, id 3890, seq 1, length 64
5.5 Principle Analysis
The key here for one-to-many IP tunnels is
-
- The IP of the one-to-many IP tunnel device is 0.0.0.0, and the broadcast address is also 0.0.0.0. The point-to-point IP tunnel we set up before, such as
remote 192.168.154.12 local 192.168.154.11
. You can check the difference between the two setting results through ip a: tunl0 is set tolink/ipip 0.0.0.0 brd 0.0.0.0
, and tun0 is set tolink/ipip 192.168.154.11 peer 192.168. 154.12
.
- The IP of the one-to-many IP tunnel device is 0.0.0.0, and the broadcast address is also 0.0.0.0. The point-to-point IP tunnel we set up before, such as
4: tunl0@NONE: <NOARP,UP,LOWER_UP> mtu 1480 qdisc noqueue state UNKNOWN group default qlen 1000 link/ipip 0.0.0.0 brd 0.0.0.0 6: tun0@NONE: <POINTOPOINT,NOARP> mtu 1480 qdisc noop state DOWN group default qlen 1000 link/ipip 192.168.154.11 peer 192.168.154.12
-
- One-to-many IP tunnels need to specify the gateway
192.168.156.0/24 via 192.168.154.12 dev tunl0 onlink
when setting up routing, so that data can be sent to the gateway using the configured routing protocol in the tunnel. When configuring a one-to-one tunnel, there is no need to specify the routing gateway, such as192.168.156.0/24 dev tun0
, becauseremote 192.168.154.12 local 192.168.154.11< is specified in the tunnel. /code>.
- One-to-many IP tunnels need to specify the gateway
Linux ipip tunnel technology test 1 (dual hosts, dual network cards) - Code Positive Energy