Table of Contents
Introduction
Create Linux users and user groups
Configure LINUX static IP
Edit IP mapping
SSH password-free login configuration
Login test
Introduction
SSH (Secure shell) is a protocol that can provide secure communication in applications. Network data can be transmitted securely through SSH. Its main principle is to use an asymmetric encryption system to encrypt all data to be transmitted to ensure that the data is transmitted safely. It will not be maliciously destroyed, leaked or tampered with during transmission. However, the big data cluster (mainly refers to the Hadoop cluster, and can be referred to as Hadoop) uses SSH not mainly for data transmission, but when the Hadoop cluster starts or stops, the master node needs to use the SSH protocol to slave the process on the node. Start or stop. If you do not configure SSH password-free login, it will have no impact on the normal use of the Hadoop cluster. However, when starting or stopping the Hadoop cluster, you need to enter the password of each slave node user name. You can imagine that when the cluster scale is relatively large, such as reaching hundreds or thousands of nodes, it will be quite troublesome if you have to enter the password of the cluster node each time. This method is definitely not advisable, so you need to perform the Hadoop cluster SSH password-free login configuration, and SSH is currently most commonly used in remote management environments.
The function of SSH password-free login is closely related to the user. For which user SSH is configured, that user will have the function of SSH password-free login. Users who are not configured will not have this function. Here we choose to configure SSH password-free login for the hadoop user.
Create Linux users and user groups
In the process of building a big data platform, for system security reasons, the super user root is generally not used directly. Instead, a new user and user group need to be created. In Linux systems, you can directly use the groupadd command to create a new user group. The method of using groupadd is as shown in the figure below.
Enter the command groupadd hadoop on the console to create a user group named hadoop. In Linux systems, you can directly use the useradd command to create a new user. The usage of useradd is as shown in the figure below.
Enter the command useradd -g hadoop hadoop on the console to create a user named hadoop and specify the user group as hadoop. In addition, you can use the passwd command under the root user to set the password for the hadoop user you just created. The password can be set by yourself. The specific operation is as shown in the figure.
[root@vm06 ~]# groupadd hadoop [root@vm06 ~]# useradd -g hadoop hadoop
Configure LINUX static IP
In practical applications, since Dynamic Host Configuration Protocol (DHCP) servers are often used to allocate IP addresses, the IP address may change every time the DHCP server is restarted. When using a Linux system to build a big data platform, it is hoped that the IP address will be fixed. Because many places in the cluster configuration involve IP addresses, the Linux system needs to be configured as a static IP. So how to configure it?
Taking the vm06 node as an example, first enter the command vim /etc/sysconfig/network-scripts/ifcfg-ens33 on the console to open the configuration file, and then modify the network card information of the vm06 node. The specific configuration is as shown in the figure.
Add DNS1=8.8.8.8
Change the BOOTPROTO parameter from dhcp to static, which means changing the dynamic IP to a static IP;
Add a fixed IP address IPADDR to 10.0.0.106;
Add the subnet mask NETMASK to 255.255.255.0;
The gateway GATEWAY is set to 10.0.0.254. (You can select Edit→Virtual Network Editor option in the VMware Workstation navigation bar to view the network segment and gateway you have set)
After modifying the network card configuration, you need to enter the command systemctl restart network on the console to restart the network service to take effect.
Edit IP mapping
In fact, both the IP address and the host name are used to identify a host or server. An IP address is a logical address assigned to a host by the IP protocol when it accesses the Internet. The host name is equivalent to giving the machine a name. Various names can be given to the host. If you want to access a host by name, how does the system identify a host by name? This requires configuring the correspondence between hostname and IP address. Enter the command vi /etc/hosts on the console to open the configuration file, and add the corresponding relationship between the IP address and the host name at the end of the hosts file according to the corresponding format. At this time, the IP address is 10.0.0.105, and the corresponding hostname is vm05. Note that there must be a space between them. The specific configuration results are shown in the figure
SSH password-free login configuration
First, in the console, use the su command to switch to the hadoop user. The specific operation is shown in the figure below. Use the mkdir command to create the .ssh directory in the root directory of the hadoop user, and use the command ssh-keygen-t rsa (ssh-keygen is a key generator, -t is a parameter, rsa is an encryption algorithm) to generate a key pair (i.e. public key file id_rsa.pub and private key file id_rsa),
[hadoop@vm06 ~]$ mkdir -p .ssh [hadoop@vm06 ~]$ ll -a total 12 drwx------ 3 hadoop hadoop 74 Nov 13 19:58 . drwxr-xr-x. 4 root root 30 Nov 13 19:53 .. -rw-r--r-- 1 hadoop hadoop 18 Nov 25 2021 .bash_logout -rw-r--r-- 1 hadoop hadoop 193 Nov 25 2021 .bash_profile -rw-r--r-- 1 hadoop hadoop 231 Nov 25 2021 .bashrc drwxrwxr-x 2 hadoop hadoop 6 Nov 13 19:58 .ssh
Copy the contents of the public key file id_rsa.pub to the authorized_keys file in the same directory. The specific operation is as shown in the figure.
[hadoop@vm06 .ssh]$ ll total 8 -rw------- 1 hadoop hadoop 1679 Nov 13 20:00 id_rsa -rw-r--r-- 1 hadoop hadoop 393 Nov 13 20:00 id_rsa.pub [hadoop@vm06 .ssh]$ cp id_rsa.pub authorized_keys [hadoop@vm06 .ssh]$ll total 12 -rw-r--r-- 1 hadoop hadoop 393 Nov 13 20:05 authorized_keys -rw------- 1 hadoop hadoop 1679 Nov 13 20:00 id_rsa -rw-r--r-- 1 hadoop hadoop 393 Nov 13 20:00 id_rsa.pub [hadoop@vm06 .ssh]$ cat authorized_keys ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDq0fksmfO4IKYl7gp87JdgU6SEo2l/1 + pd4XCsPCzV1PEm + WXP0MWm2TEnjYbD6wBeQaoA5cXCMCt/rNT19lLAL4tZhKeBUJBKln4OrwcPlwVEihBiDC 2QDYeEqtlJTGWrYpBECET3Tg3tlBkLWmDf4maYNdDjKLOEbeEl2gZ + vmyNyuO3OrIoIf0Y3ZuDqZtHZl9VQPSacPcAeRJ3KUEO56bP4mT7EgyF44GS2fzRIFnvXfrqqcYPCvS9KSinMXXSXDSa13Lp ducKJnJK4xQw/7 + Ulj1nYiQ31lF561F7RMlLwEssy9X7X46kxMJcO9Hrqn7YVeYT0MAxOCsI1mJV hadoop@vm06 [hadoop@vm06 .ssh]$
Summarize the authorized_keys key contents of each node and then distribute them to each node
[hadoop@vm05 .ssh]$ scp authorized_keys vm06:/home/hadoop/.ssh/ The authenticity of host 'vm06 (10.0.0.106)' can't be established. ECDSA key fingerprint is SHA256:6x51ixbdcREkjq5gPhLOrvwDUoDUWPXUFU0CqE + 3E5Y. ECDSA key fingerprint is MD5:52:99:a2:2e:c9:a1:09:4b:28:48:33:ff:c6:c2:e2:10. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'vm06,10.0.0.106' (ECDSA) to the list of known hosts. hadoop@vm06's password: authorized_keys 100% 1180 1.4MB/s 00:00 [hadoop@vm05 .ssh]$ scp authorized_keys vm07:/home/hadoop/.ssh/ The authenticity of host 'vm07 (10.0.0.107)' can't be established. ECDSA key fingerprint is SHA256:6x51ixbdcREkjq5gPhLOrvwDUoDUWPXUFU0CqE + 3E5Y. ECDSA key fingerprint is MD5:52:99:a2:2e:c9:a1:09:4b:28:48:33:ff:c6:c2:e2:10. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'vm07,10.0.0.107' (ECDSA) to the list of known hosts. hadoop@vm07's password: authorized_keys 100% 1180 1.2MB/s 00:00 [hadoop@vm05 .ssh]$
Switch to the root directory of the hadoop user, and then grant corresponding permissions to the .ssh directory and files. The specific operations are as shown in the figure.
[hadoop@vm06 ~]$ ll -a total 12 drwx------ 3 hadoop hadoop 74 Nov 13 20:00 . drwxr-xr-x. 4 root root 30 Nov 13 19:53 .. -rw-r--r-- 1 hadoop hadoop 18 Nov 25 2021 .bash_logout -rw-r--r-- 1 hadoop hadoop 193 Nov 25 2021 .bash_profile -rw-r--r-- 1 hadoop hadoop 231 Nov 25 2021 .bashrc drwxrwxr-x 2 hadoop hadoop 38 Nov 13 20:00 .ssh [hadoop@vm06 ~]$ cd .ssh/ [hadoop@vm06 .ssh]$ll total 8 -rw------- 1 hadoop hadoop 1679 Nov 13 20:00 id_rsa -rw-r--r-- 1 hadoop hadoop 393 Nov 13 20:00 id_rsa.pub [hadoop@vm06 .ssh]$ cd .. [hadoop@vm06 ~]$ chmod 700 .ssh/ [hadoop@vm06 ~]$ chmod 600 .ssh/* [hadoop@vm06 ~]$ ll -a total 12 drwx------ 3 hadoop hadoop 74 Nov 13 20:00 . drwxr-xr-x. 4 root root 30 Nov 13 19:53 .. -rw-r--r-- 1 hadoop hadoop 18 Nov 25 2021 .bash_logout -rw-r--r-- 1 hadoop hadoop 193 Nov 25 2021 .bash_profile -rw-r--r-- 1 hadoop hadoop 231 Nov 25 2021 .bashrc drwx------ 2 hadoop hadoop 61 Nov 13 20:05 .ssh [hadoop@vm06 ~]$ cd .ssh [hadoop@vm06 .ssh]$ll total 12 -rw------- 1 hadoop hadoop 393 Nov 13 20:05 authorized_keys -rw------- 1 hadoop hadoop 1679 Nov 13 20:00 id_rsa -rw------- 1 hadoop hadoop 393 Nov 13 20:00 id_rsa.pub
Login Test
Use the ssh command to log in to hadoop01. You need to enter yes for confirmation for the first login. This is not required for the second and subsequent logins. This indicates that the setting is successful. The specific operations are as shown in the figure.
[hadoop@vm05 .ssh]$ ssh vm06 Last login: Mon Nov 13 21:12:13 2023 [hadoop@vm06 ~]$ ssh vm07 Last login: Mon Nov 13 21:12:14 2023 [hadoop@vm07 ~]$