Minio distributed cluster deployment (4 nodes 4 clusters)

Centos7.6

Minio

Minio distributed cluster deployment (4 nodes 4 clusters)
1.Environment preparation
2-Add new disk and mount it
3. Install minio cluster
4. Configure systemctl system service
5. Login page testing

1. Environment preparation

4 servers with multiple hard drives

 Node data directory (cannot be under /root) process/script directory
192.168.156.2 /data1/minio/{<!-- -->data1.data2} /opt/apps/minio
192.168.156.3 /data1/minio/{<!-- -->data1.data2} /opt/apps/minio
192.168.156.4 /data1/minio/{<!-- -->data1.data2} /opt/apps/minio
192.168.156.5 /data1/minio/{<!-- -->data1.data2} /opt/apps/minio

4 servers with single hard drive (this time)

 Node data directory (cannot be under /root) process/script directory
192.168.156.2 /data1/minio/data /opt/apps/minio
192.168.156.3 /data1/minio/data /opt/apps/minio
192.168.156.4 /data1/minio/data /opt/apps/minio
192.168.156.5 /data1/minio/data /opt/apps/minio

Time guarantee synchronization All nodes

[root@localhost ~]# date

#If the time changes,
[root@localhost ~]# date -s 17:08

Modify the host name and hosts file All nodes

#Permanently modify the host name
[root@localhost ~]# hostnamectl set-hostname sp-001
[root@localhost ~]# bash

#Modify mapping
[root@sd-001 ~]# vim /etc/hosts
#Add to
192.168.156.2 sp-001
192.168.156.3 sp-002
192.168.156.4 sp-003
192.168.156.5 sp-004

Prepare installation package

#Download address
wget https://dl.minio.io/server/minio/release/linux-amd64/minio

[root@sp-001 ~]# ll
-rw-r--r-- 1 root root 94044160 Nov 9 12:13 minio

2-Add new disk and mount

List information about all available block devices

[root@docker01 ~]# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sdb 8:16 0 22G 0 disk #Such as the disk just added

new disk partition

[root@docker01 ~]# fdisk /dev/sdb

Command (enter m for help): m #m

Command (enter m for help): n #n

Select (default p): p #p
Partition number (1-4, default 1): 1 #1
Starting sector (2048-46137343, default is 2048): #Enter
Last sector, + sector (2048-46137343, default is 46137343): #Enter (all allocated)


Command (enter m for help): w #Save

View disk information

[root@docker01 ~]# lsblk -f
NAME FSTYPE LABEL UUID MOUNTPOINT
sdb
└─sdb1 #Existing sdb1 partition

Format disk

[root@docker01 ~]# mkfs -t ext4 /dev/sdb1

mount

[root@docker01 ~]# mkdir /data1
[root@docker01 ~]# mount /dev/sdb2 /data1/

#View mounting information
[root@docker01 ~]# df -h
File System Capacity Used Available % Used Mount Points
/dev/sdb2 22G 45M 21G 1% /data1 #21G has been mounted to /data1

Automatically mount on boot

[root@minio-1 ~]# vi /etc/rc.d/rc.local
#Add to:
mount /dev/vdb1 /data1/

Give the file execution permission

[root@minio-1 ~]# chmod + x /etc/rc.d/rc.local

3. Install minio cluster

Create the installation directory First node

[root@sp-001 ~]# mkdir -p /opt/apps/minio

Move the installation package to the installation directory and other servers First node

[root@sp-001 ~]# mv minio /opt/apps/minio/
[root@sp-001 ~]# scp -r /opt/apps/minio/ 192.168.156.3:/opt/apps/
[root@sp-001 ~]# scp -r /opt/apps/minio/ 192.168.156.4:/opt/apps/
[root@sp-001 ~]# scp -r /opt/apps/minio/ 192.168.156.5:/opt/apps/

Create directory All nodes

[root@sp-001 ~]# mkdir -p /data1/minio/data/ #Data directory
[root@sp-001 ~]# mkdir -p /data1/minio/logs/ #Log directory
[root@sp-001 ~]# mkdir -p /etc/minio #Configuration file directory

Cluster startup file description

#1.Minio defaults to port 9000. Add –address “127.0.0.1:9029” to the configuration file to change the port.
#2.MINIO_ACCESS_KEY: User name, the minimum length is 5 characters
#3.MINIO_SECRET_KEY: Password. The password cannot be set too simple, otherwise minio will fail to start. The minimum length is 8 characters.
#4.–config-dir: Specify the cluster configuration file directory

Node 1 startup script

[root@sp-001 ~]# vim /opt/apps/minio/minio-run.sh
#新add
#!/bin/bash
export MINIO_ACCESS_KEY=mhsp
export MINIO_SECRET_KEY=mhsp@1234
/opt/apps/minio/minio server --config-dir /etc/minio \
--address "192.168.156.2:9000" \
http://192.168.156.2/data1/minio/data \
http://192.168.156.3/data1/minio/data \
http://192.168.156.4/data1/minio/data \
http://192.168.156.5/data1/minio/data > /data/minio/logs/minio.log

Node 2 startup script

[root@sp-002 ~]# vim /opt/apps/minio/minio-run.sh
#新add
#!/bin/bash
export MINIO_ACCESS_KEY=mhsp
export MINIO_SECRET_KEY=mhsp@1234
/opt/apps/minio/minio server --config-dir /etc/minio \
--address "192.168.156.3:9000" \
http://192.168.156.2/data1/minio/data \
http://192.168.156.3/data1/minio/data \
http://192.168.156.4/data1/minio/data \
http://192.168.156.5/data1/minio/data > /data/minio/logs/minio.log

Node 3 startup script

[root@sp-003 ~]# vim /opt/apps/minio/minio-run.sh
#新add
#!/bin/bash
export MINIO_ACCESS_KEY=mhsp
export MINIO_SECRET_KEY=mhsp@1234
/opt/apps/minio/minio server --config-dir /etc/minio \
--address "192.168.156.4:9000" \
http://192.168.156.2/data1/minio/data \
http://192.168.156.3/data1/minio/data \
http://192.168.156.4/data1/minio/data \
http://192.168.156.5/data1/minio/data > /data/minio/logs/minio.log

Node 4 startup script

[root@sp-004 ~]# vim /opt/apps/minio/minio-run.sh
#新add
#!/bin/bash
export MINIO_ACCESS_KEY=mhsp
export MINIO_SECRET_KEY=mhsp@1234
/opt/apps/minio/minio server --config-dir /etc/minio \
--address "192.168.156.5:9000" \
http://192.168.156.2/data1/minio/data \
http://192.168.156.3/data1/minio/data \
http://192.168.156.4/data1/minio/data \
http://192.168.156.5/data1/minio/data > /data/minio/logs/minio.log

Multi-hard disk (multi-directory) startup script – sample

#!/bin/bash
export MINIO_ACCESS_KEY=admin
export MINIO_SECRET_KEY=admin123
/home/minio/run/minio server --config-dir /etc/minio \
--address "10.34.252.93:9000" \
http://10.34.252.87/home/minio/data1 http://10.34.252.87/home/minio/data2 \
http://10.34.252.88/home/minio/data1 http://10.34.252.88/home/minio/data2 \
http://10.34.252.89/home/minio/data1 http://10.34.252.89/home/minio/data2 \
http://10.34.252.93/home/minio/data1 http://10.34.252.93/home/minio/data2

4. Configure systemctl system service

Create Minio.server and add minio to the system service All nodes

[root@sp-001 ~]# vim /etc/systemd/system/minio.service
#新add
[Unit]
Description=Minio service
Documentation=https://docs.minio.io/

[Service]
WorkingDirectory=/opt/apps/minio/
ExecStart=/opt/apps/minio/minio-run.sh

Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

Load configuration All nodes

[root@sp-001 ~]# systemctl daemon-reload

Permission modification All nodes

[root@sp-001 ~]# chmod + x /opt/apps/minio/minio & amp; & amp; chmod + x /opt/apps/minio/minio-run.sh

Start the cluster and automatically start all nodes

[root@sp-001 ~]# systemctl start minio
[root@sp-001 ~]# systemctl enable minio

View cluster status

root@sp-001 ~]# systemctl status minio.service -l
● minio.service - Minio service
   Loaded: loaded (/etc/systemd/system/minio.service; enabled; vendor preset: disabled)
   Active: active (running) since Tue 2021-11-23 21:32:06 CST; 14h ago
     Docs: https://docs.minio.io/
 Main PID: 19733 (minio-run.sh)
   CGroup: /system.slice/minio.service
           ├─19733 /bin/bash /opt/apps/minio/minio-run.sh
           └─19758 /opt/apps/minio/minio server --config-dir /etc/minio --address 192.168.156.2:9000 http://192.168.156.2/data/minio/data http://192.168.156.3/ data/minio/data http://192.168.156.4/data/minio/data http://192.168.156.5/data/minio/data

Nov 24 sp-001 minio-run.sh[19733]: Error: Disk: http://192.168.156.3:9000/data/minio/data returned disk not found (*fmt.wrapError)
Nov sp-001 minio-run.sh[19733]: endpoint=http://192.168.156.3:9000/data/minio/data
Nov sp-001 minio-run.sh[19733]: 2: cmd/prepare-storage.go:52:cmd.glob..func7.1()
Nov sp-001 minio-run.sh[19733]: 1: cmd/erasure-sets.go:218:cmd.(*erasureSets).connectDisks.func1()
Nov sp-001 minio-run.sh[19733]: Client http://192.168.156.4:9000/minio/lock/v6 online
Nov sp-001 minio-run.sh[19733]: Client http://192.168.156.4:9000/minio/s.../data/v29 online
Nov sp-001 minio-run.sh[19733]: Client http://192.168.156.5:9000/minio/lock/v6 online
Nov sp-001 minio-run.sh[19733]: Client http://192.168.156.5:9000/minio/st.io/data/v29 online
Nov sp-001 minio-run.sh[19733]: Client http://192.168.156.3:9000/minio/storage.o/data/v29 online
Nov sp-001 minio-run.sh[19733]: Client http://192.168.156.3:9000/minio/lock/v6 online

5. Login page test

Visit any address

http://192.168.156.2:9000
http://192.168.156.3:9000
http://192.168.156.4:9000
http://192.168.156.5:9000

Create bucket

>Upload files in the newly created bucket

Check the four minio data directories (the data storage directory cannot be the /root directory, otherwise an error will be reported)

#Enter the data directory
[root@hhht-1 ~]# cd /data1/minio/data/

#The c1-n1-bk1 directory just created exists
[root@hhht-1 data]#ll
drwxr-xr-x 5 root root 4096 Nov 8 12:00 c1-n1-bk1

#Enter the directory, the uploaded file exists
[root@hhht-1 data]# cd c1-n1-bk1/
[root@hhht-1 c1-n1-bk1]# ll
drwxr-xr-x 2 root root 4096 Nov 8 11:56 hosts
drwxr-xr-x 3 root root 4096 Nov 8 12:00 redis-3.0.0.gem
drwxr-xr-x 2 root root 4096 Nov 8 11:59 Inner Mongolia host table.xlsx

#Check that the c1-n1-bk1 directories and files on the other three servers exist