This cluster construction is based on Elasticsearch 7.8.1
Download address (Elastic Chinese community): https://elasticsearch.cn/download/#seg-12
1. Node planning
| ES Server | Node Allocation |
|---|---|
| 192.168.0.01 | master |
| 192.168.0.02 | slave |
| 192.168.0.03 | slave |
| 192.168.0.04 | slave |
| 192.168.0.05 | slave, kibana |
2. es new user
useradd elastic passwd***
3. Modify the maximum number of files
Execute as root user: vim /etc/security/limits.conf
Add to:
elastic soft nofile 65536 elastic hard nofile 65536 elastic soft nproc 65536 elastic hard nproc 65536 elastic soft memlock unlimited elastic hard memlock unlimited
4. Modify the maximum memory of the virtual machine
Execute as root user: vim /etc/sysctl.conf
Add to:
vm.max_map_count=262144 vm.swappiness=1
The modification takes effect: sysctl -p
Or use temporary modifications (which will become invalid after the server is restarted):
sysctl -w vm.max_map_count=262144 sysctl -w vm.swappiness=1
5. Configure memory lock
The elasticsearch official website recommends that the production environment needs to set bootstrap.memory_lock: true
The explanation from the official website is: When system swapping (memory swapping) occurs, the performance of the ES node will be very poor, and it will also affect the stability of the node. So avoid swapping at all costs. Swapping will cause the cycle delay of Java GC to deteriorate from milliseconds to minutes. More seriously, it will cause node response delays or even leave the cluster.
vim /etc/systemd/system.conf
Add at the end of the file
DefaultLimitNOFILE=65536 DefaultLimitNPROC=32000 DefaultLimitMEMLOCK=infinity
But after saving, it will not take effect immediately. There are two ways to make it take effect.
- Restart the system
- Execution:
systemctl daemon-reexec
6. Switch users and create directories
Switch to the elastic user and check the disk mount point
su elastic df-h
Find a mount point with the largest disk space and create a new elasticsearch data storage directory
For this deployment, we communicated with the technical service and mounted the 2T hard disk under /data, and directly created a new directory under /data.
- elasticsearch – application directory
- data – data storage directory
- logs – store logs
- keystore – xpack cluster communication key storage
mkdir -p /data/elasticsearch mkdir -p /data/data mkdir -p /data/logs mkdir -p /data/keystore
Be sure to ensure that the permissions of each file under /data belong to the non-root account you prepared for elasticsearch. If not, use the root user to execute the following
chown -R elastic:elastic /data
7. Upload tar package
Upload the installation package to /data/elasticsearch, decompress it, and then delete the compressed package
tar -zxvf elasticsearch-7.8.1.tar.gz
rm -rf elasticsearch-7.8.1.tar.gz
8. Modify JVM memory
Enter /data/elasticsearch/elasticsearch-7.8.1/config
Modify jvm configuration
vim jvm.options
-Xms8g # Note that this value should not exceed 32G, the best value is 26g -Xmx8g
In order to ensure JVM utilization, it is recommended that the above two values are the same
9. Modify node configuration
vim elasticsearch.yml
master node:
cluster.name: es #cluster name node.name: node-01 #Node name. It is recommended that nodes 1-9 be named 01-09 for easy viewing. path.data: /data/data #Data storage path path.logs: /data/logs #Log storage path bootstrap.memory_lock: true #Enable memory locking network.host: 192.168.0.01 #Node IP http.port: 9200 #Exposed port discovery.seed_hosts: ["192.168.0.01:9300", "192.168.0.02:9300","192.168.0.03:9300","192.168.0.04:9300","192.168.0.05:9300"] #Discoverable within the cluster Node and IP cluster.initial_master_nodes: ["192.168.0.01:9300"]#Initialized master node # Enable xpack authentication xpack.security.enabled: true xpack.security.transport.ssl.enabled: true xpack.security.transport.ssl.verification_mode: certificate xpack.security.transport.ssl.keystore.path: elastic-certificates.p12 xpack.security.transport.ssl.truststore.path: elastic-certificates.p12
slave node:
cluster.name: es #cluster name node.name: node-02 #Node name. It is recommended that nodes 1-9 be named 01-09 for easy viewing. path.data: /data/data #Data storage path path.logs: /data/logs #Log storage path bootstrap.memory_lock: true #Enable memory locking network.host: 192.168.0.02 #Node IP http.port: 9200 #Exposed port discovery.seed_hosts: ["192.168.0.01:9300", "192.168.0.02:9300","192.168.0.03:9300","192.168.0.04:9300","192.168.0.05:9300"] #Discoverable within the cluster Node and IP cluster.initial_master_nodes: ["192.168.0.01:9300"]#Initialized master node # Enable xpack authentication xpack.security.enabled: true xpack.security.transport.ssl.enabled: true xpack.security.transport.ssl.verification_mode: certificate xpack.security.transport.ssl.keystore.path: elastic-certificates.p12 xpack.security.transport.ssl.truststore.path: elastic-certificates.p12
10. Specify built-in jdk
Enter /data/elasticsearch/elasticsearch-7.8.1/bin
vim elasticsearch
Add at the beginning
export JAVA_HOME=/data/elasticsearch/elasticsearch-7.8.1/jdk/
export PATH=$JAVA_HOME/bin:$PATH
#Add jdk judgment
if [ -x "$JAVA_HOME/bin/java" ]; then
JAVA="/data/elasticsearch/elasticsearch-7.8.1/jdk/bin/java"
else
JAVA=`which java`
fi
11. Add word segmenter
Download the corresponding version of the word segmenter, unzip it, then organize it by folder and place it in /data/elasticsearch/elasticsearch-7.8.1/plugins
This installation uses the following three word breakers: ik, pinyin, and stconvert.
12. CA authorization certification
1. Institutional authorization
Enter the /data/elasticsearch/elasticsearch-7.8.1/bin directory and execute
./elasticsearch-certutil ca
After execution, the following question will appear. Please enter the storage path in elasticsearch.yml and use elastic-stack-ca.p12 as the file name.
Please enter the desired output file [elastic-stack-ca.p12]:/data/keystore/elastic-stack-ca.p12
Then continue to enter the ==Password of the authorized authority== that you set. This password will be very useful when the cluster is expanded in the future and needs to be remembered.
2. Generate certificate
After entering the password, the authority generates and then generates an authorization certificate
./elasticsearch-certutil cert --ca /data/keystore/elastic-stack-ca.p12
There are three inputs in this process
- Enter ==Password of the authorized authority==
- Enter the absolute path of the CA certificate:/data/keystore/elastic-certificates.p12
- Enter the CA certificate password (for convenience of management, it is the same as the authorized authority password)
Finally, the following two files are generated in the /data/keystore directory
- elastic-certificates.p12
- elastic-stack-ca.p12
3. Certificate encryption
Place elastic-certificates.p12 and elastic-stack-ca.p12 in the /data/keystore directory into the config directory
Then execute the following command
./elasticsearch-keystore add xpack.security.transport.ssl.keystore.secure_password ./elasticsearch-keystore add xpack.security.transport.ssl.truststore.secure_password
The password is the one you just entered ==Password of the authorized agency==
13. Deploy the remaining nodes
Distribute the configured elasticsearch to the corresponding location of each node, and configure the response for each node
Pay special attention to the places that need to be modified: Node name, host name, etc.
- The remaining nodes need to perform the certificate encryption operation again
14. Start
Execute ./elasticsearch -d in the bin directory to start, start all nodes
15. Configure built-in user password
./elasticsearch-setup-passwords interactive
| Username | Function |
|---|---|
| elastic | Super user |
| kibana | is used to connect Kibana to Elasticsearch |
| logstash_system | Logstash is used when storing monitoring information in Elasticsearch |
| beats_system | Beats is used when storing monitoring information in Elasticsearch |
| apm_system | The APM server uses it when storing monitoring information in Elasticsearch |
| remote_monitoring_user | Metricbeat user Used when collecting and storing monitoring information in Elasticsearch |
Sixteen, kibana
vim kibana.yml
server.port: 5601 server.host: "192.168.0.05" #Local IP elasticsearch.hosts: ["http://192.168.0.01:9200","http://192.168.0.02:9200","http://192.168.0.03:9200","http://192.168.0.04: 9200","http://192.168.0.05:9200"] #ES cluster IP elasticsearch.username: "elastic" elasticsearch.password: "***" i18n.locale: "zh-CN"
nohup ./kibana &