Local construction of Elastic Stack (Elasticsearch + kibana + Logstash + FileBeat + APM) 7.17.14 version

Local construction of Elastic Stack (Elasticsearch + kibana + Logstash + FileBeat + APM) 7.17.14 version

1. Foreword:

1. This article is a non-cluster environment configuration description for the installation of Elastic Stack version 7.17.14. If you are using the 8.x version of the Elastic Stack product, this article does not apply.
2. The example in this article is to use Kibana to view logs, use FileBeat to monitor nginx logs, and APM to monitor JAVA performance indicators.

2. Software and hardware environment involved in the document

2.1.Hardware

3 computers with the following configurations

Device 1: elasticsearch + kibana

# Equipment
1 CPU 2 cores and 4 threads and above, 8 cores and 16 threads are recommended
2 Memory 16GB
4 SSD hard drive 100GB or more
5 Network Set a fixed IP, for example: 192.168.5.181

Device 2: logstash + apm server

# Equipment
1 CPU 2 cores and 4 threads and above, 8 cores and 16 threads are recommended
2 Memory 16GB
4 SSD hard drive 100GB or more
5 Network Set a fixed IP, for example: 192.168.5.182

Device 3: nginx + filebeat

# Equipment
1 CPU 2 cores, 4 threads and above
2 Memory 8GB
4 SSD hard drive 20GB or more
5 Network Set a fixed IP, for example: 192.168.5.184

2.2. Software

Note: All software must be of the same version

# Equipment Version and download address
1 CentOS 7.9 CentOS-7-x86_64-Everything-2009.iso
2 elasticsearch elasticsearch-7.17.14-x86_64.rpm
3 kibana kibana-7.17.14-x86_64.rpm
4 logstash logstash-7.17.14-x86_64.rpm
5 filebeat filebeat-7.17.14-x86_64.rpm
6 apm-server apm-server-7.17.14-x86_64.rpm
2.3. Software installation sequence
  1. CentOS 7.9
  2. elasticsearch-7.17.14-x86_64.rpm (server 192.168.5.181)
  3. kibana-7.17.14-x86_64.rpm (server 192.168.5.181)
  4. logstash-7.17.14-x86_64.rpm (server 192.168.5.182)
  5. filebeat-7.17.14-x86_64.rpm (server 192.168.5.184)
  6. apm-server-7.17.14-x86_64.rpm (server 192.168.5.182)
  7. Java agent

3. Install Centos7.9

3.1.1 Load centos image, select [Install CentOS 7]

3.1.2 System language The default is English, click [Continue]

3.1.3 [SOFTWARE SELECTION] Make modifications

3.1.4. Select the base environment with GUI image and system administrator tools

3.1.5. Modify disk layout

3.1.6. Select Configure partitions yourself

3.1.7. Select to automatically create partitions

3.1.8. Adjust the size of Home and root paths
This is modified to /home 10 GB, root path 81 GB

3.1.9. Apply disk changes

3.1.10. Modify network and host name

3.1.11. Set the static IP and server name of server 1

3.1.12. Set the static IP and server name of server 2

3.1.13. Click to start installing the operating system

3.1.14. Modify the root account password and create an administrator account

3.1.15. Click Restart

3.1.16. Accept the license to complete the installation

4. Install elasticsearch

4.1 Turn off the firewall and set the Beijing time zone

cp -f /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
systemctl stop firewalld
systemctl disable firewalld

4.2. Copy the downloaded elasticsearch-7.17.14-x86_64.rpm to the server
Here, you can upload the downloaded files to the /root directory of the server through the FTP tool or set up an intranet file server for downloading.

4.3. Use the rpm command to install elasticsearch-7.17.14 and set it to start automatically at boot.

sudo rpm --install elasticsearch-7.17.14-x86_64.rpm
sudo /bin/systemctl daemon-reload
sudo /bin/systemctl enable elasticsearch.service


4.4. Edit the /etc/elasticsearch/elasticsearch.yml file

vim /etc/elasticsearch/elasticsearch.yml


4.4.Basic configuration content

Keywords value Remarks
cluster.name elk01 Cluster name
path.data /var/lib/elasticsearch elasticsearch data storage directory
path.logs /var/log/elasticsearch elasticsearch log storage directory
network.host 0.0.0.0 Allow access from any IP address in any network segment
http.port 9200 Default port number
discovery.type single-node Cluster single node mode
xpack.security.enabled true Enable elasticsearch encryption mode, this project must be enabled
xpack.security.authc.api_key.enabled true Enable elasticsearch api encryption mode. If fleet is used, the project must be enabled 
# ======================== Elasticsearch Configuration ==================== =====
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
# Before you set out to tweak and tune the configuration, make sure you
# understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster --------------- --------------------------
#
# Use a descriptive name for your cluster:
#
cluster.name: elk01
#
# ------------------------------------ Node ------------ --------------------------
#
# Use a descriptive name for the node:
#
#node.name: node-1
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ---------------------------------- Paths ------------- -----------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /var/lib/elasticsearch
#
# Path to log files:
#
path.logs: /var/log/elasticsearch
#
# ---------------------------------- Memory ------------- -----------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network --------------- --------------------------
#
# By default Elasticsearch is only accessible on localhost. Set a different
# address here to expose this node on the network:
#
network.host: 0.0.0.0
#
# By default Elasticsearch listens for HTTP traffic on the first free port it
# finds starting at 9200. Set a specific HTTP port here:
#
http.port: 9200
#
# For more information, consult the network module documentation.
#
# ---------------------------------- Discovery --------------- -------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.seed_hosts: ["host1", "host2"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
#cluster.initial_master_nodes: ["node-1", "node-2"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Various --------------- --------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true
#
# ---------------------------------- Security --------------- -------------------
#
#***WARNING***
#
# Elasticsearch security features are not enabled by default.
# These features are free, but require configuration changes to enable them.
# This means that users don’t have to provide credentials and can get full access
# to the cluster. Network connections are also not encrypted.
#
# To protect your data, we strongly encourage you to enable the Elasticsearch security features.
# Refer to the following documentation for instructions.
#
# https://www.elastic.co/guide/en/elasticsearch/reference/7.16/configuring-stack-security.html
discovery.type: single-node
xpack.security.enabled: true
xpack.security.authc.api_key.enabled: true
</code><img class="look-more-preCode contentImg-no-view" src="//i2.wp.com/csdnimg.cn/release/blogv2/dist/pc/img/newCodeMoreBlack. png" alt="" title="">

4.5. Start the elasticsearch server and check the status

sudo systemctl start elasticsearch.service
sudo systemctl status elasticsearch.service


4.6. Set the default password for each module, and set the password to 123456 uniformly.

/usr/share/elasticsearch/bin/elasticsearch-setup-passwords interactive -b


4.7. Open the http://192.168.5.181:9200 web page and confirm the elasticsearch status
Enter username and password
Username: elastic
Password: 123456

Login successful, return elasticsearch status

5. Install kibana

5.1. Copy the downloaded kibana-7.17.14-x86_64.rpm to the server
Here, you can upload the downloaded files to the /root directory of the server through the FTP tool or set up an intranet file server for downloading.

5.2. Use the rpm command to install kibana-7.17.14 and set it to start automatically at boot

sudo rpm --install kibana-7.17.14-x86_64.rpm
sudo /bin/systemctl daemon-reload
sudo /bin/systemctl enable kibana.service


5.3. Edit the /etc/kibana/kibana.yml file and modify the basic configuration

vim /etc/kibana/kibana.yml

5.4.Basic configuration content

Keywords value Remarks
server.port 5601 kibana’s default port number
server.host “0.0.0.0” Allow any IP in any network segment to access kibana
server.name Elastic-Kibana kibana name
elasticsearch.hosts [“http://192.168.5.181:9200”] Elasticsearch server address, if it is a cluster, multiple addresses need to be filled in
kibana.index .kibana kibana index
elasticsearch.username kibana_system kibana administrator
elasticsearch.password 123456 Kibana administrator user password
i18n.locale “zh-CN” Modify the page to display in Chinese 
# Kibana is served by a back end server. This setting specifies the port to use.
server.port: 5601

# Specifies the address to which the Kibana server will bind. IP addresses and host names are both valid values.
# The default is 'localhost', which usually means remote machines will not be able to connect.
# To allow connections from remote users, set this parameter to a non-loopback address.
server.host: "0.0.0.0"

# Enables you to specify a path to mount Kibana at if you are running behind a proxy.
# Use the `server.rewriteBasePath` setting to tell Kibana if it should remove the basePath
# from requests it receives, and to prevent a deprecation warning at startup.
# This setting cannot end in a slash.
#server.basePath: ""

# Specifies whether Kibana should rewrite requests that are prefixed with
# `server.basePath` or require that they are rewritten by your reverse proxy.
# This setting was effectively always `false` before Kibana 6.3 and will
# default to `true` starting in Kibana 7.0.
#server.rewriteBasePath: false

# Specifies the public URL at which </code><img class="look-more-preCode contentImg-no-view" src="//i2.wp.com/csdnimg.cn/release/blogv2/dist /pc/img/newCodeMoreBlack.png" alt="" title="">