2. Elastic kibana security configuration and Logstash installation

2. Installation of ik word segmenter

2.1 Reference

https://blog.csdn.net/u011296165/article/details/107996049?ops_request_misc= & amp;request_id= & amp;biz_id=102 & amp;utm_term=IK tokenizer installation& amp;utm_medium=distribute. pc_search_result.none-task-blog-2~all~sobaiduweb~default-6-107996049.nonecase &spm=1018.2226.3001.4187

2.2 github download ik URL

https://github.com/medcl/elasticsearch-analysis-ik/releases/tag/v7.17.3

2.3 Installation

Go to the installation directory of elasticsearch and find the bin directory. Execute the elasticsearch-plugin command inside. This command manages the plug-ins in es. The ik word segmenter is a plug-in.

Execute the following command to install it. Note that the version of the IK word segmenter must be consistent with the version of es. You can find the specific version in github.
#Execute on all nodes in cluster mode
bin/elasticsearch-plugin install https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v7.17.3/elasticsearch-analysis-ik-7.17.3.zip

Note that the built-in jdk must be used here (refer to 1.8 modification)

After the installation is complete, you need to restart the es service. Just kill it and start it again.

2.4 Test ik word segmenter

POST /_analyze
{
  "text": "The road to Shu is difficult, and there are many divergent roads now.",
  "analyzer": "ik_max_word"
  //ik_max_word: will split the text into the finest granularity. ik_smart: will split the text into the coarsest granularity.
 
}

4. kibana installation

4.1 Official website address

https://www.elastic.co/cn/downloads/past-releases#kibana

4.2 Download software (download the same version)

wget https://artifacts.elastic.co/downloads/kibana/kibana-7.17.3-linux-x86_64.tar.gz
#decompression software
tar -zxvf kibana-7.17.3-linux-x86_64.tar.gz -C /opt/installed/

4.3 Generate certificate file for kibana

# Generate the certificate in the ES server and press Enter
cd /opt/installed/elasticsearch-7.17.3 bin/elasticsearch-certutil csr -name kibana -dns elastic01
 
# unzip files 
unzip csr-bundle.zip
 
# Move the decompressed file to kibana's config directory
mv kibana.csr kibana.key /opt/installed/kibana-7.17.3/config/
# Generate crt file
openssl x509 -req -in kibana.csr -signkey kibana.key -out kibana.crt

4.4 Modify the configuration file: kibana.yml

## Service port
server.port: 5601
# Service host name
server.host: "elastic01"
i18n.locale: "zh-CN"
#ES service host address
elasticsearch.hosts: ["https://:9200"]
#Account password to access ES service
elasticsearch.username: "elastic"
elasticsearch.password: ""
elasticsearch.ssl.verificationMode: none
elasticsearch.ssl.certificateAuthorities: ["/opt/installed/elasticsearch-7.17.3/config/certs/elasticsearch-ca.pem"]
#Configure SSL encrypted transmission of Kibana UI
server.ssl.enabled: true
server.ssl.certificate: /opt/installed/kibana-7.17.3/config/kibana.crt
server.ssl.key: /opt/installed/kibana-7.17.3/config/kibana.key

4.5 Modify kibana file permissions to elastic user

4.6 Start with elastic

4.7 Question

4.7.1 Question 1

It can be used normally but there is an error in the log (normal use has not yet been resolved)

[error][client][connection] Error: ,139632920631232:error:14094416:SSL routines:ssl3_read_bytes:sslv3 alert certificate unknown:../deps/openssl/openssl/ssl/record/rec_layer_s3.c:1546: SSL alert number 46

4.7.2 Problem 2 startup error

[Screenshot here] log [02:25:38.464] [error][reporting] The Reporting plugin encountered issues launching Chromium in a self-test. You may have trouble generating reports. log [02:25:38.465] [ error][reporting] ErrorEvent { target: WebSocket { _events: [Object: null prototype] { open: [Function], error: [Function] }, _eventsCount: 2, _maxListeners: undefined, readyState: 3, protocol: '', _binaryType: 'nodebuffer', _closeFrameReceived: false, _closeFrameSent: false, _closeMessage: '', _closeTimer: null, _closeCode: 1006, _extensions: {}, _receiver: null, _sender: null, _socket: null, _isServer: false, _redirects: 0, url: 'ws://127.0.0.1:45871/devtools/browser/7e246271-fac4-4eed-99c0-28ace04c7e45', _req: null }, type: 'error', message: 'socket hang up', error : { Error: socket hang up at createHangUpError (_http_client.js:323:15) at Socket.socketOnEnd (_http_client.js:426:23) at Socket.emit (events.js:194:15) at endReadableNT (_stream_readable.js :1103:12) at process._tickCallback (internal/process/next_tick.js:63:19) code: 'ECONNRESET' } } log [02:25:38.469] [warning][reporting] See Chromium's log output at “/ kibana/kibana-7.5.1-linux-x86_64/data/headless_shell-linux/chrome_debug.log” log [02:25:38.469] [warning][reporting] Reporting plugin self-check failed. Please check the Kibana Reporting settings. Error: Could not close browser client handle!

Install the package files from the following link:

yum install the following package files and then restart.

ipa-gothic-fonts
xorg-x11-fonts-100dpi
xorg-x11-fonts-75dpi
xorg-x11-utils
xorg-x11-fonts-cyrillic
xorg-x11-fonts-Type1
xorg-x11-fonts-misc
fontconfig
freetype

4.8 kibana second certificate generation configuration

refer to

https://blog.csdn.net/h952520296/article/details/112008365?ops_request_misc= & amp;request_id= & amp;biz_id=102 & amp;utm_term=es configure https default password&utm_medium =distribute.pc_search_result.none-task-blog-2~all~sobaiduweb~default-1-112008365.142^v95^control & amp;spm=1018.2226.3001.4187

Generate PKI client certificate

The certificate that will be used for PKI authentication must be signed by the same CA as the certificate used to encrypt HTTP communications. Because we have used a self-signed CA, we must also use the elastic-stack-ca.p12CA from before to sign the HTTP client certificate. As follows:

Enter the password and press Enter directly. Do not set a password. An error will be reported when starting.

Note that this block has been generated before and I have not generated it again.

bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12
ENTER
ENTER
ENTER
 
mv elastic-certificates.p12 client.p12
Named separately from the previous key

This will create a file called elastic-certificates.p12 that contains all the information needed to authenticate to PKI for our Elasticsearch cluster. However, in order to use this certificate, it needs to be broken down into its private key, public certificate and CA certificate:

//Private Key private key
openssl pkcs12 -in client.p12 -nocerts -nodes > client.key
// Public Certificate public certificate
openssl pkcs12 -in client.p12 -clcerts -nokeys > client.cer
// CA Certificate The CA that signed the public certificate
openssl pkcs12 -in client.p12 -cacerts -nokeys -chain > client-ca.cer

Configure verification from Kibana to Elasticsearch

Now that security is enabled on the Elasticsearch cluster, Kibana security must be enabled and configured to authenticate to the cluster over HTTPS. Create the config/certs directory in Kibana and copy the generated client certificate into the directory and add the following lines to our kibana.yml file:

## Service port
server.port: 5601
# Service host name
server.host: "elastic02"
i18n.locale: "zh-CN"
#ES service host address
elasticsearch.hosts: ["https://ip:9200"]
xpack.security.enabled: true
#Account password to access ES service
elasticsearch.username: ""
elasticsearch.password: ""
elasticsearch.ssl.certificate: config/cert/client.cer

elasticsearch.ssl.key: config/cert/client.key

elasticsearch.ssl.certificateAuthorities: [ "config/cert/client-ca.cer" ]

I have not configured this part

kibana user is like a service user that can work behind the scenes to authenticate Kibana applications for Elasticsearch clusters. We usually do not use the kibana user to log in directly to the Elasticsearch cluster or Kibana UI.

When Elasticsearch has not enabled HTTP encrypted communication, you can directly use the kibana user and password for authentication.

Add the following three configurations to the kibana.yml file. These are necessary configurations to enable Elastic alarms. The parameters inside can be configured at will:
# Note: The parameter value must be at least 32 bits, otherwise an error message will be reported during startup.
xpack.encryptedSavedObjects.encryptionKey: encryptedSavedObjects12345678909876543210
xpack.security.encryptionKey: encryptionKeysecurity12345678909876543210
xpack.reporting.encryptionKey: encryptionKeyreporting12345678909876543210

Configure SSL encrypted transmission of Kibana UI

For security reasons, Kibana UI to kibana service should also enable SSL encrypted transmission communication. To simplify things, we use the same certificate that we already use for transport communication, configured as follows:

server.ssl.enabled: true
server.ssl.certificate: config/cert/client.cer
server.ssl.key: config/cert/client.key

4.8.1 All configurations

## Service port
server.port: 5601
# Service host name
server.host: "elastic02"
i18n.locale: "zh-CN"
#ES service host address
elasticsearch.hosts: ["https://ip:9200"]
xpack.security.enabled: true
#Account password to access ES service
elasticsearch.username: "elastic"
elasticsearch.password: ""
elasticsearch.ssl.certificate: config/cert/client.cer

elasticsearch.ssl.key: config/cert/client.key

elasticsearch.ssl.certificateAuthorities: [ "config/cert/client-ca.cer" ]

elasticsearch.ssl.verificationMode: certificate
xpack.encryptedSavedObjects.encryptionKey: encryptedSavedObjects12345678909876543210
xpack.security.encryptionKey: encryptionKeysecurity12345678909876543210
xpack.reporting.encryptionKey: encryptionKeyreporting12345678909876543210
server.ssl.enabled: true
server.ssl.certificate: config/cert/client.cer
server.ssl.key: config/cert/client.key

5. Logstash installation

It is recommended that kibana, es, and logstash have the same versions, otherwise compatibility issues will occur.

5.1 Download

Download URL: https://www.elastic.co/downloads/past-releases#logstash

5.2 Installation

# Decompress the compressed package
tar -zxvf logstash-7.17.3.tar.gz -C /opt/installed/

Catalog description

[root@localhost logstash]# ls
bin data jdk logstash-core NOTICE.TXT x-pack
config Gemfile lib logstash-core-plugin-api tools
CONTRIBUTORS Gemfile.lock LICENSE.txt modules vendor

#bin: startup file, executable file
# config: configuration file
#data: data history file
#lib: dependent jar package
# logstash-core-plugin-api: core plug-in
# tools: tools
# modules: components
# logstash-core: logstash core



[root@localhost logstash]# ls config/
jvm.options logstash-sample.conf pipelines.yml
log4j2.properties logstash.yml startup.options
# jvm.options: Optimize configuration
# log4j2.properties: log configuration
# logstash.yml: core configuration
# logstash-sample.conf: sample file
# pipelines.yml: pipeline configuration
# startup.options: startup log information

5.3 Modify owner permissions

chown -R elastic:elastic logstash-7.17.3

5.4 Add mysqljar

mysql-connector-java-8.0.29.jar is placed under /opt/installed/logstash-7.17.3/lib

5.5 Configure TLS encryption from Logstash to Elasticsearch

The file elastic-certificates.p12 contains all the information needed to authenticate to PKI for our Elasticsearch cluster. However, in order to use this certificate, it needs to be broken down into its private key, public certificate and CA certificate:

Create a certs file in the Logstash config folder
cp /opt/installed/elasticsearch-7.17.3/config/certs/elastic-certificates.p12 Copy to the Logstash certs folder
#Decompose the certificate in Logstash/config/certs
// Private Key private key
openssl pkcs12 -in client.p12 -nocerts -nodes > client.key
// Public Certificate public certificate
openssl pkcs12 -in client.p12 -clcerts -nokeys > client.cer
// CA Certificate CA signing public certificate
openssl pkcs12 -in client.p12 -cacerts -nokeys -chain > client-ca.cer

5.6 Logstash uses synchronization configuration

output {
  stdout {
  #Output method When using codec => rubydebug in Logstash, it will convert each event received into a Ruby object and output it to the console or file so that you can view the details of the event.
interest
      codec => rubydebug
        }
          elasticsearch {
               #Security configuration
                cacert => "/opt/installed/logstash-7.17.3/config/cert/client-ca.cer"
                ssl => true
                ssl_certificate_verification => false
                hosts => ["https://ip:9200"]
                   # Index name, must be lowercase
                       index =>t"
                       # Unique data index (it is recommended to use the database primary key id)
                        document_id => "%{uuid}"
                        user => ""
                        password => ""
        }
      }

Reference article

https://blog.csdn.net/fen_fen/article/details/123265224?utm_medium=distribute.pc_relevant.none-task-blog-2~default~baidujs_utm_term~default-0-123265224-blog-127590339.235^v38 ^pc_relevant_default_base3 & amp;spm=1001.2101.3001.4242.1 & amp;utm_relevant_index=3
https://blog.csdn.net/h952520296/article/details/112008365?ops_request_misc= & amp;request_id= & amp;biz_id=102 & amp;utm_term=es configure https default password & amp;utm_medium=distribute. pc_search_result.none-task-blog-2~all~sobaiduweb~default-1-112008365.142^v95^control & amp;spm=1018.2226.3001.4187