2. Installation of ik word segmenter
2.1 Reference
https://blog.csdn.net/u011296165/article/details/107996049?ops_request_misc= & amp;request_id= & amp;biz_id=102 & amp;utm_term=IK tokenizer installation& amp;utm_medium=distribute. pc_search_result.none-task-blog-2~all~sobaiduweb~default-6-107996049.nonecase &spm=1018.2226.3001.4187
2.2 github download ik URL
https://github.com/medcl/elasticsearch-analysis-ik/releases/tag/v7.17.3
2.3 Installation
Go to the installation directory of elasticsearch and find the bin directory. Execute the elasticsearch-plugin command inside. This command manages the plug-ins in es. The ik word segmenter is a plug-in. Execute the following command to install it. Note that the version of the IK word segmenter must be consistent with the version of es. You can find the specific version in github. #Execute on all nodes in cluster mode bin/elasticsearch-plugin install https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v7.17.3/elasticsearch-analysis-ik-7.17.3.zip
Note that the built-in jdk must be used here (refer to 1.8 modification)
After the installation is complete, you need to restart the es service. Just kill it and start it again.
2.4 Test ik word segmenter
POST /_analyze { "text": "The road to Shu is difficult, and there are many divergent roads now.", "analyzer": "ik_max_word" //ik_max_word: will split the text into the finest granularity. ik_smart: will split the text into the coarsest granularity. }
4. kibana installation
4.1 Official website address
https://www.elastic.co/cn/downloads/past-releases#kibana
4.2 Download software (download the same version)
wget https://artifacts.elastic.co/downloads/kibana/kibana-7.17.3-linux-x86_64.tar.gz #decompression software tar -zxvf kibana-7.17.3-linux-x86_64.tar.gz -C /opt/installed/
4.3 Generate certificate file for kibana
# Generate the certificate in the ES server and press Enter cd /opt/installed/elasticsearch-7.17.3 bin/elasticsearch-certutil csr -name kibana -dns elastic01 # unzip files unzip csr-bundle.zip # Move the decompressed file to kibana's config directory mv kibana.csr kibana.key /opt/installed/kibana-7.17.3/config/ # Generate crt file openssl x509 -req -in kibana.csr -signkey kibana.key -out kibana.crt
4.4 Modify the configuration file: kibana.yml
## Service port server.port: 5601 # Service host name server.host: "elastic01" i18n.locale: "zh-CN" #ES service host address elasticsearch.hosts: ["https://:9200"] #Account password to access ES service elasticsearch.username: "elastic" elasticsearch.password: "" elasticsearch.ssl.verificationMode: none elasticsearch.ssl.certificateAuthorities: ["/opt/installed/elasticsearch-7.17.3/config/certs/elasticsearch-ca.pem"] #Configure SSL encrypted transmission of Kibana UI server.ssl.enabled: true server.ssl.certificate: /opt/installed/kibana-7.17.3/config/kibana.crt server.ssl.key: /opt/installed/kibana-7.17.3/config/kibana.key
4.5 Modify kibana file permissions to elastic user
4.6 Start with elastic
4.7 Question
4.7.1 Question 1
It can be used normally but there is an error in the log (normal use has not yet been resolved)
[error][client][connection] Error: ,139632920631232:error:14094416:SSL routines:ssl3_read_bytes:sslv3 alert certificate unknown:../deps/openssl/openssl/ssl/record/rec_layer_s3.c:1546: SSL alert number 46
4.7.2 Problem 2 startup error
[Screenshot here] log [02:25:38.464] [error][reporting] The Reporting plugin encountered issues launching Chromium in a self-test. You may have trouble generating reports. log [02:25:38.465] [ error][reporting] ErrorEvent { target: WebSocket { _events: [Object: null prototype] { open: [Function], error: [Function] }, _eventsCount: 2, _maxListeners: undefined, readyState: 3, protocol: '', _binaryType: 'nodebuffer', _closeFrameReceived: false, _closeFrameSent: false, _closeMessage: '', _closeTimer: null, _closeCode: 1006, _extensions: {}, _receiver: null, _sender: null, _socket: null, _isServer: false, _redirects: 0, url: 'ws://127.0.0.1:45871/devtools/browser/7e246271-fac4-4eed-99c0-28ace04c7e45', _req: null }, type: 'error', message: 'socket hang up', error : { Error: socket hang up at createHangUpError (_http_client.js:323:15) at Socket.socketOnEnd (_http_client.js:426:23) at Socket.emit (events.js:194:15) at endReadableNT (_stream_readable.js :1103:12) at process._tickCallback (internal/process/next_tick.js:63:19) code: 'ECONNRESET' } } log [02:25:38.469] [warning][reporting] See Chromium's log output at “/ kibana/kibana-7.5.1-linux-x86_64/data/headless_shell-linux/chrome_debug.log” log [02:25:38.469] [warning][reporting] Reporting plugin self-check failed. Please check the Kibana Reporting settings. Error: Could not close browser client handle!
Install the package files from the following link:
yum install the following package files and then restart.
ipa-gothic-fonts xorg-x11-fonts-100dpi xorg-x11-fonts-75dpi xorg-x11-utils xorg-x11-fonts-cyrillic xorg-x11-fonts-Type1 xorg-x11-fonts-misc fontconfig freetype
4.8 kibana second certificate generation configuration
refer to
https://blog.csdn.net/h952520296/article/details/112008365?ops_request_misc= & amp;request_id= & amp;biz_id=102 & amp;utm_term=es configure https default password&utm_medium =distribute.pc_search_result.none-task-blog-2~all~sobaiduweb~default-1-112008365.142^v95^control & amp;spm=1018.2226.3001.4187
Generate PKI client certificate
The certificate that will be used for PKI authentication must be signed by the same CA as the certificate used to encrypt HTTP communications. Because we have used a self-signed CA, we must also use the elastic-stack-ca.p12CA from before to sign the HTTP client certificate. As follows:
Enter the password and press Enter directly. Do not set a password. An error will be reported when starting.
Note that this block has been generated before and I have not generated it again.
bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12 ENTER ENTER ENTER mv elastic-certificates.p12 client.p12 Named separately from the previous key
This will create a file called elastic-certificates.p12 that contains all the information needed to authenticate to PKI for our Elasticsearch cluster. However, in order to use this certificate, it needs to be broken down into its private key, public certificate and CA certificate:
//Private Key private key openssl pkcs12 -in client.p12 -nocerts -nodes > client.key // Public Certificate public certificate openssl pkcs12 -in client.p12 -clcerts -nokeys > client.cer // CA Certificate The CA that signed the public certificate openssl pkcs12 -in client.p12 -cacerts -nokeys -chain > client-ca.cer
Configure verification from Kibana to Elasticsearch
Now that security is enabled on the Elasticsearch cluster, Kibana security must be enabled and configured to authenticate to the cluster over HTTPS. Create the config/certs directory in Kibana and copy the generated client certificate into the directory and add the following lines to our kibana.yml file:
## Service port server.port: 5601 # Service host name server.host: "elastic02" i18n.locale: "zh-CN" #ES service host address elasticsearch.hosts: ["https://ip:9200"] xpack.security.enabled: true #Account password to access ES service elasticsearch.username: "" elasticsearch.password: "" elasticsearch.ssl.certificate: config/cert/client.cer elasticsearch.ssl.key: config/cert/client.key elasticsearch.ssl.certificateAuthorities: [ "config/cert/client-ca.cer" ]
I have not configured this part
kibana user is like a service user that can work behind the scenes to authenticate Kibana applications for Elasticsearch clusters. We usually do not use the kibana user to log in directly to the Elasticsearch cluster or Kibana UI. When Elasticsearch has not enabled HTTP encrypted communication, you can directly use the kibana user and password for authentication. Add the following three configurations to the kibana.yml file. These are necessary configurations to enable Elastic alarms. The parameters inside can be configured at will: # Note: The parameter value must be at least 32 bits, otherwise an error message will be reported during startup. xpack.encryptedSavedObjects.encryptionKey: encryptedSavedObjects12345678909876543210 xpack.security.encryptionKey: encryptionKeysecurity12345678909876543210 xpack.reporting.encryptionKey: encryptionKeyreporting12345678909876543210
Configure SSL encrypted transmission of Kibana UI
For security reasons, Kibana UI to kibana service should also enable SSL encrypted transmission communication. To simplify things, we use the same certificate that we already use for transport communication, configured as follows:
server.ssl.enabled: true server.ssl.certificate: config/cert/client.cer server.ssl.key: config/cert/client.key
4.8.1 All configurations
## Service port server.port: 5601 # Service host name server.host: "elastic02" i18n.locale: "zh-CN" #ES service host address elasticsearch.hosts: ["https://ip:9200"] xpack.security.enabled: true #Account password to access ES service elasticsearch.username: "elastic" elasticsearch.password: "" elasticsearch.ssl.certificate: config/cert/client.cer elasticsearch.ssl.key: config/cert/client.key elasticsearch.ssl.certificateAuthorities: [ "config/cert/client-ca.cer" ] elasticsearch.ssl.verificationMode: certificate xpack.encryptedSavedObjects.encryptionKey: encryptedSavedObjects12345678909876543210 xpack.security.encryptionKey: encryptionKeysecurity12345678909876543210 xpack.reporting.encryptionKey: encryptionKeyreporting12345678909876543210 server.ssl.enabled: true server.ssl.certificate: config/cert/client.cer server.ssl.key: config/cert/client.key
5. Logstash installation
It is recommended that kibana, es, and logstash have the same versions, otherwise compatibility issues will occur.
5.1 Download
Download URL: https://www.elastic.co/downloads/past-releases#logstash
5.2 Installation
# Decompress the compressed package tar -zxvf logstash-7.17.3.tar.gz -C /opt/installed/
Catalog description
[root@localhost logstash]# ls bin data jdk logstash-core NOTICE.TXT x-pack config Gemfile lib logstash-core-plugin-api tools CONTRIBUTORS Gemfile.lock LICENSE.txt modules vendor #bin: startup file, executable file # config: configuration file #data: data history file #lib: dependent jar package # logstash-core-plugin-api: core plug-in # tools: tools # modules: components # logstash-core: logstash core [root@localhost logstash]# ls config/ jvm.options logstash-sample.conf pipelines.yml log4j2.properties logstash.yml startup.options # jvm.options: Optimize configuration # log4j2.properties: log configuration # logstash.yml: core configuration # logstash-sample.conf: sample file # pipelines.yml: pipeline configuration # startup.options: startup log information
5.3 Modify owner permissions
chown -R elastic:elastic logstash-7.17.3
5.4 Add mysqljar
mysql-connector-java-8.0.29.jar is placed under /opt/installed/logstash-7.17.3/lib
5.5 Configure TLS encryption from Logstash to Elasticsearch
The file elastic-certificates.p12
contains all the information needed to authenticate to PKI for our Elasticsearch cluster. However, in order to use this certificate, it needs to be broken down into its private key, public certificate and CA certificate:
Create a certs file in the Logstash config folder cp /opt/installed/elasticsearch-7.17.3/config/certs/elastic-certificates.p12 Copy to the Logstash certs folder #Decompose the certificate in Logstash/config/certs // Private Key private key openssl pkcs12 -in client.p12 -nocerts -nodes > client.key // Public Certificate public certificate openssl pkcs12 -in client.p12 -clcerts -nokeys > client.cer // CA Certificate CA signing public certificate openssl pkcs12 -in client.p12 -cacerts -nokeys -chain > client-ca.cer
5.6 Logstash uses synchronization configuration
output { stdout { #Output method When using codec => rubydebug in Logstash, it will convert each event received into a Ruby object and output it to the console or file so that you can view the details of the event. interest codec => rubydebug } elasticsearch { #Security configuration cacert => "/opt/installed/logstash-7.17.3/config/cert/client-ca.cer" ssl => true ssl_certificate_verification => false hosts => ["https://ip:9200"] # Index name, must be lowercase index =>t" # Unique data index (it is recommended to use the database primary key id) document_id => "%{uuid}" user => "" password => "" } }
Reference article
https://blog.csdn.net/fen_fen/article/details/123265224?utm_medium=distribute.pc_relevant.none-task-blog-2~default~baidujs_utm_term~default-0-123265224-blog-127590339.235^v38 ^pc_relevant_default_base3 & amp;spm=1001.2101.3001.4242.1 & amp;utm_relevant_index=3 https://blog.csdn.net/h952520296/article/details/112008365?ops_request_misc= & amp;request_id= & amp;biz_id=102 & amp;utm_term=es configure https default password & amp;utm_medium=distribute. pc_search_result.none-task-blog-2~all~sobaiduweb~default-1-112008365.142^v95^control & amp;spm=1018.2226.3001.4187