How to configure TLS encrypted transmission in RocketMQ 5.0?

The author of this article: Li Wei, everyone in the community is Xiaowei, Apache RocketMQ Committer, RocketMQ Python client project owner, Apache Doris Contributor, Tencent Cloud RocketMQ development engineer.

01 Transmission architecture diagram

Namesrv: 5.1.0

Broker: 5.1.0

Dashboard: 1.0.1-SNAPSHOT

02 Prepare CA certificates and keys for Namesrv, Broker, and Client

The directory for all the following operations is: /etc/rocketmq, and Namesrv, Broker, and Dashboard are on the same machine

In actual operation, the dashboard or the client can be other machines

1. Generate a ca-signed certificate

Fill in and repeat the ca certificate password. When actually filling in, the entered characters are invisible.

openssl req -newkey rsa:2048 -keyout ca_rsa_private.pem -x509 -days 365 -out ca.pem

Fill in other information, if not fill in, use “.”

Generate a ca-signed certificate

2. Generate public and private keys. Provided for client-server encrypted transmission

openssl req -newkey rsa:2048 -keyout server_rsa.key -out server.csr

Generating a 2048 bit RSA private key

generate encryption key pair

3. Generate Namesrv, Broker encryption key pair, and issue Namesrv, Broker certificate

openssl req -newkey rsa:2048 -keyout server_rsa.key -out server.csr Generating a 2048 bit RSA private key

Generate Namesrv, Broker keys, and issue certificates

4. Pack and encrypt Namesrv, Broker private key

5. Add the tls configuration file used by Namesrv and Broker

tls-broker.properties

tls.test.mode.enable=false
tls.server.need.client.auth=none
tls.server.keyPath=/etc/rocketmq/server.key
tls.server.keyPassword=123456
tls.server.certPath=/etc/rocketmq/server.pem
tls.client.authServer=false
tls.client.trustCertPath=/etc/rocketmq/ca.pem

tls-namesrv.properties

tls.test.mode.enable=false
tls.server.need.client.auth=none
tls.server.keyPath=/etc/rocketmq/server.key
tls.server.keyPassword=123456
tls.server.certPath=/etc/rocketmq/server.pem

tls-client.properties

tls.client.trustCertPath=/etc/rocketmq/ca.pem

So far, we have got all the tls configuration files:

All configuration files

03 Modify startup script

3.1 Modify the namesrv startup script

vim bin/runserver.sh

Modify the namesrv startup script

3.2 Modify broker startup configuration

Modify the broker startup script and set jvm to support tls

vim bin/runbroker.sh

Modify the broker startup script

Add broker.conf

brokerClusterName = DefaultCluster
brokerName = broker-a
brokerId = 0
deleteWhen = 04
fileReservedTime = 48
brokerRole = ASYNC_MASTER
flushDiskType = ASYNC_FLUSH
namesrvAddr = 127.0.0.1:9876

3.3 Modify the dashboard configuration

Modify namesrv address

Modify namesrv address

Turn on the tls switch

vim rocketmq-dashboard-1.0.1-SNAPSHOT.jar

Modify the dashboard configuration

Note: If it is client-side production and consumption, the settings are as follows

The consumer turns on the tls switch

The producer turns on the tls switch

04 Start Namesrv, Broker, Dashboard

start namesrv

nohup sh bin/mqnamesrv &

start broker

nohup sh bin/mqbroker -c conf/broker.conf &

Start the dashboard

java -Dtls.client.authServer=true -Dtls.enable=true -Dtls.test.mode.enable=false -Dtls.config.file=/etc/rocketmq/tls-client.properties -jar rocketmq-dashboard -1.0.1-SNAPSHOT.jar

05 Verification

tcpdump packet capture verification

TLS packet capture result

Rocketmq dashboard log verification: ~/logs/rocketmqlogs/rocketmq_client.log

Add image annotations, no more than 140 words (optional)

06 Question: Why is there TCP protocol in the packet capture result?

In the packet capture result, why is there a TCP protocol?

The client can be enabled by setting: -Dtls.enable=true, but actually still need to set the code “producer.setUseTLS(useTls);” or “consumer.setUseTLS(useTls);”, why?

07 See what is the final generated file?