Current situation
Currently, there are only docker images under the x86 architecture of rocketmq on the market and officially. With the increasing demand for localization and Xinchuang adaptation, it is obvious that the existing docker images under the x86 architecture cannot meet the diverse needs. Therefore, we It is necessary to create multi-architecture images that meet the needs according to the officially released version, so that they can be deployed and used under different CPU architectures.
rocketmq official website: RocketMQ · Official website | RocketMQ
For an introduction to rocketmq, please read the official documentation, and I won’t go into details here.
Prerequisite
1. The machine that builds the image needs to have access to the Internet and a good network environment.
2. Before building, please build a private warehouse, such as harbor and registry, so that multi-architecture images can be pushed to the mirror warehouse.
3. Docker needs to be installed, version 19.03 or above, and supports buildx plug-in
Preparation
1. Write Dockerfile
# Multi-stage construction method cat Dockerfile-centos ################################################ ############################## # Build stage 1 `builder`: # Download and extract RocketMQ ################################################ ############################## FROM eclipse-temurin:8-jdk-centos7 AS builder ARG version RUN set -eux \ & amp; & amp; yum -y update \ & amp; & amp; yum -y install curl gnupg unzip \ & amp; & amp; yum clean all -y RUN curl -L https://archive.apache.org/dist/rocketmq/${version}/rocketmq-all-${version}-bin-release.zip -o rocketmq.zip \ & amp; & amp; curl -L https://archive.apache.org/dist/rocketmq/${version}/rocketmq-all-${version}-bin-release.zip.asc -o rocketmq.zip. asc \ & amp; & amp; curl -L https://www.apache.org/dist/rocketmq/KEYS -o KEYS \ & amp; & amp; gpg --import KEYS \ & amp; & amp; gpg --batch --verify rocketmq.zip.asc rocketmq.zip RUN unzip rocketmq.zip \ & amp; & amp; mkdir -p /tmp/rocketmq-${version} \ & amp; & amp; mv rocketmq*/* /tmp/rocketmq-${version} ################################################ ############################## #Build stage 2: # Make the actual RocketMQ docker image ################################################ ############################## FROM eclipse-temurin:8-jdk-centos7 ARG user=rocketmq ARG group=rocketmq ARG uid=3000 ARG gid=3000 ARG version # Rocketmq version ENV ROCKETMQ_VERSION ${version} # Rocketmq home ENV ROCKETMQ_HOME /home/rocketmq/rocketmq-${ROCKETMQ_VERSION} # expose namesrv port EXPOSE 9876 # expose ports broker EXPOSE 10909 10911 10912 # RocketMQ is run with user `rocketmq`, uid = 3000 # If you bind mount a volume from the host or a data container, # ensure you use the same uid RUN groupadd -g ${gid} ${group} \ & amp; & amp; useradd -l -u ${uid} -g ${gid} -m -s /bin/bash ${user} \ & amp; & amp; yum -y update \ & amp; & amp; yum -y install less openssl which bash wget curl tzdata \ & amp; & amp; yum clean all -y & amp; & amp; rm -rf /var/cache/yum # Copy customized scripts COPY scripts/ ${ROCKETMQ_HOME}/bin/ # Copy RocketMQ artifact from builder COPY --from=builder --chown=${uid}:${gid} /tmp/rocketmq-${version}/ ${ROCKETMQ_HOME} # Override customized scripts for namesrv # Override customized scripts for broker # Export Java options # Add ${JAVA_HOME}/lib/ext as java.ext.dirs RUN mv ${ROCKETMQ_HOME}/bin/runserver-customize.sh ${ROCKETMQ_HOME}/bin/runserver.sh \ & amp; & amp; mv ${ROCKETMQ_HOME}/bin/runbroker-customize.sh ${ROCKETMQ_HOME}/bin/runbroker.sh \ & amp; & amp; chmod -R a + x ${ROCKETMQ_HOME}/bin/ \ & amp; & amp; export JAVA_OPT=" -Duser.home=/opt" \ & amp; & amp; sed -i 's/${JAVA_HOME}\/jre\/lib\/ext/${JAVA_HOME}\/jre\/lib\/ext:${JAVA_HOME }\/lib\/ext/' ${ROCKETMQ_HOME}/bin/tools.sh \ & amp; & amp; chown -R ${uid}:${gid} ${ROCKETMQ_HOME} \ & amp; & amp; localedef -i en_US -f UTF-8 en_US.UTF-8 \ & amp; & amp; ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime \ & amp; & amp; export LANG=C.UTF-8 USER ${user} WORKDIR ${ROCKETMQ_HOME}/bin ENTRYPOINT ["./docker-entrypoint.sh"] # Dummy overridable parameter parsed by entrypoint CMD ["dummy"]2. Write the required script files
# Script file docker-entrypoint.sh cat docker-entrypoint.sh #!/bin/bash set -e # Allow user specify custom CMD, maybe run /bin/bash to check the image if [[ "$1" == "nameserver" || "${NODE_ROLE}" == "nameserver" ]]; then shift exec ./mqnamesrv "${@}" elif [[ "$1" == "broker" || "${NODE_ROLE}" == "broker" ]]; then shift exec ./mqbroker "${@}" elif [[ "$1" == "controller" || "${NODE_ROLE}" == "controller" ]]; then shift exec ./mqcontroller "${@}" else # Run whatever command the user wants exec "$@" fi# Script file runbroker-customize.sh #!/bin/bash # Licensed to the Apache Software Foundation (ASF) under one or more # contributor license agreements. See the NOTICE file distributed with # this work for additional information regarding copyright ownership. # The ASF licenses this file to You under the Apache License, Version 2.0 # (the "License"); you may not use this file except in compliance with # the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. #================================================== ========================================== # Java Environment Setting #================================================== ========================================== error_exit() { echo "ERROR: $1 !!" exit 1 } find_java_home() { case "`uname`" in Darwin) JAVA_HOME=$(/usr/libexec/java_home) ;; *) JAVA_HOME=$(dirname $(dirname $(readlink -f $(which javac)))) ;; esac } find_java_home [ ! -e "$JAVA_HOME/bin/java" ] & amp; & amp; JAVA_HOME=$HOME/jdk/java [ ! -e "$JAVA_HOME/bin/java" ] & amp; & amp; JAVA_HOME=/usr/java [ ! -e "$JAVA_HOME/bin/java" ] & amp; & amp; error_exit "Please set the JAVA_HOME variable in your environment, We need java(x64)!" export JAVA_HOME export JAVA="$JAVA_HOME/bin/java" export BASE_DIR=$(dirname $0)/.. export CLASSPATH=.:${BASE_DIR}/conf:${CLASSPATH} #================================================== ========================================== #JVM Configuration #================================================== ========================================== calculate_heap_sizes() { case "`uname`" in Linux) system_memory_in_mb=`free -m| sed -n '2p' | awk '{print $2}'` system_cpu_cores=`egrep -c 'processor([[:space:]] + ):.*' /proc/cpuinfo` ;; FreeBSD) system_memory_in_bytes=`sysctl hw.physmem | awk '{print $2}'` system_memory_in_mb=`expr $system_memory_in_bytes / 1024 / 1024` system_cpu_cores=`sysctl hw.ncpu | awk '{print $2}'` ;; SunOS) system_memory_in_mb=`prtconf | awk '/Memory size:/ {print $3}'` system_cpu_cores=`psrinfo | wc -l` ;; Darwin) system_memory_in_bytes=`sysctl hw.memsize | awk '{print $2}'` system_memory_in_mb=`expr $system_memory_in_bytes / 1024 / 1024` system_cpu_cores=`sysctl hw.ncpu | awk '{print $2}'` ;; *) # assume reasonable defaults for e.g. a modern desktop or #cheapserver system_memory_in_mb="2048" system_cpu_cores="2" ;; esac # some systems like the raspberry pi don't report cores, use at least 1 if [ "$system_cpu_cores" -lt "1" ] then system_cpu_cores="1" fi # set max heap size based on the following # max(min(1/2 ram, 1024MB), min(1/4 ram, 8GB)) # calculate 1/2 ram and cap to 1024MB # calculate 1/4 ram and cap to 8192MB # pick the max half_system_memory_in_mb=`expr $system_memory_in_mb / 2` quarter_system_memory_in_mb=`expr $half_system_memory_in_mb / 2` if [ "$half_system_memory_in_mb" -gt "1024" ] then half_system_memory_in_mb="1024" fi if [ "$quarter_system_memory_in_mb" -gt "8192" ] then quarter_system_memory_in_mb="8192" fi if [ "$half_system_memory_in_mb" -gt "$quarter_system_memory_in_mb" ] then max_heap_size_in_mb="$half_system_memory_in_mb" else max_heap_size_in_mb="$quarter_system_memory_in_mb" fi MAX_HEAP_SIZE="${max_heap_size_in_mb}M" # Young gen: min(max_sensible_per_modern_cpu_core * num_cores, 1/4 * heap size) max_sensible_yg_per_core_in_mb="100" max_sensible_yg_in_mb=`expr $max_sensible_yg_per_core_in_mb "*" $system_cpu_cores` desired_yg_in_mb=`expr $max_heap_size_in_mb / 4` if [ "$desired_yg_in_mb" -gt "$max_sensible_yg_in_mb" ] then HEAP_NEWSIZE="${max_sensible_yg_in_mb}M" else HEAP_NEWSIZE="${desired_yg_in_mb}M" fi } calculate_heap_sizes # Dynamically calculate parameters, for reference. Xms=$MAX_HEAP_SIZE Xmx=$MAX_HEAP_SIZE Xmn=$HEAP_NEWSIZE MaxDirectMemorySize=$MAX_HEAP_SIZE # Set for `JAVA_OPT`. JAVA_OPT="${JAVA_OPT} -server -Xms${Xms} -Xmx${Xmx} -Xmn${Xmn}" JAVA_OPT="${JAVA_OPT} -XX: + UseG1GC -XX:G1HeapRegionSize=16m -XX:G1ReservePercent=25 -XX:InitiatingHeapOccupancyPercent=30 -XX:SoftRefLRUPolicyMSPerMB=0 -XX:SurvivorRatio=8" JAVA_OPT="${JAVA_OPT} -verbose:gc -Xloggc:/dev/shm/mq_gc_%p.log -XX: + PrintGCDetails -XX: + PrintGCDateStamps -XX: + PrintGCApplicationStoppedTime -XX: + PrintAdaptiveSizePolicy" JAVA_OPT="${JAVA_OPT} -XX: + UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=30m" JAVA_OPT="${JAVA_OPT} -XX:-OmitStackTraceInFastThrow" JAVA_OPT="${JAVA_OPT} -XX: + AlwaysPreTouch" JAVA_OPT="${JAVA_OPT} -XX:MaxDirectMemorySize=${MaxDirectMemorySize}" JAVA_OPT="${JAVA_OPT} -XX:-UseLargePages -XX:-UseBiasedLocking" JAVA_OPT="${JAVA_OPT} -Djava.ext.dirs=${JAVA_HOME}/jre/lib/ext:${BASE_DIR}/lib" #JAVA_OPT="${JAVA_OPT} -Xdebug -Xrunjdwp:transport=dt_socket,address=9555,server=y,suspend=n" JAVA_OPT="${JAVA_OPT} ${JAVA_OPT_EXT}" JAVA_OPT="${JAVA_OPT} -cp ${CLASSPATH}" numactl --interleave=all pwd > /dev/null 2> & amp;1 if [ $? -eq 0 ] then if [ -z "$RMQ_NUMA_NODE" ] ; then numactl --interleave=all $JAVA ${JAVA_OPT} $@ else numactl --cpunodebind=$RMQ_NUMA_NODE --membind=$RMQ_NUMA_NODE $JAVA ${JAVA_OPT} $@ fi else $JAVA ${JAVA_OPT} $@ fi# Script file runserver-customize.sh #!/bin/bash # Licensed to the Apache Software Foundation (ASF) under one or more # contributor license agreements. See the NOTICE file distributed with # this work for additional information regarding copyright ownership. # The ASF licenses this file to You under the Apache License, Version 2.0 # (the "License"); you may not use this file except in compliance with # the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. #================================================== ========================================== # Java Environment Setting #================================================== ========================================== error_exit() { echo "ERROR: $1 !!" exit 1 } find_java_home() { case "`uname`" in Darwin) JAVA_HOME=$(/usr/libexec/java_home) ;; *) JAVA_HOME=$(dirname $(dirname $(readlink -f $(which javac)))) ;; esac } find_java_home [ ! -e "$JAVA_HOME/bin/java" ] & amp; & amp; JAVA_HOME=$HOME/jdk/java [ ! -e "$JAVA_HOME/bin/java" ] & amp; & amp; JAVA_HOME=/usr/java [ ! -e "$JAVA_HOME/bin/java" ] & amp; & amp; error_exit "Please set the JAVA_HOME variable in your environment, We need java(x64)!" export JAVA_HOME export JAVA="$JAVA_HOME/bin/java" export BASE_DIR=$(dirname $0)/.. export CLASSPATH=.:${BASE_DIR}/conf:${CLASSPATH} #================================================== ========================================== #JVM Configuration #================================================== ========================================== calculate_heap_sizes() { case "`uname`" in Linux) system_memory_in_mb=`free -m| sed -n '2p' | awk '{print $2}'` system_cpu_cores=`egrep -c 'processor([[:space:]] + ):.*' /proc/cpuinfo` ;; FreeBSD) system_memory_in_bytes=`sysctl hw.physmem | awk '{print $2}'` system_memory_in_mb=`expr $system_memory_in_bytes / 1024 / 1024` system_cpu_cores=`sysctl hw.ncpu | awk '{print $2}'` ;; SunOS) system_memory_in_mb=`prtconf | awk '/Memory size:/ {print $3}'` system_cpu_cores=`psrinfo | wc -l` ;; Darwin) system_memory_in_bytes=`sysctl hw.memsize | awk '{print $2}'` system_memory_in_mb=`expr $system_memory_in_bytes / 1024 / 1024` system_cpu_cores=`sysctl hw.ncpu | awk '{print $2}'` ;; *) # assume reasonable defaults for e.g. a modern desktop or #cheapserver system_memory_in_mb="2048" system_cpu_cores="2" ;; esac # some systems like the raspberry pi don't report cores, use at least 1 if [ "$system_cpu_cores" -lt "1" ] then system_cpu_cores="1" fi # set max heap size based on the following # max(min(1/2 ram, 1024MB), min(1/4 ram, 8GB)) # calculate 1/2 ram and cap to 1024MB # calculate 1/4 ram and cap to 8192MB # pick the max half_system_memory_in_mb=`expr $system_memory_in_mb / 2` quarter_system_memory_in_mb=`expr $half_system_memory_in_mb / 2` if [ "$half_system_memory_in_mb" -gt "1024" ] then half_system_memory_in_mb="1024" fi if [ "$quarter_system_memory_in_mb" -gt "8192" ] then quarter_system_memory_in_mb="8192" fi if [ "$half_system_memory_in_mb" -gt "$quarter_system_memory_in_mb" ] then max_heap_size_in_mb="$half_system_memory_in_mb" else max_heap_size_in_mb="$quarter_system_memory_in_mb" fi MAX_HEAP_SIZE="${max_heap_size_in_mb}M" # Young gen: min(max_sensible_per_modern_cpu_core * num_cores, 1/4 * heap size) max_sensible_yg_per_core_in_mb="100" max_sensible_yg_in_mb=`expr $max_sensible_yg_per_core_in_mb "*" $system_cpu_cores` desired_yg_in_mb=`expr $max_heap_size_in_mb / 4` if [ "$desired_yg_in_mb" -gt "$max_sensible_yg_in_mb" ] then HEAP_NEWSIZE="${max_sensible_yg_in_mb}M" else HEAP_NEWSIZE="${desired_yg_in_mb}M" fi } calculate_heap_sizes # Dynamically calculate parameters, for reference. Xms=$MAX_HEAP_SIZE Xmx=$MAX_HEAP_SIZE Xmn=$HEAP_NEWSIZE # Set for `JAVA_OPT`. JAVA_OPT="${JAVA_OPT} -server -Xms${Xms} -Xmx${Xmx} -Xmn${Xmn}" JAVA_OPT="${JAVA_OPT} -XX: + UseConcMarkSweepGC -XX: + UseCMSCompactAtFullCollection -XX:CMSInitiatingOccupancyFraction=70 -XX: + CMSParallelRemarkEnabled -XX:SoftRefLRUPolicyMSPerMB=0 -XX: + CMSClassUnloadingEnabled -XX:SurvivorRatio=8 -XX:- UseParNewGC" JAVA_OPT="${JAVA_OPT} -verbose:gc -Xloggc:/dev/shm/rmq_srv_gc.log -XX: + PrintGCDetails" JAVA_OPT="${JAVA_OPT} -XX:-OmitStackTraceInFastThrow" JAVA_OPT="${JAVA_OPT} -XX:-UseLargePages" JAVA_OPT="${JAVA_OPT} -Djava.ext.dirs=${JAVA_HOME}/jre/lib/ext:${BASE_DIR}/lib" #JAVA_OPT="${JAVA_OPT} -Xdebug -Xrunjdwp:transport=dt_socket,address=9555,server=y,suspend=n" JAVA_OPT="${JAVA_OPT} ${JAVA_OPT_EXT}" JAVA_OPT="${JAVA_OPT} -cp ${CLASSPATH}" $JAVA ${JAVA_OPT} $@
Write the build script build-image.sh
# Script to build image
#!/usr/bin/env bash
checkVersion() {
echo "Version = $1"
echo $1 |grep -E "^[0-9] + \.[0-9] + \.[0-9] + " > /dev/null
if [ $? = 0 ]; then
return 1
fi
echo "Version $1 illegal, it should be X.X.X format(e.g. 4.5.0), please check released versions in 'https://archive.apache.org/dist/rocketmq/'"
exit -1
}
if [ $# -lt 2 ]; then
echo -e "Usage: sh $0 Version BaseImage"
exit -1
fi
ROCKETMQ_VERSION=$1
BASE_IMAGE=centos
checkVersion $ROCKETMQ_VERSION
# Build rocketmq
case "${BASE_IMAGE}" in
centos)
docker run --privileged --rm harbor.codemiracle.com.cn/baseapp/binfmt:latest --install all
docker buildx create --use --name=mybuilder-rocketmq --driver docker-container --driver-opt image=harbor.codemiracle.com.cn/baseapp/buildkit:master
docker buildx build --no-cache -f Dockerfile-centos --platform=linux/amd64,linux/arm64 -t harbor.codemiracle.com.cn/baseapp/rocketmq:${ROCKETMQ_VERSION} --build-arg version=$ {ROCKETMQ_VERSION} . --push
docker buildx rm mybuilder-rocketmq
;;
*)
echo "${BASE_IMAGE} is not supported, supported base images: centos"
exit -1
;;
esac
How to build
# How to use script to build?
1. Create relevant directories and place files in the corresponding directories
mkdir scripts
mv docker-entrypoint.sh scripts/
mv runbroker-customize.sh scripts/
mv runserver-customize.sh scripts/
2. Log in to harbor
docker login harbor.codemiracle.com.cn
3. Execute the build script
sh build-image.sh [rocketmq version number]
eg:
sh build-image.sh 4.5.2
How to deploy
https://blog.csdn.net/qq_34777982/article/details/133806164
References:
GitHub – apache/rocketmq-docker: Apache RocketMQ Docker