openEuler 22.03 Docker runs containers of arm and other architectures under the x86 architecture – the road to dream building

Why do this?

With the popularization of localization and the support of national policies for the information and innovation industry, some financial and securities industries, government units, etc., have gradually begun to take the route of localization of information and innovation, and more and more people have come into contact with domestic CPUs (arm platform, such as Huawei’s Kunpeng processor)

The cost of buying a CPU for the arm platform was too much to bear, so I tried to use the x86 platform to run containers of the arm platform. Reduce costs and increase efficiency.

Environmental description

1. Operating system version: Huawei openEuler 22.03 lts x86_64

2. Docker version: 20.10.12 binary installation

3. buildx plug-in version: 0.9.1

Docker binary installation and buildx plug-in installation

1. Install docker in binary mode and configure it

# Install docker in binary mode and configure it

#Download binary offline compressed package
wget https://download.docker.com/linux/static/stable/x86_64/docker-20.10.12.tgz

#unzip files
tar -zxvf docker-20.10.12.tgz

chmod +x docker/*

#Copy binary files to /usr/bin/
cp docker/* /usr/bin/

#Register to system service
vim /usr/lib/systemd/system/docker.service
[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network-online.target firewalld.service
Wants=network-online.target
  
[Service]
Type=notify
#Specify docker storage directory/TRS/APP/docker/lib
ExecStart=/usr/bin/dockerd --graph /APP/docker/
ExecReload=/bin/kill -s HUP $MAINPID
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
asksMax=infinity
TimeoutStartSec=0
Delegate=yes
KillMode=process
Restart=on-failure
StartLimitBurst=3
StartLimitInterval=60s
  
[Install]
WantedBy=multi-user.target
  
#overload
mkdir -p /APP/docker/lib
systemctl daemon-reload

#Start the service and add it to boot
systemctl enable docker --now

#examine
systemctl status docker

#Configuration
mkdir -p /etc/docker

cat > /etc/docker/daemon.json <<EOF
{
  "registry-mirrors": ["https://docker.mirrors.ustc.edu.cn"],
  "exec-opts": ["native.cgroupdriver=systemd"],
  "experimental": true,
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m"
  },
  "storage-driver": "overlay2",
  "storage-opts": [
    "overlay2.override_kernel_check=true"
  ]
}
EOF

2. Install docker-buildx plug-in

# Create directory

mkdir -p ~/.docker/cli-plugins/

# Download the plug-in and change its name

wget -c https://github.com/docker/buildx/releases/download/v0.9.1/buildx-v0.9.1.linux-amd64

mv buildx-v0.9.1.linux-amd64 docker-buildx

chmod +x docker-buildx

mv docker-buildx ~/.docker/cli-plugins/

# Check verification

docker buildx version

docker buildx ls

Register a supported architecture interpreter

Without specifying the CPU platform, use register to register supported architecture parsers

docker run --rm \
> --privileged \
> multiarch/qemu-user-static:register \
> --reset

ls /proc/sys/fs/binfmt_misc/ 

cat /proc/sys/fs/binfmt_misc/qemu-mips64el

cat /proc/sys/fs/binfmt_misc/qemu-aarch64

cat /proc/sys/fs/binfmt_misc/qemu-ppc64le

Start the arm architecture container

1. Download qemu-aarch64-static

GitHub – multiarch/qemu-user-static: :earth_africa: `/usr/bin/qemu-*-static`

wget -c https://github.com/multiarch/qemu-user-static/releases/download/v7.2.0-1/qemu-aarch64-static

# Other architectures

https://github.com/multiarch/qemu-user-static/releases/download/v7.2.0-1/qemu-arm-static

https://github.com/multiarch/qemu-user-static/releases/download/v7.2.0-1/qemu-mips-static

https://github.com/multiarch/qemu-user-static/releases/download/v7.2.0-1/qemu-mips64-static

https://github.com/multiarch/qemu-user-static/releases/download/v7.2.0-1/qemu-mipsel-static

https://github.com/multiarch/qemu-user-static/releases/download/v7.2.0-1/qemu-ppc64-static

https://github.com/multiarch/qemu-user-static/releases/download/v7.2.0-1/qemu-s390x-static

https://github.com/multiarch/qemu-user-static/releases/download/v7.2.0-1/qemu-x86_64-static

chmod + x qemu-aarch64-static

When starting the container, bring qemu-aarch64-static into the container

Note that the path of the qemu-aarch64-static binary file can be summarized to the specified path by yourself. You only need to bring it into the /usr/bin directory in the container

Simple test

# Run a command to view the architecture

docker run -t --rm --platform arm64 -v `pwd`/qemu-aarch64-static:/usr/bin/qemu-aarch64-static alpine uname -m

docker run -i --rm --platform arm64 -v `pwd`/qemu-aarch64-static:/usr/bin/qemu-aarch64-static debian:11 uname -m

#Create a container and open a terminal

docker run -it --rm --platform arm64 -v `pwd`/qemu-aarch64-static:/usr/bin/qemu-aarch64-static debian:11 /bin/bash

Similarly, the image to start the ppc64le architecture is as follows:

wget -c https://github.com/multiarch/qemu-user-static/releases/download/v7.2.0-1/qemu-ppc64le-static

chmod +x qemu-ppc64le-static

# Simple test

docker run -t \
--rm \
--platform ppc64le \
-v $(pwd)/qemu-ppc64le-static:/usr/bin/qemu-ppc64le-static \
alpine \
uname -m

Two non-x86 architecture images were tested above, and both can run normally. Other architectures are similar, so I won’t go into details here.

Loongson architecture here, special instructions

loongarch architecture support

1. Register interpreter

wget https://gitee.com/michael0066/qemu-loongarch64-static/blob/master/qemu-loongarch64-static

chmod + x qemu-loongarch64-static
# Register

echo ":qemu-loongarch64:M::\x7fELF\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00 \x02\x00\x02\x01:\xff\xff\xff\xff\xff\xfe\xfe\x00\xff\xff\xff\xff\ \xff\xff\xff\xff\xfe\xff\xff\xff:/usr/bin/qemu-loongarch64-static:" > /proc/sys/fs/binfmt_misc/register

# Check
ls /proc/sys/fs/binfmt_misc/ |grep loonqemu-loongarch64

2. Simple test

docker run --rm \
--platform loongarch64 -t \
-v `pwd`/qemu-loongarch64-static:/usr/bin/qemu-loongarch64-static \
loongsongd/debian10_loongarch64_x64:e15.0 \
uname -m

References:

x86 architecture operation other architecture images – Jianshu

https://hub.docker.com/r/loongsongd/debian10_loongarch64_x64

Build arm image

Take building an arm64 image as an example to illustrate how to build images for other platform architectures.

1. Write Dockerfile

FROM centos:7.9.2009
COPY ./qemu-aarch64-static /usr/bin/qemu-aarch64-static
RUN yum install -y net-tools gcc gcc-c + + make vim & amp; & amp; \
    yum clean all

2. Build image

docker build \
--platform arm64 \
-t centos_make:7.9_aarch64 .

3. Check and verify

docker inspect centos_make:7.9_aarch64 | grep -i 'architecture'

another way:

docker build --rm -t "arm64v8/redis_new" -<<EOF
FROM multiarch/qemu-user-static:x86_64-aarch64 as qemu
FROM arm64v8/redis
COPY --from=qemu /usr/bin/qemu-aarch64-static /usr/bin
EOF

docker run --rm -t "arm64v8/redis_new" uname -m
#aarch64

The buildx plug-in is not used here to build images of other architectures. It provides another idea and method for building images of other architectures. However, when building multi-architecture docker images in general cloud native scenarios, it is recommended to use the docker buildx method, which is more efficient.