Analyze the principle of Node.js mirroring and easily build an efficient CI/CD process

Big factory technology advanced front-end Node advanced

Click above for Programmer Growth Guide and follow the official account

Reply 1, join the advanced Node communication group

1. Preface

If the company’s project uses containerized deployment, then you have more or less knowledge of nodejs mirroring, because front-end projects or BFF projects based on nodejs will rely on nodejs mirroring during the construction or deployment process.

Some students may have questions, what is there to know about the nodejs image? Just go to the docker image official website to search for the corresponding node version, and then find the corresponding version number and see if it is available. For example, if you want to find 16.20.0, as follows As shown in the picture

Then write the corresponding image version in the Dockerfile, as shown below

javascript
Copy code
FROM node:16.20.0

Sharp-eyed students may see that the same version, such as 16.20.0 and 14.19.1, has different tags, as shown in the figure below

Why does a node image version have so many tags? how should i choose

Why are there differences between tags 14 and 16 besides version numbers? Why

2. Parse nodejs image

Before understanding the nodejs image, if you are not familiar with Linux, we can first understand what debian is and what Alpine is. If you are familiar with this part, you can jump directly to the nodejs official image composition.

2.1 What is Debian

There are many Linux distributions, which are divided in nature into commercial versions maintained by commercial companies and free distributions maintained by open source communities. The commercial version is represented by Redhat, and the open source community version is represented by Debian. Each of these versions has different characteristics and plays different roles in different application fields.

Current popular distributions are as follows

Generally speaking, as an operating system suitable for servers, Debian is much more stable than Ubuntu. Debian’s entire system, as long as there are no logical flaws at the application level, there will basically be no problems. The basic core of Debian’s entire system is very small, which is not only stable, but also takes up little hard disk space and memory.

For more information, you can check out the similarities and differences between CentOS, Ubuntu, and Debian Linux.

Debian Distribution Versions Debian has maintained at least three distribution versions: stable version (stable), testing version (testing) and unstable version (unstable).

• Stable: The stable version contains the latest officially released software packages from Debian. As the official release version of Debian, it is the version we recommend to users first. The current stable version number of Debian is 12, and the development code is bookworm. The initial version was 12.0, released on June 10, 2023, and its update 12.1 was released on July 22, 2023.

• Testing: The testing version contains software packages that have not yet been included in the stable version, but they have entered the candidate queue. The biggest benefit of using this version is that it has more newer versions of the software. The current beta version is codenamed trixie.

• Unstable: The unstable version stores the current development work of Debian. Usually, only developers and those who like to live a thrilling life use this version, and the version code name of the unstable version is always called sid.

Debian Release Life Cycle and CatalogDebian usually releases a new stable version at regular intervals according to certain rules. For each stable release, users receive three years of full support and an additional two years of long-term support. The current release timeline is shown below

You can see that the latest stable version is 12

2.2 What is Alpine

Alpine is a lightweight, security-oriented Linux distribution. It is different from ordinary Linux distributions. Alpine uses musl libc and busybox to reduce the system size and runtime resource consumption. However, its functions are much more complete than busybox, so it is increasingly favored by the open source community. While maintaining a slim profile, Alpine also provides its own package management tool apk, which can directly query and install various software through the apk command.

2.3 Differences between Debian and Alpine

• The space size difference is about 5M by default for alpine and about 200M for debian.

• The default software packages are different. Alpine uses busybox, while debian uses bash + coreutils.

• In alpine, international components have been optimized away.

• Another point is that alpine uses “minimum dependencies”, which is similar to archlinux. For example, the openssh package does not come with its own pam plug-in, so it does not support ldap. I raised an issue with alpinelinux official about this. Unlike php, php can be made into php-pdo, php-dom packages, and then dynamically load shared libraries. openssh doesn’t work, “If you don’t bring it, you haven’t written it.”

• Differences in glibc, alpine uses musl, centos, etc. use glibc. Others are fine. The difference in libc is very important for development.

For more information, see What is the difference between Alpine Linux and CentOS?

2.4 nodejs official image composition

Now let’s look at the nodejs image composition

• node:: Based on the latest version of Debian, relying on the tool package buildpack-deps, the most basic image, providing the most commonly used Debian software packages, such as curl, bash, git, etc.

• node:alpine: Based on the popular Alpine Linux project, it provides the smallest node image. It does not have built-in commonly used software packages and needs to be installed through apk. In addition, musl libc is used instead of glibc. It only affects packages that rely on glibc, such as grpc. Category

• node:buster: image based on Debian10 version, dependent on tool package buildpack-deps

• node:bullseye: Based on the Debian11 version of the image, relying on the tool package buildpack-deps

• node:bookworm: image based on Debian12 version, dependent on toolkit buildpack-deps

• node:slim: Based on the current version of Debian, it only contains the most basic packages that can make the image run, such as bash and git, but does not include curl, etc.

Of course, there will be mirrors based on new Debian versions in the future.

Whether the image contains a certain toolkit can be searched directly on the image details page, as shown in the figure below

The overall structure of the mirror is shown in the figure below

The differences between images with different tags are as follows, taking 16.20.0 (arm64) as an example

Image tag	node version	Number of packages	Number of system vulnerabilities	Mirror Size	Whether yarn
node	16.20.0	751	230	856MB	?
node:bullseye	16.20 .0	751	121	890MB	?
node: bullseye-slim	16.20.0	347	26	184MB	?
node:bookworm	16.20.0	750	92	1.04GB	?
node:bookworm-slim	16.20.0	334	15	206MB	?
node:alpine	16.20.0	230	4	116MB	?

It can be concluded from the above table

• Alpine and slim have the smallest sizes. The image sizes of bullseye, bookworm and node:version are several times larger than those of alpine and slim.

• Alpine and Slim have the least number of vulnerabilities. Bullseye, bookworm and node:version have several times more vulnerabilities than Alpine and Slim.

So which version node:version corresponds to can be viewed through the versions.json file. Taking version 16.20.0 as an example, then node:16.20.0 is actually node:16.20.0-buster

In addition, you can also look at the composition of the mirror, as shown in the figure below

At this point we should be able to answer the previous two questions

Why does a node image version have so many tags? how should i choose

The reason is: the images built by nodejs based on different Linux versions can be selected according to the image size and function

Why are there differences between tags 14 and 16 besides version numbers? Why

The reason is: Debian is always releasing new versions, and nodejs only releases images based on the new Debian version for the latest nodejs version under maintenance

3. How to choose nodejs mirror

From the above we already know the meaning of each tag and the composition of the corresponding tag image, then we can choose according to our own usage scenarios.

3.1 web project

When building web projects, we recommend alpine mirroring. The reason is that when we build web projects, we generally only rely on the construction tools webpack, etc., and do not rely on some underlying tool libraries, and generally do not need to enter the container. Specific examples are as follows

dockerfile
Copy code
FROM node:16.20.0-alpine AS builder_web

WORKDIR/app

ADD ./package.json /app/package.json
ADD ./pnpm-lock.yaml /app/pnpm-lock.yaml

# Install dependencies
RUN pnpm install --frozen-lockfile

ADD ./app

# Build code
RUN pnpm build

FROM nginx:1.21.0

# The CDN synchronization script reads files from the /app/dist directory by default
COPY --from=builder_web /app/dist /app/dist
COPY --from=builder_web /app/config/nginx.conf /etc/nginx/conf.d/default.conf

3.2 BFF Project

To build a BFF project based on nodejs, it is recommended to use the alpine image during the construction phase and the slim image during the running and deployment phase. The reason is that the image contains some basic third-party toolkits to facilitate secondary installation when we enter the container.

dockerfile
Copy code
#Alpine can be used during the build phase
FROM node:16.20.0-alpine as bff_build

# Work list
WORKDIR/workspace/app


#Copy dependent files
COPY ./package.json /workspace/app/package.json
COPY ./yarn.lock /workspace/app/yarn.lock

# Install dependencies
RUN yarn --frozen-lockfile --check-files

#Copy source code
COPY ./workspace/app

# Construct
RUN yarn run build

#Basic image during deployment phase, use slim image
FROM node:16.20.0-slim

# Work list
WORKDIR/workspace/app

#Copy dependencies
COPY --from=bff_build /workspace/app/package.json /workspace/app/package.json
COPY --from=bff_build /workspace/app/node_modules /workspace/app/node_modules

#Copy content
COPY --from=bff_build /workspace/app /workspace/app

# start up
CMD yarn run run

3.3 npm package scenario

To publish npm packages, take gitlab-ci as an example. Alpine or slim images are recommended because they do not rely on toolkits. However, when encountering multi-package management tools such as lerna, they rely on git and can be based on the existing alpine or slim images. Install git online

yaml
Copy code
stages: stages
  - publish

before_script:
  - echo $CI_COMMIT_TAG

publish:
  stage: publish
  image: node:16.20.0-slim
  script:
    - echo -e $NPM_AUTH_CONTENT >> ~/.npmrc # Inject private warehouse token
    - yarn install --frozen-lockfile
    - |
      if [[ $CI_COMMIT_TAG == *"beta"* ]]; then
        echo 'release beta tag'
        yarn publish --tag beta --non-interactive --no-commit-hooks
      else
        echo 'Publish official tag'
        yarn publish --non-interactive --no-commit-hooks
      fi
  only:
    refs:
      - tags

Of course there are more scenarios, but according to the characteristics of each tag, you can choose the tag that suits your project.

4. Customize nodejs image

In addition to directly using the official nodejs image, we may also encapsulate the nodejs image suitable for our own company’s projects. The purpose is to add customized logic to the image to facilitate unified processing of common issues for all the company’s projects, such as setting the company’s npm Proxy sources, set some commonly used npm package third-party dependency variables, install pnpm, handle build failures caused by buildkit cache errors during the install and build processes, etc. There are two ideas for encapsulation

• Idea 1: Encapsulate from scratch

• Idea 2: Secondary packaging based on official images

4.1 Encapsulate your own nodejs image from scratch

Take the alpine image as an example

dockerfile
Copy code
# Select alpine version image
FROM alpine:3.18

# Define node version
ENV NODE_VERSION 16.20.0

# Set the source of apk to Alibaba source
RUN sed -i 's/dl-cdn.alpinelinux.org/mirrors.aliyun.com/g' /etc/apk/repositories & amp; & amp; apk update

# Set the source and set some binary dependency file hosts that are difficult to download in China.
COPY ./.npmrc /root/.npmrc

# Use apk to install dependencies
RUN addgroup -g 1000 node \
     & amp; & amp; adduser -u 1000 -G node -s /bin/sh -D node \
     & amp; & amp; apk add --no-cache \
        libstdc + + \
     & amp; & amp; apk add --no-cache --virtual .build-deps \
        curl \
     & amp; & amp; ARCH= & amp; & amp; alpineArch="$(apk --print-arch)" \
       & amp; & amp; case "${alpineArch##*-}" in \
        x86_64)\
          ARCH='x64' \
          CHECKSUM="d2df78a192bd78b958e19a77821916a38def5e9e46c0c9a0989fdf5eb6c14a7e" \
          ;; \
        *) ;; \
      esac \
   & amp; & amp; if [ -n "${CHECKSUM}" ]; then \
    set -eu; \
    curl -fsSLO --compressed "https://unofficial-builds.nodejs.org/download/release/v$NODE_VERSION/node-v$NODE_VERSION-linux-$ARCH-musl.tar.xz"; \
    echo "$CHECKSUM node-v$NODE_VERSION-linux-$ARCH-musl.tar.xz" | sha256sum -c - \
       & amp; & amp; tar -xJf "node-v$NODE_VERSION-linux-$ARCH-musl.tar.xz" -C /usr/local --strip-components=1 --no-same-owner \
       & amp; & amp; ln -s /usr/local/bin/node /usr/local/bin/nodejs; \
  else \
    echo "Building from source" \
    # backup build
     & amp; & amp; apk add --no-cache --virtual .build-deps-full \
        binutils-gold \
        g + + \
        gcc\
        gnupg \
        libgcc \
        linux-headers\
        make\
        python3\
    # use pre-existing gpg directory, see https://github.com/nodejs/docker-node/pull/1895#issuecomment-1550389150
     & amp; & amp; export GNUPGHOME="$(mktemp -d)" \
    # gpg keys listed at https://github.com/nodejs/node#release-keys
     & amp; & amp; for key in \
      4ED778F539E3634C779C87C6D7062848A1AB005C \
      141F07595B7B3FFE74309A937405533BE57C7D57 \
      74F12602B6F1C4E913FAA37AD3A89613643B6201 \
      DD792F5973C6DE52C432CBDAC77ABFA00DDBF2B7 \
      61FC681DFB92A079F1685E77973F295594EC4689 \
      8FCCA13FEF1D0C2E91008E09770F7A9A5AE15600 \
      C4F0DFFF4E8C1A8236409D08E73BC641CC11F4C8 \
      890C08DB8579162FEE0DF9DB8BEAB4DFCF555EF4 \
      C82FA3AE1CBEDC6BE46B9360C43CEC45C17AB93C \
      108F52B48DB57BB0CC439B2997B01419BD92F80A \
    ; do \
      gpg --batch --keyserver hkps://keys.openpgp.org --recv-keys "$key" || \
      gpg --batch --keyserver keyserver.ubuntu.com --recv-keys "$key" ; \
    done \
     & amp; & amp; curl -fsSLO --compressed "https://nodejs.org/dist/v$NODE_VERSION/node-v$NODE_VERSION.tar.xz" \
     & amp; & amp; curl -fsSLO --compressed "https://nodejs.org/dist/v$NODE_VERSION/SHASUMS256.txt.asc" \
     & amp; & amp; gpg --batch --decrypt --output SHASUMS256.txt SHASUMS256.txt.asc \
     & amp; & amp; gpgconf --kill all \
     & amp; & amp; rm -rf "$GNUPGHOME" \
     & amp; & amp; grep " node-v$NODE_VERSION.tar.xz\$" SHASUMS256.txt | sha256sum -c - \
     & amp; & amp; tar -xf "node-v$NODE_VERSION.tar.xz" \
     & amp; & amp; cd "node-v$NODE_VERSION" \
     & amp; & amp; ./configure \
     & amp; & amp; make -j$(getconf _NPROCESSORS_ONLN) V= \
     & amp; & amp; make install \
     & amp; & amp; apk del .build-deps-full \
     & amp; & amp; cd .. \
     & amp; & amp; rm -Rf "node-v$NODE_VERSION" \
     & amp; & amp; rm "node-v$NODE_VERSION.tar.xz" SHASUMS256.txt.asc SHASUMS256.txt; \
  fi \
   & amp; & amp; rm -f "node-v$NODE_VERSION-linux-$ARCH-musl.tar.xz" \
   & amp; & amp; apk del .build-deps \
  # smoke tests
   & amp; & amp; node --version \
   & amp; & amp; npm --version

ENV YARN_VERSION 1.22.19

RUN apk add --no-cache --virtual .build-deps-yarn curl gnupg tar \
  # use pre-existing gpg directory, see https://github.com/nodejs/docker-node/pull/1895#issuecomment-1550389150
   & amp; & amp; export GNUPGHOME="$(mktemp -d)" \
   & amp; & amp; for key in \
    6A010C5166006599AA17F08146C2130DFD2497F5 \
  ; do \
    gpg --batch --keyserver hkps://keys.openpgp.org --recv-keys "$key" || \
    gpg --batch --keyserver keyserver.ubuntu.com --recv-keys "$key" ; \
  done \
   & amp; & amp; curl -fsSLO --compressed "https://yarnpkg.com/downloads/$YARN_VERSION/yarn-v$YARN_VERSION.tar.gz" \
   & amp; & amp; curl -fsSLO --compressed "https://yarnpkg.com/downloads/$YARN_VERSION/yarn-v$YARN_VERSION.tar.gz.asc" \
   & amp; & amp; gpg --batch --verify yarn-v$YARN_VERSION.tar.gz.asc yarn-v$YARN_VERSION.tar.gz \
   & amp; & amp; gpgconf --kill all \
   & amp; & amp; rm -rf "$GNUPGHOME" \
   & amp; & amp; mkdir -p /opt \
   & amp; & amp; tar -xzf yarn-v$YARN_VERSION.tar.gz -C /opt/ \
   & amp; & amp; ln -s /opt/yarn-v$YARN_VERSION/bin/yarn /usr/local/bin/yarn \
   & amp; & amp; ln -s /opt/yarn-v$YARN_VERSION/bin/yarnpkg /usr/local/bin/yarnpkg \
   & amp; & amp; rm yarn-v$YARN_VERSION.tar.gz.asc yarn-v$YARN_VERSION.tar.gz \
   & amp; & amp; apk del .build-deps-yarn \
  # smoke test
   & amp; & amp; yarn --version

# Install pnpm dependencies
RUN npm install -g pnpm

COPY docker-entrypoint.sh /usr/local/bin/
ENTRYPOINT ["docker-entrypoint.sh"]

CMD [ "node" ]

In fact, it is to refer to the construction method of the official image and make appropriate modifications in it.

Of course, it can also be packaged based on some other Linux distribution, as shown below

yaml
Copy code
FROM centos:7
RUN curl -L https://dl.yarnpkg.com/rpm/yarn.repo -o /etc/yum.repos.d/yarn.repo
RUN curl --silent --location https://rpm.nodesource.com/setup_14.x | bash -
RUN yum install -y nodejs yarn
WORKDIR/code
EXPOSE 80
CMD npm start

4.2 Secondary packaging based on official image

Take the slim image as an example

dockerfile
Copy code
FROM node:16.20.0-slim

RUN sed -i s/deb.debian.org/archive.debian.org/g /etc/apt/sources.list & amp; & amp; \
    sed -i 's|security.debian.org|archive.debian.org|g' /etc/apt/sources.list & amp; & amp; \
    sed -i '/stretch-updates/d' /etc/apt/sources.list & amp; & amp; \
    apt-get clean & amp; & amp; \
    apt-get update & amp; & amp; \
    apt-get install -y tzdata tree git & amp; & amp; \
    ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime

ARG CUSTOM_NODE_VERSION

COPY ./.npmrc /root/.npmrc

RUN npm install -g [email protected] \
     & amp; & amp; pnpm config set store-dir /root/.pnpm-store \
     & amp; & amp; npm config set custom_node_version $CUSTOM_NODE_VERSION

In this case, the packaging will be more refreshing

As for the nodejs image you encapsulate, you can send it to your own private warehouse, such as the private warehouse built by Nexus, or the private warehouse purchased on Alibaba Cloud, Tencent Cloud, etc.

Of course, if we want to install some additional toolkits during packaging, due to domestic network problems, we generally set the debian source to the domestic source.

5. Summary

The official image of nodejs is mainly composed of three parts: linux version + toolkit collection + nodejs runtime. The three parts can be combined into different tags. Each combination has its application scenarios. We can customize it according to the size, Choose based on safety and functionality

Recommended for general projects: alpine or slim images, not recommended for complete imagesFor special projects, it is recommended to use alpine or slim images for secondary packaging to solve the company’s common problems and improve R&D efficiency >

For example, our company chooses to encapsulate its own nodejs image and will do the following things in the image

• Set up the company’s own npm proxy source

• Set the third-party dependent environment variables of the npm package, such as node-sass, puppeteer, etc.

• Install pnpm

• Solve the problem of install failure or build failure in CI scenarios caused by buildkit cache issues

• Pre-check before bff starts, etc.

Of course, if you pursue the ultimate in image size, you can delete the toolkit in the image, or copy only the executable file of the tool to reduce the image size.

Finally, by choosing the appropriate image, it can help us build an efficient CI/CD process.

Reference link

How to check Debian version

Debian distribution

Debian source usage help

What is the difference between Alpine Linux and CentOS?

Choosing the best Node.js Docker image

Original link: https://juejin.cn/user/3650034333393575/posts

If you like it, like it, read it again, and forward it. Thank you!

Node community

I have formed a Node.js community with a very good atmosphere. There are many Node.js friends in it. If you are interested in learning Node.js (you can also have plans in the future), we can do Node.js-related work together. Exchange, learn and build together. Add Koala friends below and reply “Node”.

"Share, Like, Watch" to support