4 Case Demonstration – Customized image to run Nginx and tomcat services and realize dynamic and static separation based on NFS

One Early Environment Instructions

Prepare harbor and nfs server, nfs service: 172.31.7.122
The harbor service is harbor.magedu.com, and the project has been created in advance

Two prepare docker image

2.1 Create 4 basic images, centos, nginx, tomcat, jdk

2.1.1 centos mirror

dockfile

[root@k8s-master1 centos]# cat Dockerfile
#Customize Centos base image
FROM centos:7.9.2009
MAINTAINER Jack.Zhang [email protected]

ADD filebeat-7.12.1-x86_64.rpm /tmp
RUN yum install -y /tmp/filebeat-7.12.1-x86_64.rpm vim wget tree lrzsz gcc gcc-c + + automake pcre pcre-devel zlib zlib-devel openssl openssl-devel iproute net-tools iotop & amp; & amp ; rm -rf /etc/localtime /tmp/filebeat-7.12.1-x86_64.rpm & amp; & amp; ln -snf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime

Build and upload

[root@k8s-master1 centos]# cat build-command.sh
#!/bin/bash
docker build -t harbor.magedu.com/baseimages/magedu-centos-base:7.9.2009 .

docker push harbor.magedu.com/baseimages/magedu-centos-base:7.9.2009

2.1.2 jdk image construction

Directory structure

dockfile

[root@k8s-master1 jdk-1.8.212]# cat Dockerfile
#JDK Base Image
FROM harbor.magedu.com/baseimages/magedu-centos-base:7.9.2009

MAINTAINER zhangshijie "[email protected]"


ADD jdk-8u212-linux-x64.tar.gz /usr/local/src/
RUN ln -sv /usr/local/src/jdk1.8.0_212 /usr/local/jdk
ADD profile /etc/profile


ENV JAVA_HOME /usr/local/jdk
ENV JRE_HOME $JAVA_HOME/jre
 
ENV CLASSPATH $JAVA_HOME/lib/:$JRE_HOME/lib/
ENV PATH $PATH:$JAVA_HOME/bin

Build and upload

#!/bin/bash
docker build -t harbor.magedu.com/pub-images/jdk-base:v8.212 .
sleep 1
docker push harbor.magedu.com/pub-images/jdk-base:v8.212

2.1.3 nginx image building

dockfile content

[root@k8s-master1 nginx-base]# cat Dockerfile
#Nginx Base Image
FROM harbor.magedu.com/baseimages/magedu-centos-base:7.9.2009

MAINTAINER [email protected]

RUN yum install -y vim wget tree lrzsz gcc gcc-c + + automake pcre pcre-devel zlib zlib-devel openssl openssl-devel iproute net-tools iotop
ADD nginx-1.20.2.tar.gz /usr/local/src/
RUN cd /usr/local/src/nginx-1.20.2 & amp; & amp; ./configure & amp; & amp; make & amp; & amp; make install & amp; & amp; ln -sv /usr/ local/nginx/sbin/nginx /usr/sbin/nginx & amp; & amp;rm -rf /usr/local/src/nginx-1.20.2.tar.gz

Build and upload

#!/bin/bash
docker build -t harbor.magedu.com/pub-images/nginx-base:v1.20.2 .
sleep 1
docker push harbor.magedu.com/pub-images/nginx-base:v1.20.2

2.1.4 tomcat image building

dockfile content:

[root@k8s-master1 tomcat-base-8.5.43]# cat Dockerfile
#Tomcat 8.5.43 base image
FROM harbor.magedu.com/pub-images/jdk-base:v8.212

MAINTAINER zhangshijie "[email protected]"

RUN mkdir /apps /data/tomcat/webapps /data/tomcat/logs -pv
ADD apache-tomcat-8.5.43.tar.gz /apps
RUN useradd tomcat -u 2050 & amp; & amp; ln -sv /apps/apache-tomcat-8.5.43 /apps/tomcat & amp; & amp; chown -R tomcat.tomcat /apps /data

Build and upload

[root@k8s-master1 tomcat-base-8.5.43]# cat build-command.sh
#!/bin/bash
docker build -t harbor.magedu.com/pub-images/tomcat-base:v8.5.43 .
sleep 3
docker push harbor.magedu.com/pub-images/tomcat-base:v8.5.43

2.2 Create two business images, tomcat, nginx (used by k8s)

Directory structure

2.2.1 tomcat image construction

dockfile content

#tomcat web1
FROM harbor.magedu.com/pub-images/tomcat-base:v8.5.43

ADD catalina.sh /apps/tomcat/bin/catalina.sh
ADD server.xml /apps/tomcat/conf/server.xml #Configuration file
#ADD myapp/* /data/tomcat/webapps/myapp/
ADD app1.tar.gz /data/tomcat/webapps/myapp/ #Code directory
ADD run_tomcat.sh /apps/tomcat/bin/run_tomcat.sh #Startup script
#ADD filebeat.yml /etc/filebeat/filebeat.yml
RUN chown -R tomcat.tomcat /data/ /apps/
#ADD filebeat-7.5.1-x86_64.rpm /tmp/
#RUN cd /tmp & amp; & yum localinstall -y filebeat-7.5.1-amd64.deb

EXPOSE 8080 8443

CMD ["/apps/tomcat/bin/run_tomcat.sh"]

Build and upload. When executing this script, you need to pass a parameter, which is the version number of your code.

#!/bin/bash
TAG=$1
docker build -t harbor.magedu.com/magedu/tomcat-app1:${TAG} .
sleep 3
docker push harbor.magedu.com/magedu/tomcat-app1:${TAG}

2.2.2 nginx image building

dockfile content

#Nginx 1.20.2
FROM harbor.magedu.com/pub-images/nginx-base:v1.20.2


RUN useradd tomcat -u 2050
ADD nginx.conf /usr/local/nginx/conf/nginx.conf
ADD app1.tar.gz /usr/local/nginx/html/webapp/ #Static file path
ADD index.html /usr/local/nginx/html/index.html #Static file path

#Static resource mounting path
RUN mkdir -p /usr/local/nginx/html/webapp/static /usr/local/nginx/html/webapp/images & amp; & amp; chown tomcat.tomcat -R /usr/local/nginx/html/webapp /static /usr/local/nginx/html/webapp/images

EXPOSE 80 443

CMD ["nginx"]

Build and upload, but also pass a parameter

[root@k8s-master1 nginx]# cat build-command.sh
#!/bin/bash
TAG=$1
docker build -t harbor.magedu.com/magedu/nginx-web1:${TAG} .
echo "The image is built and will be uploaded to harbor"
sleep 1
docker push harbor.magedu.com/magedu/nginx-web1:${TAG}
echo "Image upload to harbor completed"

nginx.conf configuration file description: We want to use nginx to proxy tomcat, so the svc address of tomcat must be written behind the server.

user tomcat tomcat;
worker_processes auto;

#error_log logs/error.log;
#error_log logs/error.log notice;
#error_log logs/error.log info;

#pid logs/nginx.pid;
daemon off;

events {
    worker_connections 1024;
}


http {
    include mime.types;
    default_type application/octet-stream;

    #log_format main '$remote_addr - $remote_user [$time_local] "$request" '
    # '$status $body_bytes_sent "$http_referer" '
    # '"$http_user_agent" "$http_x_forwarded_for"';

    #access_log logs/access.log main;

    sendfile on;
    #tcp_nopush on;

    #keepalive_timeout 0;
    keepalive_timeout 65;

    #gzip on;

upstream tomcat_webserver {
        server magedu-tomcat-app1-service.magedu.svc.magedu.local:80;
}

    server {
        listen 80;
        server_name localhost;

        #charset koi8-r;

        #access_log logs/host.access.log main;

        location/{
            roothtml;
            index index.html index.htm;
        }

        location /webapp {
            roothtml;
            index index.html index.htm;
        }

        location /myapp {
             proxy_pass http://tomcat_webserver;
             proxy_set_header Host $host;
             proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
             proxy_set_header X-Real-IP $remote_addr;
        }
}

Three Prepare yaml file

3.1 nginx’s yaml

kubectl apply -f /yaml/docker/yaml/magedu/nginx/nginx.yaml

kind: Deployment
apiVersion: apps/v1
metadata:
  labels:
    app: magedu-nginx-deployment-label
  name: magedu-nginx-deployment
  namespace: magedu
spec:
  replicas: 1
  selector:
    matchLabels:
      app: magedu-nginx-selector
  template:
    metadata:
      labels:
        app: magedu-nginx-selector
    spec:
      containers:
      - name: magedu-nginx-container
        image: harbor.magedu.com/magedu/nginx-web1:202205041446 #nginx business mirror address
        #command: ["/apps/tomcat/bin/run_tomcat.sh"]
        #imagePullPolicy: IfNotPresent
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 80
          protocol: TCP
          name: http
        - containerPort: 443
          protocol: TCP
          name: https
        env:
        - name: "password"
          value: "123456"
        - name: "age"
          value: "20"

        volumeMounts:
        - name:magedu-images
          mountPath: /usr/local/nginx/html/webapp/images
          readOnly: false
        - name: magedu-static
          mountPath: /usr/local/nginx/html/webapp/static
          readOnly: false
      volumes:
      - name:magedu-images
        nfs:
          server: 172.31.7.122
          path: /data/k8sdata/magedu/images
      - name: magedu-static
        nfs:
          server: 172.31.7.122
          path: /data/k8sdata/magedu/static
      #nodeSelector:
      #group: magedu

    

---
Kind: Service
apiVersion: v1
metadata:
  labels:
    app: magedu-nginx-service-label
  name: magedu-nginx-service
  namespace: magedu
spec:
  type: NodePort
  ports:
  - name: http
    port: 80
    protocol: TCP
    targetPort: 80
    nodePort: 30090
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
    nodePort: 30091
  selector:
    app: magedu-nginx-selector

3.2 tomcat’s yaml

kubectl apply -f tomcat-app1.yaml

kind: Deployment
#apiVersion: extensions/v1beta1
apiVersion: apps/v1
metadata:
  labels:
    app: magedu-tomcat-app1-deployment-label
  name: magedu-tomcat-app1-deployment
  namespace: magedu
spec:
  replicas: 2
  selector:
    matchLabels:
      app: magedu-tomcat-app1-selector
  template:
    metadata:
      labels:
        app: magedu-tomcat-app1-selector
    spec:
      containers:
      - name: magedu-tomcat-app1-container
        image: harbor.magedu.com/magedu/tomcat-app1:202205041153
        #command: ["/apps/tomcat/bin/run_tomcat.sh"]
        imagePullPolicy: IfNotPresent
        #imagePullPolicy: Always
        ports:
        - containerPort: 8080
          protocol: TCP
          name: http
        env:
        - name: "password"
          value: "123456"
        - name: "age"
          value: "18"
        resources:
          limits:
            cpu: 1
            memory: "512Mi"
          requests:
            cpu: 500m
            memory: "512Mi"
        volumeMounts:
        - name:magedu-images
          mountPath: /usr/local/nginx/html/webapp/images
          readOnly: false
        - name: magedu-static
          mountPath: /usr/local/nginx/html/webapp/static
          readOnly: false
      volumes:
      - name:magedu-images
        nfs:
          server: 172.31.7.122
          path: /data/k8sdata/magedu/images
      - name: magedu-static
        nfs:
          server: 172.31.7.122
          path: /data/k8sdata/magedu/static
# nodeSelector:
#project: magedu
# app: tomcat
---
Kind: Service
apiVersion: v1
metadata:
  labels:
    app: magedu-tomcat-app1-service-label
  name: magedu-tomcat-app1-service
  namespace: magedu
spec:
  #type: NodePort
  ports:
  - name: http
    port: 80
    protocol: TCP
    targetPort: 8080
    #nodePort: 30092
  selector:
    app: magedu-tomcat-app1-selector

After creating the two yamls, view the results

Four-verification results

Finally accessed through the domain name, here we use haproxy and the virtual IP generated by keppalived

4.1 haproxy configuration

Resolve the www.mysite.com domain name to 172.31.7.188 and change the host file

4.2 Access back-end tomcat content

Myapp is a service provided by my backend tomcat. It has two pods and is accessed in the default rotation mode, as shown in the figure:

4.3 Access nginx

4.3 Access the images in nginx, which exist on nfs storage