Instructions for setting up and using Docker

Article directory

  • Preface
  • 1. Introduction
  • 2. Install docker
  • 3. docker image
  • 4. Create a new docker image
    • 1. Build through Dockerfile
    • 2. Modify based on the open source image on DockerHub and build an image suitable for yourself.
  • 5. Create a new docker container
    • 1. Enter docker
    • 2.Example
    • 3. Enter the container you created
    • 4. You can exit docker
  • 6. docker commit: Create a new image from the container.
  • 7. Package the existing docker into tar and load the packaged docker image.
  • 8. Connect to your own docker in the server in vscode
    • 1. Open the vscode remote resource manager and connect to the server

Foreword

Mainly records the installation of docker in a linux server environment and related instructions for using docker

1. Introduction

  • Docker is an open source application container engine that allows developers to package their applications and dependencies into a portable image and then publish it to any popular Linux or Windows machine, which can also be virtualized.
  • Containers completely use the sandbox mechanism and will not have any interfaces with each other.
  • Docker is a large virtual machine environment, which contains many virtual machine images (images), and these images will contain configured environments; containers are based on these images, instantiated virtual machines, in a certain container The environment configuration can be modified at will without mutual interference between environments.
  • One of the benefits of Docker is that you can do whatever you want in the Container without worrying about crashing the Host environment.

2. Install docker

Docker has been installed on the server and there is no need to install it. For specific installation steps, please refer to:
https://blog.csdn.net/qq_41204464/article/details/97539265

3. docker image

Docker official documentation: https://docs.docker.com/
Docker image warehouse: https://hub.docker.com/search?type=image
Docker image warehouse domestic version: http://hub.daocloud.io/

  • docker basic operations
    Please refer to: https://blog.csdn.net/runningman0000/article/details/107289037
docker ps -a # View docker

docker images -a # View docker images

docker rm [docker id] # Delete docker

docker rmi [docker image id] # Delete docker image

4. Create a new docker image

1. Build through Dockerfile

For details, please refer to
https://blog.csdn.net/shiqiangdexin/article/details/52472195

2. Modify based on the open source image on DockerHub and build an image suitable for yourself.

Taking pytorch as an example, use docker search pytorch to search for public images on DockerHub (https://hub.docker.com/).
Find a torch1.7 docker (find whichever one you need) and pull it down
For example, we can find the torch1.7 develop version we want from the official docker tags of torch. Once found, run locally (on the server):

docker pull pytorch/pytorch:1.7.1-cuda11.0-cudnn8-devel

The above operation is to pull this docker to the local server first.

5. Create a new docker container

When using docker for the first time, you need to create a new container of your own. All subsequent training tasks can be completed in this container. The common instructions for creating a new container are:

1. Enter docker

docker run --runtime=nvidia -it -v DIR_LOCAL:DIR_CON -p PORT_LOCAL:PORT_CON --name=test --ipc=host --net=host pytorch/pytorch:1.7.1-cuda11.0-cudnn8 -devel

Among them, -it means using the interactive interface to create a new container, that is, working in /bash/bin;
-v means mapping the local path to the corresponding path in the container;
DIR_LOCAL represents the local path;
DIR_CON represents the path within the container;
-p means mapping the local port to the container port, which is used when using tensorboard in deep learning;
–name means to give your own container a name so that you can use NAME to recall it next time you use the container without using the ID.

2.Example

docker run -it --gpus all --shm-size 128G -p 1003:1003 -v /data1:/data1 -v /home/moxue:/home/moxue -d --name moxue_v1 -- b07463d2d541 bash

–gpus all: indicates using all cards of the server; –shm-size: shared memory size

3. Enter the container you created

docker exec -it moxue_v1 /bin/bash

After entering docker, you can configure your own environment. At this time, you can jump to
https://blog.csdn.net/weixin_41891632/article/details/134164079?spm=1001.2014.3001.5502 Use conda to create your own python virtual environment.

4. You can exit docker

exit

6. docker commit: Create a new image from the container.

This is created directly locally. If you want to commit to docker hub, you need to connect to docker hub over the wall.

docker commit [OPTIONS] CONTAINER [REPOSITORY[:TAG]]
Among them, OPTIONS is optional:
-a: submitted image author
-c: Use Dockerfile instructions to create the image
-m: Description text when submitting
-p: Pause the container when committing.

View and find the configured docker id

docker ps -a

Find the docker id you want to commit. I am using a container named moxue_v1, the id is 31405b5f1856, and then

docker commit -a "moxue" -m "this is pytorch1.7_python3.7" 31405b5f1cf2 moxue_v1:pytorch1.7

Entering the above command will create an image mirror locally with Tag pytorch1.7 and repository moxue_v1.
Use docker images to view:

docker images

Continue to step five to start creating containers and other operations.

7. Package the existing docker into tar and load the packaged docker image. 

docker ps -a View the id and name of the container image to be packaged

docker commit [OPTIONS] CONTAINER [REPOSITORY[:TAG]], commit the docker image into a new image
docker commit 31405b5f1cf2 moxue_test:cuda11.7_torch1.12

docker images -a View the newly generated image name and tag moxue_test:cuda11.7_torch1.12

docker save -o tar package name.tar Image name to be packaged: Image version to be packaged (executed in a certain path, the tar package will be saved in that path)
docker save -o moxue_test.tar moxue_test:cuda11.7_torch1.12

scp -r moxue_test.tar target_dir Transfer the generated tar package to the new machine using scp and other transfer commands.

The moxue_test:cuda11.7_torch1.12 image will be generated in the new machine. Use the load command to load it directly.
docker load [OPTIONS] tar package name.tar
        docker load < imageName.tar is used to load the image from standard input.
        docker load -i imageName.tar is used to load an image from a file.
docker load -i moxue_test.tar
At this time, a new image will be generated. Enter the fifth step above to start creating a new container.

8. Connect to your own docker in the server in vscode

1. Open the vscode remote resource manager and connect to the server

Install the SSH extension



The server’s IP appears in the lower left corner, indicating that the server has been connected.

For specific docker related information, please refer to:
https://zhuanlan.zhihu.com/p/83663496
https://blog.csdn.net/QMW19910301/article/details/88070159
https://www.cnblogs.com/kevingrace/p/9599988.html