Modality-invariant Visual Odometry for Embodied Vision code reproduction (using docker)

Code address

https://github.com/memmelma/VO-Transformer/tree/dev

Environment configuration

1. Pull the github library

git clone https://github.com/memmelma/VO-Transformer.git
cd VO-Transformer/

2. Create an environment
Create environment.yml

name: vot_nav
channels:
  - pytorch
  -conda-forge
dependencies:
  -python=3.7
  -cmake=3.14.0
  - numpy
  - numba
  -tqdm
  -tbb
  - joblib
  - h5py
  -pytorch=1.10.0
  -torchvision=0.8.0
  -cudatoolkit=11.0
  - pip
  -pip:
    -yacs
    - lz4
    -opencv-python
    -future
    - numba
    - numpy
    -tqdm
    -tbb
    - joblib
    - h5py
    -opencv-python
    -lz4
    -yacs
    -wandb
    -tensorboard==1.15
    -ifcfg
    - jupyter
    -gpustat
    -moviepy
    - imageio
    -einops

conda env create -f environment.yml
conda activate vot_nav

3. Install habitat-sim

conda install -c aihabitat -c conda-forge habitat-sim=0.1.7 headless

4. Install habitat-lab

git clone https://github.com/facebookresearch/habitat-lab.git -b v0.1.7
cd habitat-lab/
pip install -r requirements.txt
python setup.py develop --all

5.Install timm and other

cd ..
git clone https://github.com/rwightman/pytorch-image-models.git -b v0.6.12
cd pytorch-image-models
pip install -e .

pip install protobuf==3.20

6. Configure data sets and models
Directly refer to the original github library

Generate data

./generate_data.sh --act_type -1 --N_list 250000 25000 --name_list 'train' 'val'

A long process…

6. Run the training

Before training, change the batch_size of configs/rl/evaluation/example_rl.yaml. It is too big to run. Change N_GPU as well.

./start_vo.sh --config-yaml configs/vo/example_vo.yaml

7. Run the eval script

./start_rl.sh --run-type eval --config-yaml configs/rl/evaluation/example_rl.yaml

Docker learning

A brief review of docker

Build the image (vot_dk is the name of the built image, which is built in the form of dockerfile)

docker build -t vot_dk .

Create a container (the container created in this way can specify a name, mycontainer is to get a name for its own container)
Just run it once at the beginning. The image can be used multiple times. It is recommended to always use one container

docker run -it --name mycontainer image name /bin/bash

docker file mapping

docker run -v /host path:/container path -it --name mycontainer image name /bin/bash

A commonly used command to create a container (the code in the container here is related to the host)

docker run -v ./docker_data:/home/masteruser/code -it --name ql vot_dk /bin/bash

Start container

docker start -ai mycontainer

Delete container

docker rm mycontainer

View image

docker images

View the container. Without -a, only the running docker will be viewed.

docker pa -a

Using docker in Vscode

Use docker in VSCODE, the kind that can directly modify files

VS code + Docker container development workflow

After the connection is entered, enter bash first to enter the terminal.

(base) here is also only

conda init
source ~/.bashrc

Will appear later

If you want to compile files in vscode, you also need to assign permissions.
File and folder permissions

sudo chmod -R 777 /path/to/directory
sudo chmod 777 /home/masteruser/data/test.py

Docker visualization

Reference https://blog.csdn.net/qq_42731705/article/details/130798908

When creating the container add

--env="DISPLAY=$DISPLAY" --net=host --volume="$HOME/.Xauthority:/root/.Xauthority:rw" --env="QT_X11_NO_MITSHM=1" -v /tmp /.X11-unix:/tmp/.X11-unix:ro

For example

docker run --env="DISPLAY=$DISPLAY" --net=host --volume="$HOME/.Xauthority:/root/.Xauthority:rw" --env="QT_X11_NO_MITSHM=1" -v /tmp/.X11-unix:/tmp/.X11-unix:ro -it -v ./docker_data:/home/masteruser/code --name ql_vis vot_dk /bin/bash

Use GPU

--gpus all

Set shared memory

The shared memory of the docker generated by default is very small. You need to set the shared memory of docker in this way.

--shm-size=20g

View shared memory

df -h /dev/shm

Configuration using docker

1. Keep only this part of the original Dockerfile

FROM nvidia/cudagl:11.0-base-ubuntu18.04
LABEL maintainer "Memmel Marius <[email protected]>"

USER root

# fix https://github.com/NVIDIA/nvidia-docker/issues/1632
RUN rm /etc/apt/sources.list.d/cuda.list
RUN rm /etc/apt/sources.list.d/nvidia-ml.list

# Setup basic packages
RUN apt-get update & amp; & amp; \
    apt-get install -y --no-install-recommends \
    build-essential\
    git \
    curl \
    vim \
    locales \
    htop \
    ca-certificates\
    libjpeg-dev \
    libpng-dev \
    libglfw3-dev \
    libglm-dev \
    libx11-dev \
    libomp-dev \
    libegl1-mesa-dev \
    pkg-config \
    wget\
    zip \
    sudo\
    net-tools \
    cmake\
    vim\
    locales \
    wget \
    git\
    nano\
    screen\
    gcc \
    python3-dev\
    bzip2\
    libx11-6\
    libssl-dev\
    libffi-dev\
    parallel\
    tmux\
    g + + \
    unzip & amp; & amp;\
    sudo rm -rf /var/lib/apt/lists/*


ENV user masteruser
RUN useradd -m -d /home/${user} ${user} & amp; & amp; \
    chown -R ${user} /home/${user} & amp; & amp; \
    adduser ${user} sudo & amp; & amp; \
    echo '%sudo ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers

ENV HOME=/home/$user
USER masteruser
WORKDIR /home/masteruser

# Create conda environment
RUN curl -Lso ~/miniconda.sh https://repo.anaconda.com/miniconda/Miniconda3-py37_4.11.0-Linux-x86_64.sh \
  & amp; & amp; chmod + x ~/miniconda.sh \
  & amp; & amp; ~/miniconda.sh -b -p ~/miniconda \
  & amp; & amp; rm ~/miniconda.sh
ENV PATH=$HOME/miniconda/bin:$PATH
ENV CONDA_AUTO_UPDATE_CONDA=false

RUN $HOME/miniconda/bin/conda create -y --name py37 python=3.7 \
  & amp; & amp; $HOME/miniconda/bin/conda clean -ya
ENV CONDA_DEFAULT_ENV=py37
ENV CONDA_PREFIX=$HOME/miniconda/envs/$CONDA_DEFAULT_ENV
ENV PATH=$CONDA_PREFIX/bin:$PATH
RUN $HOME/miniconda/bin/conda clean -ya

2. Build the image using dockerfile
Run in the same folder as dockerfile

docker build -t vot_dk .

3. Create a container using an image
(20G dynamic memory is configured here for training. The GPU memory error is not the problem here)

docker run --shm-size=20g -v ./docker_data:/home/masteruser/code --gpus all -it --name ql vot_dk /bin/bash

4. Start the container

docker start -ai ql

5. Configure the agent in docker to facilitate subsequent downloads
By the way, switch the pip source

python -m pip config set global.index-url https://mirrors.aliyun.com/pypi/simple

6. Build install.sh

git clone https://github.com/facebookresearch/habitat-sim.git \
     & amp; & amp; cd habitat-sim \
     & amp; & amp; git checkout 020041d75eaf3c70378a9ed0774b5c67b9d3ce99 \
     & amp; & amp; pip install -r requirements.txt \
     & amp; & python setup.py install --headless \
     & amp; & amp; cd ..

git clone https://github.com/facebookresearch/habitat-lab.git \
     & amp; & amp; cd habitat-lab \
     & amp; & amp; git checkout d0db1b55be57abbacc5563dca2ca14654c545552 \
     & amp; & amp; pip install -e . \
     & amp; & amp; cd ..

#Install timm
git clone https://github.com/rwightman/pytorch-image-models.git -b v0.6.12\
     & amp; & amp; cd pytorch-image-models \
     & amp; & amp; pip install -e .

#Install pytorch
conda install pytorch==1.7.0 torchvision==0.8.0 torchaudio==0.7.0 cudatoolkit=11.0 -c pytorch

#Install other stuff
pip install future \
    numba \
    numpy\
    tqdm\
    tbb\
    joblib\
    h5py\
    opencv-python\
    lz4\
    yacs\
    wandb\
    tensorboard==1.15 \
    ifcfg\
    jupyter\
    gpustat\
    moviepy \
    imageio\
    einops \
    protobuf==3.20


export GLOG_minloglevel=2
export MAGNUM_LOG="quiet"
export HOROVOD_GLOO_IFACE=em2

7. Run the installation script and pay attention to the running conda environment.

bash install.sh

8. Subsequent operations will be consistent with the environment configuration.