Server – configure and install Git LFS | BWM-NG | Tmux | BOS and other commands

Welcome to my CSDN: https://spike.blog.csdn.net/
Address of this article: https://blog.csdn.net/caroline_wendy/article/details/131302104

Tmux

Build the experimental Docker:

nvidia-docker run -it --name git-lfs-[your name] --net=host -p [port]:[port] -v [nfs path]:[nfs path] glm:nvidia-pytorch -1.11.0-cu116-py3

1. Configure Git LFS

Git LFS is a tool that extends Git to allow you to more easily manage large files such as audio, video, datasets, etc. The principle of Git LFS is to store large files on a remote server, and only keep a pointer to the file in the Git repository. This way, you avoid downloading unnecessarily large files when cloning or pulling repositories, saving time and space. Git LFS also supports version control, branching, merging and other operations on large files, just like ordinary Git files. To use Git LFS, you need to install and initialize it in your project, and then use the git lfs track command to specify which files or file types need to be managed by LFS. Afterwards, you can commit your changes with git add and git commit as usual, and Git LFS will automatically handle uploads and downloads of large files.

Excuting an order:

git lfs install

If the error is as follows, you need to install the git-lfs command:

git: 'lfs' is not a git command. See 'git --help'

Install git-lfs, it is recommended to execute in the Docker environment, with the highest operating authority, or add sudo:

curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.deb.sh | bash
apt-get install git-lfs

Excuting an order:

git-lfs install

# output log
Git LFS initialized.

When using the git lfs command, you can choose to download the project first, and then download the lfs format file. That is, modify the path and add .git at the end, for example:

git lfs install
git clone https://huggingface.co/lmsys/vicuna-13b-delta-v0.git # Add additional .git suffix

# Execute again
cd vicuna-13b-delta-v0
git lfs pull

However, the trial speed is relatively slow, and it is recommended to download manually within 10G.

2. Configuring BWM-NG & Tmux

The full name of bwm-ng is Bandwidth Monitor NG, and NG stands for Next Generation. bwm-ng is a command-line tool for monitoring network bandwidth, displaying traffic, speed, errors, and other information for each interface in real time. bwm-ng supports multiple output formats, such as plain text, HTML, CSV, etc., and can also redirect the output to a file or pipe. The usage of bwm-ng is very simple, just enter bwm-ng in the terminal to start. Different options can be used to adjust the style and content of the output, such as -u for specifying units, -o for specifying the output format, -I for specifying the interface to monitor, etc. bwm-ng is a lightweight and powerful network bandwidth monitoring tool.

To install bwm-ng, execute the following command, refer to: How To Install “bwm-ng” Package on Ubuntu:

apt-get update -y
apt-get install -y bwm-ng

You can use bwm-ng to view the download speed of the network.

Tmux is a terminal multiplexer that allows you to run multiple terminal sessions simultaneously in one terminal window. The main features of Tmux are:

  • Multiple panes can be split in the same window, each pane is an independent terminal session.
  • Multiple windows can be created in the same session, and each window can contain multiple panes, making it easy to switch between windows.
  • It is possible to run a tmux session in the background, even if the terminal window is closed, it will not affect the process in the tmux session. You can reconnect to your tmux session at any time, restoring your previous working state.
  • The same tmux session can be shared on different terminals or different machines to realize multi-person collaboration or remote control.
  • You can customize the appearance and behavior of tmux, adjusting various parameters through configuration files and command line options.

Install tmux, execute the following command, refer to: How to Use tmux for Remote & amp; Local Development

apt-get install tmux

The usage skills of Tmux are as follows:

  1. You need to enter prefix (prefix) in advance, and then press the command after letting go. The default prefix is Ctrl + B. For example, press Ctrl + B, let go, and then press % (Shift + 5), that is split-screen operation.
  2. By modifying ~/.tmux.conf, you can modify the prefix.
  3. After entering Tmux, display the current prefix by viewing tmux show-options -g | grep prefix.
  4. After entering Tmux, modify the configuration tmux source ~/.tmux.conf

The way to modify the prefix, for example, from Ctrl + A to Ctrl + B, add the command as follows:

set-option -g prefix C-a
unbind-key C-b
bind-key C-a send-prefix

The split screen operation of Tmux is as follows:

Split screen up and down: Ctrl + b and then press "
Split screen left and right: Ctrl + b and then press %
Switch screens: Ctrl + b then o
Close a terminal: Ctrl + b then x
Switch between up and down split screen and left and right split screen: Ctrl + b and then press the space bar

3. Configure BOS

BOS CMD is a command-line tool for Baidu Intelligent Cloud Object Storage BOS, which can easily manage and operate BOS services. BOS CMD supports multiple upload, download, synchronization, delete and other functions, as well as selection and conversion of multiple storage types. BOS CMD also supports the S3 interface protocol and is compatible with applications and SDKs developed based on S3. BOS CMD is easy to use, just enter the bcecmd command in the command line to access BOS services.

Download BOS CMD, select the mac version, and download the URL

[External link image transfer failed, the source site may have an anti-leeching mechanism, it is recommended to save the image and upload it directly (img-GA4PnVzj-1687229265751)(/Users/wang/Library/Application Support/typora-user-images/image-20230620101800615 .png)]

After decompression, it is a file bcecmd, put it into the bin folder. At the end of the .bashrc or .zshrc file, add the command:

export PATH=$PATH:$HOME/bin/
alias bos='bcecmd --conf-path /etc/bceconf/bos'

At the same time, download the /etc/bceconf/ folder from the server and put it locally, including 3 files:

bucket_endpoint_cache
credentials
multiupload_infos

In this way, files can be uploaded or downloaded, and local and server can be connected.

Bugfixes and References

sudo apt-get update couldn’t create temporary file

Cause: Error caused by deleting the temporary folder /tmp

Execute the following command to repair the /tmp folder, the command is as follows:

mkdir /tmp
ls -lad /tmp
chmod 1777 /tmp
mount | grep /tmp

Execute the update again apt-get , it will work normally:

apt-get update -y

Git: gnutls_handshake() failed: Error in the pull function

In fact, the main problem is caused by the network, and the server is abnormally connected to the external network.

add-apt-repository: command not found error

Excuting an order:

sudo apt update
sudo apt install software-properties-common
sudo apt update

You can use add-apt-repository

refer to

  • StackOverflow – git: ‘lfs’ is not a git command unclear

  • Zhihu – How to elegantly download the huggingface-transformers model

  • SuperUser – sudo apt-get update couldn’t create temporary file

  • CSDN – Modify the shortcut key of Ctrl + B prefix of Tmux

  • StackOverflow – Git: gnutls_handshake() failed: Error in the pull function

  • Steps to Resolve add-apt-repository: command not found error