[Moore Thread] Magic Pen Ma Liang U disk Ubuntu system deployment

MobiMaliang Deploy on Ubuntu20.04
[Moore Thread] Magic Pen Ma Liang U disk Ubuntu system deployment

PDF document and automatic installation script link, Lanzouyun: https://wwft.lanzoul.com/b052nz4rc Password: 95s7
The script needs to change the txt suffix to sh. and add execution permissions. sudo chmod + x ./UbuntuDeploy.sh

0. Preset

0. Please first apply for a developer account and password [required] from the official website of Mormaliang
This account will be used to download necessary resources in the future.
Please send an email to [email protected] and provide basic information such as name, mobile phone number, industry, etc. to apply for a developer account, which will be used to pull the necessary information to run torch_musa mirror
After the application is successful, please edit line 18 of install.sh and replace Account and Password

1. It is recommended to use the English system: it can avoid a lot of unnecessary trouble!!!
After Magic Pen Ma Liang can run successfully, go to the system and change the language back to Chinese, which can reduce many unnecessary errors in the early stage.**

2. Linux(Ubuntu) to go U disk system will not affect the existing system
If you only need Linux (Ubuntu), plug in the USB flash drive, and then select the USB flash drive to start when booting.

  • Check current language:
    Open Ubuntu’s Settings, and then go to the fifth from the bottom Region & amp; & amp;Language to check whether the current language is English. If not, change it to English (United States). Then restart Ubuntu.

1. To install Ubuntu 20.04, you can refer to these two videos:

  1. [[Detailed Tutorial] Use ventoy to create a super 8-in-1 system to boot the USB disk and insert WinTOGO, PE, LinuxTOGO#1 at the same time] [Accurate airdrop to 09:02]
  2. [[Detailed tutorial] Use ventoy to make linuxTOGO all-in-one portable U disk to install complete ubuntu (without persistence) – 8-in-1 U disk #2]
    For the above two videos, you can ignore the first half of the Win to go part and start watching it directly from 09:02.
    The above reprinted link has been approved by the original author.

2. Plug in the prepared Linux (Ubuntu) to go – USB flash drive to the computer.

Boot option menu selection keys for some computers:

Assembly machine Notebook Brand desktop machine
Motherboard brand Start button Laptop brand Start button Desktop brand Start button
ASUS motherboard F8 Lenovo Notebook F12 Lenovo Desktop F12
GIGABYTE Motherboard F12 Acer notebook F12 HP desktop F12
MSI motherboard F11 ASUS notebook ESC Acer desktop F12
Biostar motherboard F9 HP notebook F9 Dell desktop ESC
Meijie motherboard ESC or F12 Lenovo Thinkpad F12 Shenzhou Desktop F12
Colorful Motherboard ESC or F11 Dell Notebook F12 ASUS Desktop F8
ASRock Motherboard F11 Shenzhou Notebook F12 Founder Desktop F12
Spartak motherboard ESC Toshiba notebook F12 Tsinghua Tongfang desktop F12
Onda motherboard F11 Samsung notebook F12 Haier Desktop F12
Dual Sensitive Motherboard ESC IBM Notebook F12 BenQ Desktop F8
ASL Motherboard F10 Fujitsu Notebook F12 ? ?
Elite Motherboard ESC or F11 Haier notebook F12 ? ?
Guanmeng motherboard F11 or F12 Founder notebook F12 ? ?
Foxconn motherboard ESC or F12 Tsinghua Tongfang notebook F12 ? ?
Topstar motherboard F11 or F12 MSI notebook F11 ? ?
Maxsuan motherboard ESC or F11 BenQ notebook F9 ? ?
Yeston motherboard F8 GIGABYTE Notebook F12 ? ?
Jeway Motherboard ESC Gateway Notebook F12 ? ?
Intel motherboard F12 eMachines notebook F12 ? ?
Jiewei motherboard ESC or F8 Sony notebook ESC ? ?
Zhiming motherboard F12 Apple notebook Press and hold the “option” key ? ?
Panying motherboard ESC ? ? ? ?
Panzheng motherboard ESC ? ? ? ?
Guanming motherboard F9 ? ? ? ?
Note: For computer models not mentioned above, please try or refer to the common startup hotkeys of the same brand
Content referenceNext I tell you

3. Install the Magic Pen Ma Liang driver on the host Ubuntu

After successfully entering the Ubuntu system

  • Explanation: Host refers to the Ubuntu bare metal you are using now.
  • Essential knowledge you need to know:
    There are two ways to open Terminal:
    1. Right-click on the desktop or folder and select Open in Terminal to open the Terminal
    2. Press the mouse Ctrl + Alt + T, where T is the abbreviation of Terminal.
1. Automatic installation script

After opening the Terminal anywhere and in any way, enter the following content.

sudo apt install git # Install git
git --version # Check whether git is installed successfully

# Download and run the UbuntuDeploy.sh script
sudo wget https://gitee.com/Credhat/MobiMaliangSDK/raw/main/UbuntuDeploy.sh -O - | sudo bash
2. Manual step-by-step installation:
1. Install Git.

After opening the Terminal anywhere and in any way, enter the following contents in sequence. If asked to enter password, enter your account password.

sudo apt install git #Install git
git --version #Check whether git is installed successfully

git

If there is no error in the middle, and the second line of git --version outputs something similar to the above, it means the installation was successful.

2. Create mtai_workspace folder
mkdir -pv $HOME/Desktop/mtai_workspace # Create mtai_workspace folder
cd $HOME/Desktop/mtai_workspace # Enter the mtai_workspace folder

sdk

The two green lines above appear to indicate success.
need to wait a while

3. Download the Magic Pen Ma Liang project file in the mtai_workspace folder

Continue to enter the following content line by line in the Terminal:

git clone https://github.com/MooreThreads/MobiMaliangSDK #Download project
cd MobiMaliangSDK # Enter the downloaded MobiMaliang project folder

4. Install the GPU driver. If you encounter an error, [try multiple times]
sudo bash install_driver.sh

If the UI of Configuring lightdm appears, please select lightdm as the Default display manager. Only when the selection process of lightdm appears during the installation process, the Moore thread will be considered. Complete installation of GPU driver.

If it does not appear, then [try multiple times]. It will not be considered successful until it appears and the following content is finally entered.
lightdm

And output the following content:
installdriver

Once it appears, restart Ubuntu.

reboot

After the restart is complete, verify whether the driver (GPU Driver) is successfully installed.
Open Terminal and enter the following:

mthreads-gmi

mthreads-gmi

If the input content is similar, it will be successful.

5. Install Docker

Open Terminal and enter the following:

sudo apt install docker.io # Install Docker
sudo docker ps # Check whether Docker is installed successfully

dockerinstallation

6. Pull the official image of Magic Pen Ma Liang

Important: The developer account we originally applied for from the official website must now be used.
Please edit line 18 of install.sh and replace Account and Password
usrpasswd

Open Terminal and enter the following:

cd $HOME/Desktop/mtai_workspace/MobiMaliangSDK
sudo bash install.sh

install

7. Run the image

Execute the following command in the terminal

sudo docker run -id --name mtai_workspace --privileged -e MTHREADS_VISIBLE_DEVICES=all -p 1001:1001 -p 1002:1002 -p 1003:1003 -p 1004:1004 -p 1005:1005 -v ~/Desktop /mtai_workspace:/mtai_workspace:rw --shm-size 64G registry.mthreads.com/mcconline/musa-pytorch-release-public:v1.0.0

sudo docker exec -it mtai_workspace /bin/bash

intodocker
Then continue typing

cd /mtai_workspace/MobiMaliangSDK/
bash install_inside_docker.sh

installinside

We continue to enter the following content. If there is an error?, just ignore it.

apt-get update & amp; & amp; apt-get install ffmpeg libsm6 libxext6 -y

pip install -r requirements.txt --no-deps

mkdir -p ~/.cache/huggingface/hub/;rm -rf ~/.cache/huggingface/hub/models--openai--clip-vit-large-patch14/;cp -r models/tools/models-- openai--clip-vit-large-patch14/ ~/.cache/huggingface/hub/
8. Open Magic Pen Ma Liang WEB UI

Continue execution in the terminal:

cd /mtai_workspace/MobiMaliangSDK/
streamlit run frontend/main.py --server.port 1001

link

The web page link in the picture appears. Use Ctrl + mouse click to open it. Or enter the browser address, localhost:1001 to access the WebUI

If you want to stop the WebUI service, return to the Terminal and press Ctrl + C;
After stopping the service, if you want to exit the container container, enter exit

9.Happy drawing

After clicking on the WEB UI, an error will be reported because the official model does not come with a model.
model wrong

We need to download the model ourselves and put it in this directory

$HOME/Desktop/mtai_workspace/MobiMaliangSDK/models/Stable-diffusion/

After placing the model in the above folder, it is recommended to restart the container
Return to the Terminal, press Ctrl + C, then enter the following content to restart the WEB UI.

streamlit run frontend/main.py --server.port 1001

success
After waiting for the running to end, click on the random description and click on create to use it.
running

Issues

0. Where can I download the model?
———-Model download———-
1. Stable Diffusion: Stable Diffusion: Both ckpt and safetensors are available. They can be downloaded from tusiart.com, civitai.com and other websites, such as GhostMix2.0. After downloading, place it in models/Stable-diffusion/
2. Controlnet:
canny: Implement ControlNet edge control generation, download it and put it in models/controlnet/canny_v11/
openpose: implement ControlNet skeleton control generation, download it and put it in models/controlnet/openpose_v11/
tile: implement the ControlNet Tile fine drawing function, download it and put it in models/controlnet/tile_v11/
mlsd: Implement ControlNet edge control generation, download it and put it in models/controlnet/mlsd_v11/
Skeleton detection models: body_pose_model.pth, facenet.pth, hand_pose_model.pth, which realize the detection of the whole body, face and hands respectively, download and put them in models/controlnet/annotators/
Edge detection model: mlsd_large_512_fp32.pth, download it and put it in models/controlnet/annotators/
3. LoRA: safetensors format, which can be downloaded from tusiart.com, civitai.com and other websites. After downloading, place it in models/lora/
4. Textural Inversion: pt format, can be downloaded from tusiart.com, civitai.com and other websites. After downloading, place it in models/embeddings/
5. Super score: RealESRGAN_x4plus, more download links: https://huggingface.co/schwgHao/RealESRGAN_x4plus/blob/main/RealESRGAN_x4plus.pth, download and put to models/tools/
6. Picture text: DeepDanbooru, more download links: https://huggingface.co/Renqf/model-resnet_custom_v3.pt/blob/main/model-resnet_custom_v3.pt, download and put it in models/tools/
7. Translation: Supports Chinese prompt and negative prompt, please download https://huggingface.co/Helsinki-NLP/opus-mt-zh-en/blob/main/pytorch_model.bin and put it in models/ tools/zh2en/

Please read and understand the following matters carefully before using the recommended models:

  1. The recommended models are third-party software or code libraries. When customers download and use the recommended models, they need to understand and comply with the user agreement or open source agreement requirements of the model.
  2. If you plan to use a recommended model, please ensure that your use complies with the requirements of the appropriate user agreement or open source license. Some open source agreements may have specific restrictions on commercial use or require you to disclose source code, etc. You are solely responsible for understanding and complying with these regulations
  3. The Company makes no warranties or commitments, express or implied, regarding the recommended models. You use these models at your own risk and responsibility. The Company is not responsible for any direct or indirect loss or damage arising from the use of these models

1. Why can’t the computer start after restarting after installing the driver?
– For example, the current interface appears:
fail boot

- Try pressing Ctrl + Alt + F2 on the stuck interface, enter the Ubuntu account password, log in to the Terminal, and then restore the file to unmodified
Enter the following command
1. `sudo mv /boot/initrd.img-5.4.0-166-generic /boot/initrd.img-5.4.0-166-generic.bkp.new`
2. `sudo mv /boot/initrd.img-5.4.0-166-generic.old-dkms /boot/initrd.img-5.4.0-166-generic`
3. `reboot`
 ![fixboot](https://img-blog.csdnimg.cn/301e17f33697409b81fb42c85e1b4a75.png#pic_center)

2. Why didn’t I appear in the process of selecting Lightdm?
– It is recommended to run it multiple times [multiple attempts] (###### 4. Install the GPU driver. If you encounter an error, re-enter this command and wait for the installation to complete). Generally, the lightdm selection will appear after three to four times. process.
– Then the Moore thread GPU driver is successfully installed.

3. Why did my Ubuntu to go storage disappear so quickly?
1. This problem may be caused by the system log file being too large. Use the command df -h to check it.
2. If you find that /var/log/syslog and /var/log/kern.log take up a lot of storage, you can continue to check what is causing it.
3. tail -f /var/log/syslog # Check the cause of the syslog error
4. tail -f /var/log/kern.log # Check the error cause of kern.log
pcie
5. If it is PCIe Bus Error, you can refer to this post # Solve the PCIE Bus Error problem under Ubuntu

4. Why does Magic Pen Ma Liang occasionally report an error ml is not defined
1. Check whether the model contains Chinese characters. Please ensure that the model name does not contain Chinese characters or spaces.
2. Then restart the WEB UI service, refer to [Step 9] (##### 9. Happy drawing).
mlnotdefined

5. Error MUSA out of memory.
1. This error is because too much video memory is occupied and there is no way to allocate more video memory for drawing.
2. Solution: Adjust the image resolution to the minimum, such as 512 * 512 to 384 * 384. Then run a random image. In this way, the GPU memory usage is actively refreshed. This reduces the memory usage.
out of memory

Finally:

UbuntuDeploy.sh file contents

#!/bin/bash

echo "--------- Start trying to install the Magic Pen Ma Liang environment ------------"

mbmlpath=$HOME/Desktop/mtai_workspace/MobiMaliangSDK

#Create mtai_workspace folder

mkdir -pv $mbmlpath # Create mtai_workspace folder

echo "-------------Download project file ------------------"
echo "-------------Download project file ------------------"
echo "-------------Download project file ------------------"

max_attempts=3
attempts=0

while [ $attempts -lt $max_attempts ]; do
    # Delete existing directory
    rm -rf "$mbmlpath"
    # Try the first git clone
    git clone https://github.com/MooreThreads/MobiMaliangSDK "$mbmlpath"
    
    # Check whether the first git clone is successful
    if [ $? -eq 0 ]; then
        echo "The first git clone was successful."
        break
    else
        echo "The first git clone failed, try the second git clone..."
        # Try a second git clone
        git clone https://gitee.com/Credhat/MobiMaliangSDK.git "$mbmlpath" # Download the project and replace gitee, which is more suitable for China
        
        # Check whether the second git clone is successful
        if [ $? -eq 0 ]; then
            echo "The second git clone was successful."
            break
        else
            echo "The second git clone also failed."
            ((attempts + + ))
            sleep 5 # Wait for a while and try again. You can adjust the waiting time as needed.
        fi
    fi
done

  

if [ $attempts -eq $max_attempts ]; then
    echo "The maximum number of attempts has been reached without success. Please check the network or download the project manually."
    exit 1
fi

cd $mbmlpath # Enter the downloaded Magic Pen Ma Liang project folder


echo "-------------Install GPU driver------------------"
echo "-------------Install GPU driver------------------"
echo "-------------Install GPU driver------------------"

max_attempts=3
attempts=0

while [ $attempts -lt $max_attempts ]; do
    #Install GPU driver
    sudo bash install_driver.sh -y

    # Run mthreads-gmi
    mthreads-gmi

    if [ $? -eq 0 ]; then
        echo "GPU driver installation and check successful."
        break
    else
        echo "The GPU device has not been found, re-execute the installation..."
        ((attempts + + ))
        sleep 5 # Wait for a while and try again. You can adjust the waiting time as needed.
    fi
done

if [ $attempts -eq $max_attempts ]; then
    echo "The maximum number of attempts has been reached and still failed. Please check and process manually."
fi

# Install Docker
sudo apt install docker.io # Install Docker
sudo docker ps # Check whether Docker is installed successfully
  

max_attempts=3
attempts=0
  
while [ $attempts -lt $max_attempts ]; do
    wget "https://gitee.com/Credhat/MobiMaliangSDK/raw/main/UbuntuDeploy2.sh" -O UbuntuDeploy2.sh

    # Check the status returned by the wget command
    if [ $? -eq 0 ]; then
        echo "Download successful, UbuntuDeploy2.sh has been written."
        break
    else
        echo "Download failed, try to download again..."
        ((attempts + + ))
        sleep 5 # Wait for a while and try again. You can adjust the waiting time as needed.
    fi
done

  
if [ $attempts -eq $max_attempts ]; then
    echo "The maximum number of attempts has been reached and the download has not been successful. Please check the network connection."
fi

sudo chmod +x ./UbuntuDeploy2.sh

echo "------------------------------------------------"
echo "------------------------------------------------"
echo The first phase of automatic installation is completed.
echo The first phase of automatic installation is completed.
echo "------------------------------------------------"
echo requires you to manually replace the account and password in the install.sh file with your developer account password
echo requires you to manually replace the account and password in the install.sh file with your developer account password
echo "------------------------------------------------"
echo and execute ./UbuntuDeploy2.sh again in the terminal
echo and execute ./UbuntuDeploy2.sh again in the terminal

UbuntuDeploy2.sh file contents

#!/bin/bash

echo "---------- If it doesn't work, it is recommended to start the manual installation from step 6 of this tutorial ----------"
echo "---------- If it doesn't work, it is recommended to start the manual installation from step 6 of this tutorial ----------"
echo "---------- If it doesn't work, it is recommended to start the manual installation from step 6 of this tutorial ----------"

echo "-------- Start trying to install Magic Pen Ma Liang Environment 2 -----------"
mbmlpath=$HOME/Desktop/mtai_workspace/MobiMaliangSDK
workpath=$HOME/Desktop/mtai_workspace
# Pull the official image of Magic Pen Ma Liang
cd $mbmlpath
sudo bash install.sh

# Check return status
if [ $? -eq 0 ]; then
    echo "Magic Pen Ma Liang official image was successfully pulled."
else
    echo "An error occurred while pulling the image, please check the installation script and network connection."
    echo "Please check whether you have replaced the account and password in the install.sh file with your developer account password"
    echo "For example: docker login -u 'Abc' -p 'password' registry.mthreads.com"
    break
fi

# Start the magic pen Ma Liang image
sudo docker run -id --name mtai_workspace --privileged -e MTHREADS_VISIBLE_DEVICES=all -p 1001:1001 -p 1002:1002 -p 1003:1003 -p 1004:1004 -p 1005:1005 -v $workpath:/mtai_workspace: rw --shm-size 64G registry.mthreads.com/mcconline/musa-pytorch-release-public:v1.0.0
# Enter the magic pen Ma Liang image

# Execute multiple commands in the container
docker exec -it mtai_workspace /bin/bash -c "\
    cd /mtai_workspace/MobiMaliangSDK/ & amp; & amp; \
    bash install_inside_docker.sh
"

docker exec -it mtai_workspace /bin/bash -c "\
    apt-get update & amp; & amp; apt-get install ffmpeg libsm6 libxext6 -y & amp; & amp; \
    pip install -r requirements.txt --no-deps & amp; & amp; \
    mkdir -p ~/.cache/huggingface/hub/ & amp; & amp; \
    rm -rf ~/.cache/huggingface/hub/models--openai--clip-vit-large-patch14/ & amp; & amp; \
    cp -r models/tools/models--openai--clip-vit-large-patch14/ ~/.cache/huggingface/hub/
"

docker exec -it mtai_workspace /bin/bash -c "
    cd /mtai_workspace/MobiMaliangSDK/ & amp; & amp; \
    streamlit run frontend/main.py --server.port 1001
"

echo "---------If it doesn't work, it is recommended to start the manual installation from step 6 of this tutorial ----------"
echo "---------If it doesn't work, it is recommended to start the manual installation from step 6 of this tutorial ----------"
echo "---------If it doesn't work, it is recommended to start the manual installation from step 6 of this tutorial ----------"