[Orange Pie-OpenCV-Torch-dlib] TF damaged into RAW format solution and python environment configuration

Foreword

This article will introduce the detailed steps for software configuration and environment setup on the Orange Pi development board in order to run Python applications. This covers the following main topics:

  1. Get the required software: Download links for the Orange Pi operating system and the balenaEtcher tool are provided to ensure proper operating system installation.

  2. TF card erasure and partition reconstruction: Under the Windows operating system, detailed steps for erasing and optionally partitioning and formatting the TF card through the command line tool (Command Prompt), making it suitable for different needs, such as reburning the image or Custom storage equipment.

  3. Common disk management and repair methods: Introduces common operations using Diskpart and chkdsk tools to manage disk partitions, format and repair file systems to maintain disk health.

  4. Python environment configuration: including installing Python 3.8 on the Orange Pi, updating pip, replacing pip sources, and installing commonly used Python libraries to prepare the development environment.

2023-10-16

  • orangepi-lts3 recommends burning uabntu version starting with version 2. Currently, 2.2.2 is the smoothest and adapts to the new Torch environment.
  • Orangepi-5b version burns the system and just runs the Flaring. The verification part may be because the disk partition cannot be verified normally after burning.
  • In addition, the built-in cable screen may not display the system in your TF card. When the red light comes on again and the yellow light flashes, there is a high probability that the system has entered. You need to connect an external screen through the HDMI interface.

Article directory

  • Preface
    • Orange Pie related software download link
    • TF card erasure and partition reconstruction
      • Step 1: Insert TF card and card reader
      • Step 2: Open the command line tool and enter diskpart
      • Step 3: List available drives
      • Step 4: Select TF card drive and clear contents
      • Step 5: Create partition and format TF card (optional)
      • Step 6: Repair and format the TF card file system type to NTFS (optional)
        • method one
        • Method Two
      • Step 7: Repair the disk using chkdsk
      • Step 8: Burn again using Etcher
      • Checksum does not match for range expansion card
      • unknown error, write disk loss problem
        • 1. The TF card cannot be opened using diskpart format and cannot be formatted normally.
        • 2. Repair and format the TF card file system type to NTFS
        • 3. Use chkdsk to repair the disk
    • Diskpart common methods
    • Common methods of chkdsk
    • python environment configuration
      • system version
      • Start configuration
      • Run package configuration
      • torch
        • Resolution process
      • dlib
        • method one
        • Method Two
      • run your code
      • complete process
  • Summarize

Orange Pie related software download link

orange pie
balenaEtcher

TF card clearing and partition rebuilding

This section will introduce how to use the command line tool (Command Prompt) under the Windows operating system to clear its contents by inserting a TF card and executing a series of commands, and optionally partition and format it to make it resemble ordinary storage. The device works the same. This process is extremely useful, whether for reburning an image or customizing a storage device.

Step 1: Insert TF card and card reader

First, make sure the TF card is inserted into the card reader, and then connect the card reader to the computer. TF card is a common storage medium, usually used in embedded devices, cameras, smartphones, etc.

Step 2: Open the command line tool and enter diskpart

To open the Windows command line tool, you can search for “cmd” in the Start menu or press Windows + R keys, then type “cmd” and press Enter to open it. The command line tool is a powerful tool used to perform various system-level tasks.

Enter the following command at the command line to enter the diskpart tool:

diskpart.exe

Diskpart is a Windows command line utility that can be used to manage disk operations such as disk partitioning and formatting.

Step 3: List available drives

In diskpart, use the following command to list all disk drives on the computer and their numbers:

list disk

Each disk drive will have a unique number, usually represented numerically. In this step, you need to find the disk drive number associated with the TF card (usually the one that identifies its capacity).

Step 4: Select the TF card drive and clear the contents

Use the following command to select the disk drive where the TF card is located (replace x1 with the number of the drive where the TF card is located):

select disk x1

After selecting the TF card drive, use the following command to wipe any existing file system to completely erase the TF card contents:

clean

At this point, the TF card has been completely cleared and you can re-burn the image or partition and format it, depending on your needs.

Step 5: Create partition and format TF card (optional)

If you want to partition and format the TF card so that it works like an ordinary storage device, you can enter the commands in the following order:

create partition primary
select partition 1
format quick

This will create a primary partition and quickly format it, making the TF card ready to store data.

The above are the steps to use command line tools to clear the TF card, partition and format it under the Windows operating system. This process is very common in embedded system development, data backup, and storage device maintenance.

Note: Before performing any disk operations, please be careful to make sure you select the correct disk drive to avoid accidentally deleting important data. Depending on your needs, you can choose to only erase the TF card or create a partition and format the TF card at the same time.

Step 6: Repair and format the TF card file system type to NTFS (optional)

After following the above steps, you should be able to access the TF card normally, but you may still be unable to burn the system normally.

PhysicalDrive2: Checksum does not match for range[0,72351743]: "13908ca859e2ff5c" != "240694foa08e4cb8"

  • If you succeed, you can skip here. I also recorded the common methods of Diskpart and chkdsk later.
  • Otherwise you need to repeat steps 1-5 again, and then format the TF card file system type as NTFS

  • You may not be able to convert. I also encountered this problem. Maybe the card I bought is an expansion card.
  • The next steps are the solutions to the problems I encountered when burning the system with the new card I bought.
  • You can also try to convert the TF card file system using the following two formats
Method 1

Step 1. Press and hold “Windows + R” simultaneously to enter the Diskpart program

diskpart.exe

Step 2. Enter the following command:

list volume

select volume X ("X" is the RAW volume code)

format fs=ntfs quick

exit

you can also

format fs=fat32 quick
Method 2

(“x” is the RAW volume code)

convert x: /fs:ntfs

Step 7: Use chkdsk to repair the disk

Press the “Win + R” keys simultaneously to open the run window. Then, enter

chkdsk H:/F

H refers to the drive letter of the TF card in the machine, F refers to the repair parameters, and press the “Enter” key to confirm and wait for repair.

Step 8: Burn again using Etcher

Checksum does not match for range expansion card

If you get this error, there is a high probability that your card is completely damaged or you bought an expansion card. The expansion card can often complete the Flashing stage normally, but problems will occur during the system verification stage.

PhysicalDrive2: Checksum does not match for range[0,72351743]: "13908ca859e2ff5c" != "240694foa08e4cb8"


You can download MYdisktest to test your TF card. Here I found that the 16gTF card I bought is only 146MB.

unknown error, write disk loss problem

l-\PhysicalDrive2: UNKNOWN: unknown error, write

When I switch systems for burning, I often hear a prompt sound that the TF card is disconnected from the computer when Flashing reaches about 80% or even 20%. At this time, the program is interrupted and cannot continue to burn, and the error l-\PhysicalDrive2 is reported. : UNKNOWN: unknown error, write

Then it is recommended to repeat the following steps. At the end of each step, eject the TF card and insert it again.

1. The TF card cannot be opened using diskpart format and cannot be formatted normally
diskpart.exe

list disk
select disk x1
clean
create partition primary
select partition 1
format quick
2. Repair and format the TF card file system type to NTFS

3. Use chkdsk to repair the disk

Press the “Win + R” keys simultaneously to open the run window. Then, enter it (replace H with your drive letter)

chkdsk H:/F

It is recommended to try a few more times to safely eject and re-plug after each step. You can also try to restart the computer. When you have safely finished Flashing and entered the verification stage, the burning is almost complete.

Common methods of Diskpart

When you use Diskpart for disk management, you can perform various operations to manage the disks and partitions on your computer. Here are some detailed examples and instructions:

  1. List disk and partition information:

    • List all disk information:

      list disk

    • List all partition and volume information:

      list volume

  2. Select a disk or partition:

    • Select a disk (assume disk number is N):

      select disk N

    • Select a partition (assuming partition number is N):

      select volume N

  3. Clear disk or partition:

    • Clear all partitions and volumes on the disk, wipe data:

      clean

    • Clear data on a specific partition or volume:

      clean all

  4. Create partition:

    • Create primary partition:

      create partition primary

    • Create logical partition:

      create partition logical

  5. Format partition:

    • Quickly format a partition:

      format quick

    • Fully format the partition:

      format fs=ntfs # or other file system type

  6. Extended partition:

    • Extend an existing partition to take advantage of unallocated space:

      extend

  7. Device signature:

    • Set the disk’s device signature to make it easier to distinguish between different disks:

      uniqueid disk ID=NewID

  8. Offline and online disks:

    • Set the disk offline:

      offline disk

    • Set the disk online:

      online disk

  9. View disk and partition properties:

    • View the properties of a partition or volume:

      attributes volume

  10. Script Action:

    • Create a text file containing the Diskpart commands, such as commands.txt, and then execute the script:

      Diskpart /s commands.txt

These examples demonstrate how to use Diskpart for disk management operations. Proceed with caution and make sure you understand the impact of each command, especially when deleting or formatting a partition. Disk operations may result in data loss, so please back up important data before performing any important operations.

Common methods of chkdsk

chkdsk (Check Disk) is a command line tool in Windows operating system used to check and repair file system errors and hard disk problems on the hard drive. The following is an introduction to some commonly used chkdsk methods:

  1. Run basic chkdsk:

    chkdsk [drive name]:

    This will start chkdsk to scan the file system on the specified drive and report any errors found. Typically, you need to run this command with administrator privileges.

  2. Fix file system errors:

    chkdsk [drive name]: /f

    Running chkdsk with the /f parameter can repair detected file system errors. It will try to fix the problems found during the scan.

  3. Scan and repair bad sectors:

    chkdsk [drive name]: /r

    Adding the /r parameter will scan and attempt to repair bad sectors on any hard drive. This process may take some time, especially with large capacity drives.

  4. Force unmount the drive:

    chkdsk [drive name]: /x

    Use the /x parameter to force the specified drive to be unmounted, and then run chkdsk. This is typically used for system drives and requires a computer restart to complete the scan and repair process.

  5. Show scan progress:

    chkdsk [drive name]: /i

    Adding the /i parameter can display the scanning progress of chkdsk. This can be helpful to see how the scan is progressing.

  6. Save scan results to file:

    chkdsk [drive name] > file path.txt

    You can save the results of chkdsk to a text file to view the scan report later.

  7. Check system drive:

    chkdsk C: /f /r

    If you want to check the system drive (usually the C: drive), you can run chkdsk using the /f and /r parameters together to fix file system errors and bad sectors.

Please note that running chkdsk may take some time, especially on large hard drives or when repairing a large number of errors. Make sure to back up important data before running chkdsk to prevent accidental data loss. Also, it is recommended to close all running applications before performing disk maintenance operations.

python environment configuration

System version

The Orange Pi system used in this environment configuration is

Orangepi3-lts_3.0.8_ubuntu_jammy_desktop_xfce_linux5.16.17.7z

The ubantu system provided by Orange Pi is often divided into two versions. The desktop_xfce version is a desktop version. With the desktop version, you can turn off the desktop and get a Linux command line server version that is roughly the same as the server version.

sudo orangepi-config

Start configuration

This is a set of steps on installing Python 3.8 on a Linux system and changing pip sources. Here’s an explanation of each step:

1) Install the dependency packages required to compile Python
First, you need to update the system package list and install the dependency packages required to compile Python. These packages include build tools and various libraries for Python that support different features and modules of Python.

sudo apt-get update
sudo apt install -y build-essential zlib1g-dev \
libncurses5-dev libgdbm-dev libnss3-dev libssl-dev libsqlite3-dev \
libreadline-dev libffi-dev curl libbz2-dev

The above is the configuration method in the document. It appeared when I configured the Torch library.

ModuleNotFoundError: No module named '_lzma'

We need to install lzma dependencies first and then compile python after sudo apt-get update

apt-get install liblzma-dev -y

# centos system execution
yum install xz-devel -y
yum install python-backports-lzma -y

#ubuntu system execution
apt-get install liblzma-dev -y

2) Download the latest version of Python 3.8 source code and unzip it
Use wget to download the source code zip file of Python 3.8, and use the tar command to unzip it.

wget https://www.python.org/ftp/python/3.8.0/Python-3.8.0.tgz
tar xvf Python-3.8.0.tgz

3) Run configuration command
Enter the decompressed Python source code directory and run the configure command to prepare to compile Python. The –enable-optimizations flag is used to enable compile-time optimizations.

cd Python-3.8.0
./configure --enable-optimizations

4) Compile and install Python 3.8
Compile Python using the make command. This may take a while, around half an hour or so, depending on the performance of your system.

make -j4 # Use the -j flag to compile in parallel, the number 4 means using 4 CPU cores at the same time
sudo make altinstall # Use altinstall to prevent overwriting the system's default Python version

5) Check Python version
Once the installation is complete, you can verify the Python version using the following command:

python3.8 --version

6) Update pip
Update pip itself using the newly installed pip of Python 3.8:

/usr/local/bin/python3.8 -m pip install --upgrade pip

7) How to change the pip source
In regions such as mainland China, accessing the official Python source may be slow, so it is a good idea to replace the pip source to increase download speed.

How to permanently change the pip source:
a. Create the ~/.pip directory and create the pip.conf configuration file in it, and set the source to Tsinghua source.

mkdir -p ~/.pip
cat <<EOF > ~/.pip/pip.conf
[global]
timeout=6000
index-url = https://pypi.tuna.tsinghua.edu.cn/simple
trusted-host = pypi.tuna.tsinghua.edu.cn
EOF

b. Now, when using pip3 to install the Python library, it will be downloaded from the Tsinghua source and the speed will be faster.

How to temporarily change the pip source:
If you only want to change the pip source for a specific package, you can use the following command, replacing with the name of the package you want to install.

pip3 install <packagename> -i https://pypi.tuna.tsinghua.edu.cn/simple --trusted-host pypi.tuna.tsinghua.edu.cn

This will use the Tsinghua source to install specific packages on a temporary basis.

Run package configuration

The packaging method is based on
Semi-automated use of .bat to manually package and migrate python projects

  • Python 3.8
  • OpenCV
  • Pytorch
  • dlib
  • Gradio

Here we also implement a clean new environment for testing by creating a virtual environment

python3.8 -m venv venv
source venv/bin/activate
python3.8 -m pip install -i https://mirrors.aliyun.com/pypi/simple/ --upgrade pip setuptools
pip install -i https://mirrors.aliyun.com/pypi/simple/ scipy==1.7.1
pip install -i https://mirrors.aliyun.com/pypi/simple/ numpy==1.24.4
pip install -i https://mirrors.aliyun.com/pypi/simple/ matplotlib==3.4.3
pip install -i https://mirrors.aliyun.com/pypi/simple/ tqdm==4.62.2
pip install -i https://mirrors.aliyun.com/pypi/simple/ Pillow==10.0.2
pip install -i https://mirrors.aliyun.com/pypi/simple/ h5py==3.9.0
pip install -i https://mirrors.aliyun.com/pypi/simple/gradio
pip install -i https://mirrors.aliyun.com/pypi/simple/ opencv-python==4.5.3.56
pip install -i https://mirrors.aliyun.com/pypi/simple/ scikit-learn

torch

pip install torch==2.1.0 torchvision==0.16.0 torchaudio==0.9.0 -f https://download.pytorch.org/whl/torch_stable.html
Solution process

There is downloadable torch1.7.1 in Orange Pie, but I did not find the compatible torch1.7.1 + cu110
At the same time, torchvision0.8.2 + cu110 and torchaudio0.7.2 cannot be downloaded.

pip install torch==1.7.1 + cu110 torchvision==0.8.2 + cu110 torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html

I tried to upgrade the torch version to torch==1.9.0, and then adapted torchvision and torchaudio one by one. At this time, the versions match, but when running, the prompt is:

pip install torch==1.9.0 torchvision==0.10.0 torchaudio==0.9.0 -f https://download.pytorch.org/whl/torch_stable.html

Illegal instruction

Cause: CPU architecture mismatch: The program was compiled to run against a mismatched CPU architecture.
Compiler problem: The program is compiled using an incompatible compiler, or there is a problem with the compiler installation. Unsupported instruction used: The program is trying to execute an unsupported CPU instruction.
Solution: Check the CPU architecture: Before running the program, you can view the CPU architecture information by executing the uname -a command. If the program is targeting an incompatible CPU
Programs compiled for an architecture should be recompiled using a target platform that is compatible with that architecture.

This is the regular official explanation, but I don’t understand the CPU architecture version matching rules, even using:

uname -a

There is a high probability that I will not be able to solve the problem.

So at this time I asked pip to update torch, and it automatically found the adapted version.

python3.8 -m pip install --upgrade torch

The final torch version is as follows

pip install torch==2.1.0 torchvision==0.16.0 torchaudio==0.9.0

dlib

Method 1

Before installing face_recognition, you first need to install the dlib library. Since the dlib library is relatively slow to compile and install on the development board, we use the dlib whl file compiled on gitee and install it directly after downloading. The download address of dlib whl file is as follows:
python_whl
You can clone it directly to the desktop

git clone --depth=1 https://gitee.com/leeboby/python_whl

In the python_whl folder, you can see that there are multiple versions of dlib installation packages and different versions of dlib
The corresponding Linux system is as follows:
Ubuntu18.04 dlib-19.24.0-cp36-cp36m-linux_aarch64.whl
Ubuntu20.04 dlib-19.24.0-cp38-cp38-linux_aarch64.whl
Ubuntu22.04 dlib-19.24.0-cp310-cp310-linux_aarch64.whl
Debian10 dlib-19.24.0-cp37-cp37m-linux_aarch64.whl
Debian11 dlib-19.24.0-cp39-cp39-linux_aarch64.whl

We use

pip install dlib-19.24.0-cp38-cp38-linux_aarch64.whl
Method 2

Theoretically you can choose to
dlib official website source
Download directly to get, for example, dlib-19.19.0.tar.gz or other versions of dlib and then

pip install dlib-19.19.0.tar.gz

you can also

pip download dlib==19.19.0
pip install dlib-19.19.0.tar.gz

renew

python3.8 -m pip install --upgrade dlib

The installation can be successful, but the board performance is insufficient and it does take a long time to compile. It is recommended to use method one.
This method is based on
Comprehensive sweep: dlib Python API configuration solutions on Linux and Windows

Run your code

GUI.py can be replaced with any .py file you need to compile and run

python3.8 GUI.py

You can transfer the complete file to Windows and then package your code like this
Semi-automated use of .bat to manually package and migrate python projects

The above environment configuration is the process of my recall. If there is a red flag, please install the version in the prompt. I will format and configure it again after a while.
When you can’t solve a problem,
I recommend using the simplest pip install automatic matching version first.
Secondly use
python3.8 -m pip install --upgrade
To update the package to the latest version,
Finally, use pip to install the oldest version that can be installed, with pip uninstall and python3.8 -m pip install --upgrade
Automatically match versions and try to upgrade one by one

Complete process

sudo apt-get update
sudo apt-get install liblzma-dev -y
sudo apt install -y build-essential zlib1g-dev \
libncurses5-dev libgdbm-dev libnss3-dev libssl-dev libsqlite3-dev \
libreadline-dev libffi-dev curl libbz2-dev
wget https://www.python.org/ftp/python/3.8.0/Python-3.8.0.tgz
tar xvf Python-3.8.0.tgz
cdPython-3.8.0
./configure --enable-optimizations
make -j4 # Use the -j flag to compile in parallel, the number 4 means using 4 CPU cores at the same time
sudo make altinstall # Use altinstall to prevent overwriting the system's default Python version
python3.8 --version
/usr/local/bin/python3.8 -m pip install --upgrade pip
mkdir -p ~/.pip
cat <<EOF > ~/.pip/pip.conf
[global]
timeout=6000
index-url = https://pypi.tuna.tsinghua.edu.cn/simple
trusted-host = pypi.tuna.tsinghua.edu.cn
EOF
python3.8 -m venv venv
source venv/bin/activate
python3.8 -m pip install -i https://mirrors.aliyun.com/pypi/simple/ --upgrade pip setuptools
pip install -i https://mirrors.aliyun.com/pypi/simple/ scipy==1.7.1
pip install -i https://mirrors.aliyun.com/pypi/simple/ numpy==1.24.4
pip install -i https://mirrors.aliyun.com/pypi/simple/ matplotlib==3.4.3
pip install -i https://mirrors.aliyun.com/pypi/simple/ tqdm==4.62.2
pip install -i https://mirrors.aliyun.com/pypi/simple/ Pillow==10.0.2
pip install -i https://mirrors.aliyun.com/pypi/simple/ h5py==3.9.0
pip install -i https://mirrors.aliyun.com/pypi/simple/gradio
pip install -i https://mirrors.aliyun.com/pypi/simple/ opencv-python==4.5.3.56
pip install -i https://mirrors.aliyun.com/pypi/simple/ scikit-learn
pip install -i https://mirrors.aliyun.com/pypi/simple/ imutils
pip install torch==2.1.0 torchvision==0.16.0 torchaudio==0.9.0 -f https://download.pytorch.org/whl/torch_stable.html
git clone --depth=1 https://gitee.com/leeboby/python_whl
pip install python_whl/dlib-19.24.0-cp38-cp38-linux_aarch64.whl
python3.8 GUI.py

Summary

The purpose of software configuration and environment building on the Orange Pi is to enable it to run Python applications. First, we introduced the connections for downloading related software, including operating systems and burning tools. Then, we detailed the steps to clear and partition the TF card to create a clean storage device for the development board.

Additionally, we provide common methods for disk management and repairing file system problems using the Diskpart and chkdsk tools. Finally, we introduced the steps for Python environment configuration, including installing Python 3.8, updating pip, replacing pip sources, and installing various Python libraries to run Python applications on the Orange Pi. These steps will help users prepare a powerful development environment for software development and project implementation.