Yolo-Fastest with environment + training + testing

Yolo-Fastest configuration environment + training + testing

• Table of contents
• • 1. Matching environment
  • 2. Training
  • 3. Test
• 1. Matching environment
• • • opencv451
    • vs2019
    • cmake3.28.0
• 2. Training
• • 3. Test

Directory

1. Environment configuration

2. Training

3. Test

Reference for the entire process:
YOLO-Fastest trains its own data set:
The whole process of training Yolo-Fastest model on Win10 platform
YOLO-Fastest compilation and testing

1. Environment configuration

[Required configuration version (without CUDA version)]
1.win10
2.opencv451
3.vs2019
4.cmake3.28.0
[Required configuration version (CUDA version available)]
1.win10
2.opencv451
3.vs2019
4.cmake3.28.0
5.CUDA 10.1 updates 2 and above

illustrate:

• Other methods use vs2015, cuda10.2, etc., which do not affect yolo-fastest.

• I used vs2019 because when cmake was compiled, there was only the 2019 version of Windows.
  The SDK version can be used, but the 2015 and 2017 versions are lower than the cmake compiled version and cannot be applied.

• I tried to download the SDK manually, but I didn’t know how to introduce it in VS2015 and 2017, so I gave up. Friends who know how to install it themselves can also install it manually. The official website for manual SDK download is: https://developer.microsoft.com/zh-cn/windows/downloads/sdk-archive/

• (1 is the SDK version included with VS download, 2 is the minimum version required for cmake compilation, your own SDK must exceed this minimum version)
  

• The reason why there is no CUDA: cmake compilation can never find CUDA, even if it is already in the path. So the choice has to be deprecated so that the code can run normally.

• Later, there was another reason for CUDA: an error was reported after forcibly adding the CUDA path. The final analysis found that the previous CUDA10.1 version was low, and it must be 10.1 updates2 and above. Also, installing VS first and then CUDA will also eliminate the need to configure the CUDA environment in cmake (I reinstalled CUDA with VS)

opencv451

opnecv download 4.5.1
The following picture shows, select the opencv version you want to install. It is not yet clear whether other higher versions are suitable, so I will use 4.5.1 as a reference based on other people’s procedures.

vs2019

Official website: https://learn.microsoft.com/zh-cn/visualstudio/releases/2019/release-notes#release-notes-icon-visual-studio-2019-version-161125
Download reference: https://blog.csdn.net/qiaodahua/article/details/130194176

cmake3.28.0

Download official website: https://cmake.org/download/
It is best to install on a drive other than C.

After installation, check whether the installation is complete: press win + R and enter cmd to enter the terminal, enter
Install other older versions of cmake: https://cmake.org/files/
Note: I tried installing version 3.18.0, but cmake reported an error saying that version 3.28 is required. So try to use the latest version.

cmake -V

Then open cmake, select the YOLO-fastest folder as the target folder, click configure, cancel CUDA related, and add the opnecv path:
(When choosing a version, you must be careful about the year. It matches the year of the VS you downloaded. Look at the last number, not the previous 17, 16, and 15!! I mistakenly read the previous part before, causing it to keep compiling. Failed, and finally found out that it was this stupid mistake, damn it!)

(After downloading opnecv451.zip, you can choose a folder to unzip it. You can name it whatever you want. The opencv451 in the picture below is the folder name I named myself)

No CUDA version: Uncheck the 3 selected boxes.

There is a CUDA version: If the path is correct, no error will be reported and the compilation will be completed directly. If an error is reported, follow the error reporting requirements to find and modify it yourself.
Click configure again and the compilation is completed.

[Reference here: YOLO-Fastest complete graphic tutorial from Darknet source code compilation, testing to training]

After configure is done, press generate again:

————————————Dividing line———— ———————–
If you find that you have selected the wrong VS version after compiling, or you want to recompile everything
Just click delete cache.

————————————-Dividing line———– —————————————-
After generate is successful, the following files will be generated:
(Picture reference: https://blog.csdn.net/OrangeOne1/article/details/109187641)

Open with VS2019. (Picture reference: https://blog.csdn.net/OrangeOne1/article/details/109187641)

After opening:
Change the circled area to release and x64

Then select the project on the side, right-click, and generate the solution:
Note: If it is a version without CUDA, it is 7 projects, if it is a version with CUDA, it is 6 projects. (This is my personal situation, I’m not sure if it’s correct, but the result is the same, it must be success.)
Without CUDA:

If you get the following picture, you are successful:

With CUDA:

2. Training

Reference: https://blog.csdn.net/weixin_41868104/article/details/115795348

To train your own data set, first use LabelImage software to obtain the XML file in VOC format, which I will not go into here.
Subsequently, the labels in xml format are converted into yolo format.

1) Create a new mydata folder, continue to create a data folder under this file, and continue to create Annotations, images, ImageSets (create a Main folder inside), and labels under the data file. As shown in the picture:

2) Create the makedata.py file in this directory. Generate train.txt and test.txt

import os
import random
 
trainval_percent = 0.2 #Can be adjusted by yourself
train_percent = 1
xmlfilepath = 'Annotations'
txtsavepath = 'ImageSets\Main'
total_xml = os.listdir(xmlfilepath)
 
num = len(total_xml)
list = range(num)
tv = int(num * trainval_percent)
tr = int(tv * train_percent)
trainval = random.sample(list, tv)
train = random.sample(trainval, tr)
 
#ftrainval = open('ImageSets/Main/trainval.txt', 'w')
ftest = open('ImageSets/Main/test.txt', 'w')
ftrain = open('ImageSets/Main/train.txt', 'w')
#fval = open('ImageSets/Main/val.txt', 'w')
 
for i in list:
    name = total_xml[i][:-4] + '\\
'
    if i in trainval:
        #ftrainval.write(name)
        if i in train:
            ftest.write(name)
        #else:
            #fval.write(name)
    else:
        ftrain.write(name)
 
#ftrainval.close()
ftrain.close()
#fval.close()
ftest.close()

3) Create a new file voc_label.py in the mydata folder

import xml.etree.ElementTree as ET
import pickle
import os
from os import listdir, getcwd
from os.path import join
 
sets = ['train', 'test']
 
classes = ['human'] #Self-trained categories such as ship, cat, etc.
 
 
def convert(size, box):
    dw = 1./size[0]
    dh = 1. / size[1]
    x = (box[0] + box[1]) / 2.0
    y = (box[2] + box[3]) / 2.0
    w = box[1] - box[0]
    h = box[3] - box[2]
    x = round(x * dw, 3) # In order to make the normalized number accurate to the last three digits, if it is not accurate, an error may be reported.
    w = round(w * dw, 3)
    y = round(y * dh, 3)
    h = round(h * dh, 3)
    return (x, y, w, h)
 
 
def convert_annotation(image_id):
    in_file = open('data/Annotations/%s.xml' % (image_id))
    out_file = open('data/labels/%s.txt' % (image_id), 'w')
    tree = ET.parse(in_file)
    root = tree.getroot()
    size = root.find('size')
    w = int(size.find('width').text)
    h = int(size.find('height').text)
 
    for obj in root.iter('object'):
        difficult = obj.find('difficult').text
        cls = obj.find('name').text
        if cls not in classes or int(difficult) == 1:
            continue
        cls_id = classes.index(cls)
        xmlbox = obj.find('bndbox')
        b = (float(xmlbox.find('xmin').text), float(xmlbox.find('xmax').text), float(xmlbox.find('ymin').text) ,
             float(xmlbox.find('ymax').text))
        bb = convert((w, h), b)
        out_file.write(str(cls_id) + " " + " ".join([str(a) for a in bb]) + '\\
')
 
 
wd = getcwd()
for image_set in sets:
    if not os.path.exists('data/labels/'):
        os.makedirs('data/labels/')
    image_ids = open('data/ImageSets/Main/%s.txt' % (image_set)).read().strip().split()
    list_file = open('data/%s.txt' % (image_set), 'w')
    for image_id in image_ids:
        list_file.write('data/images/%s.jpg\\
' % (image_id))
        convert_annotation(image_id)
    list_file.close()

Note:

• Modify your training category. Mine only has one category, human.
• In the third to last line, the image format needs to correspond. If it is not in .jpg format, remember to change it to your own image format.

Run the above two scripts in sequence to obtain the data format that meets yolo.

4) When I ran the code later, I found that there was a problem with the label path and the txt file could not be found, so I changed it to an absolute path:

'''Read each line of txt and add a sentence ''' in front of each line
trainfiletxt = r'D:\essayexp\Yolo-Fastest-master\build\darknet\x64\data\train.txt'
filetxt = r'D:\essayexp\Yolo-Fastest-master\build\darknet\x64\data\test.txt'

with open(filetxt, 'r') as f:
    lines = f.readlines()

with open(filetxt, 'w') as f:
    for line in lines:
        f.write('D:/essayexp/Yolo-Fastest-master/build/darknet/x64/' + line) # Change here to the path where your own code is placed

The result is as follows:

5) Copy the data folder obtained above directly to the Yolo-Fastest-master\build\darknet\x64 file.

If you are worried about problems, you can rename the data folder in the original code for backup. I just added the words “original code” in front.

6) Modify configuration file
Find yolo-fastest-1.1.cfg in the Yolo-Fastest-master\build\darknet\x64\cfg folder (it can also be other, just modify it later). I also made a backup for the same reason.

cfg file is opened with notepad.

• It stands to reason that batch and subdivisions can be changed to what you want to set, but when I ran the code later, I got an error saying that both must be 64, so I changed both to 64.
  
• Change the classes in [yolo] below and the filters above [yolo].
  Only change one filter above yolo, there are two [yolo] in total, so you only need to change the numbers in 4 places in total, not every filter needs to be changed
  filters = (classes + 5)*3
  classes is the number of categories in your own data set. Write as many as you like. I only have one category, so it’s 1.
  
  7) Create new .data and .names files.
  Create new data and names files in the Yolo-Fastest-master\build\darknet\x64\data folder.
  You can choose between two names, I chose human.
  
  Write the category name of your own data set in .names. If you are not sure, you can refer to the .names file given in the original code to check. The paths in .data are the paths to the previously generated train and test txt files.
  
  8) Generate pre-trained model
• Create a new pretrained_model folder under the Yolo-Fastest-master\build\darknet\x64 folder (the pretrained model will be generated in this folder later)
• Create a new .bat file, write the following instructions and double-click it. New method: First create a new txt file and write the following content in it:

darknet partial cfg\yolo-fastest-1.1.cfg cfg\yolo-fastest-1.1.weights pretrained_model\yolo-fastest-1.1.conv.109 109
pause

After saving, change the suffix of the txt file to .bat.

Double-clicking the .bat file will generate pretrained weights in the pretrained model.

9) Formal training
Continue to create a new train.bat file in the Yolo-Fastest-master\build\darknet\x64 folder and write

darknet detector train data\human.data cfg\yolo-fastest-1.1.cfg pretrained_model\yolo-fastest-1.1.conv.109 backup\
pause

You can also give it another name, as long as it is a .bat file. Pay attention to the above content. The .data file and cfg file must be changed to the names of the files you set previously.
Double-click the train.bat file to run it! If there is an error, correct your error according to the error content.
Training situation:

The image recording the loss will only have a value after the loss drops to 18. However, the loss value is very large at the beginning and there will be no movement. It is not a problem with the model. It will be displayed when the loss drops to 18. A weight file will be generated in the backup. Without CUDA, training will be slower.

3. Test

darknet.exe detector test ./data/human.data ./cfg/yolo-fastest-1.1.cfg ./backup/yolo-fastest-1_last.weights ./data/00064.jpg

Note: The .data file, .cfg file and the name of the picture you want to test must be replaced with your own.