1. Foreword
Detectron2 is a widely used deep learning framework for target detection and segmentation. I recently configured the environment on my computer and was ready to run it. I found that it was not so convenient to get started, and the official tutorial was not clear. I had to explore it myself, so I will take a note. Personal methods for successful training and solutions to bugs encountered.
If you want to see the bug solution, jump directly to the end of the article.
2. Training detectron2 (self-made target training set for training)
1. The data set needs to be converted into the format of the COCO data set, that is
—-dataset
——–annotations stores annotation files
—————instances_train2017.json
—————instances_val2017.json
——–train2017 stores training set images
——–val2017 stores test set images
The above is the location of the folder file corresponding to the COCO data set format. If you lack experience, follow the standard.
COCO data set format reference official website introduction: https://cocodataset.org/#format-data
Among them target detection:
If the target representation format of the annotation file (such as yolo’s txt) is different from the format on COCO, you need to convert it to a unified format yourself.
2. Configure network and data set path:
You can find a lot of tutorials in this area, either official or from bloggers on major websites. I was a little confused at first and thought it was quite complicated and not clear enough. I fumbled around for a while, and now it’s clearer.
To summarize, there are two methods. One is not very flexible and is suitable for people who don’t want to change the code and follows the steps; the second is more flexible and requires you to know what is needed for training to perform flexible configuration.
The first is step-by-step:
① First register your data set. If it is a self-made target detection data set, it is in the following location of detectron2/detectron2/data/datasets/builtin.py:
_PREDEFINED_SPLITS_COCO = {<!-- -->}
_PREDEFINED_SPLITS_COCO["coco"] = {<!-- -->
"coco_2014_train": ("coco/train2014", "coco/annotations/instances_train2014.json"),
"coco_2014_val": ("coco/val2014", "coco/annotations/instances_val2014.json"),
"coco_2014_minival": ("coco/val2014", "coco/annotations/instances_minival2014.json"),
"coco_2014_valminusminival": (
"coco/val2014",
"coco/annotations/instances_valminusminival2014.json",
),
"coco_2017_train": ("coco/train2017", "coco/annotations/instances_train2017.json"),
"coco_2017_val": ("coco/val2017", "coco/annotations/instances_val2017.json"),
"coco_2017_test": ("coco/test2017", "coco/annotations/image_info_test2017.json"),
"coco_2017_test-dev": ("coco/test2017", "coco/annotations/image_info_test-dev2017.json"),
"coco_2017_val_100": ("coco/val2017", "coco/annotations/instances_val2017_100.json"),
#Make your own data set, be careful not to write the wrong path.
"my_train": ('coco/train2017/images', "coco/annotations/instances_train2017.json"),
"my_val": ('coco/val2017/images', 'coco/annotations/instances_val2017.json'),
# "my_train" : Dataset name
# 'coco/train2017/images': Image storage path
# 'coco/annotations/instances_train2017.json' : Annotation information json path
}
"""The following code was found under the same code"""
# True for open source;
# Internally at fb, we register them elsewhere
if __name__.endswith(".builtin"):
# Assume pre-defined datasets live in `./datasets`.
#_root = os.path.expanduser(os.getenv("DETECTRON2_DATASETS", "datasets"))
"""The path of datasetroot needs to be at the upper level of my_train and my_dval, that is, the upper level of coco.
For example, my_train='./xxx/xxx/coco/annotations/xxx.json'
"""
datasetroot = './xxx/xxx/'
_root = os.path.expanduser(os.getenv("DETECTRON2_DATASETS", datasetroot))
register_all_coco(_root)
register_all_lvis(_root)
register_all_cityscapes(_root)
register_all_cityscapes_panoptic(_root)
register_all_pascal_voc(_root)
register_all_ade20k(_root)
②Then modify detectron2/detectron2/data/datasets/builtin_meta.py
Add the category information of your own data set and a function to obtain your own data set information:
"""Add """
MY_COCO_CATE=[
{<!-- -->"color": [220, 20, 60], "isthing": 1, "id": 1, "name": "my_cate"},
]
"""To add"""
def _get_coco_meta_mycate():
thing_ids = [k["id"] for k in MY_COCO_CATE if k["isthing"] == 1]
thing_colors = [k["color"] for k in MY_COCO_CATE if k["isthing"] == 1]
assert len(thing_ids) == 1, len(thing_ids) #If your data set has several categories, change it to numbers.
# Mapping from the incontiguous COCO category id to an id in [0, 79]
thing_dataset_id_to_contiguous_id = {<!-- -->k: i for i, k in enumerate(thing_ids)}
thing_classes = [k["name"] for k in MY_COCO_CATE if k["isthing"] == 1]
ret = {<!-- -->
"thing_dataset_id_to_contiguous_id": thing_dataset_id_to_contiguous_id,
"thing_classes": thing_classes,
"thing_colors": thing_colors,
}
return ret
"""To change"""
def _get_builtin_metadata(dataset_name):
if dataset_name == "coco":
#return _get_coco_instances_meta()
return _get_coco_meta_mycate()
if dataset_name == "coco_panoptic_separated":
return _get_coco_panoptic_separated_meta()
elif dataset_name == "coco_panoptic_standard":
meta = {<!-- -->}
③This basically completes the configuration and you can train directly:
The –config-file parameter can select a specific network. If you want to pre-train the model, you can download it yourself first.
python ./tools/train_net.py --config-file ./configs/COCO-Detection/retinanet_R_50_FPN_1x.yaml
The second more flexible method can run the network directly in one code, so you need to add relevant information to this code
①Copy a train_net.py file and directly configure the data set and other information in it
#!/usr/bin/env python
# Copyright (c) Facebook, Inc. and its affiliates.
import logging
import os
from collections import OrderedDict
import detectron2.utils.comm as comm
from detectron2.checkpoint import DetectionCheckpointer
from detectron2.config import get_cfg
from detectron2.data import MetadataCatalog
from detectron2.engine import DefaultTrainer, default_argument_parser, default_setup, hooks, launch
from detectron2.evaluation import (
CityscapesInstanceEvaluator,
CityscapesSemSegEvaluator,
COCOEvaluator,
COCOPanopticEvaluator,
DatasetEvaluators,
LVISEvaluator,
PascalVOCDetectionEvaluator,
SemSegEvaluator,
verify_results,
)
from detectron2.modeling import GeneralizedRCNNWithTTA
"""Add code"""
from detectron2.data.datasets import register_coco_instances
from detectron2.data import MetadataCatalog
classnames=['person']
mydata_train_images='./xx/xxx/train2017' #You can write the absolute path directly
mydata_train_labels='./xx/xxa/train.json' #You can write the absolute path directly
mydata_val_images='./xx/xxb/val2017' #You can write the absolute path directly
mydata_val_labels='./xx/xxa/val.json' #You can write the absolute path directly
register_coco_instances("mydata_train", {<!-- -->}, mydata_train_labels, mydata_train_images)
MetadataCatalog.get("mydata_train").thing_classes=classnames
register_coco_instances("mydata_val", {<!-- -->}, mydata_val_labels, mydata_val_images)
MetadataCatalog.get("mydata_val").thing_classes=classnames
"""Here, then back"""
def build_evaluator(cfg, dataset_name, output_folder=None):
if output_folder is None:
output_folder = os.path.join(cfg.OUTPUT_DIR, "inference")
evaluator_list = []
evaluator_type = MetadataCatalog.get(dataset_name).evaluator_type
if evaluator_type in ["sem_seg", "coco_panoptic_seg"]:
evaluator_list.append(
SemSegEvaluator(
dataset_name,
distributed=True,
output_dir=output_folder,
)
)
if evaluator_type in ["coco", "coco_panoptic_seg"]:
evaluator_list.append(COCOEvaluator(dataset_name, output_dir=output_folder))
if evaluator_type == "coco_panoptic_seg":
evaluator_list.append(COCOPanopticEvaluator(dataset_name, output_folder))
if evaluator_type == "cityscapes_instance":
return CityscapesInstanceEvaluator(dataset_name)
if evaluator_type == "cityscapes_sem_seg":
return CityscapesSemSegEvaluator(dataset_name)
elif evaluator_type == "pascal_voc":
return PascalVOCDetectionEvaluator(dataset_name)
elif evaluator_type == "lvis":
return LVISEvaluator(dataset_name, output_dir=output_folder)
if len(evaluator_list) == 0:
raise NotImplementedError(
"no Evaluator for the dataset {} with the type {}".format(dataset_name, evaluator_type)
)
elif len(evaluator_list) == 1:
return evaluator_list[0]
return DatasetEvaluators(evaluator_list)
class Trainer(DefaultTrainer):
@classmethod
def build_evaluator(cls, cfg, dataset_name, output_folder=None):
return build_evaluator(cfg, dataset_name, output_folder)
@classmethod
def test_with_TTA(cls, cfg, model):
logger = logging.getLogger("detectron2.trainer")
# In the end of training, run an evaluation with TTA
# Only support some R-CNN models.
logger.info("Running inference with test-time augmentation ...")
model = GeneralizedRCNNWithTTA(cfg, model)
evaluators = [
cls.build_evaluator(
cfg, name, output_folder=os.path.join(cfg.OUTPUT_DIR, "inference_TTA")
)
for name in cfg.DATASETS.TEST
]
res = cls.test(cfg, model, evaluators)
res = OrderedDict({<!-- -->k + "_TTA": v for k, v in res.items()})
return res
def setup(args):
"""
Create configs and perform basic setups.
"""
cfg = get_cfg()
cfg.merge_from_file(args.config_file)
cfg.merge_from_list(args.opts)
cfg.DATASETS.TRAIN = ("mydata_train",)
cfg.DATASETS.TEST = ("mydata_val",) # No need to fill in if not
cfg.DATALOADER.NUM_WORKERS = 2
#Pre-training model files, you can download them in advance
cfg.MODEL.WEIGHTS = r"detectron2://COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x/137849600/model_final_f10217.pkl"
#Or use your own pre-trained model
# cfg.MODEL.WEIGHTS = "../tools/output/model_00999.pth"
cfg.SOLVER.IMS_PER_BATCH = 2
cfg.OUTPUT_DIR = "./output_trainsample/"
cfg.SOLVER.MAX_ITER = 10000
cfg.SOLVER.BASE_LR = 0.002
cfg.TEST.EVAL_PERIOD= 100
# cfg.INPUT.MAX_SIZE_TRAIN = 400
# cfg.INPUT.MAX_SIZE_TEST = 400
# cfg.INPUT.MIN_SIZE_TRAIN = (160,)
# cfg.INPUT.MIN_SIZE_TEST = 160
cfg.MODEL.DEVICE = 'cpu'
cfg.MODEL.ROI_HEADS.NUM_CLASSES = 1# Number of categories
cfg.MODEL.WEIGHTS = "./model_final.pth" # Pre-training model weights
#cfg.SOLVER.IMS_PER_BATCH = 6 # batch_size=2; iters_in_one_epoch = dataset_imgs/batch_size
ITERS_IN_ONE_EPOCH = int(118/ cfg.SOLVER.IMS_PER_BATCH)
# (ITERS_IN_ONE_EPOCH * ) - 1 # 12 epochs
cfg.SOLVER.MOMENTUM = 0.9
cfg.SOLVER.WEIGHT_DECAY = 0.0001
cfg.SOLVER.WEIGHT_DECAY_NORM = 0.0
cfg.SOLVER.GAMMA = 0.1
cfg.SOLVER.STEPS = (500,)
cfg.SOLVER.WARMUP_FACTOR = 1.0 / 1000
cfg.SOLVER.WARMUP_ITERS = 300
cfg.SOLVER.WARMUP_METHOD = "linear"
cfg.SOLVER.CHECKPOINT_PERIOD = ITERS_IN_ONE_EPOCH - 1
os.makedirs(cfg.OUTPUT_DIR, exist_ok=True) #Pass in the output folder to store trained weights, etc.
cfg.freeze()
default_setup(cfg, args)
returncfg
def main(args):
cfg = setup(args)
if args.eval_only:
model = Trainer.build_model(cfg)
DetectionCheckpointer(model, save_dir=cfg.OUTPUT_DIR).resume_or_load(
cfg.MODEL.WEIGHTS, resume=args.resume
)
res = Trainer.test(cfg, model)
if cfg.TEST.AUG.ENABLED:
res.update(Trainer.test_with_TTA(cfg, model))
if comm.is_main_process():
verify_results(cfg, res)
return res
trainer = Trainer(cfg)
trainer.resume_or_load(resume=args.resume)
if cfg.TEST.AUG.ENABLED:
trainer.register_hooks(
[hooks.EvalHook(0, lambda: trainer.test_with_TTA(cfg, trainer.model))]
)
return trainer.train()
if __name__ == "__main__":
args = default_argument_parser().parse_args()
print("Command Line Args:", args)
launch(
main,
args.num_gpus,
num_machines=args.num_machines,
machine_rank=args.machine_rank,
dist_url=args.dist_url,
args=(args,),
)
In this way, everything is configured in one code, and then you can train directly:
python ./tools/train_net_copy.py --config-file ./configs/COCO-Detection/retinanet_R_50_FPN_1x.yaml
This is the introduction of both methods. The second method is obviously much more convenient.
3. Bug resolution: KeyError: Dataset m’ is not registered! Available datasets are…
During the configuration process, this error was suddenly reported. I was very puzzled when I saw the error. I searched and found that other people had encountered it, but I didn’t see an effective solution. Later, in my personal attempts, I found that this bug was very annoying. It’s easy to solve, just be careful.
Because I was able to run it successfully using the first method, but it was the second method that stuck for me. It mainly happened during the verification phase. I thought the verification set data was not loaded. I was very puzzled. After looking at other people’s configuration files, I found a very stupid operation, which is:
cfg.DATASETS.TRAIN = ("mydata_train")
cfg.DATASETS.TEST = ("mydata_val") # No need to fill in
My validation set cfg.DATASETS.TEST = (“mydata_val”) does not include commas in the brackets, change it to
cfg.DATASETS.TRAIN = ("mydata_train")
cfg.DATASETS.TEST = ("mydata_val",)
That’s it, and only cfg.DATASETS.TEST = (“mydata_val”) is affected. It doesn’t matter if the training set is added or not.
The method I looked for said that the tuple format is required in the parentheses, but it didn’t mention a direct method and took a bit of a detour.