Image data enhancement technology in Python

Data enhancement technology in Python

In this article, you will explore different data augmentation techniques in Python using the imgaug library

What is image enhancement

Image enhancement is a powerful technique used to artificially create changes in existing images to expand image datasets. This is achieved by applying different transformation techniques, such as scaling, rotating, shearing or cropping the existing image. The goal is to create a comprehensive set of possible images that represent a variety of variations.

Why image enhancement is needed

Image enhancement is crucial in the context of deep learning convolutional neural networks (CNNs) as it helps meet the need for large and diverse datasets for effective model training. CNNs require large amounts of images to train effectively, and image augmentation provides a way to artificially extend existing datasets. Image enhancement helps generate a more comprehensive set of possible images by creating image variations through techniques such as scaling, rotation, shearing, or cropping. This diverse data set enables the model to generalize better, reduce overfitting, and improve its performance when evaluated on previously unseen data during testing or validation. Therefore, image enhancement is crucial to improve the quality and quantity of training data, ultimately leading to more powerful and accurate CNN models.

When to use image enhancement

Image enhancement can be used as a pre-processing step before training the model, or it can be applied in real time during the training process. When used as a preprocessing step, apply augmentation to increase the size of the dataset, especially when dealing with small training datasets that require scaling. This approach, called offline or preprocessing augmentation, involves generating variations of existing images to create a more diverse dataset. It is important to carefully consider the problem domain when applying image enhancement, as some enhancement strategies may be irrelevant or useless for a specific task. For example, flipping a car vertically may not add value to the data set when classifying different types of cars. Therefore, the application of image enhancement should be tailored to the specific requirements of the problem domain.

Offline or preprocessing enhancement

Offline or preprocessing enhancement refers to applying image enhancement as a preprocessing step to increase the size of the dataset. This approach is typically used when working with small training data sets that require scaling. By generating variations of existing images, such as flipping, rotating, or scaling, offline augmentation can create more diverse datasets for training machine learning models. When applying offline enhancements, especially when working with larger data sets, it is important to consider disk space. This approach allows the creation of a comprehensive set of possible images before the training process begins, ultimately improving the quality and quantity of training data.

Online or real-time enhancement

Online or real-time augmentation involves applying augmentation techniques in real time during training. This method is typically used with larger data sets because it does not require the augmented image to be saved on disk. By applying augmentation in real time, the model sees a different image at each epoch, thus contributing to the diversity of the training data. Live enhancement is particularly useful when working with larger data sets because it reduces the storage requirements associated with saving enhanced images. This approach allows for dynamic application of augmentation techniques during training, contributing to the model’s ability to learn from a variety of image changes.

We will use the imgaug class to demonstrate image enhancement. imgaug supports a wide range of data augmentation techniques

Basic data enhancement techniques

  • Flipping: Flip the image vertically or horizontally

  • Rotation: Rotate the image to a specified degree

  • Shearing: moving part of an image like a parallelogram

  • Cropping: objects appear in different locations in the image at different proportions

  • Zoom in, Zoom out

  • Change brightness or contrast

These data augmentation techniques will now be explored using the imgaug library

Practice Imgaug

imgaug is a library for image enhancement, including keypoints/landmarks, bounding boxes, heatmaps and segmentation maps.

  • Git project address: github.com/aleju/imgau…
  • Project document address: imgaug.readthedocs.io/en/latest

examples

Installation
pip install imageio imgaug
pip install imgaug - upgrade - no-deps # Use this when an error occurs
Import related dependency packages
import imageio
import imgaug as ia
import imgaug.augmenters as iaa
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import matplotlib.patches as patches
import matplotlib
%matplotlib inline
Show original image
image = imageio.v2.imread('./Lenna.jpg')
ia.imshow(image)

Lenna.png

Rotating imageRotating

You can rotate the image by specifying the rotation degree, randomly rotating the image between -50 degrees and 30 degrees. The rotation angle can be a negative value, indicating counterclockwise rotation, or a positive value, indicating clockwise rotation.

rotate = iaa.Affine(rotate=(-50, 30))
rotated_image = rotate.augment_image(image)
ia.imshow(rotated_image)

rotate.png

Add noise to image Noise

Different noise values sampled from Gaussian distribution elements are added to the image. The parameters 10 and 20 represent the mean and standard deviation of the noise respectively. Gaussian noise is a common random noise whose values conform to Gaussian distribution (also called normal distribution). The mean parameter determines the center position of the noise, and the standard deviation parameter determines the intensity of the noise.

gaussian_noise = iaa.AdditiveGaussianNoise(10, 20)
noise_image = gaussian_noise.augment_image(image)
ia.imshow(noise_image)

noise.png

Image cropping Cropping

Cropping will remove columns/rows of pixels on either side of the image. In the example below, one side of the image is randomly cropped by 0~30%. Random cropping gives the image more diversity and change. Cropping operations can be used to remove useless information from the edges of an image, adjust the composition of the image, or create a denser copy of the image.

crop = iaa.Crop(percent=(0, 0.3)) # crop image
corp_image = crop.augment_image(image)
ia.imshow(corp_image)

crop.png

Image shearingShearing

Cut the image randomly from 0 to 40 degrees. The miscut transformation will translate part of the image at a certain angle, thereby changing the shape and content of the image. It can be used to create perspective effects, distort the shape of an image, or simulate deformation in certain scenes.

shear = iaa.Affine(shear=(0, 40))
shear_image=shear.augment_image(image)
ia.imshow(shear_image)

shearing.png

Image flipping

Images can be flipped vertically or horizontally, here we show Fliplr flipping images horizontally. Horizontal flipping can be used to correct the orientation of an image, increase the diversity of training data, or simulate mirror-symmetric scenes.

#flipping image horizontally
flip_hr = iaa.Fliplr(p=1.0) # The flip probability is 100%. When it is less than 1.0, it is a random flip and may not flip.
flip_hr_image = flip_hr.augment_image(image)
ia.imshow(flip_hr_image)

fliplr.png

Fipud flips image vertically

flip_vr = iaa.Flipud(p=1.0) # The flip probability is 100%
flip_vr_image = flip_vr.augment_image(image)
ia.imshow(flip_vr_image)

flipud.png

Image changes brightness

Adjust the brightness of the image by scaling pixel values. Values in the range of Gamma=(0.5, 2.0) are reasonable. You can also use symbolic contrast or linear contrast to change the brightness of the image. The Gamma Contrast Enhancer applies a gamma transform to an image, changing its appearance by adjusting its brightness and contrast. The gamma value determines the strength of the transformation. A higher gamma value will increase the contrast of the image, making the dark parts of the image darker and the bright parts brighter.

contrast = iaa.GammaContrast(gamma=2.0)
contrast_image =contrast.augment_image(image)
ia.imshow(contrast_image)

brightness.png

Image scalingScale

You can use the scale bar to zoom in or out of the image. The image below scales to 150% to 80% of the image height/width. Alternatively, each axis can be scaled independently. The function randomly selects a scaling factor between 1.0 and 1.5 and applies it to the x and y directions of the image. Smaller zoom ratios make the image smaller, while larger zoom ratios make the image larger. Scaling transformations can be used to resize images, change the proportions of objects, or simulate images at different distances.

scale_im = iaa.Affine(scale={"x": (1.5, 1.0), "y": (1.5, 1.0)})
scale_image = scale_im.augment_image(image)
ia.imshow(scale_image)

scale.png

Enhancement of target detection

Draw bounding boxes for object detection. When an image is enhanced, it is expected that the bounding box will also be updated accordingly. imgaug provides support for bounding boxes. When an image is rotated, sheared, or cropped, the bounding box around the object is updated accordingly.

Import bounding box
from imgaug.augmentables.bbs import BoundingBox, BoundingBoxesOnImage
Original picture out of frame
bbs = BoundingBoxesOnImage([BoundingBox(x1=80, x2=140, y1=70, y2=150)], shape=image.shape)
ia.imshow(bbs.draw_on_image(image, size=2))

detection.png

In the code below, a translation percentage is used to move the image, enlarge the bounding box, and apply it to the image

move = iaa.Affine(translate_percent={"x": 0.1}, scale=0.8)
image_aug, bbs_aug = move(image=image, bounding_boxes=bbs)
ia.imshow(bbs_aug.draw_on_image(image_aug, size=2))

move.png

After applying image enhancement, process the bounding box outside the image

The bounding box may sometimes be outside the image, requiring additional code to handle such cases, rotating the image and trying to draw the bounding box around the object.

rotate_bb = iaa.Affine(rotate=(-50, 30))
image_aug, bbs_aug = rotate_bb(image=image, bounding_boxes=bbs)
ia.imshow(bbs_aug.draw_on_image(image_aug, size=2))

rotate.png

Some parts of the bounding box are outside the image. In the code below, you will see these issues:

  • Completely or partially remove the bounding box outside the image
  • Clipping parts of the bounding box that are outside so that they are completely inside the image

Create a fill function that fills the image with a 1 pixel white and 1 pixel black border:

# padding
def pad(image, by):
    image_border1 = iaa.size.pad(image, top=1, right=1, bottom=1, left=1, mode="constant", cval=255)
    image_border2 = iaa.size.pad(image_border1, top=by-1, right=by-1, bottom=by-1, left=by-1, mode="constant", cval=0)
    return image_border2

Then, draw a bounding box on the image. First extend the image plane by boundary pixels, then mark the bounding box within the image plane

def draw_bbs(image, bbs, border):
    GREEN = [0, 255, 0]
    ORANGE = [255, 140, 0]
    RED = [255, 0, 0]
    image_border = pad(image, border)
    for bb in bbs.bounding_boxes:
        if bb.is_fully_within_image(image.shape):
            color=GREEN
        elif bb.is_partly_within_image(image.shape):
            color = ORANGE
        else:
            color=RED
        image_border = bb.shift(left=border, top=border).draw_on_image(image_border, size=2, color=color)
        return image_border

Now, apply the same rotation to the image and draw the bounding box

rotate = iaa.Affine(rotate=(-50, 30))
image_aug, bbs_aug = rotate(image=image, bounding_boxes=bbs)
image_after = draw_bbs(image_aug,
bbs_aug.remove_out_of_image().clip_out_of_image(), 100)
ia.imshow(image_after)

pad-rotate.png


———————————END——————- ——–

Digression

Interested friends will receive a complete set of Python learning materials, including interview questions, resume information, etc. See below for details.

CSDN gift package:The most complete “Python learning materials” on the entire network are given away for free! (Safe link, click with confidence)

1. Python learning routes in all directions

The technical points in all directions of Python have been compiled to form a summary of knowledge points in various fields. Its usefulness is that you can find corresponding learning resources according to the following knowledge points to ensure that you learn more comprehensively.

img
img

2. Essential development tools for Python

The tools have been organized for you, and you can get started directly after installation! img

3. Latest Python study notes

When I learn a certain basic and have my own understanding ability, I will read some books or handwritten notes compiled by my seniors. These notes record their understanding of some technical points in detail. These understandings are relatively unique and can be learned. to a different way of thinking.

img

4. Python video collection

Watch a comprehensive zero-based learning video. Watching videos is the fastest and most effective way to learn. It is easy to get started by following the teacher’s ideas in the video, from basic to in-depth.

img

5. Practical cases

What you learn on paper is ultimately shallow. You must learn to type along with the video and practice it in order to apply what you have learned into practice. At this time, you can learn from some practical cases.

img

6. Interview Guide

CSDN gift package:The most complete “Python learning materials” on the entire network are given away for free! (Safe link, click with confidence)

If there is any infringement, please contact us for deletion.