Crack type
There are many types of cracks. Here we only judge linear cracks and network cracks. Linear cracks can be divided into transverse cracks, longitudinal cracks and oblique cracks according to their trends.
I think everyone should have this awareness. In the face of network cracks, do its two-dimensional parameters make sense? The answer is no! If a mesh crack is detected, I think everyone’s first reaction is that it is serious and needs to be repaired. What if it’s a linear crack? I’d also consider whether it’s damaged enough to need repair.
So according to my idea, we can find the area of network cracks to evaluate the degree of damage, and find the area, length and width of linear cracks to evaluate the degree of damage.
Histogram projection method
Histogram projection of mesh cracks
Histogram projection of transverse cracks
Histogram projection of longitudinal cracks
Histogram projection of oblique cracks
The above four pictures are histograms corresponding to the four types of cracks. Combining some of the above characteristics, we can classify the types according to our own data sets.
Get the minimum enclosing rectangle information
Next, the get_minAreaRect_information function extracts relevant information about the minimum circumscribed rectangle from the binary mask image, including center point coordinates, width, height, and rotation angle. The inference_minAreaRect function is used to calculate the width, height and angle information of the minimum enclosing rectangle, and convert the angle into an angle relative to the horizontal direction of the image.
def inference_minAreaRect(minAreaRect):
w, h = minAreaRect[1]
if w > h:
angle = int(minAreaRect[2])
else:
angle = -(90 - int(minAreaRect[2]))
return w, h, angle
def _get_minAreaRect_information(mask):
mask = pz.BinaryImg(mask)
contours, hierarchy = cv2.findContours(mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contour_merge = np.vstack(contours)
minAreaRect = cv2.minAreaRect(contour_merge)
return minAreaRect
pz.BinaryImg obtains the binary image of the image. Please ensure that it is a BGR image when reading.
Initialize parameters for classifying cracks
A ClassificationCrack class is created and the classification parameters of cracks are initialized, including the threshold threshold for classifying cracks, the height-width ratio threshold HWration for classifying cracks, and the histogram ratio threshold Histration for classifying cracks.
class CrackType():
"""Histogram projection method to infer crack type"""
def __init__(self, threshold=3, HWratio=10, Histratio=0.5):
"""
Initialize parameters for classifying cracks
:param threshold: threshold, the threshold used to classify cracks
:param HWratio: Height-to-width ratio, the height-to-width ratio threshold used to classify cracks
:param Histratio: Histogram scale, histogram scale threshold used to classify cracks
"""
self.threshold = threshold
self.HWratio = HWratio
self.Histratio = Histratio
self.types = {0: 'Horizontal',
1: 'Vertical',
2: 'Oblique',
3: 'Mesh'}
Here we use the dictionary self.types so that we can determine the type of crack through key-value pairs.
Bone point projection histogram
Under the ClassificationCrack class, we define a hist_judge method, less_than_T counts the number of pixels in the histogram that are greater than 0 and less than or equal to the threshold self.threshold, and more_than_T counts the number of pixels in the histogram that is greater than the threshold self.threshold. Compare whether the histogram scale threshold is exceeded by more_than_T / (less_than_T + 1e-5).
def hist_judge(self, hist_v):
less_than_T = np.count_nonzero((hist_v > 0) & amp; (hist_v <= self.threshold))
more_than_T = np.count_nonzero(hist_v > self.threshold)
return more_than_T / (less_than_T + 1e-5) > self.Histratio
Crack classification
The classify method is another member method in the ClassificationCrack class. It receives three values. minAreaRect is a tuple, representing the information of the minimum enclosing rectangle, including center point coordinates, width, height and rotation angle; skeleton_pts is an array, representing bones. The coordinates of the point; HW are the height and width of the current patch.
def classify(self, minAreaRect, skeleton_pts, HW):
H, W = HW
w, h, angle = inference_minAreaRect(minAreaRect)
if w / h < self.HWratio or h / w < self.HWratio:
pts_y, pts_x = skeleton_pts[:, 0], skeleton_pts[:, 1]
hist_x = np.histogram(pts_x, W)
hist_y = np.histogram(pts_y, H)
if self.hist_judge(hist_x[0]) and self.hist_judge(hist_y[0]):
return 3
return self.angle2cls(angle)
@staticmethod
def angle2cls(angle):
angle = abs(angle)
assert 0 <= angle <= 90, "ERROR: The angle value exceeds the limit and should be between 0 and 90 degrees!"
if angle < 35:
return 0
elif 35 <= angle <= 55:
return 2
elif angle > 55:
return 1
else:
return None
Use the inference_minAreaRect function to obtain the width w, height h and angle angle of the rotated rectangular box from minAreaRect. Next, determine whether the aspect ratio of the rotated rectangular frame meets the classification conditions by determining whether w / h and h / w are less than self.HWratio.
If the aspect ratio meets the conditions, project skeleton_pts to histograms hist_x and hist_y in the x and y directions, and then use the self.hist_judge method to determine whether the two histograms meet the classification conditions.
If the above conditions are met, it will be considered as a network crack, otherwise angle2cls will be used for angle classification.
Cracks are divided into the following three categories according to the size of the angle:
- If the angle is less than 35 degrees, 0 is returned, indicating a horizontal crack.
- If the angle is between 35 and 55 degrees, 2 is returned, indicating a tilted crack.
- If the angle is greater than 55 degrees, 1 is returned, indicating a vertical crack.
- If the angle is not within the above range, None is returned.
Test file main
"""
How to judge crack classification
Horizontal, longitudinal, network, oblique cracks
"""
import os
import matplotlib.pyplot as plt
import numpy as np
import cv2
import pyzjr as pz
from skimage.morphology import skeletonize
from skimage.filters import threshold_otsu
from skimage.color import rgb2gray
classCrackType():
"""Histogram projection method to infer crack type"""
def __init__(self, threshold=3, HWratio=10, Histratio=0.5):
"""
Initialize parameters for classifying cracks
:param threshold: threshold, the threshold used to classify cracks
:param HWratio: Height-to-width ratio, the height-to-width ratio threshold used to classify cracks
:param Histratio: Histogram scale, histogram scale threshold used to classify cracks
"""
self.threshold = threshold
self.HWratio = HWratio
self.Histratio = Histratio
self.types = {0: 'Horizontal',
1: 'Vertical',
2: 'Oblique',
3: 'Mesh'}
def inference_minAreaRect(self, minAreaRect):
"""
The angle between the long side of the rotated rectangle and the x-axis.
The rotation angle angle is the angle relative to the horizontal direction of the image, ranging from -90 to +90 degrees.
However, in general, we are used to defining an angle as the angle relative to the positive x-axis, ranging from -180 to + 180 degrees.
"""
w, h = minAreaRect[1]
if w > h:
angle = int(minAreaRect[2])
else:
angle = -(90 - int(minAreaRect[2]))
return w, h, angle
def classify(self, minAreaRect, skeleton_pts, HW):
"""
Classify the current crack instance;
It mainly uses the bone point bidirectional projection histogram and the rotated rectangular frame aspect ratio/angle;
:param minAreaRect: Minimum enclosing rectangle, [(cx, cy), (w, h), angle];
:param skeleton_pts: skeleton point coordinates;
:param HW: the height and width of the current patch;
"""
H, W = HW
w, h, angle = self.inference_minAreaRect(minAreaRect)
if w / h < self.HWratio or h / w < self.HWratio:
pts_y, pts_x = skeleton_pts[:, 0], skeleton_pts[:, 1]
hist_x = np.histogram(pts_x, W)
hist_y = np.histogram(pts_y, H)
if self.hist_judge(hist_x[0]) and self.hist_judge(hist_y[0]):
return 3
return self.angle2cls(angle)
def hist_judge(self, hist_v):
less_than_T = np.count_nonzero((hist_v > 0) & amp; (hist_v <= self.threshold))
more_than_T = np.count_nonzero(hist_v > self.threshold)
return more_than_T / (less_than_T + 1e-5) > self.Histratio
@staticmethod
def angle2cls(angle):
angle = abs(angle)
assert 0 <= angle <= 90, "ERROR: The angle value exceeds the limit and should be between 0 and 90 degrees!"
if angle < 35:
return 0
elif 35 <= angle <= 55:
return 2
elif angle > 55:
return 1
else:
return None
def _get_minAreaRect_information(mask):
"""
Obtain the relevant information of the minimum circumscribed rectangle from the binary mask image
:param mask: Binarized mask image, including the white area of the target area
:return: Information about the minimum enclosing rectangle, including center point coordinates, width, height and rotation angle
"""
mask = pz.BinaryImg(mask)
contours, hierarchy = cv2.findContours(mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contour_merge = np.vstack(contours)
minAreaRect = cv2.minAreaRect(contour_merge)
return minAreaRect
def SkeletonMap(target):
"""
Get skeleton diagram information
:param target: target map
:return: Skeleton diagram and an array, where each row represents the index (y, x) of a non-zero element, including row index and column index
"""
gray = rgb2gray(target)
thresh = threshold_otsu(gray)
binary = gray > thresh
skimage = skeletonize(binary)
skepoints = np.argwhere(skimage)
skimage = skimage.astype(np.uint8)
return skimage, skepoints
if __name__ == '__main__':
plt.switch_backend('TkAgg')
masks_dir = r"D:\PythonProject\RoadCrack\dimension2_data\
um" # Change here to the path to store the above image
results_save_dir = "A_results"
os.makedirs(results_save_dir, exist_ok=True)
classifier = CrackType()
imgfile,_ = pz.getPhotopath(masks_dir, debug=False)
for path in imgfile:
mask = cv2.imread(path)
H, W = mask.shape[:2]
mask_copy = mask.copy()
skeimage, skepoints = SkeletonMap(mask_copy)
minAreaRect=_get_minAreaRect_information(mask)
pts_y, pts_x = skepoints[:, 0], skepoints[:, 1]
hist_x = np.histogram(pts_x, W)
hist_y = np.histogram(pts_y, H)
result = classifier.classify(minAreaRect, skepoints, HW=(H, W))
crack_type = classifier.types[result]
print(crack_type)
T = classifier.threshold
plt.figure(figsize=(10, 5))
plt.subplot(121)
plt.plot(hist_x[1][:-1], [T] * len(hist_x[0]), 'r')
plt.bar(hist_x[1][:-1], hist_x[0])
plt.title("Histogram X")
plt.subplot(122)
plt.plot(hist_y[1][:-1], [T] * len(hist_y[0]), 'r')
plt.bar(hist_y[1][:-1], hist_y[0])
plt.title("Histogram Y")
plt.tight_layout() # Automatically adjust sub-picture layout to prevent overlap
plt.show()
Comparing with our actual pictures, the detection results are pretty good. The three initial values of threshold, HWratio, and Histraio are all obtained from experience and should be set according to your own data. The SkeletionMap function here will obtain the index points in the skeleton diagram. It does not eliminate burrs and does not actually affect it. Because of the method we use, some burrs will not affect the judgment.
Now we only need to write a function to push the crack type, which can be used to directly determine the crack type we set:
def infertype(mask):
"""Derivation of crack types"""
crack = CrackType()
H, W = mask.shape[:2]
mask_copy = mask.copy()
skeimage, skepoints = SkeletonMap(mask_copy)
minAreaRect = _get_minAreaRect_information(mask)
result = crack.classify(minAreaRect, skepoints, HW=(H, W))
crack_type = crack.types[result]
return result, crack_type