Computer Vision: Using opencv to realize bank card number recognition

1 Overview

1.1 Opencv introduction

OpenCV is the abbreviation of Open Source Computer Vision Library. It was proposed and established by Intel in 1999 and is now supported by Willow Garage. It is a highly open source computer vision library that can implement Windows, Linux, Cross-platform operation on Mac and other platforms. opencv is an open source function library used for image processing, analysis, and machine vision. It has become a powerful tool for learning computer vision. In the fields of intrusion detection, specific target tracking, target detection, face detection, face recognition, face tracking and other fields, opencv can be said to be showing its talents. In this article, opencv is mainly used for bank card number identification.

1.2 Bank card number identification steps

The recognition process of bank card numbers mainly includes basic image operations of reading images, using templates to match the processed bank cards, and finally identifying the bank card number. The image operations involved include: grayscale conversion, binary conversion, threshold segmentation, contour detection, top hat operation, gradient operation, closing operation, and template matching.

1.2.1 Preprocessing template images

First, you need to cut out the numbers in the template separately, then cut out the numbers on the bank card separately, and finally compare the numbers on the bank card one by one against the template (0-9, 10 numbers).

The original image is as follows:

The storage path is: “../data/card_template.jpg”

Assume that each number in the template is cut into a rectangle. You can first find the outer contour of each number, and then obtain the outer rectangle based on the outline, and then you can cut it out. For outer contour processing, a binary image needs to be passed in. So the steps are as follows:

• Read in image template

template = cv2.imread('../data/card_template.jpg')
ShowImage('template', template)

• Convert to grayscale

# Convert image to grayscale
image_Gray = cv2.cvtColor(template, cv2.COLOR_RGB2GRAY)
ShowImage('gray', image_Gray)

• Convert to binary image

# Convert to a binary image, [1] means return a binary image, [0] means return a threshold of 177
image_Binary = cv2.threshold(image_Gray, 177, 255, cv2.THRESH_BINARY_INV)[1]
ShowImage('binary', image_Binary)

• Draw the outer outlines of the 10 numbers 0-9

# Extract contours
refcnts, his = cv2.findContours(image_Binary.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cv2.drawContours(template, refcnts, -1, (0,0,255), 2)
ShowImage('contour', template)

• Calculate the bounding rectangle and resize it to the appropriate size

# Traverse each contour
for (i, c) in enumerate(refCnts):
# Calculate the bounding rectangle and resize it to the appropriate size
(x, y, w, h) = cv2.boundingRect(c) #External rectangle
roi = ref[y:y + h, x:x + w]
roi = cv2.resize(roi, (57, 88))
# Each number corresponds to each template
digits[i] = roi

1.2.2 Preprocessing bank card images

For bank card images, the background needs to be filtered out and the main information retained (steps 1-6 below). The above template is cut out in a rectangular shape, so the card number is also cut out in a rectangular shape for easy matching. The bank card number position is a group of four digits. You can process one group first, and then cut each number in each group for template matching. You can filter out other information on the bank card that is not the card number through the aspect ratio.

Bank card image storage path: “../data/credit03.jpg”

• Read in the bank card to be identified and convert it into a grayscale image

# Read the image and perform preprocessing
image = cv2.imread("../data/credit03.jpg")
ShowImage('card', image)

The displayed results are as follows:

image = resize(image, width=300)
#Convert the image to grayscale
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
ShowImage('card_gray', gray)

The displayed results are as follows:

• Top hat operation: Top hat operation can highlight brighter areas (original input – opening operation (corrosion first and then expansion))

# Use the top hat operation to highlight brighter areas
tophat = cv2.morphologyEx(gray, cv2.MORPH_TOPHAT, rectKernel)
ShowImage('tophat_card', tophat)

The displayed results are as follows:

• Gradient operation (Sobel operator): edge detection, can calculate the contour

gradx = cv2.Sobel(tophat, ddepth=cv2.CV_32F, dx=1, dy=0, ksize=-1)
grady = cv2.Sobel(tophat, ddepth=cv2.CV_32F, dx=0, dy=1, ksize=-1)
gradx = np.absolute(gradx)
minVal = np.min(gradx)
maxVal = np.max(gradx)
# (minVal, maxVal) = (np.min(gradx), np.max(gradx))
# The guaranteed value range is between 0-255
gradx = (255 * ((gradx - minVal) / (maxVal - minVal)))
gradx = gradx.astype("uint8")

print(np.array(gradx).shape)
ShowImage('gradx_card', gradx)

The displayed results are as follows:

• Closing operation: connect the numbers together through the closing operation (first expansion, then erosion), which facilitates the subsequent calculation of the rectangular frame

# Through the closing operation, first expand and then corrode, connect the numbers together
gradx = cv2.morphologyEx(gradx, cv2.MORPH_CLOSE,rectKernel)
ShowImage('gradx_card', gradx)

The displayed results are as follows:

• Threshold Segmentation: Use thresholds to binarize images and focus on the processing part

# THRESH_OTSU will automatically find a suitable threshold, suitable for double peaks, and the threshold needs to be set to 0
thresh = cv2.threshold(gradx, 0, 255, cv2.THRESH_BINARY | cv2.THRESH_OTSU)[1]
ShowImage('thresh_card', thresh)

The displayed results are as follows:

• Then perform the closing operation: fill in the connected numbers in the picture a little fuller

# Perform another closing operation to fill the black area within the white frame.
thresh = cv2.morphologyEx(thresh, cv2.MORPH_CLOSE, sqKernel)
ShowImage('thresh2_card', thresh)

The displayed results are as follows:

• Calculate the outer outline: After the above series of operations, we have clear candidates for the places where the numbers are in the bank card. Just like processing the template object, draw all the rectangular frames through the outer outline of the places that may be the numbers. You can filter again later.

# Calculate outline
threshCnts, his = cv2.findContours(thresh.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = threshCnts
cur_img = image.copy()
cv2.drawContours(cur_img, cnts, -1, (0,0,255), 2)
ShowImage('contour_card', cur_img)

The displayed results are as follows:

• Calculate the bounding rectangle and filter the matching rectangle

locs = []
# Traverse contours
for (i, c) in enumerate(cnts): # Function is used to iterate over the elements in the sequence and their subscripts
    # Calculate rectangle
    (x, y, w, h) = cv2.boundingRect(c)
    ar = w/float(h)
    # Select the appropriate area, based on the actual task, here are four numbers as a group
    if ar > 2.5 and ar < 5.0:
        if (w > 40 and w < 85) and (h > 10 and h < 20):
            # Leave the ones that match
            locs.append((x,y,w,h))

# Sort the matching contours from left to right according to the value of x
locs = sorted(locs, key=lambda x: x[0])

• Process each rectangle individually

output =[]
# Iterate through each number in the contour
for (i,(gx, gy, gw, gh)) in enumerate(locs):
    #Initialize the linked list
    groupOutput = []
    # Extract each group according to the coordinates, and take a little more outside, otherwise you can’t see clearly.
    group = gray[gy-5:gy + gh + 5,gx-5:gx + gw + 5]
    ShowImage('group', group)
    # Preprocessing
    group = cv2.threshold(group, 0, 255, cv2.THRESH_BINARY | cv2.THRESH_OTSU)[1] # Binarization
    ShowImage('group', group)

    # Find the outline of each group
    digitCnts, his = cv2.findContours(group.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
    # digitCnts = sortContours(digitCnts, method="LefttoRight")[0]
    # Sort the found contours
    digitCnts = sort_contours(digitCnts, method="left-to-right")[0] 

1.2.3 Template matching score calculation

# Calculate each value in each group
    for c in digitCnts:
        # Find the outline of the current value and resize it to a suitable size
        (x,y,w,h) = cv2.boundingRect(c)
        roi = group[y:y + h, x:x + w]
        roi = cv2.resize(roi, (57,88))
        ShowImage('roi', roi)
        scores = []
        for(digit, digitROI) in digits.items():
            # Template matching
            #
            result = cv2.matchTemplate(roi, digitROI, cv2.TM_CCOEFF)
            (_, score, _, _) = cv2.minMaxLoc(result)
            scores.append(score)
        # Get the most appropriate number
        groupOutput.append(str(np.argmax(scores)))

1.2.4 Drawing results

# Draw rectangle and font
        cv2.rectangle(image, (gx - 5, gy - 5), (gx + gw + 5, gy + gh + 5), (0,0,255),1)
        cv2.putText(image, "".join(groupOutput), (gx, gy-15), cv2.FONT_HERSHEY_SIMPLEX,0.65, (0,0,255),2)
        # got the answer
        output.extend(groupOutput)

2 Complete bank card number identification code

import cv2
import numpy as np


def ShowImage(name, image):
    cv2.imshow(name, image)
    cv2.waitKey(0) #Waiting time, 0 means exit with any key
    cv2.destroyAllWindows()


def sort_contours(cnts, method="left-to-right"):
    # reverse = False means ascending order. If reverse is not specified, ascending order will be defaulted.
    reverse=False
    i = 0

    if method == "right-to-left" or method == "bottom-to-top":
        reverse = True # reverse = True means descending order

    if method == "top-to-bottom" or method == "bottom-to-top":
        i=1

    # Use a smallest rectangle to wrap the found shape, represented by x, y, h, w
    boundingBoxes = [cv2.boundingRect(c) for c in cnts]

    # The zip function is used to pack iterable data and obtain the final output cnts and boundingBoxes.
    (cnts, boundingBoxes) = zip(*sorted(zip(cnts, boundingBoxes),
                                        key=lambda b: b[1][i], reverse=reverse))

    return cnts, boundingBoxes


def resize(image, width=None, height=None, inter=cv2.INTER_AREA):
    dim=None
    (h, w) = image.shape[:2] # Get the height and width of the image
    if width is None and height is None:
        return image
    if width is None:
        r = height / float(h)
        dim = (int(w * r), height)
    else:
        r = width / float(w)
        dim = (width, int(h * r))
    resized = cv2.resize(image, dim, interpolation=inter) # Use the resize function of the cv library
    return resized


template = cv2.imread('../data/card_template.jpg')
ShowImage('template', template)

#Convert the image to grayscale
image_Gray = cv2.cvtColor(template, cv2.COLOR_RGB2GRAY)
ShowImage('gray', image_Gray)

# Convert to a binary image, [1] means return a binary image, [0] means return a threshold of 177
image_Binary = cv2.threshold(image_Gray, 177, 255, cv2.THRESH_BINARY_INV)[1]
ShowImage('binary', image_Binary)

#Extract contours
refcnts, his = cv2.findContours(image_Binary.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cv2.drawContours(template, refcnts, -1, (0, 0, 255), 2)
ShowImage('contour', template)

refcnts = sort_contours(refcnts, method="left-to-right")[0]
digits = {}

# Iterate through each contour
for (i, c) in enumerate(refcnts): # The enumerate function is used to iterate over the elements in the sequence and their subscripts
    (x, y, w, h) = cv2.boundingRect(c)
    roi = image_Binary[y:y + h, x:x + w]
    roi = cv2.resize(roi, (57, 88))

    digits[i] = roi

#Initialize the convolution kernel
rectKernel = cv2.getStructuringElement(cv2.MORPH_RECT, (9,3))
sqKernel = cv2.getStructuringElement(cv2.MORPH_RECT, (5,5))

# Read the image and perform preprocessing
image = cv2.imread("../data/credit03.jpg")
ShowImage('card', image)

image = resize(image, width=300)
#Convert the image to grayscale
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
ShowImage('card_gray', gray)

# Use the top hat operation to highlight brighter areas
tophat = cv2.morphologyEx(gray, cv2.MORPH_TOPHAT, rectKernel)
ShowImage('tophat_card', tophat)

gradx = cv2.Sobel(tophat, ddepth=cv2.CV_32F, dx=1, dy=0, ksize=-1)
grady = cv2.Sobel(tophat, ddepth=cv2.CV_32F, dx=0, dy=1, ksize=-1)
gradx = np.absolute(gradx)
minVal = np.min(gradx)
maxVal = np.max(gradx)
# (minVal, maxVal) = (np.min(gradx), np.max(gradx))
# The guaranteed value range is between 0-255
gradx = (255 * ((gradx - minVal) / (maxVal - minVal)))
gradx = gradx.astype("uint8")

print(np.array(gradx).shape)
ShowImage('gradx_card', gradx)

# Through the closing operation, first expand and then corrode, connect the numbers together
gradx = cv2.morphologyEx(gradx, cv2.MORPH_CLOSE,rectKernel)
ShowImage('gradx_card', gradx)

# THRESH_OTSU will automatically find a suitable threshold, suitable for double peaks. The threshold needs to be set to 0
thresh = cv2.threshold(gradx, 0, 255, cv2.THRESH_BINARY | cv2.THRESH_OTSU)[1]
ShowImage('thresh_card', thresh)

# Do another closing operation to fill the black area in the white frame
thresh = cv2.morphologyEx(thresh, cv2.MORPH_CLOSE, sqKernel)
ShowImage('thresh2_card', thresh)

# Calculate contour
threshCnts, his = cv2.findContours(thresh.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = threshCnts
cur_img = image.copy()
cv2.drawContours(cur_img, cnts, -1, (0,0,255), 2)
ShowImage('contour_card', cur_img)

locs = []
# Traverse contours
for (i, c) in enumerate(cnts): # Function is used to iterate over the elements in the sequence and their subscripts
    # Calculate rectangle
    (x, y, w, h) = cv2.boundingRect(c)
    ar = w/float(h)
    # Select the appropriate area, based on the actual task, here are four numbers as a group
    if ar > 2.5 and ar < 5.0:
        if (w > 40 and w < 85) and (h > 10 and h < 20):
            # Leave the ones that match
            locs.append((x,y,w,h))

# Sort the matching contours from left to right according to the value of x
locs = sorted(locs, key=lambda x: x[0])

output=[]
# Iterate through each number in the contour
for (i,(gx, gy, gw, gh)) in enumerate(locs):
    #Initialize the linked list
    groupOutput = []
    # Extract each group according to the coordinates, and take a little more outside, otherwise you can’t see clearly.
    group = gray[gy-5:gy + gh + 5,gx-5:gx + gw + 5]
    ShowImage('group', group)
    # Preprocessing
    group = cv2.threshold(group, 0, 255, cv2.THRESH_BINARY | cv2.THRESH_OTSU)[1] # Binarization
    ShowImage('group', group)

    # Find the outline of each group
    digitCnts, his = cv2.findContours(group.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
    # digitCnts = sortContours(digitCnts, method="LefttoRight")[0]
    # Sort the found contours
    digitCnts = sort_contours(digitCnts, method="left-to-right")[0]

    # Calculate each value in each group
    for c in digitCnts:
        # Find the outline of the current value and resize it to a suitable size
        (x,y,w,h) = cv2.boundingRect(c)
        roi = group[y:y + h, x:x + w]
        roi = cv2.resize(roi, (57,88))
        ShowImage('roi', roi)
        scores = []
        for(digit, digitROI) in digits.items():
            # Template matching
            #
            result = cv2.matchTemplate(roi, digitROI, cv2.TM_CCOEFF)
            (_, score, _, _) = cv2.minMaxLoc(result)
            scores.append(score)
        # Get the most appropriate number
        groupOutput.append(str(np.argmax(scores)))

        # Draw rectangle and font
        cv2.rectangle(image, (gx - 5, gy - 5), (gx + gw + 5, gy + gh + 5), (0,0,255),1)
        cv2.putText(image, "".join(groupOutput), (gx, gy-15), cv2.FONT_HERSHEY_SIMPLEX,0.65, (0,0,255),2)
        # got the answer
        output.extend(groupOutput)

ShowImage('card_result', image)

operation result:

The knowledge points of the article match the official knowledge files, and you can further learn related knowledge. OpenCV skill tree Home page Overview 24094 people are learning the system