Algorithm introduction
FMM (Fast Marching Method) and NS (Navier-Stokes) are two different image inpainting methods commonly used to fill missing, damaged or contaminated image areas. They have different application scenarios and principles in image processing.
FMM algorithm
FMM application scenarios: FMM is mainly used for image restoration based on physical models, such as medical image processing, geographic information systems (GIS) and natural image restoration.
FMM principle: FMM is based on the numerical solution of the wave equation. It fills in the missing areas by following the path of the wave, starting from a known pixel value and expanding outward. This method simulates the way waves propagate and is therefore able to handle the inpainting of continuous and natural images well, producing smooth and natural results. FMM usually better preserves detail information in images
NS algorithm
NS scene: NS method is usually used for image restoration, including denoising and repair. This is a method that is more widely used in practical applications, including tasks such as red-eye repair in digital photography, stain removal, and cable or watermark removal.
NS principle: The NS method is based on the equations of fluid dynamics and uses partial differential equations (PDE) to simulate the image restoration process. By solving the PDE, the NS method can naturally fill in the missing parts in the image and preserve edge and detail information in the image. NS methods are generally more suitable for real-time or batch processing.
Summary
In practical applications, the choice of which method to use depends on the nature and requirements of the image inpainting task. Typically, the NS method is more commonly used because it has lower computational cost and is suitable for many image repair scenarios. If higher-precision repair is required and computing resources permit, the FMM method can provide better results.
Gui interface design
Use the Tkinter library to create a simple graphical user interface (GUI) for image repair. Restoration is the process of filling in missing or damaged parts of an image. Implementation to load an image, draw over damaged or missing areas in the image, and then perform repair using OpenCV
All code
from tkinter import *
import tkinter.filedialog
from PIL import Image, ImageFilter, ImageTk
import os
import tkinter.messagebox
importtkinter.ttk
import numpy as np
import cv2 as cv
sizex=0
sizey=0
quality=100
path=''
output_path=None
output_file=None
root = Tk()
root.geometry()
label_img=None
#Set window title
root.title('Picture Intelligent Repair')
# OpenCV utility class for handling mouse
classSketcher:
def __init__(self, windowname, dests, colors_func):
self.prev_pt = None
self.windowname = windowname
self.dests = dests
self.colors_func = colors_func
self.dirty = False
self.show()
cv.setMouseCallback(self.windowname, self.on_mouse)
def show(self):
cv.imshow(self.windowname, self.dests[0])
#cv.imshow(self.windowname + ": mask", self.dests[1])
# onMouse function for mouse processing
def on_mouse(self, event, x, y, flags, param):
pt = (x, y)
if event == cv.EVENT_LBUTTONDOWN:
self.prev_pt = pt
elif event == cv.EVENT_LBUTTONUP:
self.prev_pt = None
if self.prev_pt and flags & amp; cv.EVENT_FLAG_LBUTTON:
for dst, color in zip(self.dests, self.colors_func()):
cv.line(dst, self.prev_pt, pt, color, 5)
self.dirty = True
self.prev_pt = pt
self.show()
#Load image
def loadimg():
global path
global sizex
global sizey
path = tkinter.filedialog.askopenfilename()
lb.config(text=path)
if path != '':
try:
img = Image.open(path)
sizex=img.size[0]
sizey=img.size[1]
img=img.resize((400,400),Image.ANTIALIAS)
global img_origin
img_origin = ImageTk.PhotoImage(img)
global label_img
label_img.configure(image=img_origin)
label_img.pack()
except OSError:
tkinter.messagebox.showerror('Error', 'The picture format is wrong and cannot be recognized')
def inpaint(path):
def function(img):
try:
# Create a copy of the original image
img_mask = img.copy()
# Create a black copy of the original image
# Acts as a mask
inpaintMask = np.zeros(img.shape[:2], np.uint8)
# Create sketch using OpenCV Utility Class: Sketcher
sketch = Sketcher('image', [img_mask, inpaintMask], lambda: ((255, 255, 255), 255))
ch = cv.waitKey()
if ch == ord('t'):
# Use the algorithm proposed by Alexendra Telea. fast travel method
res = cv.inpaint(src=img_mask, inpaintMask=inpaintMask, inpaintRadius=3, flags=cv.INPAINT_TELEA)
cv.imshow('Inpaint Output using FMM', res)
cv.waitKey()
cv.imwrite(path, res)
if ch == ord('n'):
# Using algorithms proposed by Bertalmio, Marcelo, Andrea L. Bertozzi and Guillermo Sapiro: Navier-Stokes, fluid dynamics, and rendering of images and videos
res = cv.inpaint(src=img_mask, inpaintMask=inpaintMask, inpaintRadius=3, flags=cv.INPAINT_NS)
cv.imshow('Inpaint Output using NS Technique', res)
cv.waitKey()
cv.imwrite(path, res)
if ch == ord('r'):
img_mask[:] = img
inpaintMask[:] = 0
sketch.show()
cv.destroyAllWindows()
except ValueError as e:
tkinter.messagebox.showerror('',repr(e))
if path != '':
try:
img = Image.open(path)
img1=cv.imread(path,cv.IMREAD_COLOR)
img1 = function(img1)
except OSError:
lb.config(text="You did not select any files")
tkinter.messagebox.showerror('Error', 'The picture format is wrong and cannot be recognized')
else:
tkinter.messagebox.showerror('error', 'path not found')
lb = Label(root,text = 'The image will be saved in the original path')
lb.pack()
lb1 = Label(root,text = 'Warning: The original image will be overwritten',width=27,height=2,font=("Arial", 10),bg="red")
lb1.pack(side='top')
btn = Button(root,text="Select image",command=loadimg)
btn.pack()
lb2 = Label(root,text = 'Press to start drawing the location you want to repair')
lb2.pack()
btn2 = Button(root,text="Start",command=lambda:inpaint(path))
btn2.pack()
lb3 = Label(root,text = 'Use the following steps to complete the drawing')
lb3.pack()
lb4 = Label(root,text = 't-Use FMM to repair\
n-Use NS method to repair\
r-Redraw area')
lb4.pack()
label_img = tkinter.Label(root, text='original picture')
label_img.pack()
root.mainloop()
Operating effects and usage
After copying the above code, you only need to configure the environment and run the interface
1. Select a picture
2. Click the start button and draw on the original image
3. After the drawing is completed, press the t key on the keyboard to use the FMM repair algorithm, press the n key on the keyboard to use the NS repair algorithm, press the r key on the keyboard to redraw, and press Enter. The results are as follows
Environment installation
The requirements.txt file is as follows. You can create a new requirements.txt in your computer and copy the following code into it.
numpy==1.16.4 opencv_python==4.1.0.25 Pillow==8.1.2
Summary
shortcoming:
1. Traditional methods are usually based on mathematical models and signal processing techniques, such as wave equations or partial differential equations. These methods rely on mathematical models to fill in missing or damaged image regions
2. The performance of traditional methods is usually limited by the accuracy and complexity of mathematical models. They may not be able to handle complex image restoration tasks in some cases, especially where highly natural compositions are required.
It is basically possible to complete a small job, and we will continue to write about deep learning image repair later.