Resource download address: https://download.csdn.net/download/sheziqiong/88251303
Resource download address: https://download.csdn.net/download/sheziqiong/88251303
Seamless image stitching
Operating instructions
First install the dependent third-party libraries:
sudo apt install cmake libcgal-dev libcgal-qt5-dev sudo pip2 install opencv-python
Then compile the C++ file:
cmake . make
Generate the executable file main in the current directory.
Use command
./main <args>
You can run or view help, as shown in Figure 1:
Figure 1: View help
Implementation method
Considering the implementation efficiency, I use C++ to write the code, in which the reading and output of images are implemented using the open source warehouse “stb” on GitHub, and the extraction of image edges is implemented using OpenCV.
Algorithm details
MVC
The MVC[3, 2] algorithm can be divided into the following steps:
Find the positions of all edge pixels;
Use CGAL to perform a constrained triangulation of these points;
For non-edge pixels in the triangulation results, calculate the weight to be raised for this point according to the weight calculation method given in the paper;
For each triangular patch, the weights of the three vertices have been calculated, and since the function is assumed to be smooth within this range, a plane is directly used to fit the weights of all pixels in the triangle, that is Can.
Figure 2 shows the intermediate results of the MVC algorithm. Figure 2(a) shows the visualization after finding the edges from the mask. Figure 2(b) shows the triangulated mesh generated from the points on these boundaries. Effect. Please note that the original image is not a convex polygon, so after triangulation, you need to manually remove those triangles that are not within the outline. Specifically, you can use the ray method to determine whether a certain point is within the polygon.
Boundary segmentation (b) Triangulation results Figure 2: MVC visualization intermediate steps
1.2 Poisson
In addition, I also implemented the Poisson image fusion algorithm [4, 5] for comparison, and its gradient calculation method is
(x,y) = 4I(x,y) I(x 1,y) I(x,y 1) I(x + 1,y) I(x,y + 1)
After calculating the gradient of the original image, matching the gradient to a certain area on the target image is essentially a problem of solving a system of linear equations. Formally, if N pixels need to be matched to the target image, a linear equation system Ax =b needs to be solved.
where x represents the value of the pixel in the fused image, the size of the matrix A is N × N, the size of the column vectors x and b is N, and each row of A has at most 5 non-zero elements, and the elements on the diagonal Both are 4.
is a huge sparse matrix. Considering that the complexity of matrix inversion is O(N3) which is too high, and in some cases it cannot be stored directly in matrix form, so it cannot be directly obtained from the formula
x = A1b
Find x. The Jacobi Method is used here to iteratively solve for the value of x, see [1] for details.
1.3 Experimental results
1.3.1 Test1
Use command
time ./main -s img/src3.jpg -t img/target1.jpg -m img/mask3.jpg -o result_MVC.png -h -135 -w -30 -a MVC time ./main -s img/src3.jpg -t img/target1.jpg -m img/mask3.jpg -o result_Poisson.png -i 5000 -h 50 -w 100 -a Poisson
The following results can be obtained
Done with 262 vertices and 466 triangles.
real 0m0.243s
user 0m0.287s
sys 0m0.233s
-----
iter 5001
err 0.001767 0.001913 0.003486
real 0m0.361s
user 0m0.345s
sys 0m0.016s
It can be seen that MVC only uses a total of 262 triangle vertices and takes 0.243s; Poisson uses an iterative method to solve the matrix. The more iterations, the more accurate the solution. After 5000 rounds, the error is almost 0, and the running speed is 0.36s. , both of which are very fast. The synthetic effect is shown in Figure 3. Since this sample is too easy to synthesize, there is no obvious difference in the effect between the two.
Figure 3: The first set of data synthesis effects
1.3.2 Test2
Use command
time ./main -s img/src4.png -t img/target2.png -m img/mask4.png -o result_MVC.png -h 142 -w 132 -a MVC time ./main -s img/src4.png -t img/target2.png -m img/mask4.png -o result_Poisson.png -i 10000 -h 150 -w 150 -a Poisson
The following results can be obtained
Done with 836 vertices and 1580 triangles.
real 0m0.233s
user 0m0.228s
sys 0m0.305s
-----
iter 10001
err 82.519269 49.665685 58.341544
real 0m5.356s
user 0m5.351s
sys 0m0.004s
It can be seen that MVC only used a total of 836 triangle vertices and took 0.228s; Poisson’s total error after 10,000 rounds was less than 100, the average error per pixel was less than 0.01, and the running speed was about 5s. The synthetic effect is shown in Figure 4. By comparison, it can be found that the edges of MVC are smoother than Poisson.
MVC (b) Poisson Figure 4: The second set of data synthesis effects
1.3.3 Test3
Use command
time ./main -s img/src0.jpg -t img/target0.jpg -m img/mask0.png -h
318 -w 370 -o result_MVC.png
time ./main -a Poisson -s img/src0.jpg -t img/target0.jpg -m img/ mask0.png -o result_poisson.png -i 10000 -h 350 -w 400
The following results can be obtained
Done with 1702 vertices and 3346 triangles.
real 0m0.420s
user 0m0.458s
sys 0m0.253s
-----
iter 10001
err 2238.275301 1683.477450 1885.838338
real 0m43.901s
user 0m43.816s
sys 0m0.040s
This is the picture in the MVC paper. You can see that MVC only uses a total of 1702 triangle vertices and takes 0.458s;
The total error of Poisson after 10,000 rounds is about 2,000, and the average error per pixel is less than 0.03, but the running speed reaches about 43 seconds. The synthetic effect is shown in Figure 5. In terms of effectiveness, MVC is better.
MVC (b) Poisson Figure 5: The third set of data synthesis effects
References
Jacobi method. . Accessed: 2018-04-12.
Kewei Chen and Chenen Wu. Mvc. . Accessed: 2018-06-12.
Zeev Farbman, Gil Hoffer, Yaron Lipman, Daniel Cohen-Or, and Dani Lischinski. Coordinates for instant image cloning. ACM Transactions on Graphics (TOG), 28(3):67, 2009.
Patrick Pérez, Michel Gangnet, and Andrew Blake. Poisson image editing. ACM Transactions on graphics (TOG), 22(3):313--318, 2003.
Christopher J. Tralie. Poisson image editing. Accessed: 2018-04-12.
net, and Andrew Blake. Poisson image editing. ACM Transactions on graphics (TOG), 22(3):313–318, 2003.
Christopher J. Tralie. Poisson image editing. Accessed: 2018-04-12.
“`
Resource download address: https://download.csdn.net/download/sheziqiong/88251303
Resource download address: https://download.csdn.net/download/sheziqiong/88251303