1. Camera calibration principle
In-depth science popularization: understand camera calibration in one article – Zhihu (zhihu.com)
2. Install the camera calibration package and start calibration
Now we know that the purpose of calibrating the camera is to obtain the internal parameter matrix and distortion coefficient, and to obtain the internal parameter matrix and distortion coefficient is to later use the function solvepnp that comes with opencv to solve the external parameter matrix.
First install the usb-cam function package
sudo apt install ros-noetic-usb-cam
Then install the camera calibration package that comes with ros
#noetic camera calibration tool installation sudo apt-get install ros-noetic-camera-calibration
Start the camera and start calibration
#Start camera roslaunch usb_cam usb_cam-test.launch #Start calibration rosrun camera_calibration cameracalibrator.py --size 8x6 --square 0.024 image:=/usb_cam/image_raw camera:=/usb_cam
Note before calibration:
It is recommended that you first look at the length and width of each frame obtained by opening the camera when processing the image, and then change the image obtained during calibration to this size. Because if the size of the processed image is different from the image obtained during calibration, the internal parameters will also be different.
How to change it specifically:
Find the file usb_cam-test.launch, usually under the path /opt/ros/noetic/share/usb_cam/launch. The original file is generally read-only. We can copy the contents and put it in a directory called my_usb_cam. -In the test.launch file, change the image_width and image_height parameters to the default sizes obtained when the camera is turned on. Here they are 1280 and 720.
Then put this launch file into the launch folder under a function package
For example, I put it in the opencv function package, create a new launch folder and put it in it.
At this time
#Start camera roslaunch usb_cam usb_cam-test.launch
Change to:
#Start camera roslaunch opencv my_usb_cam-test.launch roslaunch package name launch file name
after that
#Start calibration rosrun camera_calibration cameracalibrator.py --size 8x6 --square 0.024 image:=/usb_cam/image_raw camera:=/usb_cam
The parameters in the calibration command mean as follows:
1) size: Calibrate the number of internal corner points of the chessboard. The chessboard used here has a total number of rows, and each row has an internal corner point.
2) square: This parameter corresponds to the side length of each checkerboard, and the unit is meters.
3) image and camera: Set the image topic published by the camera.
Move the chessboard according to x (left and right), y (up and down), size (front and back), skew (tilt), etc., until the progress bars of x, y, size, and skew all turn to green positions.
If you feel it’s almost done, you can press CALIBRATE.
Just copy the internal parameters and distortion coefficients in the terminal and put them into a txt file you created yourself.
3. Installation of QR code recognition function package
QR code identifies the package that needs to be downloaded
- Install the ZBar library, here use apt installation instead of source code
sudo apt install libzbar-dev
Using OpenCV-C++ and ZBar library in Ubuntu_Installing zbar on Raspberry Pi-CSDN blog
You can check whether your zbar is installed through the above blog.
4. opencv installation
How to install OpenCV on Ubuntu 20.04_ubuntu import cv2-CSDN Blog
You can try the first one first, and if you have any problems, try the following tutorials, or search for other tutorials yourself.
If you enter
pkg-config --modversion opencv4
Output:
4.2.0
This means that version 4.2.0 of opencv is successfully installed.
opencv is under /usr/share/opencv4; the header file is under /usr/include/opencv4, you can check it yourself.
5. Use of solvepnp and zbar (implementation of vscode + zbar + opencv + ros programming)
This is a function that comes with opencv, you can learn about it yourself. We want to calculate the external parameter matrix through this function.
OpenCV: solvePnP parameter introduction_cv::solvepnp-CSDN blog
You can see that the results of our previous camera calibration are used here, that is, the camera’s internal parameters and distortion coefficients.
The following is the code implementation of ros plus C++ plus opencv plus zbar
Create a function package “opencv” under the workspace. You can choose another name.
Create a new QR_ROS.cpp in the src directory of opencv. The code is as follows:
#include"ros/ros.h"
#include <iostream>
#include <opencv2/opencv.hpp>
#include <zbar.h>
using namespace std;
using namespace cv;
using namespace zbar;
//Camera internal parameter matrix
const Mat camera_matrix = (Mat_<double>(3, 3) << 970.59132483, 0, 646.87144122,0, 971.20780134, 427.7375449,0.00, 0.0, 1.0);
//Camera distortion coefficient
const Mat dist_coeffs = (Mat_<double>(5, 1) << -0.12721462, -1.10308713,0.045049, -0.01256439, 2.92627787);
int main(int argc, char *argv[]){
ros::init(argc,argv,"QR_ROS");//ros node name cannot be repeated
ros::NodeHandle nh;
cv::VideoCapture cap(0); //Open the camera device, 0 represents the default camera
if (!cap.isOpened()) {
std::cerr << "Error: Couldn't open the camera." << std::endl;
return 1;
}
// Initialize the ZBar scanner
zbar::ImageScanner scanner;
cv::Mat frame;
cv::Mat frame_gray;
cv::Scalar color = cv::Scalar(0, 255, 255);
cv::Scalar color1 = cv::Scalar(0, 0, 255);
cv::Scalar color2 = cv::Scalar(0, 255, 0);
cv::Scalar color3 = cv::Scalar(255, 0, 0);
cv::Scalar color4 = cv::Scalar(255, 255, 0);
while (true) {
//cv::Mat frame;
cap >> frame; // Capture frame from camera
// Convert OpenCV frame to grayscale
cv::cvtColor(frame, frame_gray, cv::COLOR_BGR2GRAY);
//Create a ZBar image
zbar::Image zbar_image(frame_gray.cols, frame_gray.rows, "Y800", frame_gray.data, frame_gray.cols * frame_gray.rows);
// Scan the image for barcodes
int result = scanner.scan(zbar_image);
// Iterate through the results
for (zbar::Image::SymbolIterator symbol = zbar_image.symbol_begin(); symbol != zbar_image.symbol_end(); + + symbol) {
std::cout << "Data: " << symbol->get_data() << std::endl;
//Positioning information,
int x = symbol->get_location_x(0);
int y = symbol->get_location_y(0);
cv::circle(frame, cv::Point(x,y), 15, color, 2, cv::LINE_8);
//Calculate the width and height based on the pixel position of the polygon returned by zbar
//The default QR code is vertical and horizontal
int min_x = 0, min_y=0, max_x=0, max_y=0;
Symbol::PointIterator pt = symbol->point_begin();
Symbol::Point p = *pt;
Symbol::PointIterator pt1 = symbol->point_begin();
Symbol::Point p1 = *pt1;
Symbol::PointIterator begin = symbol->point_begin();
Symbol::Point begin_1 = *begin;
//Draw a circle point by point and frame the QR code
+ + pt1;
p1 = *pt1;
//Draw a solid circle
cv::circle(frame, cv::Point(p.x,p.y), 10, color1, 2, cv::LINE_8);
cv::line(frame, cv::Point(p.x,p.y), cv::Point(p1.x, p1.y), color, 3);
+ + pt;
+ + pt1;
p1 = *pt1;
p = *pt;
cv::circle(frame, cv::Point(p.x,p.y), 10, color2, 2, cv::LINE_8);
cv::line(frame, cv::Point(p.x,p.y), cv::Point(p1.x, p1.y), color, 3);
+ + pt;
+ + pt1;
p1 = *pt1;
p = *pt;
cv::circle(frame, cv::Point(p.x,p.y), 10, color3, 2, cv::LINE_8);
cv::line(frame, cv::Point(p.x,p.y), cv::Point(p1.x, p1.y), color, 3);
+ + pt;
p = *pt;
cv::circle(frame, cv::Point(p.x,p.y), 10, color4, 2, cv::LINE_8);
cv::line(frame, cv::Point(p.x,p.y), cv::Point(begin_1.x, begin_1.y), color, 3);
// Counterclockwise starting from the upper left corner
std::vector<Point2d> image_points;
pt = symbol->point_begin();
for (; pt != (Symbol::PointIterator)symbol->point_end(); + + pt) {
p = *pt;
image_points.push_back(Point2d(p.x, p.y));
}
// 3D feature point world coordinates, corresponding to pixel coordinates, unit is mm
std::vector<Point3d> model_points;
model_points.push_back(Point3d(-30.0, -30.0, 0.0)); // Upper left corner unit mm
model_points.push_back(Point3d(-30.0, + 30.0, 0.0));
model_points.push_back(Point3d( + 30.0, + 30.0, 0.0));
model_points.push_back(Point3d( + 30.0, -30.0, 0.0));
cout << "Camera Matrix " << endl << camera_matrix << endl << endl;
// rotation vector
Mat rotation_vector;
// translation vector
Mat translation_vector;
// pnp solution
solvePnP(model_points, image_points, camera_matrix, dist_coeffs, rotation_vector, translation_vector);
//Default ITERATIVE method, you can try to modify it to EPNP (CV_EPNP), P3P (CV_P3P)
cout << "Rotation Vector " << endl << rotation_vector << endl << endl;
cout << "Translation Vector" << endl << translation_vector << endl << endl;
}
// Display the frame with detected barcodes
cv::imshow("Barcode Scanner", frame);
// Exit on 'q' key press
if (cv::waitKey(1) == 'q') {
break;
}
}
//zbar_image.set_data(NULL, 0); //Clear cache
return 0;
}
The directory structure is as follows, don’t worry about the open_camera.cpp
Write the CMake file (the one in the src directory, don’t make a mistake) as follows:
cmake_minimum_required(VERSION 3.0.2)
project(opencv)
## Compile as C++11, supported in ROS Kinetic and newer
# add_compile_options(-std=c++11)
#set(OpenCV_DIR /usr/share/opencv4/)
## Find catkin macros and libraries
## if COMPONENTS list like find_package(catkin REQUIRED COMPONENTS xyz)
## is used, also find other catkin packages
set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} "/home/qly/VS_Project/Ros_Debug/src/opencv/cmake") #Write the absolute path of FindZBar
find_package(catkin REQUIRED COMPONENTS
roscpp
rospy
std_msgs
)
find_package(OpenCV REQUIRED)
#set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} "${CMAKE_SOURCE_DIR}/cmake")
# Find the ZBar library in CMake
find_package(ZBar REQUIRED)
## System dependencies are found with CMake's conventions
# find_package(Boost REQUIRED COMPONENTS system)
## Uncomment this if the package has a setup.py. This macro ensures
## modules and global scripts declared therein get installed
## See http://ros.org/doc/api/catkin/html/user_guide/setup_dot_py.html
#catkin_python_setup()
###############################################
## Declare ROS messages, services and actions ##
###############################################
## To declare and build messages, services or actions from within this
## package, follow these steps:
## * Let MSG_DEP_SET be the set of packages whose message types you use in
## your messages/services/actions (e.g. std_msgs, actionlib_msgs, ...).
## * In the file package.xml:
## * add a build_depend tag for "message_generation"
## * add a build_depend and a exec_depend tag for each package in MSG_DEP_SET
## * If MSG_DEP_SET isn't empty the following dependency has been pulled in
## but can be declared for certainty nevertheless:
## * add a exec_depend tag for "message_runtime"
## * In this file (CMakeLists.txt):
## * add "message_generation" and every package in MSG_DEP_SET to
## find_package(catkin REQUIRED COMPONENTS ...)
## * add "message_runtime" and every package in MSG_DEP_SET to
## catkin_package(CATKIN_DEPENDS ...)
## * uncomment the add_*_files sections below as needed
## and list every .msg/.srv/.action file to be processed
## * uncomment the generate_messages entry below
## * add every package in MSG_DEP_SET to generate_messages(DEPENDENCIES ...)
## Generate messages in the 'msg' folder
# add_message_files(
#FILES
#Message1.msg
#Message2.msg
# )
## Generate services in the 'srv' folder
# add_service_files(
#FILES
#Service1.srv
#Service2.srv
# )
## Generate actions in the 'action' folder
# add_action_files(
#FILES
# Action1.action
# Action2.action
# )
## Generate added messages and services with any dependencies listed here
# generate_messages(
#DEPENDENCIES
# std_msgs
# )
###############################################
## Declare ROS dynamic reconfigure parameters ##
###############################################
## To declare and build dynamic reconfigure parameters within this
## package, follow these steps:
## * In the file package.xml:
## * add a build_depend and a exec_depend tag for "dynamic_reconfigure"
## * In this file (CMakeLists.txt):
## * add "dynamic_reconfigure" to
## find_package(catkin REQUIRED COMPONENTS ...)
## * uncomment the "generate_dynamic_reconfigure_options" section below
## and list every .cfg file to be processed
## Generate dynamic reconfigure parameters in the 'cfg' folder
# generate_dynamic_reconfigure_options(
#cfg/DynReconf1.cfg
#cfg/DynReconf2.cfg
# )
###################################
## catkin specific configuration ##
###################################
## The catkin_package macro generates cmake config files for your package
## Declare things to be passed to dependent projects
## INCLUDE_DIRS: uncomment this if your package contains header files
## LIBRARIES: libraries you create in this project that dependent projects also need
## CATKIN_DEPENDS: catkin_packages dependent projects also need
## DEPENDS: system dependencies of this project that dependent projects also need
catkin_package(
# INCLUDE_DIRS include
# LIBRARIES opencv
#CATKIN_DEPENDS roscpp rospy std_msgs
# DEPENDS system_lib
)
###########
## Build ##
###########
## Specify additional locations of header files
## Your package locations should be listed before other locations
include_directories(
#include
${catkin_INCLUDE_DIRS}
)
## Declare a C++ library
# add_library(${PROJECT_NAME}
# src/${PROJECT_NAME}/opencv.cpp
# )
## Add cmake target dependencies of the library
## as an example, code may need to be generated before libraries
## either from message generation or dynamic reconfigure
# add_dependencies(${PROJECT_NAME} ${${PROJECT_NAME}_EXPORTED_TARGETS} ${catkin_EXPORTED_TARGETS})
## Declare a C++ executable
## With catkin_make all packages are built within a single CMake context
## The recommended prefix ensures that target names across packages don't collide
add_executable(open_camera src/open_camera.cpp)
add_executable(QR_ROS src/QR_ROS.cpp)
## Rename C++ executable without prefix
## The above recommended prefix causes long target names, the following renames the
## target back to the shorter version for ease of user use
## e.g. "rosrun someones_pkg node" instead of "rosrun someones_pkg someones_pkg_node"
# set_target_properties(${PROJECT_NAME}_node PROPERTIES OUTPUT_NAME node PREFIX "")
## Add cmake target dependencies of the executable
## same as for the library above
# add_dependencies(${PROJECT_NAME}_node ${${PROJECT_NAME}_EXPORTED_TARGETS} ${catkin_EXPORTED_TARGETS})
## Specify libraries to link a library or executable target against
target_link_libraries(open_camera
${OpenCV_LIBS}
)
target_link_libraries(open_camera
${catkin_LIBRARIES}
)
target_link_libraries(QR_ROS
${OpenCV_LIBS}
${ZBAR_LIBRARIES}
)
target_link_libraries(QR_ROS
${catkin_LIBRARIES} # Without this, an error will be reported related to ros code
)
#############
## Install ##
#############
# all install targets should use catkin DESTINATION variables
# See http://ros.org/doc/api/catkin/html/adv_user_guide/variables.html
## Mark executable scripts (Python etc.) for installation
## in contrast to setup.py, you can choose the destination
# catkin_install_python(PROGRAMS
# scripts/my_python_script
# DESTINATION ${CATKIN_PACKAGE_BIN_DESTINATION}
# )
## Mark executables for installation
## See http://docs.ros.org/melodic/api/catkin/html/howto/format1/building_executables.html
# install(TARGETS ${PROJECT_NAME}_node
# RUNTIME DESTINATION ${CATKIN_PACKAGE_BIN_DESTINATION}
# )
## Mark libraries for installation
## See http://docs.ros.org/melodic/api/catkin/html/howto/format1/building_libraries.html
# install(TARGETS ${PROJECT_NAME}
# ARCHIVE DESTINATION ${CATKIN_PACKAGE_LIB_DESTINATION}
# LIBRARY DESTINATION ${CATKIN_PACKAGE_LIB_DESTINATION}
# RUNTIME DESTINATION ${CATKIN_GLOBAL_BIN_DESTINATION}
# )
## Mark cpp header files for installation
# install(DIRECTORY include/${PROJECT_NAME}/
# DESTINATION ${CATKIN_PACKAGE_INCLUDE_DESTINATION}
# FILES_MATCHING PATTERN "*.h"
# PATTERN ".svn" EXCLUDE
# )
## Mark other files for installation (e.g. launch and bag files, etc.)
# install(FILES
# # myfile1
# # myfile2
# DESTINATION ${CATKIN_PACKAGE_SHARE_DESTINATION}
# )
#############
## Testing ##
#############
## Add gtest based cpp test target and link libraries
# catkin_add_gtest(${PROJECT_NAME}-test test/test_opencv.cpp)
# if(TARGET ${PROJECT_NAME}-test)
# target_link_libraries(${PROJECT_NAME}-test ${PROJECT_NAME})
#endif()
## Add folders to be run by python nosetests
#catkin_add_nosetests(test)
In fact, these are the places that need to be added:
1. Set the absolute path of the FindZBar.cmake package. Here is my own path. Everyone needs to change it. This is to find the ZBAR package when compiling
set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} "/home/qly/VS_Project/Ros_Debug/src/opencv/cmake") #Write the absolute path of FindZBar
2. Find the packages of opencv and zbar
find_package(OpenCV REQUIRED)
#set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} "${CMAKE_SOURCE_DIR}/cmake")
# Find the ZBar library in CMake
find_package(ZBar REQUIRED)
3. Add executable file
add_executable(QR_ROS src/QR_ROS.cpp)
4. Explain the libraries to be linked. The first one is to link Zbar and opencv. The second one should be to link libraries related to ros. Without this ros code, an error will be reported.
target_link_libraries(QR_ROS
${OpenCV_LIBS}
${ZBAR_LIBRARIES}
)
target_link_libraries(QR_ROS
${catkin_LIBRARIES} # Without this, an error will be reported related to ros code
)
Then compile and run it.