Tucker Robot runs ORB_SLAM3 on Ubuntu18.04 (including running the Euroc data set and obtaining front-end data in real time through monocular and RGB-D camera modes)

I looked at visual slam related directions during the front-end time. There happened to be a small car in the laboratory, so I tried to run ORB-SLAM3 on it. The following is a summary of the operation steps and problems encountered. If you need to run ORB_SLAM3 directly on Ubuntu 18.04, you can ignore the content of the first section.

1. Install NoMachine software

1. Install NoMachine software on the PC to conveniently operate the robot system. After the installation is completed, open the NoMachine software and the following interface will appear. Click OK.
   
   2. The NoMachine software automatically searches for robot systems under the same LAN, and it will take some time. When the icon appears, double-click it to open it. (Note: The PC and the robot must be in the same LAN. I am using a mobile phone hotspot)
   
   If an error occurs:
   
   The error message means that the robot system with the IP address 192.168.55.1 cannot be connected to the port 4000.
   
   Check the IP address of the robot system, open a terminal in the robot system, enter ifconfig, the following information will appear:
   
   Therefore, the IP address of the robot system is 192.168.43.64, which should be modified on the host, as shown in the figure:
   
   3. Enter your username and password and click login.
   
   4. The connection is successful.
   

2. ORB_SLAM3 related environment installation

1. Check C++ version

First go to github and read the readme:

Verify C++ version, must show support for C++11 features
(1) Open the terminal and create a new main.cpp

touch main.cpp
vim main.cpp

(2) Compile main.cpp

g + + -std=c + + 11 -o main main.cpp

(3) Check whether the compilation is successful

echo $?

The meaning of this command is: return the execution result of the previous command. If it is successful, it will return 0. If it is unsuccessful, it will return an integer other than 0. You can also print out the results of main.cpp here to advise whether the compilation is successful. Enter ./main

(4) Check gcc version

gcc -v

2. Install Pangolin

(1) Download the Pangolin package on github locally

git clone https://github.com/stevenlovegrove/Pangolin.git

(2) After decompressing the Pangolin package, open a terminal and enter the command to enter the Pangolin file.

cd Pangolin

(3) Use mkdir to create a build folder

mkdir build

(4) Enter the build file

cd build

(5) Link configuration file

cmake ..

(6) Call build commands related to the build platform

cmake --build

(7) Installation

sudo make install

(8) 8. Check whether the installation is successful. Entered the build/examples/HelloPangolin folder and ran a HelloPangolin example. If a cube appears, the installation is successful!

cd build/examples/HelloPangolin
./HelloPangolin

3. Install Eigen3.5

(1) Download Eigen3.3.5. ORB_SLAM’s github says that it is at least 3.1.0. However, after my installation test, 3.1.0 cannot meet our needs. When starting with ROS later, an error will be reported when compiling 3.1.0. Take 3.3.5. In order to reduce the trouble of uninstalling and installing later, we use 3.3.5 directly. Eigen3.3.5’s gitlab address: Eigen3.3.5
(2) Put the compressed package into our workspace to decompress it, then enter the eigen-3.3.5 folder, open the terminal, and create a new build file mkdir build
(3) Enter the build file cd build
(4) Link setting file cmake ..
(5) Install sudo make install

(6) Check whether the installation is successful. Open a new terminal and run the following command locate eigen3 to view the location of eigen3.

4. Install OpenCV

Because the Tucker robot has its own ROS system, and the ROS system comes with opencv3.3.1, which can be queried through the command pkg-config --modversion opencv, there is no need to install it separately here. If you need it, you can refer to OpenCV installation

5. Install DBoW2 and g2o

It is included in the source code, no need to download, it will be compiled later when compiling ORB_SLAM3

6. Install Python

Run the following command to install: sudo apt install libpython2.7-dev
There is a problem that cannot be downloaded:

Solution: Run the command sudo apt-get update and then run sudo apt install libpython2.7-dev to install successfully.

3. Compile ORB_SLAM3

(1) In the workspace, open the terminal and obtain the source code of ORB_SLAM3

git clone https://github.com/UZ-SLAMLab/ORB_SLAM3.git ORB_SLAM3

A timeout error may occur, which is caused by an error in the agent. My solution is:

git clone http://github.com/UZ-SLAMLab/ORB_SLAM3.git ORB_SLAM3

download successful

(2) After the download is successful, enter the ORB_SLAM3 folder cd ORB_SLAM3
(3) Grant allowable permissions to build.sh chmod + x build.sh
(4) 4. The method of many bloggers on the Internet to get to this step is to directly enter the ./build.sh command to directly run the build.sh script. Running it directly will lead to compilation errors without knowing the cause of the problem. View the build.sh script

We will find that this script directly compiles 5 documents and compiles 5 documents continuously. It is easy to not know which document the error is, so we compile it separately here.
(a) Open the terminal under the ORB_SLAM3 folder and compile Thirdparty/DBoW2

cd Thirdparty/DBoW2
mkdir build
cd build
cmake .. -DCMAKE_BUILD_TYPE=Release
make -j

(b) After successful compilation, compile g2o

cd ../../g2o
mkdir build
cd build
cmake .. -DCMAKE_BUILD_TYPE=Release
make -j

(c) After successful compilation, compile Sophus

cd ../../Sophus
mkdir build
cd build
cmake .. -DCMAKE_BUILD_TYPE=Release

! ! Note: make -j (don’t run this step for now, the explanation is as follows) needs to be changed to make -l

We can see that in the build.sh script, the last line is the make -j command. However, after entering make -j in the terminal, the robot system will freeze. This is caused by insufficient running memory. Then try the make -2 and make -j4 commands (make -j takes a parameter, which can compile the project in parallel. For example, make -j4 allows make to allow up to 4 compilation commands to be executed at the same time, which can effectively utilize the CPU). However, although these two commands can be compiled, the compilation speed is very slow. After a certain progress, the robot operating system will freeze again. After the robot is stuck, execute ctrl + C to exit the command. As shown below

Then try the make -l command (the compilation process still takes a long time, please be patient) and the compilation is successful! After consulting the information, we learned: the meaning of make with two parameters:
-j actually corresponds to –jobs, which indicates tasks that can be performed synchronously
-l actually corresponds to –load-average, which indicates the number of tasks loaded by the system

(d) After successful compilation, enter Vocabulary to decompress the file.

cd ../../../
cd Vocabulary
tar -xf ORBvoc.txt.tar.gz
cd..

(e) Finally build ORB_SLAM3

mkdir build
cd build
cmake .. -DCMAKE_BUILD_TYPE=Release

If the following error occurs:

The error message is that an OpenCV configuration file that is compatible with version 4.4 cannot be found. A configuration file with version 4.1.1 is found, but it cannot be used. As shown, we now need to modify the location of the above picture in CMakeList.txt under the ORB_SLAM3 folder.
original:

find_package(OpenCV 4.4) was changed to find_package(OpenCV4.1.1)
LIST(APPENDCMAKE_MODULE_PATH${PROJECT_SOURCE_DIR}/cmake_modules)
find_package(OpenCV 4.4)
if(NOT OpenCV_FOUND)
message(FATAL_ERROR “OpenCV > 4.4 not found.”)
endif()

Change to:

LIST(APPENDCMAKE_MODULE_PATH${PROJECT_SOURCE_DIR}/cmake_modules)
find_package(OpenCV 4.1.1)
if(NOT OpenCV_FOUND)
message(FATAL_ERROR “OpenCV > 4.4 not found.”)
endif()

After modification, click Save and run the cmake .. -DCMAKE_BUILD_TYPE=Release command again, success!

Change make -j4 to make -l. The principle is the same as the third document. If you run make -j4, the robot system will be directly stuck. Change it to make -l to compile. This process is relatively slow and takes about an hour. It requires Wait patiently. Don’t worry if the following prompt appears. This is a reminder that the system is running out of memory.


At this point, the ORB_SLAM3 installation is complete! ! !

4. ORB_SLAM3 runs public data sets

1. Download the data set from the official website. What is downloaded here is the Euroc data set Euroc data set download address in ASL format, as shown in the figure below.

2. Modification of path. Here you first need to create a new dataset folder under the ORB_SLAM3 folder, and extract the downloaded Euroc dataset into the dataset folder (my path here is like this, the path must not be wrong!!!)

3. Double-click to open the script “euroc_examples.sh” in the ORB_SLAM3 source code, but found the ORB_SLAM3 source code
This file does not exist in the file. It must have been deleted by the author. After checking the information, I found that the downloaded version 1.0 had “euroc_examples.sh” in the previous version 0.4. It can be found in other github repositories, address: version 0.4 ORB_SLAM3
As shown in the figure: This startup file contains a collection of startup file commands. We can select one of the command lines to run. Here I select euroc_examples.sh

Unzip the newly downloaded ORB_SLAM3 and copy euroc_examples.sh to the original ORB_SLAM3 workspace. Found the instruction containing MH01

Modified to: (note that the path is the same as the one I installed before)

./Examples/Monocular/mono_euroc ./Vocabulary/ORBvoc.txt ./Examples/Monocular/EuRoC.yaml ./dataset/MH01 ./Examples/Monocular/EuRoC_TimeStamps/MH01.txt

4. Open the terminal in ORB_SLAM3 and run the command. The data set runs successfully and ORB_SLAM3 is installed successfully!

./Examples/Monocular/mono_euroc ./Vocabulary/ORBvoc.txt ./Examples/Monocular/EuRoC.yaml ./dataset/MH01 ./Examples/Monocular/EuRoC_TimeStamps/MH01.txt

Although it runs successfully, the interface cannot appear because I installed the OpenCV1.0 version. Solution:
Find the mono_eurocc.cc file under ORB_SLAM3/Examples/Monocular. Change false in this line to true, ORB_SLAM3::System SLAM(argv[1],argv[2],ORB_SLAM3::System::MONOCULAR, false); Pangolin’s visual interface will appear.

After clicking save, you still need to recompile (note: files ending with .cc file must be recompiled after modification). Open a terminal in the ORB_SLAM3 file directory and enter

chmod +x build.sh
./build.sh

This compilation is different from the first time and will pass quickly! After the compilation is successful, run it again and the interface will appear!

./Examples/Monocular/mono_euroc ./Vocabulary/ORBvoc.txt ./Examples/Monocular/EuRoC.yaml ./dataset/MH01 ./Examples/Monocular/EuRoC_TimeStamps/MH01.txt

5. Run ORB_SLAM3 in monocular mode

1.1.Compilation of ROS part. According to the ROS part of github, first add environment variables in the ~/.bashrc environment. It is worth mentioning that due to the update of the orb_slam author, the ROS examples have been placed in the Examples_old location, so you should pay attention when compiling the file. We are no longer Go to Examples, but go to Examples_old. Add environment variables to ~/.bashrc as follows:

• Open ~/.bashrc sudo gedit ~/.bashrc
• Add the environment variable in the last line (must be added to the last line!!! If it is not placed in the last line of bashrc, the environment variable cannot be written!) where PATH should be replaced with your own workspace path.

export ROS_PACKAGE_PATH=${ROS_PACKAGE_PATH}:PATH/ORB_SLAM3/Examples_old/ROS

• Verify the ROS environment variables and enter echo $ROS_PACKAGE_PATH
  
  If the display is incorrect, enter the command source .bashrc in the terminal to refresh the file, and then run the command echo $ROS_PACKAGE_PATH again to display it correctly.
• Finally enter Examples_old/ORS/ORB_SLAM3 to compile

cd Examples_old/ROS/ORB_SLAM3
mkdir build
cd build
cmake .. -DROS_BUILD_TYPE=Release
make l (there is an error when running this command, the solution is below)

Error answer:
Error 1: cv::Mat Tcw = mpSLAM->TrackMonocular(cv_ptr->image,cv_ptr->header.stamp.toSec());
Find the ros_mono_ar.cc file under ORB_SLAM3/Examples_old/ROS/ORB_SLAM3/src/AR:
cv::Mat Tcw = mpSLAM->TrackMonocular(cv_ptr->image,cv_ptr->header.stamp.toSec());
Replace with:
cv::Mat Tcw;
Sophus::SE3f Tcw_SE3f = mpSLAM->
TrackMonocular(cv_ptr->image,cv_ptr->header.stamp.toSec());
Eigen::Matrix4f Tcw_Matrix = Tcw_SE3f.matrix();
cv::eigen2cv(Tcw_Matrix, Tcw);



Error 2: vPoints.push_back(pMP->GetWorldPos());
Finding the ViewerAR.cc file under ORB_SLAM3/Examples_old/ROS/ORB_SLAM3/src/AR will cause an error
vPoints.push_back(pMP->GetWorldPos()); changed to:
cv::Mat WorldPos;
cv::eigen2cv(pMP->GetWorldPos(), WorldPos);
vPoints.push_back(WorldPos);



**Error three:** ViewerAR.cc file also under ORB_SLAM3/Examples_old/ROS/ORB_SLAM3/src/AR
Change cv::Mat Xw = pMP->GetWorldPos(); to:
cv::Mat Xw;
cv::eigen2cv(pMP->GetWorldPos(), Xw);

Error 4: error: eigen2cv’ is not a member of cv’
Find CMakeLists.txt under the ORB_SLAM3/Examples_old/ROS/ORB_SLAM3 path and comment out the part about AR to solve the problem



After the error is resolved, run make -l again and the compilation is successful!

2. Open a terminal and run: roscore

3. Open another terminal to run, connect to the robot through SSH command, and run the camera node:

roslaunch xtark_ros_wrapper xtark_camera.launch

When the following interface appears, it means that the camera has been successfully started!

4. Open another terminal and enter rviz to open the rviz tool to view the camera interface.

Then follow the steps in the order in the picture. The following interface will appear, which means the camera has been started successfully! After seeing success, you can close the rviz tool.


5. Run the command to view all subscription topics: rostopic list

Among them/usb_cam/image_raw is the topic of the camera
6. Modify the code and find it in the /home/xtark/ORB_SLAM3/Examples_old/ROS/ORB_SLAM3/src directory: ros_mono.cc file, open it, find and modify it to the subscription /usb_cam/image_raw of our own camera


7. Return to the ORB_SLAM main directory, open the terminal, and compile the code you just modified ./build_ros.sh

8. Open another terminal and run (/home/xtark/note that the path here needs to be modified to your own file path)

Rosrun ORB_SLAM3 Mono /home/xtark/ORB_SLAM3/Vocabulary/ORBvoc.txt /home/xtark/ORB_SLAM3/Examples_old/ROS/ORB_SLAM3/Asus.yaml

Error 1: Preferred here[rosrun] Couldn’t find executable named Mono error (forgot to take a screenshot), the solution is in the /ORB_SLAM3/Examples_old/ROS/ORB_SLAM3 file Open the terminal and run:

chmod +x build_ros.sh
./build_ros.sh

After the operation is successful, the Mono file is generated under the /ORB_SLAM3/Examples_old/ROS/ORB_SLAM3 file!
**Mistake 2:This next error takes a long time to resolve! (I forgot to take a screenshot here) After running to the last step in the picture, the error Monocular Segmentation fault (core dumped)** appeared. The Chinese word is segmentation fault (core dumped), which I don’t understand.

After consulting the information on the Internet, it was said that gdb debugging is required (this method does not work and cannot solve this error). There are not many blogs on the Internet about how to solve this error. Finally, I found a related issue in an article on github (https://github.com/UZ-SLAMLab/ORB_SLAM3/issues/333)
As mentioned in the comments below this article: This may be caused by different versions of OpenCV used in ./build.sh and ./build_ros.sh. In ORB_SLAM3/Thirdparty/DBoW2/CMakeLists.txt and ORB_SLAM3/CMakeLists.txt, it requires opencv 4. But in ORB_SLAM3/Examples_old/ROS/ORB_SLAM3/CMakeLists.txt, it requires opencv 3. I have installed opencv4.1.0 outside and opencv3.3.1 in ros. Finally, it leads to segmentation errors.
Follow the prompts to query our OpenCV version. The ROS robot system has opencv3.3.1 installed. But when we installed ORB_SLAM3 and compiled build.sh, the error message given to us required version 4.1.1, so we changed the opencv version in the CMakeLists file to 4.1.1. The compilation can pass. However, when compiling ROS, you need to unify opencv to version 3.3.1. So the error we have here is caused by inconsistent OpenCV versions. Next, perform the following operations to unify the OpenCV versions:
(1) Change the opencv in the CMakeLists file in the /ORB_SLAM3 folder and the /ORB_SLAM3/Thirdparty/DBoW2 folder to the 3.3.1 version. At the same time, you also need to add a statement to set up the opencv file: set(OpenCV_DIR /home/xtark/Software/opemcv-3.3.1/build) This sentence must be added. I did not add it at the beginning. If you directly perform the rosrun operation after modifying the previous version number, the same error will be reported! ! !


(2) Then find the OpenCVConfig.cmake file in the path /usr/lib/aarch64-linux-gnu/cmake/opencv4, and change all the codes containing 4.1.1 to 3.3.1


(3) Finally, return to the ORB_SLAM3 folder and recompile.

Chmod + x build.sh
./build.sh

Chmod + x build_ros.sh
/build_ros.sh

roslaunch xtark_ctl xtark_keyboard.launch

roslaunch astra_camera astrapro.launch

sudo chmod 777 /dev/bus/usb/001/008

rosrun ORB_SLAM3 RGBD /home/xtark/ORB_SLAM3/Vocabulary/ORBvoc.txt /home/xtark/ORB_SLAM3/Examples_old/ROS/ORB_SLAM3/Asus.yaml