Compile BVLC/Caffe under msys2@mingw

(Refer to https://github.com/lemonsqueeze/mingw-caffe, the attempts on this project are written at the end of the article)

1. Prepare development environment

Regarding the tools that need to be installed, the above article has made it very clear, so I will copy them below.

pacman -S --needed git make patch diffutils
pacman -S --needed \
mingw-w64-${MSYSTEM_CARCH}-cmake \
mingw-w64-${MSYSTEM_CARCH}-python \
mingw-w64-${MSYSTEM_CARCH}-tools-git \
mingw-w64-${MSYSTEM_CARCH}-gcc \
mingw-w64-${MSYSTEM_CARCH}-boost \
mingw-w64-${MSYSTEM_CARCH}-protobuf-c \
mingw-w64-${MSYSTEM_CARCH}-gflags \
mingw-w64-${MSYSTEM_CARCH}-glog \
mingw-w64-${MSYSTEM_CARCH}-hdf5 \
mingw-w64-${MSYSTEM_CARCH}-openblas \
mingw-w64-${MSYSTEM_CARCH}-leveldb \
mingw-w64-${MSYSTEM_CARCH}-lmdb \
mingw-w64-${MSYSTEM_CARCH}-snappy \
mingw-w64-${MSYSTEM_CARCH}-python-matplotlib \
mingw-w64-${MSYSTEM_CARCH}-python-pytest \
mingw-w64-${MSYSTEM_CARCH}-python-scipy

Also download the caffe source code

wget https://github.com/BVLC/caffe/archive/refs/tags/1.0.tar.gz
tar xf 1.0.tar.gz
cd caffe

2. Compile script

#!/bin/bash
 
BUILD_DIR=${1:-build}
export PATH=/mingw64/bin:/mingw64/include:$PATH
 
[ -e ${BUILD_DIR} ] & amp; & rm -r ${BUILD_DIR}/* || mkdir ${BUILD_DIR}
 
cmake \
-B${BUILD_DIR} \
-DBLAS=open \
caffe-1.0
 
#Build
[ $? -eq 0 ] & amp; & amp; cmake --build ${BUILD_DIR} -j 8

3. Bug fixes

Error 1:

/home/rd/NN/caffe/src/caffe/layers/window_data_layer.cpp: In member function virtual void caffe::WindowDataLayer::load_batch(caffe::Batch*)’:
/home/rd/NN/caffe/src/caffe/layers/window_data_layer.cpp:293:42: error: CV_LOAD_IMAGE_COLOR’ was not declared in this scope
293 | cv_img = cv::imread(image.first, CV_LOAD_IMAGE_COLOR);
| ^~~~~~~~~~~~~~~~~~~
make[2]: *** [src/caffe/CMakeFiles/caffe.dir/build.make:1090: src/caffe/CMakeFiles/caffe.dir/layers/window_data_layer.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:400:src/caffe/CMakeFiles/caffe.dir/all] Error 2

Cause of the problem: codes for opencv2 or opencv3 are not compatiable to opencv4, so add below codes

Solution: Add patches in front of caffe-1.0/src/caffe/layers/window_data_layer.cpp and caffe-1.0/src/caffe/util/io.cpp files:

#if (CV_MAJOR_VERSION > 3)
#include "opencv2/imgcodecs/legacy/constants_c.h"
#endif

Error 2:

/home/rd/NN//caffe-1.0/src/caffe/util/db_lmdb.cpp:13:19: error: too many arguments to function ‘int mkdir(const char*)’
13 | CHECK_EQ(mkdir(source.c_str(), 0744), 0) << "mkdir " << source << " failed";

Reference: Function ‘int mkdir(const char*) has too many parameters

Solution: Add macro in front of caffe-1.0/src/caffe/layers/window_data_layer.cpp

#if (defined(_WIN32) || defined(__WIN32__))
#define mkdir(A, B) mkdir(A)
#endif

Error 3:

/home/rd/NN/caffe-1.0/src/caffe/util/io.cpp: In function bool caffe::ReadProtoFromBinaryFile(const char*, google::protobuf::Message*)’:
/home/rd/NN/caffe/src/caffe/util/io.cpp:60:66: error: no matching function for call to ‘google::protobuf::io::CodedInputStream::SetTotalBytesLimit(const int & amp ;, int)’
60 | coded_input->SetTotalBytesLimit(kProtoReadBytesLimit, 536870912);

Cause of the problem : refer to https://github.com/onnx/onnx/issues/2678

Solution: Modify the corresponding code of caffe-1.0/src/caffe/util/io.cpp

#if GOOGLE_PROTOBUF_VERSION >= 3002000
    coded_input->SetTotalBytesLimit(kProtoReadBytesLimit);
#else
    coded_input->SetTotalBytesLimit(kProtoReadBytesLimit, 536870912);
#endif

Error 4:

D:/work/caffe-1.0/src/caffe/util/signal_handler.cpp:31:22: error: aggregate ‘{anonymous}::HookupHandler()::sigaction sa’ has incomplete type and cannot be defined
31 | struct sigaction sa;
| ^~
D:/work/caffe-1.0/src/caffe/util/signal_handler.cpp:35:19: error: ‘SA_RESTART’ was not declared in this scope
35 | sa.sa_flags = SA_RESTART;
| ^~~~~~~~~~
D:/work/caffe-1.0/src/caffe/util/signal_handler.cpp:37:5: error: ‘sigfillset’ was not declared in this scope
37 | sigfillset( & amp;sa.sa_mask);
| ^~~~~~~~~~
D:/work/caffe-1.0/src/caffe/util/signal_handler.cpp:39:19: error: ‘SIGHUP’ was not declared in this scope
39 | if (sigaction(SIGHUP, & amp;sa, NULL) == -1) {
| ^~~~~~
D:/work/caffe-1.0/src/caffe/util/signal_handler.cpp:39:36: error: invalid use of incomplete type ‘struct {anonymous}::HookupHandler()::sigaction’
39 | if (sigaction(SIGHUP, & amp;sa, NULL) == -1) {
| ^
D:/work/caffe-1.0/src/caffe/util/signal_handler.cpp:31:12: note: forward declaration of ‘struct {anonymous}::HookupHandler()::sigaction’
31 | struct sigaction sa;
| ^~~~~~~~~
D:/work/caffe-1.0/src/caffe/util/signal_handler.cpp:42:36: error: invalid use of incomplete type ‘struct {anonymous}::HookupHandler()::sigaction’
42 | if (sigaction(SIGINT, & amp;sa, NULL) == -1) {
| ^
D:/work/caffe-1.0/src/caffe/util/signal_handler.cpp:31:12: note: forward declaration of ‘struct {anonymous}::HookupHandler()::sigaction’
31 | struct sigaction sa;

Reference: c++ – Sigaction and porting Linux code to Windows – Stack Overflow

Cause of the problem: sigaction is part of the UNIX signal API. Windows only provides signal and does not support SIGHUP or any flags (such as SA_RESTART). However, very basic support is still there, so if you just use signal (instead of sigaction), the code should still work fine.

Solution: Refer to the code in the link

Error 5:

/usr/local/lib/libgflags.a(gflags.cc.o): relocation R_X86_64_PC32 against symbol `stderr@@GLIBC_2.2.5′ can not be used when making a shared object;

Cause of the problem: libgflags library is not built as shared. You may recompile it with below commands

Reference https://github.com/BVLC/caffe/issues/2171, 2016-08-26/2022-06-21

solution :

cd build/
cmake .. -DBUILD_SHARED_LIBS=ON
make
sudo make install

Error 6:

[106/138] Linking CXX shared library bin\libcaffe.dll
FAILED: bin/libcaffe.dll lib/libcaffe.dll.a
cmd.exe /C “cd . & amp; & amp; D:\msys64\mingw64\bin\c + + .exe -Wno-sign-compare -Wno-uninitialized -O3 -DNDEBUG -shared – o bin\libcaffe.dll -Wl,–out-implib,lib\libcaffe.dll.a -Wl,–major-image-version,1,–minor-image-v
ersion,0 @CMakeFiles\caffe.rsp & amp; & amp; cd .”
D:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/13.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: src/caffe/CMakeFiles/caffe.dir/blob.cpp.obj:blob.cpp:(.rdata$.refptr.__emutls_v._ZN6google8protobuf8interna
l15ThreadSafeArena13thread_cache_E[.refptr.__emutls_v._ZN6google8protobuf8internal15ThreadSafeArena13thread_cache_E] + 0x0): undefined reference to `__emutls_v._ZN6google8protobuf8internal15ThreadSafeArena13thre
ad_cache_E’
collect2.exe: error: ld returned 1 exit status
[113/138] Building CXX object tools/CMakeFiles/train_net.dir/train_net.cpp.ob

Reproducible by manually executing link instructions in the build directory

 c++.exe -Wno-sign-compare -Wno-uninitialized -O3 -DNDEBUG -shared -o bin/libcaffe.dll -Wl,--out-implib,lib/libcaffe.dll.a -Wl, --major-image-version,1,--minor-image-version,0 @CMakeFiles/caffe.rsp 

4. Try https://github.com/lemonsqueeze/mingw-caffe

This project is as old as caffe, so it makes sense that its boost_headers and other patches cannot be patched.

You can notice the software version mentioned in Release caffe-cpu 20180127 · lemonsqueeze/mingw-caffe.

mingw-w64-*-boost 1.66.0-1
mingw-w64-*-gflags 2.2.1-1
mingw-w64-*-glog 0.3.5-1
mingw-w64-*-hdf5 1.8.20-1
mingw-w64-*-openblas 0.2.20-1
mingw-w64-*-protobuf 3.5.0-1
mingw-w64-*-protobuf-c 1.3.0-1

Rewrite scripts/msys_build.sh, the original is not very concise.

#!/bin/sh
# msys build script for appveyor

die() { echo "$@"; exit 1; }

# Run in top-level directory
cd `dirname "$0"`/..

# Show arch
echo "MINGW_INSTALLS: $MINGW_INSTALLS"
echo ""

# Update pacman db and packages
pacman --noconfirm -Syu
pacman --noconfirm -Su
echo ""

pacman -S --noconfirm --needed \
${MINGW_PACKAGE_PREFIX}-boost \
${MINGW_PACKAGE_PREFIX}-protobuf-c \
${MINGW_PACKAGE_PREFIX}-gflags \
${MINGW_PACKAGE_PREFIX}-glog \
${MINGW_PACKAGE_PREFIX}-hdf5 \
${MINGW_PACKAGE_PREFIX}-openblas \
git make patch diffutils

echo "Using packages:"
pacman -Q | grep 'boost\|protobuf\|gflags\|glog\|hdf5\|openblas\|-gcc '
echo ""

# Fix bad header in mingw-w64-i686-boost-1.66.0-1
if [ -f /mingw32/include/boost/winapi/basic_types.hpp ]; then
    echo "Patching boost headers for ming32 ..."
    patch -d/ -p0 < boost_header_fix_mingw32.patch
    echo ""
fi

if [ -f /mingw64/include/boost/winapi/basic_types.hpp ]; then
    echo "Patching boost headers for mingw64 ..."
    patch -d/ -p0 < boost_header_fix_mingw64.patch
    echo ""
fi

#Build
cd mingw-w64-caffe
makepkg-mingw

#Copy built package
mkdir -p ../build
cp *.pkg.tar.xz ../build/

The knowledge points of the article match the official knowledge files, and you can further learn relevant knowledge. Python entry skill treeBasic skillsOperating system and environment 366,186 people are learning the system