C++ project – cloud backup – ③ – Design and implementation of server-side utility tools

Article directory

• Column introduction
• 1. Design of file utility class
• 2. Implementation of file utility class
• • 2.1 Prerequisite knowledge supplement
  • • 2.1.1Introduction to struct stat and stat
    • 2.1.2std::experimental::filesystem understanding
  • 2.2FileUtil implementation
• 3. Design of JSON utility class
• 4. Implementation of JSON utility class
• 5. Organizing practical tools

Introduction to the column

About the author: Hua Xiangyun, undergraduate student One, a rising star creator in the C/C++ field, a mentor in the Rising Star Program, an expert blogger at Alibaba Cloud, and a CSDN content partner… dedicated to learning C/C++ and Linux.

Introduction to the column: This article is included in the C + + project – cloud backup

Related column recommendations: C Language Beginner Series, C Language Advanced Series, C++ Series, Data Structures and Algorithms, Linux
Project Gitee link: https://gitee.com/li-yuanjiu/cloud-backup

1. Design of file utility class

Regardless of whether it is the client or the server, file transfer and backup involves reading and writing files, including the persistence of data management information. Therefore, first design an encapsulated file operation class. After this class is encapsulated, it can be processed in any module. File operations will become simpler.

The file utility class FileUtil mainly includes the following members:

class FileUtil
{<!-- -->
public:
    FileUtil(const std::string & amp;filename) :_filename(filename);
    int64_t FileSize(); // Get the file size
    time_t LastMtime(); // Get the last modification time of the file
    time_t LastATime(); // Get the last access time of the file
    std::string FileName(); // Get the pure file name in the file path a/b/c/test.cc --> test.cc
    bool GetPosLen(std::string *body, size_t pos, size_t len); // Get the data in the specified range of the file
    bool GetContent(std::string *body); // Get file content
    bool SetContent(const std::string & amp;body); // Write data to the file
    bool Compress(const std::string & amp;packname); // Compressed file
    bool UnCompress(const std::string & amp;filename); // Decompress the file
    bool Exists(); // Determine whether the file exists
    bool Remove(); // Delete file
    bool CreateDirectory(); // Create file directory
    bool ScanDirectory(std::vector<std::string> *array); // Browse all files in the specified directory
private:
    std::string _filename; // file name--including path
};

• _filename: file name (including path), such as a/b/c/test.cc;
• FileUtil: constructor;
• FileSize: Get the file size;
• LastMtime: Get the last modification time of the file;
• LastATime: Get the last access time of the file;
• FileName: Get the pure file name in the file path a/b/c/test.cc --> test.cc;
• GetPosLen: Get the data in the specified interval of the file;
• GetContent: Get file content;
• SetContent: Write data to the file;
• Compress: compressed files;
• UnCompress: Uncompress the file;
• Exists: Determine whether the file exists;
• Remove: delete files;
• CreateDirectory: Create a file directory;
• ScanDirectory: Browse all files in the specified directory;

2. Implementation of file utility class

2.1 Prerequisite knowledge supplement

In the implementation of FileUtil, we use a lot of relatively uncommon library functions. Let’s review them first.

2.1.1Introduction to struct stat and stat

Before getting to know the stat function, we first get to know the struct stat type.

In C language, struct stat is a structure type used to represent attributes of files or file system objects. This structure is usually used for operations related to files and directories, such as obtaining file size, access permissions, last modification time and other information. The definition of the struct stat type is usually provided by the operating system, so its specific fields may vary depending on the operating system.

The following are the fields of a typical struct stat structure, although the specific fields may vary depending on the operating system:

struct stat {<!-- -->
    dev_t st_dev; // ID of the device where the file is located
    ino_t st_ino; //The inode number of the file
    mode_t st_mode; //File access permissions and types
    nlink_t st_nlink; //The number of hard links to the file
    uid_t st_uid; // User ID of the file owner
    gid_t st_gid; // Group ID of the file owner
    off_t st_size; // Size of the file (in bytes)
    time_t st_atime; //The last access time of the file
    time_t st_mtime; //The last modification time of the file
    time_t st_ctime; //The last status change time of the file
    blksize_t st_blksize; // Optimal block size for file system I/O operations
    blkcnt_t st_blocks; //The number of blocks occupied by the file
};

These fields in the struct stat structure provide various information about the file or directory. Different operating systems may provide additional fields, or the meaning of the fields may vary.

The stat function is used to get the attributes of the file or directory associated with the specified path name and fill these attributes into a struct stat structure middle. The following is the function prototype of the stat function:

int stat(const char *pathname, struct stat *statbuf);

• pathname is the path name of the file or directory whose attributes are to be obtained;
• statbuf is a pointer to the struct stat structure, used to store the obtained attribute information;
• The stat function returns an integer value. If the operation is successful, it returns 0; if an error occurs, it returns -1 and sets errnoglobal variable to indicate the type of error.

Note that the header files and need to be included to use the stat function.

2.1.2std::experimental::filesystem understanding

The std::experimental::filesystem library is part of the C++ standard library, first appeared in C++ 17 and is considered experimental File system library. It provides a set of classes and functions for handling file system operations, such as creation, access, traversal, copying, deletion of files and directories, etc. The purpose of this library is to make file system operations more convenient and cross-platform, so you can perform the same file operations on different operating systems without having to worry about the underlying details.

Here are some of the main features and functionality of the std::experimental::filesystem library:

• std::experimental::filesystem::path class: used to represent file paths. It can automatically handle the path separators of different operating systems, making the code more portable.

• File and directory operations: This library provides many functions to perform common file and directory operations, including file creation, copying, moving, deleting, and directory creation, deletion, traversal, etc.

• File attribute query: You can use this library to query the attributes of files and directories, such as file size, modification time, etc.

• Exception handling: The std::experimental::filesystem library defines some exception classes to handle errors related to file system operations, such as files that do not exist or are inaccessible. .

• Iterator: You can use iterators to traverse files and subdirectories in a directory. This is a very convenient feature for recursively traversing the file system.

It is important to note that although std::experimental::filesystem was introduced in C++17, it is an experimental feature and is not necessarily fully implemented on all compilers and platforms. support. Therefore, some compilers may require specific compilation options or configurations to use this library.

Starting with C++17, the filesystem library has officially become part of the C++ standard and moved into the std::filesystem namespace, and is no longer an experimental feature. Therefore, in the new C++ standard, it is recommended to use the std::filesystem library to perform file system operations.

2.2FileUtil implementation

namespace fs = std::experimental::filesystem;
class FileUtil
{<!-- -->
public:
    FileUtil(const std::string & amp;filename) :_filename(filename)
    {<!-- -->}
    int64_t FileSize()
    {<!-- -->
        struct stat st;
        if(stat(_filename.c_str(), & amp;st) < 0)
        {<!-- -->
            std::cout << "get file size failed!" << std::endl;
            return -1;
        }
        return st.st_size;
    }
    time_t LastMtime()
    {<!-- -->
        struct stat st;
        if(stat(_filename.c_str(), & amp;st) < 0)
        {<!-- -->
            std::cout << "get last modify time failed!" << std::endl;
            return -1;
        }
        return st.st_mtime;
    }
    time_t LastATime()
    {<!-- -->
        struct stat st;
        if(stat(_filename.c_str(), & amp;st) < 0)
        {<!-- -->
            std::cout << "get last access time failed!" << std::endl;
            return -1;
        }
        return st.st_atime;
    }
    std::string FileName()
    {<!-- -->
        size_t pos = _filename.find_last_of("/");
        if(pos == std::string::npos)
        {<!-- -->
            return _filename;
        }
        return _filename.substr(pos + 1);
    }
    bool GetPosLen(std::string *body, size_t pos, size_t len)
    {<!-- -->
        size_t fsize = FileSize();
        if(pos + len > fsize)
        {<!-- -->
            std::cout << "GetPosLen: get file len error" << std::endl;
            return false;
        }
        std::ifstream ifs;
        ifs.open(_filename, std::ios::binary);
        if(ifs.is_open() == false)
        {<!-- -->
            std::cout << "GetPosLen: open file failed!" << std::endl;
            return false;
        }
        ifs.seekg(pos, std::ios::beg); // Position the file pointer to pos
        body->resize(len);
        ifs.read( & amp;(*body)[0], len);
        if(ifs.good() == false)
        {<!-- -->
            std::cout << "GetPosLen: get file content failed" << std::endl;
            ifs.close();
            return false;
        }
        ifs.close();
        return true;
    }
    bool GetContent(std::string *body)
    {<!-- -->
        size_t fsize = FileSize();
        return GetPosLen(body, 0, fsize);
    }
    bool SetContent(const std::string & amp;body)
    {<!-- -->
        std::ofstream ofs;
        ofs.open(_filename, std::ios::binary);
        if(ofs.is_open() == false)
        {<!-- -->
            std::cout << "SetContent: write open file failed" << std::endl;
            return false;
        }
        ofs.write( & amp;body[0], body.size());
        if(ofs.good() == false)
        {<!-- -->
            std::cout << "SetContent: write open file failed" << std::endl;
            ofs.close();
            return false;
        }
        ofs.close();
        return true;
    }
    bool Compress(const std::string & amp;packname)
    {<!-- -->
        // 1. Get source file data
        std::string body;
        if(GetContent( & amp;body) == false)
        {<!-- -->
            std::cout << "compress get file content failed" << std::endl;
            return false;
        }
        // 2. Compress the data
        std::string packed = bundle::pack(bundle::LZIP, body);
        // 3. Store the compressed data in the compressed package file
        FileUtil fu(packname);
        if(fu.SetContent(packed) == false)
        {<!-- -->
            std::cout << "compress write packed data failed!" << std::endl;
            return false;
        }
        return true;
    }
    bool UnCompress(const std::string &filename)
    {<!-- -->
        // 1. Read the current compressed package data
        std::string body;
        if(GetContent( & amp;body) == false)
        {<!-- -->
            std::cout << "Uncompress get file content failed!" << std::endl;
            return false;
        }
        // 2. Decompress the compressed data
        std::string unpacked = bundle::unpack(body);
        // 3. Write the decompressed data to a new file
        FileUtil fu(filename);
        if(fu.SetContent(unpacked) == false)
        {<!-- -->
            std::cout << "Uncompress write packed data failed!" << std::endl;
            return false;
        }
        return true;
    }
    bool Exists()
    {<!-- -->
        return fs::exists(_filename);
    }
    bool Remove()
    {<!-- -->
        if(Exists() == false)
        {<!-- -->
            return true;
        }
        remove(_filename.c_str());
        return true;
    }
    bool CreateDirectory()
    {<!-- -->
        if(Exists()) return true;
        return fs::create_directories(_filename);
    }
    bool ScanDirectory(std::vector<std::string> *array)
    {<!-- -->
        for(auto & amp; p : fs::directory_iterator(_filename)) // The iterator traverses the files in the specified directory
        {<!-- -->
            if(fs::is_directory(p) == true) continue;
            // relative_path file name with path
            array->push_back(fs::path(p).relative_path().string());
        }
        return true;
    }
private:
    std::string _filename;
};

3. Design of JSON utility class

Jsoncpp has provided us with serialization and deserialization interfaces, but in order to make it more practical, we can encapsulate a JsonUtil class ourselves.

The JsonUtil class contains the following members:

class JsonUtil
{<!-- -->
public:
    static bool Serialize(const Json::Value & amp;root, std::string *str); // Serialization operation
    static bool Unserialize(const std::string & amp;str, Json::Value *root); // Deserialization operation
};

Since the use of Json has been introduced in the previous chapter, let’s look directly at the implementation of the function.

4.Implementation of JSON utility class

class JsonUtil
{<!-- -->
public:
    static bool Serialize(const Json::Value & amp;root, std::string *str)
    {<!-- -->
        Json::StreamWriterBuilder swb;
        std::unique_ptr<Json::StreamWriter> sw(swb.newStreamWriter());
        std::stringstream ss;
        if(sw->write(root, & amp;ss) != 0)
        {<!-- -->
            std::cout << "json write failed!" << std::endl;
            return false;
        }
        *str = ss.str();
        return true;
    }
    static bool Unserialize(const std::string & amp;str, Json::Value *root)
    {<!-- -->
        Json::CharReaderBuilder crb;
        std::unique_ptr<Json::CharReader> cr(crb.newCharReader());
        std::string err;
        bool ret = cr->parse(str.c_str(), str.c_str() + str.size(), root, & amp;err);
        if(ret == false)
        {<!-- -->
            std::cout << "parse error" << std::endl;
            return false;
        }
        return true;
    }
};

5. Organizing of practical tools

Finally, we put both utility classes into the header file of util.hpp.

// util.hpp
#ifndef __MY_UTIL__
#define __MY_UTIL__
/*
    1. Get the file size
    2. Get the last modification time of the file
    3. Get the last access time of the file
    4. Get the file name in the file path name /abc/test.txt -> test.txt
    5. Write data to the file
    6. Get file data
    7. Get the specified location of the file and specify the data length
    8. Determine whether the file exists
    9.Create a directory
    10. Browse and obtain all file path names in the directory
    11. Compress files
    12. Unzip all files
    13. Delete files
*/
#include <iostream>
#include <fstream>
#include <string>
#include <vector>
#include <sys/stat.h>
#include <experimental/filesystem>
#include <jsoncpp/json/json.h>
#include "bundle.h"

namespace cloud
{<!-- -->
    namespace fs = std::experimental::filesystem;
    class FileUtil
    {<!-- -->
    public:
        FileUtil(const std::string & amp;filename) :_filename(filename)
        {<!-- -->}
        int64_t FileSize()
        {<!-- -->
            struct stat st;
            if(stat(_filename.c_str(), & amp;st) < 0)
            {<!-- -->
                std::cout << "get file size failed!" << std::endl;
                return -1;
            }
            return st.st_size;
        }
        time_t LastMtime()
        {<!-- -->
            struct stat st;
            if(stat(_filename.c_str(), & amp;st) < 0)
            {<!-- -->
                std::cout << "get last modify time failed!" << std::endl;
                return -1;
            }
            return st.st_mtime;
        }
        time_t LastATime()
        {<!-- -->
            struct stat st;
            if(stat(_filename.c_str(), & amp;st) < 0)
            {<!-- -->
                std::cout << "get last access time failed!" << std::endl;
                return -1;
            }
            return st.st_atime;
        }
        std::string FileName()
        {<!-- -->
            size_t pos = _filename.find_last_of("/");
            if(pos == std::string::npos)
            {<!-- -->
                return _filename;
            }
            return _filename.substr(pos + 1);
        }
        bool GetPosLen(std::string *body, size_t pos, size_t len)
        {<!-- -->
            size_t fsize = FileSize();
            if(pos + len > fsize)
            {<!-- -->
                std::cout << "GetPosLen: get file len error" << std::endl;
                return false;
            }
            std::ifstream ifs;
            ifs.open(_filename, std::ios::binary);
            if(ifs.is_open() == false)
            {<!-- -->
                std::cout << "GetPosLen: open file failed!" << std::endl;
                return false;
            }
            ifs.seekg(pos, std::ios::beg); // Position the file pointer to pos
            body->resize(len);
            ifs.read( & amp;(*body)[0], len);
            if(ifs.good() == false)
            {<!-- -->
                std::cout << "GetPosLen: get file content failed" << std::endl;
                ifs.close();
                return false;
            }
            ifs.close();
            return true;
        }
        bool GetContent(std::string *body)
        {<!-- -->
            size_t fsize = FileSize();
            return GetPosLen(body, 0, fsize);
        }
        bool SetContent(const std::string & amp;body)
        {<!-- -->
            std::ofstream ofs;
            ofs.open(_filename, std::ios::binary);
            if(ofs.is_open() == false)
            {<!-- -->
                std::cout << "SetContent: write open file failed" << std::endl;
                return false;
            }
            ofs.write( & amp;body[0], body.size());
            if(ofs.good() == false)
            {<!-- -->
                std::cout << "SetContent: write open file failed" << std::endl;
                ofs.close();
                return false;
            }
            ofs.close();
            return true;
        }
        bool Compress(const std::string & amp;packname)
        {<!-- -->
            // 1. Get source file data
            std::string body;
            if(GetContent( & amp;body) == false)
            {<!-- -->
                std::cout << "compress get file content failed" << std::endl;
                return false;
            }
            // 2. Compress the data
            std::string packed = bundle::pack(bundle::LZIP, body);
            // 3. Store the compressed data in the compressed package file
            FileUtil fu(packname);
            if(fu.SetContent(packed) == false)
            {<!-- -->
                std::cout << "compress write packed data failed!" << std::endl;
                return false;
            }
            return true;
        }
        bool UnCompress(const std::string &filename)
        {<!-- -->
            // 1. Read the current compressed package data
            std::string body;
            if(GetContent( & amp;body) == false)
            {<!-- -->
                std::cout << "Uncompress get file content failed!" << std::endl;
                return false;
            }
            // 2. Decompress the compressed data
            std::string unpacked = bundle::unpack(body);
            // 3. Write the decompressed data to a new file
            FileUtil fu(filename);
            if(fu.SetContent(unpacked) == false)
            {<!-- -->
                std::cout << "Uncompress write packed data failed!" << std::endl;
                return false;
            }
            return true;
        }
        bool Exists()
        {<!-- -->
            return fs::exists(_filename);
        }
        bool Remove()
        {<!-- -->
            if(Exists() == false)
            {<!-- -->
                return true;
            }
            remove(_filename.c_str());
            return true;
        }
        bool CreateDirectory()
        {<!-- -->
            if(Exists()) return true;
            return fs::create_directories(_filename);
        }
        bool ScanDirectory(std::vector<std::string> *array)
        {<!-- -->
            for(auto & amp; p : fs::directory_iterator(_filename)) // The iterator traverses the files in the specified directory
            {<!-- -->
                if(fs::is_directory(p) == true) continue;
                // relative_path file name with path
                array->push_back(fs::path(p).relative_path().string());
            }
            return true;
        }
    private:
        std::string _filename;
    };
    classJsonUtil
    {<!-- -->
    public:
        static bool Serialize(const Json::Value & amp;root, std::string *str)
        {<!-- -->
            Json::StreamWriterBuilder swb;
            std::unique_ptr<Json::StreamWriter> sw(swb.newStreamWriter());
            std::stringstream ss;
            if(sw->write(root, & amp;ss) != 0)
            {<!-- -->
                std::cout << "json write failed!" << std::endl;
                return false;
            }
            *str = ss.str();
            return true;
        }
        static bool Unserialize(const std::string & amp;str, Json::Value *root)
        {<!-- -->
            Json::CharReaderBuilder crb;
            std::unique_ptr<Json::CharReader> cr(crb.newCharReader());
            std::string err;
            bool ret = cr->parse(str.c_str(), str.c_str() + str.size(), root, & amp;err);
            if(ret == false)
            {<!-- -->
                std::cout << "parse error" << std::endl;
                return false;
            }
            return true;
        }
    };
}
#endif