Use FastDFS and Nginx for port mapping to achieve remote access to local file servers

Article directory

  • Preface
  • 1. Build the FastDFS file system locally
    • 1.1 Environment installation
    • 1.2 Install libfastcommon
    • 1.3 Install FastDFS
    • 1.4 Configure Tracker
    • 1.5 Configure Storage
    • 1.6 Test upload and download
    • 1.7 Integration with Nginx
    • 1.8 Install Nginx
    • 1.9 Configure Nginx
  • 2. LAN test access to FastDFS
  • 3. Install cpolar intranet penetration
  • 4. Configure public network access address
  • 5. Fixed public network address
    • 5.1 Reserve second-level subdomain names
    • 5.2 Configure the second-level subdomain name
  • 6. Test access to fixed second-level subdomain names

Foreword

FastDFS is an open source lightweight distributed file system that manages files. Its functions include: file storage, file synchronization, file access (file upload, file download), etc., solving the problems of large-capacity storage and load balancing. It is especially suitable for online services based on files, such as photo album websites, video websites, etc.

FastDFS is tailor-made for the Internet, fully considering redundant backup, load balancing, linear expansion and other mechanisms, and focusing on high availability, high performance and other indicators. Using FastDFS, it is easy to build a high-performance file server cluster to provide file upload and download. Waiting for service.

Equipping a public network server for file upload and download requires a lot of resources and overhead. If you build a FastDFS service locally, you can only configure the uploaded file to be accessed through the Internet through the cpolar intranet port forwarding tool, which greatly reduces the cost. Server costs and expenses, this article introduces a detailed tutorial on setting up FastDFS, and combines Nginx + cpolar tools to achieve remote access to files.

1. Build FastDFS file system locally

1.1 Environment installation

Since fastDFS is written in c language, we need to install gcc

yum -y install gcc-c++

FastDFS depends on libevent library and needs to be installed. Execute the following command:

yum -y install libevent

1.2 Install libfastcommon

Enter the local directory

cd /usr/local

libfastcommon is officially provided by FastDFS. libfastcommon contains some basic libraries required for FastDFS to run. Click the link below to get it:

https://github.com/happyfish100/libfastcommon/releases/tag/V1.0.7

After obtaining it, upload it to the local directory

1679321808236

Unzip libfastcommon

tar -zxvf libfastcommonV1.0.7.tar.gz

Go to the unzipped folder

cd libfastcommon-1.0.7/

compile

./make.sh

Install

./make.sh install

After libfastcommon is installed, it will automatically copy the library files to /usr/lib64. Since the FastDFS program refers to the usr/lib directory, it is necessary to copy the library files under /usr/lib64 to /usr/lib. .

cp /usr/lib64/libfastcommon.so /usr/lib/

1.3 Install FastDFS

Enter the local directory

cd /usr/local

Click the following address to download the FastDFS_v5.05.tar.gz file:

https://sourceforge.net/projects/fastdfs/

After downloading the file, upload FastDFS_v5.05.tar.gz to /usr/local/

1679322527568

unzip files

tar -zxvf FastDFS_v5.05.tar.gz

Go to the unzipped folder

cd FastDFS

compile

./make.sh

Install

./make.sh install

After the installation is successful, copy the files under conf in the installation directory to /etc/fdfs/

cd conf/

copy

cp * /etc/fdfs/

1.4 Configuring Tracker

Next, configure the configuration. After successful installation, enter the /etc/fdfs directory.

cd /etc/fdfs/

Modify tracker.conf

vim tracker.conf

Change base_path=/home/yuqing/fastdfs to base_path=/home/fastdfs

1679321890447

Save and exit after editing, and then create a corresponding fastdfs folder

mkdir /home/fastdfs

then start

/usr/bin/fdfs_trackerd /etc/fdfs/tracker.conf restart

Set up startup

vim /etc/rc.d/rc.local

Add the run command line to the file

/usr/bin/fdfs_trackerd /etc/fdfs/tracker.conf restart

1679321943356

1.5 Configuring Storage

Enter the /etc/fdfs directory

cd /etc/fdfs

Modify storage.conf

vim storage.conf

Change base_path=/home/yuqing/fastdfs to base_path=/home/fastdfs

1679322281423

Change store_path0=/home/yuqing/fastdfs to store_path0=/home/fastdfs/fdfs_storage

1679322153239

Continue to modify, set tracker server: IP, tracker_server=Linux LAN ip: 22122, save and exit after setting.

1679322199689

Create the corresponding fdfs_storage folder

mkdir /home/fastdfs/fdfs_storage

then start

/usr/bin/fdfs_storaged /etc/fdfs/storage.conf restart

Set to start automatically at boot

vim /etc/rc.d/rc.local

Add the run command line to the file and save it

/usr/bin/fdfs_storaged /etc/fdfs/storage.conf restart

image-20230319225348678

1.6 Test upload and download

If FastDFS is installed successfully, you can use the /usr/bin/fdfs_test command to test upload, download and other operations.

Modify /etc/fdfs/client.conf

vim /etc/fdfs/client.conf

Modify the following content and save:

base_path=/home/fastdfs

tracker_server=Linux LAN ip:22122

1679322420123

Next, test uploading files. Here, a picture is uploaded to the /usr/local directory for testing.

1679322482853

FastDFS upload file usage format:

/usr/bin/fdfs_test /etc/fdfs/client.conf upload + path to upload file

Then according to the format, test uploading the test23.png image to FastDFS

/usr/bin/fdfs_test /etc/fdfs/client.conf upload /usr/local/test23.png

After execution, we can see the return address information, indicating success, but it is not accessible yet. It needs to be combined with Nginx for access and download.

1679322601788

1.7 Integration with Nginx

Click to download the link below to download fastdfs-nginx-module_v1.16:

[https://sourceforge.net/projects/fastdfs/files/FastDFS Nginx Module Source Code/fastdfs-nginx-module_v1.16.tar.gz/download](https://sourceforge.net/projects/fastdfs/files/ FastDFS Nginx Module Source Code/fastdfs-nginx-module_v1.16.tar.gz/download)

Enter the /usr/local/ folder

cd /usr/local/

Upload fastdfs-nginx-module_v1.16.tar.gz

1679322637070

Unzip fastdfs-nginx-module_v1.16.tar.gz

tar -zxvf fastdfs-nginx-module_v1.16.tar.gz

Enter src and edit the configuration file

cd fastdfs-nginx-module/src/

Modify config file

vim config

Modify these three locations, change the path that was originally /usr/local/ to /usr/, then save and exit.

1679322770876

Then copy mod_FastDFS.conf under FastDFS-nginx-module/src to /etc/fdfs/

cp mod_fastdfs.conf /etc/fdfs/

Modify the contents of mod_FastDFS.conf:

vim /etc/fdfs/mod_fastdfs.conf

The modifications are as follows:

base_path=/home/fastdfs

tracker_server=Linux LAN ip address:22122

url_have_group_name=true #The url contains the group name

store_path0=/home/fastdfs/fdfs_storage #Specify the file storage path

1679405651550

Next, copy libfdfsclient.so to /usr/lib

cp /usr/lib64/libfdfsclient.so /usr/lib/

Just create the nginx/client directory

mkdir -p /var/temp/nginx/client

1.8 Install Nginx

First install the series environment

yum -y install pcre pcre-devel

yum -y install zlib zlib-devel

yum -y install openssl openssl-devel

Then click to download the Nginx package:

nginx-1.16.1.tar.gz

Upload to the root directory

1679323002737

Unzip and enter the unzipped folder

tar -zxvf nginx-1.16.1.tar.gz & amp; & amp; cd nginx-1.16.

Add the FastDFS-nginx-module module and execute the following command:

./configure \
--prefix=/usr/local/nginx \
--pid-path=/var/run/nginx/nginx.pid \
--lock-path=/var/lock/nginx.lock \
--error-log-path=/var/log/nginx/error.log \
--http-log-path=/var/log/nginx/access.log \
--with-http_gzip_static_module \
--http-client-body-temp-path=/var/temp/nginx/client \
--http-proxy-temp-path=/var/temp/nginx/proxy \
--http-fastcgi-temp-path=/var/temp/nginx/fastcgi \
--http-uwsgi-temp-path=/var/temp/nginx/uwsgi \
--http-scgi-temp-path=/var/temp/nginx/scgi \
--add-module=/usr/local/fastdfs-nginx-module/src

1679320271108

then compile

make

Install

make install

1.9 Configuring Nginx

Modify Nginx configuration file

vim /usr/local/nginx/conf/nginx.conf

Add server, save and exit after adding:

server {<!-- -->
        listen 8089;#In order to prevent port 80, we changed it to 8089
        server_name 192.168.59.133;#You need to use your local area network IP here

        location /group1/M00/{<!-- -->
                ngx_fastdfs_module;
        }
}

1679406356346

Start Nginx

/usr/local/nginx/sbin/nginx

If you start nginx the following error occurs:

nginx: [emerg] open() “/var/run/nginx/nginx.pid” failed (2: No such file or directory)

You can execute the following command and then start again:

mkdir /var/run/nginx

Set to start automatically at boot.

vim /etc/rc.d/rc.local

Add the run command line to the file, then save and exit:

/usr/local/nginx/sbin/nginx

1679321237969

Turn off firewall

service iptables stop

Set the firewall not to start automatically

chkconfig iptables off

2. LAN test access to FastDFS

Then according to the format, test upload the test23.png image we uploaded above to FastDFS

/usr/bin/fdfs_test /etc/fdfs/client.conf upload /usr/local/test23.png

After execution, we can see the return address information, indicating success

1679323425917

Then use the above address to access it in the browser, you can see the pictures we uploaded, and fastDFS is built.

1679406429350

3. Install cpolar intranet penetration

Above we installed the fastDFS distributed file service on the local Linux virtual machine. Next we installed the cpolar intranet penetration tool. Through cpolar’s http public network address, we can easily access the files uploaded in fastDFS remotely without having to do it ourselves. Register a domain name and purchase a cloud server. The following are the steps to install cpolar intranet penetration

cpolar official website: https://www.cpolar.com/

  • Use one-click script installation command
curl -L https://www.cpolar.com/static/downloads/install-release-cpolar.sh | sudo bash
  • token authentication

Log in to the cpolar official website backend, click Verify on the left to view your authentication token, and then paste the token into the command line:

cpolar authtoken xxxxxxx

20230206171248

  • Add services to the system
sudo systemctl enable cpolar
  • Start cpolar service
sudo systemctl start cpolar

4. Configure public network access address

After successfully starting the cpolar service, we access the Linux LAN IP address + port 9200 on the browser and log in to the cpolar web UI management interface.

After successful login, click Tunnel Management – Create Tunnel on the left dashboard to create an http protocol tunnel pointing to the Nginx8089 port set above:

  • Tunnel name: It can be customized. Be careful not to duplicate the existing tunnel name.
  • Protocol: http
  • Local address: 8089
  • Domain name type: Choose a random domain name for free
  • Region: Select China VIP

Click Create

1679406828417

Then open the online tunnel list, view and copy the public network address

1679406894080

Then open the browser, enter the public network address, access the files we uploaded in fastDFS, and the remote access will be successful.

1679406972587

5. Fixed public network address

Since the tunnel just created uses a random temporary address, the address will change within 24 hours. For long-term and more stable remote access, we will configure this public network address as fixed.

5.1 Reserve second-level subdomain names

You need to upgrade to the basic package or above to support the configuration of second-level subdomain names. If you need to download files, you can upgrade to a higher package to obtain greater bandwidth and increase download speed.

Log in to the cpolar official website backend, click Reserve on the left dashboard, find Reserve second-level subdomain name, and reserve a second-level subdomain name for the http tunnel.

  • Region: Select server region
  • Name: Fill in the second-level subdomain name you want to reserve (can be customized)
  • Description: Notes, which can be customized

1679407261443

This example reserves a second-level subdomain named fasttest. After the subdomain name is successfully reserved, we copy the subdomain name and then configure it into the tunnel.

1679407306636

5.2 Configure second-level subdomain names

Log in to the cpolar web ui management interface. Click Tunnel Management – Tunnel List on the left dashboard, find the tunnel that needs to be configured with a second-level subdomain name, and click Edit on the right

1679407950343

Modify the tunnel information and configure the second-level subdomain name into the tunnel:

  • Domain name type: select Second-level subdomain name instead
  • Sub Domain: Fill in the second-level subdomain name we just reserved (in this case, fasttest)

After the modification is completed, click Update

1679407459593

After the tunnel is successfully updated, click Status – Online Tunnel List on the left dashboard. You can see that the public network address of the tunnel has been updated to a second-level subdomain name. Copy the public network address.

1679407508347

6. Test access to fixed second-level subdomain name

Let’s test accessing the successfully configured second-level subdomain name, open the browser, enter the public network address and resource path, access the file we uploaded in fastDFS, and the remote access will be successful when the picture appears. Now, our only private second-level subdomain name in the entire network has been created.

And this address will no longer change randomly. It is fixed. As long as the tunnel is kept online, we can remotely access it anytime and anywhere through this public network address without the need for a public network IP or setting up a router.

1679407663302