MinIO (3) Use Webhook to synchronize files in real time

Foreword

In the process of using MinIO, we encountered a requirement here. The system needs to preview 3D files and perform front-end operations. Use a tool to browse 3D files to open the directory of the 3D file to achieve the online preview function. At this time, the problem arises. After MinIO uploads the file Compiled, as shown below

At this time, if you want to read the file (such as xx\xx\bb.txt), it cannot be read. Therefore, when uploading the file, save the uploaded file in another folder and send it to the front end. Read using. At first I thought of putting it in the same folder, but testing found that although the folder name uploaded by MinIO is bb.txt, the saved file cannot be placed in the same directory as the bb.txt folder.

The first step is to develop a program to synchronize files

Create a new WebApi project. The configuration information is the same as the previous article. I will not go into details here. I will mainly talk about the differences.

“BucketDirectory” in the configuration: “D:\aaa\bbb\\ccc” refers to the directory where the file is to be saved.

Main code for saving files

using Microsoft.AspNetCore.Mvc;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.Logging;
using Minio;
using Newtonsoft.Json;
using System.IO;
using System.Linq;
using System.Threading.Tasks;

namespace MinIOTest.Controllers
{
    public class MinioCallbackController : ControllerBase
    {
        private static string _bucketName = string.Empty;//"demo";//Default bucket

        private readonly MinioClient _client;
        private readonly IConfiguration _configuration;
        private readonly ILogger<MinioCallbackController> _logger;

        public MinioCallbackController(MinioClient client,
            IConfiguration configuration,
            ILogger<MinioCallbackController> logger)
        {
            _client = client;
            _configuration = configuration;
            _bucketName = configuration["MinIO:Bucket"];
            _logger = logger;
        }

        [Route("/path")] //This cannot be changed casually, it must correspond to the path configured by the webhook
        [HttpPost]
        //public IActionResult Post([FromBody]Dictionary<string,object> args)
        public async Task<IActionResult> CallBack([FromBody] object args)
        {
            try
            {
                //_logger.LogInformation("aaaaaaaaaaa");

                string jsonStr = args.ToString();
                var jsonObj = JsonConvert.DeserializeObject<Root>(jsonStr);
                string bucketDirectory = _configuration["MinIO:BucketDirectory"];//File saving directory
                var pathAll = jsonObj.Key;//Get the file path
                var fileName = pathAll.Split('/').Last();

                //Add new
                if (jsonObj.EventName.Contains("Put"))
                {
                    if (!System.IO.File.Exists(bucketDirectory))
                    {
                        DirectoryInfo directoryInfo = new DirectoryInfo(bucketDirectory);
                        directoryInfo.Create();
                    }

                    var memoryStream = new MemoryStream();
                    var path = jsonObj.Key.Substring(pathAll.IndexOf('/') + 1, pathAll.Length - pathAll.IndexOf('/') - 1);
                    await _client.GetObjectAsync(_bucketName, path,
                                    (stream) =>
                                    {
                                        stream.CopyTo(memoryStream);
                                    });

                    using FileStream targetFileStream = new FileStream($"{bucketDirectory}\{fileName}", FileMode.OpenOrCreate);

                    memoryStream.WriteTo(targetFileStream);
                }
                else if (jsonObj.EventName.Contains("Delete"))
                {
                    var path = $"{bucketDirectory}\{fileName}";
                    System.IO.File.Delete(path);
                }


                return Ok();
            }
            catch (System.Exception ex)
            {
                _logger.LogError($"ERROR:{ex}");
            }

            return null;
        }

    }
}

Pay attention to routing [Route(“/path”)], which is a path that must be consistent with the path when configuring Webhook below.

Root class

This entity is complicated to write. In fact, only the first layer of structure is enough. Here, several layers are added to it. Let’s do it according to the actual situation.

using MinIOTest;
using System.Collections.Generic;

namespace MinIOTest
{

    public class ValueKind
    {

    }
    
    public class UserIdentity
    {
        /// <summary>
        ///
        /// </summary>
        public string principalId { get; set; }
    }

    public class @object
    {
        /// <summary>
        ///
        /// </summary>
        public string key { get; set; }
        /// <summary>
        ///
        /// </summary>
        public int size { get; set; }
        /// <summary>
        ///
        /// </summary>
        public string eTag { get; set; }
        /// <summary>
        ///
        /// </summary>
        public string contentType { get; set; }
        /// <summary>
        ///
        /// </summary>
        public object userMetadata { get; set; }
        /// <summary>
        ///
        /// </summary>
        public string sequencer { get; set; }
    }

    public class S3
    {
        /// <summary>
        ///
        /// </summary>
        public string s3SchemaVersion { get; set; }
        /// <summary>
        ///
        /// </summary>
        public string configurationId { get; set; }
        /// <summary>
        ///
        /// </summary>
        public object bucket { get; set; }
        /// <summary>
        ///
        /// </summary>
        public @object @object { get; set; }
    }

    public class Source
    {
        /// <summary>
        ///
        /// </summary>
        public string host { get; set; }
        /// <summary>
        ///
        /// </summary>
        public string port { get; set; }
        /// <summary>
        ///
        /// </summary>
        public string userAgent { get; set; }
    }

    public class RecordsItem
    {
        /// <summary>
        ///
        /// </summary>
        public string eventVersion { get; set; }
        /// <summary>
        ///
        /// </summary>
        public string eventSource { get; set; }
        /// <summary>
        ///
        /// </summary>
        public string awsRegion { get; set; }
        /// <summary>
        ///
        /// </summary>
        public string eventTime { get; set; }
        /// <summary>
        ///
        /// </summary>
        public string eventName { get; set; }
        /// <summary>
        ///
        /// </summary>
        public UserIdentity userIdentity { get; set; }
        /// <summary>
        ///
        /// </summary>
        public object requestParameters { get; set; }
        /// <summary>
        ///
        /// </summary>
        public object responseElements { get; set; }
        /// <summary>
        ///
        /// </summary>
        public S3 s3 { get; set; }
        /// <summary>
        ///
        /// </summary>
        public Source source { get; set; }
    }

    public class Root
    {
        /// <summary>
        ///
        /// </summary>
        public string EventName { get; set; }
        /// <summary>
        /// demo/engineering drawings/001/bb.txt
        /// </summary>
        public string Key { get; set; }
        /// <summary>
        ///
        /// </summary>
        public List<RecordsItem> Records { get; set; }
    }



}

Step 2 Configure Webhook

When reaching the next step, you need to open the service in advance. The service address is usually http://192.168.x.xx:5000. If the service is not opened, the next step of creation will fail, and there will usually be something like “Remote Service Rejected” Connect” prompt. To put it bluntly, the file synchronization program written above needs to be run.

After the creation is completed, you cannot see the newly created content. You need to restart the MinIO service. See the figure below for how to restart the service. Click Restart, or Stop-Start (Note: A problem was discovered during the deployment process. Some The computer needs to restart the service. Some computers can see the newly created Webhook without restarting the service. If you can see the result after the new creation is completed, then don’t restart the service.)