SpringBoot + minio realizes fragment upload, second transmission and resume transmission

Lao Pao said Java 2023-05-24 20:15 Published in Shanxi

Included in collection

#老炒说Java541

#springboot28

Old Cannon Talks about Java

Ten-year-old programmers take you to play with technology

No public

What is minio

MinIO is a Go-based high-performance object storage compatible with the S3 protocol. It adopts the GNU AGPL v3 open source protocol, and the project address is https://github.com/minio/minio.

To quote the official website:

MinIO is a high-performance object store released under the GNU Affero General Public License v3.0. It is compatible with Amazon S3 cloud storage service. Use MinIO to build high-performance infrastructure for machine learning, analytics, and application data workloads.

Official website address:

https://min.io/

Document address:

https://docs.min.io/

1. Use docker to build minio service

GNU/Linux and macOS

docker run -p 9000:9000 \
  --name minio1 \
  -v /mnt/data:/data\
  -e "MINIO_ROOT_USER=AKIAIOSFODNN7EXAMPLE" \
  -e "MINIO_ROOT_PASSWORD=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY" \
  minio/minio server /data

windows

docker run -p 9000:9000 \
  --name minio1 \
  -v D:\data:/data\
  -e "MINIO_ROOT_USER=AKIAIOSFODNN7EXAMPLE" \
  -e "MINIO_ROOT_PASSWORD=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY" \
  minio/minio server /data
  • MINIO_ROOT_USER: user key

  • MINIO_ROOT_PASSWORD: user key

The above builds are all stand-alone versions. If you want to know how to distribute, please check the official website documentation.

This is what runs on docker for win.

More open source projects: https://www.yoodb.com/projects/springboot-user-manger.html

After starting, visit http://localhost:9000 in the browser to access the minio graphical interface, as shown in the figure:

2. Let’s start to build the springboot environment

Everyone can initialize a springboot project, so I won’t introduce it here.

Mainly introduce the dependencies that need to be introduced:

<!-- thymeleaf template rendering engine -->
  <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-thymeleaf</artifactId>
        </dependency>
<!-- Java client for operating minio -->
         <dependency>
            <groupId>io.minio</groupId>
            <artifactId>minio</artifactId>
            <version>8.2.1</version>
        </dependency>
<!-- lombok plugin -->
         <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>

The dependency can be found in the official document: https://docs.min.io/docs/java-client-quickstart-guide.html

Here are the configuration files:

spring:
  servlet:
    multipart:
      max-file-size: 10MB
      max-request-size: 10MB
#minioconfiguration
  minio:
    access-key: AKIAIOSFODNN7EXAMPLE #key is the docker initialization is set, the key is the same
    secret-key: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
    url: http://localhost:9000
    bucket-name: wdhcr
  thymeleaf:
    cache: false

Create minio configuration class:

@Configuration
@ConfigurationProperties(prefix = "spring. minio")
@Data
public class MinioConfiguration {
    private String accessKey;

    private String secretKey;

    private String url;

    private String bucketName;

    @Bean
    public MinioClient minioClient() {
        return MinioClient. builder()
                .endpoint(url)
                .credentials(accessKey, secretKey)
                .build();
    }
}

Use configuration property binding for parameter binding, and initialize a minio client object into the container.

The following are the components of the simple method of operating minio by the minio client I encapsulated.

@Component
public class MinioComp {

    @Autowired
    private MinioClient minioClient;

    @Autowired
    private MinioConfiguration configuration;

    /**
     * @description: Obtain the temporary signature for uploading, public account Java selection
     * @dateTime: 2021/5/13 14:12
     */
    public Map getPolicy(String fileName, ZonedDateTime time) {
        PostPolicy postPolicy = new PostPolicy(configuration. getBucketName(), time);
        postPolicy.addEqualsCondition("key", fileName);
        try {
            Map<String, String> map = minioClient.getPresignedPostFormData(postPolicy);
            HashMap<String, String> map1 = new HashMap<>();
            map.forEach((k,v)->{
               map1.put(k.replaceAll("-",""),v);
           });
            map1.put("host", configuration. getUrl() + "/" + configuration. getBucketName());
            return map1;
        } catch (ErrorResponseException e) {
            e.printStackTrace();
        } catch (InsufficientDataException e) {
            e.printStackTrace();
        } catch (InternalException e) {
            e.printStackTrace();
        } catch (InvalidKeyException e) {
            e.printStackTrace();
        } catch (InvalidResponseException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        } catch (NoSuchAlgorithmException e) {
            e.printStackTrace();
        } catch (ServerException e) {
            e.printStackTrace();
        } catch (XmlParserException e) {
            e.printStackTrace();
        }
        return null;
    }

    /**
     * @description: Obtain the url of the uploaded file, public account Java selection, there are surprises!
     * @dateTime: 2021/5/13 14:15
     */
    public String getPolicyUrl(String objectName, Method method, int time, TimeUnit timeUnit) {
        try {
            return minioClient.getPresignedObjectUrl(GetPresignedObjectUrlArgs.builder()
                    .method(method)
                    .bucket(configuration. getBucketName())
                    .object(objectName)
                    .expiry(time, timeUnit).build());
        } catch (ErrorResponseException e) {
            e.printStackTrace();
        } catch (InsufficientDataException e) {
            e.printStackTrace();
        } catch (InternalException e) {
            e.printStackTrace();
        } catch (InvalidKeyException e) {
            e.printStackTrace();
        } catch (InvalidResponseException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        } catch (NoSuchAlgorithmException e) {
            e.printStackTrace();
        } catch (XmlParserException e) {
            e.printStackTrace();
        } catch (ServerException e) {
            e.printStackTrace();
        }
        return null;
    }


    /**
     * @description: upload file
     * @dateTime: 2021/5/13 14:17
     */
    public void upload(MultipartFile file, String fileName) {
        // Upload a file to the bucket using putObject.
        try {
            InputStream inputStream = file. getInputStream();
            minioClient.putObject(PutObjectArgs.builder()
                    .bucket(configuration. getBucketName())
                    .object(fileName)
                    .stream(inputStream, file. getSize(), -1)
                    .contentType(file.getContentType())
                    .build());
        } catch (ErrorResponseException e) {
            e.printStackTrace();
        } catch (InsufficientDataException e) {
            e.printStackTrace();
        } catch (InternalException e) {
            e.printStackTrace();
        } catch (InvalidKeyException e) {
            e.printStackTrace();
        } catch (InvalidResponseException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        } catch (NoSuchAlgorithmException e) {
            e.printStackTrace();
        } catch (ServerException e) {
            e.printStackTrace();
        } catch (XmlParserException e) {
            e.printStackTrace();
        }
    }
  /**
     * @description: Get file access address according to filename
     * @dateTime: 2021/5/17 11:28
     */
    public String getUrl(String objectName, int time, TimeUnit timeUnit) {
        String url = null;
        try {
            url = minioClient.getPresignedObjectUrl(GetPresignedObjectUrlArgs.builder()
                    .method(Method.GET)
                    .bucket(configuration. getBucketName())
                    .object(objectName)
                    .expiry(time, timeUnit).build());
        } catch (ErrorResponseException e) {
            e.printStackTrace();
        } catch (InsufficientDataException e) {
            e.printStackTrace();
        } catch (InternalException e) {
            e.printStackTrace();
        } catch (InvalidKeyException e) {
            e.printStackTrace();
        } catch (InvalidResponseException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        } catch (NoSuchAlgorithmException e) {
            e.printStackTrace();
        } catch (XmlParserException e) {
            e.printStackTrace();
        } catch (ServerException e) {
            e.printStackTrace();
        }
        return url;
    }
}

Simple explanation:

  • Use MultipartFile to receive the front-end file stream, and then upload it to minio.

  • Build a formData signature data, give it to the front end, and let the front end upload it to minio before.

  • Build a temporary URL that can be uploaded to the front end, and the front end requests the URL to upload by carrying the file.

  • Use filename to request the server to obtain the URL of the temporary access file. (The maximum time is 7 days. If you want permanent access, you need other settings, which will not be explained here.)

  • Recommended, public account Java selection, reply to java interviews, obtain interview materials, and support online quizzes.

The following shows the page html, using VUE + element-ui for rendering.

<!DOCTYPE html>
<html>
<head>
    <meta charset="UTF-8">
    <!-- import CSS -->
    <link rel="stylesheet" href="https://unpkg.com/element-ui/lib/theme-chalk/index.css">
    <title>Upload picture</title>
</head>
<body>
<div id="app">

    <el-row:gutter="2">
        <el-col :span="8">
            <div class="div-center-class">
                <div class="">
                    <center><h3>Traditional upload</h3></center>
                    <el-upload
                            class="upload-demo"
                            action="#"
                            drag
                            :http-request="uploadHandle">
                        <i class="el-icon-upload"></i>
                        <div class="el-upload__text">Drag files here, or <em>click upload</em></div>
                        <div class="el-upload__tip" slot="tip">Only upload jpg/png files, and no more than 500kb</div>
                    </el-upload>
                    <div v-if="imgUrl">
                        <img :src="imgUrl" style="width: 40px;height: 40px"></img>
                    </div>
                </div>
            </div>
        </el-col>
        <el-col :span="8">
            <div class="div-center-class">
                <div class="">
                    <center><h3>Front-end formData direct transmission</h3></center>
                    <el-upload
                            class="upload-demo"
                            action="#"
                            drag
                            :http-request="httpRequestHandle">
                        <i class="el-icon-upload"></i>
                        <div class="el-upload__text">Drag files here, or <em>click upload</em></div>
                        <div class="el-upload__tip" slot="tip">Only upload jpg/png files, and no more than 500kb</div>
                    </el-upload>
                    <div v-if="directUrl">
                        <img :src="directUrl" style="width: 40px;height: 40px"></img>
                    </div>
                </div>
            </div>
        </el-col>
        <el-col :span="8">
            <div class="div-center-class">
                <div class="">
                    <center><h3>Front-end Url direct transmission</h3></center>
                    <el-upload
                            class="upload-demo"
                            action="#"
                            drag
                            :http-request="UrlUploadHandle">
                        <i class="el-icon-upload"></i>
                        <div class="el-upload__text">Drag files here, or <em>click upload</em></div>
                        <div class="el-upload__tip" slot="tip">Only upload jpg/png files, and no more than 500kb</div>
                    </el-upload>
                    <div v-if="uploadUrl">
                        <img :src="uploadUrl" style="width: 40px;height: 40px"></img>
                    </div>
                </div>
            </div>
        </el-col>
    </el-row>
</div>
</body>
<!-- import Vue before Element -->
<script src="//i2.wp.com/unpkg.com/vue/dist/vue.js"></script>
<!-- import JavaScript -->
<script src="//i2.wp.com/unpkg.com/element-ui/lib/index.js"></script>
<!--import axios -->
<script src="//i2.wp.com/unpkg.com/axios/dist/axios.min.js"></script>
<script>
    new Vue({
        el: '#app',
        data: function () {
            return {
                imgUrl: '',
                directUrl: '',
                uploadUrl: ''
            }
        },
        methods: {

            uploadHandle(options) {
                let {file} = options;
                this.traditionPost(file);
            },
            traditionPost(file) {
                _that = this
                const form = new FormData();
                form.append("fileName", file.name);
                form.append("file", file);
                this.axiosPost("post", "/upload", form).then(function (res) {
                    if (res. status === 200) {
                        _that.imgUrl = res.data.data
                    } else {
                        alert("Upload failed!")
                    }
                })
            },
            getpolicy(file) {
                _that = this
                axios.get('policy?fileName=' + file.name)
                    .then(function (response) {
                        let {xamzalgorithm, xamzcredential, policy, xamzsignature, xamzdate, host} = response.data.data;
                        let formData = new FormData();
                        formData.append("key", file.name);
                        formData.append("x-amz-algorithm", xamzalgorithm); // Let the server return 200, if not set, return 204 by default.
                        formData.append("x-amz-credential", xamzcredential);
                        formData.append("policy", policy);
                        formData.append("x-amz-signature", xamzsignature);
                        formData.append("x-amz-date", xamzdate);
                        formData.append("file", file);
                        // send a POST request
                        _that.axiosPost("post", host, formData).then(function (res) {
                            if (res. status === 204) {
                                axios.get('url?fileName=' + file.name).then(function (res) {
                                    _that.directUrl = res.data.data;
                                })
                            } else {
                                alert("Upload failed!")
                            }
                        })
                    })
            },
            httpRequestHandle(options) {
                let {file} = options;
                this. getpolicy(file);
            },

            UrlUploadHandle(options) {
                let {file} = options;
                this. getUploadUrl(file);
            },
            getUploadUrl(file) {
                _that = this
                console. log(file)
                axios.get('uploadUrl?fileName=' + file.name)
                    .then(function (response) {
                        let url = response.data.data;
                        // send put request
                        let config = {'Content-Type': file.type}
                        _that.axiosPost("put", url, file, config).then(function (res) {
                            if (res. status === 200) {
                                axios.get('url?fileName=' + file.name).then(function (res) {
                                    _that.uploadUrl = res.data.data;
                                })
                            } else {
                                alert("Upload failed!")
                            }
                        })
                    })
            },
            //package
            //axios package post request
            axiosPost(method, url, data, config) {
                let result = axios({
                    method: method,
                    url: url,
                    data: data,
                    headers: config
                }).then(resp => {
                    return resp
                }).catch(error => {
                    return "exception=" + error;
                });
                return result;
            }

        }
    })
</script>
<style>
    .div-center-class {
        padding: 28% 0%;
        text-align: center;
        background: beige;
    }
</style>
</html>

The effect of the page is as shown above.

Different implementation effects can be experienced separately.

The above are all the steps to use springboot to build a minio-based high-performance storage service.

Address of this project:

https://gitee.com/jack_whh/minio-upload

Author: HUWD

https://blog.csdn.net/weixin_45089791/article/details/116921075/