Teach you how to download Java file breakpoints

Foreword

?

Internet connections are slow and unstable, and there is a possibility of disconnection due to network failures.

When the client downloads a large object, the probability of upload and download failure due to network disconnection becomes non-negligible.

?

When requesting a GET object, the client tells the interface service where to start outputting the data of the object by setting the Range header.

Determine whether to support breakpoint download, according to the document: 14.35.1 Byte Rangeshttps://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html

// directly judge whether there is Accept-Ranges = bytes
boolean support = urlConnection.getHeaderField("Accept-Ranges").equals("bytes");
System.out.println("Partial content retrieval support = " + (support ? "Yes" : "No));

For example:

<a href="/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="acc8c3c2cdc0c8ecc8c3c2cdc0c881dcdec3">[email protected]</a>:~$ curl -i --range 0-9 http://localhost:8080/file/chunk/download
HTTP/1.1 206
Accept-Ranges: bytes
Content-Disposition: inline;filename=pom.xml
Content-Range: bytes 0-9/13485
Content-Length: 10
Date: Mon, 01 Nov 2021 09:53:25 GMT

Directly judge the header HEAD, for example:

?

The HeadObject interface is used to obtain the meta information of a file (Object). Using this interface will not return the file content.

?

HEAD /ObjectName HTTP/1.1
Host: BucketName.oss-cn-hangzhou.aliyuncs.com
Date: GMT Date
Authorization: SignatureValue

It should be noted that the corresponding HTTP status code:

  • 206 Partial Content: HTTP Range request succeeded

  • 416 Requested Range Not Satisfiable status: HTTP Range request out of bounds

  • 200 OK: range request not supported

The summary is as follows:

  1. HTTP range request: HTTP/1.1 and above are required. If a segment of the double-end is lower than this version, it is considered unsupported.

  2. Whether range requests are supported is determined by Accept-Ranges in the response header.

  3. Specify the byte range of the requested content entity by adding the request header Range to the request header.

  4. In the response header, use Content-Range to identify the range of the currently returned content entity, and use Content-Length to identify the length of the currently returned content entity range.

  5. During the request process, If-Range can be used to distinguish whether the resource file has changed, and its value comes from ETag or Last-Modifled. If the resource file is changed, the download process will be repeated.

Practical production

Development also has to rely on evidence and set boundaries in order to control the overall situation.

There are ready-made documents, see Alibaba Cloud Documentation https://help.aliyun.com/document_detail/39571.html

  • Range: bytes=0-499: Indicates the content in the range of 0~499 bytes.

  • Range: bytes=500-999: Indicates the content of the 500th~999th byte range.

  • Range: bytes=-500: indicates the content of the last 500 bytes.

  • Range: bytes=500-: Indicates the content from the 500th byte to the end of the file.

  • Range: bytes=0-: Indicates the first byte to the last byte, that is, the complete file content.

Whether HTTP Range is valid or not:

  • If the HTTP Range request is valid, the response returns 206 and contains the Content-Range in the response header

  • If the HTTP Range request is illegal, or the specified range is not in the valid range, the Range will not take effect, the response return value will be 200, and the entire Object content.

?

The following are examples and error descriptions of illegal HTTP Range requests: Assume that the resource size of Object is 1000 bytes, and the valid range of Range is 0~999

  • Range: byte=0-499: wrong format, byte should be bytes.

  • Range: bytes=0-1000: The last byte 1000 exceeds the valid range.

  • Range: bytes=1000-2000: The specified range exceeds the valid range.

  • Range: bytes=1000-: The first byte exceeds the valid range.

  • Range: bytes=-2000: The specified range exceeds the valid range.

?

Take some chestnuts:

> Background management system + user applet based on Spring Cloud Alibaba + Gateway + Nacos + RocketMQ + Vue & amp; Element, supporting RBAC dynamic permissions, multi-tenancy, data permissions, workflow, three-party login, payment, SMS , shopping mall and other functions
>
> * Project address: <https://github.com/YunaiV/yudao-cloud>
> * Video tutorial: <https://doc.iocoder.cn/video/>

# normal range download
<a href="/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="8aeee5e4ebe6eecaeee5e4ebe6eea7faf8e5">[email protected]</a>:~$ curl -i --range 0-9 http://localhost:8080/file/chunk/download
HTTP/1.1 206
Accept-Ranges: bytes
Content-Disposition: inline; filename=Screen_Recording_20211101-162729_Settings.mp4
Content-Range: bytes 0-9
Content-Type: application/force-download;charset=UTF-8
Content-Length: 16241985
Date: Wed, 03 Nov 2021 09:50:50 GMT

Server – Business Development

Here we take SpringBoot as a chestnut:

  1. External support range download

  2. Underlying storage: use ceph

  3. The Controller is as follows:

@Slf4j
@RestController
public class Controller {
    @Autowired
    private FileService fileService;
    
    /**
     * download file
     *
     * Provided externally
     *
     * @param fileId fileId
     * @param token token
     * @param accountId Account Id
     * @param response response
     */
    @GetMapping("/oceanfile/download")
    public void downloadOceanfile(@RequestParam String fileId,
                                  @RequestHeader(value = "Range") String range,
                                  HttpServletResponse response) {

        this.fileService.downloadFile(fileId, response, range);
    }
}

  1. The Service is as follows:

@Slf4j
@Service
public class FileService {
    @Autowired
    private CephUtils cephUtils;
    
    /**
     * Download files directly
     *
     * Tips: Support breakpoint download
     * @param fileId fileId
     * @param response return
     * @param range range
     */
    public void downloadFile(String fileId, HttpServletResponse response, String range) {
        // Get file information according to fileId
        FileInfo fileInfo = getFileInfo(fileId);

        String bucketName = fileInfo. getBucketName();
        String relativePath = fileInfo. getRelativePath();

        // process range, range information
        RangeDTO rangeInfo = executeRangeInfo(range, fileInfo. getFileSize());

        // rangeInfo = null, download the entire file directly
        if (Objects. isNull(rangeInfo)) {
        
            cephUtils.downloadFile(response, bucketName, relativePath);
            return;
        }
        // download some files
        cephUtils.downloadFileWithRange(response, bucketName, relativePath, rangeInfo);
    }

    private RangeDTO executeRangeInfo(String range, Long fileSize) {

        if (StringUtils.isEmpty(range) || !range.contains("bytes=") || !range.contains("-")) {

            return null;
        }

        long startByte = 0;
        long endByte = fileSize - 1;

        range = range.substring(range.lastIndexOf("=") + 1).trim();
        String[] ranges = range. split("-");

        if (ranges. length <= 0 || ranges. length > 2) {

            return null;
        }

        try {
            if (ranges. length == 1) {
                if (range. startsWith("-")) {

                    //1. For example: bytes=-1024 data from the start byte to the 1024th byte
                    endByte = Long. parseLong(ranges[0]);
                } else if (range. endsWith("-")) {

                    //2. For example: bytes=1024- the data from the 1024th byte to the last byte
                    startByte = Long. parseLong(ranges[0]);
                }
            } else {
                //3. For example: bytes=1024-2048 The data from the 1024th byte to the 2048th byte
                startByte = Long. parseLong(ranges[0]);
                endByte = Long. parseLong(ranges[1]);
            }
        } catch (NumberFormatException e) {
            startByte = 0;
            endByte = fileSize - 1;
        }
        
        if (startByte >= fileSize) {
            
            log.error("range error, startByte >= fileSize." +
                    "startByte: {}, fileSize: {}", startByte, fileSize);
            return null;
        }

        return new RangeDTO(startByte, endByte);
    }
}

You can collect the above content, if you encounter such a scene in the future, you can get it done in minutes!