Netty–file programming

3. File programming

3.1 FileChannel

FileChannel working mode

FileChannel can only work in blocking mode

Get

FileChannel cannot be opened directly. FileChannel must be obtained through FileInputStream, FileOutputStream or RandomAccessFile. They all have getChannel methods.

• The channel obtained through FileInputStream can only be read
• The channel obtained through FileOutputStream can only be written
• Whether it can be read and written through RandomAccessFile is determined by the read and write mode when constructing RandomAccessFile.

Read

Data will be read from the channel to fill the ByteBuffer. The return value indicates how many bytes have been read. -1 indicates that the end of the file has been reached.

int readBytes = channel.read(buffer);

Write

The correct way to write is as follows, SocketChannel

ByteBuffer buffer = ...;
buffer.put(...); // Store data
buffer.flip(); // Switch read mode

while(buffer.hasRemaining()) {<!-- -->
    channel.write(buffer);
}

Channel.write is called in while because the write method does not guarantee that all the contents of the buffer will be written to the channel at one time

Close

The channel must be closed, but calling the close method of FileInputStream, FileOutputStream or RandomAccessFile will indirectly call the channel’s close method.

Location

Get current location

long pos = channel.position();

Set current location

long newPos = ...;
channel.position(newPos);

When setting the current position, if set to the end of the file

• At this time, reading will return -1
• When writing at this time, content will be appended, but please note that if the position exceeds the end of the file, there will be a hole (00) between the new content and the original end when writing again.

Size

Use the size method to get the size of a file

Force writing

For performance reasons, the operating system caches data instead of writing it to disk immediately. You can call the force(true) method to immediately write the file content and metadata (file permissions and other information) to the disk.

3.2 Two Channels transmit data

String FROM = "helloword/data.txt";
String TO = "helloword/to.txt";
long start = System.nanoTime();
try (FileChannel from = new FileInputStream(FROM).getChannel();
     FileChannel to = new FileOutputStream(TO).getChannel();
    ) {<!-- -->
    from.transferTo(0, from.size(), to);
} catch (IOException e) {<!-- -->
    e.printStackTrace();
}
long end = System.nanoTime();
System.out.println("transferTo time: " + (end - start) / 1000_000.0);

output

transferTo time: 8.2011

File transfer exceeding 2g size

public class TestFileChannelTransferTo {<!-- -->
    public static void main(String[] args) {<!-- -->
        try (
                FileChannel from = new FileInputStream("data.txt").getChannel();
                FileChannel to = new FileOutputStream("to.txt").getChannel();
        ) {<!-- -->
            // High efficiency, the bottom layer will use the zero copy of the operating system for optimization, 2g data
            long size = from.size();
            // The left variable represents how many bytes are left
            for (long left = size; left > 0; ) {<!-- -->
                System.out.println("position:" + (size - left) + " left:" + left);
                left -= from.transferTo((size - left), left, to);
            }
        } catch (IOException e) {<!-- -->
            e.printStackTrace();
        }
    }
}

Actual transfer of a very large file

position:0 left:7769948160
position:2147483647 left:5622464513
position:4294967294 left:3474980866
position:6442450941 left:1327497219

3.3 Path

jdk7 introduced the Path and Paths classes

• Path is used to represent the file path
• Paths is a tool class used to obtain Path instances

Path source = Paths.get("1.txt"); // Relative path uses user.dir environment variable to locate 1.txt

Path source = Paths.get("d:\1.txt"); // The absolute path represents d:\1.txt

Path source = Paths.get("d:/1.txt"); // Absolute path also represents d:\1.txt

Path projects = Paths.get("d:\data", "projects"); // represents d:\data\projects

• . represents the current path
• .. represents the upper level path

For example, the directory structure is as follows

d:
|- data
|- projects
|-a
|- b

code

Path path = Paths.get("d:\data\projects\a\..\b");
System.out.println(path);
System.out.println(path.normalize()); // Normalize path

will output

d:\data\projects\a\..\b
d:\data\projects\b

3.4 Files

Check if the file exists

Path path = Paths.get("helloword/data.txt");
System.out.println(Files.exists(path));

Create a first-level directory

Path path = Paths.get("helloword/d1");
Files.createDirectory(path);

• If the directory already exists, an exception FileAlreadyExistsException will be thrown.
• Multi-level directories cannot be created at one time, otherwise NoSuchFileException will be thrown.

Used to create multi-level directories

Path path = Paths.get("helloword/d1/d2");
Files.createDirectories(path);

Copy files

Path source = Paths.get("helloword/data.txt");
Path target = Paths.get("helloword/target.txt");

Files.copy(source, target);

• If the file already exists, an exception FileAlreadyExistsException will be thrown.

If you want to use source to overwrite target, you need to use StandardCopyOption to control

Files.copy(source, target, StandardCopyOption.REPLACE_EXISTING);

Move files

Path source = Paths.get("helloword/data.txt");
Path target = Paths.get("helloword/data.txt");

Files.move(source, target, StandardCopyOption.ATOMIC_MOVE);

• StandardCopyOption.ATOMIC_MOVE ensures atomicity of file movement

Delete Files

Path target = Paths.get("helloword/target.txt");

Files.delete(target);

• If the file does not exist, an exception NoSuchFileException will be thrown.

delete directory

Path target = Paths.get("helloword/d1");

Files.delete(target);

• If the directory still contains content, an exception DirectoryNotEmptyException will be thrown.

Traverse directory files

public static void main(String[] args) throws IOException {<!-- -->
    Path path = Paths.get("C:\Program Files\Java\jdk1.8.0_91");
    AtomicInteger dirCount = new AtomicInteger();//Folder atomic counter, thread safe
    AtomicInteger fileCount = new AtomicInteger();//File atomic counter, thread safe
    Files.walkFileTree(path, new SimpleFileVisitor<Path>(){<!-- -->
        @Override
        public FileVisitResult preVisitDirectory(Path dir, BasicFileAttributes attrs)
            throws IOException {<!-- -->
            System.out.println(dir);
            dirCount.incrementAndGet();//Increase the count when traversing to the folder
            return super.preVisitDirectory(dir, attrs);
        }

        @Override
        public FileVisitResult visitFile(Path file, BasicFileAttributes attrs)
            throws IOException {<!-- -->
            System.out.println(file);
            fileCount.incrementAndGet();//Increase the count when traversing to the file
            return super.visitFile(file, attrs);
        }
    });
    System.out.println(dirCount); // 133
    System.out.println(fileCount); // 1479
}

Count the number of jars

Path path = Paths.get("C:\Program Files\Java\jdk1.8.0_91");
AtomicInteger fileCount = new AtomicInteger();
Files.walkFileTree(path, new SimpleFileVisitor<Path>(){<!-- -->
    @Override
    public FileVisitResult visitFile(Path file, BasicFileAttributes attrs)
        throws IOException {<!-- -->
        if (file.toFile().getName().endsWith(".jar")) {<!-- -->//file.toString().endsWith(".jar")
            fileCount.incrementAndGet();
        }
        return super.visitFile(file, attrs);
    }
});
System.out.println(fileCount); // 724

Delete multi-level directories

Path path = Paths.get("d:\a");
Files.walkFileTree(path, new SimpleFileVisitor<Path>(){<!-- -->
    @Override
    public FileVisitResult visitFile(Path file, BasicFileAttributes attrs)
        throws IOException {<!-- -->
        Files.delete(file);
        return super.visitFile(file, attrs);
    }

    @Override
    public FileVisitResult postVisitDirectory(Path dir, IOException exc)
        throws IOException {<!-- -->
        Files.delete(dir);
        return super.postVisitDirectory(dir, exc);
    }
});

Deletion is dangerous

Deletion is a dangerous operation, make sure there is no important content in the folder you want to delete recursively

Copy multi-level directories

long start = System.currentTimeMillis();
String source = "D:\Snipaste-1.16.2-x64";
String target = "D:\Snipaste-1.16.2-x64aaa";

Files.walk(Paths.get(source)).forEach(path -> {<!-- -->
    try {<!-- -->
        String targetName = path.toString().replace(source, target);
        // is a directory
        if (Files.isDirectory(path)) {<!-- -->
            Files.createDirectory(Paths.get(targetName));
        }
        // It is a normal file
        else if (Files.isRegularFile(path)) {<!-- -->
            Files.copy(path, Paths.get(targetName));
        }
    } catch (IOException e) {<!-- -->
        e.printStackTrace();
    }
});
long end = System.currentTimeMillis();
System.out.println(end - start);