Hadoop Mapper calculation and adjustment logic under various InputFormats

FileInputFormat Calculate splitSize goalSize total file size minSize configuration mapreduce.input.fileinputformat.split.minsize, default value 1 blockSize configuration dfs.block.size default value 128m computeSplitSize protected long computeSplitSize(long goalSize, long minSize, long blockSize) { return Math.max(minSize, Math.min(goalSize, blockSize)); } Calculate the number of MapTasks job.setLong(NUM_INPUT_FILES, files.length); long totalSize = 0; //compute total size for (FileStatus file: files) { // check […]

Conversion of dates in different formats (strings, timestamps) in the front end js related

I encountered it in the project and needed to convert strings and Unix timestamps to each other. I recorded it casually. The component library I use is Naive UI, and the component involved is Date Picker. The author writes in the document: To be honest, I don’t like this feature because in most cases, passing […]

osg implements a custom plug-in to read model files in custom formats into the scene

Table of Contents 1 Introduction 2. Preliminary knowledge 3. Tools and raw materials 4. Code implementation 1. Preface osg provides many plug-ins to read model files into scenes. These plug-ins support about 70 formats and types of files. However, in reality, files are of various types. osg cannot cover all types of files. When osg […]

The relationship between various encoding formats (GB2312, GBK, GB18030, unicode, utf-8)

Common encoding formats for Chinese characters To display characters on the screen. The following steps are required: Make fonts corresponding to all characters. For example, what does the capital letter A look like. This appearance is the final graphic displayed on the screen, which is the character A we see. To encode all characters. For […]

ChatGPT: Understanding HTTP request data formats: JSON, x-www-form-urlencoded and form-data

ChatGPT: Understanding HTTP request data formats: JSON, x-www-form-urlencoded and form-data Use postman to send a post request, add form-data data in the body, name=xxx, age=23, why do you get this result when outputting request.body() —————————-817240066476907930266144 Content-Disposition: form-data; name=”name” xxx —————————-817240066476907930266144 Content-Disposition: form-data; name=”age” 24 —————————-817240066476907930266144 ChatGPT: This is because when you send a POST request, […]

Use ffmpeg to pull streams from RTSP servers and save files in various formats

Add command options: RTSP url Video file saving format number of frames to save The first frame is an I frame Without further ado, here’s the code: #include<stdio.h> #include<libavformat/avformat.h> #include<libavutil/mathematics.h> #include<libavutil/time.h> #include <unistd.h> #define USAGE “rtsp2x -i <rtsp url> -t [avi | flv | mp4] -n <number of frames you want to save>” #define OPTS […]