How to use Serverless to achieve batching, automation and customization of video editing?

Before I start, let me first resolve the three doubts in everyone’s mind when they saw this title:

  • Can’t video editing be done using Adobe software?

  • Why use Serverless?

  • How to write code to make video editing?

1. Which video editing scenes cannot be completed by Adobe and other software

The video editing that everyone usually comes into contact with usually uses professional tools such as Premiere and AE to complete the video editing. They can complete some complex effects, such as making promotional videos, advertising videos, etc.

However, some companies hope to complete video editing in batches and automatically in certain business scenarios.

For example, the following scenarios:

Suppose the school hopes to present exciting videos of all students’ learning processes immediately after students finish online classes, coupled with the school’s logo and slogans, etc., so that students can share their results with one click. Assume that there are 10,000 students and a unique video needs to be produced for each student, so 10,000 different video clips need to be completed in batches and automatically.

In a certain marketing campaign, different avatar videos need to be generated for different users to attract user participation. Each user’s avatar is unique, and the generated video is also unique. There may be thousands of users, so automation is a must.

Internet celebrity operating companies hope to generate unified business videos for all anchors. There may be 100 anchors, and finding one person to edit 100 videos seems barely acceptable, but what if you have to edit a different video every week? So automated, batch and customizable editing have become major requirements.

There are three characteristics in the above scene:

  • batch

  • automation

  • customizable

For scenes that meet the above characteristics, traditional video editing tools or templated video processing software cannot easily complete them.

2. Why is Serverless recommended?

Because a business like video editing has several characteristics:

  • Concentrated use period.

  • It is computationally intensive.

The utilization rate of purchasing high-specification servers separately is very low, and the computing power of cheap servers cannot keep up.

Therefore, Serverless’ pay-as-you-go features and high-performance computing capabilities perfectly match such demand scenarios.

It can not only achieve 100% utilization, but also use its high-performance computing power according to the amount. At the same time, Tencent Cloud Serverless cloud functions have a changeable programmable environment and can use any familiar programming language, which is very flexible.

3. How to make video editing by writing code

All the video editing functions mentioned in this article use the tool FFmpeg. Let me first tell you what FFmpeg is.

FFmpeg (http://ffmpeg.org/) is an open source tool used for video processing. It has very powerful functions. It supports video editing, video transcoding, video editing, audio processing, adding text, video splicing, Functions such as pull streaming and push streaming live broadcast.

We can program different video editing functions through different FFmpeg commands, and when combined and arranged, we can cope with various batch automation scenarios.

4. Practice of batching, automation and customization of video editing

Common video editing scenes mainly include the following:

  • Video transcoding

  • Video cropping

  • Video plus text

  • Video plus pictures

  • Video splicing

  • video plus audio

  • Video transition

  • video effects

  • Video acceleration and slow playback

Next, I will show you some specific FFmpeg command examples. If you have FFmpeg installed locally, you can also execute these commands locally. Regarding how to install FFmpeg, you can read the tutorial on the official website (http://ffmpeg.org/).

//Convert MOV video to mp4 video
ffmpeg -i input.mov output.mp4

// Modify the frame rate of the original video to 24
ffmpeg -i input.mp4 -r 24 -an output.mp4

// Convert mp4 video into a video stream that can be used for live broadcast
ffmpeg -i input.mp4 -codec: copy -bsf:v h264_mp4toannexb -start_number 0 -hls_time 10 -hls_list_size 0 -f hls output.m3u8

//Change the video to 480x360 and change the bit rate to 400
ffmpeg -i input.mp4 -vf scale=480:360,pad=480:360:240:240:black -c:v libx264 -x264-params nal-hrd=cbr:force-cfr=1 -b:v 400000 -bufsize 400000 -minrate 400000 -maxrate 400000 output.mp4

// Add text to the video, such as subtitles, titles, etc.
// `fontfile` is the path to the font to be used, `text` is the text you want to add,
// `fontcolor` is the color of the text, `fontsize` is the size of the text, and `box` is to add a bottom frame to the text.
// `box=1` means enable, `0` means disable, `boxcolor` is the color of the bottom box, [email protected] means the black transparency is 50%, `boxborderw` is the width of the bottom box from the text
// `x` and `y` are the positions of text. `x` and `y` not only support numbers, but also support various expressions. You can check the official website for details.
ffmpeg -i input.mp4 -vf "drawtext=fontfile=/path/to/font.ttf:text='your text':fontcolor=white:fontsize=24:box=1:[email protected] :boxborderw=5:x=(w-text_w)/2:y=(h-text_h)/2" -codec:a copy output.mp4

// Add pictures to the video, such as adding logos, avatars, expressions, etc. filter_complex represents a composite filter, overlay represents the x and y of the image, and enable represents the time period during which the image appears, from 0-20 seconds.
ffmpeg -i input.mp4 -i avatar.JPG -filter_complex "[0:v][1:v] overlay=25:25:enable='between(t,0,20)'" -pix_fmt yuv420p -c:a copy output.mp4

// For video splicing, list.txt contains the file paths of all the videos to be spliced in order, as follows.
// Note that if the video resolutions are inconsistent, the splicing will fail.
ffmpeg -f concat -safe 0 -i list.txt -c copy -movflags + faststart output.mp4

// The format of list.txt is as follows file 'xx.mp4' file 'yy.mp4'
// Video plus audio, stream_loop indicates whether to loop the audio content, -1 indicates infinite loop, 0 indicates no loop. shortest indicates the end of the shortest MP3 input stream to complete encoding.
ffmpeg -y -i input.mp4 -stream_loop -1 -i audio.mp3 -map 0:v -map 1:a -c:v copy -shortest output.mp4

FFmpeg can do a lot of things, so I won’t explain them one by one here. More ways to play can be explored on the FFmpeg official website.

The same is true for audio editing. FFmpeg also supports separate audio editing.

5. Run FFmpeg command

Because Python is more convenient to run these commands, we can use Python to run all FFmpeg commands.

At the same time, Python has better running performance on Tencent Cloud Serverless cloud functions and is easy to deploy.

The video editing code for using FFmpeg through Python has an open source link at the end of the article, and there are also templates on the official website that can be used directly, covering almost all common audio and video editing operations.

Here is a simple calling code example:

child = subprocess.run('./ffmpeg -i input.mov output.mp4', stdout=subprocess.PIPE, stderr=subprocess.PIPE, close_fds=True, shell=True)if child.returncode = = 0: print("success:", child)else: print("error:", child) raise KeyError("Failed to process video, error: ", child)

6. Deployment in Serverless

I have implemented the common video editing scenarios mentioned above and made them open source. Download the code and deploy it directly to Serverless for use. Download address:

https://github.com/woodyyan/ffmpeg-composition

https://github.com/woodyyan/ffmpeg-splice

There are two functions here, one is responsible for processing a single video, and the other is responsible for splicing multiple videos into one video with background music.

Currently the following features are supported:

  • Add text to video

  • Video resolution conversion

  • Add pictures to video

  • Video splicing

  • Add background music

The source code shows only some common video editing scenes. You can write your own video editing logic according to your business needs.

(1) Method 1: Github Action automatic deployment

  • Fork warehouse.

  • Add the two keys TENCENT_SECRET_ID and TENCENT_SECRET_KEY in the Settings-Secrets-Actions of the warehouse. The ID and KEY can be obtained in Tencent Cloud’s access control.

  • After adding it, deployment can be initiated in Action. Every time the modified code is pushed, Action deployment will be automatically triggered.

  • If you need some customized configuration, please modify serverless.yml.

  • The cloud function will eventually be automatically deployed to the account where TENCENT_SECRET_ID is located.

(2) Method 2: Manual deployment on the cloud function console

  • Download the code.

  • Pack all files and folders into a ZIP file in the root directory.

  • Go to the cloud function console (https://console.cloud.tencent.com/scf/list) and create a new function.

  • Choose to start from scratch:

  • Choose python language.

  • Upload the ZIP file.

  • It is recommended to choose larger memory for function memory.

  • Enable asynchronous execution.

  • It is recommended that the execution timeout be set longer depending on the size of the video, such as more than 30 seconds.

  • Configure triggers, select API gateway triggers, and turn off integrated responses.

  • After completing the deployment, you can start calling through the URL of the API gateway.

7. Review of real cases

A school that offers online classes needs to make a 30-second video of the online class after the students have finished the online class, as the students’ learning results. There are several key information points in this case:

  • Usually there are 200 students in a class and 200 videos need to be produced at the same time.

  • The one-hour class video needs to be edited into 30 seconds.

  • Since each student’s class screen is different, the recorded videos will be different.

  • The final product video also needs to include the students’ names and avatars.

  • The time when students finish class is very concentrated, so there will be short periods of high concurrency when making videos.

  • Videos are only required to be made after each class, and the time period is relatively fixed and concentrated.

Based on the above characteristics, using Tencent Cloud Serverless cloud functions to do such video editing brings multiple benefits:

  • The problem of 200 concurrency is solved, and there is no need to build too many servers yourself.

  • This solves the problem of using it only during the occurrence period, and no cost will be incurred in other periods.

  • It solves the problem of quickly producing videos that requires strong computing power.

The following is the reference architecture diagram of this case:

Summary

By arranging, combining, and reusing the various audio and video editing scenes listed above, you can create a variety of desired effects.

Then, the parameters used to control various effects in the video clip are changed into parameters passed in when calling the service, so that various effects can be customized.

Finally, let’s summarize the usage scenarios of video editing through this way of writing code:

  • Solve the scenario of batch production of videos by modifying individual parameters.

  • Solve the scenario of automating video production through user triggers.

  • Solving different scenarios requires different customized video production scenarios.

At the same time, using Tencent Cloud Serverless cloud functions to complete video editing also solves the following problems:

  • Because video editing usually does not run all day long, using the pay-as-you-go feature of Tencent Cloud Serverless cloud functions can optimize costs.

  • Because video editing is usually a heavy computing scenario, use the optional high-specification configuration of Tencent Cloud Serverless cloud functions to deal with this heavy computing scenario.

  • High concurrency usually exists in batch video production scenarios. The automatic elastic scaling feature of Tencent Cloud Serverless cloud functions can easily cope with high concurrency.

Note: If you are interested in using Serverless to implement batch, automated and customized video editing solutions, please feel free to add a little assistant (WeChat ID: skychoud) to learn more~