Table of Contents
1. Overview of FFMPEG
?2. FFMPEG decoding
2.1 Decoding process
2.2 Decoding example
1. FFMPEG Overview
- FFmpeg is a set of open source computer programs that can be used to record, convert, and convert digital audio and video into streams.
- FFmpeg adopts LGPL or GPL license; it provides a complete solution for recording, converting and streaming audio and video; it also includes a very advanced audio\video codec library libavcodec. In order to ensure high portability and codec quality, Many codes in libavcodec are developed from scratch.
- FFmpeg is developed under the Linux platform, but it can also be compiled and run in other operating system environments, including Windows, Mac OS X, etc.
- FFmpeg was first initiated by Fabrice Bellard and was mainly maintained by Michael Niedermayer from 2004 to 2015. Many FFmpeg developers come from the MPlayer project, and currently FFmpeg is also placed on the server of the MPlayer project team.
- The name FFmpeg comes from the MPEG video coding standard, and the “FF” in front of it stands for “Fast Forward”.
Download link:Download FFmpeg
For specific download details, please see: Qt + FFmpeg—-environment construction under windows_Yanli Zhu’s blog-CSDN blog_ffmpeg development environment construction
?
?2. FFMPEG decoding
2.1 decoding process
Decoding: The process of decoding the compressed stream data (encoded data) obtained after decapsulating the video in the encapsulated format to obtain pixel data.
For example: decode
H.264
Compressed code stream data is obtained
YUV
(or
RGB
) pixel data.
The decoding flow chart is as follows:
Decoding idea analysis:
- Register all components av_register_all()
- Open video file avformat_open_input() may fail to open
- Get video information Video code stream, audio code stream, text code stream
- Find stream information avformat_find_stream_info()
- Find decoder avcodec_find_decoder() may not be found
- Open decoder avcodec_open2()
- Read a frame of code stream data in the code stream av_read_frame()
- Decode and read one frame of code stream data Get one frame of pixel data YUV RGB
- Repeat steps 7-8 until all frames of the video have been processed
- Turn off decoder
- Close video file
Several important structures in the decoding process:
- AVFormatContext
The encapsulation format context structure is also the overall structure that holds information related to the video file encapsulation format.
- AVInputFormat//AVOutpufFormat
Each packaging format (such as FLV, MKV, MP4, AVI) corresponds to one structure.
- AVStream
Each video (audio) stream in the video file corresponds to one this structure.
- AVCodecContext
The encoder context structure stores video (audio) encoding and decoding related information.
- AVCodec
Each video (audio) codec (such as H.264 decoder) corresponds to one structure.
- AVPacket
Stores one frame of compressed encoded data.
- AVFrame
Stores one frame of decoded pixel (sampling) data.
2.2 decoding example
Definition of decoding class:
//ffmpeg is implemented in C language. To introduce code written in C, you need to use extern.
extern "C"
{
#include <libavcodec/avcodec.h> //Encoding
#include <libavdevice/avdevice.h>
#include <libavformat/avformat.h> //Encapsulation format processing
#include <libavutil/error.h>
#include <libswscale/swscale.h> //Pixel processing
#include <libswresample/swresample.h> //Zoom
}
class fdecode
{
public:
fdecode();
//Register component
void registerFFmpeg();
//Open video stream
void openVIdeoStream(QString filename);
//Video name
QString filename;
protected:
private:
};
The specific implementation is as follows:
Register all components
void fdecode::registerFFmpeg()
{
//Register all components
av_register_all();
}
Open video file
AVFormatContext *forContent;//Structure used to save video-related information
forContent=avformat_alloc_context();//Allocate space
//Open video file
int res=avformat_open_input( & amp;forContent,filename.toStdString().c_str(),nullptr,nullptr);
if(res!=0)//Determine whether to open the video file
{
qDebug()<<"Cannot open video file";
return;
}
Get video file information
//Open video file successfully and obtain file information
res = avformat_find_stream_info(forContent,nullptr);//Check whether there is relevant video stream information
if(res<0)
{
qDebug()<<"No streaming information"<<endl;
return;
}
//A video stream has multiple streams, which are stored in the streams array in forContentext.
int videoType=-1;
//nb_streams represents how many structure information there are in the encapsulation format. Normally there are two: audio information and video information.
for(int i=0;i<forContent->nb_streams;i + + ) //i is less than the number of streams
{
if(forContent->streams[i]->codec->codec_type == AVMEDIA_TYPE_VIDEO)//Video stream
{
videoType=i;//Identification type
break;
}
}
//Determine whether there is video stream information
if(videoType==-1)
{
qDebug()<<"No video stream related information"<<endl;
return;
}
Find the corresponding decoder according to the encoding ID in the encoding and decoding context
//Context object structure corresponding to the codec: saves the decoder information and the width, height, and pixel information of the graphics
AVCodecContext *codec=forContent->streams[videoType]->codec;
//Find the corresponding video stream decoder
AVCodec *decoder = avcodec_find_decoder(codec->codec_id);
if(decoder==nullptr)//Determine whether the decoder is found
{
qDebug()<<"No corresponding decoder"<<endl;
return;
}
Open the decoder
//The decoder is found, open the decoder
res = avcodec_open2(codec,decoder,nullptr);
if(res!=0)
{
qDebug()<<"Decoder opening failed"<<endl;
return;
}
Read preparation to obtain YUV and RGB pixel data
//Prepare to read frame data--AVPacket is used to store compressed data frame by frame (h264)
AVPacket *pkt=nullptr;
//Set the buffer and open space
pkt=(AVPacket *) malloc(sizeof(AVPacket));
int size=codec->width*codec->height;//Calculate the data size of an image
av_new_packet(pkt,size);
/* pictureRGB saves the decoded RGB pixel data
* pictureYUV saves the decoded YUV pixel data
* picture saves unprocessed pixel data
*/
AVFrame *pictureRGB,*pictureYUV,*picture=nullptr;
//memory allocation
pictureRGB=av_frame_alloc();
pictureYUV=av_frame_alloc();
picture=av_frame_alloc();
//Size and format settings RGB
pictureRGB->width=codec->width;//Width
pictureRGB->height=codec->height;//height
pictureRGB->format=codec->pix_fmt;//Format setting
//Size and format settings YUV
pictureYUV->width=codec->width;//width
pictureYUV->height=codec->height;//height
pictureYUV->format=codec->pix_fmt;//Format setting
//How big is the YUV RGB pixel data obtained after decoding one frame of code stream data?
int numByte_RGB=avpicture_get_size(AV_PIX_FMT_RGB32,codec->width,codec->height);
int numByte_YUV=avpicture_get_size(AV_PIX_FMT_YUV420P,codec->width,codec->height);
//The open space is used to save the YUV RGB pixel data size
uint8_t *buffer_RGB=(uint8_t *)av_malloc(numByte_RGB*sizeof(uint8_t));
uint8_t *buffer_YUV=(uint8_t *)av_malloc(numByte_YUV*sizeof(uint8_t));
//Filling of pixel data
avpicture_fill((AVPicture *)pictureRGB,buffer_RGB,AV_PIX_FMT_RGB32,codec->width,codec->height);
avpicture_fill((AVPicture *)pictureYUV,buffer_YUV,AV_PIX_FMT_YUV420P,codec->width,codec->height);
//conversion rules
SwsContext *sws_RGB=nullptr;//Structure to save conversion rules
SwsContext *sws_YUV=nullptr;//Structure to save conversion rules
//Conversion rule settings AV_PIX_FMT_YUV420P AV_PIX_FMT_RGB32
sws_RGB=sws_getContext(codec->width,codec->height,codec->pix_fmt,
codec->width,codec->height,AV_PIX_FMT_RGB32, //target format
SWS_BICUBIC,nullptr,nullptr,nullptr); //Conversion rules
sws_YUV=sws_getContext(codec->width,codec->height,codec->pix_fmt,
codec->width,codec->height,AV_PIX_FMT_YUV420P,//target format
SWS_BICUBIC,nullptr,nullptr,nullptr); //Conversion rules
//File that saves h.264 compressed code stream data
FILE *fp=fopen("fileout/alenH264.h264","wb + ");//The file name can be defined by yourself
//File to save yuv pixel data
FILE *fp_yuv=fopen("fileout/alenyuv.yuv","wb + ");//The file name can be defined by yourself
Read compressed data frame by frame and save code stream data and YUV and RGB pixel data
int count=0;//Save pictures
//Read compressed data frame by frame
while(av_read_frame(forContent,pkt)==0)//read data
{
if(pkt->stream_index == videoType)//Determine whether a frame of code stream data is the video stream that needs to be obtained
{
fwrite(pkt->data,pkt->size,1,fp);//Write to file
int got_picture_ptr=-1;
//Decode to get YUV
res = avcodec_decode_video2(codec,picture, & amp;got_picture_ptr,pkt);
if(res < 0)
{
qDebug()<<"Decoding error"<<endl;
return;
}
//Compress code stream data, decoded pixel data, determine whether there is data that can be decoded, and who to decode
if(got_picture_ptr!=0)//Decoding operation
{
//Remove the bad data obtained by decoding
sws_scale(sws_RGB,picture->data,picture->linesize,0,picture->height,
pictureRGB->data,pictureRGB->linesize);//RGB
sws_scale(sws_YUV,picture->data,picture->linesize,0,picture->height,
pictureYUV->data,pictureYUV->linesize);//YUV
//Output YUV file
//AVFrame pixel frame is written to the file
fwrite(pictureYUV->data[0],size,1,fp_yuv); //y data
fwrite(pictureYUV->data[1],size/4,1,fp_yuv);//udata
fwrite(pictureYUV->data[2],size/4,1,fp_yuv);//v data
//Set to output a picture every 25 frames
count + + ;
if(count%==0)
{
QImage image((uchar *)pictureRGB->data[0],pictureRGB->width,pictureRGB->height,QImage::Format_RGB32);//pixel data
QString imageName = QString("image/test%1.jpg").arg(count);
image.save(imageName);//Save image function
}
}
}
//Release package and AVFrame resources
av_packet_unref(pkt);
av_frame_unref(picture);
}
qDebug()<<"Save code stream data successfully"<<endl;
Resource release
//Close h.264
fclose(fp);
//Close YUV file
fclose(fp_yuv);
//Release AVFrame
av_frame_free( & amp;picture);
//Close decoder
avcodec_close(codec);
//Release the video information structure
avformat_free_context(forContent);
Test the main function code as follows:
int main(int argc, char *argv[])
{
//QApplication a(argc, argv);
//Widget w;
//w.show();
fdecode *decode = new fdecode;
decode->registerFFmpeg();
decode->openVIdeoStream("filein/alen.avi");
//return a.exec();
}
The saved H.264, yuv files and image data are as follows:
H.264, yuv file
Picture data (one screenshot every 25 frames)
Use the following applications to play the H.264 and YUV files we obtained
Let’s play Japanese comics that bloggers like!
The effect is as follows:
?FFMPEG technology—environment configuration, see:
FFmpeg + Qt development (1): Detailed steps for setting up the environment under Windows_Yuanlizhu’s blog-CSDN blog_ffmpeg library
?FFMPEG technology—encoding process, see:
[FFmpeg + Qt development] Encoding process, ordinary video encoding + detailed examples, learn it in one go_Yuanlizhu’s blog-CSDN blog
?FFMPEG technology—transcoding process, see:
[FFmpeg + Qt development] Transcoding process H.264 conversion (mov, mp4, avi, flv) and other video formats with detailed examples_Yanli Zhu’s Blog-CSDN Blog
? This article mainly introduces the decoding part of FFmpeg technology. If you have any questions, you are welcome to learn and communicate in the comment area!
? I think the blogger’s writing is good, trouble! like! Comment! collect! Please support me! Crabs, you guys!
