android ANativeWindow rotation rendering angle
MediaCodec rotation angle reference
videoExtractor opens an angled video file mediaFormat.getInteger(MediaFormat.KEY_ROTATION); gets the angle
MediaFormat mediaFormat = videoExtractor.getTrackFormat(j);
String mime = mediaFormat.getString(MediaFormat.KEY_MIME);
if (mime.startsWith(KEY_VIDEO)) {//Match the track corresponding to the video
videoExtractor.selectTrack(j);//Select the track corresponding to the video
long duration = mediaFormat.getLong(MediaFormat.KEY_DURATION);
int width = mediaFormat.getInteger(MediaFormat.KEY_WIDTH);
int height = mediaFormat.getInteger(MediaFormat.KEY_HEIGHT);
int degrees = 0;
try {
if (mediaFormat.containsKey(MediaFormat.KEY_ROTATION))
degrees = mediaFormat.getInteger(MediaFormat.KEY_ROTATION);//Some videos will have a null pointer without this parameter
} catch (Exception e) {
e.printStackTrace();
}
}
Pass in the angle when MediaCodec is created and configure, and decode mediaCodec.releaseOutputBuffer(outputBufferIndex, true); during rendering, you find that your video is straight and has no rotation angle.
mediaCodec = MediaCodec.createDecoderByType("video/avc");
mediaCodec.configure(mediaFormat surface, null, 0);
According to the Android API and framework above, you can find that the bottom layer is rotated during rendering in mediaCodec.releaseOutputBuffer(outputBufferIndex, true); decoding does not help you rotate the angle. Verify that the rotation angle is only valid when MediaCodec is directly decoded to Surface. .
The flow process of rotation angle in MediaCodec
1. MediaCodec::configure configures the decoder, using MediaFormat as a parameter.
2. MediaCodec sends the kWhatConfigure message to configure the decoder in its own thread.
3. Call ACodec->initiateConfigureComponent (MediaFormat is the parameter) to configure ACodec.
4. ACodec sends the kWhatConfigureComponent message to configure ACodec in its own thread.
5. Next is the ACodec::LoadedState::onConfigureComponent method.
6. Then there is the ACodec.configureCodec method, which is responsible for configuring ACodec through MediaFormat, in which “rotation-degrees” is taken out from the format and stored in the ACodec.mRotationDegrees variable.
7. Finally, call the global function setNativeWindowSizeFormatAndUsage in ACodec.setupNativeWindowSizeFormatAndUsage to set the transform flag for Surface (ANativeWindow). The core code is in the setNativeWindowSizeFormatAndUsage function:
// Get the transform flag based on the rotation angle
int transform = 0;
if ((rotation % 90) == 0) {
switch ((rotation / 90) & amp; 3) {
case 1: transform = HAL_TRANSFORM_ROT_90; break;
case 2: transform = HAL_TRANSFORM_ROT_180; break;
case 3: transform = HAL_TRANSFORM_ROT_270; break;
default: transform = 0; break;
}
}
//Set transform for Surface (ANativeWindow)
err = native_window_set_buffers_transform(nativeWindow, transform);
Can the ANativeWindow be rotated when filling in your decoded data
Drawing on MediaCodec from above, it can be implemented normally through the API of ndk and android. ndk provides the ANativeWindow_setBuffersTransform method.
Realization conditions:
1. Call the implementation to configure the OpenGL environment in GLSurfaceView to obtain the surface and convert it to nativewindow in jni.
2. Gradle’s android version 8.0 is only compatible with minSdkVersion 26 or above.
3. CMakeLists should be associated with the nativewindow library
Implementation code:
Key header files #include
void WlVideo::postDataFromNative(const AVFrame *avFrame,int angle) {
int retval;
char overlay_format[5] = {'Y', 'V', '1', '2', 0};
int curr_w = ANativeWindow_getWidth(native_window);
int curr_h = ANativeWindow_getHeight(native_window);
int curr_format = ANativeWindow_getFormat(native_window);
int buff_w = avFrame->width;
int buff_h = avFrame->height;
if (curr_format != HAL_PIXEL_FORMAT_YV12) {
LOGE("ANativeWindow_setBuffersGeometry: w=%d, h=%d, f=%.4s(0x%x) => w=%d, h=%d, f=%.4s",
curr_w, curr_h, (char *) & curr_format, curr_format,
buff_w, buff_h, (char *) overlay_format);
retval = ANativeWindow_setBuffersGeometry(native_window, buff_w, buff_h,
HAL_PIXEL_FORMAT_YV12);
if (retval < 0) {
LOGE("ANativeWindow_setBuffersGeometry: failed %d", retval);
return;
}
}
ANativeWindow_Buffer out_buffer;
retval = ANativeWindow_lock(native_window, & amp;out_buffer, NULL);
if (retval < 0) {
LOGE("ANativeWindow_lock: failed %d", retval);
return;
}
LOGE("postDataFromNative1 native window buffer (%p)(out_buffer.width:%d, out_buffer.height:%d, out_buffer.format:'%.4s'0x%x), expecting (buff_w:%d, buff_h:%d, overlay_format:'%.4s')",
native_window,
out_buffer.width, out_buffer.height, (char *) & & out_buffer.format, out_buffer.format,
buff_w, buff_h, (char *) overlay_format);
if (out_buffer.width != buff_w || out_buffer.height != buff_h) {
LOGE("unexpected native window buffer (%p)(w:%d, h:%d, fmt:'%.4s'0x%x), expecting (w:%d, h:%d, fmt :'%.4s')",
native_window,
out_buffer.width, out_buffer.height, (char *) & & out_buffer.format, out_buffer.format,
buff_w, buff_h, (char *) overlay_format);
// TODO: 8 set all black
ANativeWindow_unlockAndPost(native_window);
ANativeWindow_setBuffersGeometry(native_window, buff_w, buff_h, HAL_PIXEL_FORMAT_YV12);
return;
}
int render_ret = android_render_yv12_on_yv12( & amp;out_buffer, avFrame);
if (render_ret < 0) {
// TODO: 8 set all black
// return after unlock image;
}
/**
* minSdkVersion=26 gradle’s sdk version is 26 larger
* CMakeLists.txt needs target_link_libraries to be added to ndk's nativewindow, otherwise an error will be reported undefined reference to 'ANativeWindow_setBuffersTransform'
* The function of ANativeWindow_setBuffersTransform refers to mediacodec. It rotates the angle when decoding and rendering.
*/
switch (angle) {
case 90:
retval = ANativeWindow_setBuffersTransform(native_window, ANATIVEWINDOW_TRANSFORM_ROTATE_90);
break;
case 180:
retval = ANativeWindow_setBuffersTransform(native_window, ANATIVEWINDOW_TRANSFORM_ROTATE_180);
break;
case 270:
retval = ANativeWindow_setBuffersTransform(native_window, ANATIVEWINDOW_TRANSFORM_ROTATE_270);
break;
default:
break;
}
if (retval < 0) {
LOGE("ANativeWindow_setBuffersTransform: failed %d", retval);
}
LOGE("ANativeWindow_lock: ============= %d", render_ret);
retval = ANativeWindow_unlockAndPost(native_window);
if (retval < 0) {
LOGE("ANativeWindow_unlockAndPost: failed %d", retval);
return;
}
return;
}
The int WlVideo::android_render_yv12_on_yv12(ANativeWindow_Buffer *out_buffer, const AVFrame *avFrame) corresponding to the above method is the data filling of ANativeWindow by yuv420p. When the data is filled, the corresponding listening method of SurfaceTexture.OnFrameAvailableListener will be called and onFrameAvailable can refresh OpenGL rendering.
@Override
public void onFrameAvailable(SurfaceTexture surfaceTexture) {
Log.e("========","========onRender====");
surfaceView.requestRender();
}