Tucao
After changing to a new company, I faced two difficult problems as soon as I came up. The two problems were delayed for several months, and there was no solution after communicating with Spreadtrum.
Originally, I was working on the MTK platform, but now I need to work on the Spreadtrum platform and the Qualcomm platform.
It’s time to prove your abilities!
One week was used to get familiar with the Spreadtrum Camera driver. The two problems were basically overtime and the source code, and the solution was found in about 3 days!
In fact, there is still a sense of accomplishment. After all, the first thing we are facing is a new platform. In addition, especially the second problem, Spreadtrum has not been able to solve it for several months.
Cut the nonsense, the knowledge points of this article:
Platform: Sprd-Spreadtrum Platform
Knowledge points
- 1. Calculation method of frame rate
- 2. Frame skipping solves the problem of flashing red instantly during the photo taking process
Question 1: In the video call, the fixed frame rate is 10 frames, and the power consumption is optimized without affecting the number of frames for taking pictures.
Difficulty:
1. Distinguish whether it is a video call or a normal use of the camera.
Because the video call and the normal use of the camera are called camera preivew for preview.
Solution:
1. Distinguish at the upper level:
Contact the WeChat APP manufacturer to send a msg to the bottom layer when answering the call (complex, not advisable)
2. Distinguish at the HAL layer:
The width and height of the camera preview during video calls (480, 360), and the width and height of the camera preview when taking pictures (480, 480)
void SprdCamera3OEMIf::setCameraPreviewMode(bool isRecordMode) {
···
if (height==360) distinguish according to height
{
//Set a flag to indicate that a video call is in progress
video_call_flag = true;
}
···
}
But how to pass the value of video_call_flag to the bottom layer is a bit complicated.
\t\tmethod one:
Add the persist_sys_video_call_flag node and read this node at the bottom layer (no permission, not advisable)
Method 2:
Pass parameters from the hal layer through pointers (already implemented, complex changes, not desirable)
3. Distinguish at the bottom layer (optimal solution) Through log analysis: Since the video call, the fixed output frame rate is 10 frames, Therefore, the destination frame length is always dest_fre_len = dummy_line + shutter = 4326; The target frame length of this value is constant, and we can use this point to distinguish it at the bottom layer. \t
2. How to modify and modify the frame rate.
Camera sensor knowledge points (model: SP2609)
1. fps (frame rate) calculation method:
//cur_fr_len indicates the length of the current frame
if (cur_fr_len > shutter) {
fps = 1000000.0 / (frame length* line_time);
} else {
fps = 1000000.0 / ((shutter + dummy_line) * line_time);
}
- The shutter is written to P1: 0x03, 0x04 registers
- dummy_line: P1: 0x05 (high 8 bits), 0x06 (low 8 bits) can adjust the frame length by changing dummy_line -> then adjust the frame rate
- frame length (namely cur_fr_len): can be read out through P1:0x4e, 0x4f (only read operation)
Since 0x4e (higher 8 bits) and 0x4f (lower 8 bits) that control the frame length are read-only and cannot be written, how to change the frame length?
2. The calculation method of frame length
if (frame length > shutter) {
frame length = 1217 + dummy_line, (initial value + dummy_line)
//1217 here is when the value of dummy_line is 0, you can use the ISPTool tool to read the value of 4e, 4f to get, you can also ask FAE
//For example: if dummy_line(0x05,0x0C),(0x06,0x25)->0C25->3109 decimal
//Then frame length = 1217 + 3109 = 4326
} else {
frame length = shutter + -1;
}
- frame_length: Indicates the length of a frame, which can be understood as the number of lines in a frame;
- shutter: the number of exposure lines, the unit is line_time
- dummy_line: used to change the frame length to adjust the frame rate, but there is no actual data output;
- line_time: one line exposure time (fixed value: 23114)
Frame length: limit the maximum fps, for example:
During initialization, write (0x05,0x0c),(0x06,0x25) to dummy_line;
Forget about registers 05 and 06 and write 0c25->3109 (decimal)
Then frame_length = 1217 + dummy_line = 1217 + 3109;
The maximum output frame rate max_fps = 1000000.0/(frame length* line_time)
= 1000000.0/(4326*23114)
= 10 (fps)
The above logic is done inside the sensor, so the exposure interface function only needs to be configured. The exp variable in the AE Pline of the platform is converted into the exposure line count, and then written into the exp control registers P1:0x03, 0x04 and P1:0x05, 0x06. The method can refer to the following:
//shutter
static uint32_t sp2609_read_shutter(void)
{
uint16_t shutter_h = 0;
uint16_t shutter_l = 0;
Sensor_WriteReg(0xfd, 0x01);//fd register selects the first page
shutter_h = Sensor_ReadReg(0x03) & amp; 0xff;//write 03 register (high 8 bits)
shutter_l = Sensor_ReadReg(0x04) & amp; 0xff;//write 04 register (lower 8 bits)
return (shutter_h << 8) | shutter_l;//Return the value of high 8 bits and low 8 bits spliced together
}
static void sp2609_write_shutter(uint32_t shutter)
{
Sensor_WriteReg(0xfd, 0x01);
Sensor_WriteReg(0x03, (shutter >> 8) & 0xff);
Sensor_WriteReg(0x04, shutter & 0xff);
Sensor_WriteReg(0x01, 0x01); //Let the written value take effect immediately
}
//dummy_line
static uint32_t sp2609_read_dummy_line(void)
{
uint16_t dummy_line_h = 0;
uint16_t dummy_line_l = 0;
Sensor_WriteReg(0xfd, 0x01);//fd register selects the first page
shutter_h = Sensor_ReadReg(0x05) & amp; 0xff;//write 03 register (high 8 bits)
shutter_l = Sensor_ReadReg(0x06) & amp; 0xff;//write 04 register (lower 8 bits)
return (dummy_line_h << 8) | dummy_line_l;//Return the value of the upper 8 bits and the lower 8 bits
}
static void sp2609_write_dummy_line(uint32_t dummy_line)
{
Sensor_WriteReg(0xfd, 0x01);
Sensor_WriteReg(0x05, (dummy_line >> 8) & 0xff);
Sensor_WriteReg(0x06, dummy_line & 0xff);
Sensor_WriteReg(0x01, 0x01);
}
3. Solution:
vendor/sprd/modules/libcamera/sensor/sensor_drv/classic/Superpix/SP2609/sensor_sp2609_mipi_raw.c
//add dummy_line interface
+ static void sp2609_drv_write_dummy_line(cmr_handle handle, cmr_u32 dest_fr_len) {
SENSOR_IC_CHECK_HANDLE_VOID(handle);
struct sensor_ic_drv_cxt * sns_drv_cxt = (struct sensor_ic_drv_cxt *)handle;
if(dest_fr_len == 4326) {
//fix max fps = 10 in video call
hw_sensor_write_reg(sns_drv_cxt->hw_handle, 0xfd, 0x01);
hw_sensor_write_reg(sns_drv_cxt->hw_handle, 0x05, 0x0c);
hw_sensor_write_reg(sns_drv_cxt->hw_handle, 0x06, 0x25);
hw_sensor_write_reg(sns_drv_cxt->hw_handle, 0x01, 0x01);
} else {
//fix max fps = 30 in normol use camera
hw_sensor_write_reg(sns_drv_cxt->hw_handle, 0xfd, 0x01);
hw_sensor_write_reg(sns_drv_cxt->hw_handle, 0x05, 0x00);
hw_sensor_write_reg(sns_drv_cxt->hw_handle, 0x06, 0xe1);
hw_sensor_write_reg(sns_drv_cxt->hw_handle, 0x01, 0x01);
}
}
//Called when write_exposure_dummy
cmr_int sp2609_drv_write_exposure_dummy(cmr_handle handle,···)
···
+ if (dest_fr_len != cur_fr_len) {
+ write_sensor_dummy_line:
+ sp2609_drv_write_dummy_line(handle, dest_fr_len);
+ }
···
}
Video call out frame rate
Question 2. Under the yellow light or sunset environment, the problem of flashing red instantly during the photo taking process.
1. Phenomena
2. Root cause of the problem
This phenomenon is the same as when the Camera is started.
When the camera is started, it is judged that the current environment is a yellow light environment (according to the color temperature). At this time, the preview interface is reddish, and after the AWB (automatic white balance) algorithm adjustment, it will become whiter.
So why does this happen when taking pictures?
That is to say, when the camera is turned on or the data stream is switched when taking pictures, AE will readjust, which leads to this problem.
3. Solutions
1. Skipping frame
When taking pictures, during the process of data stream switching, we skip one frame to solve it.
Modifying this frame skip at the bottom layer is invalid.
Make changes on the ISP side
vendor/sprd/modules/libcamera/camdrv/isp2.3/driver/src/dcam_u_raw_aem.c
cmr_s32 dcam_u_raw_aem_skip_num(cmr_handle handle, cmr_u32 skip_num)
{
···
+ if(when taking a photo){
+ skip_num = 1
}
param.property_param = &skip_num;
···
}
We can’t simply and rudely set skip_num = 1, otherwise it will cause the AWB adjustment to be obviously slower when the camera is started. In fact, the dcam_u_raw_aem_skip_num function will be called when the camera is started or when taking pictures.
Therefore, we need to skip a frame when taking pictures, and not skip frames when starting up.
Stay hungry, Stay foolish!
———————
Author: c Feng_The day of the code
Source: CSDN
Original text: https://blog.csdn.net/justXiaoSha/article/details/86685116
Copyright statement: This article is the author’s original article, please attach the blog post link for reprinting!
Content analysis By: CSDN, CNBLOG blog post one-click reprint plug-in