ArmSom——Camera Development Guide (1)

1. Introduction

• RK3588 from beginner to proficient
• Development board: ArmSoM-W3
• Kernel: 5.10.160
• OS: Debian11
• This article mainly introduces the camera related code configuration under the Rockchip platform and the path analysis of MIPI-CSI debugging.

Glossary:

• CSI (Camera Serial Interface): high-speed serial interface between the host processor and the camera module
• DSI (Display Serial Interface): high-speed serial interface between the host processor and the display module
• ISP (Image Signal Processor): This is the image signal processing module. Its main function is to perform post-processing on the signal output by the front-end image sensor. Relying on ISP, we can better restore on-site details under different optical conditions.
• VICAP (Video capture): video capture unit
• MIPI-DPHY: Controller in Rockchip chip that complies with MIPI-DPHY protocol.

2. MIPI-CSI basic concepts

MIPI: Mobile Industry Processor Interface (Mobile Industry Processor Interface) is an open standard for mobile application processors initiated by the MIPI Alliance.

CSI: The MIPI-CSI-2 protocol is a sub-protocol of the MIPI Alliance protocol and is specifically designed for the interface of camera chips.

Due to its high speed and low power consumption, the MIPI-CSI2 protocol has greatly supported the development of the high-definition camera field. There are two physical standards followed by the CSI-2 protocol, namely C-PHY and D-PHY.

The difference between D-PHY and C-PHY: From a practical point of view, it is mainly the difference between data lines and clock lines, as well as the transmission rate. C-PHY makes data transmission faster through certain technical improvements.

DPHY-v1.2 & amp; & amp;(D/C-PHY) DPHY-v2.0 used by Rockchip 3588.

3. MIPI-CSI hardware configuration

RK3588 camera resource hardware has 2 DCPHY, 2 DPHY, 1 DVP, 6 CSI HOST, a vicap controller, and 2 isp controllers. Among them, 2-way DPHY can be decomposed into 4x2lane mode to work.

The hardware path block diagram is as follows:

MIPI-CSI resource introduction:

Type	Max bandwidth	NUM	Mode
DPHY-v1.2	2.5Gbps x 4 lanes	2	4lane or 2lane + 2lane
(D/C-PHY) DPHY-v2.0	DPHY-v2.0: 2.5Gbps x 2lanes	2	DPHY-v2.0: 2lane
CSI-Host	For MIPI D-PHY v1.2/D-PHY v2.0 /C-PHY v1.1	6

The maximum bandwidth of 2lane is 5G, the resolution can reach 8M30 frames, and the maximum bandwidth of 4lane reaches 10G.

3. MIPI CSI usage

3.1 DPHY

1. rk3588 supports two dcphy, the node names are csi2_dcphy0/csi2_dcphy1. Each dcphy hardware supports RX/TX
   Used at the same time, RX is used for camera input. Supports DPHY/CPHY protocol reuse; it should be noted that the TX/RX of the same dcphy
   Can only use DPHY at the same time or CPHY at the same time

2. rk3588 supports 2 dphy hardware, here we call it dphy0_hw/dphy1_hw, both dphy hardware can work in full
   mode and split mode

?Full Mode:
? Only use csi2_dphy0, csi2_dphy0 and csi2_dphy1/csi2_dphy2 are mutually exclusive and cannot be used at the same time;

? data lane maximum 4 lanes;

? Maximum rate 2.5Gbps/lane;

? Split Mode:
? Only use csi2_dphy1 and csi2_dphy2, which are mutually exclusive with csi2_dphy0 and cannot be used at the same time;

? csi2_dphy1 and csi2_dphy2 can be used at the same time;

? The maximum data lane of csi2_dphy1 and csi2_dphy2 is 2 lanes;

? csi2_dphy1 corresponds to lane0/lane1 of physical dphy;

? csi2_dphy2 corresponds to lane2/lane3 of physical dphy;

? Maximum rate 2.5Gbps/lane;

3.2 Multi-sensor support

The hardware supports up to 7 channels of sensor acquisition: 6mipi + 1dvp. The multi-sensor software paths are as follows:

4. Camera related configuration

4.1 Camera Software driver directory

Camera related driver files are as follows:

|– arch/arm/boot/dts DTS configuration file
|– drivers/phy/rockchip
|– phy-rockchip-mipi-rx.c mipi dphy driver
|– phy-rockchip-csi2-dphy-common.h
|– phy-rockchip-csi2-dphy-hw.c
|– phy-rockchip-csi2-dphy.c
|– drivers/media
|– platform/rockchip/cif RKCIF driver
|– platform/rockchip/isp RKISP driver
|– dev includes probe, asynchronous registration, clock, pipeline, iommu and media/v4l2 framework
|– capture includes mp/sp/rawwr configuration and vb2, frame interrupt processing
|– dmarx includes rawrd configuration and vb2, frame interrupt processing
|– isp_params 3A related parameter settings
|– isp_stats 3A related statistics
|– isp_mipi_luma mipi data brightness statistics
|– Read and write operations related to regs register
|– rkisp isp subdev and entity registration
|– csi csi subdev and mipi configuration
|– bridge bridge subdev, isp and ispp interactive bridge
|– platform/rockchip/ispp rkispp driver
|– dev includes probe, asynchronous registration, clock, pipeline, iommu and media/v4l2 framework
|– stream includes 4-channel video output configuration and vb2, frame interrupt processing
|– rkispp ispp subdev and entity registration
|– params TNR/NR/SHP/FEC/ORB parameter settings
|– stats ORB statistics
|– i2c
|– imx415.c CIS (cmos image sensor) driver

4.2 Sensor driver development and transplantation

The Sensor driver is located in the drivers/media/i2c directory. The Sensor driver is independent from the RKCIF or RKISP1 driver to the greatest extent. The two are registered asynchronously and are declared by remote-endpoint in dts.
connection relationship.

Sensor driver development and transplantation are summarized in 5 parts

• Write the power-on sequence according to the datasheet, mainly including vdd, reset, powerdown, clk, etc.
• Configure the sensor’s register to output the required resolution and format
• Write the callback functions required for struct v4l2_subdev_ops, generally including set_fmt, get_fmt, s_stream, s_power
• Add v4l2 controller to set fps, exposure, gain, test pattern
• Write the probe() function and add Media Control and Sub Device initialization code

Documentation/devicetree/bindings/media/i2c/The following is the driver’s Documentation for reference. Board-level dts can be quickly configured according to this document.

In board-level dts, to reference the Sensor driver, you generally need:

• Configure the correct clk and io mux
• Set the regulator and gpio required for the power-on sequence according to the schematic diagram
• Add port sub-node to establish connection with cif or isp

4.2.1 Power-on sequence

Different Sensors have different power-on timing requirements. Most Sensors may not have strict timing requirements. As long as the mclk, vdd, reset and powerdown states are correct, I2C communication can be performed correctly and pictures can be output without worrying about the power-on sequence. Regarding the sequence and delay, the DataSheet provided by the Sensor manufacturer generally has a power-on sequence diagram, which only needs to be configured in sequence.

__imx415_power_on() //Control the power-on sequence part __imx415_power_off() //Control the power-off sequence part

In the probe() stage, it will try to read the chip id, such as imx415_check_sensor_id of imx415.

ret = __imx415_power_on(imx415);
if (ret)
goto err_free_handler;

ret = imx415_check_sensor_id(imx415, client);
if (ret)
goto err_power_off;

If the chip ID can be read correctly, it is generally considered that the power-on sequence is correct and the Sensor can communicate normally with i2c.

Generally, problems occur in imx415_check_sensor_id(). Please troubleshoot as follows:

• First comment out __imx415_power_off()
• Check i2c slave address, i2c read function
• You can capture the i2c waveform

4.2.2 Sensor initialization register list

In the imx415 driver, struct imx415_mode supported_modes[] is defined to represent the different initialization modes supported by the Sensor, that is, the Sensor can output images of different resolutions, different fps, etc. Mode can include resolution, Mbus Code, fps, register initialization list, etc.

static const struct imx415_mode supported_modes[] = {
/*
* frame rate = 1 / (Vtt * 1H) = 1 / (VMAX * 1H)
* VMAX >= (PIX_VWIDTH / 2) + 46 = height + 46
*/
...
{
/* 1H period = (1100 clock) = (1100 * 1 / 74.25MHz) */
.bus_fmt = MEDIA_BUS_FMT_SGBRG12_1X12,
.width = 3864,
.height = 2192,
.max_fps = {
.numerator = 10000,
.denominator = 300000,
},
.exp_def = 0x08ca - 0x08,
.hts_def = 0x044c * IMX415_4LANES * 2,
.vts_def = 0x08ca,
.global_reg_list = imx415_global_12bit_3864x2192_regs,
.reg_list = imx415_linear_12bit_3864x2192_891M_regs,
.hdr_mode = NO_HDR,
.mipi_freq_idx = 1,
.bpp = 12,
.vc[PAD0] = V4L2_MBUS_CSI2_CHANNEL_0,
},
{
.bus_fmt = MEDIA_BUS_FMT_SGBRG12_1X12,
.width = 3864,
.height = 2192,
.max_fps = {
.numerator = 10000,
.denominator = 300000,
},
.exp_def = 0x08CA * 2 - 0x0d90,
.hts_def = 0x0226 * IMX415_4LANES * 2,
/*
* IMX415 HDR mode T-line is half of Linear mode,
* make vts double(that is FSC) to workaround.
*/
.vts_def = 0x08CA * 2,
.global_reg_list = imx415_global_12bit_3864x2192_regs,
.reg_list = imx415_hdr2_12bit_3864x2192_1782M_regs,
.hdr_mode = HDR_X2,
.mipi_freq_idx = 3,
.bpp = 12,
.vc[PAD0] = V4L2_MBUS_CSI2_CHANNEL_1,
.vc[PAD1] = V4L2_MBUS_CSI2_CHANNEL_0,//L->csi wr0
.vc[PAD2] = V4L2_MBUS_CSI2_CHANNEL_1,
.vc[PAD3] = V4L2_MBUS_CSI2_CHANNEL_1,//M->csi wr2
},
...
};

Many resolution configurations of imx415 modules are supported here. The default is 3864×2192@30fp. Looking at the code below, you can see that one resolution will be used as the default configuration:

ret = of_property_read_u32(node, OF_CAMERA_HDR_MODE, & amp;hdr_mode);
if (ret) {
hdr_mode = NO_HDR;
dev_warn(dev, "Get hdr mode failed! no hdr default\\
");
}
imx415->client = client;
imx415->cfg_num = ARRAY_SIZE(supported_modes);
for (i = 0; i < imx415->cfg_num; i + + ) {
if (hdr_mode == supported_modes[i].hdr_mode) {
imx415->cur_mode = & amp;supported_modes[i];
break;
}
}

Adapting to new resolutions requires replacing a new initialization list. For example, the imx415 camera only supports 30 frames, but needs to be increased to 60 frames.

The maximum mipi transmission rate supported by the sensor itself, the amount of data transmitted per second is width * height * 10bit * fps. This also has an upper limit and cannot be increased without limit.
What is written on the DateSheet of IMX415 is that it supports 3864×2192@30fps. To adjust to 60 frames, the manufacturer needs to provide a set of low-resolution sensor configurations, such as 1080P@60fps, and then add it to the .reg_list of struct imx415_mode supported_modes[], reg_list list Finally, REG_NULL is used to indicate the end.

4.2.3 Callback function

The v4l2_subdev_ops callback function is the core of logic control in the Sensor driver and contains rich interfaces for upper-layer applications to call.

static const struct v4l2_subdev_ops imx415_subdev_ops = {
.core = &imx415_core_ops,
.video = &imx415_video_ops,
.pad = &imx415_pad_ops,
};

Some member functions:

1. open – When Userspace opens the /dev/v4l-subdev? node, the .open() function will be called.
2. s_power – includes power on and power off. Power on or off here
3. enum_mbus_code – used to enumerate supported Media Bus format codes. Media bus is the standard used in V4L2 systems to describe image formats. This function may list the different media bus format codes supported by the IMX415 sensor.
4. enum_frame_size – Used to enumerate the different frame sizes (resolutions) supported by the sensor. This function may return a set of available frame size options for the application to choose from.
5. enum_frame_interval – Used to enumerate the different frame intervals (frame rates) supported by the sensor. It returns the available frame interval options for the application to choose from.
6. get_fmt – used to get the image format of the current sensor. Applications can use this to query the currently set image format.
7. set_fmt – used to set the image format of the sensor. Applications can use this to configure the desired image format.
8. get_selection – used to get the image selection (ROI – Region of Interest) of the current sensor. This allows the application to understand the current region of interest settings.
9. get_mbus_config – used to obtain media bus configuration, including data bus width, timing and other information.

These functions play a key role in the V4L2 subdevice driver, allowing applications to configure and control the IMX415 sensor to capture image and video data.

4.3 DTS configuration

Here are the dts configuration instructions for a single camera, taking the imx415 camera as an example.

• Case scenario: The single-channel camera configuration of csi2_dphy0 is used here:
• Link configuration: imx415 -> csi2_dphy0 -> mipi2_csi2 -> rkcif_mipi_lvds2 ->rkcif_mipi_lvds2_sditf ->rkisp0_vir2

Under this path, the two nodes medio0 and medio1 will be registered.

4.3.1 Configure sensor end

We need to find which I2C bus the sensor is connected to based on the MIPI CSI interface of the board schematic diagram, then configure the camera node on the corresponding I2C node, and correctly configure the I2C device address, pins and other attributes of the camera module. The corresponding driver path for the sensor is under kernel\drivers\media\i2c.
The following is the imx415 configuration:

 &i2c3 {
status = "okay";

imx415: imx415@1a {
status = "okay";
compatible = "sony,imx415"; // Needs to be consistent with the matching string in the driver
reg = ?; // sensor I2C device address, 7 bits
clocks = < & amp;cru CLK_MIPI_CAMARAOUT_M3>; // sensor mclk source configuration
clock-names = "xvclk";
pinctrl-names = "default";
pinctrl-0 = < & amp;mipim0_camera3_clk>; //sensor related power domain enable
power-domains = < & amp;power RK3588_PD_VI>;
pwdn-gpios = < & amp;gpio1 RK_PB0 GPIO_ACTIVE_HIGH>;
reset-gpios = < & amp;gpio4 RK_PA0 GPIO_ACTIVE_LOW>;
rockchip,camera-module-index = <0>;
rockchip,camera-module-facing = "back"; //Module facing, including "back" and "front"
rockchip,camera-module-name = "CMK-OT2022-PX1";
rockchip,camera-module-lens-name = "IR0147-50IRC-8M-F20";
port {
imx415_out0: endpoint {
remote-endpoint = < & amp;mipidphy0_in_ucam0>;
data-lanes = <1 2 3 4>;
};
};
};
};

Notice:

data-lanes must specify the specific number of lanes used, otherwise it cannot be recognized as a mipi type;

module-index is related to the moduleId in the iq file. <0> corresponds to moduleId configured as m00, <1> corresponds to moduleId configured as m01, m is the abbreviation of “module”, and 01 is a decimal number;

The module-name and module-lens-name commands are the same as the end of the iq file name of the corresponding sensor in the device/etc/iqfiles. The corresponding Sensor iq file here is “imx415_CMK-OT2022-PX1_IR0147-50IRC-8M-F20.json”, please note that there is a case difference

4.3.2 csi2_dphy0 configuration

csi2_dphy0 and csi2_dphy1/csi2_dphy2 are mutually exclusive and cannot be used at the same time. In addition, the csi2_dphy0_hw physical node needs to be enabled.

 &csi2_dphy0_hw {<!-- -->
status = "okay";
};

 & amp;csi2_dphy0 {<!-- -->
status = "okay";

ports {<!-- -->
#address-cells = <1>;
#size-cells = <0>;

port@0 {<!-- -->
reg = <0>;
#address-cells = <1>;
#size-cells = <0>;

mipidphy0_in_ucam0: endpoint@1 {<!-- -->
reg = <1>;
remote-endpoint = < & amp;imx415_out0>; // The port name of the sensor side
data-lanes = <1 2 3 4>;
};

};

port@1 {<!-- -->
reg = <1>;
#address-cells = <1>;
#size-cells = <0>;

csidphy0_out: endpoint@0 {<!-- -->
reg = <0>;
remote-endpoint = < & amp;mipi2_csi2_input>; csi2 host port name
};
};
};
};

 & amp;mipi2_csi2 {<!-- -->
status = "okay";

ports {<!-- -->
#address-cells = <1>;
#size-cells = <0>;

port@0 {<!-- -->
reg = <0>;
#address-cells = <1>;
#size-cells = <0>;

mipi2_csi2_input: endpoint@1 {<!-- -->
reg = <1>;
remote-endpoint = < & amp;csidphy0_out>;
};
};

port@1 {<!-- -->
reg = <1>;
#address-cells = <1>;
#size-cells = <0>;

mipi2_csi2_output: endpoint@0 {<!-- -->
reg = <0>;
remote-endpoint = < & amp;cif_mipi2_in0>;
};
};
};
};

 & amp;rkcif {<!-- -->
status = "okay";
};

 & amp;rkcif_mmu {<!-- -->
status = "okay";
};

 & amp;rkcif_mipi_lvds2 {<!-- -->
status = "okay";

port {<!-- -->
cif_mipi2_in0: endpoint {<!-- -->
remote-endpoint = < & amp;mipi2_csi2_output>;
};
};
};

4.3.3 isp related configuration

 & amp;rkcif_mipi_lvds2_sditf {
status = "okay";

port {
mipi_lvds2_sditf: endpoint {
remote-endpoint = < & amp;isp0_vir0>;
};
};
};

 & amp;rkisp0 {
status = "okay";
};

 & amp;isp0_mmu {
status = "okay";
};

 & amp;rkisp0_vir0 {
status = "okay";

port {
#address-cells = <1>;
#size-cells = <0>;

isp0_vir0: endpoint@0 {
reg = <0>;
remote-endpoint = < & amp;mipi_lvds2_sditf>;
};
};
};

An ISP can connect multiple Sensors, but they can only be multiplexed in a time-sharing manner. By configuring dts, after linking multiple Sensors to MIPI DPHY, you can switch Sensors through media-ctl.

4.4 Multi-camera configuration

The above is the configuration of a single 4lan camera, with DPHY in Full Mode. Here is an example of configuring four 2lan cameras with DPHY in Split Mode:

// SPDX-License-Identifier: (GPL-2.0 + OR MIT)
/*
 * Copyright (c) 2021 Rockchip Electronics Co., Ltd.
 *
 */
/{
vcc_mipicsi0: vcc-mipicsi0-regulator {
compatible = "regulator-fixed";
gpio = < & amp;gpio1 RK_PB1 GPIO_ACTIVE_HIGH>;
pinctrl-names = "default";
pinctrl-0 = < & amp;mipicsi0_pwr>;
regulator-name = "vcc_mipicsi0";
enable-active-high;
};

vcc_mipicsi1: vcc-mipicsi1-regulator {
compatible = "regulator-fixed";
gpio = < & amp;gpio1 RK_PB2 GPIO_ACTIVE_HIGH>;
pinctrl-names = "default";
pinctrl-0 = < & amp;mipicsi1_pwr>;
regulator-name = "vcc_mipicsi1";
enable-active-high;
};
};

 &pinctrl {
cam {
mipicsi0_pwr: mipicsi0-pwr {
rockchip, pins =
<1 RK_PB1 RK_FUNC_GPIO & amp;pcfg_pull_none>;
};

mipicsi1_pwr: mipicsi1-pwr {
rockchip, pins =
<1 RK_PB2 RK_FUNC_GPIO & amp;pcfg_pull_none>;
};
};
};

 & amp;csi2_dphy0_hw {
status = "okay";
};

 & amp;csi2_dphy1_hw {
status = "okay";
};

 & amp;csi2_dphy1 {
status = "okay";

ports {
#address-cells = <1>;
#size-cells = <0>;

port@0 {
reg = <0>;
#address-cells = <1>;
#size-cells = <0>;

mipi_in_ucam2: endpoint@1 {
reg = <1>;
remote-endpoint = < & amp;imx464_out2>;
data-lanes = <1 2>;
};
};

port@1 {
reg = <1>;
#address-cells = <1>;
#size-cells = <0>;

csidphy1_out: endpoint@0 {
reg = <0>;
remote-endpoint = < & amp;mipi2_csi2_input>;
};
};
};
};

 & amp;csi2_dphy2 {
status = "okay";

ports {
#address-cells = <1>;
#size-cells = <0>;

port@0 {
reg = <0>;
#address-cells = <1>;
#size-cells = <0>;

mipi_in_ucam3: endpoint@1 {
reg = <1>;
remote-endpoint = < & amp;imx464_out3>;
data-lanes = <1 2>;
};
};

port@1 {
reg = <1>;
#address-cells = <1>;
#size-cells = <0>;

csidphy2_out: endpoint@0 {
reg = <0>;
remote-endpoint = < & amp;mipi3_csi2_input>;
};
};
};
};

 & amp;csi2_dphy4 {
status = "okay";

ports {
#address-cells = <1>;
#size-cells = <0>;

port@0 {
reg = <0>;
#address-cells = <1>;
#size-cells = <0>;

mipi_in_ucam4: endpoint@1 {
reg = <1>;
remote-endpoint = < & amp;imx464_out4>;
data-lanes = <1 2>;
};
};

port@1 {
reg = <1>;
#address-cells = <1>;
#size-cells = <0>;

csidphy4_out: endpoint@0 {
reg = <0>;
remote-endpoint = < & amp;mipi4_csi2_input>;
};
};
};
};

 & amp;csi2_dphy5 {
status = "okay";

ports {
#address-cells = <1>;
#size-cells = <0>;

port@0 {
reg = <0>;
#address-cells = <1>;
#size-cells = <0>;

mipi_in_ucam5: endpoint@1 {
reg = <1>;
remote-endpoint = < & amp;imx464_out5>;
data-lanes = <1 2>;
};
};

port@1 {
reg = <1>;
#address-cells = <1>;
#size-cells = <0>;

csidphy5_out: endpoint@0 {
reg = <0>;
remote-endpoint = < & amp;mipi5_csi2_input>;
};
};
};
};

 & amp;i2c5 {
status = "okay";

pinctrl-0 = < & amp;i2c5m3_xfer>;
/* module 77/79 0x1a 78/80 0x36 */
imx464_2: imx464-2@1a {
compatible = "sony,imx464";
status = "okay";
reg = ?;
clocks = < & amp;cru CLK_MIPI_CAMARAOUT_M3>;
clock-names = "xvclk";
power-domains = < & amp;power RK3588_PD_VI>;
pinctrl-names = "default";
pinctrl-0 = < & amp;mipim0_camera3_clk>;
avdd-supply = < & amp;vcc_mipicsi0>;
reset-gpios = < & amp;gpio1 RK_PA4 GPIO_ACTIVE_HIGH>;
pwdn-gpios = < & amp;gpio1 RK_PB3 GPIO_ACTIVE_HIGH>;
rockchip,camera-module-sync-mode = "internal_master";
rockchip,camera-module-index = <2>;
rockchip,camera-module-facing = "back";
rockchip,camera-module-name = "CMK-OT1980-PX1";
rockchip,camera-module-lens-name = "SHG102";
port {
imx464_out2: endpoint {
remote-endpoint = < & amp;mipi_in_ucam2>;
data-lanes = <1 2>;
};
};
};

imx464_3: imx464-3@36 {
compatible = "sony,imx464";
status = "okay";
reg = 6;
clocks = < & amp;cru CLK_MIPI_CAMARAOUT_M3>;
clock-names = "xvclk";
power-domains = < & amp;power RK3588_PD_VI>;
avdd-supply = < & amp;vcc_mipicsi0>;
pwdn-gpios = < & amp;gpio1 RK_PA7 GPIO_ACTIVE_HIGH>;
rockchip,camera-module-sync-mode = "external_master";
rockchip,camera-module-index = <3>;
rockchip,camera-module-facing = "back";
rockchip,camera-module-name = "CMK-OT1980-PX1";
rockchip,camera-module-lens-name = "SHG102";
port {
imx464_out3: endpoint {
remote-endpoint = < & amp;mipi_in_ucam3>;
data-lanes = <1 2>;
};
};
};
};

 & amp;i2c4 {
status = "okay";

pinctrl-0 = < & amp;i2c4m3_xfer>;
/* 77/79 0x1a 78/80 0x36 */
imx464_4: imx464-4@1a {
compatible = "sony,imx464";
status = "okay";
reg = ?;
clocks = < & amp;cru CLK_MIPI_CAMARAOUT_M4>;
clock-names = "xvclk";
power-domains = < & amp;power RK3588_PD_VI>;
pinctrl-names = "default";
pinctrl-0 = < & amp;mipim0_camera4_clk>;
avdd-supply = < & amp;vcc_mipicsi1>;
reset-gpios = < & amp;gpio1 RK_PB5 GPIO_ACTIVE_HIGH>;
pwdn-gpios = < & amp;gpio1 RK_PB4 GPIO_ACTIVE_HIGH>;
rockchip,camera-module-sync-mode = "external_master";
rockchip,camera-module-index = <0>;
rockchip,camera-module-facing = "back";
rockchip,camera-module-name = "CMK-OT1980-PX1";
rockchip,camera-module-lens-name = "SHG102";
port {
imx464_out4: endpoint {
remote-endpoint = < & amp;mipi_in_ucam4>;
data-lanes = <1 2>;
};
};
};

imx464_5: imx464-5@36 {
compatible = "sony,imx464";
status = "okay";
reg = 6;
clocks = < & amp;cru CLK_MIPI_CAMARAOUT_M4>;
clock-names = "xvclk";
power-domains = < & amp;power RK3588_PD_VI>;
avdd-supply = < & amp;vcc_mipicsi1>;
pwdn-gpios = < & amp;gpio1 RK_PB0 GPIO_ACTIVE_HIGH>;
rockchip,camera-module-sync-mode = "external_master";
rockchip,camera-module-index = <1>;
rockchip,camera-module-facing = "back";
rockchip,camera-module-name = "CMK-OT1980-PX1";
rockchip,camera-module-lens-name = "SHG102";
port {
imx464_out5: endpoint {
remote-endpoint = < & amp;mipi_in_ucam5>;
data-lanes = <1 2>;
};
};
};
};

 & amp;mipi2_csi2 {
status = "okay";

ports {
#address-cells = <1>;
#size-cells = <0>;

port@0 {
reg = <0>;
#address-cells = <1>;
#size-cells = <0>;

mipi2_csi2_input: endpoint@1 {
reg = <1>;
remote-endpoint = < & amp;csidphy1_out>;
};
};

port@1 {
reg = <1>;
#address-cells = <1>;
#size-cells = <0>;

mipi2_csi2_output: endpoint@0 {
reg = <0>;
remote-endpoint = < & amp;cif_mipi_in2>;
};
};
};
};

 & amp;mipi3_csi2 {
status = "okay";

ports {
#address-cells = <1>;
#size-cells = <0>;

port@0 {
reg = <0>;
#address-cells = <1>;
#size-cells = <0>;

mipi3_csi2_input: endpoint@1 {
reg = <1>;
remote-endpoint = < & amp;csidphy2_out>;
};
};

port@1 {
reg = <1>;
#address-cells = <1>;
#size-cells = <0>;

mipi3_csi2_output: endpoint@0 {
reg = <0>;
remote-endpoint = < & amp;cif_mipi_in3>;
};
};
};
};

 & amp;mipi4_csi2 {
status = "okay";

ports {
#address-cells = <1>;
#size-cells = <0>;

port@0 {
reg = <0>;
#address-cells = <1>;
#size-cells = <0>;

mipi4_csi2_input: endpoint@1 {
reg = <1>;
remote-endpoint = < & amp;csidphy4_out>;
};
};

port@1 {
reg = <1>;
#address-cells = <1>;
#size-cells = <0>;

mipi4_csi2_output: endpoint@0 {
reg = <0>;
remote-endpoint = < & amp;cif_mipi_in4>;
};
};
};
};

 & amp;mipi5_csi2 {
status = "okay";

ports {
#address-cells = <1>;
#size-cells = <0>;

port@0 {
reg = <0>;
#address-cells = <1>;
#size-cells = <0>;

mipi5_csi2_input: endpoint@1 {
reg = <1>;
remote-endpoint = < & amp;csidphy5_out>;
};
};

port@1 {
reg = <1>;
#address-cells = <1>;
#size-cells = <0>;

mipi5_csi2_output: endpoint@0 {
reg = <0>;
remote-endpoint = < & amp;cif_mipi_in5>;
};
};
};
};

 &rkcif {
status = "okay";
};

 & amp;rkcif_mipi_lvds2 {
status = "okay";

port {
cif_mipi_in2: endpoint {
remote-endpoint = < & amp;mipi2_csi2_output>;
};
};
};

 & amp;rkcif_mipi_lvds2_sditf {
status = "okay";

port {
mipi2_lvds_sditf: endpoint {
remote-endpoint = < & amp;isp0_vir0>;
};
};
};

 & amp;rkcif_mipi_lvds3 {
status = "okay";

port {
cif_mipi_in3: endpoint {
remote-endpoint = < & amp;mipi3_csi2_output>;
};
};
};

 & amp;rkcif_mipi_lvds3_sditf {
status = "okay";

port {
mipi3_lvds_sditf: endpoint {
remote-endpoint = < & amp;isp1_vir0>;
};
};
};

 & amp;rkcif_mipi_lvds4 {
status = "okay";

port {
cif_mipi_in4: endpoint {
remote-endpoint = < & amp;mipi4_csi2_output>;
};
};
};

 & amp;rkcif_mipi_lvds4_sditf {
status = "okay";

port {
mipi4_lvds_sditf: endpoint {
remote-endpoint = < & amp;isp0_vir1>;
};
};
};

 & amp;rkcif_mipi_lvds5 {
status = "okay";

port {
cif_mipi_in5: endpoint {
remote-endpoint = < & amp;mipi5_csi2_output>;
};
};
};

 & amp;rkcif_mipi_lvds5_sditf {
status = "okay";

port {
mipi5_lvds_sditf: endpoint {
remote-endpoint = < & amp;isp1_vir1>;
};
};
};

 & amp;rkcif_mmu {
status = "okay";
};

 & amp;rkisp0 {
status = "okay";
};

 & amp;isp0_mmu {
status = "okay";
};

 & amp;rkisp0_vir0 {
status = "okay";

port {
#address-cells = <1>;
#size-cells = <0>;

isp0_vir0: endpoint@0 {
reg = <0>;
remote-endpoint = < & amp;mipi2_lvds_sditf>;
};
};
};

 & amp;rkisp0_vir1 {
status = "okay";

port {
#address-cells = <1>;
#size-cells = <0>;

isp0_vir1: endpoint@0 {
reg = <0>;
remote-endpoint = < & amp;mipi4_lvds_sditf>;
};
};
};

 & amp;rkisp1 {
status = "okay";
};

 & amp;isp1_mmu {
status = "okay";
};

 & amp;rkisp1_vir0 {
status = "okay";

port {
#address-cells = <1>;
#size-cells = <0>;

isp1_vir0: endpoint@0 {
reg = <0>;
remote-endpoint = < & amp;mipi3_lvds_sditf>;
};
};
};

 & amp;rkisp1_vir1 {
status = "okay";

port {
#address-cells = <1>;
#size-cells = <0>;

isp1_vir1: endpoint@0 {
reg = <0>;
remote-endpoint = < & amp;mipi5_lvds_sditf>;
};
};
};

Link configuration:

imx464 _1-> csi2_dphy1 -> mipi2_csi2 -> rkcif_mipi_lvds2 -> rkcif_mipi_lvds2_sditf -> rkisp0_vir0

imx464 _2-> csi2_dphy2 -> mipi3_csi2 -> rkcif_mipi_lvds3 -> rkcif_mipi_lvds3_sditf -> rkisp0_vir1

imx464 _3-> csi2_dphy4 -> mipi4_csi2 -> rkcif_mipi_lvds4 -> rkcif_mipi_lvds4_sditf -> rkisp1_vir0

imx464 _4-> csi2_dphy5 -> mipi5_csi2 -> rkcif_mipi_lvds5 -> rkcif_mipi_lvds5_sditf -> rkisp1_vir0

Four cameras of the same model are configured here. If there are other types of cameras that need to be added, update and replace the corresponding sensor drivers.

5. Conclusion

Here we borrow the imx415 module on the ArmSoM-W3 development board to introduce some configurations of the camera on the rockchip platform. When developing and configuring the camera driver, each specific model of camera may have its own unique settings and requirements. If you have questions about other camera models or other embedded system components, or need more customization help, you can ask questions on the ArmSom forum to share experiences with other developers and get support.