.. include:: /keyword.rst .. _feature-g1200-cam-imgsensor-yuv-sensor: =============================== YUV Sensor (MediaTek Imgsensor) =============================== .. contents:: Sections :local: :depth: 2 .. note:: All command operations presented in this chapter are based on the IoT Yocto v23.1, |G1200-EVK-REF-BOARD| and Onsemi AP1302 ISP and AR0830 sensor. You might get different operation results depending on the platform you use. Camera Daughter Board --------------------- The YUV camera DTB for |G1200-EVK| is ``CAM DTB-D6``. It contains an Onsemi AP1302 ISP and an AR0830 Sensor. .. figure:: /_asset/sw_yocto_app-dev_camera_cam-dtb-d6.jpeg :align: center :scale: 30% Camera DTB D6 - Onsemi AP1302 ISP and AR0830 Sensor Connect The Camera To The EVK ----------------------------- There are 3 MIPI-CSI sockets on |G1200-EVK|, which are ``CSI0``, ``CSI1``, and ``CSI2``. In the following figure, the camera DTB is connected to the ``CSI0`` socket. .. figure:: /_asset/sw_yocto_app-dev_camera_g1200_connect-d6.png :align: center :scale: 30% Connect Camera DTB D6 to |G1200-EVK| The hardware connection is ``AR0830 ---> AP1302 ---> MIPI-CSI On EVK``. The camera daughter board can be configured in ISP mode and Bypass mode. In ISP mode, the images from the AR0830 camera module are passed to AP1302 ISP before being received by the EVK. In Bypass mode, the images from the AR0830 camera module are transferred to the EVK directly. By default, |IOT-YOCTO| only supports ISP mode. To configure the camera DTB D6 to ISP mode, you MUST set 6 jumpers and 1 camera sensor as shown in the figure below. The jumpers MUST be correctly set and the camera sensor MUST be connected to the CAMERA 0 slot. .. figure:: /_asset/sw_yocto_app-dev_camera_cam-dtb-d6-isp-mode.png :align: center :scale: 30% Camera DTB D6 - ISP mode .. warning:: Please make sure all the above settings, including the jumpers, camera slot, and MIPI-CSI socket, are correct. Otherwise, the I2C connection between the sensor, ISP, and SoC will fail. Select Camera Device Tree Blob Overlay -------------------------------------- The camera device is inactive by default, so the kernel has to load a specific device tree blob overlay to enable it. Please refer to the :ref:`bl33 (u-boot) section ` for more details. .. csv-filter:: YUV Camera DTBO for |G1200-EVK| :header-rows: 1 :file: ../../../../_asset/tables/camera-platform-sensor-dtbo.csv :include: {0: 'Genio 1200-EVK'} :exclude: {5: '[^1.*]', 6: '[^0.*]'} :included_cols: 0,1,2,3,4,7 .. prompt:: bash > auto > genio-flash --list-dtbo List of available DTBO: - camera-ar0830-ap1302-2lanes-csi0.dtbo - camera-ar0830-ap1302-csi0.dtbo - camera-ar0830-ap1302-csi1.dtbo - camera-ar0830-ap1302-csi2.dtbo - ... > genio-flash --load-dtbo camera-ar0830-ap1302-csi0.dtbo --load-dtbo gpu-mali.dtbo --load-dtbo apusys.dtbo --load-dtbo video.dtbo .. warning:: Please select the DTBO according to the CSI slot to which the camera sensor is connected. For example, if the camera is connected to the ``CSI0`` slot, please load the `dtbo` ``camera-ar0830-ap1302-csi0.dtbo``. Otherwise, the camera initialization will fail. Supported Formats And Sizes --------------------------- .. csv-table:: :header: "Platform", "Sensor", "Stream Type", "Size", "Frame rate", "Format", "MIPI Lanes" :widths: 10, 20, 5, 5, 5, 5, 5 :align: left "|G1200-EVK|", "Onsemi AP1302 & AR0830", "Preview", "1920x1080", "30", "UYVY", "4" "|G1200-EVK|", "Onsemi AP1302 & AR0830", "Preview", "2560x1440", "30", "UYVY", "4" "|G1200-EVK|", "Onsemi AP1302 & AR0830", "Preview", "3840x2160", "24", "UYVY", "4" "|G1200-EVK|", "Onsemi AP1302 & AR0830", "Preview", "1920x1080", "30", "UYVY", "2" "|G1200-EVK|", "Onsemi AP1302 & AR0830", "Preview", "2560x1440", "30", "UYVY", "2" "|G1200-EVK|", "Onsemi AP1302 & AR0830", "Preview", "3840x2160", "12", "UYVY", "2" .. note:: The supported format, resolution, and frame rate are related to the capability of the sensor and SoC. V4L2 Device Node ---------------- Camera ISP is a complicated feature that has a large number of device nodes. The camera ISP driver creates 123 video devices in total. - ``MTK-ISP-DIP-V4L2`` has 68 devices - ``mtk-cam`` has 54 devices - ``mtk-v4l2-camera`` has 1 device ``MTK-ISP-DIP-V4L2`` and ``mtk-cam`` are used by the camera ISP middleware. The user can just focus on ``mtk-v4l2-camera`` which is for streaming. .. prompt:: bash # auto # v4l2-ctl --list-devices ... mtk-cam (platform:16000000.camisp): (54 video devices) /dev/media2 mtk-v4l2-camera (platform:mtkcam0): /dev/video74 Launch Camera ------------- The V4L2 nodes with the name ``mtk-v4l2-camera`` can be used for streaming. For the YUV sensor, there's only 1 V4L2 device which is for the preview stream. The user can find the devices by the following command: .. prompt:: bash # auto # v4l2-ctl --list-device mtk-v4l2-camera (platform:mtkcam0): /dev/video74 # cat /sys/class/video4linux/video74/name mtk-v4l2-camera@0-Preview .. important:: The above v4l2 device node number is for example. The node number may change every time the system boots. To quickly fetch the correct node and assign the environment variable: .. prompt:: bash # auto # declare -a VIDEO_DEV=(`v4l2-ctl --list-devices | grep mtk-v4l2-camera -A 1 | grep video | tr -d "\n"`) # printf "Preview Node\t= ${VIDEO_DEV[0]}\n" Preview Node = /dev/video74 Section `Supported formats and Sizes`_ shows the supported formats and sizes. The user can also use the v4l2 utility, ``v4l2-ctl``, to list formats. .. prompt:: bash # auto # v4l2-ctl -d ${VIDEO_DEV[0]} --list-formats-ext ioctl: VIDIOC_ENUM_FMT Type: Video Capture Multiplanar [0]: 'UYVY' (UYVY 4:2:2) Size: Discrete 1920x1080 Interval: Discrete 0.033s (30.000 fps) ... To launch the camera, the user can use ``v4l2-ctl``, ``gst-launch-1.0``, or other V4L2 applications. - Use ``v4l2-ctl`` .. prompt:: bash # auto # v4l2-ctl -d ${VIDEO_DEV[0]} --set-fmt-video=width=1920,height=1080,pixelformat=UYVY --stream-mmap --stream-count=10 --stream-to=/tmp/ap1302.yuv --verbose - Use ``gst-launch-1.0`` .. prompt:: bash # auto # gst-launch-1.0 v4l2src device=${VIDEO_DEV[0]} ! video/x-raw,width=1920,height=1080,format=UYVY ! v4l2convert output-io-mode=dmabuf-import ! video/x-raw,width=1280,height=720 ! waylandsink sync=false .. figure:: /_asset/sw_yocto_app-dev_camera_yuv-show-on-weston.png :align: center :scale: 50% Show Preview stream on Weston .. note:: To use MediaTek video codec and MDP hardware, the user has to load ``video.dtbo``. For more details, please refer to :doc:`Genio 1200-EVK Video Codec `. .. _feature-g1200-cam-yuv-sensor-launch-time: Sensor Launch Time ------------------ In our camera operation sequence, we follow the steps below: * **Open Camera** 1. Power on the sensor 2. Release the reset pin 3. Configure the sensor settings 4. Stream on the sensor * **Close Camera** 1. Stream off the sensor 2. Hold the reset pin 3. Power off the sensor When using the AP1302 ISP, it is necessary to download a specific firmware binary during initialization. The launch time of the camera depends on the size of the binary file. The log provided below illustrates the start-up process of the AP1302 ISP and displays the corresponding launch time. .. figure:: /_asset/sw_yocto_app-dev_camera_ap1302-start-up-time.svg :align: center :scale: 100% The presented log provides an overview of the AP1302 ISP start-up process and focuses on the launch time, which primarily measures the duration it takes to transmit the firmware through I2C at a speed of 400KHz.