.. include:: /keyword.rst .. _feature-g1200-cam-v4l2-yuv-sensor: ======================== YUV Sensor (V4L2 Sensor) ======================== .. contents:: Sections :local: :depth: 2 .. note:: All command operations presented in this chapter are based on the IoT Yocto v23.2, |G1200-EVK-REF-BOARD| and Onsemi AP1302 ISP and AR0830 sensor. You might get different operation results depending on the platform you use. This chapter shows how to receive YUV sensor data and directly dump it to the DRAM under **V4L2 Sensor** architecture on |G1200-EVK|. There are 6x camera DMA engines in Genio 1200. Camera Daughter Board --------------------- The YUV camera DTB for |G1200-EVK| is ``CAM DTB-D6``. It contains Onsemi AP1302 ISP and AR0830 Sensor. .. figure:: /_asset/sw_yocto_app-dev_camera_cam-dtb-d6.jpeg :align: center :scale: 30% Camera DTB D6 - Onsemi AP1302 ISP and AR0830 Sensor Connect The Camera To The EVK ----------------------------- There are 3 MIPI-CSI sockets on |G1200-EVK|, which are ``CSI0``, ``CSI1``, and ``CSI2``. In the following figure, the camera DTB is connected to the ``CSI0`` socket. .. figure:: /_asset/sw_yocto_app-dev_camera_g1200_connect-d6.png :align: center :scale: 30% Connect Camera DTB D6 to |G1200-EVK| The hardware connection is ``AR0830 ---> AP1302 ---> MIPI-CSI On EVK``. The camera daughter board can be configured in ISP mode and Bypass mode. In ISP mode, the images from the AR0830 camera module are passed to AP1302 ISP before being received by the EVK. In Bypass mode, the images from the AR0830 camera module are transferred to the EVK directly. By default, |IOT-YOCTO| only supports ISP mode. To configure the camera DTB D6 to ISP mode, you MUST set 6 jumpers and 1 camera sensor as shown in the figure below. The jumpers MUST be correctly set and the camera sensor MUST be connected to the CAMERA 0 slot. .. figure:: /_asset/sw_yocto_app-dev_camera_cam-dtb-d6-isp-mode.png :align: center :scale: 30% Camera DTB D6 - ISP mode .. warning:: Please make sure all the above settings, including the jumpers, camera slot, and MIPI-CSI socket, are correct. Otherwise, the I2C connection between the sensor, ISP, and SoC will fail. Select Camera Device Tree Blob Overlay -------------------------------------- The camera device is inactive by default, so the kernel has to load a specific device tree blob overlay to enable it. Please refer to the :ref:`bl33 (u-boot) section ` for more details. The following table shows the supported multi-sensor combinations in V4L2 sensor architecture. IT6510FN and LT6911UXE are the proof-of-concept components. Please refer to :ref:`Proof-of-Concept ` for the usage. .. csv-filter:: YUV Camera DTBO for |G1200-EVK| :header-rows: 1 :file: ../../../../_asset/tables/camera-platform-sensor-dtbo.csv :include: {0: 'Genio 1200-EVK'} :exclude: {5: '[^1.*]', 6: '[^1.*]'} :included_cols: 0,1,2,3,4,7 .. prompt:: bash > auto > genio-flash --list-dtbo List of available DTBO: - camera-ar0830-ap1302-csi0-std.dtbo - camera-it6510-csi0-std.dtbo - camera-lt6911uxe-csi0-std.dtbo - ... > genio-flash --load-dtbo camera-ar0830-ap1302-csi0-std.dtbo --load-dtbo gpu-mali.dtbo --load-dtbo apusys.dtbo --load-dtbo video.dtbo Supported Formats And Sizes --------------------------- The array of supported formats and sizes depends on the two primary aspects: #. The capabilities of the sensor and the corresponding driver implementation. #. The capabilities of the SoC (System on a Chip) and the corresponding driver implementation. A couple of examples would further explain this: #. An external ISP (Image Signal Processor) such as the Onsemi AP1302 can scale the size of sensor images up to a size of 13MP. It can then export these images in some formats, including YUV422, YUV420, RGB888, etc. However, the SoC can only accept the YUV422-8bit format and can process images up to 16MP in size. It is also crucial to consider other influential factors such as the throughput of the MIPI C/D-PHY data lane. #. The ITE IT6510 is a bridge IC converting the input from a DP signal into MIPI CSI (Video) and I2S (Audio) signals. It can support up to an FHD@120Hz, 4K@30Hz DP signal, but the signal is also somewhat dependent on the DP signal source. The sizes reported by the IT6510 driver are from the IT6510 hardware, which also represents the size of the DP source. The degree of flexibility for setting the format and the size is subject to the sensor design and the driver implementation. Some sensors (e.g., ITE IT6510) provide a fixed set of formats and sizes, while some (e.g., Onsemi AP1302) can scale up/down the size in a wide range. Regardless of how the sensor determines its format and size, if the SoC supports those specifications, the user can set the SoC through the standard V4L2 interface and obtain the images from the video device node. Media Device ------------ The media device represents the internal topology of a media system like ISP. The topology setting including links, pads, and entities can be set through the media device. Therefore, the first step is to find the target media device node. .. prompt:: bash # auto # export MEDIA_DEV=$(v4l2-ctl -z platform:$(basename `find /sys/bus/platform/devices/ -name "*camisp"`) --list-devices | grep media) # echo ${MEDIA_DEV} /dev/media3 .. warning:: The assigned node number for the media device may vary depending on the sequence of the probing process during the boot-up phase. As such, it is crucial to ensure the accuracy of the media device node before using the camera. Video Device ------------ The video device is the interface for users to obtain the images. On |G1200-EVK|, they are called ``mtk-cam camsv-* main-stream``. The ``media-ctl`` is a useful tool to list video devices. .. prompt:: bash # auto # declare -a VIDEO_DEV=(`for i in {0..5}; do media-ctl -d ${MEDIA_DEV} --entity "mtk-cam camsv-$i main-stream"; done | tr "\n" " "`) # echo ${VIDEO_DEV[*]} /dev/video53 /dev/video54 /dev/video55 /dev/video56 /dev/video57 /dev/video58 .. warning:: The assigned node number for the video device may vary depending on the sequence of the probing process during the boot-up phase. As such, it is crucial to ensure the accuracy of the video device node before using the camera. Launch Camera ------------- The initial step is to link all the necessary components within the media device. The structure of the YUV camera pipeline is quite straightforward. It fundamentally incorporates only three components: the sensor, SENINF, and CAMSV. .. prompt:: bash # auto # media-ctl -d ${MEDIA_DEV} -l "'seninf-0':1 -> 'mtk-cam camsv-0':0 [5]" # media-ctl -d ${MEDIA_DEV} -l "'ap1302.0-003c':2 -> 'seninf-0':0 [1]" The second step is to assign the correct format and size to each component incorporated in the camera pipeline. .. prompt:: bash # auto # media-ctl -d ${MEDIA_DEV} -V "'ap1302.0-003c':2 [fmt:UYVY8_1X16/1920x1080 field:none]" # media-ctl -d ${MEDIA_DEV} -V "'seninf-0':1 [fmt:UYVY8_1X16/1920x1080 field:none]" # media-ctl -d ${MEDIA_DEV} -V "'mtk-cam camsv-0':1 [fmt:UYVY8_1X16/1920x1080 field:none]" For the third step, it's necessary to identify the correct video device node. For further information on this, please refer to `Video Device`_. .. prompt:: bash # auto # declare -a VIDEO_DEV=(`for i in {0..5}; do media-ctl -d ${MEDIA_DEV} --entity "mtk-cam camsv-$i main-stream"; done | tr "\n" " "`) # echo ${VIDEO_DEV[*]} /dev/video53 /dev/video54 /dev/video55 /dev/video56 /dev/video57 /dev/video58 .. important:: You have to select the video node that corresponds to the CAMSV instance linked in the second step. In this given example, ``mtk-cam camsv-0`` is the linked instance, hence the video device node should be ``${VIDEO_DEV[0]}`` or ``/dev/video53``. Lastly, to activate the camera, the user can use ``v4l2-ctl``, ``gst-launch-1.0``, or other V4L2 applications. - Use ``v4l2-ctl`` .. prompt:: bash # auto # v4l2-ctl -d ${VIDEO_DEV[0]} --set-fmt-video=width=1920,height=1080,pixelformat=UYVY --stream-mmap --stream-count=10 --stream-to=/tmp/ap1302.yuv --verbose - Use ``gst-launch-1.0`` .. prompt:: bash # auto # gst-launch-1.0 v4l2src device=${VIDEO_DEV[0]} ! video/x-raw,width=1920,height=1080,format=UYVY ! v4l2convert output-io-mode=dmabuf-import ! video/x-raw,width=1280,height=720 ! waylandsink sync=false .. figure:: /_asset/sw_yocto_app-dev_camera_yuv-show-on-weston.png :align: center :scale: 50% Show Preview stream on Weston .. note:: To use MediaTek video codec and MDP hardware, the user has to load ``video.dtbo``. For more details, please refer to :doc:`Genio 1200-EVK Video Codec `.