.. include:: /keyword.rst .. _feature-g700-cam-imgsensor-raw-sensor: =============================== RAW Sensor (MediaTek Imgsensor) =============================== .. important:: All the bash commands shown here are based on |G510-G700-EVK|. For |G510-EVK|, users can follow the same step to do the whole setup by changing the naming term from `700` to `510`. .. contents:: Sections :local: :depth: 2 .. note:: All command operations presented in this chapter are based on the IoT Yocto v24.0, |G700-EVK-REF-BOARD| and Sony IMX214 sensor. You might get different operation results depending on the platform you use. Camera Daughter Board --------------------- The RAW camera DTB for |G510-G700-EVK| is ``CAM DTB-D2``. It contains a SONY IMX214 CMOS Image Sensor. .. figure:: /_asset/sw_yocto_app-dev_camera_cam-dtb-d2.jpeg :align: center :scale: 30% Camera DTB D2 - SONY IMX214 Sensor Connect The Camera to the EVK ----------------------------- There are 2 MIPI-CSI slots on |G510-G700-EVK|, which are ``CSI0`` and ``CSI1``. In the following figure, the camera DTB is connected to the ``CSI0`` slot. .. figure:: /_asset/sw_yocto_app-dev_camera_g700_connect-d2.png :align: center :scale: 30% Connect Camera DTB D2 to |G510-G700-EVK| (Schematic) .. figure:: /_asset/sw_yocto_app-dev_camera_g700-p1v3_connect-cam.jpeg :align: center :scale: 30% Connect Camera DTB D2 to |G510-G700-EVK| (Entity) .. note:: All three CSI slots are available. The user can connect the camera sensor to any slot. .. _feature-g700-cam-imx214: Select Camera Device Tree Blob Overlay -------------------------------------- The camera device is inactive by default, so the kernel has to load a specific device tree blob overlay to enable it. Please refer to the :ref:`bl33 (u-boot) section ` for more details. .. csv-filter:: Raw Camera DTBO for |G510-G700-EVK| :header-rows: 1 :file: ../../../../_asset/tables/camera-platform-sensor-dtbo.csv :include: {0: 'Genio 510\/700-EVK'} :exclude: {5: '[^0.*]', 6: '[^0.*]'} :included_cols: 0,1,2,4,7 .. prompt:: bash > auto > genio-flash --list-dtbo List of available DTBO: - camera-imx214-2lanes-csi0.dtbo - camera-imx214-csi0.dtbo - camera-imx214-csi1.dtbo - ... > genio-flash --load-dtbo camera-imx214-csi0.dtbo --load-dtbo gpu-mali.dtbo --load-dtbo apusys.dtbo --load-dtbo video.dtbo .. warning:: Please select the DTBO according to the CSI slot to which the camera sensor is connected. For example, if the camera is connected to the ``CSI0`` slot, please load the `dtbo` ``camera-imx214-csi0.dtbo``. Otherwise, the camera initialization will fail. Supported Formats and Sizes --------------------------- The following table shows the supported formats and resolutions for each stream. .. figure:: /_asset/sw_yocto_app-dev_camera_g1200_raw_format-resolution.svg :align: center :scale: 60% Supported formats and Sizes .. note:: The supported format, resolution, and frame rate are related to the capability of the sensor and SoC. .. _camera-g700-evk-v4l2-device-node: V4L2 Device Node ---------------- Camera ISP is a complicated feature that has a large number of device nodes. The camera ISP driver creates 132 video devices in total. - ``MTK-ISP-DIP-V4L2`` has 73 devices - ``mtk-cam`` has 56 devices - ``mtk-v4l2-camera`` has 3 devices ``MTK-ISP-DIP-V4L2`` and ``mtk-cam`` are used by the camera ISP middleware. The user can just focus on ``mtk-v4l2-camera`` which is for streaming. .. prompt:: bash # auto # v4l2-ctl --list-devices ... MTK-ISP-DIP-V4L2 (platform:15000000.imgsys_fw): (73 video devices) /dev/media1 mtk-cam (platform:16000000.camisp): (56 video devices) /dev/media3 mtk-v4l2-camera (platform:mtkcam0): /dev/video74 /dev/video75 /dev/video76 /dev/media2 Launch Camera ------------- The V4L2 nodes with the name ``mtk-v4l2-camera`` can be used for streaming. There are 3 V4L2 devices representing Preview, Video, and Capture streams. The user can find the devices by the following command: .. prompt:: bash # auto # v4l2-ctl --list-device mtk-v4l2-camera (platform:mtkcam0): /dev/video74 /dev/video75 /dev/video76 # cat /sys/class/video4linux/video{74,75,76}/name mtk-v4l2-camera@0-Preview mtk-v4l2-camera@0-Video mtk-v4l2-camera@0-Capture .. important:: The above v4l2 device node number is for example. The node number may change every time the system boots. To quickly fetch the correct node and assign the environment variable: .. prompt:: bash # auto # declare -a VIDEO_DEV=(`v4l2-ctl --list-devices | grep mtk-v4l2-camera -A 3 | grep video | tr -d "\n"`) # printf "Preview Node\t= ${VIDEO_DEV[0]}\nVideo Node\t= ${VIDEO_DEV[1]}\nCapture Node\t= ${VIDEO_DEV[2]}\n" Preview Node = /dev/video74 Video Node = /dev/video75 Capture Node = /dev/video76 The Preview and Video streams can deliver images in YUV and RGB formats. The Capture stream is for JPEG outputs. The Capture stream is used to fetch the image that the user sees on the Preview or Video stream. The 3A configuration of the Capture stream relies on the Preview or Video stream. Therefore, **before starting the Capture stream, make sure one of the Preview and Video streams is running.** Section `Supported formats and Sizes`_ shows the supported formats and sizes. The user can also use the v4l2 utility, ``v4l2-ctl``, to list formats. .. prompt:: bash # auto # v4l2-ctl -d ${VIDEO_DEV[0]} --list-formats-ext # List formats/sizes of Preview Stream ioctl: VIDIOC_ENUM_FMT Type: Video Capture Multiplanar [0]: 'RGB3' (24-bit RGB 8-8-8) Size: Discrete 640x360 Interval: Discrete 0.033s (30.000 fps) ... [1]: 'YUYV' (YUYV 4:2:2) Size: Discrete 640x360 Interval: Discrete 0.033s (30.000 fps) ... # v4l2-ctl -d ${VIDEO_DEV[2]} --list-formats-ext # List formats/sizes of Capture Stream ioctl: VIDIOC_ENUM_FMT Type: Video Capture Multiplanar [0]: 'JPEG' (JFIF JPEG, compressed) Size: Discrete 320x240 Interval: Discrete 0.033s (30.000 fps) ... To launch the camera, the user can use ``v4l2-ctl``, ``gst-launch-1.0``, or other V4L2 applications. - Use ``v4l2-ctl`` .. prompt:: bash # auto # v4l2-ctl -d ${VIDEO_DEV[0]} --set-fmt-video=width=1920,height=1080,pixelformat=YUYV --stream-mmap --stream-count=1 --stream-to=/tmp/preview.yuv --verbose - Use ``gst-launch-1.0`` .. prompt:: bash # auto # gst-launch-1.0 v4l2src device=${VIDEO_DEV[0]} ! video/x-raw,width=640,height=480,format=YUY2 ! waylandsink sync=false & # gst-launch-1.0 v4l2src device=${VIDEO_DEV[1]} ! video/x-raw,width=640,height=480,format=NV12 ! waylandsink sync=false & # gst-launch-1.0 v4l2src device=${VIDEO_DEV[2]} ! image/jpeg,width=640,height=480,format=JPEG ! jpegdec ! waylandsink sync=false & .. figure:: /_asset/sw_yocto_app-dev_camera_raw-show-on-weston.png :align: center :scale: 50% Show Preview/Video/Capture streams on Weston Moreover, the user can show 720P Preview content on Weston, record 10 seconds of FHD Video to an mp4 file, and Capture 4K JPEG images by the following commands: .. prompt:: bash # auto # # Show 720P Preview on Weston # gst-launch-1.0 v4l2src device=${VIDEO_DEV[0]} ! video/x-raw,width=1280,height=720,format=YUY2 ! waylandsink sync=false & # # Record 10 seconds of FHD Video # gst-launch-1.0 v4l2src device=${VIDEO_DEV[1]} num-buffers=300 ! video/x-raw,width=1920,height=1080,format=NV12 ! v4l2h264enc ! h264parse ! queue ! mp4mux ! filesink location=video.mp4 & # # Capture 4K JPEG image # v4l2-ctl -d ${VIDEO_DEV[2]} --set-fmt-video=width=3840,height=2160,pixelformat=JPEG,sizeimage=4194304 --stream-mmap=1 --stream-count=1 --stream-to=capture.jpg --verbose .. important:: JPEG is a compression format, so the size of the compressed result may vary each time. The user should ensure that the buffer size is large enough to accommodate the worst-case compressed size. Otherwise, the JPEG content may be incomplete. To set the buffer size, the user can refer to the above example ``sizeimage=4194304``. .. note:: To use MediaTek video codec and MDP hardware, the user has to load ``video.dtbo``. For more details, please refer to :doc:`Genio 700-EVK Video Codec `. Set Camera ISP Controls ----------------------- For each stream, there are 14 V4L2 controls. These controls set 3A (Auto Focus, Auto Exposure, and Auto White Balance) and ISP pipeline features (Brightness, Contrast, Saturation, Hue, and Sharpness). The tool ``v4l2-ctl`` can list all controls through the following command: .. prompt:: bash # auto # v4l2-ctl --list-ctrls-menu -d ${VIDEO_DEV[0]} User Controls brightness 0x00980900 (int) : min=0 max=100 step=1 default=0 value=0 flags=slider contrast 0x00980901 (int) : min=0 max=10 step=1 default=0 value=0 flags=slider saturation 0x00980902 (int) : min=0 max=10 step=1 default=0 value=0 flags=slider hue 0x00980903 (int) : min=0 max=100 step=1 default=0 value=0 flags=slider white_balance_automatic 0x0098090c (bool) : default=1 value=1 exposure 0x00980911 (int) : min=-40 max=40 step=1 default=0 value=0 power_line_frequency 0x00980918 (menu) : min=0 max=3 default=3 value=3 (Auto) 0: Disabled 1: 50 Hz 2: 60 Hz 3: Auto white_balance_temperature 0x0098091a (int) : min=2700 max=6500 step=1 default=6500 value=6500 sharpness 0x0098091b (int) : min=0 max=10 step=1 default=0 value=0 flags=slider min_number_of_capture_buffers 0x00980927 (int) : min=1 max=32 step=1 default=1 value=8 flags=read-only, volatile iso 0x009819a9 (int) : min=100 max=6400 step=100 default=100 value=100 Camera Controls auto_exposure 0x009a0901 (menu) : min=0 max=1 default=0 value=0 (Auto Mode) 0: Auto Mode 1: Manual Mode exposure_time_absolute 0x009a0902 (int) : min=100000 max=100000000 step=1 default=33000000 value=33000000 focus_absolute 0x009a090a (int) : min=0 max=255 step=1 default=0 value=0 focus_automatic_continuous 0x009a090c (bool) : default=0 value=0 .. note:: Preview stream and Video stream share the same control setting, so the user modifies one and affects another. However, the control setting of Capture is separated, it doesn't affect others. Auto Focus ^^^^^^^^^^ .. note:: The SONY IMX214 sensor doesn't support AF, so this is not yet available. Auto Exposure ^^^^^^^^^^^^^ Auto Exposure can be set to AUTO mode and MANUAL mode. In AUTO mode, the exposure is calculated automatically. The user can set exposure compensation to adjust the exposure level. .. prompt:: bash # auto # v4l2-ctl -d ${VIDEO_DEV[0]} -c auto_exposure=0 # AUTO # v4l2-ctl -d ${VIDEO_DEV[0]} -c exposure=-40 # MIN # v4l2-ctl -d ${VIDEO_DEV[0]} -c exposure=0 # DEFAULT # v4l2-ctl -d ${VIDEO_DEV[0]} -c exposure=40 # MAX In MANUAL mode, the exposure is decided manually by the user. The user can set exposure time and ISO to adjust the exposure level. .. prompt:: bash # auto # v4l2-ctl -d ${VIDEO_DEV[0]} -c auto_exposure=1 # MANUAL # v4l2-ctl -d ${VIDEO_DEV[0]} -c exposure_time_absolute=100000 # MIN # v4l2-ctl -d ${VIDEO_DEV[0]} -c exposure_time_absolute=100000000 # MAX # v4l2-ctl -d ${VIDEO_DEV[0]} -c iso=100 # MIN # v4l2-ctl -d ${VIDEO_DEV[0]} -c iso=6400 # MAX Auto White Balance ^^^^^^^^^^^^^^^^^^ Auto White Balance can be set to AUTO mode or MANUAL mode. In AUTO mode, the white balance value is calculated automatically. .. prompt:: bash # auto # v4l2-ctl -d ${VIDEO_DEV[0]} -c white_balance_automatic=1 # AUTO In MANUAL mode, the white balance value is decided manually by the user. The user can set the white balance temperature value. .. prompt:: bash # auto # v4l2-ctl -d ${VIDEO_DEV[0]} -c white_balance_automatic=0 # MANUAL # v4l2-ctl -d ${VIDEO_DEV[0]} -c white_balance_temperature=2800 # INCANDESCENT # v4l2-ctl -d ${VIDEO_DEV[0]} -c white_balance_temperature=3300 # WARM_FLUORESCENT # v4l2-ctl -d ${VIDEO_DEV[0]} -c white_balance_temperature=3800 # TWILIGHT # v4l2-ctl -d ${VIDEO_DEV[0]} -c white_balance_temperature=4400 # FLUORESCENT # v4l2-ctl -d ${VIDEO_DEV[0]} -c white_balance_temperature=5200 # DAYLIGHT # v4l2-ctl -d ${VIDEO_DEV[0]} -c white_balance_temperature=5800 # CLOUDY_DAYLIGHT # v4l2-ctl -d ${VIDEO_DEV[0]} -c white_balance_temperature=6300 # SHADE .. note:: Although the temperature value is consecutive, the white balance change is discrete. Because the value is mapped to a white balance mode, such as INCANDESCENT, DAYLIGHT, etc. ISP Pipeline Feature ^^^^^^^^^^^^^^^^^^^^ There are 5 ISP pipeline feature controls (Brightness, Contrast, Saturation, Hue, and Sharpness). The user can set them and observe the image quality change. The default control value is zero which means to disable the control. - Brightness .. prompt:: bash # auto # v4l2-ctl -d ${VIDEO_DEV[0]} -c brightness=0 # DISABLED # v4l2-ctl -d ${VIDEO_DEV[0]} -c brightness=1 # MIN # v4l2-ctl -d ${VIDEO_DEV[0]} -c brightness=100 # MAX - Contrast .. prompt:: bash # auto # v4l2-ctl -d ${VIDEO_DEV[0]} -c contrast=0 # DISABLED # v4l2-ctl -d ${VIDEO_DEV[0]} -c contrast=1 # MIN # v4l2-ctl -d ${VIDEO_DEV[0]} -c contrast=10 # MAX - Saturation .. prompt:: bash # auto # v4l2-ctl -d ${VIDEO_DEV[0]} -c saturation=0 # DISABLED # v4l2-ctl -d ${VIDEO_DEV[0]} -c saturation=1 # MIN # v4l2-ctl -d ${VIDEO_DEV[0]} -c saturation=10 # MAX - Hue .. prompt:: bash # auto # v4l2-ctl -d ${VIDEO_DEV[0]} -c hue=0 # DISABLED # v4l2-ctl -d ${VIDEO_DEV[0]} -c hue=1 # MIN # v4l2-ctl -d ${VIDEO_DEV[0]} -c hue=100 # MAX - Sharpness .. prompt:: bash # auto # v4l2-ctl -d ${VIDEO_DEV[0]} -c sharpness=0 # DISABLED # v4l2-ctl -d ${VIDEO_DEV[0]} -c sharpness=1 # MIN # v4l2-ctl -d ${VIDEO_DEV[0]} -c sharpness=10 # MAX .. include:: camera-g700-g1200-evk-appendix.rst