.. include:: /keyword.rst ========== |G350-EVK| ========== .. contents:: Sections :local: :depth: 1 .. note:: Command operations and test results presented in this chapter are based on the `IoT Yocto v22.0` and |G350-EVK| board. Video CODEC Device Tree Overlay ------------------------------- Because there is no MT8365 video codec driver in Linux mainline, we have to migrate the MediaTek proprietary video codec driver to `IoT Yocto v22.2` and make it as a device tree overlay for the integration. (For how to download an image to |G350-EVK|, please refer to :doc:`Genio tools`.) Command to list the device tree overlay that supported on ``genio-350-evk`` image: .. prompt:: $ genio-flash --list-dtbo List of available DTBO: - camera-ap1302-ar0430-dual.dtbo - camera-ap1302-ar0430-single-csi0.dtbo - camera-ap1302-ar0430-single-csi1.dtbo - net-ethernet.dtbo - panel-startek-kd070fhfid015.dtbo - video.dtbo Command to flash image with ``video.dtbo`` (to install `vcodec`, `vcu`, and `mdp` drivers): .. prompt:: $ genio-flash --load-dtbo video.dtbo AIoT Tools: v1.3.3 Yocto Image: name: Rity Demo Image (rity-demo-image) distro: Rity Demo Layer 22.2-release (rity-demo) codename: kirkstone machine: genio-350-evk overlays: ['video.dtbo'] Looking for MediaTek SoC matching USB device 0e8d:0003 ... After boot to home, check if the V4L2 video devices mount successfully: .. prompt:: bash # auto # ls -l /sys/class/video4linux/ total 0 lrwxrwxrwx 1 root root 0 Jan 1 00:08 video0 -> ../../devices/platform/soc/16000000.codec/video4linux/video0 lrwxrwxrwx 1 root root 0 Jan 1 00:08 video1 -> ../../devices/platform/soc/17020000.codec/video4linux/video1 lrwxrwxrwx 1 root root 0 Jan 1 00:08 video2 -> ../../devices/platform/soc/14004000.mdp_rdma0/video4linux/video2 Supported Formats of MDP ------------------------ List `Video Capture` formats: .. prompt:: bash # auto # v4l2-ctl --list-formats -d 2 ioctl: VIDIOC_ENUM_FMT Type: Video Capture Multiplanar [0]: 'NM12' (Y/CbCr 4:2:0 (N-C)) [1]: 'NV12' (Y/CbCr 4:2:0) [2]: 'NM21' (Y/CrCb 4:2:0 (N-C)) [3]: 'NV21' (Y/CrCb 4:2:0) [4]: 'YM21' (Planar YVU 4:2:0 (N-C)) [5]: 'YM12' (Planar YUV 4:2:0 (N-C)) [6]: 'YV12' (Planar YVU 4:2:0) [7]: 'YU12' (Planar YUV 4:2:0) [8]: '422P' (Planar YUV 4:2:2) [9]: 'NV16' (Y/CbCr 4:2:2) [10]: 'NM16' (Y/CbCr 4:2:2 (N-C)) [11]: 'YUYV' (YUYV 4:2:2) [12]: 'UYVY' (UYVY 4:2:2) [13]: 'YVYU' (YVYU 4:2:2) [14]: 'VYUY' (VYUY 4:2:2) [15]: 'BA24' (32-bit ARGB 8-8-8-8) [16]: 'AR24' (32-bit BGRA 8-8-8-8) [17]: 'BX24' (32-bit XRGB 8-8-8-8) [18]: 'XR24' (32-bit BGRX 8-8-8-8) [19]: 'RGBP' (16-bit RGB 5-6-5) [20]: 'RGB3' (24-bit RGB 8-8-8) [21]: 'BGR3' (24-bit BGR 8-8-8) MDPD Daemon ----------- Like the ``VCODEC`` driver, on Genio 350, the MDP kernel driver delegates most of the hardware configuration logic to userspace daemons. The daemon is ``mdpd``. The daemon does not provide interfaces to other userspace clients. They only work with the kernel driver. All the video processing functionalities should be accessed through the V4L2 interface on |IOT-YOCTO|. Therefore, the format convert processing drivers stop working if the ``mdpd`` process is not initialized or stopped. On |IOT-YOCTO|, the ``mdpd`` is launched during the system boot process. Video Decoder Codec Types ------------------------- .. csv-table:: Codec types of `mtk-vcodec-dec` on Genio 350 :class: longtable :file: /_asset/tables/video_g350_vdec_spec.csv :width: 70% :header-rows: 1 Video Encoder Codec Types ------------------------- .. csv-table:: Codec types of `mtk-vcodec-enc` on Genio 350 :class: longtable :file: /_asset/tables/video_g350_venc_spec.csv :width: 70% :header-rows: 1 Video Codec Capabilities ------------------------ On Genio-350, the maximum instance number of VDEC is 16, and the maximum instance of VENC is 10. The maximum resolution/framerate of multiple instances might be calculated by SPEC listed in the following section. For example, the maximum spec of MPEG4 video decoding is FHD(1920x1080) 60fps. We might support two 1920x1080-30fps videos or eight 720x480-30fps videos. It is okay to run VDEC and VENC at the same time without any conflict. .. csv-table:: H264 decoder (w/ MDP) concurrent test result on Genio 350 :class: longtable :file: /_asset/tables/video_g350_vdec_concurrent.csv :width: 70% :header-rows: 1 .. csv-table:: H264 encoder concurrent test result on Genio 350 :class: longtable :file: /_asset/tables/video_g350_venc_concurrent.csv :width: 70% :header-rows: 1 .. note:: The more instances, the more SW overhead(CPU / bandwidth / IO... etc) was generated. Leave some margins for the overheads. Video Encoder Extra-Controls ============================ As a V4L2 video encoder, ``mtk-vcodec-enc`` also provides extra-controls to set encoder capabilities. .. table:: extra-controls of `mtk-vcodec-enc` on Genio 350 =================================== ======================== =========== ============= ================== CID Command(String) Value Default Value Note =================================== ======================== =========== ============= ================== V4L2_CID_MPEG_VIDEO_BITRATE ``video_bitrate`` 1~20000000 20000000 V4L2_CID_MPEG_VIDEO_GOP_SIZE ``video_gop_size`` 0~65535 0 size 0 means I-VOP only V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME ``force_key_frame`` 0~0 0 to force set I-VOP on the next output frame V4L2_CID_MPEG_VIDEO_HEADER_MODE ``sequence_header_mode`` 0~1 1 0: separate mode, 1: joined-with-1st-frame mode. V4L2_CID_MPEG_VIDEO_H264_PROFILE ``h264_profile`` 0, 2, 4 4 0: BASELINE, 2: MAIN, 4: HIGH V4L2_CID_MPEG_VIDEO_H264_LEVEL ``h264_level`` 0, 2~13 11 support LEVEL_1_0~LEVEL_4_2, exclude LEVEL_1B) =================================== ======================== =========== ============= ================== .. note:: GStreamer does **not** fully support video header mode *V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE*. For example, to compress a H.264 main profile and level 4.1 video bitstream with 512kbps bitrate: .. prompt:: bash # auto # gst-launch-1.0 -v videotestsrc num-buffers=300 ! "video/x-raw,format=NV12, width=720, height=480, framerate=30/1" ! v4l2h264enc extra-controls="cid,video_gop_size=30,video_bitrate=512000,sequence_header_mode=1" ! "video/x-h264,level=(string)4.1,profile=main" ! h264parse ! mp4mux ! filesink location=/tmp/test-h264.mp4 ... Execution ended after 0:00:01.554987154 Setting pipeline to NULL ... Freeing pipeline ... Another example, to compress a H.265 main profile and level 4.1 video bitstream with 512kbps bitrate: .. prompt:: bash $ auto $ gst-launch-1.0 -v videotestsrc num-buffers=300 ! "video/x-raw,format=NV12, width=720, height=480, framerate=30/1" ! v4l2h265enc extra-controls="cid,video_gop_size=30,video_bitrate=512000,sequence_header_mode=1" ! "video/x-h265,level=(string)4.1,profile=main" ! h265parse ! mp4mux ! filesink location=/tmp/test-h265.mp4 ... Execution ended after 0:00:00.545208846 Setting pipeline to NULL ... Freeing pipeline ... .. note:: To modify profile & level, please set it via `gst-caps`. If set by extra-controls directly, The profile & level **will be overridden during gst caps negotiation**.