.. include:: /keyword.rst

.. _video-g700-evk:

===============
|G510-G700-EVK|
===============

.. contents:: Sections
    :local:
    :depth: 1


.. important::
    
    All the bash commands shown here are based on |G700-EVK|. For |G510-EVK|, 
    users can follow the same step to do the whole setup and experiment with changing the naming term from `700` to `510`.
    
.. note::

    The test results presented in this chapter are based on the `IoT Yocto v23.0` and |G700-EVK-REF-BOARD|.


Video CODEC Device Tree Overlay
-------------------------------

Because there is no MT8370/MT8390 video codec driver in Linux mainline, we have to migrate
the MediaTek proprietary video codec driver to `IoT Yocto v23.0` and make it as a device tree overlay for the integration.
(For how to download an image to |G510-G700-EVK-REF-BOARD|, please refer to :doc:`Genio tools</sw/yocto/get-started/flash>`.)

Command to list the device tree overlay that supported on |G510-G700-EVK-REF-BOARD| image:

.. prompt::

    $ genio-flash --list-dtbo

    List of available DTBO:
        - display-dp.dtbo
        - gpu-mali.dtbo
        - video.dtbo

Command to flash image with ``video.dtbo`` (to install `vcodec` and `vcu` drivers):

.. prompt::

    $ genio-flash --load-dtbo video.dtbo

    AIoT Tools: v1.3.3
    Yocto Image:
        name:     Rity BSP Image (rity-bsp-image)
        distro:   Rity Demo Layer 23.0-release (rity-demo)
        codename: kirkstone
        machine:  genio-700-evk
        overlays: ['video.dtbo']

    Looking for MediaTek SoC matching USB device 0e8d:0003
    ...

After boot to home, check if the V4L2 video devices mount successfully:

.. prompt:: bash # auto

    # root@g700-evk:/# ls -l /sys/class/video4linux/
    total 0
    lrwxrwxrwx 1 root root 0 Apr 28  2022 video0 -> ../../devices/platform/soc/1a030000.jpgenc/video4linux/video0
    lrwxrwxrwx 1 root root 0 Apr 28  2022 video1 -> ../../devices/platform/soc/1a040000.jpgdec/video4linux/video1
    lrwxrwxrwx 1 root root 0 Dec 14 17:39 video2 -> ../../devices/platform/soc/14001000.mdp3-rdma0/video4linux/video2
    lrwxrwxrwx 1 root root 0 Dec 14 17:39 video3 -> ../../devices/platform/soc/1a020000.venc/video4linux/video3
    lrwxrwxrwx 1 root root 0 Dec 14 17:39 video4 -> ../../devices/platform/soc/18000000.vdec/video4linux/video4


Supported Formats of MDP
------------------------

List `Video Output` formats:

.. prompt:: bash # auto

    # v4l2-ctl -d2 --list-formats-out
    ioctl: VIDIOC_ENUM_FMT
        Type: Video Output Multiplanar

        [0]: 'GREY' (8-bit Greyscale)
        [1]: 'RGBR' (16-bit RGB 5-6-5 BE)
        [2]: 'RGBP' (16-bit RGB 5-6-5)
        [3]: 'RGB3' (24-bit RGB 8-8-8)
        [4]: 'BGR3' (24-bit BGR 8-8-8)
        [5]: 'AR24' (32-bit BGRA 8-8-8-8)
        [6]: 'BA24' (32-bit ARGB 8-8-8-8)
        [7]: 'UYVY' (UYVY 4:2:2)
        [8]: 'VYUY' (VYUY 4:2:2)
        [9]: 'YUYV' (YUYV 4:2:2)
        [10]: 'YVYU' (YVYU 4:2:2)
        [11]: 'YU12' (Planar YUV 4:2:0)
        [12]: 'YV12' (Planar YVU 4:2:0)
        [13]: 'NV12' (Y/CbCr 4:2:0)
        [14]: 'NV21' (Y/CrCb 4:2:0)
        [15]: 'NV16' (Y/CbCr 4:2:2)
        [16]: 'NV61' (Y/CrCb 4:2:2)
        [17]: 'NV24' (Y/CbCr 4:4:4)
        [18]: 'NV42' (Y/CrCb 4:4:4)
        [19]: 'NM12' (Y/CbCr 4:2:0 (N-C))
        [20]: 'MM21' (Mediatek 8-bit Block Format)
        [21]: 'NM21' (Y/CrCb 4:2:0 (N-C))
        [22]: 'NM16' (Y/CbCr 4:2:2 (N-C))
        [23]: 'NM61' (Y/CrCb 4:2:2 (N-C))
        [24]: 'YM12' (Planar YUV 4:2:0 (N-C))
        [25]: 'YM21' (Planar YVU 4:2:0 (N-C))
        [26]: 'YM16' (Planar YUV 4:2:2 (N-C))
        [27]: 'YM61' (Planar YVU 4:2:2 (N-C))

List `Video Capture` formats:

.. prompt:: bash # auto

    # v4l2-ctl -d2 --list-formats
    ioctl: VIDIOC_ENUM_FMT
        Type: Video Capture Multiplanar

        [0]: 'GREY' (8-bit Greyscale)
        [1]: 'RGBR' (16-bit RGB 5-6-5 BE)
        [2]: 'RGBP' (16-bit RGB 5-6-5)
        [3]: 'RGB3' (24-bit RGB 8-8-8)
        [4]: 'BGR3' (24-bit BGR 8-8-8)
        [5]: 'AR24' (32-bit BGRA 8-8-8-8)
        [6]: 'BA24' (32-bit ARGB 8-8-8-8)
        [7]: 'UYVY' (UYVY 4:2:2)
        [8]: 'VYUY' (VYUY 4:2:2)
        [9]: 'YUYV' (YUYV 4:2:2)
        [10]: 'YVYU' (YVYU 4:2:2)
        [11]: 'YU12' (Planar YUV 4:2:0)
        [12]: 'YV12' (Planar YVU 4:2:0)
        [13]: 'NV12' (Y/CbCr 4:2:0)
        [14]: 'NV21' (Y/CrCb 4:2:0)
        [15]: 'NV16' (Y/CbCr 4:2:2)
        [16]: 'NV61' (Y/CrCb 4:2:2)
        [17]: 'NV24' (Y/CbCr 4:4:4)
        [18]: 'NM12' (Y/CbCr 4:2:0 (N-C))
        [19]: 'NM21' (Y/CrCb 4:2:0 (N-C))
        [20]: 'NM16' (Y/CbCr 4:2:2 (N-C))
        [21]: 'NM61' (Y/CrCb 4:2:2 (N-C))
        [22]: 'YM12' (Planar YUV 4:2:0 (N-C))
        [23]: 'YM21' (Planar YVU 4:2:0 (N-C))
        [24]: 'YM16' (Planar YUV 4:2:2 (N-C))
        [25]: 'YM61' (Planar YVU 4:2:2 (N-C))


Micro Processor - System Companion Processor
--------------------------------------------

On Genio-700, the MDP kernel driver delegates most of the hardware configuration logic to a Micro Processor, which we called `SCP` (System Companion Processor).
The MDP kernel driver communicate with `SCP` via `Remote Processor Framework`_. All the video processing
functionalities should be accessed through the V4L2 interface on |IOT-YOCTO|.

On |IOT-YOCTO|, the ``SCP`` firmware is loaded when first launched.

.. _Remote Processor Framework: https://docs.kernel.org/staging/remoteproc.html


Video Decoder Codec Types
-------------------------

.. csv-table:: Codec types of `mtk-vcodec-dec` on |G510-EVK|
    :class: longtable
    :file: /_asset/tables/video_g510_vdec_spec.csv
    :width: 70%
    :header-rows: 1

.. csv-table:: Codec types of `mtk-vcodec-dec` on |G700-EVK|
    :class: longtable
    :file: /_asset/tables/video_g700_vdec_spec.csv
    :width: 70%
    :header-rows: 1


Video Encoder Codec Types
-------------------------

.. csv-table:: Codec types of `mtk-vcodec-enc` on |G510-G700-EVK|
    :class: longtable
    :file: /_asset/tables/video_g510_g700_venc_spec.csv
    :width: 70%
    :header-rows: 1


Video Codec Capabilities
------------------------

On Genio-700, the maximum instance number of VDEC is 16, and the maximum instance of VENC is 10.
The maximum resolution/framerate of multiple instances might be calculated by SPEC listed in the following section.
For example, the maximum spec of MPEG4 video decoding is FHD(1920x1080) 60fps. We might support two 1920x1080-30fps videos or eight 720x480-30fps videos.
It is okay to run VDEC and VENC at the same time without any conflict.

.. csv-table:: H264 decoder (w/ MDP) concurrent test result on Genio 510
    :class: longtable
    :file: /_asset/tables/video_g510_vdec_concurrent.csv
    :width: 70%
    :header-rows: 1

.. csv-table:: H264 encoder concurrent test result on Genio 510
    :class: longtable
    :file: /_asset/tables/video_g510_venc_concurrent.csv
    :width: 70%
    :header-rows: 1

.. note::

    The more instances, the more SW overhead(CPU / bandwidth / IO... etc) was generated. Leave some margins for the overheads.


Video Encoder Extra-Controls
============================

As a V4L2 video encoder, ``mtk-vcodec-enc`` also provides extra-controls to set encoder capabilities.

.. prompt:: bash # auto

    Codec Controls
                     video_b_frames 0x009909ca (int)    : min=0 max=3 step=1 default=0 value=0 flags=update
                     video_gop_size 0x009909cb (int)    : min=0 max=65535 step=1 default=0 value=0
                 video_bitrate_mode 0x009909ce (menu)   : min=0 max=2 default=0 value=0 (Variable Bitrate) flags=update
                                                            0: Variable Bitrate
                                                            1: Constant Bitrate
                                                            2: Constant Quality
                      video_bitrate 0x009909cf (int)    : min=0 max=400000000 step=1 default=5000000 value=5000000
    frame_level_rate_control_enable 0x009909d7 (bool)   : default=1 value=1
               sequence_header_mode 0x009909d8 (menu)   : min=0 max=1 default=1 value=1 (Joined With 1st Frame)
                                                            0: Separate Buffer
                                                            1: Joined With 1st Frame
           number_of_mbs_in_a_slice 0x009909dc (int)    : min=0 max=65535 step=1 default=0 value=0
                    force_key_frame 0x009909e5 (button) : value=0 flags=write-only, execute-on-write
              h264_maximum_qp_value 0x00990a62 (int)    : min=0 max=51 step=1 default=51 value=51
                h264_i_frame_period 0x00990a66 (int)    : min=0 max=65535 step=1 default=0 value=0
                         h264_level 0x00990a67 (menu)   : min=0 max=16 default=11 value=11 (4)
                                                            0: 1
                                                            1: 1b
                                                            2: 1.1
                                                            3: 1.2
                                                            4: 1.3
                                                            5: 2
                                                            6: 2.1
                                                            7: 2.2
                                                            8: 3
                                                            9: 3.1
                                                            10: 3.2
                                                            11: 4
                                                            12: 4.1
                                                            13: 4.2
                                                            14: 5
                                                            15: 5.1
                                                            16: 5.2
                       h264_profile 0x00990a6b (menu)   : min=0 max=4 default=0 value=0 (Baseline)
                                                            0: Baseline
                                                            2: Main
                                                            4: High
    video_encode_vui_sar_descriptio 0x00990a6e (u32)    : min=0 max=4294967295 step=1 default=0 dims=[4] flags=write-only, has-payload
                       hevc_profile 0x00990b67 (menu)   : min=0 max=0 default=0 value=0 (Main)
                                                            0: Main
                         hevc_level 0x00990b68 (menu)   : min=0 max=8 default=5 value=5 (4)
                                                            0: 1
                                                            1: 2
                                                            2: 2.1
                                                            3: 3
                                                            4: 3.1
                                                            5: 4
                                                            6: 4.1
                                                            7: 5
                                                            8: 5.1
                          hevc_tier 0x00990b6a (menu)   : min=0 max=1 default=0 value=0 (Main)
                                                            0: Main
                                                            1: High
    video_encode_slice_loss_indicat 0x00990b84 (int)    : min=0 max=1 step=1 default=0 value=0 flags=write-only
     video_encode_color_description 0x00992007 (u32)    : min=0 max=4294967295 step=1 default=0 dims=[18] flags=write-only, has-payload
              video_encode_scenario 0x00992014 (int)    : min=0 max=32 step=1 default=0 value=0 flags=write-only
               video_encode_nonrefp 0x00992015 (int)    : min=0 max=32 step=1 default=0 value=0 flags=write-only
      video_encode_detect_framerate 0x00992016 (int)    : min=0 max=32 step=1 default=0 value=0 flags=write-only
        video_encode_operation_rate 0x00992018 (int)    : min=0 max=2048 step=1 default=0 value=0 flags=write-only
        video_encode_heif_grid_size 0x0099201b (int)    : min=0 max=251660416 step=16 default=0 value=0 flags=write-only
                   i_frame_qp_value 0x0099201f (int)    : min=0 max=51 step=1 default=51 value=0 flags=write-only
                   p_frame_qp_value 0x00992020 (int)    : min=0 max=51 step=1 default=51 value=0 flags=write-only
                   b_frame_qp_value 0x00992021 (int)    : min=0 max=51 step=1 default=51 value=0 flags=write-only
                  video_encode_tsvc 0x00992022 (u32)    : min=0 max=15 step=1 default=0 dims=[2] flags=write-only, has-payload
               video_encode_nonrefp 0x00992023 (int)    : min=0 max=32 step=1 default=0 value=0 flags=write-only
             video_encode_multi_ref 0x00992024 (u32)    : min=0 max=65535 step=1 default=0 dims=[8] flags=write-only, has-payload
                video_encode_max_qp 0x00992026 (int)    : min=-1 max=51 step=1 default=51 value=0 flags=write-only
                video_encode_min_qp 0x00992027 (int)    : min=-1 max=51 step=1 default=0 value=0 flags=write-only
           video_encode_ip_qp_delta 0x00992028 (int)    : min=-1 max=51 step=1 default=4 value=0 flags=write-only
       video_encode_qp_control_mode 0x00992029 (int)    : min=0 max=8 step=1 default=0 value=0 flags=write-only
        video_encode_frame_level_qp 0x0099202a (int)    : min=-1 max=51 step=1 default=28 value=0 flags=write-only
              encode_max_ltr_frames 0x0099202e (int)    : min=0 max=3 step=1 default=0 value=0 flags=write-only


.. note::

    GStreamer does **not** fully support video header mode *V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE*.

For example, to compress a H.264 main profile and level 4.1 video bitstream with 512kbps bitrate:

.. prompt:: bash # auto

    # gst-launch-1.0 -v videotestsrc num-buffers=300 ! "video/x-raw,format=NV12, width=720, height=480, framerate=30/1"  ! v4l2h264enc extra-controls="cid,video_gop_size=30,video_bitrate=512000,sequence_header_mode=1" ! "video/x-h264,level=(string)4.1,profile=main" ! h264parse ! mp4mux ! filesink location=/tmp/test-h264.mp4
    ...
    Execution ended after 0:00:01.554987154
    Setting pipeline to NULL ...
    Freeing pipeline ...

Another example, to compress a H.265 main profile and level 4.1 video bitstream with 512kbps bitrate:

.. prompt:: bash $ auto

    $ gst-launch-1.0 -v videotestsrc num-buffers=300 ! "video/x-raw,format=NV12, width=720, height=480, framerate=30/1"  ! v4l2h265enc extra-controls="cid,video_gop_size=30,video_bitrate=512000,sequence_header_mode=1" ! "video/x-h265,level=(string)4.1,profile=main" ! h265parse ! mp4mux ! filesink location=/tmp/test-h265.mp4
    ...
    Execution ended after 0:00:00.545208846
    Setting pipeline to NULL ...
    Freeing pipeline ...

.. note::

    To modify profile & level, please set it via `gst-caps`. If set by extra-controls directly, The profile & level **will be overridden during gst caps negotiation**.