Genio 1200-demo

Note

Cmd operations and test results presented in this chapter are based on the AIoT Yocto v22.1 and MT8395 P1V6 (Genio 1200) demo board.

VCODEC Device Tree Overlay

Because there is no MT8395 vcodec driver in Linux mainline, we have to migrate the Mediatek propietary vcodec driver to AIoT Yocto v22.1 and make it as a device tree overlay for the integration. (For how to download an image to i1200-DEMO, please refer to AIOT tools.)

Command to list the device tree overlay that supported on i1200-DEMO image:

$ aiot-flash --list-dtbo

List of available DTBO:
    - display-dp.dtbo
    - panel-boe-ne135fbm.dtbo
    - vcodec-vcu-mt8395.dtbo

Command to flash image with vcodec-vcu-mt8395.dtbo (to install vcodec and vcu drivers):

$ aiot-flash --load-dtbo vcodec-vcu-mt8395.dtbo

AIoT Tools: v1.3.dev4+g3029a11
Yocto Image:
    name:     Rity Demo Image (rity-demo-image)
    distro:   Rity Demo Layer 22.1 (rity-demo)
    codename: kirkstone
    machine:  i1200-demo
    overlays: ['vcodec-vcu-mt8395.dtbo']

Looking for MediaTek SoC matching USB device 0e8d:0003
...

After boot to home, check if the V4L2 video devices mount successfully:

root@i1200-demo:/# ls -l /sys/class/video4linux/
total 0
lrwxrwxrwx 1 root root 0 May  3 21:05 video0 -> ../../devices/platform/soc/18000000.vdec/video4linux/video0
lrwxrwxrwx 1 root root 0 May  3 21:05 video1 -> ../../devices/platform/soc/1a020000.venc/video4linux/video1
lrwxrwxrwx 1 root root 0 May  3 21:05 video2 -> ../../devices/platform/soc/14001000.mdp_rdma0/video4linux/video2

Supported Formats of MDP

List Video Output formats:

v4l2-ctl -d2 --list-formats-out
ioctl: VIDIOC_ENUM_FMT
    Type: Video Output Multiplanar

    [0]: 'GREY' (8-bit Greyscale)
    [1]: 'RGBR' (16-bit RGB 5-6-5 BE)
    [2]: 'RGBP' (16-bit RGB 5-6-5)
    [3]: 'RGB3' (24-bit RGB 8-8-8)
    [4]: 'BGR3' (24-bit BGR 8-8-8)
    [5]: 'AR24' (32-bit BGRA 8-8-8-8)
    [6]: 'BA24' (32-bit ARGB 8-8-8-8)
    [7]: 'UYVY' (UYVY 4:2:2)
    [8]: 'VYUY' (VYUY 4:2:2)
    [9]: 'YUYV' (YUYV 4:2:2)
    [10]: 'YVYU' (YVYU 4:2:2)
    [11]: 'YU12' (Planar YUV 4:2:0)
    [12]: 'YV12' (Planar YVU 4:2:0)
    [13]: 'NV12' (Y/CbCr 4:2:0)
    [14]: 'NV21' (Y/CrCb 4:2:0)
    [15]: 'NV16' (Y/CbCr 4:2:2)
    [16]: 'NV61' (Y/CrCb 4:2:2)
    [17]: 'MT21' (Mediatek Compressed Format, compressed)
    [18]: 'MM21' (Mediatek 8-bit Block Format)
    [19]: 'NM12' (Y/CbCr 4:2:0 (N-C))
    [20]: 'NM21' (Y/CrCb 4:2:0 (N-C))
    [21]: 'NM16' (Y/CbCr 4:2:2 (N-C))
    [22]: 'NM61' (Y/CrCb 4:2:2 (N-C))
    [23]: 'YM12' (Planar YUV 4:2:0 (N-C))
    [24]: 'YM21' (Planar YVU 4:2:0 (N-C))
    [25]: 'YM16' (Planar YUV 4:2:2 (N-C))
    [26]: 'YM61' (Planar YVU 4:2:2 (N-C))

List Video Capture formats:

v4l2-ctl -d2 --list-formats
ioctl: VIDIOC_ENUM_FMT
    Type: Video Capture Multiplanar

    [0]: 'GREY' (8-bit Greyscale)
    [1]: 'RGBR' (16-bit RGB 5-6-5 BE)
    [2]: 'RGBP' (16-bit RGB 5-6-5)
    [3]: 'RGB3' (24-bit RGB 8-8-8)
    [4]: 'BGR3' (24-bit BGR 8-8-8)
    [5]: 'AR24' (32-bit BGRA 8-8-8-8)
    [6]: 'BA24' (32-bit ARGB 8-8-8-8)
    [7]: 'UYVY' (UYVY 4:2:2)
    [8]: 'VYUY' (VYUY 4:2:2)
    [9]: 'YUYV' (YUYV 4:2:2)
    [10]: 'YVYU' (YVYU 4:2:2)
    [11]: 'YU12' (Planar YUV 4:2:0)
    [12]: 'YV12' (Planar YVU 4:2:0)
    [13]: 'NV12' (Y/CbCr 4:2:0)
    [14]: 'NV21' (Y/CrCb 4:2:0)
    [15]: 'NV16' (Y/CbCr 4:2:2)
    [16]: 'NV61' (Y/CrCb 4:2:2)
    [17]: 'NM12' (Y/CbCr 4:2:0 (N-C))
    [18]: 'NM21' (Y/CrCb 4:2:0 (N-C))
    [19]: 'NM16' (Y/CbCr 4:2:2 (N-C))
    [20]: 'NM61' (Y/CrCb 4:2:2 (N-C))
    [21]: 'YM12' (Planar YUV 4:2:0 (N-C))
    [22]: 'YM21' (Planar YVU 4:2:0 (N-C))
    [23]: 'YM16' (Planar YUV 4:2:2 (N-C))
    [24]: 'YM61' (Planar YVU 4:2:2 (N-C))

Micro Processor - System Companion Processor

On i1200, the MDP kernel driver delegates most of the hardware configuration logics to a Micro Processor, which we called SCP (System Companion Processor). The MDP kernel driver communicate with SCP via Remote Processor Framework. All the video processing functionalities should be accessed through the V4L2 interface on AIoT Yocto.

On AIoT Yocto, the SCP firmware is loaded when first launched.

Note

In AIoT Yocto v22.1, the SCP will occupy UART1 for debugging purpose. Please do not use UART1.

Video Decoder Codec Types

Codec types of mtk-vcodec-dec on i1200

Codec

Profile

Max Level

Max Bitrate

Max Resolution

MPEG-4

Advanced Simple

L5

60Mbps

FHD60fps

Simple

L6

60Mbps

FHD60fps

H.264

Constrained Baseline (CBP)

L5.2

160Mbps

4K90fps

Main (MP)

L5.2

160Mbps

4K90fps

High (HiP)

L5.2

160Mbps

4K90fps

H.265(HEVC)

Main

L5.1

160Mbps

4K90fps

VP8

40Mbps

FHD60fps

VP9

0/2

120Mbps

4K90fps

Video Encoder Codec Types

Codec types of mtk-vcodec-enc on i1200

Codec

Input Format

Profile

Level

Record Spec

H.264

NV12, YV12, YUV420(I420), RGBA8888, RGB888

High

L5.1

4K30fps

H.265

NV12, YV12, YUV420(I420), RGBA8888, RGB888

Main

L5.1

4K30fps

Video Encoder Extra-Controls

As a V4L2 video encoder, mtk-vcodec-enc also provides extra-controls to set encoder capabilities.

Codec Controls
                 video_b_frames 0x009909ca (int)    : min=0 max=3 step=1 default=0 value=0 flags=update
                 video_gop_size 0x009909cb (int)    : min=0 max=65535 step=1 default=0 value=0
             video_bitrate_mode 0x009909ce (menu)   : min=0 max=2 default=0 value=0 (Variable Bitrate) flags=update
                                                        0: Variable Bitrate
                                                        1: Constant Bitrate
                                                        2: Constant Quality
                  video_bitrate 0x009909cf (int)    : min=0 max=400000000 step=1 default=5000000 value=5000000
frame_level_rate_control_enable 0x009909d7 (bool)   : default=1 value=1
           sequence_header_mode 0x009909d8 (menu)   : min=0 max=1 default=1 value=1 (Joined With 1st Frame)
                                                        0: Separate Buffer
                                                        1: Joined With 1st Frame
       number_of_mbs_in_a_slice 0x009909dc (int)    : min=0 max=65535 step=1 default=0 value=0
                force_key_frame 0x009909e5 (button) : value=0 flags=write-only, execute-on-write
          h264_maximum_qp_value 0x00990a62 (int)    : min=0 max=51 step=1 default=51 value=51
            h264_i_frame_period 0x00990a66 (int)    : min=0 max=65535 step=1 default=0 value=0
                     h264_level 0x00990a67 (menu)   : min=0 max=15 default=11 value=11 (4)
                                                        0: 1
                                                        1: 1b
                                                        2: 1.1
                                                        3: 1.2
                                                        4: 1.3
                                                        5: 2
                                                        6: 2.1
                                                        7: 2.2
                                                        8: 3
                                                        9: 3.1
                                                        10: 3.2
                                                        11: 4
                                                        12: 4.1
                                                        13: 4.2
                                                        14: 5
                                                        15: 5.1
                   h264_profile 0x00990a6b (menu)   : min=0 max=4 default=0 value=0 (Baseline)
                                                        0: Baseline
                                                        1: Constrained Baseline
                                                        2: Main
                                                        3: Extended
                                                        4: High
video_encode_vui_sar_descriptio 0x00990a6e (u32)    : min=0 max=4294967295 step=1 default=0 dims=[4] flags=write-only, has-payload
                    mpeg4_level 0x00990a95 (menu)   : min=0 max=7 default=0 value=0 (0)
                                                        0: 0
                                                        1: 0b
                                                        2: 1
                                                        3: 2
                                                        4: 3
                                                        5: 3b
                                                        6: 4
                                                        7: 5
                  mpeg4_profile 0x00990a96 (menu)   : min=0 max=0 default=0 value=0 (Simple)
                                                        0: Simple
                   hevc_profile 0x00990b67 (menu)   : min=0 max=0 default=0 value=0 (Main)
                                                        0: Main
                     hevc_level 0x00990b68 (menu)   : min=0 max=8 default=5 value=5 (4)
                                                        0: 1
                                                        1: 2
                                                        2: 2.1
                                                        3: 3
                                                        4: 3.1
                                                        5: 4
                                                        6: 4.1
                                                        7: 5
                                                        8: 5.1
                      hevc_tier 0x00990b6a (menu)   : min=0 max=1 default=0 value=0 (Main)
                                                        0: Main
                                                        1: High
video_encode_slice_loss_indicat 0x00990b84 (int)    : min=0 max=1 step=1 default=0 value=0 flags=write-only
 video_encode_color_description 0x00992007 (u32)    : min=0 max=4294967295 step=1 default=0 dims=[18] flags=write-only, has-payload
          video_sec_encode_path 0x00992011 (int)    : min=0 max=5 step=1 default=0 value=0 flags=write-only
          video_encode_scenario 0x00992014 (int)    : min=0 max=32 step=1 default=0 value=0 flags=write-only
           video_encode_nonrefp 0x00992015 (int)    : min=0 max=32 step=1 default=0 value=0 flags=write-only
  video_encode_detect_framerate 0x00992016 (int)    : min=0 max=32 step=1 default=0 value=0 flags=write-only
video_encode_slice_loss_indicat 0x00992017 (int)    : min=0 max=1 step=1 default=0 value=0 flags=write-only
    video_encode_operation_rate 0x00992018 (int)    : min=0 max=2048 step=1 default=0 value=0 flags=write-only
    video_encode_heif_grid_size 0x0099201b (int)    : min=0 max=251660416 step=16 default=0 value=0 flags=write-only
 video_encode_resolution_change 0x0099201c (u32)    : min=0 max=16777215 step=1 default=0 dims=[4] flags=read-only, volatile, has-payload
         video_encode_max_width 0x0099201d (int)    : min=0 max=3840 step=16 default=0 value=0 flags=write-only
        video_encode_max_height 0x0099201e (int)    : min=0 max=3840 step=16 default=0 value=0 flags=write-only
               i_frame_qp_value 0x0099201f (int)    : min=0 max=51 step=1 default=51 value=0 flags=write-only
               p_frame_qp_value 0x00992020 (int)    : min=0 max=51 step=1 default=51 value=0 flags=write-only
               b_frame_qp_value 0x00992021 (int)    : min=0 max=51 step=1 default=51 value=0 flags=write-only
              video_encode_tsvc 0x00992022 (u32)    : min=0 max=15 step=1 default=0 dims=[2] flags=write-only, has-payload
           video_encode_nonrefp 0x00992023 (int)    : min=0 max=32 step=1 default=0 value=0 flags=write-only
         video_encode_multi_ref 0x00992024 (u32)    : min=0 max=65535 step=1 default=0 dims=[8] flags=write-only, has-payload
            video_encode_max_qp 0x00992026 (int)    : min=-1 max=51 step=1 default=51 value=0 flags=write-only
            video_encode_min_qp 0x00992027 (int)    : min=-1 max=51 step=1 default=0 value=0 flags=write-only
       video_encode_ip_qp_delta 0x00992028 (int)    : min=-1 max=51 step=1 default=4 value=0 flags=write-only
   video_encode_qp_control_mode 0x00992029 (int)    : min=0 max=8 step=1 default=0 value=0 flags=write-only
    video_encode_frame_level_qp 0x0099202a (int)    : min=-1 max=51 step=1 default=28 value=0 flags=write-only
             encode_low_latency 0x0099202c (int)    : min=0 max=1 step=1 default=0 value=0 flags=write-only
          encode_max_ltr_frames 0x0099202e (int)    : min=0 max=3 step=1 default=0 value=0 flags=write-only

Note

Gstreamer is not fully support video header mode V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE.

For example, to compress a H.264 main profile and level 4.1 video bitstream with 512kbps bitrate:

gst-launch-1.0 -v videotestsrc num-buffers=300 ! "video/x-raw,format=NV12, width=720, height=480, framerate=30/1"  ! v4l2h264enc extra-controls="cid,video_gop_size=30,video_bitrate=512000,sequence_header_mode=1" ! "video/x-h264,level=(string)4.1,profile=main" ! h264parse ! mp4mux ! filesink location=/tmp/test-h264.mp4
...
Execution ended after 0:00:01.554987154
Setting pipeline to NULL ...
Freeing pipeline ...

Note

To modify profile & level, please set it via gst-caps. If set by extra-controls directly, The profile & level will be overridden during gst caps negotiation.