Genio 350-EVK
Note
Command operations and test results presented in this chapter are based on the IoT Yocto v22.0 and Genio 350-EVK board.
Video CODEC Device Tree Overlay
Because there is no MT8365 video codec driver in Linux mainline, we have to migrate the MediaTek proprietary video codec driver to IoT Yocto v22.2 and make it as a device tree overlay for the integration. (For how to download an image to Genio 350-EVK, please refer to Genio tools.)
Command to list the device tree overlay that supported on genio-350-evk image:
$ genio-flash --list-dtbo
List of available DTBO:
- camera-ap1302-ar0430-dual.dtbo
- camera-ap1302-ar0430-single-csi0.dtbo
- camera-ap1302-ar0430-single-csi1.dtbo
- net-ethernet.dtbo
- panel-startek-kd070fhfid015.dtbo
- video.dtbo
Command to flash image with video.dtbo (to install vcodec, vcu, and mdp drivers):
$ genio-flash --load-dtbo video.dtbo
AIoT Tools: v1.3.3
Yocto Image:
name: Rity Demo Image (rity-demo-image)
distro: Rity Demo Layer 22.2-release (rity-demo)
codename: kirkstone
machine: genio-350-evk
overlays: ['video.dtbo']
Looking for MediaTek SoC matching USB device 0e8d:0003
...
After boot to home, check if the V4L2 video devices mount successfully:
ls -l /sys/class/video4linux/
total 0
lrwxrwxrwx 1 root root 0 Jan 1 00:08 video0 -> ../../devices/platform/soc/16000000.codec/video4linux/video0
lrwxrwxrwx 1 root root 0 Jan 1 00:08 video1 -> ../../devices/platform/soc/17020000.codec/video4linux/video1
lrwxrwxrwx 1 root root 0 Jan 1 00:08 video2 -> ../../devices/platform/soc/14004000.mdp_rdma0/video4linux/video2
Supported Formats of MDP
List Video Capture formats:
v4l2-ctl --list-formats -d 2
ioctl: VIDIOC_ENUM_FMT
Type: Video Capture Multiplanar
[0]: 'NM12' (Y/CbCr 4:2:0 (N-C))
[1]: 'NV12' (Y/CbCr 4:2:0)
[2]: 'NM21' (Y/CrCb 4:2:0 (N-C))
[3]: 'NV21' (Y/CrCb 4:2:0)
[4]: 'YM21' (Planar YVU 4:2:0 (N-C))
[5]: 'YM12' (Planar YUV 4:2:0 (N-C))
[6]: 'YV12' (Planar YVU 4:2:0)
[7]: 'YU12' (Planar YUV 4:2:0)
[8]: '422P' (Planar YUV 4:2:2)
[9]: 'NV16' (Y/CbCr 4:2:2)
[10]: 'NM16' (Y/CbCr 4:2:2 (N-C))
[11]: 'YUYV' (YUYV 4:2:2)
[12]: 'UYVY' (UYVY 4:2:2)
[13]: 'YVYU' (YVYU 4:2:2)
[14]: 'VYUY' (VYUY 4:2:2)
[15]: 'BA24' (32-bit ARGB 8-8-8-8)
[16]: 'AR24' (32-bit BGRA 8-8-8-8)
[17]: 'BX24' (32-bit XRGB 8-8-8-8)
[18]: 'XR24' (32-bit BGRX 8-8-8-8)
[19]: 'RGBP' (16-bit RGB 5-6-5)
[20]: 'RGB3' (24-bit RGB 8-8-8)
[21]: 'BGR3' (24-bit BGR 8-8-8)
MDPD Daemon
Like the VCODEC driver, on Genio 350, the MDP kernel driver delegates
most of the hardware configuration logic to userspace daemons. The daemon is mdpd.
The daemon does not provide interfaces to other userspace clients. They only work with the kernel driver. All the video processing functionalities should be accessed through the V4L2 interface on IoT Yocto.
Therefore, the format convert processing drivers stop working if the mdpd process is not initialized or stopped.
On IoT Yocto, the mdpd is launched during the system boot process.
Video Decoder Codec Types
Codec |
Profile |
Max Level |
Max Bitrate |
Max Resolution |
|---|---|---|---|---|
MPEG-4 |
Advanced Simple |
L5 |
40Mbps |
1080p60fps |
Simple |
L6 |
40Mbps |
1080p60fps |
|
H.264 |
Constrained Baseline (CBP) |
L4.2 |
40Mbps |
1080p60fps |
Main (MP) |
L4.2 |
40Mbps |
1080p60fps |
|
High (HiP) |
L4.2 |
40Mbps |
1080p60fps |
|
H.265 (HEVC) |
Main |
L4 |
40Mbps |
1080p60fps |
VP8 |
40Mbps |
1080p60fps |
||
VP9 |
40Mbps |
1080p60fps |
Video Encoder Codec Types
Codec |
Input Format |
Profile |
Level |
Max Bitrate |
Record Spec |
|---|---|---|---|---|---|
H.264 |
NV12, YV12, YUV420(I420), NV21 |
Baseline Main High |
L4.1 |
17Mbps |
|
H.265 |
NV12, YV12, YUV420(I420), NV21 |
Main |
L5.1 |
17Mbps |
Video Codec Capabilities
On Genio-350, the maximum instance number of VDEC is 16, and the maximum instance of VENC is 10. The maximum resolution/framerate of multiple instances might be calculated by SPEC listed in the following section. For example, the maximum spec of MPEG4 video decoding is FHD(1920x1080) 60fps. We might support two 1920x1080-30fps videos or eight 720x480-30fps videos. It is okay to run VDEC and VENC at the same time without any conflict.
Width |
Height |
FPS |
Instance |
Test result |
Note |
|---|---|---|---|---|---|
1920 |
1080 |
60 |
1 |
Pass |
|
1920 |
1080 |
30 |
2 |
Pass |
|
1280 |
720 |
30 |
11 |
Pass |
Width |
Height |
FPS |
Instance |
Test result |
Note |
|---|---|---|---|---|---|
1920 |
1080 |
60 |
1 |
Pass |
|
1920 |
1080 |
30 |
2 |
Pass |
|
1280 |
720 |
30 |
4 |
Pass |
Note
The more instances, the more SW overhead(CPU / bandwidth / IO… etc) was generated. Leave some margins for the overheads.
Video Encoder Extra-Controls
As a V4L2 video encoder, mtk-vcodec-enc also provides extra-controls to set encoder capabilities.
CID |
Command(String) |
Value |
Default Value |
Note |
|---|---|---|---|---|
V4L2_CID_MPEG_VIDEO_BITRATE |
|
1~20000000 |
20000000 |
|
V4L2_CID_MPEG_VIDEO_GOP_SIZE |
|
0~65535 |
0 |
size 0 means I-VOP only |
V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME |
|
0~0 |
0 |
to force set I-VOP on the next output frame |
V4L2_CID_MPEG_VIDEO_HEADER_MODE |
|
0~1 |
1 |
0: separate mode, 1: joined-with-1st-frame mode. |
V4L2_CID_MPEG_VIDEO_H264_PROFILE |
|
0, 2, 4 |
4 |
0: BASELINE, 2: MAIN, 4: HIGH |
V4L2_CID_MPEG_VIDEO_H264_LEVEL |
|
0, 2~13 |
11 |
support LEVEL_1_0~LEVEL_4_2, exclude LEVEL_1B) |
Note
GStreamer does not fully support video header mode V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE.
For example, to compress a H.264 main profile and level 4.1 video bitstream with 512kbps bitrate:
gst-launch-1.0 -v videotestsrc num-buffers=300 ! "video/x-raw,format=NV12, width=720, height=480, framerate=30/1" ! v4l2h264enc extra-controls="cid,video_gop_size=30,video_bitrate=512000,sequence_header_mode=1" ! "video/x-h264,level=(string)4.1,profile=main" ! h264parse ! mp4mux ! filesink location=/tmp/test-h264.mp4
...
Execution ended after 0:00:01.554987154
Setting pipeline to NULL ...
Freeing pipeline ...
Another example, to compress a H.265 main profile and level 4.1 video bitstream with 512kbps bitrate:
gst-launch-1.0 -v videotestsrc num-buffers=300 ! "video/x-raw,format=NV12, width=720, height=480, framerate=30/1" ! v4l2h265enc extra-controls="cid,video_gop_size=30,video_bitrate=512000,sequence_header_mode=1" ! "video/x-h265,level=(string)4.1,profile=main" ! h265parse ! mp4mux ! filesink location=/tmp/test-h265.mp4
...
Execution ended after 0:00:00.545208846
Setting pipeline to NULL ...
Freeing pipeline ...
Note
To modify profile & level, please set it via gst-caps. If set by extra-controls directly, The profile & level will be overridden during gst caps negotiation.