RAW Sensor (MediaTek Imgsensor)
Important
All the bash commands shown here are based on Genio 510/700-EVK. For Genio 510-EVK, users can follow the same step to do the whole setup by changing the naming term from 700 to 510.
Note
All command operations presented in this chapter are based on the IoT Yocto v24.0, Genio 700-EVK and Sony IMX214 sensor. You might get different operation results depending on the platform you use.
Camera Package And Specification
The RAW camera DTB for Genio 510/700-EVK is CAM DTB-D2.
It contains a SONY IMX214 CMOS Image Sensor.
Camera DTB D2 - SONY IMX214 Sensor
Connect The Camera To The EVK
There are 2 MIPI-CSI slots on Genio 510/700-EVK, which are CSI0 and CSI1.
In the following figure, the camera DTB is connected to the CSI0 slot.
Connect Camera DTB D2 to Genio 510/700-EVK (Schematic)
Connect Camera DTB D2 to Genio 510/700-EVK (Entity)
Note
All three CSI slots are available. The user can connect the camera sensor to any slot.
Select Camera Device Tree Blob Overlay
The camera device is inactive by default, so the kernel has to load a specific device tree blob overlay to enable it. Please refer to bl33u-boot for more details.
Platform |
CSI0 |
CSI1 |
DTBO |
Usage |
|---|---|---|---|---|
Genio 510/700-EVK |
IMX214 |
x |
camera-imx214-csi0.dtbo |
Enable 4-lane CAM-DTB-D2 w/ Sony IMX214 on CSI0 |
Genio 510/700-EVK |
IMX214 (2 lane) |
x |
camera-imx214-2lanes-csi0.dtbo |
Enable 2-lane CAM-DTB-D2 w/ Sony IMX214 on CSI0 |
Genio 510/700-EVK |
x |
IMX214 |
camera-imx214-csi1.dtbo |
Enable 4-lane CAM-DTB-D2 w/ Sony IMX214 on CSI1 |
genio-flash --list-dtbo
List of available DTBO:
- camera-imx214-2lanes-csi0.dtbo
- camera-imx214-csi0.dtbo
- camera-imx214-csi1.dtbo
- ...
genio-flash --load-dtbo camera-imx214-csi0.dtbo --load-dtbo gpu-mali.dtbo --load-dtbo apusys.dtbo --load-dtbo video.dtbo
Warning
Please select the DTBO according to the CSI slot to which the camera sensor is connected.
For example, if the camera is connected to the CSI0 slot, please load the dtbo camera-imx214-csi0.dtbo.
Otherwise, the camera initialization will fail.
Supported Formats And Sizes
The following table shows the supported formats and resolutions for each stream.
Supported formats and Sizes
Note
The supported format, resolution, and frame rate are related to the capability of the sensor and SoC.
V4L2 Device Node
Camera ISP is a complicated feature that has a large number of device nodes. The camera ISP driver creates 132 video devices in total.
MTK-ISP-DIP-V4L2has 73 devicesmtk-camhas 56 devicesmtk-v4l2-camerahas 3 devices
MTK-ISP-DIP-V4L2 and mtk-cam are used by the camera ISP middleware.
The user can just focus on mtk-v4l2-camera which is for streaming.
v4l2-ctl --list-devices
...
MTK-ISP-DIP-V4L2 (platform:15000000.imgsys_fw):
(73 video devices)
/dev/media1
mtk-cam (platform:16000000.camisp):
(56 video devices)
/dev/media3
mtk-v4l2-camera (platform:mtkcam0):
/dev/video74
/dev/video75
/dev/video76
/dev/media2
Launch Camera
The V4L2 nodes with the name mtk-v4l2-camera can be used for streaming.
There are 3 V4L2 devices representing Preview, Video, and Capture streams.
The user can find the devices by the following command:
v4l2-ctl --list-device
mtk-v4l2-camera (platform:mtkcam0):
/dev/video74
/dev/video75
/dev/video76
cat /sys/class/video4linux/video{74,75,76}/name
mtk-v4l2-camera@0-Preview
mtk-v4l2-camera@0-Video
mtk-v4l2-camera@0-Capture
Important
The above v4l2 device node number is for example. The node number may change every time the system boots.
To quickly fetch the correct node and assign the environment variable:
declare -a VIDEO_DEV=(`v4l2-ctl --list-devices | grep mtk-v4l2-camera -A 3 | grep video | tr -d "\n"`)
printf "Preview Node\t= ${VIDEO_DEV[0]}\nVideo Node\t= ${VIDEO_DEV[1]}\nCapture Node\t= ${VIDEO_DEV[2]}\n"
Preview Node = /dev/video74
Video Node = /dev/video75
Capture Node = /dev/video76
The Preview and Video streams can deliver raw format images. The Capture stream is for JPEG outputs. The Capture stream is used to fetch the image that the user sees on the Preview or Video stream. The 3A configuration of the Capture stream relies on the Preview or Video stream. Therefore, before starting the Capture stream, make sure one of the Preview and Video streams is running.
Section Supported formats and Sizes shows the supported formats and sizes.
The user can also use the v4l2 utility, v4l2-ctl, to list formats.
v4l2-ctl -d ${VIDEO_DEV[0]} --list-formats-ext # List formats/sizes of Preview Stream
ioctl: VIDIOC_ENUM_FMT
Type: Video Capture Multiplanar
[0]: 'RGB3' (24-bit RGB 8-8-8)
Size: Discrete 640x360
Interval: Discrete 0.033s (30.000 fps)
...
[1]: 'YUYV' (YUYV 4:2:2)
Size: Discrete 640x360
Interval: Discrete 0.033s (30.000 fps)
...
v4l2-ctl -d ${VIDEO_DEV[2]} --list-formats-ext # List formats/sizes of Capture Stream
ioctl: VIDIOC_ENUM_FMT
Type: Video Capture Multiplanar
[0]: 'JPEG' (JFIF JPEG, compressed)
Size: Discrete 320x240
Interval: Discrete 0.033s (30.000 fps)
...
To launch the camera, the user can use v4l2-ctl, gst-launch-1.0, or other V4L2 applications.
Use
v4l2-ctlv4l2-ctl -d ${VIDEO_DEV[0]} --set-fmt-video=width=1920,height=1080,pixelformat=YUYV --stream-mmap --stream-count=1 --stream-to=/tmp/preview.yuv --verboseUse
gst-launch-1.0gst-launch-1.0 v4l2src device=${VIDEO_DEV[0]} ! video/x-raw,width=640,height=480,format=YUY2 ! waylandsink sync=false & gst-launch-1.0 v4l2src device=${VIDEO_DEV[1]} ! video/x-raw,width=640,height=480,format=NV12 ! waylandsink sync=false & gst-launch-1.0 v4l2src device=${VIDEO_DEV[2]} ! image/jpeg,width=640,height=480,format=JPEG ! jpegdec ! waylandsink sync=false &
Show Preview/Video/Capture streams on Weston
Moreover, the user can show 720P Preview content on Weston, record 10 seconds of FHD Video to an mp4 file, and Capture 4K JPEG images by the following commands:
# Show 720P Preview on Weston
gst-launch-1.0 v4l2src device=${VIDEO_DEV[0]} ! video/x-raw,width=1280,height=720,format=YUY2 ! waylandsink sync=false &
# Record 10 seconds of FHD Video
gst-launch-1.0 v4l2src device=${VIDEO_DEV[1]} num-buffers=300 ! video/x-raw,width=1920,height=1080,format=NV12 ! v4l2h264enc ! h264parse ! queue ! mp4mux ! filesink location=video.mp4 &
# Capture 4K JPEG image
v4l2-ctl -d ${VIDEO_DEV[2]} --set-fmt-video=width=3840,height=2160,pixelformat=JPEG,sizeimage=4194304 --stream-mmap=1 --stream-count=1 --stream-to=capture.jpg --verbose
Important
JPEG is a compression format, so the size of the compressed result may vary each time.
The user should ensure that the buffer size is large enough to accommodate the worst-case compressed size.
Otherwise, the JPEG content may be incomplete.
To set the buffer size, the user can refer to the above example sizeimage=4194304.
Note
To use MediaTek video codec and mdp hardware, the user has to load video.dtbo. For more details, please refer to Genio 700-EVK Video Codec.
Set Camera ISP Controls
For each stream, there are several controls that can be set. The following table shows the available controls. There are 12 V4L2 standard controls and 1 vendor-specific control. The user can use the control to set 3A (Auto Focus, Auto Exposure, and Auto White Balance) and ISP pipeline feature (Brightness, Contrast, Saturation, and Sharpness).
Available camera controls
The user can also use v4l2-ctl to list all controls by the following command:
v4l2-ctl --list-ctrls -d ${VIDEO_DEV[0]}
User Controls
brightness 0x00980900 (int) : min=0 max=100 step=1 default=0 value=0 flags=slider
contrast 0x00980901 (int) : min=0 max=10 step=1 default=0 value=0 flags=slider
saturation 0x00980902 (int) : min=0 max=10 step=1 default=0 value=0 flags=slider
hue 0x00980903 (int) : min=0 max=100 step=1 default=0 value=0 flags=slider
white_balance_automatic 0x0098090c (bool) : default=1 value=1
exposure 0x00980911 (int) : min=-40 max=40 step=1 default=0 value=0
power_line_frequency 0x00980918 (menu) : min=0 max=3 default=3 value=3 (Auto)
white_balance_temperature 0x0098091a (int) : min=2700 max=6500 step=1 default=6500 value=6500
sharpness 0x0098091b (int) : min=0 max=10 step=1 default=0 value=0 flags=slider
iso 0x009819a9 (int) : min=100 max=6400 step=100 default=100 value=100
Camera Controls
auto_exposure 0x009a0901 (menu) : min=0 max=1 default=0 value=0 (Auto Mode)
exposure_time_absolute 0x009a0902 (int) : min=100000 max=100000000 step=1 default=33000000 value=33000000
focus_absolute 0x009a090a (int) : min=0 max=255 step=1 default=0 value=0
focus_automatic_continuous 0x009a090c (bool) : default=0 value=0
Note
Preview stream and Video stream share the same control setting, so the user modifies one and affects another. However, the control setting of Capture is separated, it doesn’t affect others.
Auto Focus
Note
The SONY IMX214 sensor doesn’t support AF, so this is not yet available.
Auto Exposure
Auto Exposure can be set to AUTO mode and MANUAL mode.
In AUTO mode, the exposure is calculated automatically. The user can set exposure compensation to adjust the exposure level.
v4l2-ctl -d ${VIDEO_DEV[0]} -c auto_exposure=0 # AUTO
v4l2-ctl -d ${VIDEO_DEV[0]} -c exposure=-40 # MIN
v4l2-ctl -d ${VIDEO_DEV[0]} -c exposure=0 # DEFAULT
v4l2-ctl -d ${VIDEO_DEV[0]} -c exposure=40 # MAX
In MANUAL mode, the exposure is decided manually by the user. The user can set exposure time and ISO to adjust the exposure level.
v4l2-ctl -d ${VIDEO_DEV[0]} -c auto_exposure=1 # MANUAL
v4l2-ctl -d ${VIDEO_DEV[0]} -c exposure_time_absolute=100000 # MIN
v4l2-ctl -d ${VIDEO_DEV[0]} -c exposure_time_absolute=100000000 # MAX
v4l2-ctl -d ${VIDEO_DEV[0]} -c iso=100 # MIN
v4l2-ctl -d ${VIDEO_DEV[0]} -c iso=6400 # MAX
Auto White Balance
Auto White Balance can be set to AUTO mode or MANUAL mode.
In AUTO mode, the white balance value is calculated automatically.
v4l2-ctl -d ${VIDEO_DEV[0]} -c white_balance_automatic=1 # AUTO
In MANUAL mode, the white balance value is decided manually by the user. The user can set the white balance temperature value.
v4l2-ctl -d ${VIDEO_DEV[0]} -c white_balance_automatic=0 # MANUAL
v4l2-ctl -d ${VIDEO_DEV[0]} -c white_balance_temperature=2800 # INCANDESCENT
v4l2-ctl -d ${VIDEO_DEV[0]} -c white_balance_temperature=3300 # WARM_FLUORESCENT
v4l2-ctl -d ${VIDEO_DEV[0]} -c white_balance_temperature=3800 # TWILIGHT
v4l2-ctl -d ${VIDEO_DEV[0]} -c white_balance_temperature=4400 # FLUORESCENT
v4l2-ctl -d ${VIDEO_DEV[0]} -c white_balance_temperature=5200 # DAYLIGHT
v4l2-ctl -d ${VIDEO_DEV[0]} -c white_balance_temperature=5800 # CLOUDY_DAYLIGHT
v4l2-ctl -d ${VIDEO_DEV[0]} -c white_balance_temperature=6300 # SHADE
Note
Although the temperature value is consecutive, the white balance change is discrete. Because the value is mapped to a white balance mode, such as INCANDESCENT, DAYLIGHT, etc.
ISP Pipeline Feature
There are 4 ISP pipeline feature controls (brightness, contrast, saturation, sharpness, and hue). The user can set them and observe the image quality change.
v4l2-ctl -d ${VIDEO_DEV[0]} -c brightness=0 # MIN
v4l2-ctl -d ${VIDEO_DEV[0]} -c brightness=100 # MAX
v4l2-ctl -d ${VIDEO_DEV[0]} -c contrast=0 # MIN
v4l2-ctl -d ${VIDEO_DEV[0]} -c contrast=10 # MAX
v4l2-ctl -d ${VIDEO_DEV[0]} -c saturation=0 # MIN
v4l2-ctl -d ${VIDEO_DEV[0]} -c saturation=10 # MAX
v4l2-ctl -d ${VIDEO_DEV[0]} -c sharpness=0 # MIN
v4l2-ctl -d ${VIDEO_DEV[0]} -c sharpness=10 # MAX
v4l2-ctl -d ${VIDEO_DEV[0]} -c hue=0 # MIN
v4l2-ctl -d ${VIDEO_DEV[0]} -c hue=10 # MAX