NeuroPilot Converter Tool

Overview

This page provides a comprehensive guide for converting YOLOv5s PyTorch models to TensorFlow Lite (TFLite) format using the NeuroPilot Converter Tool. It includes detailed instructions for setting up the YOLOv5 environment, applying necessary patches, and converting models to both int8 and FP32 TFLite formats. Additionally, it covers a detailed step-to-step guide for converting TFLite models to DLA format.

YOLOv5s Model Conversion

Setting Up YOLOv5 Environment

Note

For better compatibility, it is recommended to use Python 3.7 when working with these models, as it has higher compatibility with certain libraries and frameworks.

Install and Verify NeuroPilot Converter Tool

Download NeuroPilot SDK All-In-One Bundle (version:7.X.X):

Visit the download page: NeuroPilot Converter Tool

Extract the Bundle:

tar zxvf neuropilot-sdk-basic-<version>.tar.gz

Install NeuroPilot Converter Tool for Python 3.7:

pip3 install /path/to/neuropilot-sdk-basic-<version>/offline_tool/mtk_converter-<version>+release-cp37-cp37-manylinux_2_5_x86_64.manylinux1_x86_64.whl

Verify the Installation:

The following script prints the installed package version:
```
python3 -c 'import mtk_converter; print(mtk_converter.__version__)'
```

Setting Up and Exporting YOLOv5

Clone the repository:

git clone http://github.com/ultralytics/yolov5
cd yolov5
git reset --hard 485da42

Install Python packages and dependencies:
```
pip3 install -r requirements.txt
pip3 install torch==1.9.0 torchvision==0.10.0
```
Note

The mtk_converter.PyTorchConverter only supports PyTorch versions between 1.3.0 and 2.0.0. The detected version v2.3.1+cu121 is not within this supported range, causing a runtime error. Therefore, it is necessary to install a compatible version of PyTorch and torchvision to ensure compatibility.
Apply Patch:
```
wget https://mediatek-aiot.s3.ap-southeast-1.amazonaws.com/aiot/download/model-zoo/scripts/model_conversion_YOLOv5s_example_20240916.zip
unzip -j model_conversion_YOLOv5s_example_20240916.zip
git apply Fix_yolov5_mtk_tflite_issue.patch
```
Note

The Fix_yolov5_mtk_tflite_issue.patch adds support for MTK TensorFlow Lite (MTK TFLite) in the YOLOv5 model export script. It includes:
- Adding mtk_tflite as a supported export format.
- Modifying the Detect module’s forward method to only include convolution operations.
- Implementing post-processing operations for MTK TFLite.
- Extending the DetectMultiBackend class to handle MTK TFLite models.

Exporting PyTorch Model to TorchScript Format:

python3 export.py --weight yolov5s.pt --img-size 640 640 --include torchscript

Convert PyTorch Model to TFLite Model

Convert to INT8 format

Prepare Calibration Data:

To prepare the calibration data, create a new Python script named prepare_calibration_data.py in the root directory of the YOLOv5 project. This script will generate a set of images that are used for model quantization calibration.

python3 prepare_calibration_data.py

import os
import numpy as np
from utils.dataloaders import LoadImagesAndLabels
from utils.general import check_dataset

data = 'data/coco128.yaml'
num_batches = 100
calib_dir = 'calibration_dataset'
os.makedirs(calib_dir)

# Retrieve the first 100 images from the training set with batch_size = 1
dataset = LoadImagesAndLabels(check_dataset(data)['train'], batch_size=1)

for idx, (im, _target, _path, _shape) in enumerate(dataset):
    if idx >= num_batches:
        break

    # Expand shape from (3, 640, 640) to (1, 3, 640, 640)
    im = np.expand_dims(im, axis=0).astype(np.float32)
    # 0 - 255 to 0.0 - 1.0
    im /= 255
    np.save(os.path.join(calib_dir, 'batch-{:05d}.npy'.format(idx)), im)

Convert to int8 TFLite:

To perform the conversion of the PyTorch model to an int8 TFLite format, create a new Python script named convert_to_quant_tflite.py in the root directory of your YOLOv5 project. This script will handle the conversion process by utilizing the pre-generated calibration data and converting the model into the quantized TFLite format.

python3 convert_to_quant_tflite.py

import os
import numpy as np
import mtk_converter

calib_dir = 'calibration_dataset'

converter = mtk_converter.PyTorchConverter.from_script_module_file(
    'yolov5s.torchscript', input_shapes=[(1, 3, 640, 640)]
)

def data_gen():
    """Return an iterator for the calibration dataset."""
    for fn in sorted(os.listdir(calib_dir)):
        yield [np.load(os.path.join(calib_dir, fn))]

converter.quantize = True
converter.calibration_data_gen = data_gen
converter.convert_to_tflite('yolov5s_int8_mtk.tflite')

Convert to FP32 format

To convert the PyTorch model to an FP32 TFLite format, create a new Python script named convert_to_tflite.py in the root directory of your YOLOv5 project. This script will handle the conversion process to generate a non-quantized, full-precision TFLite model.

python3 convert_to_tflite.py

import mtk_converter

converter = mtk_converter.PyTorchConverter.from_script_module_file(
    'yolov5s.torchscript', input_shapes=[(1, 3, 640, 640)]
)
converter.convert_to_tflite('yolov5s_mtk.tflite')

Convert TFLite Model to DLA Model

Convert to INT8 format

Download NeuroPilot SDK All-In-One Bundle:

Visit the download page: NeuroPilot Downloads

Extract the Bundle:

tar zxvf neuropilot-sdk-basic-<version>.tar.gz

Setting Environment Variables:

export LD_LIBRARY_PATH=/path/to/neuropilot-sdk-basic-<version>/neuron_sdk/host/lib

TFLite Model convert to DLA format:

/path/to/neuropilot-sdk-basic-<version>/neuron_sdk/host/bin/ncc-tflite --arch=mdla3.0 yolov5s_int8_mtk.tflite

Note

To ensure compatibility with your device, please download and use NeuroPilot SDK version 6. Other versions might not be fully supported.

Convert to FP32 format

Setting Environment Variables:

export LD_LIBRARY_PATH=/path/to/neuropilot-sdk-basic-<version>/neuron_sdk/host/lib

Convert to DLA format:

/path/to/neuropilot-sdk-basic-<version>/neuron_sdk/host/bin/ncc-tflite --arch=mdla3.0 --relax-fp32 yolov5s_mtk.tflite