.. include:: /keyword.rst

===============
Model Converter
===============

Overview
========

This page overviews methods for converting YOLOv5s PyTorch models to TensorFlow Lite (TFLite) format. It covers two primary conversion methods: using an open-source converter script and the MTK Converter tool provided by the NeuroPilot SDK.

.. figure:: /_asset/sw_yocto_ml-guide_neuron-sdk_neuron-dev-flowl-dev_flow_overview.png
   :alt: Model Converter Overview
   :align: center
   :width: 80%

.. raw:: html

   <br><br>


YOLOv5s Model Conversion
========================

.. toctree::
    :hidden:

    Open Source Converter <model_converter/open_source_converter>
    NeuroPilot Converter Tool <model_converter/neuropilot_converter_tool>
    Accuracy Evaluation <model_converter/accuracy_evaluation>

1. **Open Source Converter**:

   - Use the `export.py` script provided by the open-source community. This script leverages TensorFlow Lite Converter to create a converter object that converts the YOLOv5 model from PyTorch format to TensorFlow Lite format.
   - **Source Link**: `YOLOv5 GitHub Repository <http://github.com/ultralytics/yolov5>`_

.. note::

   The methods provided by open source converters may vary depending on the source. The instructions in this documentation focus specifically on the end-to-end flow for converting YOLOv5s models. For other models or converters, additional research and steps might be required.

2. **NeuroPilot Converter tool**:

   - Download the NeuroPilot SDK All-In-One Bundle, install the NeuroPilot Converter tool that matches the user's Python version, and use the MTK converter to convert the YOLOv5 model from PyTorch to TensorFlow Lite format.

For detailed conversion steps, please refer to the :doc:`Open Source Converter </sw/yocto/ml-guide/neuron-dev-flow/model_converter/open_source_converter>` page and the :doc:`NeuroPilot Converter tool </sw/yocto/ml-guide/neuron-dev-flow/model_converter/neuropilot_converter_tool>` page.


.. _Unsupported Operations in DLA Conversion (Optional):

Unsupported Operations in DLA Conversion (Optional)
===================================================

If you encounter "unsupported operation" errors when converting the TFLite model to DLA format, you may need to prune the model to remove or replace unsupported operations. The following steps demonstrate how to prune a TFLite model for DLA compatibility:

Example Error Message
---------------------

Below is a detailed example of an "unsupported operation" error that may occur when attempting to convert a TFLite model to DLA format.

   .. code-block:: bash

       ncc-tflite --arch=mdla3.0 yolox_s_quant.tflite

   .. code-block:: bash

       OP[2]: STRIDED_SLICE
        ├ MDLA: Last stride size should be equal to 1
        ├ EDMA: unsupported operation
        ├ EDMA: unsupported operation
       OP[3]: STRIDED_SLICE
        ├ MDLA: Last stride size should be equal to 1
        ├ EDMA: unsupported operation
        ├ EDMA: unsupported operation
       OP[4]: STRIDED_SLICE
        ├ MDLA: Last stride size should be equal to 1
        ├ EDMA: unsupported operation
        ├ EDMA: unsupported operation
       OP[5]: STRIDED_SLICE
        ├ MDLA: Last stride size should be equal to 1
        ├ EDMA: unsupported operation
        ├ EDMA: unsupported operation
       ERROR: Cannot find an execution plan because of unsupported operations
       ERROR: Fail to compile yolox_s_quant.tflite


.. note::

    In this example:

    - The `STRIDED_SLICE` operation appears multiple times (e.g., OP[2], OP[3], OP[4], OP[5]), and each occurrence fails due to the same issue with stride size.
    - For MDLA, the TFLite model didn't meet the requirement “Last stride size should be equal to 1”, causing the operation to be unsupported in this context.
    - For EDMA, the `STRIDED_SLICE` operation is completely unsupported, leading to further execution failures.
    - As a result, the converter cannot generate a valid execution plan and ultimately fails to compile the model.

    To resolve this issue, you need to modify or prune the unsupported operations (such as `STRIDED_SLICE`) in the model, or adjust the stride size to meet the MDLA’s requirements, this will allow a successful conversion to DLA format.

1. **Inspect the model**:

   - Use tools like `Netron <https://netron.app/>`_ to inspect the model's input/output tensors and internal layers. Identify any unsupported operations that need modification.

.. figure:: /_asset/yoloX-s_unsupported_operation.png
   :alt: yoloX-s_unsupported_operation
   :align: center
   :width: 50%

.. raw:: html

   <br><br>

2. **Prune the model**:

   - Use the `TFLiteEditor` class from `mtk_converter` to prune the TFLite model by removing unsupported operations or modifying the model structure to ensure compatibility.

   Example script to prune a model:

   .. code-block:: python

       import mtk_converter

       editor = mtk_converter.TFLiteEditor("yolox_s_quant.tflite")
       output_file = "yolox_s_quant_7-303.tflite"
       input_names = ["input.32"]
       output_names = ["1360"]

       _ = editor.export(output_file=output_file, input_names=input_names, output_names=output_names)


3. **Validate the pruned model**:

   - After pruning the model, validate it by running inference and comparing results with the original model to ensure functionality is maintained.

.. note::
   The input and output tensor names (`input_names` and `output_names`) used in the script are specific to the example model. You must modify them based on your own model's structure.


.. figure:: /_asset/yoloX-s_input_name.png
   :alt: yoloX-s_input_name
   :align: center
   :width: 80%


.. raw:: html

   <br><br>


.. figure:: /_asset/yoloX-s_output_name.png
   :alt: yoloX-s_output_name
   :align: center
   :width: 80%