============================
ONNX Runtime - Analytical AI
============================

.. toctree::
   :hidden:
   :maxdepth: 1
   
   Overall Performance Comparison<onnxrt_analytical/onnxrt_performance>
   
.. important::

   ONNX Runtime support for Genio platforms is under active development. MediaTek will provide feature enhancements and expanded model coverage on a quarterly basis.
   
The resources in this section support the **Online** inference path for **ONNX Runtime-analytical AI** workloads. This section provides the necessary onnx models required to execute via the ONNX Runtime.

.. image:: /_asset/analytical-ai-onnx-workflow.png
   :alt: Analytical AI inference paths: ONNX Runtime
   :align: center
   :width: 100%

.. raw:: html

   <br>


ONNX Runtime is available on MediaTek Genio platforms to accelerate ONNX models with hardware support.

* **High Performance**: Utilize the **Neuron Execution Provider (EP)** on Genio 520 and 720 for NPU acceleration.
* **Broad Compatibility**: Standard **CPU EP** execution is available across all Genio platforms.
* **FP16 Focus**: FP16 models have the most complete hardware-acceleration support.
* **Quantized QDQ**: QDQ models have partially supported with NPU acceleration.

For best performance and compatibility, deploy FP16 ONNX models whenever possible.

Performance Comparison
======================

MediaTek provides a comprehensive matrix of performance data across different Genio platforms.

* To view a quick summary of **high-end platforms (G520/G720)**, see the tables below.
* To compare performance across **all platforms (G350, G510, G700, G1200)**, refer to the dedicated page:

.. button-link:: onnxrt_analytical/onnxrt_performance.html
   :color: primary
   :shadow:

   View Full Platform Performance Matrix


Supported Models on Genio Products
==================================

The following tables list ONNX models that have been validated on Genio platforms.
The current list contains **45 models**, grouped into three model families:

* TAO Related
* Legacy Analytical
* Robotic

If your model is **not** listed in the tables below, you are still welcome to try it.
For questions or issues, post on the `Genio Community forum <https://genio-community.mediatek.com/c/neuropilot/23>`_.

.. note::

   The performance statistics shown in these tables were measured using **NPU Execution Provider** with **performance mode enabled** across different Genio products, models, and data types.



TAO Related Models
==================

TAO-related models originate from NVIDIA TAO Toolkit or use TAO-based pretrained weights.

.. csv-table:: Models for TAO Related
   :class: longtable
   :file: /_asset/tables/ml-model-hub-onnx-tao.csv
   :width: 100%


Legacy Analytical Models
========================

Legacy analytical models are classic vision backbones and networks widely used **only for** benchmarking and reference.
The accuracy of the model is not addressed.


.. csv-table:: Models for Detection
   :class: longtable
   :file: /_asset/tables/ml-model-hub-onnx-detection.csv
   :width: 100%

.. csv-table:: Models for Classification
   :class: longtable
   :file: /_asset/tables/ml-model-hub-onnx-classification.csv
   :width: 100%

.. csv-table:: Models for Recognition
   :class: longtable
   :file: /_asset/tables/ml-model-hub-onnx-recognition.csv
   :width: 100%

Robotic Models
==============

Robotic models target robotic perception and control workloads, such as grasping, navigation, or policy learning. These models demonstrate the performance of ONNX Runtime on Genio platforms for specialized robotic tasks.

.. csv-table:: Models for Robotic
   :class: longtable
   :file: /_asset/tables/ml-model-hub-onnx-robotic.csv
   :width: 100%

Performance Notes and Limitations
=================================

.. note::

   The measurements were obtained using **onnxruntime_perf_test**, and each model’s performance can vary depending on:

   * The specific Genio platform and hardware configuration.
   * The version of the board image and evaluation kit (EVK).
   * The selected backend and model variant.

   To obtain the most accurate performance numbers for your use case, you must run the application directly on the target platform.

Current Limitations
===================

ONNX-GAI (generative AI) models are **not** officially supported at this time.
Some proof-of-concept experiments exist internally, but they are not production-ready and are not part of the validated model list.

For the latest roadmap or early-access updates on ONNX-GAI support, please refer to the `Genio Community forum <https://genio-community.mediatek.com/c/neuropilot/23>`_.