========================
Genio AI Developer Guide
========================

.. important::

   For the most comprehensive collection of pre-trained models, real-world performance benchmarks, and end-to-end deployment tutorials, please visit the official `IoT AI Hub <https://mediatek.gitlab.io/genio/doc/iot-aihub/master/>`_.

The Genio AI Developer Guide provides a comprehensive resource for building, optimizing, and deploying artificial intelligence workloads on MediaTek Genio IoT platforms. By utilizing the **NeuroPilot (NP)** software stack, developers can leverage dedicated hardware accelerators, including the Deep Learning Accelerator (MDLA) and Video Processing Unit (VPU), to achieve high-performance edge AI inference.

.. image:: /_asset/sw_yocto_ai_sw_stack_common.png
   :alt: Genio AI software stack overview for Analytical and Generative AI
   :align: center
   :width: 100%

.. raw:: html

   <br>

Core Concepts and Architecture
==============================

To deploy AI effectively on Genio platforms, developers must understand the underlying software-hardware relationship and the available inference paths.

Software-Hardware Binding
-------------------------

On Genio platforms, the NeuroPilot software version is **tightly coupled** to the hardware generation of the System-on-Chip (SoC). Each platform has a fixed NPU operator set and requires version-matched tools for conversion and compilation. Refer to :ref:`Understanding the Software-Hardware Binding <npu-soft-hard-binding>` for detailed versioning information.

Inference Paths
---------------

The Genio AI software stack supports two primary execution modes:

*   **Online Inference:** Utilizes AI frameworks such as **LiteRT (TFLite) Interpreter** or **ONNX Runtime** on the device to compile and execute models.
*   **Offline Inference:** Executes pre-compiled **Deep Learning Archive (DLA)** models directly through the **Neuron Runtime**, offering minimal overhead and predictable latency.

For a detailed comparison of these paths, see :doc:`Software Architecture </ai_hub/software_architecture>`.

AI Workload Categories
======================

MediaTek Genio platforms support a wide range of AI models, categorized into Analytical AI and Generative AI (GAI).

Analytical AI
-------------

Analytical AI focuses on traditional vision and recognition tasks. Developers can choose between the TFLite path and the ONNX Runtime path:

*   **TFLite Path:** Optimized for vision tasks such as classification, detection, and recognition. See :doc:`TFLite - Analytical AI </ai_hub/model_zoo/litert_analytical>`.
*   **ONNX Runtime Path:** Offers a cross-platform engine for running models from various frameworks. See :doc:`ONNX Runtime Overview </ai_hub/supported_os/yocto/onnxruntime/introduction>`.

Generative AI (GAI)
-------------------

Genio supports large-scale generative models, including Large Language Models (LLMs), Vision-Language Models (VLMs), and Stable Diffusion. These workloads typically follow the offline inference path for maximum efficiency. See :doc:`LiteRT - Generative AI </ai_hub/model_zoo/litert_gai>`.

Operating System Support
========================

The AI software stack behavior and update policies vary across different operating systems.

.. list-table:: AI Framework Support by OS
   :header-rows: 1
   :widths: 20 40 40

   * - OS
     - TFLite (LiteRT) Variant
     - Features
   * - **Android**
     - Proprietary MediaTek-optimized
     - Tightly coupled to Android releases with fixed CPU fallback behavior.
   * - **Yocto**
     - Open-source LiteRT and ONNX Runtime
     - Flexible customization and regular upgrades for improved compatibility.
   * - **Ubuntu**
     - Standard open-source packages
     - Suitable for desktop-like development and server-class evaluation.

For more details on OS-specific setup, refer to :doc:`Operating System Overview </ai_hub/supported_os/overview_os>`.

Development Workflow
====================

The path from a pretrained model to on-device execution involves several key stages:

1.  **Model Conversion:** Transform source models into TFLite or ONNX format using the :doc:`Model Converter </ai_hub/ai-workflow/model_converter>`.
2.  **Model Visualization:** Inspect tensor shapes and data types using :doc:`Visualization Tools </ai_hub/ai-workflow/model_visualization>`.
3.  **Compilation:** Use the :doc:`Neuron SDK </ai_hub/ai-workflow/neuron-sdk>` to generate hardware-specific DLA files for offline inference.
4.  **Profiling:** Analyze NPU utilization and performance metrics with :doc:`Neuron Studio </ai_hub/ai-workflow/neuron_studio>`.
5.  **Evaluation:** Verify model integrity through :doc:`Accuracy Evaluation </ai_hub/ai-workflow/accuracy_evaluation>`.

To start your first project, follow the :doc:`Get Started With AI Tools </ai_hub/ai-workflow/overview>` guide.

Resources and Support
=====================

*   **Model Hub:** Access a curated collection of pre-trained models and benchmark data in the :doc:`Model Zoo </ai_hub/model_zoo/overview>`.
*   **Developer Resources:** Obtain SDK bundles, documentation, and tools by platform in :doc:`AI Development Resources </ai_hub/related_resource>`.
*   **Troubleshooting:** Resolve common deployment failures and operator support issues in :doc:`Troubleshooting Operator Support </ai_hub/supported_os/yocto/litert/troubleshooting>`.