nexa-sdk

Nexa SDK is an on-device inference framework that runs any model on any device, across any backend. It runs on CPUs, GPUs, NPUs with backend support for CUDA, Metal, Vulkan, and Qualcomm NPU. It handles multiple input modalities including text 📝, image 🖼️, and audio 🎧. The SDK includes an OpenAI-compatible API server with support for JSON schema-based function calling and streaming. It supports model formats such as GGUF, MLX, Nexa AI's own .nexa format, enabling efficient quantized inference across diverse platforms.

How to Use

• Download & follow instruction on https://sdk.nexa.ai.
• Run commands in terminal. GitHub Repo: https://github.com/NexaAI/nexa-sdk

Nexa SDK's

Key Features

• Run multimodal models (text, speech, vision understanding and image generation) locally
• Integrate into on-device AI apps
• First NPU-Aware Multimodal Inference Stack
• Run models from Hugging Face
• Supports model formats such as GGUF, MLX, Nexa AI's own .nexa format
• OpenAI-compatible API server

Use Cases

• LLM - Chat, reasoning, RAG, etc.
• ASR (Real-time transcription) - Meeting, video/audio and conversation transcription, etc.
• Text to speech - Audiobook, video voiceover, accessibility features, etc.
• Image understanding - OCR, scene and sentiment description, quality control, etc.
• Image generation - Character design, image edits, e-commerce, etc.
• Tool use - AI agent, app integration, etc.