nn2

Free

Tiny zero-dependency neural network inference engine in pure C with hand-written SIMD kernels (AVX-512 / AVX2 / NEON). Runs the FaceX face recognition stack on Apple Silicon, ARM SBCs and ESP32-P4. Apache 2.0.

FreeFree tier

Type

Open Source

Visit Website GitHub (40 stars)

About nn2

nn2 is a tiny zero-dependency neural network inference engine written in pure C, featuring hand-tuned SIMD kernels (AVX-512, AVX2, NEON). It is designed for real-time object detection using YOLO models and is part of the FaceX face recognition stack. The engine achieves up to 1.50x speedup over ONNX Runtime on Intel i5-11500, supports multiple platforms (Apple Silicon, ARM SBCs, ESP32-P4), and includes a production-ready NVR server with a web dashboard, REST API, motion-gated inference, Kalman tracking, SORT tracker, line-crossing counting, and zone intrusion detection. It is licensed under Apache 2.0 and has a tiny binary size of about 520KB.

Key Features

Zero-dependency pure C inference engine

Hand-tuned SIMD kernels (AVX-512, AVX2, NEON)

1.50x faster than ONNX Runtime on single-frame inference

Supports Apple Silicon, ARM SBCs, and ESP32-P4

Production NVR server with embedded web dashboard and REST API

Motion-gated inference (~0.05ms per frame) and Kalman tracking (~0.01ms)

SORT tracker with persistent object IDs

Line-crossing people counting and zone intrusion detection

C API for easy integration

Weight export from PyTorch (YOLOv8n)

Pros & Cons

Pros

Exceptional performance: up to 1.50x faster than ONNX Runtime
Zero external dependencies – easy to build and deploy
Extremely small binary (~520KB) suitable for resource-constrained devices
Open-source under Apache 2.0 license
Includes comprehensive NVR features (dashboard, API, tracking, counting)
Supports multiple architectures (x86, ARM, RISC-V via ESP32-P4)

Cons

Currently limited to YOLO architecture models (YOLOv8n demonstrated)
Requires manual weight conversion from PyTorch using provided tools
No GPU acceleration support – CPU-only inference
Documentation is sparse beyond the README

Best For

Real-time object detection on edge devicesSmart NVR (Network Video Recorder) systemsEmbedded surveillance with motion gating and trackingFace recognition pipeline (part of FaceX stack)High-throughput camera processing (70+ cameras on a single CPU)

FAQ

What hardware does nn2 support?

nn2 supports x86 processors with AVX-512/AVX2, Apple Silicon (NEON), ARM SBCs (NEON), and ESP32-P4 (RISC-V with custom SIMD).

How fast is nn2 compared to ONNX Runtime?

On an Intel i5-11500 with 320x320 input, nn2 achieves 8.5ms (117 FPS) vs 12.7ms (78 FPS) for ONNX Runtime 1.23, a 1.50x speedup.

Does nn2 include a NVR server?

Yes, nn2 includes a production NVR server binary that provides a web dashboard, REST API, motion-gated inference, Kalman tracking, line-crossing counting, and zone intrusion detection.

Is nn2 free to use?

Yes, nn2 is open-source under the Apache 2.0 license and can be used for free.