edenhill/librdkafka
FreeThe Apache Kafka C/C++ library
About edenhill/librdkafka
librdkafka is a high-performance C library implementation of the Apache Kafka protocol, providing Producer, Consumer, and Admin clients. Designed for message delivery reliability and low latency, it achieves over 1 million messages/second for producers and 3 million messages/second for consumers. The library supports Exactly‑Once‑Semantics (EOS) including idempotent and transactional producers, a high‑level balanced KafkaConsumer, and a preview of the Share Consumer (KIP‑932). It includes built‑in compression (snappy, gzip, lz4, zstd), SSL/TLS, and multiple SASL mechanisms (GSSAPI/Kerberos, PLAIN, SCRAM, OAUTHBEARER). The API is guaranteed stable for both C and C++ with ABI safety, and it is portable across Linux, macOS, Windows, Solaris, FreeBSD, AIX, and more. Prebuilt packages are available for Debian, Ubuntu, RPM-based systems, and Gentoo. The library is licensed under the 2‑clause BSD license.
Key Features
Pros & Cons
- High performance: over 1 million msgs/sec producer, 3 million msgs/sec consumer
- Full Exactly-Once-Semantics support for reliable message processing
- Portable across multiple operating systems and architectures
- Comprehensive security features (SSL, SASL)
- Stable API with ABI safety guarantee for long-term maintenance
- Active development and community support under Confluent
- Master branch is actively developed; use latest stable release for production
- Requires familiarity with Apache Kafka concepts
- C library may require manual memory management