Neura Market
confluentinc/librdkafka logo

confluentinc/librdkafka

Free

The Apache Kafka C/C++ library

FreeFree tier
Type
Open Source
Company
Confluent Inc.
Visit Website GitHub (38 stars)

About confluentinc/librdkafka

librdkafka is a robust, high-performance C library implementation of the Apache Kafka protocol. It provides producer, consumer, and admin clients with full support for Exactly-Once-Semantics (EOS), including idempotent and transactional producers. The library handles compression (snappy, gzip, lz4, zstd), SSL/TLS encryption, and multiple SASL mechanisms (GSSAPI, PLAIN, SCRAM, OAUTHBEARER). Designed for reliability and speed, it achieves over 1 million messages/second for producers and 3 million messages/second for consumers. librdkafka is portable across Linux, macOS, Windows, Solaris, FreeBSD, AIX, and other platforms, and is distributed under the 2-clause BSD license. It also offers a high-level balanced KafkaConsumer, admin client functionality, and comprehensive statistics metrics.

Key Features

High-level Producer with Idempotent and Transactional support
High-level balanced KafkaConsumer (requires broker >= 0.9)
Admin client for topic and broker management
Exactly-Once-Semantics (EOS) support
Compression: snappy, gzip, lz4, zstd
SSL/TLS and SASL (GSSAPI, PLAIN, SCRAM, OAUTHBEARER) support
Portable across Linux, macOS, Windows, Solaris, FreeBSD, AIX
Prebuilt packages: Debian, RPM, Gentoo
Statistics metrics for monitoring
Guaranteed ABI stability for C API

Pros & Cons

Pros
  • Very high performance: >1M msg/s producer, >3M msg/s consumer
  • Full Exactly-Once-Semantics support for delivery guarantees
  • Portable across many operating systems and architectures
  • Stable C and C++ APIs with long-term backwards compatibility
  • Comprehensive documentation and configuration options
  • Open-source under permissive BSD license
Cons
  • C/C++ only – not suitable for higher-level languages without wrapping
  • Requires understanding of Kafka concepts and configuration
  • Master branch is actively developed; production use should use latest release
  • Not an AI/ML tool; does not provide AI capabilities

Best For

Building high-throughput Kafka producers and consumers in C/C++ applicationsReal-time data streaming and event-driven architecturesData pipeline integration with Apache KafkaEmbedded systems requiring a lightweight Kafka clientPerformance-critical applications needing low-latency message delivery

FAQ

What is librdkafka?
librdkafka is a C library implementation of the Apache Kafka protocol, providing producer, consumer, and admin clients. It is designed for high performance and reliability.
What license is librdkafka released under?
librdkafka is licensed under the 2-clause BSD license.
Is librdkafka production-ready?
Yes, but the master branch is actively developed; the latest release should be used for production environments.
What platforms does librdkafka support?
librdkafka is portable and runs on Linux, macOS, Windows, Solaris, FreeBSD, AIX, and other platforms.