|Sep 24, 2023
|May 2, 2022
|Sep 10, 2019
|Oct 17, 2017
|May 25, 2015
#50 in Asynchronous
10,377 downloads per month
Used in 21 crates (17 directly)
Kafka Rust Client
This project is starting to be maintained by John Ward, the current status is that I am bringing the project up to date with the latest dependencies, removing deprecated Rust code and adjusting the tests.
Welcome to kafka-rust's new home: https://github.com/kafka-rust
- This library is primarily documented through examples in its API documentation.
- Documentation about Kafka itself can be found at its project home page.
Thank you to our sponsors, this helps me to spend more time on this project and also helps with infrastructure
Upstash: Serverless Kafka
This crate works with Cargo and is on crates.io. The API is currently under heavy movement although we do follow semantic versioning (but expect the version number to grow quickly).
kafka = "0.9"
To build kafka-rust the usual
cargo build should suffice. The crate
supports various features which can be turned off at compile time.
Cargo.toml and cargo's documentation.
Supported Kafka version
kafka-rust is tested for compatibility with a select few Kafka versions from 0.8.2 to 3.1.0. However,
not all features from Kafka 0.9 and newer are supported yet.
This is a higher-level consumer API for Kafka and is provided by the
kafka::consumer. It provides convenient offset management
support on behalf of a specified group. This is the API a client
application of this library wants to use for receiving messages from
This is a higher-level producer API for Kafka and is provided by the
kafka::producer. It provides convenient automatic partition
assignment capabilities through partitioners. This is the API a
client application of this library wants to use for sending messages
KafkaClient in the
kafka::client module is the central point of
this API. However, this is a mid-level abstraction for Kafka rather
suitable for building higher-level APIs. Applications typically want
to use the already mentioned
Nevertheless, the main features or
- Loading metadata
- Fetching topic offsets
- Sending messages
- Fetching messages
- Committing a consumer group's offsets
- Fetching a consumer group's offsets
Bugs / Features / Contributing
There's still a lot of room for improvement on
Not everything works right at the moment, and testing coverage could be better.
Use it in production at your own risk. Have a look at the
issue tracker and feel free
to contribute by reporting new problems or contributing to existing
ones. Any constructive feedback is warmly welcome!
As usually with open source, don't hesitate to fork the repo and
submit a pull request if you see something to be changed. We'll be
happy to see
kafka-rust improving over time.
When working locally, the integration tests require that you must have
Docker (1.10.0+) and docker-compose (1.6.0+) installed and run the tests via the
run-all-tests script in the
tests directory. See the
script itself for details on its usage.
Creating a topic
Note unless otherwise explicitly stated in the documentation, this library will ignore requests to topics which it doesn't know about. In particular it will not try to retrieve messages from non-existing/unknown topics. (This behavior is very likely to change in future version of this library.)
Given a local kafka server installation you can create topics with the
following command (where
kafka-topics.sh is part of the Kafka
kafka-topics.sh --topic my-topic --create --zookeeper localhost:2181 --partitions 1 --replication-factor 1
See also Kafka's quickstart guide for more information.
- rust-rdkafka is an emerging alternative Kafka client library for Rust based on
librdkafka. rust-rdkafka provides a safe Rust interface to librdkafka.