9 releases (breaking)

0.7.0 Sep 16, 2023
0.6.1 Jul 26, 2023
0.6.0 Mar 1, 2023
0.5.1 Feb 23, 2023
0.1.0 Dec 14, 2021

#101 in Network programming

Download history 3456/week @ 2023-06-08 3262/week @ 2023-06-15 2589/week @ 2023-06-22 2724/week @ 2023-06-29 3797/week @ 2023-07-06 2916/week @ 2023-07-13 3104/week @ 2023-07-20 3660/week @ 2023-07-27 3121/week @ 2023-08-03 3046/week @ 2023-08-10 3070/week @ 2023-08-17 3484/week @ 2023-08-24 3706/week @ 2023-08-31 4242/week @ 2023-09-07 3846/week @ 2023-09-14 3697/week @ 2023-09-21

16,019 downloads per month
Used in 4 crates (3 directly)


42K SLoC

Kafka-Protocol Build crates.io docs.rs

Rust implementation of the Kafka wire protocol.

Unlike other Kafka protocol implementations, this project uses code generation to cover the entire Kafka API surface, including different protocol versions. See Kafka's repo for an example of protocol schema.


Protocol messages are generated against the most recent stable Kafka release, currently 3.3.2.

Although the Kafka protocol remains relatively stable and strives to be backwards compatible, new fields are occasionally added. In order to ensure forward compatibility with the protocol, this crate marks all exported items as #[non-exhaustive]. Protocol messages can be constructed using either Default::default or their provided builder.

Working with messages

Using Default::default:

use kafka_protocol::messages::{ApiKey, MetadataRequest, RequestHeader};
use kafka_protocol::protocol::StrBytes;

let mut header = RequestHeader::default();
header.client_id = Some(StrBytes::from_str("my-client"));
header.request_api_key = ApiKey::MetadataKey as i16;
header.request_api_version = 12;

let mut request = MetadataRequest::default();
request.topics = None;
request.allow_auto_topic_creation = true;

Using kafka_protocol::protocol::Builder:

use kafka_protocol::messages::{ApiKey, MetadataRequest, RequestHeader};
use kafka_protocol::protocol::{Builder, StrBytes};

let header = RequestHeader::builder()
    .request_api_key(ApiKey::MetadataKey as i16)
let request = MetadataRequest::builder()


Once a message has been created, it can be serialized using Encodable, writing the struct to a provided bytes::BytesMut. The API version for the given message matching the version specified in the request header must be provided.

use bytes::BytesMut;
use kafka_protocol::messages::MetadataRequest;
use kafka_protocol::protocol::Encodable;

let mut bytes = BytesMut::new();
let request = MetadataRequest::default();
request.encode(&mut bytes, 12).unwrap();


Messages can be decoded using Decodable and providing the matching API version from their corresponding request.

use bytes::Bytes;
use kafka_protocol::messages::ApiVersionsRequest;
use kafka_protocol::protocol::Decodable;

let bytes: [u8; 25] = [
        0x12, 0x61, 0x70, 0x61, 0x63, 0x68, 0x65, 0x2d,
        0x6b, 0x61, 0x66, 0x6b, 0x61, 0x2d, 0x6a, 0x61,
        0x76, 0x61, 0x06, 0x32, 0x2e, 0x38, 0x2e, 0x30,
let res = ApiVersionsRequest::decode(&mut Bytes::from(bytes.to_vec()), 3).unwrap();


Run cargo run -p protocol_codegen in the root path of this repo to generate/update the Rust codes via the latest Kafka protocol schema.

Originally implemented by @Diggsey in a minimal Kafka client implementation Franz


~70K SLoC