madsim-rdkafka

The rdkafka simulator on madsim. Mirrors rdkafka v0.34.0 and librdkafka 2.3.0.

Usage

Replace all rdkafka entries in your Cargo.toml:

[dependencies]
rdkafka = { version = "0.3", package = "madsim-rdkafka" }

API Modification

This crate roughly follows the rdkafka API but is NOT exactly the same.

The following functions are modified to be async:

• FromClientConfig::from_config
• FromClientConfigAndContext::from_config_and_context
• ClientConfig::create
• ClientConfig::create_with_context
• Client::fetch_metadata
• Client::fetch_watermarks[^1]
• Client::fetch_group_list
• Consumer::seek
• Consumer::seek_partitions
• Consumer::commit
• Consumer::commit_consumer_state
• Consumer::commit_message
• Consumer::committed
• Consumer::committed_offsets
• Consumer::offsets_for_timestamp
• Consumer::offsets_for_times
• Consumer::fetch_metadata
• Consumer::fetch_watermarks[^1]
• Consumer::fetch_group_list
• Producer::flush
• Producer::init_transactions
• Producer::send_offsets_to_transaction
• Producer::commit_transaction
• Producer::abort_transaction

[^1]: wrapped in tokio::task::spawn_blocking

DNS Resolution

This crate has cherry-picked a commit from Materialize to support rewriting broker addresses.

A new method is added to ClientContext:

• ClientContext::rewrite_broker_addr