#tcp-udp #kafka #tls #udp #tcp #tokio

crab-kafka

Forward <TCP|UDP> + TLS traffic to kafka

8 releases

Uses new Rust 2024

0.0.8 Feb 24, 2025
0.0.7 Nov 14, 2024
0.0.6 Sep 13, 2024
0.0.5 Jul 6, 2024
0.0.2 May 28, 2023

#1642 in Network programming

Download history 122/week @ 2024-11-13 8/week @ 2024-11-20 1/week @ 2025-02-05 4/week @ 2025-02-12 105/week @ 2025-02-19 37/week @ 2025-02-26

147 downloads per month

Apache-2.0

52KB
1K SLoC

crab_kafka

Crates.io Apache licensed Docs.rs

Forward <TCP|UDP> + TLS traffic to kafka.

Based on tokio and rust rdkafka

Basic Usage

It's strongly encouraged the use of alternative allocator like MiMalloc

use anyhow::Result;
use std::collections::HashMap;
use mimalloc::MiMalloc;
use crab_kafka::{forwarder::ForwarderBuilder,Receiver,PartitionStrategies,CheckpointStrategies,TransformStrategies};

#[global_allocator]
static GLOBAL: MiMalloc = MiMalloc;

#[tokio::main]
async fn main() -> Result<()> {
    ForwarderBuilder::default()
    .receiver(Receiver::new_tcp_stream("127.0.0.1".to_owned(), "8888".to_owned(), 2000))
    .checkpoint(CheckpointStrategies::OpenDoors)
    .partition(PartitionStrategies::new_sticky_round_robin())
    .transform(TransformStrategies::NoTransform)
    .kafka_settings(HashMap::from([("bootstrap.servers".to_owned(),"broker:29091".to_owned())]))
    .topic("test_topic".to_owned())
    .cache_size(1000)
    .stats_interval(10)
    .build()
    .unwrap()
    .await
}

Examples

Dependencies

~26–42MB
~663K SLoC