1 unstable release
0.1.3 | Nov 1, 2023 |
---|---|
0.1.2 |
|
0.1.1 |
|
0.1.0 |
|
#12 in #described
35 downloads per month
19KB
176 lines
hedged
This crate provides functionality to perform hedged requests, inspired by the strategies described in "The Tail at Scale".
Hedged requests help in mitigating latency variability in distributed systems by initiating redundant operations and using the result of the first one to complete.
Features
tokio
: Enables asynchronous support, including theHedge::send
method for performing hedged requests.
Usage
use std::{
sync::{
atomic::{AtomicU32, Ordering},
Arc,
},
time::Duration,
};
use hedged::Hedge;
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let hedge = Hedge::new(7, 64, Duration::from_secs(30), 1, 0.95)?;
let fast_request = || async { "ok" };
let (result, rem) = hedge.send(fast_request).await;
assert!(rem.is_none());
assert_eq!(result, "ok");
let called = Arc::new(AtomicU32::new(0));
let slow_request = {
|| {
called.fetch_add(1, Ordering::SeqCst);
async move {
tokio::time::sleep(Duration::from_millis(500)).await;
"ok"
}
}
};
let (_, rem) = hedge.send(slow_request).await;
// a second request was performed.
assert!(rem.is_some());
assert_eq!(called.load(Ordering::SeqCst), 2);
// Wait for completion in a new task if the request is not cancel-safe.
// For example a connection might need to be returned to a pool for reuse.
if let Some(rem) = rem {
tokio::spawn(rem);
}
Ok(())
}
Dependencies
~0.3–6.5MB
~37K SLoC