#metrics #memory #data-structures #storage #no-std

nightly no-std downsample

keep downsampled history of data over long period of time

3 releases

0.0.3 Oct 22, 2023
0.0.2 Jan 17, 2023
0.0.1 Jan 6, 2023

#2307 in Algorithms

Apache-2.0 OR MIT

27KB
597 lines

Crates.io docs.rs pipeline status coverage report license dependencies lines of code

downsample

no-std library to downsample fixed-frequency or time-based metrics for long history storage.

Let's assume you have a temperature sensor that can do 100 measurements per second and stores the data in a f32. You want to keep track of some historic samples, like from last year, but you don't need them at the same 100Hz as your raw data as 365 * 24 * 60 * 60 * 100 * 4byte = 12GB is just to much data.

Alternativly it will provide a time-based downsampler where you can store data in levels with no fixed size but a fixed interval. time-based downsampling is not yet implemented.

Usage

use downsample::FixedFrequencyBuilder;
use rand::Rng;

fn main() {
    // 0s - 1s : 100Hz
    // 1s - 1m : 1Hz
    // 1m - 1h : 1/60 Hz
    // 1h - 1d : 1/3600 Hz
    let mut temperature_measurements = FixedFrequencyBuilder::new(100, 100)
        .level(59, 100)
        .level(59, 60)
        .level(23, 60)
        .build();
    for _ in 0..1000 {
        temperature_measurements.push(rng.gen::<f32>());
    }
}

Limitations

  • Bucket size must be known on creation of the object.
  • Only whole numbers are allowed as a downsample factor. e.g. 100Hz -> 25Hz -> 5Hz is possible with factor 4 and 5, but 100Hz -> 25Hz -> 10Hz isn't as factor 2.5 is no integer.

Early release version

This crate is in version 0.0.x that means the api will change rapidly, traits will change nothing is even nearly stable yet

No runtime deps