#parallelism #rayon #mpi #universe #arc #intranode #internode

balancer

Use rayon and mpi to achieve simple forms of internode + intranode parallelism

1 unstable release

0.2.0 Mar 5, 2023
0.1.0 May 26, 2019

#8 in #universe

MIT/Apache

12KB
112 lines

balancer: simple multinode parallelism

With the power of rayon and mpi, internode and intranode parallelism is achieved for simple tasks.

Example Usage

See examples/simple.rs.

use std::sync::Arc;

use balancer::Balancer;
use mpi::environment::Universe;

fn main() {
    let universe = Arc::new(mpi::initialize().unwrap());
    experiment(universe.clone());
    experiment(universe.clone());
}

fn experiment(universe: Arc<Universe>) {
    // Get relevant portion of data on this node
    let data: Vec<f64> = (0..100_000).map(|x| x as f64 / 100_000.0).collect();

    // Define task
    let work = |x: &f64| x * x;

    // Initialize balancer, work and collect
    let verbose = false;
    let balancer = Balancer::new(universe, verbose);
    balancer.work_local(&data, work);
    let output = balancer.collect();

    // That's it!
    // Let's do some verification
    if balancer.rank == 0 {
        for (expected, actual) in data.iter().map(work).zip(output.as_ref().unwrap()) {
            assert_eq!(expected, *actual);
        }
    }
}

Dependencies

~1.6–4MB
~75K SLoC