#file-io #aio #agilulf #linux #abstract #layer #async

nightly agilulf_fs

Async file IO abstract layer (with Linux AIO)

1 unstable release

0.1.0 Jul 19, 2019

#8 in #aio


Used in 2 crates (via agilulf)

MIT license

9KB
164 lines

Agilulf

A fast and asynchronous KV database

Build Status Crates.io Docs

Usage

Detail about the protocol can be found in documents about protocol.

Server

Installing and starting a Agilulf Server is quite simple.

cargo install agilulf

agilulf_server --help

If you want to start a memory only server (like redis without persistence)

agilulf_server --mem --addr <ADDR>

If you need a server with data persistence on disk (with sstable and LSM tree structure. However, compaction is not implemented yet. So it is much slower after responding to much PUT requests)

agilulf_server --addr <ADDR>

Client

If you need a client, agilulf_driver is an asynchronous Rust client for this protocol.

Add this to your Cargo.toml:

[dependencies]
agilulf_driver = "0.1.0"

Now you can use the client:

use agilulf_driver::MultiAgilulfClient;

futures::executor::block_on(async {
    let client = MultiAgilulfClient::connect(address, 128).await;
    
    client.put(Slice(b"HELLO".to_vec()), Slice(b"WORLD".to_vec())).await;
})

Benchmark

A simple benchmark is provided in agilulf_driver/examples the benchmark.rs example will start up a server itself. And then send request to that server.

cargo run --example benchmark --release

Note: It will use /var/tmp/agilulf as it's base directory, so make sure this directory exists. Agilulf will not create directory for you.

Another choice is remote_benchmark.rs

cargo run --example remote_benchmark --release -- --addr <ADDR>

You can setup a server yourself and send requests to that server. The default addr is "127.0.0.1:3421" which is same with default addr of Agilulf Server.

TODO

  • AIO for writing files
  • Wait for crossbeam-skiplist to be stable and migrate to it
  • Compact SSTables into higher level
  • Restore data from frozen logs
  • Automatically increase the highest level of skipmap

lib.rs:

An abstract layer for async write file.

Tokio doesn't support latest async/await syntax in rust which is unbearable for this project (as this project is an experiment for writing projects with async/await syntax in rust). So I have to write an asynchronous file I/O library.

The implementation of it is AIO so it supports Linux only.

Dependencies

~5MB
~110K SLoC