8 releases

new 0.3.2 Oct 31, 2024
0.3.1 Oct 25, 2024
0.2.1 Oct 6, 2024
0.1.4 Sep 29, 2024
0.0.0 Jun 19, 2024

#40 in Database implementations

Download history 1/week @ 2024-07-29 384/week @ 2024-08-26 126/week @ 2024-09-02 3/week @ 2024-09-09 1/week @ 2024-09-16 102/week @ 2024-09-23 373/week @ 2024-09-30 268/week @ 2024-10-07 35/week @ 2024-10-14 96/week @ 2024-10-21

866 downloads per month

MIT/Apache

92KB
1.5K SLoC

Append Only Log

Generic purpose append only log implementation.

github LoC Build codecov

docs.rs crates.io crates.io

license

English | 简体中文

Introducation

When developing infrastructure softwares, write-ahead log or append-only log plays an important role, and people re-implement same funcationalities multiple times, but actually, the core for append-only log is just atomic append, append_batch, replay, and rewrite.

This crate provides generic purpose append-only log implementation based on std::fs::File.

  • aol::fs::AppendLog:

    Generic purpose append-only log implementation based on std::fs::File.

    • It is good for:

      • The encoded entry size is smaller than 64 bytes.
      • Manifest file.
      • Write is not too frequently.
    • Pros:

      • It is growable, do not require pre-allocated.
      • Support automatically rewrite.
      • No holes in the file.

File Structure

+----------------------+--------------------------+-----------------------+
| magic text (4 bytes) | external magic (2 bytes) | magic (2 bytes)       |
+----------------------+--------------------------+-----------------------+-----------------------+-----------------------+
| op (1 bit)           | custom flag (7 bits)     | len (4 bytes)         | data (N bytes)        | checksum (8 bytes)    |
+----------------------+--------------------------+-----------------------+-----------------------+-----------------------+
| op (1 bit)           | custom flag (7 bits)     | len (4 bytes)         | data (N bytes)        | checksum (8 bytes)    |
+----------------------+--------------------------+-----------------------+-----------------------+-----------------------+
| ...                  | ...                      | ...                   | ...                   | ...                   |
+----------------------+--------------------------+-----------------------+-----------------------+-----------------------+

Installation

[dependencies]
aol = "0.3"

Example

License

aol is under the terms of both the MIT license and the Apache License (Version 2.0).

See LICENSE-APACHE, LICENSE-MIT for details.

Copyright (c) 2024 Al Liu.

Dependencies

~2–11MB
~148K SLoC