0.1.0-dev.4 |
|
---|
#6 in #pliant-db
5KB
63 lines
PliantDb
PliantDb aims to be a Rust-written, ACID-compliant, document-database inspired by CouchDB. While it is inspired by CouchDB, this project will not aim to be compatible with existing CouchDB servers, and it will be implementing its own replication, clustering, and sharding strategies.
Project Goals
The high-level goals for this project are:
- ☑️ Be able to build a document-based database's schema using Rust types.
- ☑️ Run within your Rust binary, simplifying basic deployments.
- ☑️ Run as a local-only file-based database with no networking involved.
- ☑️ Run as a networked server using QUIC with TLS enabled by default
- Easily set up read-replicas between multiple servers.
- Easily run a highly-available quorum-based cluster across at least 3 servers
- ☑️ Expose a Publish/Subscribe eventing system
- Expose a Job queue and scheduling system -- a la Sidekiq or SQS
⚠️ Status of this project
You should not attempt to use this software in anything except for experiments. This project is under active development (), but at the point of writing this README, the project is too early to be used.
If you're interested in chatting about this project or potentially wanting to contribute, come chat with us on Discord: .
Example
Check out ./pliantdb/examples for examples. To get an idea of how it works, this is a simple schema:
#[derive(Debug, Serialize, Deserialize)]
struct Shape {
pub sides: u32,
}
impl Collection for Shape {
fn collection_name() -> Result<CollectionName, InvalidNameError> {
CollectionName::new("khonsulabs", "shapes")
}
fn define_views(schema: &mut Schematic) -> Result<(), Error> {
schema.define_view(ShapesByNumberOfSides)
}
}
#[derive(Debug)]
struct ShapesByNumberOfSides;
impl View for ShapesByNumberOfSides {
type Collection = Shape;
type Key = u32;
type Value = usize;
fn version(&self) -> u64 {
1
}
fn name(&self) -> Result<Name, InvalidNameError> {
Name::new("by-number-of-sides")
}
fn map(&self, document: &Document<'_>) -> MapResult<Self::Key, Self::Value> {
let shape = document.contents::<Shape>()?;
Ok(Some(document.emit_key_and_value(shape.sides, 1)))
}
fn reduce(
&self,
mappings: &[MappedValue<Self::Key, Self::Value>],
_rereduce: bool,
) -> Result<Self::Value, view::Error> {
Ok(mappings.iter().map(|m| m.value).sum())
}
}
After you have your collection(s) defined, you can open up a database and insert documents:
let db =
Database::<Shape>::open_local("view-examples.pliantdb", &Configuration::default()).await?;
// Insert a new document into the Shape collection.
db.collection::<Shape>().push(&Shape::new(3)).await?;
And query data using the Map-Reduce-powered view:
let triangles = db
.view::<ShapesByNumberOfSides>()
.with_key(3)
.query()
.await?;
println!("Number of triangles: {}", triangles.len());
Why write another database?
- Deploying highly-available databases is hard (and often expensive). It doesn't need to be.
- We are passionate Rustaceans and are striving for an ideal of supporting a 100% Rust-based deployment ecosystem for newly written software.
- Specifically for the founding author @ecton, the idea for this design dates back to thoughts of fun side-projects while running my last business which was built atop CouchDB. Working on this project is fulfilling a long-time desire of his.
let triangles = db
.view::<ShapesByNumberOfSides>()
.with_key(3)
.query()
.await?;
println!("Number of triangles: {}", triangles.len());
Feature Flags
No feature flags are enabled by default in the pliantdb
crate. This is
because in most Rust executables, you will only need a subset of the
functionality. If you'd prefer to enable everything, you can use the full
feature:
[dependencies]
pliantdb = { version = "*", default-features = false, features = "full" }
full
: Enableslocal-full
,server-full
, andclient-full
.cli
: Enables thepliantdb
executable.
Local databases only
[dependencies]
pliantdb = { version = "*", default-features = false, features = "local-full" }
local-full
: Enableslocal
,local-cli
,local-keyvalue
, andlocal-pubsub
local
: Enables thelocal
module, which re-exports the cratepliantdb-local
.local-cli
: Enables theStructOpt
structures for embedding database management commands into your own command-line interface.local-pubsub
: EnablesPubSub
forpliantdb-local
.local-keyvalue
: Enables the key-value store forpliantdb-local
.
PliantDb
server
[dependencies]
pliantdb = { version = "*", default-features = false, features = "server-full" }
server-full
: Enablesserver
,server-websockets
,server-keyvalue
, andserver-pubsub
server
: Enables theserver
module, which re-exports the cratepliantdb-server
.server-websockets
: EnablesWebSocket
support forpliantdb-server
.server-pubsub
: EnablesPubSub
forpliantdb-server
.server-keyvalue
: Enables the key-value store forpliantdb-server
.
Client for accessing a PliantDb
server
[dependencies]
pliantdb = { version = "*", default-features = false, features = "client-full" }
client-full
: Enablesclient
,client-trusted-dns
,client-websockets
,client-keyvalue
, andclient-pubsub
client
: Enables theclient
module, which re-exports the cratepliantdb-client
.client-trusted-dns
: Enables using trust-dns for DNS resolution. If not enabled, all DNS resolution is done with the OS's default name resolver.client-websockets
: EnablesWebSocket
support forpliantdb-client
.client-pubsub
: EnablesPubSub
forpliantdb-client
.client-keyvalue
: Enables the key-value store forpliantdb-client
.
Backups
Exporting and restoring databases with direct access
If you have a local PliantDb
database, you can use the local-backup
command to save and load backups:
pliantdb local-backup <database-path> save
pliantdb local-backup <destination-database-path> load <backup-path>
The format of this export should be easy to work with if you're either transitioning from PliantDb to another solution or needing to do complicated disaster recovery work. It is described here.
Developing PliantDb
Pre-commit hook
Our CI processes require that some commands succeed without warnings or errors. To ensure that code you submit passes the basic checks, install the included pre-commit hook:
./git-pre-commit-hook.sh install
Once done, tools including cargo fmt
, cargo doc
, and cargo test
will all be checked before git commit
will execute.
Open-source Licenses
This project, like all projects from Khonsu Labs, are open-source. This repository is available under the MIT License or the Apache License 2.0.
Dependencies
~1.5MB
~36K SLoC