1 unstable release
0.1.0 | Apr 12, 2022 |
---|
#2050 in Database interfaces
65KB
1K
SLoC
A database-backed logger for use with the log crate
db_logger is a Rust crate providing an implementation of the log crate's logging facade to write structured log entries to a database. Just add a few lines of code to the beginning of your program and all logging will be saved for later analysis, which is especially suited to (distributed) services.
db_logger currently supports PostgreSQL and SQLite and is backed by the sqlx crate.
The latest version of db_logger is 0.1.0 and was released on 2022-04-12.
Usage
To use db_logger, you need to add a dependency to your project with the right set of features, create a database with the expected schema, and then initialize the logger during program initialization.
As a logging facade implementation, db_logger should only be depended upon from binary crates (never from libraries).
Usage with PostgreSQL
-
Add the following to your list of dependencies in
Cargo.toml
:[dependencies.db_logger] version = "0.1" default-features = false features = ["postgres"]
-
Create a PostgreSQL database and initialize it with the
schemas/postgres.sql
schema. -
Initialize the logger in your code with one of:
-
Explicit configuration:
use db_logger::postgres; let conn = postgres::connect_lazy(postgres::ConnectionOptions { host: "some host".to_owned(), port: 5432, database: "some database".to_owned(), username: "some username".to_owned(), password: "some password".to_owned(), ..Default::default() }); let _handle = db_logger::init(conn).await;
-
Environment-based configuration:
use db_logger::postgres; let conn = postgres::connect_lazy( postgres::ConnectionOptions::from_env("LOGGER").unwrap()); let _handle = db_logger::init(conn).await;
This will cause your program to recognize variables of the form
LOGGER_HOST
,LOGGER_PORT
,LOGGER_DATABASE
,LOGGER_USERNAME
andLOGGER_PASSWORD
to configure the PostgreSQL connection.
-
-
Make sure to keep
_handle
alive for the duration of the program in an async context, because the handle keeps the background logging task alive.
Usage with SQLite
-
Add the following to your list of dependencies in
Cargo.toml
:[dependencies.db_logger] version = "0.1" default-features = false features = ["sqlite"]
-
Create an SQLite database and initialize it with the
schemas/sqlite.sql
schema. -
Initialize with:
use db_logger::sqlite; let conn = sqlite::connect(sqlite::ConnectionOptions { uri: "file:/path/to/database?mode=rw", ..Default::default() }).await.unwrap(); let _handle = db_logger::init(conn).await;
-
Make sure to keep
_handle
alive for the duration of the program in an async context, because the handle keeps the background logging task alive.
Environment configuration
db_logger recognizes the RUST_LOG
environment variable to configure the
maximum level of the log messages to record, the same way as the
env_logger crate does.
Schema initialization
As indicated above, you should create the database and its schema by hand
before establishing a connection using the reference files provided in the
schemas
directory. This is a one-time operation.
You also have the option of invoking the Connection::create_schema()
method
to initialize the database schema. You probably don't want to do this in
production but this is useful if you are using ephemeral SQLite databases.
Limitations
The code in this crate was extracted from the EndBASIC cloud service and then cleaned for separate publication. This code was written as a fun experiment as part of that service and it has not received a lot of real-world stress testing. As a result, expect this to have a bunch of limitations, including:
-
Because db_logger runs in-process, the logger must filter out any log messages that it can generate while writing to the database (or else it would enter an infinite logging loop). This includes log messages for its own dependencies, such as those emitted by sqlx or tokio, which means that if you are using this crate, you won't get their logs saved into the database.
-
Performance is not great. While the logger tries to avoid blocking caller code by buffering log messages, if you log a lot, your application will slow down. I'm positive some profiling could make this much better without a ton of effort, but I just haven't spent the time doing it.
-
No tracing support. This crate may be a dead end. Saving raw logs to a database is interesting but what would be more interesting is saving the traces generated when integrating with the tracing crate (so as to trace individual requests as they flow through an async server, which was the original desire).
-
stderr pollution. Right now, any errors encountered while persisting logs to the database and any log messages that are filtered out are dumped to stderr in an ad-hoc format. This behavior should be configurable but isn't.
Dependencies
~19–27MB
~594K SLoC