2 unstable releases
0.2.0 | Jan 27, 2025 |
---|---|
0.1.0 | Aug 24, 2024 |
#108 in Configuration
147 downloads per month
Used in meadows
180KB
2.5K
SLoC
Tracing Config
NOTE
: For README.md
viewers, the links in the documentation do not work for you.
The documentation is best read online at
docs.rs
or locally.
To generate the documentation locally :
- Download the source
cargo doc --open
The primary purpose of this crate is to allow rust programs to configure the tracing
crate using the Registry
Subscriber
implementation from the tracing-subscriber
crate given a toml
configuration file and a simple init!()
macro call in main()
or
in any #[test]
function.
This crate is not meant to be used by library authors. If your project contains a lib.rs
file,
remove tracing-config
from your Cargo.toml
project file.
Performance penalties / Memory overhead
If you use this crate to build and set up your global tracing
Subscriber
,
the implementation will be a tracing-subscriber
Registry
and all Layer
s added
to said Registry
will be dynamic dispatch Box<dyn Layer>
.
Moreover tracing-config
's own SpanRecordLayer
will be added to the Registry
right after the root EnvFilter
which will essentially keep an in memory serde_json Value
representation of all (non filtered out) Span
Value
s practically negating any and
all performance gained by tracing
s visitor pattern
which does not keep an in-memory
representation of a span data after it's been created/entered.
The SpanRecordLayer
visits the values and leverages tracing-subscriber
s
span Extensions
, to persist span data for the remaining of the span's lifetime.
If you suspect that your application suffers performance penalties due to how tracing is configured:
- Submit a bug report
- Try a stricter filter or entirely remove some high verbosity tracing events
(see
level_filters
) - Consider emitting less events, you should not debug your application using tracing, use a debugger instead.
- Try building your subscriber manually in
main()
doing so removes the need for dynamic dispatch layers. - Lastly you can remove
tracing-config
from yourCargo.toml
project file and find a different way to configure tracing.
Note
: Given that there are a myriad of programming languages that only use dynamic dispatch or
heavily rely on it for logging/tracing purposes.
I think that having the same in rust is no big deal especially because once your configuration
is mature enough you can easily construct your subscriber without dynamic dispatch
or the SpanRecordLayer
.
Getting started
Cargo.toml
tracing-config = { version = "0.2" }
tracing = { version = "0.1", features = [
"max_level_trace", # trace for debug
"release_max_level_info" # info for release
]}
main.rs
use tracing::*;
fn main() {
tracing_config::init!(); // panics; read the docs on why and when.
let _main_info_span = info_span!("main").entered();
info!("Hello World");
}
Environment variable tracing_config
Set the environment variable so that it points directly to the tracing.toml
file.
Current directory
If setting up an environment variable is too much work, you can also place tracing.toml
in the
current directory (usually near Cargo.toml
), cargo run
as well as your IDE will both work.
Rudimentary configuration file.
tracing.toml
title = "Pretty colored ts-fmt to stdout"
[layer.ts-fmt]
type = "fmt"
writer = "stdout"
formatter = "pretty"
span_events = "none"
ansi = true
[writer.stdout]
type = "standard_output"
[filter.root]
level = "trace"
Configuration file search path
Quick setup
- Place
tracing.toml
in the current directory (nearCargo.toml
) andcargo run
. - You can set the
tracing_config
environment variable and have it point directly to a.toml
configuration file. - You can set the
tracing_config_test
environment variable if you want a separate configuration file for tests.
Both environment variables can also point to a directory containing :
tracing-${name}.toml
; where${name}
is replaced by thepackage.name
in yourCargo.toml
.tracing.toml
.- ... And only for tests (
#[test]
), files with the-test
suffix are checked first (i.e.:tracing-${name}-test.toml
andtracing-test.toml
).
Init parameters
The init!()
macro or the initialize
function can be supplied with:
path
: APath
pointing directly to a.toml
configuration file or to a directory.env
: The key of an environment variable, pointing directly to a.toml
configuration file or to a directory.name
defaults topackage.name
in yourCargo.toml
.qualifier
defaults to an empty string (readProjectDirs
).organization
defaults to an empty string (readProjectDirs
).
Directories
In order to understand the directories mentioned in the following search path, please
read the documentation of the directories
crate. ProjectDirs
is constructed by default
with empty strings except for the application
parameter which is the same name
that you supply
to the init!()
macro (that defaults to package.name
if not set).
qualifier
and/or organization
are optional but can either or both be supplied to init!()
which will forward them to the ProjectDirs
constructor.
The search path
The search path
is an ordered list of either environment variables
or files
or directories
.
The first element in the list has the highest priority. The search function will loop trough all
elements from highest priority to least priority returning the first existing .toml
configuration
file.
path
(supplied in source to theinit!()
macro or theinitialize
function).env
(supplied in source to theinit!()
macro or theinitialize
function).env_parent
ifenv
is set but does not exist.- only for tests:
tracing_config_test
environment variable. - only for tests:
tracing_config_test_parent
iftracing_config_test
is set but does not exist. tracing_config
environment variable.tracing_config_parent
iftracing_config
is set but does not exist.project_dirs_preference_dir
project_dirs_config_dir
project_dirs_config_local_dir
base_dirs_preference_dir
base_dirs_config_dir
base_dirs_config_local_dir
user_dirs_home_dir
base_dirs_home_dir
current_exe_dir
current_dir
Within each directory in the search path
the first file that matches the following is accepted :
tracing-${name}.toml
; where${name}
is replaced by thepackage.name
in yourCargo.toml
or by thename
override passed toinit!()
orinitialize
.tracing.toml
- ... And only for tests (
#[test]
), files with the-test
suffix are checked first (e.g.:tracing-${name}-test.toml
)
The search path
is processed as follows :
- Environment variable entries (
env
,tracing_config
andtracing_config_test
) specifying (or pointing to) a direct.toml
configuration file or directory are accepted if the file/directory exists, otherwise demoted to the directory in which the (non existing) file or directory resides (i.e.: it's parent). - Entries specifying (or pointing to) a directory that does NOT exist are ignored.
- Environment variable values containing tokens in the form of
${env:key}
, areresolve
d by replacing the token with the value of the environment variable specified bykey
if it exists; this is done recursively up to a certain depth (>=25). - If an environment variable points to an existing
.toml
file, said file is accepted regardless of it's name, though calling ittracing.toml
is recommended.
Debugging the initialization process
By default, during initialization, tracing-config
will only emit errors and warnings in ansi
color to the program's standard output, this can be changed by setting a different verbosity
level
when calling init!()
(e.g: init! { verbosity : "trace", };
) or
initialize
(Some(verbosity))
otherwise, an environment variable tracing_config_verbosity
can
be set; accepted values are : trace
, debug
, info
, warn
, error
, none
.
Setting this to debug
or trace
will cause the function responsible to evaluate the search path
to "print" information about where it's looking and which file is accepted.
Suppressing output
Call init!()
with verbosity : "none"
this overrides the tracing_config_verbosity
environment
variable, otherwise make sure it's value is set to "none", this is not recommended though, as the
default verbosity is set to warn
which only outputs warnings that should be resolved and hard
errors which eventually panic if initialized by macro.
Configuration file
To fully understand the nomenclature of the configuration file, a thorough read of the documentation
on both tracing
and tracing-subscriber
crates is required; however, here is a brief summary:
writer
is used by alayer
and is responsible to write the data incoming form alayer
to a destination, which can be anything, e.g.(standard_output, file, network, database, etc...). A writer as a component in the system is not strictly necessary since alayer
could do the writing itself.layer
is something that receives structured events and spans (i.e.: all the information thatevent!
andspan!
macro calls contain) and is responsible to either ignore such events and spans or format them and either directly write somewhere or send the formatted events and spans to awriter
.filter
is a special kind oflayer
with the sole purpose of filtering out events and spans. A filter is exclusive, in that it allows everything by default unless there is an exclusion rule.
The "flow"
that events and spans usually go trough is : filter
->layer
->writer
.
You can find a detailed example and how the configuration file works
in the docs for the config::model module
.
For a full understanding of the configuration file structure, start by reading the docs for the root
level configuration structure i.e.: a TracingConfig
structure.
Basic configuration file mini guide.
Note
: The configuration file can include environment variables in the form of${env:key}
tokens in any toml string in the file, they areresolve
d withdepth=25
.
In this basic example, we will configure a compact fmt layer with colors for the terminal/console.
Next we will require an environment variable named tracing_config_logs
(the name is arbitrary)
which will point to a directory where we will save both .log files and .json files in subdirectories
(.log for humans, and .json for machines).
- Start by setting the required
title
title = "Basic configuration file"
- Declare the
layer
that will output in color to the terminal/console. - Call it
color-terminal
(again, name is arbitrary). - The
type
of the layer isfmt
fromtracing-subscriber
- We will have to later declare a
writer
but we can name it now "terminal" (name is arbitrary). - We want the
fmt
layer to use thecompact
formatter
. - Since this is the terminal, we disable
span_events
- Lastly we enable
ansi
which means pretty colors.
[layer.color-terminal]
type = "fmt"
writer = "terminal"
formatter = "compact"
span_events = "none"
ansi = true
- We now have to declare the writer named
terminal
since thelayer
color-terminal
is using it. - We wanted the console as output so set the
type
tostandard_output
no other configuration is necessary for this type of writer.
[writer.terminal]
type = "standard_output"
- Next we need to declare 2 more layers, 1 that will write .log and the other that will write .json
- Let's start with the .log, it's the same as the
color-terminal
except : - It's called
log-file
- Set the formatter to
full
- We want
active
span_events
since these will emit an event on span enter and exit. - This is a file, so we don't want
ansi
- Lastly we will have to declare another writer, in this example I will give it the same name as the layer i.e.: log-file.
[layer.log-file]
type = "fmt"
writer = "log-file"
formatter = "full"
span_events = "active"
ansi = false
- Let's also declare the layer for the json file before starting with the
writer
s. - We'll use
tracing-config
s custom json layer, settype
tojson
. - The default config for the json layer is acceptable, although the
pretty
flag is off by default, we set it explicitly here to show that pretty output is possible. - We'll call the writer
machine-readable
.
[layer.json-file]
type = "json"
writer = "machine-readable"
pretty = false
- Now we have to declare 2 writers, one
log-file
and anothermachine-readable
. - Start with the
log-file
- Set the
type
tofile
- The output directory will be a
human
subdirectory inside whatever directory the environment variabletracing_config_logs
points to. We setdirectory_path
with a reference to${env:tracing_config_logs}
. - Set
file_name
andfile_ext
[writer.log-file]
type = "file"
directory_path = "${env:tracing_config_logs}/human"
file_name = "my_app"
file_ext = "log"
max_log_files = 7 # keep at most 7 log files
rotation = "daily" # 7 log files daily means a week of history
non_blocking = true # async writes
lossy = true # it's okay if we loose some, we still have the json
- The json file now
[writer.machine-readable]
type = "file"
directory_path = "${env:tracing_config_logs}/machine"
file_name = "my_app"
file_ext = "json"
rotation = "never" # This will be GB in size
non_blocking = true
lossy = false
- Lastly the root filter.
[filter.root]
level = "trace"
directives = [
"hyper=error",
"hyper::client::connect::dns=error",
"hyper::proto::h1::conn=error",
"hyper::proto::h1::conn=error",
"hyper::proto::h1::io=error",
"hyper::proto::h1::role=error",
"hyper::proto::h1::encode=error",
"hyper::client::pool=error",
]
Combining it together:
tracing-basic.toml
title = "Basic configuration file"
[layer.color-terminal]
type = "fmt"
writer = "terminal"
formatter = "compact"
span_events = "none"
ansi = true
[writer.terminal]
type = "standard_output"
[layer.log-file]
type = "fmt"
writer = "log-file"
formatter = "full"
span_events = "active"
ansi = false
[layer.json-file]
type = "json"
writer = "machine-readable"
pretty = false
[writer.log-file]
type = "file"
directory_path = "${env:tracing_config_logs}/human"
file_name = "my_app"
file_ext = "log"
max_log_files = 7 # keep at most 7 log files
rotation = "daily" # 7 log files daily means a week of history
non_blocking = true # async writes
lossy = true # it's okay if we loose some, we still have the json
[writer.machine-readable]
type = "file"
directory_path = "${env:tracing_config_logs}/machine"
file_name = "my_app"
file_ext = "json"
rotation = "never" # This will be GB in size
non_blocking = true
lossy = false
[filter.root]
level = "trace"
directives = [
"hyper=error",
"hyper::client::connect::dns=error",
"hyper::proto::h1::conn=error",
"hyper::proto::h1::conn=error",
"hyper::proto::h1::io=error",
"hyper::proto::h1::role=error",
"hyper::proto::h1::encode=error",
"hyper::client::pool=error",
]
- Test it
#[test]
fn test_basic_config() {
use tracing::*;
std::env::set_var("tracing_config_logs", "path/to/some/dir");
tracing_config::init! {
path : std::path::Path::new("path/to/tracing-basic.toml")
};
// do more tests here
let _span = info_span!("my_span").entered();
info!("Test done!");
}
Dependencies
~10–19MB
~244K SLoC