sonde

sonde is a library to compile USDT probes into a Rust library, and to generate a friendly Rust idiomatic API around it.

Userland Statically Defined Tracing probes (USDT for short) is a technique inherited from DTrace (see OpenDtrace to learn more). It allows user to define statically tracing probes in their own application; while they are traditionally declared in the kernel.

USDT probes can be naturally consumed with DTrace, but also with eBPF (bcc, bpftrace…).

Lightweight probes by design

USDT probes for libraries and executables are defined in an ELF section in the corresponding application binary. A probe is translated into a nop instruction, and its metadata are stored in the ELF's .note.stapstd section. When registering a probe, USDT tool (like dtrace, bcc, bpftrace etc.) will read the ELF section, and instrument the instruction from nop to breakpoint, and after that, the attached tracing event is run. After deregistering the probe, USDT will restore the nop instruction from breakpoint.

The overhead of using USDT probes is almost zero when no tool is listening the probes, otherwise a tiny overhead can be noticed.

The workflow

Everything is automated. dtrace must be present on the system at compile-time though. Let's imagine the following sonde-test fictitious project:

/sonde-test
├── src
│  ├── main.rs
├── build.rs
├── Cargo.toml
├── provider.d

Start with the obvious thing: let's add the following lines to the Cargo.toml file:

[build-dependencies]
sonde = "0.1"

Now, let's see what is in the provider.d file. It's not a sonde specific vendor format, it's the canonical way to declare USDT probes (see Scripting)!

provider hello {
    probe world(); 
    probe you(char*, int);
};

It describes a probe provider, hello, with two probes:

1. world,
2. you with 2 arguments: char* and int.

Be careful, D types aren't the same as C types, even if they look like the same.

At this step, one needs to play with dtrace -s to compile the probes into systemtrap headers or an object file, but forget about that, sonde got you covered. Let's see what's in the build.rs script:

fn main() {
    sonde::Builder::new()
        .file("./provider.d")
        .compile();
}

That's all. That's the minimum one needs to write to make it work.

Ultimately, we want to fire this probe from our code. Let's see what's inside src/main.rs then:

// Include the friendly Rust idiomatic API automatically generated by
// `sonde`, inside a dedicated module, e.g. `tracing`.
mod tracing {
    include!(env!("SONDE_RUST_API_FILE"));
}

fn main() {
    tracing::hello::world();

    println!("Hello, World!");
}

What can we see here? The tracing module contains a hello module, corresponding to the hello provider. And this module contains a world function, corresponding to the world probe. Nice!

/// Bindings from Rust to the C FFI small library that calls the
/// probes.

use std::os::raw::*;

extern "C" {
    #[doc(hidden)]
    fn hello_probe_world();

    #[doc(hidden)]
    fn hello_probe_you(arg0: *mut c_char, arg1: c_int);
}

/// Probes for the `hello` provider.
pub mod r#hello {
    use std::os::raw::*;

    /// Call the `world` probe of the `hello` provider.
    pub fn r#world() {
        unsafe { super::hello_probe_world() };
    }

    /// Call the `you` probe of the `hello` provider.
    pub fn r#you(arg0: *mut c_char, arg1: c_int) {
        unsafe { super::hello_probe_you(arg0, arg1) };
    }
}

Let's see it in action:

$ cargo build --release
$ sudo dtrace -l -c ./target/release/sonde-test | rg sonde-test
123456 hello98765 sonde-test hello_probe_world world

Neat! Our sonde-test binary contains a world probe from the hello provider!

$ # Let's execute `sonde-test` as usual.
$ ./target/release/sonde-test
Hello, World!
$
$ # Now, let's execute it with `dtrace` (or any other tracing tool).
$ # Let's listen the `world` probe and prints `gotcha!` when it's executed.
$ sudo dtrace -n 'hello*:::world { printf("gotcha!\n"); }' -q -c ./target/release/sonde-test
Hello, World!
gotcha!

Eh, it works! Let's try with the you probe now:

fn main() {
    {
        let who = std::ffi::CString::new("Gordon").unwrap();
        tracing::hello::you(who.as_ptr() as *mut _, who.as_bytes().len() as _);
    }

    println!("Hello, World!");
}

Time to show off:

$ cargo build --release
$ sudo dtrace -n 'hello*:::you { printf("who=`%s`\n", stringof(copyin(arg0, arg1))); }' -q -c ./target/release/sonde-test
Hello, World!
who=`Gordon`

Successfully reading a string from Rust inside a USDT probe!

With sonde, you can add as many probes inside your Rust library or binary as you need by simply editing your canonical .d file.

Bonus: sonde generates documentation for your probes automatically. Run cargo doc --open to check.

Possible limitations

Types

DTrace has its own type system (close to C) (see Data Types and Sizes). sonde tries to map it to the Rust system as much as possible, but it's possible that some types could not match. The following types are supported:

Parser

The .d files are parsed by sonde. For the moment, only the provider blocks are parsed, which declare the probes. All the pragma (#pragma) directives are ignored for the moment.

License

BSD-3-Clause, see LICENSE.md.