Rust conf_test

Run configuration tests from build.rs and set available features, similar to autotools configure scripts.

Description

ConfTest::run() called from 'build.rs' parses 'Cargo.toml'. Then for each [feature] defined, it checks if that feature was not set manually (with --features) and a test in 'conf_tests/' exists. This test is then compiled and build. When that succeeds the feature becomes enabled automatically.

lib.rs:

Run configuration tests from build.rs and set available features, similar to autotools configure scripts.

Description

ConfTest::run() called from 'build.rs' parses 'Cargo.toml'. Then for each [feature] defined, it checks if that feature was not set manually (with --features) and a test in 'conf_tests/' exists. This test is then compiled and build. When that succeeds the feature becomes enabled automatically.

Special case for 'docs.rs'

When a packages is build for documentation on 'docs.rs' then conf_test detects and checks if a docs_rs = [] features is available in 'Cargo.toml', if so, then this becomes enabled. When not, then nothing is probed.

Rationale

Compiler versions and Operating Systems have sometimes subtle differences in features and standards conformance. Sometimes non-standard features are added for improved performance. These differences make it hard to write portable software that still offer optimal performance for different Operating Systems. Moreover often a developer doesn't even know what featureset other Operating Systems may provide or this may be changed by kernel or userland version or configuration. Probing the presence of such features at build time can solve these problems.

Further it becomes possible to test for rust stdlib functionality such as if nightly features are available or became stabilized.

How To

Checking OS features

When present 'cargo' (builds and) executes a build.rs script while building crate. This is the place where ConfTest is hooked in at first:

fn main() {
    conf_test::ConfTest::run();
    // any other build.rs steps follow below
}

Further one has to define a set of features and dependencies in the 'Cargo.toml'. Note that 'build.rs' will be run before the '[dependencies]' are build. Thus all dependencies needed for the tests must go into '[build-dependencies]' as well. For Example:

[dependencies]
libc = "0.2.34"

[build-dependencies]
libc = "0.2.34"
conf_test = "0.4"

[features]
default = []
o_path = []

And as final step the crate directory 'conf_tests/' need to be created which contain rust files named after the features to be probed. Containing a single fn main() which shall probe one single thing.

// This goes into conf_tests/o_path.rs
extern crate libc;

fn main() {
    unsafe {
        let conf_tests = std::ffi::CString::new("conf_tests").unwrap();
        // Compilation will fail when the libc does not define O_PATH
        libc::open(conf_tests.as_ptr(), libc::O_PATH);
    }
}

Later in the crate implementation source code one uses conditional compilation as usual with #[cfg(feature = "o_path")].

Test depending on other Features

Tests may depend on features that are discovered by other tests or set manually. For simplicity there is no dependency resolver about this but tests are run in sort order of the feature name. Every subsequent test is compiled with the the feature flags already discovered so far. To leverage this functionality one rarely needs to change the feature names. For example when 'bar' depends on 'foo' it is required to enforce the sort order by renaming these features to 'aa_foo' and 'bb_bar'. Only features that get discovered are used for the test compilations features set by printing cargo instructions from the test scripts are not used.

Detailed Control

Tests can emit special instructions to cargo on stdout. These become only effective when the test exits successful. See https://doc.rust-lang.org/cargo/reference/build-scripts.html#outputs-of-the-build-script

One can control ConfTest by setting the environment variable CONF_TEST_INHIBIT to one of the following:

• skip Will not execute any conf_tests but proceed with 'build.rs'.
• stop Exits 'build.rs' sucessfully, not executing any tests.
• fail Exits 'build.rs' with an failure, not executing any tests.

Any other value will make the script panic.

Limitations

• The tests running on the machine where the software is build, using the build-dependencies. This will be a problem when Software gets cross-compiled. For cross compilation set 'CONF_TEST_INHIBIT=skip' and set the desired features manually with the '--features' option.

• Features can only be set, not unset. This is deliberate and not a limitation. Do only positive tests checking for the presence of a feature.

Good Practices

• Only use ConfTest when other things (like factoring out OS specific thing into their own crates) are not applicable.

• Provide a baseline implementation which is portable with no features enabled. This may not perform as well or lack some special features but should compile nevertheless.