1 unstable release
0.1.0 | Jan 5, 2024 |
---|
#1 in #s2n-netbench
205KB
6K
SLoC
s2n-netbench
An efficiency, performance, and correctness analysis tool for transport protocols.
Why does this exist?
There are many transport protocols and several implementations of each. This tool exists to provide users with the ability to perform a direct comparison and decide the best implementation for their workloads.
Here are a few examples of questions that s2n-netbench aims to answer:
- What is the cost of encrypting traffic?
- How much more capacity will I need when deploying this?
- What transport protocol performs best
- in a data center?
- in networks with high packet loss?
- in networks with high latency?
- with many concurrent, multiplexed streams?
- Which implementation of "X" protocol is best for my workload?
- What is the optimal configuration of the transport's settings for my workload?
- How does certificate chain length affect handshake throughput?
- Is implementation "X" interoperable with implementation "Y" of "Z" protocol?
Quickstart
A basic use of s2n-netbench is demonstrated in the netbench-run.sh
script. This script will
- compile all necessary s2n-netbench utilities
- generate scenario files
- execute the
request-response.json
scenario usings2n-quic
ands2n-tls
drivers - execute the
connect.json
scenario usings2n-quic
ands2n-tls
drivers - collect statistics from the drivers using
netbench-collector
- generate a report in the
./target/netbench/report
directory
From the main netbench
folder, run the following commands
./scripts/netbench-run
cd target/netbench/report
python3 -m http.server 9000
Then navigate to localhost:9000
in a browser to view the netbench results.
Note that this script does not support bpftrace, as it runs without any of the elevated permissions required for bpf programs.
How it works
netbench-scenarios
netbench
provides tools to write scenarios that describe application workloads. An example of a scenario is a simple request/response pattern between a client and server:
use netbench_scenario::prelude::*;
config!({
/// The size of the client's request to the server
let request_size: Byte = 1.kilobytes();
/// The size of the server's response to the client
let response_size: Byte = 10.megabytes();
});
pub fn scenario(config: Config) -> Scenario {
let Config {
request_size,
response_size,
} = config;
Scenario::build(|scenario| {
let server = scenario.create_server();
scenario.create_client(|client| {
client.connect_to(server, |conn| {
conn.open_bidirectional_stream(
|local| {
local.send(request_size);
local.receive(response_size);
},
|remote| {
remote.receive(request_size);
remote.send(response_size);
},
);
});
});
})
}
This scenario generates a json file of instructions. These instructions are protocol and language independent, which means they can easily be executed by a "netbench driver", written in any language or runtime.
netbench-driver
Netbench drivers are responsible for executing netbench scenarios. Each transport protocol has a client
and server
implementation. Each of these implementations is a self-container binary that consumes a scenario.json
file. Implemented drivers include:
TCP
native-tls
- OpenSSL on Linux
- Secure Transport on macOS
- SChannel on Windows
s2n-quic
s2n-tls
netbench-collector
Driver metrics are collected with the netbench-collector
utility. There are two implementation of this available - a generic utility and a bpftrace utility. The generic utility uses the proc fs
to gather information about the process, while the bpftrace
implementation is able to collect a wider variety of statistics through ebpf probes.
The collector binary takes a netbench-driver
as an argument. The driver binary is spawned as a child process. The collector will continuously gather metrics from the driver and emit those metrics to stdout
.
netbench-cli
netbench-cli
is used to visualize the results of the scenarios. This reports use vega which is "a declarative format for creating, saving, and sharing visualization designs".
report
is used to generate individual .json
reports. These can be visualized by pasting them into the vega editor.
report-tree
is used to to generate a human-readable .html
report. Given a directory structure like the following
request-response/ # scenario
├─ tls/ # driver
│ ├─ client.json
│ ├─ server.json
├─ quic/
├─ client.json
├─ server.json
report-tree
will generate the individual reports
and package them into a human readable index.html
file that can be used to view graphs of the results.
A sample report can be found here.
Note that you will not be able to open the report directly since the report relies on the jsdelivr cdn. This request will fail when the URL is a local file scheme with a CORS request not HTTP error.
To get around this, use a local server.
# assuming the report is in ./report
cd report
# start a local server on port 9000
python3 -m http.server 9000
In a browser, navigate to localhost:9000
to view the netbench report.
Dependencies
~14–24MB
~357K SLoC