Benchmarks

What the benchmarks measure, how to run them, and what the numbers do and do not mean.

What is measured

crates/rlg/benches/competitive_bench.rs times the cost to the calling thread of emitting a record, in four groups:

The three contenders do different work, and the numbers only make sense with that in mind:

• rlg fire() checks the level and pushes the record into the ring buffer. Formatting and I/O happen later on the flusher thread and are not in the timed path. That is the design being measured.
• tracing::info! runs through a tracing_subscriber::fmt subscriber that formats the event on the calling thread and writes it to std::io::sink.
• log::info! goes to a logger that does nothing: no formatting, no I/O. It is the floor, the cost of the facade alone.

So rlg against tracing compares “enqueue and return” with “format on this thread”, and neither is comparable to the log floor.

Running them

cargo bench -p rlg --bench competitive_bench      # this suite
cargo bench --workspace                           # every crate's benches

Numbers from a laptop swing by tens of percent between runs; compare results from the same machine, back to back.

Published results

bench-publish.yml runs every workspace benchmark on a GitHub-hosted ubuntu-latest runner for each release tag, and uploads the Criterion report and a JSON summary as workflow artifacts (kept 90 days). Until 0.0.13 that workflow ran no benchmarks at all: its output directory did not exist, and the failure was swallowed.

0.0.13 (release branch)

Two runs on GitHub-hosted ubuntu-latest, stable Rust, release profile, before and after the hot-path fix below. Typical time per iteration with Criterion’s 95% confidence interval.

Runners differ in speed between runs: tracing, whose code did not change, measured 353 ns in one and 591 ns in the other. So compare within a run (the ratio to tracing), not across runs.

After (run 36802323103):

Before (run 36773354952):

Reading these honestly. After the fix, fire() costs about what tracing::info! costs on the calling thread, and less with attributes, while tracing formats the event there and rlg does not. rlg’s advantage remains that the caller never waits on a sink’s I/O, which this suite’s discarding writer does not exercise.

Where the per-record cost went

Timing each piece of fire() in isolation (release build, one machine) showed most of it was formatting that had stayed on the calling thread, not the queue or the wake-up:

The timestamp is now written digit by digit into a fixed buffer instead of through format! (output identical, checked against the old code on two million instants), and the caller attribute is built without format!.