Criterion benchmarks for Tonic

Running the benchmarks

From the root Tonic directory, cargo bench

After running, the reports can be found in tonic/target/criterion/report/index.html

Gnuplot is required for graph generation. If gnuplot is not installed, Criterion will display: Gnuplot not found, disabling plotting at the console.

Notes

1. Currently, these benchmarks only test the performance of constructing Tonic Requests and Responses, not over-the-wire throughput.
2. The thrpt value generated by Criterion is simply a measure of bytes consumed by the target function.
3. As we are not testing tonic-build compile time, the tests reference pre-compiled .rs files in ‘benchmarks/compiled_protos’.
4. The original proto files are in the proto directory for reference.
5. This used the Criterion 3.0 Criterion Group functionality. Details here: https://docs.rs/criterion/0.3.0/criterion/

Interpreting Results

Criterion is particularly useful for establishing a first-run baseline and then comparing after code-changes - e.g. Performance has regressed below.

Request_Response/request/100000                                                                             
                        time:   [2.7231 us 2.7588 us 2.7969 us]
                        thrpt:  [33.298 GiB/s 33.758 GiB/s 34.200 GiB/s]
                 change:
                        time:   [+16.073% +17.871% +19.980%] (p = 0.00 < 0.05)
                        thrpt:  [-16.653% -15.162% -13.847%]
                        Performance has regressed.
Found 3 outliers among 100 measurements (3.00%)
  1 (1.00%) high mild
  2 (2.00%) high severe