| ## Criterion benchmarks for Tonic |
| |
| ### Running the benchmarks |
| From the root Tonic directory, `cargo bench` |
| |
| After running, the reports can be found in `tonic/target/criterion/report/index.html` |
| |
| [Gnuplot](http://www.gnuplot.info/) is required for graph generation. If gnuplot is not installed, Criterion will display: `Gnuplot not found, disabling plotting` at the console. |
| |
| ### Notes |
| 1) Currently, these benchmarks only test the performance of constructing Tonic Requests and Responses, not over-the-wire throughput. |
| 2) The `thrpt` value generated by Criterion is simply a measure of bytes consumed by the target function. |
| 3) As we are not testing tonic-build compile time, the tests reference pre-compiled .rs files in 'benchmarks/compiled_protos'. |
| 4) The original proto files are in the `proto` directory for reference. |
| 5) This used the Criterion 3.0 `Criterion Group` functionality. Details here: https://docs.rs/criterion/0.3.0/criterion/ |
| |
| ### Interpreting Results |
| |
| Criterion is particularly useful for establishing a first-run baseline and then comparing after code-changes - e.g. `Performance has regressed` below. |
| |
| ```bash |
| Request_Response/request/100000 |
| time: [2.7231 us 2.7588 us 2.7969 us] |
| thrpt: [33.298 GiB/s 33.758 GiB/s 34.200 GiB/s] |
| change: |
| time: [+16.073% +17.871% +19.980%] (p = 0.00 < 0.05) |
| thrpt: [-16.653% -15.162% -13.847%] |
| Performance has regressed. |
| Found 3 outliers among 100 measurements (3.00%) |
| 1 (1.00%) high mild |
| 2 (2.00%) high severe |
| ``` |
| |
| |
| |
| |
| |
