blob: cfc9889b1c8acde3626a13938fb3b54c3b33e188 [file] [log] [blame] [view]
Jakub Gielzak206ba4c2024-01-04 13:49:41 +00001# Overview
2
3The project provides an easy way to save before/after CPU traces from Microbenchmark runs, and compare them visually using Differential Flame Graphs.
4
5![Differential graph illustration](assets/illustration-diff.webp)
6
7Areas where the code got slower are highlighted in red, while areas that are now faster are marked in blue; the intensity of the colour is proportional to the size of the difference.
8
9See also the [end-to-end demo (video)](https://drive.google.com/file/d/119nI_zlAMbTHzh-Rdzf8UuUVCGEKnKFQ/view?usp=drive_link&resourcekey=0-SRRmKgVZYfAlnkL4Hvh-cg).
10
11# Usage
12
13## Interacting with the script
14
15- Overview of all commands: `./bench-flame-diff.sh -h`
16- Help for a specific command: `./bench-flame-diff.sh <command> -h`
17
18## First usage
19
20On first usage, initialise all dependencies by running: `./bench-flame-diff.sh init`
21
22## General workflow
23
241. Run a specific Microbenchmark with CPU Stack sampling enabled (see below for instructions)
251. Save the trace as _base_ for comparison using `./bench-flame-diff.sh save`. It's worth picking a good names for the saved traces since you're likely going to e.g. re-use the _base_ while iterating on code changes.
261. Apply changes in your code and run the same benchmark as in step 1
271. Save the trace as _current_ `./bench-flame-diff.sh save`
281. Compare both traces using `./bench-flame-diff.sh diff` which will create and open a diff in a web browser
291. Toggle between graphs using the buttons on the top:
30 - `base`: flamegraph for the _base_ trace
31 - `base-vs-curr`: differential flame graph showing _base_ vs _current_ on the _base_ trace
32 - `curr`: flamegraph for the _current_ trace
33 - `curr-vs-base`: differential flame graph showing _base_ vs _current_ on the _current_ trace
341. You can later go back to generated diffs using `./bench-flame-diff.sh open`
35
36# Misc
37
38## Enabling stack sampling in Benchmark traces
39
40This can be done in CLI or by editing `build.gradle`. Full documentation is [here](https://developer.android.com/topic/performance/benchmarking/microbenchmark-profile).
41
42Quick CLI example:
43```
44# pick a target benchmark
45tgt=:compose:foundation:foundation-benchmark:connectedCheck
46
47# create a regex that targets a specific benchmark (test)
48test_rx="androidx.compose.foundation.benchmark.lazy.LazyListScrollingBenchmark.scrollProgrammatically_noNewItems\[.*Row.*\]"
49
50# run the benchmark and gather a 5 second (default) stack sample at 1000 Hz (default)
51./gradlew $tgt -Pandroid.testInstrumentationRunnerArguments.tests_regex="$test_rx" \
52 -P android.testInstrumentationRunnerArguments.androidx.benchmark.profiling.mode=StackSampling \
53 -P android.testInstrumentationRunnerArguments.androidx.benchmark.profiling.sampleDurationSeconds=5 \
54 -P android.testInstrumentationRunnerArguments.androidx.benchmark.profiling.sampleFrequency=1000
55```
56
57## CLI completion
58
Chris Craik2c246682024-03-18 10:42:28 -070059Generate shell-specific completion files with `./generate-completion.sh`.
60
61Then, source in your shell config, e.g.:
Jakub Gielzak206ba4c2024-01-04 13:49:41 +000062- For `bash`: `dst="$(pwd)/completion_bash.sh"; echo "source '$dst'" >> ~/.bashrc`
63- For `zsh`: `dst="$(pwd)/completion_zsh.sh"; echo "source '$dst'" >> ~/.zshrc`
64
65After restarting the shell session, you will be able to 'tab-autocomplete' commands and argument names.
66
67# Dependencies
68
69On top of dependencies discoverable with `./gradlew app:dependencies` the project depends on:
70- https://github.com/brendangregg/FlameGraph
71- https://android.googlesource.com/platform/system/extras/+/refs/heads/main/simpleperf/scripts
72
73Both are fetched from the network in the `init` command and pinned to known-good-revisions.
74
75# Reporting issues
76
77File an issue on Buganizer using [this link](https://b.corp.google.com/issues/new?component=1229612&hotlistIds=3622386&hotlistIds=5709693&assignee=jgielzak@google.com&title=bench-flame-diff:%20) or reach out directly to [jgielzak@](http://go/moma/chat?with=jgielzak).
78
79Known issues and future work items are tracked [here](https://b.corp.google.com/hotlists/5709693).