Dave Mankoff | fa7aca1 | 2021-03-24 14:14:38 -0400 | [diff] [blame] | 1 | # Falsing in SystemUI |
| 2 | |
| 3 | Phones are easily and often accidentally-activated in owners' pockets ("falsing" or "pocket |
| 4 | dialing"). Because a phone's screen can be turned on with a single tap, and because we have further |
| 5 | actions that be activated with basic tapping and swiping, it is critical that we |
| 6 | analyze touch events on the screen for intentional vs accidental behavior. With analysis, |
| 7 | features within SystemUI have an opportunity to ignore or even undo accidental interactions as they |
| 8 | are occurring. |
| 9 | |
| 10 | ## Technical Details |
| 11 | |
| 12 | The `FalsingManager` tracks all touch interactions happening on a phone's lock screen. |
| 13 | |
| 14 | If you support any sort of touch gestures on the lock screen, you **must**, at a |
| 15 | minimum, inform the `FalsingManager` of what touches are on touch targets vs not (things that may be |
| 16 | intentional). If you do not tell the `FalsingManager`, it will assume touches on your feature are |
| 17 | always accidental and penalize the session accordingly. |
| 18 | |
| 19 | Individual touch targets do not _have_ to be separated out; it's acceptable to |
| 20 | wrap your whole feature in one virtual block that reports touches to the |
| 21 | `FalsingManager`, however more granular tracking will result in better results |
| 22 | across the whole lock screen. |
| 23 | |
| 24 | You can _act_ on the results of the `FalsingManager`. Instead of only telling |
| 25 | the `FalsingManager` that touch events were on touch targets, you can further use the |
| 26 | returned results to decide if you want to respond to an owner's touch, if you |
| 27 | want to prompt them to confirm their action, or if you simply want to ignore the |
| 28 | touch. |
| 29 | |
| 30 | The flow through the system looks like such: |
| 31 | |
| 32 | 1. Gesture on the screen. |
| 33 | 2. The `FalsingManager` makes a note of all of the `MotionEvents`. |
| 34 | * If no feature/touch target receives the `MotionEvents`, skip to 4. |
| 35 | 3. Your touch target receives the `MotionEvents`. |
| 36 | * Once your feature is ready to respond to the gesture in a substantive manner, it queries |
| 37 | the `FalsingManager`. |
| 38 | - Dragging animations, touch ripples, and other purely visual effects should not query. |
| 39 | - Query once you are ready to launch a new feature or dialogue, or are otherwise going to |
| 40 | change the state of the UI. |
| 41 | - Generally, wait until `MotionEvent.ACTION_UP` to query or `View.OnClickListener#onClick`. |
| 42 | - Only query once per gesture, at the end. |
| 43 | * If the `FalsingManager` says it looks good, respond to the touch. |
| 44 | 4. The `FalsingManager` checks to see if anyone queried about the gesture. If not, mark it as |
| 45 | accidental. |
| 46 | |
| 47 | There is also an event fired by the `FalsingManager` that can be listened to by anyone, that |
| 48 | indicates that the the `FalsingManager` believes the phone is actively being pocket-dialed. When |
| 49 | fired, modal features, such as quick settings, keyguard bouncer, and others should retract |
| 50 | themselves to prevent further pocket-dialing. |
| 51 | |
| 52 | ## Falsing "Belief" and History |
| 53 | |
| 54 | The `FalsingManager` maintains a recent history of false analyses. Using |
| 55 | Bayesian statistics, it updates a "belief" in whether recent |
| 56 | gestures are intentional or not. Any gesture that it is not explicitly queried about is treated as |
| 57 | accidental, increasing the overall belief in |
| 58 | false-iness. Gestures that are explicitly queried and that pass the relevant heuristics |
| 59 | reduce belief that falsing is occurring. This information is tracked within the `HistoryTracker`. |
| 60 | |
| 61 | Changes in belief may influence internal heurstics within the `FalsingManager`, |
| 62 | making it easier or harder for an owner to interact with their device. (An owner |
| 63 | will always be able to interact with their device, but we may require double |
| 64 | taps, or more deliberate swipes.) |
| 65 | |
| 66 | ## Responding to Touch Events |
| 67 | |
| 68 | The methods below inform the `FalsingManager` that a tap is occurring within an expected touch |
| 69 | target. Match the methods with the gesture you expect the device owner to use. |
| 70 | |
| 71 | ### Single Tap |
| 72 | |
Dave Mankoff | 58718c7 | 2021-03-30 14:08:01 -0400 | [diff] [blame] | 73 | `FalsingManager#isSimpleTape()`. This method |
| 74 | performs a only very basic checking, checking that observed `MotionEvent`s are |
| 75 | all within some small x & y region ("touch slop"). Useful for only the most simple of scenarios, |
| 76 | you probably want `FalsingManager#isFalseTap` method for most cases. |
| 77 | |
| 78 | `FalsingManager#isFalseTap(@Penalty int penalty)`. This |
| 79 | method tells the `FalsingManager` that you want to thoroughly validate a single tap. It |
Dave Mankoff | fa7aca1 | 2021-03-24 14:14:38 -0400 | [diff] [blame] | 80 | returns true if it thinks the tap should be rejected (i.e. the tap looks more |
| 81 | like a swipe) and false otherwise. |
| 82 | |
Dave Mankoff | 58718c7 | 2021-03-30 14:08:01 -0400 | [diff] [blame] | 83 | It runs through the following heuristics to validate a tap: |
Dave Mankoff | fa7aca1 | 2021-03-24 14:14:38 -0400 | [diff] [blame] | 84 | |
| 85 | 1. If the device recognizes a face (i.e. face-auth) the tap is **accepted**. |
| 86 | 2. If the tap is the _second_ tap in recent history and looks like a valid Double Tap |
| 87 | the tap is **accepted**. This works exactly like `FalsingManager#isFalseDoubleTap`. |
| 88 | 3. If the `HistoryTracker` reports strong belief in recent falsing, the tap is |
| 89 | **rejected**. |
| 90 | 4. Otherwise the tap is **accepted**. |
| 91 | |
| 92 | All the above rules are applied only after first confirming the gesture does |
Dave Mankoff | 58718c7 | 2021-03-30 14:08:01 -0400 | [diff] [blame] | 93 | in fact look like a simple tap. |
Dave Mankoff | fa7aca1 | 2021-03-24 14:14:38 -0400 | [diff] [blame] | 94 | |
Dave Mankoff | 58718c7 | 2021-03-30 14:08:01 -0400 | [diff] [blame] | 95 | `penalty` is a measure of how much the `HistoryTracker`'s belief should be |
Dave Mankoff | fa7aca1 | 2021-03-24 14:14:38 -0400 | [diff] [blame] | 96 | penalized in the event that the tap is rejected. This value is only used if |
Dave Mankoff | 58718c7 | 2021-03-30 14:08:01 -0400 | [diff] [blame] | 97 | the gesture fails to validate as a simple tap. |
Dave Mankoff | fa7aca1 | 2021-03-24 14:14:38 -0400 | [diff] [blame] | 98 | |
Dave Mankoff | 58718c7 | 2021-03-30 14:08:01 -0400 | [diff] [blame] | 99 | The `@FalsingManager.Penalty` values are fairly straightforward, but note that you |
| 100 | should generally be choosing `LOW_PENALTY`. It is inherently difficult to know if a |
| 101 | tap is truly false or not, so a single mis-tap should apply only a small penalty. |
| 102 | If the owner is further along in a UX flow, and is still mis-tapping, it may make more |
| 103 | sense to increase the penalty as mis-taps should be less likely to occur after |
| 104 | several successful gestures. |
Dave Mankoff | fa7aca1 | 2021-03-24 14:14:38 -0400 | [diff] [blame] | 105 | |
| 106 | ### Double Tap |
| 107 | |
| 108 | `FalsingManager#isFalseDoubleTap()`. This method tells the `FalsingManager` that |
| 109 | your UI wants to validate a double tap. There are no parameters to pass to this method. |
| 110 | Call this when you explicitly receive and want to verify a double tap, _not_ a single tap. |
| 111 | |
| 112 | Note that `FalsingManager#isFalseTap(boolean robustCheck, double falsePenalty)` |
| 113 | will also check for double taps when `robustCheck` is set to true. If you are |
| 114 | willing to use single taps, use that instead. |
| 115 | |
| 116 | ### Swipes and Other Gestures |
| 117 | |
| 118 | `FalsingManager#isFalseTouch(@Classifier.InteractionType int interactionType)`. |
| 119 | Use this for any non-tap interactions. This includes expanding notifications, |
| 120 | expanding quick settings, pulling up the bouncer, and more. You must pass |
| 121 | the type of interaction you are evaluating when calling it. A large set of |
| 122 | heuristics will be applied to analyze the gesture, and the exact rules vary depending upon |
| 123 | the `InteractionType`. |
| 124 | |
| 125 | ### Ignoring A Gesture |
| 126 | |
| 127 | `FalsingCollector#avoidGesture()`. Tell the `FalsingManager` to pretend like the |
| 128 | observed gesture never happened. **This method must be called when the observed |
| 129 | `MotionEvent` is `MotionEvent.ACTION_DOWN`.** Attempting to call this method |
| 130 | later in a gesture will not work. |
| 131 | |
| 132 | Notice that this method is actually a method on `FalsingCollector`. It is |
| 133 | forcefully telling the `FalsingManager` to wholly pretend the gesture never |
| 134 | happened. This is intended for security and PII sensitive gestures, such as |
| 135 | password inputs. Please don't use this as a shortcut for avoiding the |
| 136 | FalsingManager. Falsing works better the more behavior it is told about. |
| 137 | |
| 138 | ### Other Considerations |
| 139 | |
| 140 | Please try to call the `FalsingManager` only once per gesture. Wait until you |
| 141 | are ready to act on the owner's action, and then query the `FalsingManager`. The `FalsingManager` |
| 142 | will update its belief in pocket dialing based only on the last call made, so multiple calls per |
| 143 | gesture are not well defined. |
| 144 | |
Dave Mankoff | c5755b2 | 2021-04-01 17:03:29 -0400 | [diff] [blame] | 145 | The `FalsingManager` does not update its belief in pocket-dialing until after a gesture completes. |
| 146 | That is to say, if the owner makes a bad tap on your feature, the "belief" in pocket dialing will |
| 147 | not incorporate this new data after processing on the final `ACTION_UP` or `ACTION_CANCEL` event |
| 148 | occurs. |
Dave Mankoff | fa7aca1 | 2021-03-24 14:14:38 -0400 | [diff] [blame] | 149 | |
| 150 | If you expect a mix of taps, double taps, and swipes on your feature, segment them |
| 151 | accordingly. Figure out which `FalsingManager` method you need to call first, rather than relying |
| 152 | on multiple calls to the `FalsingManager` to act as a sieve. |
| 153 | |
| 154 | Don't: |
| 155 | ``` |
| 156 | if (!mFalsingManager.isFalseTap(false, 0)) { |
| 157 | // its a tap |
| 158 | } else if (!mFalsingManager.isFalseTouch(GESTURE_A) { |
| 159 | // do thing a |
| 160 | } else if (!mFalsingManager.isFalseTouch(GESTURE_B) { |
| 161 | // do thing b |
| 162 | } else { |
| 163 | // must be a false. |
| 164 | } |
| 165 | ``` |
| 166 | |
| 167 | Do: |
| 168 | ``` |
| 169 | void onTap() { |
| 170 | if (!mFalsingManager.isFalseTap(false, 0)) { |
| 171 | // its a tap |
| 172 | } |
| 173 | |
| 174 | void onGestureA() { |
| 175 | if (!mFalsingManager.isFalseTouch(GESTURE_A) { |
| 176 | // do thing a |
| 177 | } |
| 178 | } |
| 179 | |
| 180 | void onGestureB() { |
| 181 | if (!mFalsingManager.isFalseTouch(GESTURE_B) { |
| 182 | // do thing b |
| 183 | } |
| 184 | } |
| 185 | ``` |
| 186 | |
| 187 | |
| 188 | ## Influencing Belief |
| 189 | |
| 190 | `FalsingCollector#updateFalseConfidence(FalsingClassifier.Result result)`. This |
| 191 | method allows you to directly change the `FalsingManager`'s belief in the state |
| 192 | of pocket dialing. If the owner does something unusual with their phone that you |
| 193 | think indicates pocket dialing, you can call: |
| 194 | |
| 195 | ``` |
| 196 | mFalsingCollector.updateFalseConfidence( |
| 197 | FalsingClassifier.Result.falsed(0.6, "Owner is doing something fishy")); |
| 198 | ``` |
| 199 | |
| 200 | A belief value of `1` indicates a 100% confidence of false behavior. A belief |
| 201 | value of `0` would make no change in the `FalsingManager` and should be avoided |
| 202 | as it simply creates noise in the logs. Generally, a middle value between the |
| 203 | two extremes makes sense. |
| 204 | |
| 205 | A good example of where this is used is in the "Pattern" password input. We |
| 206 | avoid recording those gestures in the `FalsingManager`, but we have the pattern input update |
| 207 | the `FalsingManager` directly in some cases. If the owner simply taps on the pattern input, we |
| 208 | record it as a false, (patterns are always 4 "cells" long, so single "cell" inputs are penalized). |
| 209 | |
| 210 | Conversely, if you think the owner does something that deserves a nice reward: |
| 211 | |
| 212 | ``` |
| 213 | mFalsingCollector.updateFalseConfidence( |
| 214 | FalsingClassifier.Result.passed(0.6)); |
| 215 | ``` |
| 216 | |
| 217 | Again, useful on password inputs where the FalsingManager is avoiding recording |
| 218 | the gesture. This is used on the "pin" password input, to recognize successful |
| 219 | taps on the input buttons. |
| 220 | |
| 221 | ## Global Falsing Event |
| 222 | |
| 223 | If the `FalsingManager`'s belief in falsing crosses some internally defined |
| 224 | threshold, it will fire an event that other parts of the system can listen for. |
| 225 | This even indicates that the owner is likely actively pocket-dialing, and any |
| 226 | currently open activities on the phone should retract themselves. |
| 227 | |
| 228 | To subscribe to this event, call |
| 229 | `FalsingManager#addFalsingBeliefListener(FalsingBeliefListener listener)`. |
| 230 | `FalsingBeliefListener` is a simple one method interface that will be called |
| 231 | after when activities should retract themselves. |
| 232 | |
| 233 | **Do Listen For This**. Your code will work without it, but it is a handy, |
| 234 | universal signal that will save the phone owner a lot of accidents. A simple |
| 235 | implementation looks like: |
| 236 | |
| 237 | ``` |
| 238 | mFalsingManager.addFalsingBeliefListener(MyFeatureClass::hide); |
| 239 | ``` |