| <html devsite> |
| <head> |
| <title>Camera ITS-in-a-Box</title> |
| <meta name="project_path" value="/_project.yaml" /> |
| <meta name="book_path" value="/_book.yaml" /> |
| </head> |
| <body> |
| <!-- |
| Copyright 2017 The Android Open Source Project |
| |
| Licensed under the Apache License, Version 2.0 (the "License"); |
| you may not use this file except in compliance with the License. |
| You may obtain a copy of the License at |
| |
| http://www.apache.org/licenses/LICENSE-2.0 |
| |
| Unless required by applicable law or agreed to in writing, software |
| distributed under the License is distributed on an "AS IS" BASIS, |
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
| See the License for the specific language governing permissions and |
| limitations under the License. |
| --> |
| |
| |
| |
| <p>Android Camera Image Test Suite (ITS) is part of Android Compatibility Test |
| Suite (CTS) Verifier and includes tests that verify image content. As of CTS |
| 7.0_r8, CTS Verifier supports ITS test automation via Camera ITS-in-a-box; |
| support for manual tests continues to ensure coverage for all Android device |
| form factors.</p> |
| |
| <p>ITS-in-a-box brings the following benefits:</p> |
| <ul> |
| <li><strong>Automation</strong>. No human intervention is needed during the |
| test.</li> |
| <li><strong>Reduced testing time</strong>. Parallel testing of front/back camera |
| cuts |
| testing cycle time by 50%.</li> |
| <li><strong>Easier troubleshooting</strong>. Consistency of test environment |
| leads to fewer setup errors and increases reproducibility.</li> |
| <li><strong>Efficiency</strong>. Ability to retry for individual Camera/Scene |
| improves test execution efficiency.</li> |
| </ul> |
| |
| <h2 id=get-started>Getting started</h2> |
| <p>ITS-in-a-box consists of a plastic box that is laser cut from computer-aided |
| design (CAD) drawings, a chart tablet, and a device under test (DUT). You can |
| use the wide field of view (WFoV) ITS-in-a-box, which is capable |
| of testing both WFoV (FoV > 90 degrees) and RFoV (FoV < 90 degrees) cameras, or |
| the regular field of view (RFoV) ITS-in-a-box.</p> |
| |
| <p>To get started with the Camera ITS-in-a-box:</p> |
| <ol> |
| <li>Purchase or build a <a |
| href="/compatibility/cts/camera-wfov-box-assembly">wide field of view |
| (WFoV)</a> or <a href="/compatibility/cts/camera-its-box-assembly">regular field |
| of view (RFoV)</a> ITS-in-a-box.</li> |
| <li><a href="#configure-tablet">Configure a tablet</a> with Camera ITS |
| software.</li> |
| <li><a href="#run-tests">Run tests</a>.</li> |
| <li><a href="#get-results">Get results</a> from the DUT.</li> |
| </ol> |
| |
| <h2 id=configure-tablet>Configuring the tablet</h2> |
| <p>This section provides step-by-step instructions for setting up a |
| tablet for use with the CameraITS software. These instructions use a Pixel C as |
| an example tablet. For information on tablet |
| requirements and recommendations, see <a href="#tablet-requirements">Tablet |
| requirements</a>.</p> |
| |
| <p class="note"><strong>Note:</strong> The CameraITS python scripts |
| automatically set the following options on the tablet for you: |
| <br><em>Settings > Display > Sleep > After 30 minutes of inactivity</em> |
| <br><em>Adaptive brightness > OFF</em> |
| </p> |
| |
| <ol> |
| <li>Charge the tablet and power it on. If prompted to set up an account, skip it |
| (CameraITS does not require any account paired with the tablet).</li> |
| <li>Update the tablet to Android 7.0 or higher. Android 6.x and lower versions |
| do not support CameraITS.</li> |
| <li>Enable <a href="https://developer.android.com/studio/debug/dev-options#enable" |
| class="external">developer mode</a>.</li> |
| <li>Return to <em>Settings</em> and select <strong>Developer options</strong>. |
| |
| <table> |
| <tr> |
| <th>Enable options</th> |
| <td> |
| <ul> |
| <li>On</li> |
| <li>Stay awake</li> |
| <li>USB debugging (This allows the host to run the tablet in debug mode. When you |
| connect the tablet to the host for the first time, the tablet prompts you to |
| "Allow USB debugging?" If the tablet does not display the debug prompt, |
| disconnect then reconnect tablet.)</li> |
| </ul> |
| </td> |
| </tr> |
| <tr> |
| <th>Disable options</th> |
| <td> |
| <ul> |
| <li>Automatic system updates</li> |
| <li>Verify apps over USB</li> |
| </ul> |
| </td> |
| </tr> |
| </table> |
| </li> |
| <li>Determine DUT and chart IDs by running <code>$ adb devices</code> to list |
| available devices. To determine <code>device_id</code> and |
| <code>chart_id</code>, plug and unplug devices and observe the devices that |
| connect and disconnect.</li> |
| <li>Perform three test runs to suppress hints and user prompts that can obscure |
| charts on the tablet screen. |
| <ol style=list-style-lower-alpha> |
| <li>Position the tablet face-up on a table (do not attach the tablet to the back |
| panel of the box)</li> |
| <li>Run the following command: |
| <pre class="devsite-terminal devsite-click-to-copy"> |
| python tools/run_all_tests.py device=$device_id camera=0 chart=$chart_id scenes=2,3 |
| </pre> |
| Scenes 2 and 3 require the tablet to display an image, so the tablet prompts you |
| to "Allow Drive to access photos, media, and files on your device?". Clear this |
| prompt (and prevent future prompts) by pressing <strong>Allow</strong>.</li> |
| <li>Run the command again. The tablet prompts you to "Keep a copy of this file?" |
| and suggests Google Drive. Clear this prompt (and prevent future prompts) by |
| pressing the Drive icon then <strong>Cancel</strong> for upload to drive.</li> |
| <li>Finally, run <code>tools/run_all_tests.py</code> and confirm that scenes |
| change automatically as script cycles through different scenes. While most |
| tests will FAIL (as the camera is not pointed at the chart), you can verify the |
| tablet correctly cycles through the scenes without displaying any prompts or |
| other pop-ups on the screen.</li></ol></li></ol> |
| |
| <h2 id=run-tests>Running tests</h2> |
| <p>Before running the ITS-in-a-box, ensure your test setup includes the following |
| hardware and software:</p> |
| <ul> |
| <li>One (1) ITS-in-a-box</li> |
| <li>One (1) Pixel C for displaying Scenes, S/N: 5811000011</li> |
| <li>Two (2) DUTs that use the same build fingerprint and have the CTS Verifier |
| 7.0_8+ application installed. Example DUTs: |
| |
| <ul> |
| <li>One (1) Pixel NOF26W for the back camera(0) testing, S/N: FA6BM0305016. To |
| install the CTS Verifier app, unzip android-cts-verifier.zip then run |
| <pre class="devsite-terminal devsite-click-to-copy"> |
| adb -s FA6BM0305016 install -r android-cts-verifier/CtsVerifier.apk |
| </pre></li> |
| <li>One (1) Pixel NOF26W for the front camera(1) testing, S/N: FA6BM0305439. To |
| install the CTS Verifier app, unzip android-cts-verifier.zip then run |
| <pre class="devsite-terminal devsite-click-to-copy"> |
| adb -s FA6BM0305439 install -r android-cts-verifier/CtsVerifier.apk |
| </pre> |
| </li> |
| </ul> |
| </li></ul> |
| |
| <h3 id=scenes-0-4>Running scenes 0-4</h3> |
| <p>To run scenes 0-4 on the front and back camera in parallel:</p> |
| <pre class="devsite-click-to-copy"> |
| <code class="devsite-terminal">cd android-cts-verifier/CameraITS</code> |
| <code class="devsite-terminal">. build/envsetup.sh</code> |
| <code class="devsite-terminal">python tools/run_parallel_tests.py device0=FA6BM0305016 device1=FA6BM0305439 chart=5811000011</code> |
| </pre> |
| |
| <p>Examples:</p> |
| <table> |
| <tr> |
| <td ><img src=/compatibility/cts/images/camera_its_cam0.png align="center"></td> |
| <td align="center"><img src=/compatibility/cts/images/camera_its_cam0.png></td> |
| </tr> |
| <tr> |
| <td align="center"><p class=caption><strong>Figure 1</strong>. Camera 0 S/N: |
| FA6BM0305016</p> |
| </td> |
| <td align="center"><p class=caption><strong>Figure 2</strong>. Camera 1 S/N: |
| FA6BM0305439</p> |
| </td> |
| </tr> |
| </table> |
| |
| <h3 id=retry-scenes>Retrying scenes</h3> |
| <p>You can retry scenes for both front and back cameras or a single camera: |
| <ul> |
| <li>To retry scenes on front and back cameras in parallel: |
| <pre class="devsite-terminal devsite-click-to-copy"> |
| python tools/run_parallel_tests.py device0=FA6BM0305016 device1=FA6BM0305439 chart=5811000011 scenes=3,4 |
| </pre> |
| </li> |
| <li>To retry scenes on a single camera: |
| <pre class="devsite-terminal devsite-click-to-copy"> |
| python tools/run_all_tests.py device=FA6BM0305016 chart=5811000011 camera=1 scenes=3,4 |
| </pre> |
| </li> |
| </ul> |
| |
| <h3 id=scenes-0-4>Running scene 5</h3> |
| <p>Scene 5 requires special setup with specific lighting (for details, refer to |
| CameraITS.pdf in CTS Verifier, which you can download at <a href="/compatibility/cts/downloads">Compatibility Test Suite Downloads</a>). |
| You can run Scene 5 separately |
| (outside of the box) to test two devices in parallel.</p> |
| <ul> |
| <li>To run Scene 5 on front and back cameras on two devices in parallel: |
| <pre class="devsite-terminal devsite-click-to-copy"> |
| python tools/run_parallel_tests.py device0=FA6BM0305016 device1=FA6BM0305439 chart=5811000011 scenes=5 |
| </pre> |
| <br><img src=/compatibility/cts/images/camera_its_scene5.png width="50%"><br> |
| <strong>Figure 3</strong>. Camera scene 5.</li> |
| <li>To run Scene 5 for front and back cameras on a single device: |
| <pre class="devsite-click-to-copy"> |
| <code class="devsite-terminal">python tools/run_all_tests.py device=FA6BM0305016 camera=0 scenes=5</code> |
| <code class="devsite-terminal">python tools/run_all_tests.py device=FA6BM0305016 camera=1 scenes=5</code> |
| </pre></li> |
| </ul> |
| |
| <h2 id=get-results>Getting results</h2> |
| <p>You can view results during testing and save the completed results as a report. |
| </p> |
| <ul> |
| <li><strong>View progress of running tests</strong>. The command |
| <code>run_parallel_tests</code> prints results only after Camera-Scene tests |
| have finished, so to view results during test execution you must use Android |
| Device Monitor or <code>adb logcat</code> to verify progress and/or view |
| screenshots.<br> |
| <br>Example adb command: |
| <pre class="devsite-terminal devsite-click-to-copy"> |
| adb -s FA6BM0305016 logcat -v time |
| </pre> |
| Example screenshots command: |
| <pre class="devsite-click-to-copy"> |
| <code class="devsite-terminal">adb -s FA6BM0305016 shell screencap -p /sdcard/screencap.png</code> |
| <code class="devsite-terminal">adb -s FA6BM0305016 pull /sdcard/screencap.png</code> |
| <code class="devsite-terminal">display ./screencap.png</code> |
| </pre></li> |
| <li><strong>View results</strong>. To save Camera ITS results as a report: |
| <ol> |
| <li>Press <strong>Pass</strong> and save the report: |
| <br><img src=/compatibility/cts/images/camera_its_results.png width="50%"><br> |
| <strong>Figure 4</strong>. Camera ITS report.</li> |
| <li>Pull reports from the device: |
| <pre class="devsite-terminal devsite-click-to-copy"> |
| adb -s FA6BM0305016 pull /sdcard/verifierReports |
| </pre> |
| </li> |
| <li>Unzip the report file and view the test_result.xml. |
| <br><img src=/compatibility/cts/images/camera_its_reports.png><br> |
| <strong>Figure 5</strong>. Camera ITS reports.<br></li> |
| </ol> |
| </li> |
| </ul> |
| |
| <h2 id=tablet-requirements>Tablet requirements</h2> |
| |
| <p>Tablets must have a display size of around 10 inches with a screen resolution |
| greater than 2000 x 1500 pixels. The <code>DISPLAY_LEVEL</code> value must be |
| set in |
| <code>CameraITS/tools/wake_up_screen.py</code> according to the tablet model. |
| The table below lists the values for recommended tablets.</p> |
| |
| <p>Recommended tablets for ITS testing are shown below:</p> |
| |
| <table> |
| <tr> |
| <th>Device</th> |
| <th>Display size<br>(inches)</th> |
| <th>Display size<br>(pixels)</th> |
| <th>Tablet dimensions<br>(inches)</th> |
| <th>DISPLAY_LEVEL</th> |
| <th>OS</th> |
| </tr> |
| <tr> |
| <td>Asus Zen Pad 3</td> |
| <td>9.7</td> |
| <td>2048 x 1536</td> |
| <td>9.47 x 6.44 x 0.28</td> |
| <td>192</td> |
| <td>Android 6.0</td> |
| </tr> |
| <tr> |
| <td>Huawei Mediapad m5</td> |
| <td>10.8</td> |
| <td>2560 x 1600</td> |
| <td>10.18 x 6.76 x 0.29</td> |
| <td>192</td> |
| <td>Android 8.0</td> |
| </tr> |
| <tr> |
| <td>Pixel C</td> |
| <td>10.2</td> |
| <td>2560 x 1800</td> |
| <td>9.53 x 7.05 x 0.28</td> |
| <td>96</td> |
| <td>Android 8.0</td> |
| </tr> |
| <tr> |
| <td>Samsung S3</td> |
| <td>9.7</td> |
| <td>2048 x 1536</td> |
| <td>10.76 x 6.65 x 0.24</td> |
| <td>192</td> |
| <td>Android 7.0</td> |
| </tr> |
| <tr> |
| <td>Sony Xperia Z4</td> |
| <td>10.1</td> |
| <td>2560 x 1600</td> |
| <td>10 x 6.57 x 0.24</td> |
| <td>192</td> |
| <td>Android 7.0</td> |
| </tr> |
| </table> |
| |
| </body> |
| </html> |