1- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- 2- LLDB for Renderscript Test Suite 3- 4- 16/03/2016 5- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- 6 7Overview: 8 9 The LLDB for Renderscript test suite is written in python and relies on 10 LLDB's python bindings. The test suite will push several test app's onto 11 a target device, and makes coordinated use of LLDB and ADB to run automated 12 debug sessions. 13 14Set up the test suite: 15 16 Check out the AOSP and build it for your target. Navigate to 17 /frameworks/rs/test/lldb and type mm. This should successfully build the 18 binaries that the testsuite uses. They will be placed in 19 <path to out folder>/target/product/<product code name>/data/app, system/lib 20 and system/bin. 21 22Prerequisite: 23 24 An lldb-server executable must be present on your device/emulator. 25 LLDB must be compiled on your host machine along with its python interface. 26 lldb-server and lldb should be built from the same source revisions. 27 28Running the test suite: 29 30 The test suite can be run via the following command: 31 32 > python run_tests.py [-h] 33 [--config path] 34 [--device DEVICE] 35 [--test path] 36 [--install-only] 37 [--no-install] 38 [--no-uninstall] 39 [--print-to-stdout] 40 [--verbose] 41 [--wimpy] 42 [--run-emu] 43 [--adb-path ADB_PATH] 44 [--aosp-product-path AOSP_PRODUCT_PATH] 45 [--blacklist BLACKLIST [BLACKLIST ...]] 46 [--device-port DEVICE_PORT] 47 [--emu-cmd EMU_CMD] 48 [--host-port HOST_PORT] 49 [--lldb-path LLDB_PATH] 50 [--lldb-server-path-device LLDB_SERVER_PATH_DEVICE] 51 [--lldb-server-path-host LLDB_SERVER_PATH_HOST] 52 [--log-file-path LOG_FILE_PATH] 53 [--results-file-path RESULTS_FILE_PATH] 54 [--timeout TIMEOUT] 55 56 optional arguments: 57 -h, --help show this help message and exit 58 --config path, -c path 59 Path to a custom config file. 60 --device DEVICE, -d DEVICE 61 Specify the device id of the device to test on. 62 --test path, -t path Specify a specific test to run. 63 --install-only It only runs the pre-run stage of the test suite. It 64 installs the required APKs but does not execute the 65 tests. 66 --no-install, -n Stop the test suite installing apks to device. 67 --no-uninstall Stop the test suite uninstalling apks after 68 completion. 69 --print-to-stdout Print all logging information to standard out. 70 --verbose, -v Store extra info in the log. 71 --wimpy, -w Test only a core subset of features. 72 --run-emu Spawn an emulator and run the test suite on that. 73 Specify the emulator command line in the config file 74 or with -emu-cmd. 75 --adb-path ADB_PATH Path to android debug bridge on the host. 76 --aosp-product-path AOSP_PRODUCT_PATH 77 The path to the "out" folder of the AOSP repository. 78 --blacklist BLACKLIST [BLACKLIST ...] 79 Provide a test blacklist for skipping specific tests. 80 To specify the blacklist from the command line the 81 following can be used: --blacklist test1.py test2.py 82 ... 83 --device-port DEVICE_PORT 84 Specify the port number that lldb-server (on the 85 device) listens on. When lldb-server is spawned on the 86 device it will listen on this port. Each successive 87 test will increment onwards from this port. 88 --emu-cmd EMU_CMD The command line for the emulator (if using -run-emu). 89 --host-port HOST_PORT 90 Specify host port which lldb-server will be forwarded 91 to. Specify the starting host port number that lldb- 92 server (on the target) will be forwarded to on the 93 host. Each successive test will increment onwards from 94 this initial port. 95 --lldb-path LLDB_PATH 96 The path to lldb executable on the host. 97 --lldb-server-path-device LLDB_SERVER_PATH_DEVICE 98 Path to the lldb-server executable on the device. 99 --lldb-server-path-host LLDB_SERVER_PATH_HOST 100 Path to the lldb-server executable on host (if using 101 -run-emu). 102 --log-file-path LOG_FILE_PATH 103 The path to the folder where the log file will be 104 placed. 105 --results-file-path RESULTS_FILE_PATH 106 Output folder where junit results.xml file will be 107 placed. 108 --timeout TIMEOUT Timeout period for a single command, expressed in 109 seconds 110 111 An optional config file can be passed to the test suite which will provide 112 details of your specific environment. The user file should define a custom 113 class inheriting from Config in config.py. The class Config presents the 114 default set of options, available to be overridden. 115 116 All options in the config file can also be specified on the command line. 117 118 If your config and command line do not specify a path to the host lldb, 119 the PYTHONPATH environment variable must be set. The appropriate value to 120 set this to can be obtained by running the following command: 121 122 > lldb -P 123 124 This will print out a path to the lldb python bindings on your local machine. 125 126Build Requirements: 127 128 129 The following revisions are from the llvm git mirror: 130 131 llvm : 5786b73 132 clang: b6d0b32 133 lldb : 98712eb 134 135 lldb has the following dependencies: 136 137 Python2.7.6 138 swig2.0 139 lldb-server 140 141Building LLDB python bindings: 142 143 Linux: 144 145 Build instructions for Linux lldb can be found on the official lldb web 146 page: 147 148 http://lldb.llvm.org/build.html 149 150 The following CMake variables should be enabled when generating: 151 152 LLDB_ENABLE_PYTHON_SCRIPTS_SWIG_API_GENERATION = True 153 154 As a post build step, swig will generate the python bindings for lldb. 155 156 Windows: 157 158 Prerequisites: 159 160 Visual Studio 2015 161 Custom x64 Python2.7.10 package: 162 http://p-nand-q.com/python/2015.08.07-Python2710-x64-vs2015.7z 163 Swig Version 3.0.5 (Added to the $PATH) 164 Cmake 3.4.0 (So we can generate Visual Studio 2015 solutions) 165 166 Build Python for Windows: 167 168 http://p-nand-q.com/python/building-python-27-with-vs2010.html 169 170 The important thing here is that the above python distribution 171 contains debug versions of the libraries and is built with the same 172 Visual Studio version we are using so the runtimes do not conflict. 173 174 Build LLDB on Windows: 175 176 Select the Cmake generator "Microsoft Visual Studio 2015 Win64". 177 The following CMake variables should be enabled when generating: 178 179 LLDB_DISABLE_PYTHON=False 180 LLDB_RELOCATABLE_PYTHON=False 181 PYTHON_HOME=<path to the above python release> 182 183 Using cmake-gui is a good idea and lets you make sure that swig has 184 been correctly detected. 185 186 In the CMake configure step, you should see something similar to the 187 following: 188 -- Found Python version 2.7.9 189 -- LLDB Found PythonExecutable: E:/Python27/python.exe and 190 E:/Python27/python_d.exe 191 -- LLDB Found PythonLibs: E:/Python27/libs/python27.lib and 192 E:/Python27/libs/python27_d.lib 193 -- LLDB Found PythonDLL: E:/Python27/python27.dll and 194 E:/Python27/python27_d.dll 195 -- LLDB Found PythonIncludeDirs: E:/Python27/Include 196 LLDB version: 3.8.0 197 Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE) 198 Found PythonInterp: 199 $<$<CONFIG:Debug>:E:/Python27/python_d.exe>$<$<NOT:$<CONFIG:Debug>>:E:/Python27/python.exe> 200 (found version "1.4") 201 202 Build LLDB as a RelWithDebInfo build, because debug builds of lldb 203 produce an lldb_d.pyd file, but the __init__.py still refers to 204 lldb.pyd. 205 206 The LLDB python bindings should have built in 207 "llvm_build\RelWithDebInfo\lib\site-packages\lldb". Point the test 208 suite towards "llvm_build\RelWithDebInfo\bin\lldb.exe". 209 When running the test suite itself, make sure to use the python 210 executable from the custom package. 211 212A typical test transcript: 213 214 Located ADB 215 Located device ZX1G427S2S 216 Pushing all tests... 217 Pushed all tests 218 Pre run complete 219 Found 86 tests 220 Running test_allocation_dump_2_cpp.py: PASS 221 Running test_breakpoint_fileline_multiple_rs_files_cpp.py: PASS 222 Running test_read_local_cpp.py: PASS 223 Running test_breakpoint_kernel_multiple_rs_files_cpp.py: PASS 224 Running test_multiple_rs_files_cpp.py: PASS 225 Running test_breakpoint_kernel_all_multiple_rs_files_cpp.py: PASS 226 Running test_dwarf_lang_cpp.py: PASS 227 Running test_write_global_element_cpp.py: PASS 228 Running test_allocation_list_cpp.py: PASS 229 Running test_breakpoint_coordinate_cpp.py: PASS 230 Running test_rs_consts_cpp.py: PASS 231 Running test_allocation_file_cpp.py: PASS 232 Running test_allocation_dump_1_cpp.py: PASS 233 Running test_source_step_cpp.py: PASS 234 Running test_breakpoint_kernel_2_cpp.py: PASS 235 Running test_backtrace_cpp.py: PASS 236 Running test_language_subcmds_no_debug_cpp.py: PASS 237 Running test_breakpoint_kernel_1_cpp.py: PASS 238 Running test_language_subcmds_cpp.py: PASS 239 Running test_write_global_cpp.py: PASS 240 Running test_invoke_fun_cpp.py: PASS 241 Running test_breakpoint_fileline_cpp.py: PASS 242 Running test_write_local_cpp.py: PASS 243 Running test_breakpoint_kernel_all_cpp.py: PASS 244 Running test_write_local_element_cpp.py: PASS 245 Running test_call_api_funs_cpp.py: PASS 246 Running test_coordinates_cpp.py: PASS 247 Running test_read_global_cpp.py: PASS 248 Running test_language_subcmds.py: PASS 249 Running test_coordinates.py: PASS 250 Running test_language_subcmds_no_debug.py: PASS 251 Running test_read_local.py: PASS 252 Running test_call_api_funs.py: PASS 253 Running test_breakpoint_kernel_1.py: PASS 254 Running test_breakpoint_fileline.py: PASS 255 Running test_breakpoint_fileline_multiple_rs_files.py: PASS 256 Running test_rs_consts.py: PASS 257 Running test_invoke_fun.py: PASS 258 Running test_write_local_element.py: PASS 259 Running test_source_step.py: PASS 260 Running test_allocation_file.py: PASS 261 Running test_allocation_list.py: PASS 262 Running test_breakpoint_kernel_multiple_rs_files.py: PASS 263 Running test_allocation_dump_1.py: PASS 264 Running test_breakpoint_kernel_all.py: PASS 265 Running test_allocation_dump_2.py: PASS 266 Running test_allocation_dump_struct.py: PASS 267 Running test_read_global.py: PASS 268 Running test_language.py: PASS 269 Running test_dwarf_lang.py: PASS 270 Running test_breakpoint_coordinate.py: PASS 271 Running test_write_global.py: PASS 272 Running test_multiple_rs_files.py: PASS 273 Running test_write_global_element.py: PASS 274 Running test_breakpoint_kernel_all_multiple_rs_files.py: PASS 275 Running test_breakpoint_kernel_2.py: PASS 276 Running test_write_local.py: PASS 277 Running test_backtrace.py: PASS 278 Running test_call_api_funs_jni.py: PASS 279 Running test_invoke_fun_jni.py: PASS 280 Running test_allocation_dump_1_jni.py: PASS 281 Running test_breakpoint_fileline_multiple_rs_files_jni.py: PASS 282 Running test_allocation_file_jni.py: PASS 283 Running test_breakpoint_fileline_jni.py: PASS 284 Running test_source_step_jni.py: PASS 285 Running test_coordinates_jni.py: PASS 286 Running test_rs_consts_jni.py: PASS 287 Running test_breakpoint_kernel_all_multiple_rs_files_jni.py: PASS 288 Running test_multiple_rs_files_jni.py: PASS 289 Running test_allocation_dump_2_jni.py: PASS 290 Running test_allocation_list_jni.py: PASS 291 Running test_write_local_element_jni.py: PASS 292 Running test_breakpoint_kernel_all_jni.py: PASS 293 Running test_breakpoint_coordinate_jni.py: PASS 294 Running test_language_subcmds_no_debug_jni.py: PASS 295 Running test_read_local_jni.py: PASS 296 Running test_dwarf_lang_jni.py: PASS 297 Running test_breakpoint_kernel_2_jni.py: PASS 298 Running test_breakpoint_kernel_multiple_rs_files_jni.py: PASS 299 Running test_write_global_element_jni.py: PASS 300 Running test_breakpoint_kernel_1_jni.py: PASS 301 Running test_read_global_jni.py: PASS 302 Running test_language_subcmds_jni.py: PASS 303 Running test_write_global_jni.py: PASS 304 Running test_backtrace_jni.py: PASS 305 Running test_write_local_jni.py: PASS 306 Uninstalled/Deleted all tests 307 86 of 86 passed 308 100% rate 309