836e0f1633f6e53ccdf65d72da2acff6d4d666f2 |
|
14-Jul-2017 |
Po-Hsien Wang <pwang@chromium.org> |
graphics_dEQP.py: fix chromeperf failure records 1) Fix number mismatch from wmatrix & chromeperf 2) Use dedicate function to dispatch _run_tests_invidually & run_tests_hasty BUG=chromium:742599 TEST=test_that cave graphics_dEQP.vk-master Before: test_that cave graphics_dEQP.vk-master, cat /tmp/test_that_../.../result-charts.json "Failures_VK": { "summary": { "units": "count", "type": "scalar", "value": 134115.0, "improvement_direction": "down" } } After: test_that cave graphics_dEQP.vk-master, cat /tmp/test_that_../.../result-charts.json "Failures_VK": { "summary": { "units": "count", "type": "scalar", "value": 56.0, "improvement_direction": "down" } } Change-Id: I5390c86efd303cbf57c8bab60ec330ea5d2d4b7a Reviewed-on: https://chromium-review.googlesource.com/571407 Commit-Ready: Ilja H. Friedel <ihf@chromium.org> Tested-by: Ilja H. Friedel <ihf@chromium.org> Reviewed-by: Ilja H. Friedel <ihf@chromium.org>
/external/autotest/client/site_tests/graphics_dEQP/graphics_dEQP.py
|
b2bc3fb92b98414ea2715439a70050cc0395934a |
|
14-Jun-2017 |
Po-Hsien Wang <pwang@chromium.org> |
graphics_dEQP: Report of failures to chromeperf 1) Report of failures to chrome perf as gfx regression alerts. 2) Modify generate_controlfiles so that control.vk-master is generated correctly. 3) Modify perf_dashboard_config.json BUG=chromium:717664 TEST=test_that dut graphics_dEQP.gles2-master.hasty.8 Change-Id: I7db5d5b2e521aecc69eca596d41383c78b4e191f Reviewed-on: https://chromium-review.googlesource.com/534576 Commit-Ready: Ilja H. Friedel <ihf@chromium.org> Tested-by: Ilja H. Friedel <ihf@chromium.org> Reviewed-by: Ilja H. Friedel <ihf@chromium.org>
/external/autotest/client/site_tests/graphics_dEQP/graphics_dEQP.py
|
7c0bccc268fc4a87fa3b586387a6414ee508299b |
|
24-May-2017 |
Ilja H. Friedel <ihf@chromium.org> |
graphics_dEQP: add a hand selected bvt list. This runs on minnie on 15 seconds and exercises the gles2, gles3, gles31 and vk APIs (vk of course is skipped on minnie). BUG=chromium:725656 TEST=test_that <ip> graphics_dEQP.bvt graphics_dEQP.bvt [ PASSED ] graphics_dEQP.bvt Pass 24 graphics_dEQP.bvt Skipped 19 Change-Id: Ib2ee6edec432fa9674c170cae72a0ecb40f80781 Reviewed-on: https://chromium-review.googlesource.com/513508 Commit-Ready: Ilja H. Friedel <ihf@chromium.org> Tested-by: Ilja H. Friedel <ihf@chromium.org> Reviewed-by: Chad Versace <chadversary@chromium.org>
/external/autotest/client/site_tests/graphics_dEQP/graphics_dEQP.py
|
af271023c914bbf0870ae4d3a866ae647f754ed3 |
|
10-May-2017 |
Po-Hsien Wang <pwang@chromium.org> |
graphics_dEQP.py: rerun failing test after hasty mode Tests which is indicated as failure in hasty mode will be re-runed in individual mode as some tests cause problem to the next following test and make it fail. BUG=chromium:712299 TEST=test_that veyron_minnie graphics_dEQP.gles2-master.hasty.9 (original: 1/1415, after rerun: 0/1415) Change-Id: I43fc844ea7c12bb36b3f59ba5f83a81f8b2945fb Reviewed-on: https://chromium-review.googlesource.com/502333 Commit-Ready: Po-Hsien Wang <pwang@chromium.org> Tested-by: Ilja H. Friedel <ihf@chromium.org> Reviewed-by: Ilja H. Friedel <ihf@chromium.org>
/external/autotest/client/site_tests/graphics_dEQP/graphics_dEQP.py
|
ce4ce2601e62cf5d04ed7eb06da7badda2b21c32 |
|
10-May-2017 |
Po-Hsien Wang <pwang@chromium.org> |
graphics_dEQP.py: return failing tests in results Add names of failing tests to a list in the results in _parse_test_results. BUG=chromium:712299 TEST=test_that gles2-master.hasty.9 Change-Id: I5948ff386d932cb090b86e315dd65efda6c2d733 Reviewed-on: https://chromium-review.googlesource.com/502332 Commit-Ready: Po-Hsien Wang <pwang@chromium.org> Tested-by: Ilja H. Friedel <ihf@chromium.org> Reviewed-by: Ilja H. Friedel <ihf@chromium.org>
/external/autotest/client/site_tests/graphics_dEQP/graphics_dEQP.py
|
37f61217c3df4e5ef7688f724fa239598786033f |
|
24-Apr-2017 |
Po-Hsien Wang <pwang@chromium.org> |
graphics_dEQP: Fix not exist error for vk-master 1) Check test_names_file to see if the API is supported 2) Delete existing error.TestWarn as they might cause trouble in different system 3) Add a GraphicsApiHelper in graphics_utils.py 4) Change vk-master run frequency BUG=None TEST=test_that on veyron_minnie Change-Id: Ied15ad9ab362e8029e085585d66342d0293889ef Reviewed-on: https://chromium-review.googlesource.com/486161 Commit-Ready: Ilja H. Friedel <ihf@chromium.org> Tested-by: Ilja H. Friedel <ihf@chromium.org> Reviewed-by: Ilja H. Friedel <ihf@chromium.org>
/external/autotest/client/site_tests/graphics_dEQP/graphics_dEQP.py
|
0a37d8effca38bc9c7b36d5fa218f2561e3b24cc |
|
18-Feb-2017 |
Chad Versace <chadversary@chromium.org> |
graphics_dEQP: Prepend /usr/local/$libdir to LD_LIBRARY_PATH Prepend "/usr/local/lib:/usr/local/lib64" to LD_LIBRARY_PATH for all dEQP test processes. Fixes the Vulkan CTS. Currently, no package in the base image requires media-libs/vulkan-loader. Only media-gfx/deqp does, which is in the test image. Therefore the build_image scripts install libvulkan.so into /usr/local. When in a login shell on Chrome OS, LD_LIBRARY_PATH does contain the /usr/local libdirs. But processes spawned by autotest do not. Therefore, deqp worked when ran from a login shell; but it failed to dlopen libvulkan.so when ran by autotest. BUG=chromium:689177 TEST=Run `test_that ... graphics_dEQP.vk.info` on sentry. Inspect client.0.DEBUG to confirm that tests ran. Change-Id: Ib067acc76db1364328b45ef2c66a6a73a4e12c38 Reviewed-on: https://chromium-review.googlesource.com/444984 Commit-Ready: Ilja H. Friedel <ihf@chromium.org> Tested-by: Chad Versace <chadversary@chromium.org> Reviewed-by: Ilja H. Friedel <ihf@chromium.org> Reviewed-by: Gurchetan Singh <gurchetansingh@chromium.org>
/external/autotest/client/site_tests/graphics_dEQP/graphics_dEQP.py
|
360a64ab4f40744e63f467409d9252f39516d0d0 |
|
03-Feb-2017 |
Chad Versace <chadversary@chromium.org> |
graphics_dEQP: Add Vulkan CTS Recent versions of dEQP include the Vulkan CTS. Add the following control files for it: control.vk-master # The mustpass list from the Android CTS control.vk-master.hasty control.vk.api.hasty control.vk.api.smoke control.vk.binding_model.hasty control.vk.glsl.hasty control.vk.info control.vk.pipeline.hasty control.vk.spirv_assembly.hasty To run these tests, media-gfx/deqp must be built with USE=vulkan. I tested this on reef (which has an Apollo Lake GPU) with mesa-17.1.0-r3, and all tests passed in suite 'graphics_dEQP.vk-master.hasty'. I also did a smoke test on sentry (Skylake GPU) with 'graphics_dEQP.vk.api.smoke'. BUG=chromium:689177 TEST=$ USE=vulkan emerge-sentry deqp $ cros deploy $SENTRY_IP deqp mesa vulkan-loader $ test_that --board=sentry $SENTRY_IP graphics_dEQP.vk.api.smoke # Confirm 6/6 tests pass. v4: Fix crash when deqp-vk is not installed. For each dEQP executable (deqp-egl, deqp-vk, deqp-gles3, etc), the script chdir'd into the executable's directory before testing if the executable existed. This made Python unhappy by raising an OSError. Depends-On: I10233e70cdf2303ace300bef39896e98f2d7636b Change-Id: I854876d100ad6ece737459661ca2a6b2a05e395c Reviewed-on: https://chromium-review.googlesource.com/438766 Commit-Ready: Ilja H. Friedel <ihf@chromium.org> Tested-by: Chad Versace <chadversary@chromium.org> Reviewed-by: Ilja H. Friedel <ihf@chromium.org>
/external/autotest/client/site_tests/graphics_dEQP/graphics_dEQP.py
|
aa2187ee300bff08db48bca4c9f29f2673b88fa7 |
|
10-Jan-2017 |
Ilja H. Friedel <ihf@chromium.org> |
graphics_dEQP: add gpu in reporting to wmatrix. BUG=None. TEST=pylint Change-Id: I8e752b119e5cd4691087e94e7db30763c8877838 Reviewed-on: https://chromium-review.googlesource.com/426371 Commit-Queue: Ilja H. Friedel <ihf@chromium.org> Tested-by: Ilja H. Friedel <ihf@chromium.org> Trybot-Ready: Ilja H. Friedel <ihf@chromium.org> Reviewed-by: Haixia Shi <hshi@chromium.org>
/external/autotest/client/site_tests/graphics_dEQP/graphics_dEQP.py
|
8245a0b28b82e76d307657856e7165685444acb2 |
|
02-Nov-2016 |
Ilja H. Friedel <ihf@chromium.org> |
graphics_dEQP: enable on tegra. But don't fail tests (warning only for now) until we have new expectations. BUG=chromium:655820 TEST=Ran on nyan_blaze. Change-Id: I6b6f74be8e5b4e81d4dd6940234bc1bdd67662bc Reviewed-on: https://chromium-review.googlesource.com/406629 Commit-Ready: Ilja H. Friedel <ihf@chromium.org> Tested-by: Ilja H. Friedel <ihf@chromium.org> Reviewed-by: Haixia Shi <hshi@chromium.org>
/external/autotest/client/site_tests/graphics_dEQP/graphics_dEQP.py
|
619ee2569463359d155539539f5c0d7b2515ccd6 |
|
29-Oct-2016 |
Ilja H. Friedel <ihf@chromium.org> |
autotest graphics_*: unify TestStatus. The goal is for graphics tests to only pass or fail. As much as possible do not use TestError, TestWarn or TestNA etc. Having only Pass/Fail will make reading on external dashboards (wmatrix, sponge) less ambiguous. Also prepend all TestFail messages with "Failed:" to make reading/understanding messages easier. BUG=chromium:647465 TEST=pylint Change-Id: I92160c9def35b72e22af8c19af6275932a63d00b Reviewed-on: https://chromium-review.googlesource.com/404953 Commit-Ready: Ilja H. Friedel <ihf@chromium.org> Tested-by: Ilja H. Friedel <ihf@chromium.org> Reviewed-by: Dominik Behr <dbehr@chromium.org>
/external/autotest/client/site_tests/graphics_dEQP/graphics_dEQP.py
|
cbe677a4c77a21b1afba39d7d4552e357694c561 |
|
07-Aug-2016 |
Mike Mason <michael.w.mason@intel.com> |
graphics_dEQP.py: log message format change causes exception The following change in a previous commit causes an exception when formatting the message: - logging.info('Found gles%d.%d.' % (major, minor)) + logging.info('Found gles%d.%d.', (major, minor)) This patch removes the parenthesis from 'major, minor' so the format works. Change-Id: Ic6d942d1aef97b7d06c241c7fc4546806becca21 Signed-off-by: Mike Mason <michael.w.mason@intel.com> Reviewed-on: https://chromium-review.googlesource.com/366981 Commit-Ready: Ilja H. Friedel <ihf@chromium.org> Tested-by: Ilja H. Friedel <ihf@chromium.org> Reviewed-by: Ilja H. Friedel <ihf@chromium.org>
/external/autotest/client/site_tests/graphics_dEQP/graphics_dEQP.py
|
b413556fc5d9ccf1dff1d12ee435334a5d06530f |
|
05-Aug-2016 |
Mike Mason <michael.w.mason@intel.com> |
graphics_dEQP: count warnings as passes Android CTS counts QualityWarning and CompatibilityWarning results as passes when determining the pass/fail summary. The Chrome OS dEQP autotest does not. This patch aligns the autotest with CTS so both calculate summary values the same way. BUG=None. TEST=test_that $DUT graphics_dEQP.gles31-master Observe that warnings are now counted as passes in the fail/pass summary at the end of the test. Change-Id: I2393ea6070294cecb16cc78e2e20f2cd070f82c8 Signed-off-by: Mike Mason <michael.w.mason@intel.com> Reviewed-on: https://chromium-review.googlesource.com/366872 Commit-Ready: Ilja H. Friedel <ihf@chromium.org> Tested-by: Ilja H. Friedel <ihf@chromium.org> Reviewed-by: Ilja H. Friedel <ihf@chromium.org>
/external/autotest/client/site_tests/graphics_dEQP/graphics_dEQP.py
|
6ee34dbe7e01556c4b6e163df63d958e4d64ae70 |
|
01-Jul-2016 |
Ilja H. Friedel <ihf@chromium.org> |
graphics_dEQP: reduce spew to logs. The dEQP logs can be on the rather large size. Have less spew recorded for now. BUG=None. TEST=test_that $DUT graphics_dEQP.gles3.info Change-Id: I50e35cca091ceb5865de675b99db8554af996224 Reviewed-on: https://chromium-review.googlesource.com/357952 Reviewed-by: Ilja H. Friedel <ihf@chromium.org> Commit-Queue: Ilja H. Friedel <ihf@chromium.org> Trybot-Ready: Ilja H. Friedel <ihf@chromium.org> Tested-by: Ilja H. Friedel <ihf@chromium.org>
/external/autotest/client/site_tests/graphics_dEQP/graphics_dEQP.py
|
7beb8026cb255c27ecf622a9de5db0b24fe5ffb2 |
|
07-Jun-2016 |
Mike Mason <michael.w.mason@intel.com> |
graphics_dEQP: Add start/end times to debug logs Some dEQP tests can leave the system in a state that causes subsequent tests to fail. Adding start/end times to the per-test debug logs can help determine the order tests were run and which test may have caused other tests to fail. BUG=None. TEST=Run graphics_dEQP.py with the debug option and observe start and end times in the debug logs. Change-Id: Ie73145b6058f53d85631c5834a416ea82d3e03d7 Signed-off-by: Mike Mason <michael.w.mason@intel.com> Reviewed-on: https://chromium-review.googlesource.com/350398 Commit-Ready: Ilja H. Friedel <ihf@chromium.org> Tested-by: Ilja H. Friedel <ihf@chromium.org> Reviewed-by: Ilja H. Friedel <ihf@chromium.org>
/external/autotest/client/site_tests/graphics_dEQP/graphics_dEQP.py
|
39334f4feac6f2b6d4afca7e526f1c510ebce940 |
|
29-Apr-2016 |
Mike Mason <michael.w.mason@intel.com> |
graphics_dEQP: Debug section should be indented one more level The "if self._debug" section beginning on line 339 should be indented one more level so it falls under the "else" clause beginning on line 315. This debug code only applies to tests that are run, not skipped tests. BUG=None. TEST=Tested on two devices that generate skipped tests (gles3.1 on link and falco) to ensure the debug path is not followed. Ran same tests on cyan to ensure the debug section is run and debug files are generated. Change-Id: Ibb6a3be4b3207594ff591d7f550fa5433fbd593f Signed-off-by: Mike Mason <michael.w.mason@intel.com> Reviewed-on: https://chromium-review.googlesource.com/341332 Commit-Ready: Ilja H. Friedel <ihf@chromium.org> Tested-by: Ilja H. Friedel <ihf@chromium.org> Reviewed-by: Ilja H. Friedel <ihf@chromium.org>
/external/autotest/client/site_tests/graphics_dEQP/graphics_dEQP.py
|
3e16c8c327f7e8d9514ec1dc5150fd017b1d07b2 |
|
25-May-2016 |
Ilja H. Friedel <ihf@chromium.org> |
graphics_dEQP: switch rogue to pbuffers. BUG=chrome-os-partner:52663 TEST=None. Change-Id: I11220f99f3221d4d38e6dc297dd6a94281490f41 Reviewed-on: https://chromium-review.googlesource.com/347240 Commit-Ready: Ilja Friedel <ihf@chromium.org> Tested-by: Ilja Friedel <ihf@chromium.org> Reviewed-by: Mao Huang <littlecvr@google.com>
/external/autotest/client/site_tests/graphics_dEQP/graphics_dEQP.py
|
42c5d9dc7ddb0fca45bc9d1801246fbaf99fcc12 |
|
25-Apr-2016 |
Gurchetan Singh <gurchetansingh@chromium.org> |
graphics_dEQP: Run with pbuffer option on all boards except Rogue. There are still some changes needed to enable pbuffers on Rogue (partner bug 52663). Until then, special case the board. BUG=chromium:606472 TEST=Ran dEQP on Cyan and Oak, and verified the desired options were present. Change-Id: Id22c3e3981182ec71fc75f00086583a0d6e5a8ca Reviewed-on: https://chromium-review.googlesource.com/340495 Commit-Ready: Ilja Friedel <ihf@chromium.org> Tested-by: Ilja Friedel <ihf@chromium.org> Reviewed-by: Ilja Friedel <ihf@chromium.org>
/external/autotest/client/site_tests/graphics_dEQP/graphics_dEQP.py
|
4f4e936de19af1ed20a5cbf1b5378bab3bd9c4a1 |
|
23-Apr-2016 |
Ilja H. Friedel <ihf@chromium.org> |
graphics_dEQP: switch to rgba8888d24s8ms0. We should read configs from the xml files, but that will take a bit more work. For now use rgba8888d24s8ms0 instead of rgba8888d24s8. BUG=None. TEST=None. Change-Id: I000b78f1edec45d827446675bbc898609cf3566d Reviewed-on: https://chromium-review.googlesource.com/340363 Commit-Ready: Ilja Friedel <ihf@chromium.org> Tested-by: Ilja Friedel <ihf@chromium.org> Reviewed-by: Ilja Friedel <ihf@chromium.org>
/external/autotest/client/site_tests/graphics_dEQP/graphics_dEQP.py
|
9d61cf98e9a613713182934d0ddf62dbe0948e74 |
|
12-Apr-2016 |
Ilja H. Friedel <ihf@chromium.org> |
graphics_dEQP: raise TestNA instead of WARN. bvt-cq files automatic bugs on warning, which can happen on machines which for instance don't support gles31. BUG=chromium:601323 TEST=None. Change-Id: I7db8703fa4000f8e81bad18a5fbbc3ff15b6bb74 Reviewed-on: https://chromium-review.googlesource.com/338251 Commit-Ready: Ilja Friedel <ihf@chromium.org> Tested-by: Ilja Friedel <ihf@chromium.org> Reviewed-by: Ilja Friedel <ihf@chromium.org> Reviewed-by: Haixia Shi <hshi@chromium.org>
/external/autotest/client/site_tests/graphics_dEQP/graphics_dEQP.py
|
d82eb680b395b122c2a50a4852a2703373edf784 |
|
05-Apr-2016 |
Ilja H. Friedel <ihf@chromium.org> |
graphics_dEQP: fix string matching. gles3 is contained in gles31. Avoid this. BUG=None. TEST=Ran on link bin/autotest tests/graphics_dEQP/control.gles2.info bin/autotest tests/graphics_dEQP/control.gles3.info bin/autotest tests/graphics_dEQP/control.gles31.info Change-Id: Iec23c4e0174898734b617d4d1ad347d6f63c2313 Reviewed-on: https://chromium-review.googlesource.com/337055 Commit-Ready: Ilja Friedel <ihf@chromium.org> Tested-by: Ilja Friedel <ihf@chromium.org> Reviewed-by: Dominik Behr <dbehr@chromium.org>
/external/autotest/client/site_tests/graphics_dEQP/graphics_dEQP.py
|
e7c97f899c73c271abcd7eafa73120864e5da604 |
|
25-Mar-2016 |
Ilja H. Friedel <ihf@chromium.org> |
graphics_dEQP: add generic way to skip tests. Use wflinfo to query gles version before running tests. Refactor graphics_GLAPICheck. BUG=None. TEST=Ran graphics_dEQP.gles{2|3|31}.info on alex, daisy and lars. Ran graphics_GLAPICheck on alex. Change-Id: Ifaf1e20240152338439c050a4481d2933def9422 Reviewed-on: https://chromium-review.googlesource.com/334872 Commit-Ready: Ilja Friedel <ihf@chromium.org> Tested-by: Ilja Friedel <ihf@chromium.org> Reviewed-by: Stéphane Marchesin <marcheu@chromium.org>
/external/autotest/client/site_tests/graphics_dEQP/graphics_dEQP.py
|
798f59e7dc114377d5915785b8907518476f0761 |
|
17-Mar-2016 |
Ilja H. Friedel <ihf@chromium.org> |
graphics_dEQP: output valid filter. We don't want to output "filter = None" even when we specify tests via file as this is used in post processing expectations. BUG=None. TEST=test_that 100.96.58.232 graphics_dEQP.gles2-master 03/16 22:31:45.510 INFO | graphics_dEQP:0441| dEQP test filter = gles2-master Change-Id: Ia87ada8ea50f2197d5e10ea7945e6db96717d174 Reviewed-on: https://chromium-review.googlesource.com/333155 Reviewed-by: Ilja Friedel <ihf@chromium.org> Tested-by: Ilja Friedel <ihf@chromium.org>
/external/autotest/client/site_tests/graphics_dEQP/graphics_dEQP.py
|
029c6beee5bd404b593860396b3339061feb9a11 |
|
15-Mar-2016 |
Ilja H. Friedel <ihf@chromium.org> |
graphics_dEQP: handle pinetrail gracefully. Instead of listing anything non-gles2 in the expectations, add custom code to detect pinetrail and skip tests as needed. BUG=None. TEST=Ran locally on DUT. Change-Id: I986b1bdd922ee3f05269d351123c4549f6830506 Reviewed-on: https://chromium-review.googlesource.com/332647 Trybot-Ready: Ilja Friedel <ihf@chromium.org> Tested-by: Ilja Friedel <ihf@chromium.org> Reviewed-by: Stéphane Marchesin <marcheu@chromium.org>
/external/autotest/client/site_tests/graphics_dEQP/graphics_dEQP.py
|
61b4832727f526aeb9d90651d702204c2d572835 |
|
24-Feb-2016 |
Ilja H. Friedel <ihf@chromium.org> |
graphics_dEQP: add cts-6.0_r4 master lists. These are copied from android-cts-6.0_r4/external/deqp/android/cts/master and define the tests which need to be passed by CTS. Also refactor graphics_dEQP to execute these master lists. BUG=None. TEST=Ran on lars. Checked logs in /tmp/. test_that --args="filter=dEQP-GLES2.info" 100.96.58.232 graphics_dEQP.filter_args test_that --args="test_names=dEQP-GLES2.info.extensions,dEQP-GLES2.info.render_target,dEQP-GLES2.info.renderer" 100.96.58.232 graphics_dEQP.filter_args test_that 100.96.58.232 graphics_dEQP.gles2.functional.hasty.0 test_that 100.96.58.232 graphics_dEQP.gles2.capability test_that 100.96.58.232 graphics_dEQP.gles2-master Change-Id: I1a7c7b58d1d5427137d427c5379cb40ac4bc481a Reviewed-on: https://chromium-review.googlesource.com/329164 Reviewed-by: Ilja Friedel <ihf@chromium.org> Tested-by: Ilja Friedel <ihf@chromium.org>
/external/autotest/client/site_tests/graphics_dEQP/graphics_dEQP.py
|
b74425356a1232f57a603b8307b595845d0aefbe |
|
09-Mar-2016 |
Ilja H. Friedel <ihf@chromium.org> |
graphics_dEQP: enable pinetrail. BUG=None. TEST=sanity run on alex. Change-Id: I5d5c0cadef49a1b8b5425989b80da66ca57ddea0 Reviewed-on: https://chromium-review.googlesource.com/331764 Reviewed-by: Ilja Friedel <ihf@chromium.org> Tested-by: Ilja Friedel <ihf@chromium.org>
/external/autotest/client/site_tests/graphics_dEQP/graphics_dEQP.py
|
5f6574479479892e960f8a4332f6fae375cb66bf |
|
08-Mar-2016 |
Ilja H. Friedel <ihf@chromium.org> |
autotest: refactor ARM/mali utils. 1) Distinguish between different mali -> mali-t604. 2) Remove temperature TODO, it is long fixed. BUG=None. TEST=test_that <daisy> graphics_dEQP.gles2.performance graphics_GLBench.bvt-cq graphics_Idle Change-Id: I47568cb3369ab213592855430a647fe6366374c1 Reviewed-on: https://chromium-review.googlesource.com/331334 Commit-Ready: Ilja Friedel <ihf@chromium.org> Tested-by: Ilja Friedel <ihf@chromium.org> Reviewed-by: Stéphane Marchesin <marcheu@chromium.org>
/external/autotest/client/site_tests/graphics_dEQP/graphics_dEQP.py
|
cd1caeda9f5fbf3b06ae6b24ba5bec5ed672280a |
|
04-Mar-2016 |
Stéphane Marchesin <marcheu@chromium.org> |
deqp: Use RGBA8888 / D24S8 format by default The surfaceless deqp backend which we use in Chrome OS doesn't specify a pixel format and runs using an fbo which means that it lets deqp pick the format. The format picked for depth is the first in the list of depth formats, which is GL_DEPTH32F_STENCIL8. Sadly deqp doesn't check that this format is present (in particular, it is not a requirement for GLES2 implementations, and is only required for GLES3). This means that the driver rejects the format (rightfully so) and in turn deqp does not run on these GLES2 stacks. This patch changes the test to pass -deqp-gl-config-name=rgba8888d24s8 to deqp which forces the use of a D42S8 format, which all GLES implementations support. Note that the backends for all the other platforms (android, linux...) use this format by default already, so by changing this we are making our deqp testing more comparable with other platforms. BUG=chromium:532691 TEST=run on alex Change-Id: I26e432ec7351f636cfe16c265d13de02cd800979 Reviewed-on: https://chromium-review.googlesource.com/330464 Reviewed-by: Ilja Friedel <ihf@chromium.org> Tested-by: Ilja Friedel <ihf@chromium.org>
/external/autotest/client/site_tests/graphics_dEQP/graphics_dEQP.py
|
de00a62a7f9bf266504ca72d8a030880d6928d8f |
|
01-Mar-2016 |
Ilja H. Friedel <ihf@chromium.org> |
graphics_dEQP: run on mali. BUG=chromium:543372 TEST=None. Change-Id: I750d63def76eabda754cc810457086b2ea8299c2 Reviewed-on: https://chromium-review.googlesource.com/329823 Commit-Ready: Ilja Friedel <ihf@chromium.org> Tested-by: Ilja Friedel <ihf@chromium.org> Reviewed-by: Ilja Friedel <ihf@chromium.org>
/external/autotest/client/site_tests/graphics_dEQP/graphics_dEQP.py
|
40196cf868b79919b5d2de58771ceb9b8715adc9 |
|
15-Jan-2016 |
Mike Mason <michael.w.mason@intel.com> |
graphics_dEQP: add debug option It is sometimes useful to save additional information about what happened during a test. Some of this data is currently collected by autotest, but it is difficult to associate it with a specific test and to correlate data from multiple sources. This patch adds a debug option to collect dmesg, stdout and stderr data for each test. This data is written in JSON format to <test-name>.debug in the same directory as the individual dEQP test log files on the DUT. BUG=None TEST=test_that --args="debug=True filter=dEQP-GLES3.info" <ip> graphics_dEQP.filter_args Look for <test-name>.debug files in /tmp/dEQP-GLES3.info-logs on the DUT. Change-Id: Icd6d78ac369d288f89f9fb69dcaa60b8e3278c35 Signed-off-by: Mike Mason <michael.w.mason@intel.com> Reviewed-on: https://chromium-review.googlesource.com/322225 Commit-Ready: Ilja Friedel <ihf@chromium.org> Tested-by: Ilja Friedel <ihf@chromium.org> Reviewed-by: Ilja Friedel <ihf@chromium.org>
/external/autotest/client/site_tests/graphics_dEQP/graphics_dEQP.py
|
8b2e8bd3ecc4c8ad6aff47a2ea56029f069190b1 |
|
17-Feb-2016 |
Ilja H. Friedel <ihf@chromium.org> |
graphics_dEQP: don't warn if NotPass is empty. This warning causes too manual checking on wmatrix and has little value. BUG=None. TEST=Ran locally. Change-Id: Ibc0caafe09548de5bc0568e62e775c621d4c21bf Reviewed-on: https://chromium-review.googlesource.com/327705 Reviewed-by: Ilja Friedel <ihf@chromium.org> Tested-by: Ilja Friedel <ihf@chromium.org>
/external/autotest/client/site_tests/graphics_dEQP/graphics_dEQP.py
|
939f57efdd21c6dd47f92f6b8b6f2221c6911595 |
|
14-Jan-2016 |
Mike Mason <michael.w.mason@intel.com> |
graphics_dEQP: hasty expectation lists used in non-hasty mode The dEQP autotest mistakenly used the hasty exception lists to exclude tests even in non-hasty mode, which resulted in some tests not being run. This was caused by the wrong string being used to filter out hasty file names in non-hasty mode. This patch fixes that string. BUG=49397 TEST=test_that <ip_address> graphics_dEQP.gles3.functional This should run ~41.5K tests, but instead only runs ~22K tests. The 'dEQP-GLES3.functional.hasty.Flaky' expectation contains ~19K tests. Change-Id: Ic95aa779428df545b6ee3637a27e45f42af9e029 Signed-off-by: Mike Mason <michael.w.mason@intel.com> Reviewed-on: https://chromium-review.googlesource.com/321866 Commit-Ready: Ilja Friedel <ihf@chromium.org> Tested-by: Ilja Friedel <ihf@chromium.org> Reviewed-by: Ilja Friedel <ihf@chromium.org>
/external/autotest/client/site_tests/graphics_dEQP/graphics_dEQP.py
|
0facf6fd666d27bee7965759d71fd47d8a62d46c |
|
09-Dec-2015 |
Ilja H. Friedel <ihf@chromium.org> |
graphics_dEQP: add control files. We would like to run tests that fail on a daily schedule. Run gles3.functional in hasty/sharded mode in preparation for moving it to bvt-cq one day. All these files are now created using generate_controlfiles.py. BUG=chromium:556747 TEST=test_that <samus> graphics_dEQP.gles2.accuracy graphics_dEQP.gles3.accuracy graphics_dEQP.gles2.info graphics_dEQP.gles3.info Change-Id: I34de4f309d2cb433c1f344a3b43fadc39d32307d Reviewed-on: https://chromium-review.googlesource.com/317032 Commit-Ready: Ilja Friedel <ihf@chromium.org> Tested-by: Ilja Friedel <ihf@chromium.org> Reviewed-by: Ilja Friedel <ihf@chromium.org>
/external/autotest/client/site_tests/graphics_dEQP/graphics_dEQP.py
|
3d4a32186c4f856be868eaa815c4bfb4e2167b83 |
|
21-Oct-2015 |
Ilja H. Friedel <ihf@chromium.org> |
graphics_dEQP: warn instead of error. When we run tests that are known not to be passing they could be flaky or the failure can depend on previously ran tests. For this reason just warn if a test that we thought should be failing is now passing. It does not require immediate attention. BUG=None. TEST=None. Change-Id: I18524da875aef2fee3fe40dd49dd0997cbf4a9c9 Reviewed-on: https://chromium-review.googlesource.com/307924 Commit-Ready: Ilja Friedel <ihf@chromium.org> Tested-by: Ilja Friedel <ihf@chromium.org> Reviewed-by: Ilja Friedel <ihf@chromium.org>
/external/autotest/client/site_tests/graphics_dEQP/graphics_dEQP.py
|
20d15b2e2566840876c1a9f35051403249a98a3a |
|
15-Oct-2015 |
Ilja H. Friedel <ihf@chromium.org> |
graphics_dEQP: unsupported GPU -> TestNAError. We want to keep track of these GPUs, in particular in preparation of moving graphics_dEQP into bvt-cq. And DEPENDENCIES don't work as they should. (Might still be worth adding DEPENDENCIES in addition to avoid lab overhead though.) BUG=chromium:510208 TEST=None. Change-Id: I6ab344b17e7da103311e384c3e57b98e365f0179 Reviewed-on: https://chromium-review.googlesource.com/305715 Commit-Ready: Ilja Friedel <ihf@chromium.org> Tested-by: Ilja Friedel <ihf@chromium.org> Reviewed-by: Ilja Friedel <ihf@chromium.org>
/external/autotest/client/site_tests/graphics_dEQP/graphics_dEQP.py
|
dd11bd5cf2ae753fe9d229fc37dfb9f72135ffe8 |
|
01-Oct-2015 |
Ilja H. Friedel <ihf@chromium.org> |
graphics_dEQP: change NotPass to error on pass. And output how many tests are passing now. This should make reading the dashboard easier. BUG=chromium:510208 TEST=Ran on pixel. Change-Id: I9f7ebe9995d2a3c215a6f5083333b407de9d754e Reviewed-on: https://chromium-review.googlesource.com/303487 Commit-Ready: Ilja Friedel <ihf@chromium.org> Tested-by: Ilja Friedel <ihf@chromium.org> Reviewed-by: Haixia Shi <hshi@chromium.org>
/external/autotest/client/site_tests/graphics_dEQP/graphics_dEQP.py
|
94747e735e98a57eec2df7288170bf781983d604 |
|
01-Oct-2015 |
Ilja H. Friedel <ihf@chromium.org> |
graphics_dEQP: shard gles2.functional test. We would like to run the test in bvt-cq for which each shard needs to complete in less than one minute. For this we break it into 10 shards. BUG=chromium:510208 TEST=Ran all control files on link, checked boundaries. Change-Id: Ibed146fd9b80693b8bd982b7de3a648a62d33ebe Reviewed-on: https://chromium-review.googlesource.com/303486 Commit-Ready: Ilja Friedel <ihf@chromium.org> Tested-by: Ilja Friedel <ihf@chromium.org> Reviewed-by: Haixia Shi <hshi@chromium.org>
/external/autotest/client/site_tests/graphics_dEQP/graphics_dEQP.py
|
5d0958d60980f807a19bc8f1f6d6bef39ad3d3b3 |
|
29-Sep-2015 |
Ilja H. Friedel <ihf@chromium.org> |
graphics_dEQP: handle hasty only failures. Even when each individual functional test passes it is possible for failures to happen when batches of these tests are run in hasty mode. This prevents from dEQP moving into bvt-cq. This change adds ability to process_logs.py to parse hasty log files and generate expectations for these. It also changes graphics_dEQP.py slightly to honor these hasty failures. In addition there is some refactoring to run failing tests via 'NotPass' control files. Regularly running NotPass'ing tests allows for monitoring test fixes and updating expectations based on these. BUG=chromium:510208 TEST=Ran hasty.Fail and NotPass for testing on link. Change-Id: I12b61619f56774b8ca5ddfe67c3380bc6ba8ad9e Reviewed-on: https://chromium-review.googlesource.com/302738 Commit-Ready: Ilja Friedel <ihf@chromium.org> Tested-by: Ilja Friedel <ihf@chromium.org> Reviewed-by: Haixia Shi <hshi@chromium.org>
/external/autotest/client/site_tests/graphics_dEQP/graphics_dEQP.py
|
85262be07cbd30ea60d911bc915cff8f5f81e462 |
|
17-Sep-2015 |
Ilja H. Friedel <ihf@chromium.org> |
graphics_dEQP: update Intel whitelists. Update whitelist so we can move dEQP to bvt-cq after sharding. Also track pinetrail failure explicitly. BUG=chromium:510207, chromium:510208, chromium:532691 TEST=Ran on link. Change-Id: I54c471c791b6b087f297f9835bb6429a99c28226 Reviewed-on: https://chromium-review.googlesource.com/300280 Commit-Ready: Ilja Friedel <ihf@chromium.org> Tested-by: Ilja Friedel <ihf@chromium.org> Reviewed-by: Stéphane Marchesin <marcheu@chromium.org>
/external/autotest/client/site_tests/graphics_dEQP/graphics_dEQP.py
|
294dca534a448c835c2405b3bef98efcbef2ae21 |
|
25-Aug-2015 |
HS Liao <hs.liao@mediatek.com> |
autotest (graphics): fix dEQP log parsing Use ISO-8859-1 encoding to prevent failed XML parsing of dEQP log. BUG=chrome-os-partner:42991 test_that -b oak --results_dir=./log <IP> f:.*graphics_dEQP/control.gles2.functional.hasty Change-Id: Ie360d64dc1d638d5a6973fde12887fead959e9e2 Signed-off-by: HS Liao <hs.liao@mediatek.com> Reviewed-on: https://chromium-review.googlesource.com/295520 Commit-Ready: Ilja Friedel <ihf@chromium.org> Tested-by: Ilja Friedel <ihf@chromium.org> Reviewed-by: Ilja Friedel <ihf@chromium.org>
/external/autotest/client/site_tests/graphics_dEQP/graphics_dEQP.py
|
42d238fec9cd578bcb1039aeacd0ce6e977241e6 |
|
20-Jun-2015 |
Mike Mason <michael.w.mason@intel.com> |
graphics_dEQP: put hasty logs in same directory as non-hasty logs Now that hasty mode runs tests in batches of 100, many log files can still be generated, especially for the functional tests. This clutters up /tmp unnecessarily and makes log file management more difficult. This patch puts the hasty logs in the same directory as non-hasty logs, mitigating these issues. BUG=None TEST=test_that --args="hasty=True" <dut_addr> graphics_dEQP.gles3.accuracy The hasty logs should be in /tmp/dEQP-GLES3.accuracy-logs Change-Id: Iee79e6700b8bff852faf5a207d7ec4c664896670 Signed-off-by: Mike Mason <michael.w.mason@intel.com> Reviewed-on: https://chromium-review.googlesource.com/280417 Reviewed-by: Ilja Friedel <ihf@chromium.org> Trybot-Ready: Ilja Friedel <ihf@chromium.org> Commit-Queue: Ilja Friedel <ihf@chromium.org>
/external/autotest/client/site_tests/graphics_dEQP/graphics_dEQP.py
|
3152d7b9e9fc8f6b724bb426b8af68ba81a0077f |
|
20-Jun-2015 |
Mike Mason <michael.w.mason@intel.com> |
graphics_dEQP: pep8 and pylint cleanup Fix all pep8 and pylint warnings and errors. BUG=None TEST=None Change-Id: I83bf7f061b45ced91ae6110e01a8d863206798e2 Signed-off-by: Mike Mason <michael.w.mason@intel.com> Reviewed-on: https://chromium-review.googlesource.com/280488 Reviewed-by: Ilja Friedel <ihf@chromium.org> Commit-Queue: Ilja Friedel <ihf@chromium.org> Trybot-Ready: Ilja Friedel <ihf@chromium.org> Tested-by: Ilja Friedel <ihf@chromium.org>
/external/autotest/client/site_tests/graphics_dEQP/graphics_dEQP.py
|
46217b267744c327703be31204664268832d3e71 |
|
16-Jun-2015 |
Mike Mason <michael.w.mason@intel.com> |
graphics_dEQP: add ability to run individual tests The filter option previously allowed the running of individual tests, but no longer does. We need this capability, so this patch adds a test_names option that works with the filter_args control file. If test_names is given, the filter option and expectation lists are ignored. When test_names is not given, the filter option and expectation lists work as before. A test name can be a full name or just an initial part. If an initial part is given, all tests that start with that part will be run. The option also allows multiple test names in the form of a comma-separated list. Multiple names must be for the same API (e.g., dEQP-GLES3, dEQP-GLES2, etc.). BUG=None TEST= test_that --args="test_names=dEQP-GLES3.info.version" \ <dut_addr> graphics_dEQP.filter_args test_that --args="test_names=dEQP-GLES3.accuracy,dEQP-GLES3.stress" \ <dut_addr> graphics_dEQP.filter_args Change-Id: I215bbef8a983884481c2ca9bad1643f943dadc3e Signed-off-by: Mike Mason <michael.w.mason@intel.com> Reviewed-on: https://chromium-review.googlesource.com/278090 Reviewed-by: Ilja Friedel <ihf@chromium.org> Trybot-Ready: Frank Henigman <fjhenigman@chromium.org> Commit-Queue: Ilja Friedel <ihf@chromium.org> Trybot-Ready: Ilja Friedel <ihf@chromium.org> Tested-by: Ilja Friedel <ihf@chromium.org>
/external/autotest/client/site_tests/graphics_dEQP/graphics_dEQP.py
|
6a0b82bdd5f69d093448934ba0db14bcbfbc0e3d |
|
20-May-2015 |
Ilja H. Friedel <ihf@chromium.org> |
graphics_dEQP: ignore InternalError. This error can happen when dEQP misestimates times and they become negative. We should simply discount them. Also update blacklist/expectations. BUG=chromium:488659 TEST=None. Change-Id: Ica1ef9a12d5c268618c67173eec8315ee8d56fe6 Reviewed-on: https://chromium-review.googlesource.com/272307 Commit-Queue: Ilja Friedel <ihf@chromium.org> Trybot-Ready: Ilja Friedel <ihf@chromium.org> Tested-by: Ilja Friedel <ihf@chromium.org> Reviewed-by: Stéphane Marchesin <marcheu@chromium.org>
/external/autotest/client/site_tests/graphics_dEQP/graphics_dEQP.py
|
76f809985d8082410e695be0a64a0ab9c54e3937 |
|
15-May-2015 |
Ilja H. Friedel <ihf@chromium.org> |
graphics_dEQP: make hasty mode more robust. Instead of running all test cases at once, define a batch size of 100 tests run in groups. That way we can recover more often from test failures due to state leakage (but it isn't really a fix). Also add clean up spew to make log files a bit nicer. Add more flaky tests as harvested from wmatrix/gs. BUG=chromium:461023 TEST=test_that <link_ip> graphics_dEQP.gles2.functional.hasty Change-Id: I14717a137acd9d16d1552f732db9dbd8d78060f4 Reviewed-on: https://chromium-review.googlesource.com/271571 Trybot-Ready: Ilja Friedel <ihf@chromium.org> Tested-by: Ilja Friedel <ihf@chromium.org> Reviewed-by: Haixia Shi <hshi@chromium.org>
/external/autotest/client/site_tests/graphics_dEQP/graphics_dEQP.py
|
8af26fa68550b4a04bfe80ece52682caec08a9af |
|
01-May-2015 |
Ilja H. Friedel <ihf@chromium.org> |
deqp: rework test to use expectations. Instead of using blacklists auto generate expectations from previous runs using process_logs.py. In particular run only tests by default that have been shown to be 'Pass'ing reliably. In addition introduce a way to bootstrap incomplete expectations. This situation can happen when a GPU family is added, test are new or have been hanging leading to incomplete logs. Long term we may consider getting rid of the 'Pass'ing expectations and only use the bootstrapping. Furthermore introduce a hasty way of running tests which calls dEQP binary only once with the list of all tests on stdin. This avoid the overhead of running each test individually and is quite a bit faster. Finally add a script to process_logs.py obtained locally or stored on autotest Google Storage. BUG=chromium:461023 TEST=Ran some of the sets on link (ivybridge). Deleted Pass'ing expectations, watched bootstrapping. Change-Id: I288071d0e3a36db551c7d7981eb0a3f37dc48229 Reviewed-on: https://chromium-review.googlesource.com/268934 Trybot-Ready: Ilja Friedel <ihf@chromium.org> Tested-by: Ilja Friedel <ihf@chromium.org> Reviewed-by: Haixia Shi <hshi@chromium.org> Reviewed-by: Michael W Mason <michael.w.mason@intel.com> Tested-by: Michael W Mason <michael.w.mason@intel.com>
/external/autotest/client/site_tests/graphics_dEQP/graphics_dEQP.py
|
a8fdc75b9bdf8941b485ad8fe7fa71f9da2e89f6 |
|
25-Apr-2015 |
Ilja H. Friedel <ihf@chromium.org> |
graphics_dEQP: temporary revert blacklist. We need log files showing all failures to compile new blacklists. This change will be reverted next week. BUG=chromium:461023 TEST=None. Change-Id: I448dfe22a1f1ee31cc7d101ee296470efc1ddaa1 Reviewed-on: https://chromium-review.googlesource.com/267346 Trybot-Ready: Ilja Friedel <ihf@chromium.org> Tested-by: Ilja Friedel <ihf@chromium.org> Reviewed-by: Dominik Behr <dbehr@chromium.org>
/external/autotest/client/site_tests/graphics_dEQP/graphics_dEQP.py
|
5858f1e5fdbe558519548db3f595a1175d6c3f66 |
|
21-Apr-2015 |
Ilja H. Friedel <ihf@chromium.org> |
graphics_dEQP: minor changes. Add logging information on board/GPU/CPU to simplify postprocessing. Disable egl and gles31 tests. BUG=chromium:461023 TEST=pylint check spew of test_that 100.96.59.64 graphics_dEQP.gles2.info Change-Id: Iaefe0cf841b454a79b887ed9d6b017fb487d1f7c Reviewed-on: https://chromium-review.googlesource.com/266606 Reviewed-by: Dominik Behr <dbehr@chromium.org> Reviewed-by: Ilja Friedel <ihf@chromium.org> Tested-by: Ilja Friedel <ihf@chromium.org>
/external/autotest/client/site_tests/graphics_dEQP/graphics_dEQP.py
|
aac0527e311231a914f4ca11b55135c64a257347 |
|
07-Apr-2015 |
Ilja H. Friedel <ihf@chromium.org> |
graphics_dEQP: don't upload log files to GS. The log files contain interesting debugging output but are too many/large for permanent storage. This change moves log files to /tmp and avoids upload. Furthermore remove control.filter_args from graphics SUITE. BUG=chromium:461023, chromium:473865 TEST=ran twice test_that 100.96.59.64 graphics_dEQP.gles3.info and verified the log directory is in /tmp now. Change-Id: Ic580acdef5e9107d1407e4b6ad1c231ac0b4d517 Reviewed-on: https://chromium-review.googlesource.com/264149 Reviewed-by: Ilja Friedel <ihf@chromium.org> Tested-by: Ilja Friedel <ihf@chromium.org>
/external/autotest/client/site_tests/graphics_dEQP/graphics_dEQP.py
|
870c962b68e8babda0a03802150613af5d9bd01a |
|
09-Mar-2015 |
Mike Mason <michael.w.mason@intel.com> |
[autotest] Add autotest for DrawElements dEQP (drawElements Quality Program) is an OpenGL ES testsuite added to AOSP in the Lollipop release. It is being added to Chrome OS. This patch adds an autotest for dEQP. dEQP provides test suites for EGL, GLES2, GLES3 and GLES3.1. The suites are further broken down into info, accuracy, functional, stress and performance categories. This patch provides control files for each suite/category, although we currently only run the GLES3 tests. In addition, the filter_args control file can be used to specify tests or test sets using the --args option as shown in the TEST section below. The filter_args control file runs no tests by default. Some tests are known to be unsupported, timeout, crash the test process and/or hang the GPU or CPU on Chrome OS, sometimes leading to reboots. These tests are listed in a blacklist JSON file as regular expressions that match one or more tests. graphics_dEQP.py reads and parses the blacklist file and will not run these tests. Note that some categories can take a long time to run. For example, in test runs on auron_paine, gles3.functional took ~110 minutes and gles3.performance took ~270 minutes. By contrast, gles3.info, accuracy and stress together took ~20 minutes. Times will vary depending on board, CPU, kernel, etc. BUG=chromium:461023 TEST= emerge-link deqp; cros deploy --board=<board> <remote-addr> deqp test_that -b <board> <remote-addr> graphics_dEQP.gles3.accuracy test_that -b <board> --args="filter=dEQP-GLES3.info" <remote-addr> \ graphics_dEQP.filter_args Depends-Change-Id: Ifa07827370a140f776420808d094156809108b7e Change-Id: I09bd010f31e00f5ad939faf9224ef3243f31e7e2 Reviewed-on: https://chromium-review.googlesource.com/257651 Reviewed-by: Ilja Friedel <ihf@chromium.org> Trybot-Ready: Ilja Friedel <ihf@chromium.org> Tested-by: Michael W Mason <michael.w.mason@intel.com> Reviewed-by: Chad Versace <chad.versace@intel.com> Commit-Queue: Chad Versace <chad.versace@intel.com>
/external/autotest/client/site_tests/graphics_dEQP/graphics_dEQP.py
|