39164cad07a790017c84482bb5f68768b27313cf |
|
27-Mar-2013 |
Aviv Keshet <akeshet@chromium.org> |
[autotest] only retry tests if they fail with TestFailRetry For tests that have known flaky failure modes, it is still desirable for the test to fail (without retries) when an unknown flaky failure occurs. Therefore, this CL changes the behavior of test retries, so that retires of a test are only attempted if the test fails with a TestFailRetry exception. Any other test failure will be treated as a failure regardless of retries. This means it is up to the tests in question to catch errors / exceptions caused by known flaky failure modes, and rethrow those as TestFailRetry. For the purposes of testing this change, some changes have been made to the dummyflake suite. CQ-DEPEND=Ibaa84f42beac52881cd34351e92474ef1457b15b BUG=chromium:224372 TEST=added new unit test for test.py run_remote_tests.sh --remote=<ip> suite:dummyflake; Suite runs as expected. In particular -- - client/.../dummy_Fail/control.retry_alwaysfail fails immediately even though the control file uses retries, because the failure is of type TestFail and hence not retry-able - The same applies to server/.../dummy_Fail/control.retry_failfast - client/.../dummy_Fail/control.retry_alwaysflake fails and gets retried 5 times, failing with a TestRetryFail each time such that it can be retried Change-Id: I4d1354cb410856c9de8b720c9f8310cb10d03156 Reviewed-on: https://gerrit.chromium.org/gerrit/46696 Commit-Queue: Aviv Keshet <akeshet@chromium.org> Reviewed-by: Aviv Keshet <akeshet@chromium.org> Tested-by: Aviv Keshet <akeshet@chromium.org>
/external/autotest/client/tests/flaky_test/flaky_test.py
|