History log of /external/autotest/client/bin/parallel.py
Revision Date Author Comments (<<< Hide modified files) (Show modified files >>>)
85d2e0541b03ccd6fdd27f0378bdfc36d28fa0bf 25-Jul-2013 Dennis Jeffrey <dennisjeffrey@chromium.org> Do not raise an ImportError when attempting to translate a subprocess exception.

After running a client-side test, autotest attempts to parse and re-raise any
exceptions raised by the test subprocess. This is done by attempting to
un-pickle a text file that may contain information about the raised exception.
However, the un-pickling may fail in the event that the subprocess raises a
subprocess-specific exception that autotest proper doesn't know about
(e.g., pyauto raising a pyauto_errors.JSONInterfaceError). This causes the
immediate autotest failure to be an "ImportError" that actually masks the
root-cause failure that was originally raised by the subprocess itself.

In this CL, this situation is captured by catching the ImportError exception,
then re-raising a general error.TestError with an informative error message
indicating that the root cause exception occurred elsewhere (it also prints
out the text contents of the file that was attempted to be un-pickled, which
isn't completely human-readable, but does contain text about the root cause
exception that was originally raised. Here's an example failure message for
this case:

10:05:48 INFO | ERROR login_CryptohomeUnmounted login_CryptohomeUnmounted timestamp=1374771948 localtime=Jul 25 10:05:48 Subprocess raised an exception that could not be identified. The root cause exception is in the text that follows: cautotest_lib.client.common_lib.error
TestFailRetry
p0
(cpyauto_errors
JSONInterfaceError
p1
(S'Raised exception message here.'
p2
tp3
Rp4
tp5
Rp6
.
10:05:48 INFO | END ERROR login_CryptohomeUnmounted login_CryptohomeUnmounted timestamp=1374771948 localtime=Jul 25 10:05:48

BUG=chromium:244593
TEST=Ran login_CryptohomeUnmounted on a local device in which I forced the
underlying pyauto code to raise a pyauto_errors.JSONInterfaceError, then
verified that the example error log above was the result.

Change-Id: I3132f89dd962e2f266b65a8d96fc87ac05075649
Reviewed-on: https://gerrit.chromium.org/gerrit/63355
Reviewed-by: Prashanth Balasubramanian <beeps@chromium.org>
Commit-Queue: Dennis Jeffrey <dennisjeffrey@chromium.org>
Tested-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
/external/autotest/client/bin/parallel.py
456d3c115952bf1ae984770e226c5a50676b31c0 19-Jul-2011 Dale Curtis <dalecurtis@chromium.org> Upstream Autotest merge.

Merged from d9d64b855363d214996b187380532d4cc9991d29.

BUG=none
TEST=emerge autotest-tests, local server, run_remote_tests.

Change-Id: Id8cf1ef930bc0cd80347d77f2de65561be2a12a4
Reviewed-on: http://gerrit.chromium.org/gerrit/4664
Reviewed-by: Mike Truty <truty@chromium.org>
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Tested-by: Dale Curtis <dalecurtis@chromium.org>
/external/autotest/client/bin/parallel.py
74a314b490ff542c4dd2ae4aa0d11c6394d92960 23-Jun-2011 Dale Curtis <dalecurtis@chromium.org> Upstream Autotest merge.

As titled, a merge up to 93fc426ca133e775eb495f34d138fc57d92fb55e.

- Removes a bunch of deprecated code.
- Moves several private utilities into the private repo.
- Couple changes ported upstream and resynced.

BUG=None
TEST=In progress... will spin up new Autotest server and use
run_remote_tests for bvt, regression, smoke.

Change-Id: Id3e2ad529bb7b05f148e5d98aea46bb9ea828200
Reviewed-on: http://gerrit.chromium.org/gerrit/3350
Tested-by: Dale Curtis <dalecurtis@chromium.org>
Reviewed-by: Dale Curtis <dalecurtis@chromium.org>
/external/autotest/client/bin/parallel.py
d656d56473f50b9c1a9f5e2b2f5a9472181ee342 20-Apr-2011 Eric Li <ericli@chromium.org> Merge remote branch 'autotest-upstream/master' into autotest-merge

Merged to autotest upstream @5317.

BUG=none
TEST=
1. utils/unittest_suite.py
2. utils/packager.py --all upload -r /tmp/pkgs
3. utils/compile_gwt_clients.py

enter_chroot
emerge-x86-generic autotest-*
emerge-arm-generic autotest-*

run_remote_test suite_Smoke
run_remote_test bvt

Review URL: http://codereview.chromium.org/6883035

Change-Id: I03240d8652ffdfde4d6c8fbf474e2fad7f3cda10
/external/autotest/client/bin/parallel.py
cb2daff34efd56a0c94eb6f93b71680967cccccc 14-Jun-2010 lmr <lmr@592f7852-d20e-0410-864c-8624ca9c26a4> Clear exception information in fork_start() before exiting

If an exception is raised in a test, its traceback contains references to
objects used in the test. These references prevent garbage collection of the
objects. Normally this isn't a problem because when a program exits the
objects are garbage-collected anyway, but fork_start() calls os._exit() which
skips normal exit processing. Thus we can't rely on objects' __del__() methods
being called.

The problem can be solved by calling sys.exc_clear() to clear exception
information before exiting. It also doesn't hurt to call gc.collect() just to
be very sure that garbage collection takes place.

Signed-off-by: Michael Goldish <mgoldish@redhat.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4608 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/client/bin/parallel.py
d839e10210fc9a4c5ed7f043a825013e23ac640b 18-May-2010 lmr <lmr@592f7852-d20e-0410-864c-8624ca9c26a4> parallel.py: Fix the client test recycled pids problem

In a client control file calling a lot of times of job.run_test(), the
system's pid_max could be reached and the pids are recycled for the
tests run in child processes, causing the parent process mistakenly
determines the status of the child test.

This change fixes the problem by renaming the serialized
client/results/default/debug/error-pid file so that they are not seen by
the parent process after first use.

Signed-off-by: Jinxin Zheng <jzheng@redhat.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4501 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/client/bin/parallel.py
72817bb477fc262a86e777c4072332a96673ed1b 28-Jul-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Added support for proper subprocess killing to parallel.py, as parallel.fork_nuke_subprocess(). This involved a simple refactoring of parallel.fork_waitfor().

I also fixed a bug in parallel.fork_start() with the writing of exception pickles.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3450 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/client/bin/parallel.py
ad787c2557e90d91fd7069e8061d6e55e108ac56 06-Jul-2009 lmr <lmr@592f7852-d20e-0410-864c-8624ca9c26a4> All test failure related exception types are not supposed to print
exception traces. Recent changes on parallel.py made all exceptions
that can happen inside a test to print the exception trace, which
is very inconvenient. This patch restores previous behavior using
logging.error() instead of logging.exception() on the fork process
code.

Risk: Small
Visibility: High Users will have previous behavior back

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3377 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/client/bin/parallel.py
ed887b9372eb1bd6b061869b57426b96739507f0 22-Jun-2009 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Fix up the handling of child process exceptions to be more robust
around failure. And fix up a crappy logging.exception call.

Risk: Low
Visibility: Fix up a problem that was causing client test errors to
get dropped on the floor with no error message.

Signed-off-by: John Admanski <jadmanski@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3315 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/client/bin/parallel.py
d3e3ac94941f88ceac69834bea39c8dbe0c9d851 16-Jun-2009 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Re-write the fork handling code in parallel.py and
logging_manager.py so that in the child process we can be absolutely
certain that no matter what ends up happening, the child process will
never travel back up the call stack past the local function which
is forking.

As things are currently written exceptions being thrown at bad times
can lead to the os._exit call being bypassed. We need to use
try-except and try-finally appropriately to make this impossible.

Risk: Low
Visibility: fork() calls should no longer have edge cases where the
child fails to os._exit properly.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@3288 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/client/bin/parallel.py
c218083b6b96257788364ef8b7fa0fad5087fcbd 25-Jul-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Smarter way that tests handle default exceptions (unhandled errors) and also a cleanup on test exception classes.

From: rxaviers@br.ibm.com



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1899 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/client/bin/parallel.py
6f731364e6207c198ac8de748a043eba1e9223b3 17-Jun-2008 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Changes the semantics of UnhandledError. First, it takes in a root
exception to wrap, rather than implicitly grabbing the last caught
exception. Second, it displays the root exception message instead
of wrapping it in a new message and only displaying the root cause
in a pre-collected traceback.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1710 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/client/bin/parallel.py
0afbb6369aa5aa9a75ea67dd9e95ec4b21c0c181 06-Jun-2008 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Convert all python code to use four-space indents instead of eight-space tabs.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1658 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/client/bin/parallel.py
0e6192aa2853fb4dd98cbad705471b3fb135e70e 03-Jun-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Risk: High
Visibility: Any test run using client/bin/autotest would not go ahead.

Fix up the the core files to explicitly import common_lib/utils and bin/autotest_utils instead of using 'import *'. Prevents the showstopper where tests were not being run through client/bin/autotest

Signed-off-by: Ashwin Ganti <aganti@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1592 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/client/bin/parallel.py
0cd51493d9c31b1ed1b4b433c7e16284e57cf74e 17-Jan-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Revert broken exception handling change

Can we revert this part of the change? We were not raising a string exception
here. pickle.load() deserializes the exception raised on the client side. This
change effectively converts all exceptions thrown at the client into an
AutotestError.

For me, this breaks the case where a TestError is raised. job.run_test()
won't re-raise a TestError:

if exc_info and isinstance(exc_info[1], TestError):
return False
elif exc_info:
raise exc_info[0], exc_info[1], exc_info[2]
else:
return True

I have control files that call the same test many times with different
parameters used for each iteration. If there is a test error for one iteration,
the next test is run immediately. After this change, the entire control
file gets aborted after the first TestError gets raised. I reverted
the change in my local copy and confirm that the original behavior
returns.

Below is a patch to revert the change.

Signed-off-by: Mandeep Singh Baines <msb@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1174 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/client/bin/parallel.py
642b03e401142565a8109ac6e8495eda1dead577 14-Jan-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Now fixing the places where we're raising string exceptions on the
client code. The patch also fixes identation errors in the iostat
profiler and error.py'>error.py.

Signed-off-by: Lucas Meneghel Rodrigues <lucasmr@br.ibm.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1158 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/client/bin/parallel.py
f31b0c0060dd688c80be3bb97ffdfa193e3d924b 29-Nov-2007 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Move error.py'>error.py'>error.py out of the client and server and into common_lib.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1003 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/client/bin/parallel.py
9453ed5b27261ed817617c7da6aa52983a71c6d3 23-Nov-2006 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Move bin directory under client/



git-svn-id: http://test.kernel.org/svn/autotest/trunk@375 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/client/bin/parallel.py