History log of /external/autotest/server/autotest.py
Revision Date Author Comments (<<< Hide modified files) (Show modified files >>>)
1c3b853b2d9b80c9efcc1a27112731b96d765e6e 19-Nov-2015 J. Richard Barnette <jrbarnette@chromium.org> [autotest] Remove dead code/features from hosts factory.

The code for create_host() in server/hosts/factory.py had a large
amount of unused legacy code left behind. This change removes all
of that unused legacy.

BUG=None
TEST=run all unit tests

Change-Id: I727521aa27e82c86e04ab3d6b62cbd9e4158f00b
Reviewed-on: https://chromium-review.googlesource.com/313192
Commit-Ready: Richard Barnette <jrbarnette@chromium.org>
Tested-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
/external/autotest/server/autotest.py
a8ff78d6cc8514ad8d52274ab4b144d9e2aa3e94 18-Sep-2015 Dan Shi <dshi@chromium.org> [autotest] Collect stats for AutoservInstallError

When container has wrong privilege on ssh config files, test will fail to ssh to
the dut and lead to AutoservInstallError error.

This cl will collects some data on the error to help troubleshoot the issue.

BUG=chromium:531729
TEST=local run, unittest

Change-Id: Ide02d8fa26709bc96e4d41457f29933646fb6dc2
Reviewed-on: https://chromium-review.googlesource.com/301213
Commit-Ready: Dan Shi <dshi@chromium.org>
Tested-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Mungyung Ryu <mkryu@google.com>
/external/autotest/server/autotest.py
2de320dc5137d947ecd12859c400009a5fce2051 08-Jul-2015 Laurence Goodby <lgoodby@google.com> [autotest] makes package-based install work with dash shell

The autotest package-based install fails on DUTs running the 'dash'
shell due to a non-POSIX-compliant pathname expansion pattern.
Changes the pattern to make it POSIX-compliant.

Reference: Section 2.13.1 in
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html

http://www.gnu.org/software/bash/manual/bashref.html#index-pathname-expansion

BUG=chrome-os-partner:40702
TEST=Running autotest with this patch on moblabs.

Change-Id: Id952e19c67aaa6d74eaa09e478201693c98dc95c
Reviewed-on: https://chromium-review.googlesource.com/284360
Trybot-Ready: Laurence Goodby <lgoodby@chromium.org>
Reviewed-by: Richard Barnette <jrbarnette@chromium.org>
Commit-Queue: Laurence Goodby <lgoodby@chromium.org>
Tested-by: Laurence Goodby <lgoodby@chromium.org>
/external/autotest/server/autotest.py
1889ca1f96b253d6054fd2c0e9b31e06a34d1035 15-Apr-2015 Dan Shi <dshi@chromium.org> [autotest] Force dut to update package if test run is from test_that

test_that set use_package to False to avoid the dut to pull packages from
job_repo_url, which is set in autotest database. On the dut, autotest may not
update local package if the checksum of local cached package does not change.
The checksum is cached in [autodir]/packages.checksum.

This causes an issue for test_that. When autotest runs in dut, the test started
by test_that does not have job_repo_url, thus no new packages.checksum can be
pulled. Then the test decides that the local cache of test package's checksum
matches the existing packages.checksum, and will not pull new package from
autoserv.

This CL forces dut to use the new package from autoserv by deleting
packages.checksum file in the dut, if autotest client code is not updated using
packaging.

BUG=chromium:475261
TEST=local test_that run, make sure local test code change is used in dut.

Change-Id: Iae6bc6e2b62f2800b8f6df1561fd1f2c49047c93
Reviewed-on: https://chromium-review.googlesource.com/265833
Tested-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Puthikorn Voravootivat <puthik@chromium.org>
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
/external/autotest/server/autotest.py
b669cbdea7f4bfc9cbfcea48d9cf9ce62f0deff5 13-Sep-2013 Dan Shi <dshi@chromium.org> [autotest] Disable install autotest from job_repo_url for test_that

use_packing option for autotest._install method is True by default. When using
test_that to run test, DUT will try to install autotest package from
job_repo_url with server configured in global_config (cautotest). For DUT not
located in lab, there is no job_repo_url existed for such DUT in cautotest, and
therefore autotest will try to copy over the local files. However, when a user
try to use a DUT in the lab to run test_that, autotest package is downloaded
and installed from job_repo_url. That leads to a problem that developers can't
use test_that to verify autotest code.

The fix is to add a new option no_use_packaging to autoserv, and thread the
option to autotest._install. The option is set to False by default, and set
to True in test_that. Also, when this argument is set to True, any package
installation will use AutoservFetcher.

BUG=chromium:265673
TEST=run following command and confirm local autotest code is copied over to
the DUT: test_that -b lumpy 172.22.75.225 dummy_Pass

Change-Id: I365f5f7dc79c8e65cddaa0af77f1f1a1e4b0012a
Reviewed-on: https://chromium-review.googlesource.com/169343
Reviewed-by: Mungyung Ryu <mkryu@google.com>
Commit-Queue: Mungyung Ryu <mkryu@google.com>
Tested-by: Mungyung Ryu <mkryu@google.com>
/external/autotest/server/autotest.py
1e1c41b1b4a1b97c0b7086b8430856ed45e064d3 05-Feb-2015 Gabe Black <gabeblack@chromium.org> graphite: Separate out configuration from the statsd classes.

The new version of the statsd classes should be created using an instance of
the new Statsd class which sets up some defaults without having to specify
them over and over. This makes it essentially compatible with the existing
usage in autotest, but will allow chromite to configure things differently and
avoid having side effects from importing the module or global state.

BUG=chromium:446291
TEST=Ran unit tests, ran stats_es_functionaltest.py, ran the
stats_mock_unittest, ran a butterfly-paladin tryjob with --hwtest, testing by
fdeng.
DEPLOY=apache,scheduler,host-scheduler

Change-Id: I1071813db197c0e5e035b4d8db615030386f1c1c
Reviewed-on: https://chromium-review.googlesource.com/246428
Reviewed-by: Fang Deng <fdeng@chromium.org>
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Gabe Black <gabeblack@chromium.org>
Tested-by: Gabe Black <gabeblack@chromium.org>
/external/autotest/server/autotest.py
b9cd83c205116d37128db5599ea272fc0cf0df8b 27-Jan-2015 Fang Deng <fdeng@chromium.org> [autotest] Remove an unnecessary machine check

No need to verify machine again if the first verify passed in
autotest.py:_do_run.

TEST=Run dummy_Pass test.
BUG=None

Change-Id: I651c4784bb8b4648fb28d8778e93b912487e7a01
Reviewed-on: https://chromium-review.googlesource.com/243596
Reviewed-by: Fang Deng <fdeng@chromium.org>
Tested-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Fang Deng <fdeng@chromium.org>
/external/autotest/server/autotest.py
da8c60af1e1e3ee97170c700d0b72991687e35a2 03-Jun-2014 Michael Liang <michaelliang@chromium.org> [autotest] Migrate graphite directory to client/common_lib/cros

This change allows us to report stats in client tests.
1. Change import paths for all files that import modules from graphite
2. Clean up some unused modules
Related CL: https://chromium-review.googlesource.com/#/c/202467/
BUG=chromium:237255
TEST=Ran scheduler locally, scheduled reboot jobs, verified stats such as monitor_db_cleanup.user_cleanup._cleanup were reported on chromeos-stats.
DEPLOY = apache, scheduler, host_scheduler
Change-Id: Iebfe3b8acc1c363a0b70ea555744e85d1367cb67
Reviewed-on: https://chromium-review.googlesource.com/202727
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Michael Liang <michaelliang@chromium.org>
Tested-by: Michael Liang <michaelliang@chromium.org>
/external/autotest/server/autotest.py
14768813d90a8729c76350fb6f4861a07e4cc86b 25-Sep-2013 beeps <beeps@chromium.org> [autotest] Fix error message in autotest.py

TEST=Threw and exception.
BUG=None

Change-Id: I548d1afb936ad9f6e6e2580833c47f9dbb1f4aa6
Reviewed-on: https://chromium-review.googlesource.com/170569
Tested-by: Prashanth B <beeps@chromium.org>
Commit-Queue: Prashanth B <beeps@chromium.org>
Reviewed-by: Aviv Keshet <akeshet@chromium.org>
/external/autotest/server/autotest.py
b03ea9d5209ff5096556d6dc85b9a2cc1c38e417 16-Aug-2013 Dan Shi <dshi@chromium.org> [autotest] Force to collect /var/log if test is failed with device error.

When test running in DUT is aborted and does not get a chance to run post test
hooks, diff of /var/log can not be copied to resuts folder and autoserv will
not be able to collect any log from the DUT.

This CL saves the device error failure flag to job.failed_with_device_error.
Autoserv uses this flag to determine whether to collect crash info (through
server/control_segments/crashinfo), which will collect all files in /var/log.

BUG=chromium:271703
TEST=run autoserv in local setup. Manually reboot DUT during the middle of a
test, then confirm the results collected has content in crashinfo.[DUT name].

Change-Id: I1a3757b8933fe60deea75728e867033eeb86c7cd
Reviewed-on: https://gerrit.chromium.org/gerrit/66013
Commit-Queue: Dan Shi <dshi@chromium.org>
Reviewed-by: Dan Shi <dshi@chromium.org>
Tested-by: Dan Shi <dshi@chromium.org>
/external/autotest/server/autotest.py
209eefd78dbb3eb8f8575aa0e0f088fa0452abc0 30-May-2013 Dan Shi <dshi@chromium.org> [Autotest] Measure how long it takes to collect test results

Add stats to measure how long it takes to collect test results from DUT.

BUG=chromium:217979
TEST=tested with autoserv in local machine
DEPLOY=none

Change-Id: Id4b5a6b59cb80903968ad64e0420aba9077f596d
Reviewed-on: https://gerrit.chromium.org/gerrit/57135
Tested-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Alex Miller <milleral@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
/external/autotest/server/autotest.py
b0be88bc1176099c85abe4a5999e55fd7ca8dc1c 18-Apr-2013 Dennis Jeffrey <dennisjeffrey@chromium.org> Autotest job tags in server-side suites are passed to underlying client jobs.

This change reverts one part of:

https://gerrit.chromium.org/gerrit/#/c/41318/4

Previously, when a server-side suite invoked a client-side test with
a tag, that tag would be passed to the client job and appended to the
name of the client-side test. For example, control.power_daily
invokes power_LoadTest multiple times with different tags. This
would lead to tests with different names, such as "power_LoadTest.WIFI_1hr"
and "power_LoadTest.WIRED_1hr".

After the above CL was submitted, the tags were instead passed to the
server job, not to the client jobs. This resulted in the tags _not_
being appended to the client-side test names, meaning that we could
no longer use the test names (with tags) to distinguish test results
from different invocations of the same client-side test.

In the current change, that tags are instead passed to the underlying
client jobs as before, so the tag can be appended to the names of the
invoked client-side tests.

BUG=chromium:227172
TEST=Modified control.power_daily to invoke the "sleeptest"
client-side test with different tags, and verified that the resulting
test names _do_ include the tag with this change, but don't include
the tags without this change.

Change-Id: I44581ea84e990995aa6323e9cb517419b5e3c70e
Reviewed-on: https://gerrit.chromium.org/gerrit/48563
Reviewed-by: Andrew Bresticker <abrestic@chromium.org>
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Commit-Queue: Dennis Jeffrey <dennisjeffrey@chromium.org>
Tested-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
/external/autotest/server/autotest.py
607428b251f437f924932a7b5386cbc06d63b22b 26-Mar-2013 beeps <beeps@chromium.org> [Autotest] Unexpeted reboots should not lead to unexpected ABORTS.

If the test failed inconclusively we should try and diagnose the
DUT. If we successfully diagnose it to either be un-pingable,
un-sshable or to have rebooted unexpectedly we should fail. Only
if we are unable to diagnose it (i.e everything seems normal but
the test still failed) do we end with an unexpected abort.

TEST=Induced crashes in power resume and made sure we diagnose
them appropriately. Also made sure we END FAIL in these situations.
BUG=chromium:188217

Change-Id: I0ff60248abe0f1d0b2b0c24a268748b51c4318ae
Reviewed-on: https://gerrit.chromium.org/gerrit/46608
Commit-Queue: Prashanth Balasubramanian <beeps@chromium.org>
Reviewed-by: Prashanth Balasubramanian <beeps@chromium.org>
Tested-by: Prashanth Balasubramanian <beeps@chromium.org>
/external/autotest/server/autotest.py
bca10a6fc4527d43199f4ad7f489127662fe6c29 25-Jan-2013 Simran Basi <sbasi@chromium.org> Autotest: Add additional logging to autotestd_monitor

In order to help determine what is happening on autotestd_monitor,
we have added additional logging so that we can tell where it gets
stuck in the situations where the client-side test never exits.

BUG=chromium-os:37608
TEST=ensured the log output is recorder and did not impact status or
status.log

Change-Id: I4ee96831ab458a9e4d4400e88b6cb2daf46cc10d
Reviewed-on: https://gerrit.chromium.org/gerrit/41983
Reviewed-by: Simran Basi <sbasi@chromium.org>
Tested-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Simran Basi <sbasi@chromium.org>
/external/autotest/server/autotest.py
2da1b76ed7190a8e6c8791997624d6d07b89f650 16-Jan-2013 Andrew Bresticker <abrestic@chromium.org> Pass additional arguments from run_test() to run().

Modify BaseAutotest.run_test() to take tag, parallel_flag, background, and
client_disconnect_timeout and pass them through to BaseAutotest.run().

BUG=chromium-os:36015
TEST=run_remote_test.sh ... platform_LongPressPower

Change-Id: I2a1d3cf92322353495acad20f3418d2d1b6538d3
Reviewed-on: https://gerrit.chromium.org/gerrit/41318
Reviewed-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Commit-Queue: Andrew Bresticker <abrestic@chromium.org>
Tested-by: Andrew Bresticker <abrestic@chromium.org>
/external/autotest/server/autotest.py
07f53b97c4ca29c9613c5eff2948e10e520f37bd 08-Jan-2013 beeps <beeps@chromium.org> Check if the DUT rebooted before ABORTING.

When the DUT reboots after a suspend/resume, processes that have
been scheduled to run, exit, and we loose our ssh connection.
These events happen simultaneously, but autotest, only noticing the
exit code and a malformed last line, throws an unexpected ABORT.
This change attempts to diagnose the failure by first pinging the DUT,
then querying it for a boot_id over ssh before ABORTING.

Note this patch does not change the saved boot_id when the test
requests a reboot. Therefore we can only tell if a reboot happened,
not if it is unexpected.

TEST=Triggered the unexpected ABORT by restarting during suspend/resume.
BUG=chromium-os:37596

Change-Id: I8390f20e047034ccb9585591ccd16cf71bfd7050
Reviewed-on: https://gerrit.chromium.org/gerrit/40628
Commit-Queue: Prashanth Balasubramanian <beeps@chromium.org>
Reviewed-by: Prashanth Balasubramanian <beeps@chromium.org>
Tested-by: Prashanth Balasubramanian <beeps@chromium.org>
/external/autotest/server/autotest.py
4c3f4d3f0ace81f32abe4e4e91b69a05893fd85f 22-Oct-2012 Alex Miller <milleral@chromium.org> Revert "[autotest] Temporarily log last_line."

This reverts commit 40326d38aee2a870c259e97a387590344cd91d3b

This logging is no longer needed/useful.

Change-Id: Ied217dc8f1da171bdd85e5924bec92ff93c9629e
Reviewed-on: https://gerrit.chromium.org/gerrit/36225
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Commit-Ready: Alex Miller <milleral@chromium.org>
Tested-by: Alex Miller <milleral@chromium.org>
/external/autotest/server/autotest.py
814d8b94f436ddc79c64b8567ab56fb31a0c360a 22-Oct-2012 Alex Miller <milleral@chromium.org> Revert "[autotest] Temporarily add even more last_line logging!"

This reverts commit c792533d3e74b4f3b9fedc223587548360cf08f9

This logging is no longer useful.

Change-Id: Iece7c0155fabd832d9ceaa49cea236aacea9868c
Reviewed-on: https://gerrit.chromium.org/gerrit/36224
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Commit-Ready: Alex Miller <milleral@chromium.org>
Tested-by: Alex Miller <milleral@chromium.org>
/external/autotest/server/autotest.py
c792533d3e74b4f3b9fedc223587548360cf08f9 12-Oct-2012 Alex Miller <milleral@chromium.org> [autotest] Temporarily add even more last_line logging!

The abort is happening because this variable doesn't contain what it
should, but everything else looks fine...

TEST=local run_suite
BUG=chromium-os:34788

Change-Id: I70875a9018e96ab23c7c252a9990af2b55d544bd
Reviewed-on: https://gerrit.chromium.org/gerrit/35360
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Commit-Ready: Alex Miller <milleral@chromium.org>
Tested-by: Alex Miller <milleral@chromium.org>
/external/autotest/server/autotest.py
40326d38aee2a870c259e97a387590344cd91d3b 12-Oct-2012 Alex Miller <milleral@chromium.org> [autotest] Temporarily log last_line.

One determines the state of a client by parsing the last line of output
we get from them. It would be useful to see what this value is to
understand what autotest thinks the state of the client is.

TEST=pylint
BUG=chromium-os:34788

Change-Id: I8932c2dd5b838609ca643598914a5f7899851321
Reviewed-on: https://gerrit.chromium.org/gerrit/35348
Commit-Ready: Alex Miller <milleral@chromium.org>
Reviewed-by: Alex Miller <milleral@chromium.org>
Tested-by: Alex Miller <milleral@chromium.org>
/external/autotest/server/autotest.py
3ee5d5c574d4e6d397aff36136101c657664c26b 23-Feb-2012 Chris Sosa <sosa@chromium.org> Allow use of local packages for developers to use when running tests.

The culprit behind the checksum logic not being run is client_logger in
autotest. I've changed client_logger such that it can be Site'ized
and added the necessary implementation in site_autotest to get this
working correctly for us.

The key issues were
1) client_logger ignored all requests for checksum files.
2) client_logger assumed you always wanted to re-tar code through
the autotest workflow.

In addition, we should exclude packages from being wiped on every
installed hence the additional excluded dirs.

BUG=chromium-os:26561
TEST=Ran it, saved 30 seconds of 70 second tests. Did not re-install
unnecessarily.

Change-Id: I7ac0a878f19296472f3b79e8cbe4cee641177215
Reviewed-on: https://gerrit.chromium.org/gerrit/17572
Reviewed-by: Chris Sosa <sosa@chromium.org>
Tested-by: Chris Sosa <sosa@chromium.org>
Commit-Ready: Chris Sosa <sosa@chromium.org>
/external/autotest/server/autotest.py
386eea7066cbd9a35599e76eee56b2c98ecec9a1 22-Sep-2011 Dale Curtis <dalecurtis@chromium.org> Fix packaging issues and modification of archived controls files.

This patch fixes a couple issues:
1. N-1->N autoupdate tests were using the test packages from N-1 instead
of N.
2. Autotest was not making a copy of the local control file before
adding the repository URLs to it. Causing the local control file to
contain every repository link for every test case run! Will upstream.
3. Site packaging function was needlessly complex, replaced with a
simpler version. Will remove fetch_location from internal config.

BUG=none
TEST=Ran run_remote_tests / tested in production.

Change-Id: I6e274c000d0f880b38023c07118f957d913fbf4e
Reviewed-on: http://gerrit.chromium.org/gerrit/8107
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Tested-by: Dale Curtis <dalecurtis@chromium.org>
Commit-Ready: Dale Curtis <dalecurtis@chromium.org>
/external/autotest/server/autotest.py
74a314b490ff542c4dd2ae4aa0d11c6394d92960 23-Jun-2011 Dale Curtis <dalecurtis@chromium.org> Upstream Autotest merge.

As titled, a merge up to 93fc426ca133e775eb495f34d138fc57d92fb55e.

- Removes a bunch of deprecated code.
- Moves several private utilities into the private repo.
- Couple changes ported upstream and resynced.

BUG=None
TEST=In progress... will spin up new Autotest server and use
run_remote_tests for bvt, regression, smoke.

Change-Id: Id3e2ad529bb7b05f148e5d98aea46bb9ea828200
Reviewed-on: http://gerrit.chromium.org/gerrit/3350
Tested-by: Dale Curtis <dalecurtis@chromium.org>
Reviewed-by: Dale Curtis <dalecurtis@chromium.org>
/external/autotest/server/autotest.py
cb7bfaf79c567031a93da7390fed5d57b4c7da85 08-Jun-2011 Dale Curtis <dalecurtis@chromium.org> Reduce timeout conditions for Chrome OS for efficiency.

Many of the Autotest timeouts are far too long for Chrome OS. Where a
normal machine might take 10+ minutes to recover, Chromebooks take only
seconds. As such, I've tweaked many of the boot and crash timeouts to
reflect this.

When a host goes down the scheduler idles the job for four hours while
waiting for crash collection, causing two main problems:

1. Metahost jobs take four hours to requeue upon verify failure.
2. Results are delayed four hours when a host dies in jobs.

Additional issues fixed:

1. Timeout after reboot was essentially doubled in the default Host
implementation of wait_for_reboot. Works for us, but will need to query
upstream to ensure it works for everyone.
2. Unified timeouts across autotest.py and base_classes.py so boot
timeouts exist in only one place. Should be okay since all hosts extend
from Host in base_classes.py. Timings are now configurable in
global_config.
3. Global SSH options to ensure better timings. In order to do this, it
was necessary to add site extensibility to abstract_ssh.

Changes to crashcollect.py, autotest.py, abstract_ssh.py, and
base_classes.py will be upstreamed.

BUG=none
TEST=run_remote_tests. Will test in production as well.

Change-Id: I92964b140337e4ab919288b5190bb2e6d5a4fa2d
Reviewed-on: http://gerrit.chromium.org/gerrit/2269
Reviewed-by: Paul Pendlebury <pauldean@chromium.org>
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Tested-by: Dale Curtis <dalecurtis@chromium.org>
/external/autotest/server/autotest.py
d656d56473f50b9c1a9f5e2b2f5a9472181ee342 20-Apr-2011 Eric Li <ericli@chromium.org> Merge remote branch 'autotest-upstream/master' into autotest-merge

Merged to autotest upstream @5317.

BUG=none
TEST=
1. utils/unittest_suite.py
2. utils/packager.py --all upload -r /tmp/pkgs
3. utils/compile_gwt_clients.py

enter_chroot
emerge-x86-generic autotest-*
emerge-arm-generic autotest-*

run_remote_test suite_Smoke
run_remote_test bvt

Review URL: http://codereview.chromium.org/6883035

Change-Id: I03240d8652ffdfde4d6c8fbf474e2fad7f3cda10
/external/autotest/server/autotest.py
861b2d54aec24228cdb3895dbc40062cb40cb2ad 04-Feb-2011 Eric Li <ericli@chromium.org> Merge remote branch 'cros/upstream' into master

Merged to upstream autotest @4749~@5215.

The entire change list description is too big to enlist here. Please refer to upstream (http://autotest.kernel.org/browser) for more details.

BUG=
TEST=emerged both x86 and arm build.
Tested emerged x86 build bvt against a chromeos device.

Review URL: http://codereview.chromium.org/6246035

Change-Id: I8455f2135c87c321c6efc232e2869dc8f675395e
/external/autotest/server/autotest.py
bdaab795cffa33f9a37995bb283047cb03699272 07-Jan-2011 Kenneth Waters <kwaters@chromium.org> Revert "Merge remote branch 'cros/upstream' into autotest-rebase"

This reverts commit 7edb30498d75a29a3287fe07070f2b51a116c5d4.

TEST=Built for tegra2_seaboard
BUG=10756

Review URL: http://codereview.chromium.org/6124004

Change-Id: Ifcd1a2279f5a62b99552a8f961c1430889a3a928
/external/autotest/server/autotest.py
7edb30498d75a29a3287fe07070f2b51a116c5d4 07-Jan-2011 Eric Li <ericli@chromium.org> Merge remote branch 'cros/upstream' into autotest-rebase

Merged to upstream trunk@5066, from trunk@4749.

There is no way I could enlist each individual CL from the upstream here since it will blow up the changelist description field.

BUG=
TEST=
Had patched this CL into a fresh cut client to avoid any side effect.
run_remote_test bvt from both emerged location and third_party/autotest/file.

Both test passed!

We should also keep any eye on this to see how it gets propagated into cautotest server.
TBR=dalecurtis

Change-Id: I72f2bc7a9de530178484aea1bfb5ace68bcad029
/external/autotest/server/autotest.py
9285ddf562c698fda569390be6763170215663d7 05-Jan-2011 Dale Curtis <dalecurtis@chromium.org> Rollback ignore_aborts change.

We're no longer seeing any unexpected aborts and have not for a couple months. In order to track better with upstream, this change rolls back the work arounds we introduced for the issue.

Change-Id: I078385946a0c44d9bf90b0a7cf46c30dc765f144

BUG=chromium-os:7931
TEST=

Review URL: http://codereview.chromium.org/6020015
/external/autotest/server/autotest.py
f44ff961880151407c051a6b657ebdc9019d6945 05-Jan-2011 J. Richard Barnette <jrbarnette@chromium.org> Remove code that added bind mounts to drop the 'noexec' mount option.

At run time, autotest requires /home and /tmp mounted 'exec' to
allow execution of autotest scripts.

The bind mounts appear to be causing the file system to be left
needing journal replay in the next mount cycle. This causes
notable skewing of the boot time test results because of the
extra time required to mount the stateful partition.

Furthermore, the bind mounts appear to be unnecessary: On test
images, chromeos_startup remounts the same target directories to
drop 'noexec' during startup.

Change-Id: Ic7ae3fae28a47dae9df510e939305b8db5ed508f

BUG=chromium-os:10392
TEST=platform_BootPerfServer/control and suites/control.bvt

Review URL: http://codereview.chromium.org/5992008
/external/autotest/server/autotest.py
ac52f54319fafe29386326b4f0a4b644e564de8d 17-Dec-2010 Dale Curtis <dalecurtis@chromium.org> After reboot, Autotest wasn't remounting the exec capable version; causing a second copy of Autotest to be installed to the mount point.

Change-Id: I9af4d361fd49194706fa916f0ac97eb6686b9f32

BUG=chromium-os:10445
TEST=Running in production. Ran unit-tests.

Review URL: http://codereview.chromium.org/5867003
/external/autotest/server/autotest.py
0b6843a4482ed69c8fa965612c097711a00eca4a 16-Nov-2010 Dale Curtis <dalecurtis@chromium.org> Fix server side autotests installing to the wrong directory.

I've added two new constants to global_config.ini: chromeos_client_autodir (default: /mnt/stateful_partition/autotest) and chromeos_client_mount (default: /home/autotest_mount). Autotest will now always create these directories and then mount chromeos_client_autodir at chromeos_client_mount as executable.

A previous change did this for all client tests, but server side tests were over looked. Original CL is at http://codereview.chromium.org/3763007/show . I've merely relocated the code into autotest.py and updated unit tests and surrounding code.

Since the original variable for client installation, client_autodir_paths, has no meaning for us in Chrome OS. I've removed it and ensured there are no dependencies.

Some tests will need modification to work properly as they improperly hard coded the path to the autotest directory instead of querying the autodir variable.

Change-Id: Ic8f9171d14a754d86f38ec882006a4217eab8ec3

BUG=8336
TEST=Ran unit tests. Tested in production; ran bvt, works fine.

Review URL: http://codereview.chromium.org/4176007
/external/autotest/server/autotest.py
f9d7d8d00c06566ff75b513061be2dcb15b2257a 20-Oct-2010 Dale Curtis <dalecurtis@chromium.org> We're encountering a significant number of unexpected aborts in the pre/post-test and pre/post-iteration sysinfo collection. After a week of debugging I'm no closer to determining the underlying cause.

In the mean time, I've prepared this CL to prevent Autotest from recording a job as aborted if the abort happens in the pre/post-test or pre/post-iteration sysinfo collection phase.

Change-Id: I2082415ce85e73b469934f7ac39035ed490e98d7

BUG=7931
TEST=Running in production. Observe bug does not trigger abort.

Review URL: http://codereview.chromium.org/3919001
/external/autotest/server/autotest.py
6f27d4f22a1ba5063968b8c322fa0845f3279ade 29-Sep-2010 Eric Li <ericli@chromium.org> Merge remote branch 'cros/upstream' into tempbranch3

Merge to trunk@4817

BUG=
TEST=

Review URL: http://codereview.chromium.org/3554003

Change-Id: I83376bc7d28104ec2678e157eadbe7df7c05c0e0
/external/autotest/server/autotest.py
517d95a1ef4edb04da427763f86068a447d45ec7 29-Sep-2010 Benson Leung <bleung@chromium.org> Revert "Merge remote branch 'cros/upstream' into tempbranch2"

This reverts commit 25fc6d1f28e54c46689f12d3b93c2540ef45323a.

TBR=ericli@chromium.org

Review URL: http://codereview.chromium.org/3541002

Change-Id: Ib0165b19bfdf02264f8a6a74ddf3ae74c8c0f7df
/external/autotest/server/autotest.py
25fc6d1f28e54c46689f12d3b93c2540ef45323a 29-Sep-2010 Eric Li <ericli@chromium.org> Merge remote branch 'cros/upstream' into tempbranch2

Merged to trunk@4816.

BUG=
TEST=we will build a new autotest server instance, and keep cautotest running and then later do a cname switch.

Review URL: http://codereview.chromium.org/3511003

Change-Id: Iee5f52f45f28f84927d6c6f9a74edc370d40288a
/external/autotest/server/autotest.py
69bdaacdf2b8161ae95dded60ed6eedbeb5f8f47 28-Jul-2010 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Makes some changes to the partitioning code and the autotest.py code to make it a little easier to override more reliably. Specifically:
- Switch the partition imports from execfile back to an import * style of loading, ultimately import seems to be the most reliable and portable
- Clean up the base+site definitions in autotest.py to make Site overrides more consistent
- Drop the explicit failure if the Autotest.source_material is a file and not a directory, there's no reason to outright fail and it just introduces some unnecessary fragility

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@4736 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
c6136e92bb5aef1acda979fd303fc855cf704434 19-Jul-2010 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Add in a run_test mixin for host objects that allows you run a client
test with just host.run_test, instead of having to setup an autotest
client directly and generate a control file.

Note that since server-client job state is managed automatically, this also
supports profilers out of the box. If you do a job.profilers.add before calling
host.run_test then the client test will automatically be run with the
appropriate profilers.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@4725 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
2ab0d9ba58896a659da61b3c9242267bb3babea5 25-Jun-2010 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Change autotest to not log Autotest.install

Signed-off-by: Darren Kuo <darrenkuo@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4681 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
52053636199c9dca7cd7a4df4d8913c866dff507 11-Jun-2010 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Restores proper support for handling unexpected client terminations
after the server_job.record refactoring.

The original record implementation just dumped whatever status logs
the client sent it into the status.log, so if the client died on you
there was never a problem with the client leaving the server-side
logging in a funky state. Now that client job records are being
properly routed though the server-side job.record a client death can
leave the server logging in a broken state.

This implementation adds a special get_record_context method which can
be used by the autotest.py code to retrieve the current job.record
context at the start of the client job, and then restore it at the end
so that the server side logging is put back into a known-good state.

It also removes a bunch of self._record_prefix uses in the server_job
code which were rendered useless by the job.record changes, but
weren't removed.

Signed-off-by: John Admanski <jadmanski@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4606 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
2a89dac0b6e319ef58d41c7a591c3d88cf6dd8a1 11-Jun-2010 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Refactor the server_job code to use the base_job implementation of
record.

The two main features that the server job code added on top of the
basic job record were the ability to inject client job records into
the server job record, and the ability to inject console/vlm warnings
into the record.

The Autotest injection was implemented by adding a parse classmethod
to the status_log_entry class that does the inverse of the render
method, so that the Autotest class can construct status_log_entries
from the raw text status logs it gets from the client. The server then
re-records the messages, rather than using a low-level method to
bypass the old job.record rendering.

The warning injection was implemented by using the record_hook; to
make it work better it was moved to be called before the entry is
logged, not after. This required a slightly more involved hook, as it
was useful to be able to call job.record directly from the hook and so
the hook was wrapped in a callable class that prevents recursion.

Signed-off-by: John Admanski <jadmanski@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4605 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
b1de54924c728f321fb966eee2555f004d95e394 11-Jun-2010 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Add the hostname to the 'unexpected final status message from client'
ABORT error.

Signed-off-by: Gregory Smith <gps@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4600 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
808f4b1663600ba7e735c4d78125825c52bb804f 10-Apr-2010 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Expose args as a proper job property, fix up some unit tests. In the
future I'm just going to start reverting patches that break tests.

Signed-off-by: John Admanski <jadmanski@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4396 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
1f572e5efbaaa8bc0d1246123ab897607c9a011b 01-Apr-2010 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> enable server side test prebuild from autoserv

From: ericli@google.com



git-svn-id: http://test.kernel.org/svn/autotest/trunk@4371 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
2f076833aba3e9cc6ddc8a7e1b02b45005770dee 30-Mar-2010 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Propagate user args to the client-side control file

From: petkov@chromium.org



git-svn-id: http://test.kernel.org/svn/autotest/trunk@4355 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
cb777f1090d2fcdef8065e04e3b1d4a95fe32e93 23-Mar-2010 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Fix a bug in the warning processor where we're supposed to be processing only the "old" warnings but are incorrectly processing all of them.

Signed-off-by: John Admanski <jadmanski@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4324 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
4d03cf61d998e13d4a19a37ad92e4f0f39998322 04-Mar-2010 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Add support to the client logging config for optionally reading a log file
prefix from the global config, and change the autotest.py execution code to
make use of this to produce client.0.*, client.1.*, etc log files instead of
just capturing one set of client.* logs.

Also modifies the server-side logging to capture client.N.log files instead
of client.log.N files (to be more in line with the logs from the client) and
retains the file only if the equivalent client.N.DEBUG file cannot be captured
(e.g. in the case of a complete client crash).

Signed-off-by: John Admanski <jadmanski@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4285 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
ddc9a40e2f313399c2d3eca1125e39e4bf65882f 15-Jan-2010 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Add a little check to see if packaging is actually being used. If it is don't pass on exceptions raised when trying to setup packaging for the job instead log them as an error.

Signed-off-by: Scott Zawalski <scottz@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4132 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
c035491ba24efea9e4343982f0a7c4b92e0a8c72 12-Jan-2010 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Adds support to wait_down and wait_for_restart to watch for changed
boot_id values (using /proc/sys/kernel/random/boot_id). This avoids a
race condition where if a machine was able to successfully shutdown
and boot up again after you started a reboot and before we started
polling for the shutdown, it would just appear that the machine
failed to shutdown when told.

As a result the Host.wait_down method now treats the case where a
machine is NOT down but has a new boot_id as being "down", because
this implies that the machine did shut down (and subsequently came
back up). This does mean that you cannot assume that a successful
wait_down implies that the machine is now down, but that was never
true anyway.

Risk: High
Visibility: We can now reliably detect the restart of machines with
very fast restart times.

Signed-off-by: John Admanski <jadmanski@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4096 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
9de6ed7ab19c29b5072d52439f434453df16bec0 11-Jan-2010 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Eliminate the last uses of the public state managment interface for
storing private job state. This does it for the last bits of
client-only state.

It also changes job.record to use a _record_prefix value, instead of
a group level value, for tracking the "current" indentation of status
logging. It's only a minimal change to record, I have more significant
refactoring in mind for getting rid of the massive duplication of
the job.record infrastructure, I just targetted this specific aspect
to get rid of the last of the "old" state management.

Note that unlike the other stateful properties, job._record_prefix is
NOT a single, global value automatically shared between the server
and client job; the client puts the stateful property in the client
namespace to keep its value independent of the server one.

Risk: Medium
Visibility: Clears out the last of the "old-style" use of storing
private job state in the public state area.

Signed-off-by: John Admanski <jadmanski@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4086 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
fc3da5bbdcd63840cd224282812c050c5520db11 06-Jan-2010 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> More refactoring of the job state code to try and reduce duplication.
In the process this also refactors a bunch of job.run_test duplication,
specifically the handling of job tags (there's still lots more
duplicate code left).

The main state refactorings are:
* Storing client job steps in __steps and special-casing its removal
in autotest.py is gone. Instead the client job code stores it
directly in the client.steps state, and the autotest.py generically
removes the entire client.* namespace.
* Moving this out of the publicly-visible state and into a private
namespace allows us to drop has_state and discard_state, which
were really only needed so that we could make sure __steps was
not persisted across jobs.
* All of this client-job state managment was moved out of autotest.py
and into server_job.py. All the data this code was getting and
setting was in the job object, so it seemed like a better place
to put it. This also let us drop load_state and save_state, which
were really only there to make it easier to push and pull data
out of the server job object.
* Eliminate get_run_number and set_run_number in favour of a generic
use_sequence_number boolean property (that works on both client
and server automatically). The get_ and set_ code was only being
used to enable and disable an auto-incrementing sequence number to
be tagged to each test run, so we can just make the public
interface a boolean and handle the sequencing under the covers.
* Drop all the test_tag and test_tag_prefix code in favour of a
persistent automatic_test_tag property. Also drops the
show_kernel_in_test_tag flag in favour of just implementing this
in the control file generator using automatic_test_tag.
* Refactor all the test name tagging code into base_job, unifying
the tagging portion of the job.run_test code.

Signed-off-by: John Admanski <jadmanski@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4075 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
060c471022558cd91a03b6e8da5a98ce35cd4b54 29-Dec-2009 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Makes two major changes to the status log processing in autotest.py:
1) Adjusts the processing of warnings and status lines to remove
entries from their buffers as the values are written out, rather
than removing them all after processing an entire batch.
2) Changes the error handling during status log processing to
log-and-ignore exceptions that occur while talking to the remote
machine, rather than propagating exceptions.

These changes are needed to eliminate runaway failure logging that can
occur when an exception occurs during the remote status log processing.
If such a failure occurs and persists, then you can get into a loop
where a client-disconnect WARN message is logged, and then another error
will happen during the logging leading to another WARN being logged, and
so on as long as the failure keeps occuring (which can be forever in
some cases). The buffering fixed by 1) makes it even worse, since if an
error occurs the next logging pass will re-log everything in the buffer
which never gets cleared due to the error (so you get 1 WARN, then 2,
then 3, then 4, ...)

Risk: High
Visibility: Fixed the error handling of the status log processing in
autotest.py to be much more robust in the face of remote errors.

Signed-off-by: John Admanski <jadmanski@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4063 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
fbf73aecdd357094ae05e7d1e4ea99b1ecf93ee4 19-Dec-2009 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Refactor all of the persistent state managing code in the client job
class into the common_lib base job class. The major change is that all
the details of the job state implementation are pulled into a class
base_job.job_state. This class handles all the internal details of the
state managment that job.get_state and job.set_state did, and also
provides a mechanism for creating job properties which are automatically
backed by job state.

This also includes a major change to the semantics of state passed
between autoserv and any Autotest clients it ran. Previously the code
was a bit of a hack, with code in server/autotest.py that would
pre-generate a state file with some pre-set values, and code that would
pull back the state file after the client job and extract some values
from it, as well as some more code that would prepend a bunch of
state initialization calls at the start of the client control file.
Instead of all these hacks, autotest.py now just pushes all of the
state out to the client at the start, and pulls it all back at the end
(with a special exception for __steps that's difficult to eliminate at
this time). So any state set on the server is automatically available
on the client, and any state set on the client is automatically
available on the server.

There are a couple of outstanding issues:
* More of the set_state/get_state code needs to be refactored
to either use the automatic properties, or the job-internal state
API (so that all the internal state is not leaked into the "public"
state visible through get_state and set_state).
* The job_state class is perhaps somewhat overengineered. It's really
just a big, automatically persistect dict-of-dicts. But most of it
is actually code that was being used before, it was just scattered
all over server/autotest.py and client/bin/job.py so it was less
obvious how overgrownt he state managment is.

Risk: High
Visibility: Replaces a bunch of get_*+set_*+default_* methods with
persistent properties, refactors all of the state managment
into a single implementation consistently used in both the
server and client.

Signed-off-by: John Admanski <jadmanski@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4022 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
9dcf083957dff6257b7cbaa23db253b65bd92a25 08-Dec-2009 lmr <lmr@592f7852-d20e-0410-864c-8624ca9c26a4> server/autotest.py: Fix of an undefined variable bug

In server/autotest.py, there's an undefined variable
autodir under the the function get_install_dir.

This patch fixes the issue

Signed-off-by: Steve Howard <showard@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@4004 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
d6d27ed06b7e264b8c99862414b13a443197d64c 08-Dec-2009 lmr <lmr@592f7852-d20e-0410-864c-8624ca9c26a4> Fix a bug in autotest.py that directly raise a string for error

As suggested by John, use existing exception module AutoservInstallError
to raise the error message.

Signed-off-by: Yolkfull Chow <yzhou@redhat.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4003 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
6d08b3c3fc2858ccd60a323008703179f93ae38b 18-Nov-2009 lmr <lmr@592f7852-d20e-0410-864c-8624ca9c26a4> Make a standalone client to be able to use global_config.ini

In order to make it possible the autotest client to use the
global_config.ini configuration files:

* Modified global_config.py to support verifying if the
configuration file is under autotest's root directory,
or the client directory
* Extended the autotest run method to copy over
the configuration files to the client.

Notes for revisors (changes from previous patchset and other stuff):

* Latest comments from John adressed

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3955 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
0d0f67d6abcf6599ad626042d3236020e093558f 06-Nov-2009 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Create a base_job class in the common library as a location to
refactor common code from client/bin/job.py and server/server_job.py
into.

Initially this just creates the base class and attempts to refactor
some of the __init__ code into it. It also tries to refactor the
code somewhat so that it's easier to test without mocking out large
amounts of external dependencies.

It also tries to standardize on what all the official publicly usable
attributes are, mostly on the basis of looking at what the existing
code actually does. In general my approach was:
* If used outside of the job class, assume it's public. If it makes
sense make it a requirement of the base class, but in some cases
(e.g. serverdir) it really only makes sense for one of the
concrete classes. In that case I define it as "OPTIONAL", which
means it can be None (but must still be defined).
* If only used within the job class, make it "private", which means
it gets a name starting with _. Code outside of the job modules
shouldn't be touching these. This includes the "constants" that
we define with all-caps in the class.
* I added unit tests which try to actually validate this, by
checking that all the correct properties are defined after
__init__. The test cases are still limited (all the test_init
cases) because of the pain of stubbing out __init__, but I will
be adding more in the future as I refactor to reduce the amount
of mocking/stubbing needed.

Overall, I don't know that this change by itself makes the code
simpler or easier to understand, but I don't think it makes things
any worse in that regard, while adding a lot more testing and
finally giving us a well-defined interface. I have more changes
planned, but as new improvements kept creeping into this change I
decided to mail it out for some review instead of letting it grow
even bigger. Some ideas of what I still intend to do in followup
patches:
* Refactor more of the directory management code into job_directory, to
get the rest of the mkdir/makedirs code into there. That way all
of the directory management is concentrated in one place.
* Refactor all the of get_state/set_state code into something that
automatically handles this with properties without a lot of
boilerplate getter and setter methods.
* Refactor the job.record related code that's duplicated into the
base class.

Risk: High
Visibility: Mostly code changes to standardize the job class interface.
A lot of the visible changes just involve privatizing attributes
that aren't used outside of job anyway, or adding in attributes
that were only defined on the client or server.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@3916 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
4b976078babcf8f652a44aa84537861ac1e4d6d0 21-Oct-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> make the code that calls client/tools/make_clean a bit safer.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3873 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
ad812bf3cf46bd9da1871401042b2b0abf1a4220 21-Oct-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Cleanups to code controlling where Autotest clients get installed and used by Autoserv.

* change autotest._get_autodir() back to looking for the autotest client executable, document explicitly that it's looking for an existing autotest installation, give it a unique exception to raise upon failure, and make it a public classmethod of the Autotest class. also renamed it to get_installed_autodir() to be even more explicit about what it's doing.
* change _get_install_dir() to look through client_autodir_paths for the first one it can create (if no existing installation exists), rather than just hardcoding the default to /usr/local/autotest. also make it a public classmethod.
* make verify_software_only() use get_install_dir() rather than get_installed_autodir(), since i think that makes more sense for what it's doing.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3872 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
bd450640e5be8e64d244b65488fbfb725056b7f9 19-Oct-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Move autodir_client_paths defaults into global_config.ini. Typically when we add default to the global_config that users should be aware of we remove them from the code so the code fails if the value is not specified. So that is what i did here

Signed-off-by: Scott Zawalski <scottz@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3864 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
07e27af54bd841e9a7b8d6d189fec3dd4ae2a92b 12-Oct-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> I realized that my initial addition of the default_profile_only
property was missing proper persistent state support on the client.
It won't be preserved across reboots, and changes to it on the
client won't be pulled back to the server. This change fixes that.

It also goes back to the setter approach for setting this property.
When I fixed this it occured to me that this was the real reason
we've used setters in the past for properties like this. I have
some refactoring in mind to clean up the handling of these stateful
properties, but that's rather substantial and I don't want to block
these changes on that so for now I just put in an implementatation
that's consistent with the "standard" handling we already have.

Risk: Medium
Visibility: Properly preserves the state of default_profile_only
across reboots and after client jobs.

Signed-off-by: John Admanski <jadmanski@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3826 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
a6082efeb30cd86d09fcf25d55cde15dedd60fce 12-Oct-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Add a global mechanism to the job object to allow users to change
the behaviour of profile_only, so that all tests default to

Signed-off-by: John Admanski <jadmanski@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3818 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
2315a7e4dd3efa7cc9c937483bbd660c7b595fcb 18-Sep-2009 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Fix up the client install to properly reverse the ordering of
repos. Otherwise this breaks the priority ordering of the repos
(and tries the "last" repo first).

Risk: Low
Visibility: Should improve the speed of client installs.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@3746 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
b8aa75bded69fd07b808c61eeed8eff9c4b12d63 18-Sep-2009 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Add a control file for doing an explicit full-client install on
machines. Rather than just hack it up in the control file I dedicded
it was better to just add an explicit install_full_client method
to the Autotest class.

Risk: Low
Visibility: Adds a way to perform a full (packageless) client install.

Signed-off-by: John Admanski <jadmanski@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3742 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
0cf508398c6837ee8bac5a7f1665273f8ece0495 18-Sep-2009 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Add a "repair test" along the same lines as the verify test so that
you have the option of running a repair as a normal test. This
includes a set of control files for each of the (useful)
protection levels.

It doesn't use the host protection levels that the scheduler understands
since autoserv has no concept of them. Since you'd have to explicitly
choose to run these repairs that doesn't seem like a problem.

Risk: Low
Visibility: Adds a repair_test (in the style of verify_test)

Signed-off-by: John Admanski <jadmanski@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3739 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
2266b1547440dc8432450386bdc2cf28decbcc36 09-Sep-2009 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> use getpass.getuser, not LOGNAME

Signed-off-by: Martin J. Bligh <mbligh@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@3671 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
ea0405c4f0129ede9f0078b780014eafd7a7fde2 05-Sep-2009 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Pass the user autoserv is running as through to the client.
This can be used to su back to the running user on the remote
end for file access, etc

Signed-off-by: Martin J. Bligh <mbligh@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@3662 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
e2e0befbf9be6dd44841f2c041a9b8f15089c36c 03-Sep-2009 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Make the default client autotest dirs a config value with a default
rather than module global.

Signed-off-by: Gregory Smith <gps@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3646 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
875f53522e82a1565fc30e3717efecdbf47da9e8 03-Sep-2009 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Really fix up the lightweight install this time. We were still copying
over profilers, but rather than trying to fix up the code that copies
them over I just droped that part of the install and excluded them
entirely again. This isn't a problem so long as we drop an __init__.py
inside the dirs we exclude.

Risk: Low
Visibility: Fix up the lightweight install properly

Signed-off-by: John Admanski <jadmanski@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3639 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
e1cea0a1c89ea97622332af34a75dd7822d8fed8 03-Sep-2009 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Default no_autoserv to False, since most clients are being run via
autoserv and thus can use autoserv packaging.

Risk: Low
Visibility: sysinfo installs should be lightweight again

Signed-off-by: John Admanski <jadmanski@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3638 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
6a83777f72aadb5835e751148be0809ec6910b27 02-Sep-2009 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Change log message from error to info, no packaging server is
perfectly valid.

Signed-off-by: Martin J. Bligh <mbligh@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@3634 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
bccad480b242b63f93f35b55a0e77517e3a76feb 25-Aug-2009 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Sadly, put back the "disable packaging-only installs" used by the
profiler installs. We can't count on profilers using autoserv-based
packaging because the profiler runs are background jobs, and so
don't actually use autoserv.

I think this code can ultimately but cleaned up but for now it's
better to revert back to the old model of doing "full" installs when
profiling is running on a setup without any http repositories.

Risk: Low
Visibility: Restores server-side profilers in no-repos configs.

Signed-off-by: John Admanski <jadmanski@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3609 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
483d1da073507a9dcd9a287c3b79a98bcdc70775 25-Aug-2009 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Remove the use of the very strange /etc/autotest.conf symlink on the
target machines. Depend on /usr/local/autotest or /home/autotest
existing (as a directory itself or as a symlink to one).

Signed-off-by: Gregory Smith <gps@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3608 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
ce955fcf58207d795ff1fe0001851b8b3728452c 24-Aug-2009 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Created a common_lib host object hierarchy (with site specific override
support) and moved *some* non server specific code from server/hosts to
the common_lib version. Created a client.bin LocalHost host object
inheriting from the common_lib Host and implemented a run() method based
on utils.run() (ie a host object for the local machine). Modified
client/bin/autotest to receive a new --hostname argument to tell it the
hostname to use for the LocalHost.hostname instance attribute and
updated server/autotest.py to send use this argument. Modified
client.bin.job to set a self.host instance attribute with an instance of
LocalHost and updated the unittest. Added an AutotestHostRunError class
(raised on LocalHost.run() failures).

Risk: high (there are modifications in the core server side support code
and some core client code).

Tested with verify/repair and client/server sleeptest jobs.

To be able for SVN to remember code history (that most of the new
client/common_lib/hosts/base_classes.py is based on the old
server/hosts/base_classes.py) then the following steps are probably needed
to apply this patch:
$ svn mkdir client/common_lib/hosts
$ svn copy server/hosts/base_classes.py client/common_lib/hosts
$ patch -p1 ...
$ svn add client/bin/local_host.py client/bin/local_host_unittest.py client/common_lib/hosts/__init__.py client/common_lib/hosts/base_classes_unittest.py

Signed-off-by: Mihai Rusu <dizzy@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3594 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
27b529165c63274395ff1d207e346201cea4cb25 14-Aug-2009 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Get rid of the "lightweight" autotest install option, in favour of
just always doing a thin install whenever possible; now that the
packaging-via-autoserv is supported, this should almost always be the
case.

Risk: Low
Visibility: Get rid of explicit lightweight installs, in favour of
automatic ones whenever possible.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@3544 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
ede7e248313e4670e69b2c9c28583cbc91d6f6bc 10-Aug-2009 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Add support for autotest clients run by autoserv to request packages
from the server when no other repositories are available. The
autoserv process will automatically bundle up a tarball and push it
out to the client when a package is requested from it.

Risk: Medium
Visibility: Allows tests, profilers and deps to ALWAYS be installed
via the packaging system so long as autoserv is in use, even if
package server haven't been set up.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@3522 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
b1a5113d43c8f0a8b115c541d3e590d280e75565 07-Aug-2009 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Next step in the PackageManager restructure, replace the static list
of repo urls with a list of RepositoryFetcher instances. Drops
PackageManager.fetch_pkg_file and just calls the
RepositoryFetcher.fetch_pkg_file on each repo directly in fetch_pkg.

Also adds an explicit add_repository method as a "clean" way for
other classes (specifically client/job.py) to add repositories.

Risk: Medium
Visibility: Switches over to the polymorphic RepositoryFetcher model
of fetching repo packages.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@3512 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
234b296fb6a81f67fd429c9be87b62708738311f 28-Jul-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> When running autotest client through autoserv, pass --verbose to the client so that we get debug output in client.log.N. client.log.N is the only real-time feedback one gets when running through autoserv, so it's helpful to have all info there.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3454 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
d0e94981f3771ec95821358bee9d27fda7378288 11-Jul-2009 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Server side messages cleanup. In this patch, the aim was:

* Fix issues with spacing on some messages
* Make server output coherent with client output
* Remove verbosity turned obsolete by the logging system
(it's not necessary to manually prepend debug messages with the method they
belong to)

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3398 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
e7d9c605bacd7b1816987994ae18a68c63306a16 02-Jul-2009 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Make the job executiontag available in both the server and client side job
objects as job.tag.

This is useful if your job would like to copy its data off directly to a
results repository on its own from the client machine.

Mostly small changes to pass the data down, though I did some docstring
cleanup near code that I touched which makes the diff larger.

The execution tag is taken from the autoserv -P parameter if supplied
and no explicit --execution-tag parameter is supplied. This prevents
the need to change monitor_db.py to pass yet another autoserv parameter.

Signed-off-by: Gregory Smith <gps@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3359 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
e9633f5852b063d60fffc0a663b98643e2afbf0a 02-Jul-2009 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Change an annoyingly verbose log message to DEBUG level.

Signed-off-by: Gregory Smith <gps@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3357 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
8297cf5b8e5eab3e303d4119f7a12cd26c4cfd22 22-May-2009 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Clean up _get_autodir in autotest.py a bit. Adds in some more logging
to make it easier to see what decisions it's making, and changes
the final fallback to checking /usr/local/autotest and /home/autotest
to just look for the directories, rather than an installed client.
Only checking for existing clients at those locations is inconsistent
with the other checks this call does.

Risk: Medium
Visibility: Adds logging to autotest._get_autodir and adjusts its
search to be more consistent.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@3166 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
a61edad29fd9f81e0c20190a16d0ea9cb9f0b9cd 22-May-2009 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Fix up the logging of unexpected client aborts to flush all the
current client log buffers first so that the final client end message
is logged after all the client logs.

Risk: Low
Visibility: An aborted client should never produce status logs after
the final client end message.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@3164 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
efe4ebff90b023edf8e146b7bfa4a796d52eb1ab 22-May-2009 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> A new version of the fix to handle race conditions between console
warning logging and status warning disabling messages. Instead of using
a clearly still-unreliable sleep, just hold up the logging of WARN
messages on the server side until we start seeing new messages from
after the warning coming in.

While this kind of buffing is rather unfortunate and means we're
delaying the logging of warnings, I just don't see any other choice.
We don't have a way to re-order status logs after the fact, so if we
log a WARN as soon as we get it then there's still always the posibility
that more status messages will come in that should've been logged
before it.

This still unfortunately leaves the reverse problem (logging warnings
too late instead of too early) but this is the much less common case.
It's also a lot less serious when it occurs, since it doesn't break
the warning enabling and disabling.

Risk: Medium
Visibility: Should prevent WARN messages from being logged before
events that they occured after.

Signed-off-by: John Admanski <jadmanski@google.com>




git-svn-id: http://test.kernel.org/svn/autotest/trunk@3163 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
02260fec67182c2307da15de93b2ac9a64ecc7b7 13-May-2009 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Summary: Remove 'client global keyval' retrieving, as it is a no-op most of the time

autoserv does have code that tries to retrieve a client side 'global'
keyval file (global in the sense it's not associated to a test, and
placed on the results directory (client/results/default/keyval)). Turns
out that such a keyval is not automatically generated by the autotest
client, and the only way to generate it would be the user using the
write_keyval APIs directly at the control file level. So if someone
wants to use such a feature, it would be better handled on a site
internal extension. On the upstream repo only causes noise on the server
log files.

After conversation with Steve and John we think the best approach would
just be to get rid of this part of the code. I have a patch ready for
0.10 and this is the one for the development branch (trunk).

Risk: Low (This is a no-op in the vast majority of the cases anyway)
Visibility: Autoserv users in general will notice one less failure on
their logs.

Signed-off-by: Lucas Meneghel Rodrigues <mrodrigu@redhat.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@3119 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
4556178ac97dc6160a31ac105f5d987b6c9933d8 11-May-2009 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Added utils.get_relative_path() to compute the path of an absolute path
as relative to a given directory path and used it when creating the
test results sysinfo/current_reboot symlink to create a relative symlink
that would still be valid when relocated.

Added a "preserve_symlinks" default False keyword argument to
AbstractSSHHost.get_file/send_file to tell rsync to try to preserve the
original symlink and not transform them into the referenced
file/directory when copied (only for rsync as scp has no way to do
this). Used the new flag when fetching the test result directory to
preserve symlinks. Along with the previous change this should fix an
issue of >100k sysinfo bloat because of duplicated content instead of
preserving the client symlink.

Signed-off-by: Mihai Rusu <dizzy@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3104 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
6fc7ff59377638acd449ad5b5ff7cd44fd922b23 04-May-2009 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Change the autotest.run implementation to only automatically add
profilers to client runs on non-background runs. We can't stop the
profilers and collect the results anyway since control of the run
is left to the code launching the background client, and these runs
are generally internal processes that shouldn't be profiled anyway.

Risk: Low
Visibility: No more profilers on top of background client jobs.

Signed-off-by: John Admanski <jadmanski@google.com



git-svn-id: http://test.kernel.org/svn/autotest/trunk@3087 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
372ce3e81404405c250e31bfb011aa2d0f412102 07-Apr-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Convert a missed self.logger call to use the logging module.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@2963 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
16a7ff7af7fe2ae3093c5ad7452c516257184ffc 01-Apr-2009 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> The enabling and disabling of warning messages unfortunately seems to
be rather racy. Unfortunately, since warnings and test status messages
are being pulled asynchronously from two different sources there can
be a skew of a few seconds between when the event actually occurs
and when the server thinks it does.

Ideally we could fix this by forcing some kind of sync but there
doesn't seem to be any way of doing that; we have no way of flushing
conmux output, and /var/log monitoring is even worse since we may
also have client filesystem buffering getting in the way.

I was able to fix all the cases I could reproduce by adding a five
second sleep before and after disabling and enabling warnings. This is
an...unfortunate way to fix it, but in practice it works rather well.
I'm not actually certain we can do a lot better, at least not without
completely restructuring the way we do warnings. And even then, we
still have network latency that can kill any approach that doesn't
use a "sleep for a moments while the buffers flush" anyway.

Hopefully this fixes things well enough; if this problems comes up
again thanks to > 5s skews then we may have to revisit this.

Risk: Low
Visibility: Adds some sleep to hack around warning monitor and
client status log time skew.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2950 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
91d56a9bbeb29f4f80cd36072254977fe4530eae 01-Apr-2009 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Add a network_destabilizing flag that tests can set to disable
NETWORK warnings.

Risk: Medium
Visibility: Adds a way to flag a test as network-destabilizing.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2947 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
7c7aff3a2ed5787d502642a3f5eaa0107aad1661 25-Mar-2009 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Fix up the server-side sysinfo collection (again) to make sure all
temp directories and installs on the host machine are removed. Since
the install->run collection->remove pattern seems to be pretty common,
I extracted it out into a dectorator.

Risk: Low
Visibility: Should eliminate more sysinfo collection junk eagerly.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2938 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
b18134f8faa7c5b4623760bc88650a65e70b2cac 20-Mar-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> As discussed on the mailing list, we implemented logging with a single
root logger and logging configurations per entry points. The entry
points affected are:

* Autotest client
* Autoserv
* Scheduler

We are sending patches for each one of those entry points. Now we don't
need to 'grab' loggers anymore to log messages, only need to use the
utility functions:

logging.info('msg')
logging.debug('msg')
logging.warning('msg')
logging.error('msg')
logging.critical('msg')

Which reduces complexity of the log system, and makes it easier for
developers to log messages, just select the level, make sure the
standard logging module is loaded, and profit!

From: Lucas Meneghel Rodrigues <lmr@linux.vnet.ibm.com>
Signed-off-by: Steve Howard <showard@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2915 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
6bb32d74bb4c5c6ae8b76e01d58b8963bbf61849 19-Mar-2009 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Add some code to the server job and autotest classes to save off a
list of directories that need to be copied in order to retrieve
client log results. It uses file locking to sequence updates to this
file since multi-machine jobs will be sharing the same log file.

This is the first step in adding a way of doing post-abort (or any
post-SIGTERM) log retrieval. The next step will be adding a way in
autoserv to run a standalone "recover this data" job that will use
the data recorded in this change, and the final step will be to add
support to the scheduler for running this "recover this data" service
when catastrophic failure occurs.

Risk: Medium
Visibility: Adds an "uncollected_logs" file in the server job results
directory containing a list of remote directories that need to be
copied to recover the results of a crashed job.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2913 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
e41305384c80bd6d8a126b3ace503e179e13bf8d 17-Mar-2009 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Fixed a bug in an error message in Autotest. Looks like a chunk of
code that got moved around and no longer makes sense now that it's
out of context.

Risk: Low
Visibility: Gets rid of a reference to an undefined variable.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2884 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
3c2369470f809d4a920e8cc102c9ad54fdb5c839 04-Mar-2009 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Remove the double-quoting from the mkdir and rm commands being run
in Autotest.install.

Risk: Low
Visibility: In the case of special characters these should work
correctly now, instead of over-quoting.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2853 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
e2eef7bedc043857306fc2ea7211cbf6f48a4f09 04-Mar-2009 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Fix an issue with the automatic turning on of profilers from the
server side, where we need to disable it on lightweight (i.e. sysinfo
and server profiler installs). Only "real" installs should turn on
the profilers.

Also, this moves the job.profilers.add call in the control file down
so that it occurs after the packaging setup calls, to make sure that
the packaging system is always available when the profilers are
being added.

Risk: Medium
Visibility: We no longer will automatically run profilers on
lightweight installs.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2849 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
29a4c707b906627522fd171dd709588ca9a413d6 04-Mar-2009 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Fixes a regression; we're not properly setting
the job results dir when executing control files. This messes up
multi-machine server jobs that execute client jobs by writing
status logs into the wrong place.

Risk: Low
Visibility: Fixes some seriously broken status.log logging

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2848 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
1c3c07b9c7100c9ae28d609a4d94764a5ce0d9ab 04-Mar-2009 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Clear out the contents of the autotest results dir when installing on
a client machine. The use of symlinks in our results sometimes
confuses the rsync-based install process, so just avoid that mess by
always clearing out the directory first.

Risk: Low
Visibility: Should reduce the need to fall back from rsync to scp
during autotest installs.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2847 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
cb0e16134a76b3badc0307cd804a9e510f3e232e 27-Feb-2009 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> When profilers are turned on from the server side and a client-side
job is launched, we should add some corresponding job.profilers.add
calls to the client-side prologue so that they are turned on for
client-side tests as well.

Also fixes a "how did this ever run" type of bug in the profiler
argument handling code on the server side.

Risk: Low
Visibility: Turning on profilers from the server-side also turns them
on for client-side tests.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2840 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
fcc0d5d352f3b6e031a2080e5474bd3a70ef2ec7 12-Feb-2009 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> When we get a "test completed" signal from the client indicating that
we can go and collect test logs, we should also flush our status log
buffers. It should always be safe to do this, as the client only
signals that logs can be collected when it is not currently running
any parallel operatinons.

Risk: Low
Visibility: Logs should be flushed more eagerly to status.log on the
server.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2786 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
f37df84b97538540185f91e2e5b62890ca407b3b 11-Feb-2009 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Add job.enable_warnings and job.disable_warnings methods to the job
classes, to allow you to disable and enable specific types of console
warnings from being logged. Whenever these messages are called an
INFO line is written to the status log.

I added three categories to the patterns, based off of suggestions
from Martin: BUG, OOM, and LOCKDEP.

There are some limitations to this capability; it the warnings are
basically disabled globally, there's no easy way to disable them for
specific hosts or tests. It could probably be extended to host-specifc
controls if a need arises, but extending it to specific tests isn't
really possible, there's simply no way for the console monitoring to
determine what test the warning is actually associated with. Note that
this is only a problem for tests run in parallel; if you are just
running a series of tests obviously the warnings can be turned off at
the start of the test and then turned back on after it is finished.

Risk: Medium
Visbility: Adds categories to the console warning patterns, and a
mechanism for enabling+disabling specific categories of warnings.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2771 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
7a60367a02934d7d21a3f44cb7b1f004e97e5b25 07-Feb-2009 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> More package clean up and removing large error messages that included
redudant commands that were listed above. Augmented error messgaes to be
more precise.

Signed-off-by: Scott Zawalski <scottz@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2763 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
6dadd8360710eb88fb80a9ce85433e13e8dcabe9 06-Feb-2009 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Adds an autotestd and autotestd_monitor script to the Autotest client
to allow us to run clients from autoserv in a disconnected fashion,
and adds support to autoserv to make use of this when running clients.

The basic idea is that instead of running the client via SSH and
sending output & status info back to the server via the connection,
we kick off autotestd to run the client and write output & status info
into temp files, and then run autotestd_monitor from the server to
feed that information back to the server over SSH. Since the monitor
is completely detached from the running of the client itself, if
anything happens to the connection we can simply restart the monitor.

I also had to fix a bug in ParamikoHost that came up while working on
this. Sudden disconnects from the client-side would lead to the
commands exiting with a SIGHUP; the SSHHost implementation would
throw an exception is this case since this is a connection failure
rather than a command failure. I added an explicit SIGHUP check to
ParamikoHost.

Risk: High
Visibility: If the network connection gets killed for any reason but
we can reconnect to the remote machine we now do that.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2752 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
a700772e70548b10721ba709c7ec5194bb6d7597 13-Jan-2009 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Created a client/common_lib/utils/import_site_class function to take
care of site specific importing of classes. Modified code to use it.

Signed-off-by: Mihai Rusu <dizzy@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2626 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
b264ed09d1300aed3a6353eaa9a8545ca9c03f35 13-Jan-2009 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Add some END ABORT logging to the code that executes the Autotest
client, so that if the client fails unexpectedly without logging
anything then at the very least an "unexpected termination" log is
written.

The server message isn't very explanitory, but unfortuately there's
no easy way for the server to know what happened in general; however
the END ABORT log should at least make it clearer that the failure
was on the client and the client.log.* files should be examined.

Risk: Medium
Visibility: Should eliminate a case where the client could fail but
nothing would be logged in the status log

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2623 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
c98c47002070d684f022bf4e1d957929c309ab25 05-Jan-2009 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Drop the explicit tag parameter from Autotest.run_test and
Autotest.run_timed_test. These methods are meant to look like
job.run_test, and any tag parameter passed into them should behave
like the tag parameter to job.run_test.

Autotest.run has its own tag parameter that means something
different, and has implications for
where the client job is run and where its results are collected; if
users explicitly need that much control they should really be using
Autotest.run directly anyway.

Risk: Low
Visibility: Passing tag to Autotest.run_test will interpret the tag
parameter as a job.run_test tag parameter.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2592 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
d99d3b27fb076377354fa388e0a81e97bda443b8 22-Dec-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Boot timeout leads to hardreset attempt, then broken exception.
Exception handler on _wait_for_reboot's attempted call on optional hardreset routine
was not catching the kind of exception thrown when host has no hard reset op.

Signed-off-by: Duane Sand <duanes@google.com>




git-svn-id: http://test.kernel.org/svn/autotest/trunk@2577 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
e0c7fb6d315135b265eed8c6086ba53a7c09d59b 16-Dec-2008 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Currently, client.log.* files are numbered in a way that's only unique
within a single Autotest.run, so multiple client jobs will reuse (and
overwrite) the same files. Change this to a scheme that simple grabs
the next available filename.

This does mean that you can no longer assume that client.log.N is the
log from after reboot N in a client job, but in the context of having
multiple jobs that piece of information is a lot less valuable than
simply having unique, sequentially numbered filenames.

Risk: Low
Visibility: Multi-client-job control files will now have all the client
logs captured in unique client.log.* files.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2570 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
b3c0c91ff9020a7259377e3d607c3aa0cae9c804 27-Nov-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Adds support for using client-side profilers from the server side.
This makes a few significant infrastructure changes in order the
support this:
- it modifies autotest.py to allow "background" clients to be
launched, where the server simply kicks off a client as a
background process without bothering to monitor it
- it adds a profiler_test test which is not a "real" test but is
instead used to allow the server to start & stop the profilers
in a reasonably controlled way

Currently, this still lacks any support for dealing with reboots; if
a test reboots the remote machine then the profilers will not be
restarted and the logs collected will only go as far as the first
reboot.

Risk: Medium
Visibility: You can use profilers from the server side (via
job.profilers.add and job.profilers.delete)

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2514 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
f22fea8f311c4d955b171a2e7869732d6a1067e4 26-Nov-2008 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Add an --install-in-tmpdir option to autoserv that allows us to change
the default Autotest client install behaviour so that new clients
are installed in new temporary directories instead of the "standard"
locations. This only affects installs that don't explicitly request
a specific location.

The expected use for this is to allow the use of multiple server jobs
against the same machine without having them collide with one another.

Risk: Low
Visibility: Adds a new flag to autoserv, but the default behaviour
is unchanged.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2510 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
043e1132d1af7969a8f67871188ef34e050c2186 19-Nov-2008 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> This depends on Martin's "run autoserv without a results dir" patch
to be applied cleanly.


A rather large refactoring and stubbing change working towards
client-side profilers from the server side. This makes a few major
changes:
- refactors almost all of client/bin/profilers.py into a common lib
profiler_manager class; except for the code for actually loading
a profiler object, all of that code is already completely generic
- add a server-side profiler_proxy class that will act as a proxy
on the server for using a client-side profiler, this doesn't
actually do anything useful right now but it basically just a
stub for later changes
- add a server-side profiler_manager implementation that creates a
profiler_proxy class instead of actually loading a real profiler

The intended changes still in the pipeline that will build on this are:
- add code to the profiler_proxy for actually making sure the
profiler is set up and installed on the remote host(s)
- add a mechanism for client-side profilers to deal with reboots

Risk: Medium
Visibility: Adds a bunch of stubs that don't actually do anything yet
but will do things soon.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2447 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
5b2844f8306994a5bb09ef4f618910cc3c5d3427 19-Nov-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> git-svn-id: http://test.kernel.org/svn/autotest/trunk@2446 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
79ab928f306a222db195e0ef75806be071f8b3dd 11-Nov-2008 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Make sure that when a client-side reboot fails the server properly
logs it as an ABORT.

Risk: Medium
Visibility: Reboot failures during client jobs show up in the status.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2397 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
27b37eaaf57e5bc1eb1bbc06416f10afa4c8eb62 30-Oct-2008 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Get rid of the job.aborted flag. It's not necessary, and it actually
makes it impossible to supress client job abortions (e.g. if the
server-sysinfo client job aborts, we don't abort the server job).

Risk: Low
Visibility: autoserv won't return non-zero codes on non-aborted jobs

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2362 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
c6772063be0ad8ed171c701cd9d1749d2f78836b 29-Oct-2008 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Clean up the server-side error when we can't pull back a state file
from the client; make it clearer that this error is just a warning,
and eliminate the useless and noisy traceback.

Risk: Low
Visibility: Added reduced-scare error output.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2361 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
4600e343febea26110a920c598870f75ac394952 29-Oct-2008 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> The client logger that handles client-side logging + warnings is
broken; we would throw away the logger and create a new one for
every client test section, but that meant that between reboots
you lost all the information about the current state of the client
logging. When WARN messages got logged during this time (not usual
when the console monitor finds something in the boot log) it would
be logged inconsistently with the rest of the client logging. This
confused the parser and often resulted in false failures or even
parser errors.

Risk: Medium
Visibility: WARN messages during a reboot will no longer mess up all
the test results.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2360 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
89e258d442be542143b1c79fcd5a990bde701f34 24-Oct-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Add an option to the Host send_file and get_file methods to indicate
that whatever is at the destination should be deleted. For
AbstractSSHHost this means using --delete (for rsync) or just
pre-emptively deleting the destination directory (for scp).

The option defaults to False (the old behavour) but this changes the
autotest install to set it to True, so that we can be sure that old
.pyc* and abandoned .py files are never left on the client. This also
allows me to remove some of the "remove stale files" hacks from the
setup_modules code.

Risk: Medium
Visibility: Adds a new option to *_file methods on Host objects and
kills stale files during Autotest client installs.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2332 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
c09fc1543a3f77026bab728770aa0191feaddd07 15-Oct-2008 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Make sysinfo more configurable. This adds some methods to the job
classes, job.add_sysinfo_command and job.add_sysinfo_logfile, that
can be used to add more commands and/or logfiles to the standard
sysinfo collection for the job.

Risk: High
Visiblity: You can add code to control files to customize sysinfo
collection.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2289 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
0910844396c68a8747dc1ab580fb898ecff2469e 15-Oct-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Keep track of the last kernel we explicitly 'reboot'ed into. This
should be the default for any subsequent reboots.

Fix a bug where we would set the default label on a platform when
host.rebooting() from the server side (when only a boot_once was
called for). Also fixed a bug where label=None would NOT reboot
into what was supposed to be the default kernel.

Fix a bug in the state engine which made it impossible for None to be
the default of a state variable.

Risk: High
Visibility: High (this tweaks the semantics a bit; most people would
have called the behavior a bug, but some may have
depended on it)

Signed-off-by: Jeremy Orlow <jorlow@google.com>


Make boottool handle the None case on boot_once and sanity check
it for set_default (since it takes an index, and Null is not a valid
use).

Risk: Medium
Visibility: Medium

Signed-off-by: Jeremy Orlow <jorlow@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2287 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
76d19f708c5c7bacf7220539802086d04ed450b4 15-Oct-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Package updates:
Updated errors to reflect what package failed
Removed nc and changed it to wget (removes extra unneccessary debug lines)
renamed all file_name to filename
Added hostname to init so that repos can automatically be prioritized on initialization
Update autotest.py to pass hostname
Update run_http_test to actually write to the file that it is removing

Signed-off-by: Scott Zawalski <scottz@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2286 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
1e3b0992d23a924f0217ad4d4491bdd986575c6d 14-Oct-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Add get_mirror_list function stub to packages.py that just returns the given repo list. Useful for local sites to create a way to return a prioritized list of repos.
Modify autotest.py to call get_mirror_list before appending the repos to the control file
Risk: Medium
Visibility: Low

Signed-off-by: Scott Zawalski <scottz@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2279 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
54f90af69daac8a781678f0ccd42d6728d5d81d1 10-Oct-2008 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Add server-side sysinfo collection code (for server tests). This
is implemented by doing a temporary lightweight (no tests or
deps) autotest install on the remote machine and then running a
small script to generate the sysinfo.

The code is somewhat ugly because of the nature of what it has
to do on the remote machine (it needs to run before the test,
save some state, and then run afterwards with the saved state).
I accomplish this by pickling off of the sysinfo object after
the pre-test run, and reloading it during the post-test run.
However, I don't see a cleaner solution; we can't depend on
anything we leave behind on the remote machine while the test
is running not getting smashed.

This only applies to single-machine jobs (or multi-machine jobs
that break down into single-machine jobs via job.parallel_simple).
For multi-machine jobs we'd need to come up with a new,
multi-machine way to log sysinfo; in my mind it's a completely
separate problem.

Risk: Medium
Visibility: Adds sysinfo collection to server-side tests.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2270 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
76a42932939e6cb95ab63a131695ad6e7ab3b70c 09-Oct-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> recommit logging changes - was just spare pyc file




git-svn-id: http://test.kernel.org/svn/autotest/trunk@2266 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
b931b687e09cec9b25f37d437f33be73b6de20f3 09-Oct-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> back out logging changes for now




git-svn-id: http://test.kernel.org/svn/autotest/trunk@2265 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
681b6a05a596fb0bf379979e11bb620b58a5707f 09-Oct-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Configuring main autotest classes to use the new debug infrastructure,
converting 2 of the major verbosity sources on autotest and autoserv
executions. Logs with timestamps as a bonus :). The patch should apply
cleanly against svn trunk.

Visibility: Users and developers of the command line programs (autotest
and autoserv) will perceive a reduced verbosity, that can be increased
on the debug configuration file if needed.

Risk: Low (small number of changes, all tested and things were done
trying to isolate sources of failure as much as possible).

Signed-off-by: Lucas Meneghel Rodrigues <lucasmr@br.ibm.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2261 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
1b3b376d94edee8f235f8669a047dc00751bf97a 25-Sep-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Rename client/common_lib/logging.py to client/common_lib/log.py in order to be able to use the standard python logging module

Python has a standard module used to perform logging operations
(http://docs.python.org/lib/module-logging.html) that could be used
inside autotest, however we have a common_lib module named logging as
well, that makes difficult to import the python logging module in all
other parts of autotest (including tests).

So this patch basically renames logging to log and fixes all references
to it. All the files touched were pylinted and basic sanity testing was
made (running autotest with different control files).

About the risks, in my opinion it's worth the effort, since we gain the
ability to use a powerful logging library, that would help us to
implement a better logging system inside all autotest modules.

Risk: Medium/High - Even though care was taken to ensure all references
to logging were changed, it touches a fair bit of the code and something
might have passed unaudited.

Visibility: Low - Should be invisible for all users, and developers must
just remember that now logging is called log.

Signed-off-by: Lucas Meneghel Rodrigues <lucasmr@br.ibm.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2193 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
23afbec41f6275bef4e80c36726916b0be564204 17-Sep-2008 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Add a standard run_cleanup option to test.run_test that can be used
to enable/disable the running of test.cleanup after the test
completes. This also adds some global state on the job objects
that can change to default for tests from enabled to disabled.

Risk: Low
Visbility: Adds ways to disable test cleanup at both the job and test
level.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2167 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
a1f3c20cb943bd56c6aaee16b2b21dca83a5177d 15-Sep-2008 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Refactor the results collection code to run out of a separate class
that we can reuse in different places, and change the mkfifo code
to use a filename created on the fly, rather than hardcoding a fixed
filename.

Risk: Medium
Visibility: Mostly just a refactoring.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2159 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
b6eb2f19e1225b14dee3839c2945caf6bc90fa73 12-Sep-2008 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> First pass at pulling back test results during a client job, as each
test completes. Definitely needs some testing, cleanup and refactoring
before this can be considered complete, but this is a good, working
start point.

Risk: High
Visbility: Test results should now be pull off of client machines as
the tests finish.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2139 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
169ecad264cbf834c2c10325be7cebf2d1a91ee8 12-Sep-2008 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Fix up some bad style trash in autotest.py

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2137 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
0562e658835ab8d604f56a4b9b3a84bcadafc143 20-Aug-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> - Add a set_autodir method to ssh_host to set the autotest installation directory value
- Set the autotest installation directory in ssh_host from server/autotest.py once the installation of autotest is completed on that machine.

The idea behind the patch is that host.get_autodir is generally not
very useful, because hosts are generally constructed with
autodir=None. But for server-side code outside of autotest.py, there
really isn't any other way to figure out where autotest is installed
on the remote machine.

So this patch changes autotest.py to change the autodir on a host
object when it uses that host object to install autotest. That way
code using the host object can figure out where autotest is, as long
as it's been installed.

Signed-off-by: Ashwin Ganti <aganti@google.com>




git-svn-id: http://test.kernel.org/svn/autotest/trunk@2016 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
c5ddfd1f71caef9ec0c84c53ef7db42fcdc33e1c 04-Aug-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> I have attached the patch for the packaging system.
The documentation is available at : http://test.kernel.org/autotest/PackagingSystem

Signed-off-by: Ashwin Ganti <aganti@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1953 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
c66e93ce12e748352d1979c99acbc732b9680a25 29-Jul-2008 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Fix the parser behaviour when handling non-grouped tests inside of
a client job. The test subdir should override the job subdir (or
rather, the job subdir should not override the test subdir).

The main idea of this fix is to move the code that check if the line
currently being parsed is an unexpected abort to before the line is
actual parsed and the "current" test state of the parser is updated.
This allows us to abort the current test.

Risk: Medium
Visibility: An unexpected client failure during a test should look
like a job abort, not a build+job abort.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1923 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
3c7a1500cd0e67ce3ee4752453ea3e76eb088ccd 24-Jul-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Fixed the autotest_unittest so that it is consistent with changes made involving the new packaging system.
Also made a slight change to autotest.py as it was using "dir" as a temporary variable which has the affect
of overwriting the actual symbol for the function dir(). I doubt this is a good idea.

Signed-off-by: Travis Miller <raphtee@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1897 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
4e4961c5b5081429f8032c61a13f1ba7c94fffb4 11-Jul-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> fix typo

Signed-off-by: Martin J. Bligh <mbligh@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1819 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
78bf5355ae3455f6e18fde2b4a839b16a59db337 11-Jul-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Fix formatting / coding style in autotest.py

Signed-off-by: Martin J. Bligh <mbligh@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1818 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
8947606f972d0e78cec363e624a72066115da69c 02-Jul-2008 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Change the Autotest.run code to throw an AutotestRunError when a
client job is aborted.

Risk: Medium
Visibility: Any code depending on server-side control files to
continue execution regardless of the outcome of an Autotest.run call
would be broken, although I'm not currently aware of any examples or
use cases for this.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1758 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
0afbb6369aa5aa9a75ea67dd9e95ec4b21c0c181 06-Jun-2008 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Convert all python code to use four-space indents instead of eight-space tabs.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1658 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
1965dfa167c65fea2c49536f1fc329ac7203828a 04-Jun-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> New options and automatic ssh key setup

Risk: Medium
Visibility: Adds three new argument options --ssh-user, --ssh-port, and --ssh-pass, which overrides the global default 'root' and '22' for all machines.

Adds the option to pass user, port and password per-machine in the format user:pass@host:port (overrides --ssh-user, --ssh-port and --ssh-pass).

Adds the pexpect.py and pxssh.py modules to autotest tree, these modules are used to interact with interactive programs.

The --ssh-pass option and pass field on machine string is used only to perform automatic ssh key setup on remote machines, which is also implemented in this patch.

Signed-off-by: Ramon de Carvalho Valle <rcvalle@br.ibm.com>
Signed-off-by: Lucas Meneghel Rodrigues <lucasmr@br.ibm.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1600 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
ec0a42b6b2b4c6f0f65a9dbdfc23bf43ceaa2a6a 04-Jun-2008 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Risk: Medium
Visibility: Changes how status logs are written for tests which use
Autotest.run and pass in a results_dir different than the default.

Currently if you use Autotest.run to run a test and specify a results
directory, the status logs are not redirected for the duration of the
test, like all the other results are. This fixes that.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1596 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
02ff2d5ee0d687923ab0caf2355f89cae0fa7177 03-Jun-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Risk: Medium - Hits quite a bit of different areas
Visibility: Low
utils.py
Create a wrapper for urlopen and urlretrieve to enforce a timeout
using the socket module and setsockettimeout.

- All additional files outside of utils.py are using urllib/urllib2 in
one way or another and are being changed to use our wrapper

Signed-off-by: Scott Zawalski <scottz@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1590 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
ec85914ffa842c275d6c0c0392a36132b992249a 29-May-2008 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Risk: Low
Visibility: Adds a 2.5 hour wait when the autotest client unexpectedly
terminates on a remote machine.

This adds a 2.5 hour wait_up to the the server when the autotest client
terminates unexpected. When the client machine is still up this will
have no impact, but if the client machine actually crashed (e.g.
a kernel panic) then this provides an extra opportunity for come back
before log collection occurs.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1570 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
9a3f5e5b167d5c83caa516b98e2f85c4231e7ef4 28-May-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Move the keyval files somewhere temporary and then move them back later.

Signed-off-by: Travis Miller <raphtee@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1561 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
5b4765b8f3f734ec4b43ffadab2c152328a88e0c 28-May-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> autotest.py failed to import traceback.

Signed-off-by: Travis Miller <raphtee@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1559 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
e6020ab4f98cac2e3e16fe78310f18c04a915248 27-May-2008 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Fix an exception handler that is trying to catch two different types
of exceptions, but is using the wrong syntax.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1555 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
954e63e2f86cc7b8a15b8efc1fc05d32e84769ed 27-May-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Doing som basic cleanup of autotest.py before I start working on it for unit testing.

Signed-off-by: Travis Miller <raphtee@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1553 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
c1cbc9948f5ddaf360853afa1994d942ffd5fe09 27-May-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> 1. Enable job tags to be passed from the server side (basically use the -t option available on the client side to be passed in from the server)
2. Add code to help run multiple jobs on the same machine starting at the same time (using jobs tags so that the results don't get clobbered up and barriers for synchronization)
3. Add a sample server side control file to demonstrate the usage of running multiple tests on the same client starting at the same time by running two "sleeptests".

Signed-off-by: Ashwin Ganti <aganti@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1549 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
1a5a3fb81bc666d3dbf8efca31051f8027ad0208 21-May-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Summary: When can't figure out autotest directory, get_autodir() raises
an AutotestRunError, instead of an AutoservRunError. When I made the
first patch trying to fix the install problem, I made the mistake of
trapping an AutoservRunError exception instead of the correct one. The
patch fixes this mistake.

From: Lucas Meneghel Rodrigues <lmr@linux.vnet.ibm.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1542 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
57c34846fad5b9ba00bb20dd37db20d9fef03208 10-May-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Make usage of write_keyval and read_keyval consistent



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1502 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
2974230271313d1d98e914f3f4cf13cccfb70804 09-May-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> If we can't find the autotest dir, fall back to /usr/local/autotest



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1498 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
cb73fee8623ccf8c227cc233a82ef2aa0ca1c577 01-May-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Changed the process of copying keyval file from remote machine. New process avoids deleting existing keyval file.

Tested by running jobs and making sure that keyval values are properly appended and temporary files are properly deleted.

Signed-off-by: Svitlana Tumanova <stumanova@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1474 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
7f2befb6b97e68321009c9af5f696b2a6bca4077 21-Apr-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Provide a hook for code to be executed after every client machine reboot.

Signed-off-by: Steve Howard <showard@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1458 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
ccb9e18237137e547fc0f815506710a8b1d94453 17-Apr-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Convert all imports on the server to use absolute imports.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1452 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
caa62c2a72c0300254998f21f4aca4579f251da4 07-Apr-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Implemented hooks for external logging infrastructure for storing both server-side and client-side logs.

Signed-off-by: Svitlana Tumanova <stumanova@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1408 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
215c55acea57656f5d9abfa9eef86d7491bc3615 25-Mar-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Log a job-level ABORT into the status log whenever an Autotest client
terminates unexpectedly.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1358 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
d8b39259ca4f011cea6f804e5b407262cad57057 20-Mar-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Added support for having site version of autotest.

Signed-off-by: Colby Ranger <cranger@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1354 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
dab3966fdf65b8d047fdb4ede5d642e498d8b2f9 27-Feb-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Move the code responsible for pulling client-side status logs over to
the server into server_job so that it can be integrated with the
serial console warning monitoring.

Signed-off-by: John Admanski <jadmanski@google.com>




git-svn-id: http://test.kernel.org/svn/autotest/trunk@1270 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
6f015c44cad8984c6117a2bbf8ff8a5a16602012 12-Feb-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Move some error classes out of server/autotest.py and into the error
library, and associate them with the appropriate superclasses since
these errors are really autotest errors and not autoserv errors.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1233 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
9e787d26dfaebe9a97391d05f9ab60e1d2362cb4 11-Feb-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Fix autotest client-handling code to use wait_up rather than ensure_up.
For one, ensure_up takes 30 minutes to timeout. For another, it'll
hardreset the machine if it's down, which is not desirable in this
context

Signed-off-by: Martin Bligh <mbligh@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1232 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
09f456938e7c8593a3f6490b08917ab5db188864 26-Jan-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Get rid of a few meaningless errors from test commands

Signed-off-by: Martin J. Bligh <mbligh@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1204 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
4d6feff83e2a38c74fa7e935971baa453ed97341 14-Jan-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Fix exceptions not to be strings

Signed-off-by: Lucas Meneghel Rodrigues <lucasmr@br.ibm.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1157 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
da13d54bae2a4161d8cd85a41d10028a67435e94 03-Jan-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> I need to be able to run many autotest instances on one machine and
I need to be able to have these tests run for an indeterminite amount
of time.

From: Jeremy Orlow <jorlow@google.com>
Signed-off-by: Martin Bligh <mbligh@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1103 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
1f194909479badc386d997d272331139f5706d68 20-Dec-2007 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Modify the test results collection to try and ensure the remote host
is up before trying to grab the files.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1091 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
d528d30ade9c0ea04777555908053f324f5ca56d 19-Dec-2007 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Extend the status logging mechanism to handled multiple tests in
parallel on a single machine without interleaving the status logs from
each test. This introduces some delay into the processing of these
logs since the status logging has to delay writing out the status
logs until it can impose a serial order on them.

However, it does NOT delay the writing of the logs to stdout; since
stdout is intended for human consumption and not machine processing it
is much more convenient for the logging to be visible in the logs at
the point in time where the event occured.

This implementation tries to avoid introducing a sitation where logs
are dropped on the floor because their being held back from the status
log. In the worst case on the client side the logs if autotest fails
before accumulating the logs in status then the intermediate logs will
still be available in individual status.* files, and on the server side
the server will automatically write out whatever logs it has
accumulated so far in the case of a failure (under the assumption that
it is still possible to write to status.log).

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1089 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
4918bdb2981e3896f40072e89daf0f7dbdad7387 13-Dec-2007 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Don't remove the control file if it wasn't a temp copy

Signed-off-by: Martin J. Bligh <mbligh@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1064 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
548197f1292597dfa5247bcea157379680d1f5e8 12-Dec-2007 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Redirect output of remote ls command 'tests' to /dev/null
Now that we're logging the output of ssh commands, it's a bit noisy

Signed-off-by: Martin Bligh <mbligh@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1058 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
35b225c6a7237e4f89908b5aded18d9cda3bde27 10-Dec-2007 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> autoserv: grab test results even if we had an error

From: Adrian Bauer <adrianbg@google.com>
Signed-off-by: Martin Bligh <mbligh@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1052 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
03f4fc7be7966d839d6055021424b8a9d430faa8 29-Nov-2007 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Pull the server-side errors.py into the common error library.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1007 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
adf2aabbf6db8fd5f0fa4321022cd4f2aa33a44a 29-Nov-2007 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Have the autotest server run the client by calling host.run, instead
of explicitly doing it through ssh. Also, make sure that when
SSHHost does run commands through run it logs the commands being run
and passed on the output to stdout and stderr.

This also disables X11 forwarding in ssh (the -x option) to since we
don't use it and it may just introduce a lot of spurious "remote host
denied forwarding" warnings into the logs.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1002 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
2bf2db6379790645e2ffc45102ee729d271cf3ba 27-Nov-2007 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> When sending status logs back through the ssh stderr, we need to
prepend the logs with some sort of indicator that the error came
from autotest and not ssh so that errors generated by ssh itself
don't get mixed in with status information.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@993 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
e988aa5a8f2d4e805fd36a91b9bb05a561bec6b9 24-Nov-2007 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Fix up autotest directory detection to look at the destination of
the autotest.conf symlink, instead of making some foul effort to
parse python code.

Signed-off-by: Martin Bligh <mbligh@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@973 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
a4bece10130a2bffe4aa2d8fa5a20244040ac9b0 24-Nov-2007 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Use control.autoserv rather than control on the client.

The intent is to support the manual stepping method on the client,
which requires an init script that checks if $AUTODIR/control exists
and if so executes it from the init script. We don't want to hit that
path when executing from the server

Signed-off-by: Martin Bligh <mbligh@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@970 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
cf732d1d7978e8065ef38c1daa3bdb26dad71058 21-Nov-2007 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> The server should only consider a client job to be aborted if the
exit status is 1. Just catching all non-zero exit codes also catches
cases where the job is doing a client-side reboot.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@951 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
faf0cd4c3c8845010909a15a3c5ff355eb26b24f 19-Nov-2007 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Running autotest_client should give the same exit status as running
autotest directly (i.e. non-zero on ABORT, zero otherwise). Also,
running autoserv should produce a non-zero exit code if it runs a
client-side test that is aborted, although this does not actually
abort the server-side job itself.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@944 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
c343016711f17d8ad4c6bb1239944eee13279f0f 15-Nov-2007 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Move a lot of the ad hoc logging out of harness_simple and into job.py,
and make sure that multi-line logs (e.g. logs with tracebacks) are
not trucated to a single line in the status.log file.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@941 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
119c12a73ec74ea7f259f7db39d994055db1b609 12-Nov-2007 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Extract the logging decorator used in kernel.py into a common library
and add it to Autotest.install to provide logging of autotest
installation failure (and success).

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@937 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
3850f8930fe51e076baa5402202960e77225604d 05-Nov-2007 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Fix stupid parenthesis typos

Signed-off-by: Martin J. Bligh <mbligh@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@923 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
07c1eacb1d032e63884df9da1b1f10823608b7ab 05-Nov-2007 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Change the reboot wait time inside the autotest client from
5 to 30 minutes. There's seperate locations for server-side and
client-side induced reboots.

Signed-off-by: Martin Bligh <mbligh@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@900 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
40f122af3435761f0211cf3b473d45d397526396 04-Nov-2007 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Allow site-local extensions to autoserv, and a custom site install
of the autotest client

From: Steve Howard <showard@google.com>
Signed-off-by: Martin Bligh <mbligh@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@876 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
0e4613bf6bc2fd5e5e3c327dbf568c305ede9681 29-Oct-2007 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Add run_timed_test to autotest.py kill tests after a certain timeout.

Changed utils.run() to be more powerful, and then changed autotest.py to use the new functionality. utils.run() uses the same codepath for both the timeout and the non-timeout case. This path reads and reports data from stdout and stderr as it becomes available. It also supports an arbitrary file-like tee object for both stdout and stderr.

From: Steven Howard <showard@google.com>
Signed-off-by: Martin J. Bligh <mbligh@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@857 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
ba81c688b2720b702a2d08d7c165485f4c0cb40e 25-Oct-2007 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Fixed SSHHost.hardreset to raise an exception if the operation is unsupported
on the remote host.

From: John Admanski <jadmanski@google.com>
Signed-off-by: Martin J. Bligh <mbligh@google.com>

-This line, and those below, will be ignored--

M server/autotest.py
M server/hosts/ssh_host.py
M server/errors.py


git-svn-id: http://test.kernel.org/svn/autotest/trunk@843 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
cc53b353e421034eab903ae810777183202b0333 24-Oct-2007 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Hook site.env from host local extensions

From: adrianbg@google.com
Signed-off-by: Martin J. Bligh <mbligh@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@832 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
d2fc50f1fb970249a2787ac934387f46866da34f 24-Oct-2007 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> This includes several changes to status and console logging:
- SSHHost now turns on console and netconsole logging by default
- autotest writes status logs to status.log instead of just echoing them to stdout
- logging is now unbuffered so that services monitoring the log files see new logs as they come in

From: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@829 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
9708f739de2474408ac75eb90b3611b558f12cdb 18-Oct-2007 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> At some point, parallel_simple was broken. This updated patch fixes this
problem and a couple other potential problems. After this patch, there
should be no more silly uses of argv[0].

From: Jeremy Orlow <jorlow@google.com>
From: Steven Howard <showard@google.com>
Signed-off-by: Martin J. Bligh <mbligh@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@817 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
f1c52841de1cad263a0ec83ee551deeff0cc0bb2 16-Oct-2007 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Move the meat of autoserv into a separate job class, and provide logging
functionality to a server-level status file

Signed-off-by: Martin J. Bligh <mbligh@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@801 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
8fc0e5a4b697b886939d07228344fdda7b1e4aac 11-Oct-2007 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> The server copies it's client directory out to the clients ... if
that's not clean (has precompiled binaries in), it can cause a
variety of problems. Make some attempt to clean it first.

Signed-off-by: Martin J. Bligh <mbligh@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@790 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
7fdd16938507ba8cf362fcf882118a702a1a443c 02-Oct-2007 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> When calling autotest.run_test with a host parameter, pass that on to autotest.install. I believe these are the only 2 instances where we (incorrectly) don't pass on the host parameter.

Signed-off-by: Martin J. Bligh <mbligh@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@739 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
55a2a3baa7f51fec91a5d7bee2da1096e181863c 30-Sep-2007 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Default resultsdir to '.' - makes it easy when using the scheduler

Signed-off-by: Martin J. Bligh <mbligh@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@718 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
913349086bee255028ae2fc0277d87c841b076bd 28-Sep-2007 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Add automating hooks to autotest class to do install and get

Signed-off-by: Martin J. Bligh <mbligh@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@711 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
7386abcea7029addcc99c09240319f7659de77ad 31-Aug-2007 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Make sure host is up before installing autotest

Signed-off-by: Ryan Stutsman <stutsman@google.com>




git-svn-id: http://test.kernel.org/svn/autotest/trunk@653 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
271d5af3050de888b5d63de7fa799a1d0459082f 10-Aug-2007 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> This patch adds test argument passing support to autotest.run_test(). I
need this for the test suite I'm creating. Basically, what I'm trying to
do is this:

From: Mandeep Singh Baines (msb@google.com)




git-svn-id: http://test.kernel.org/svn/autotest/trunk@625 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
629e39e56c2ded85ca5fda03c3d48d05a0acab4b 10-Aug-2007 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> This patch makes two non-functionnal changes to autotest.py in autoserv.

From: Benjamin Poirier <poirier@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@624 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
6e37245a18a23cf3093b66be065370de486fa1f4 10-Aug-2007 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> add newline to server run_test's control file

Signed-off-by: Martin J. Bligh <mbligh@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@619 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
e4f015133f9174b0aea88235a4182c32b8b1cc12 10-Aug-2007 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Automatically install autotest if verify_machine fails

Signed-off-by: Martin J. Bligh <mbligh@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@618 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
8d7e347959aeac0fd940853d618541f6ac8ffce5 10-Aug-2007 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> unmount $AUTODIR/tmpdir in verify_machine

Signed-off-by: Martin J. Bligh <mbligh@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@617 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
dd81ef0606d45d3eb49908f1106f5dd561f268f3 10-Aug-2007 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Allow optional host argument to autotest class

Signed-off-by: Martin J. Bligh <mbligh@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@616 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
b84a1cfb02d9ebab1966c12e58f74b6919b63250 10-Aug-2007 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Make sure we install autotest before using it

Signed-off-by: Martin J. Bligh <mbligh@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@615 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
dc735a2c6c48a16666198b0a3078a1834630ced8 02-Aug-2007 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> clean up spacing everywhere

Signed-off-by: Martin J. Bligh <mbligh@google.com>

Multiline bash scripts turned into triple-quoted strings



git-svn-id: http://test.kernel.org/svn/autotest/trunk@606 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
928c0368a452961d3dae7057896dbad0e04a0132 31-Jul-2007 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> In the first line changed the if needs to be invoked when hasattr
not when not hasattr. In the second case self refers to _Run
when it should have referred to a Host object.

Signed-off-by: Ryan Stutsman <stutsman@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@601 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
3ceb720bef5ac3595577aea3aaea4db6759e711d 27-Jul-2007 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> If a host object has an env attribute it will be prepended to autotest
invocations so client environment can be modified before tests are
executed. This make it easy to customize the autotest environment
on a per host basis.

Signed-off-by: Ryan Stutsman <stutsman@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@598 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
dbe4a38e59611645868b1d7dd89145c74adbf039 26-Jul-2007 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Currently if a host is down or doesn't have an accessible sshd attempts
to execute tests on the client fail. This causes entire queues of tests
to fail when a single bad kernel is run. This blocks tests from
starting if the machine isn't in a good state. More pixie dust can
probably be added to this to make it smarter, but this at least prevents
cacading failures.

Signed-off-by: Ryan Stutsman <stutsman@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@594 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
f238b45f49b0a69363d7c4316344435b20855edf 25-Jul-2007 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> This small patch is to properly escape an mkdir command in autotest.py

From: Benjamin Poirier <poirier@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@588 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
d54832be0608b7381cba6b4ac9dae92a4bb2715d 25-Jul-2007 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Add a run_test convenience method to Autotest server to run tests by name

Signed-off-by: Ryan Stutsman <stutsman@google.com>

Requested by mbligh. Adds run_test which can run tests by name with
optional parameters without an autotest control file.

Example:
m = hosts.SSHHost('somehost')
a = autotest.Autotest()
a.run_test('kernbench', '/tmp/results', m, iterations=2, threads=16)

Note: The argument order will change moving the host to the first
argument as per a request from mbligh in a follow-up patch to run_test
and run.



git-svn-id: http://test.kernel.org/svn/autotest/trunk@585 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
c8949b85f271afc332b404624f48d2bcbf9789da 23-Jul-2007 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Add autotest support in server side

From: poirier@google.com

This patch adds some functionality to autotest in autoserv. Credit goes
to Ryan for some of this.

Autotest support in autoserv:
* automatically hardreset the machine if it never comes back online
* autotest now uses get() interface
* remove autotest results directory if it is there before run (to avoid
mixing things up)

utils has unarchive code (to extract tar bz2 gz)
DEBKernel uses get() interface



git-svn-id: http://test.kernel.org/svn/autotest/trunk@570 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py
dcd57a85fc9233c4dc9f45da55338a2c09f5a37e 12-Jul-2007 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Add initial version of autoserv



git-svn-id: http://test.kernel.org/svn/autotest/trunk@557 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/server/autotest.py