History log of /external/autotest/client/common_lib/cros/dev_server.py
Revision Date Author Comments (<<< Hide modified files) (Show modified files >>>)
e6803234ac5233c48ff50b57460d42641e04a52b 20-Jan-2016 Dan Shi <dshi@google.com> [autotest] Add switch to enable/disable devserver and drone election in restricted subnet.

This allows devserver and drone election in restricted subnet to be enabled/
disabled separatedly.

BUG=chromium:574872
TEST=unittest

Change-Id: If2c3d1ad1bd9be5546bf2f8a027bcd88627ae2c5
Reviewed-on: https://chromium-review.googlesource.com/322468
Commit-Ready: Dan Shi <dshi@google.com>
Tested-by: Dan Shi <dshi@google.com>
Reviewed-by: Dan Shi <dshi@google.com>
/external/autotest/client/common_lib/cros/dev_server.py
114e17228efd62ab595690be30cb1e3f26fabebe 11-Jan-2016 Dan Shi <dshi@google.com> [autotest] Support selecting drone in restricted subnet

For agent task uses host in restricted subnet, only use drone in the subnet.
For agent task uses host NOT in restricted subnet, only use drones NOT in any
restricted subnet.

BUG=chromium:574872
TEST=local run, unittest

Change-Id: I3492fe14660e7629f982937d428d230ca9dcf3dc
Reviewed-on: https://chromium-review.googlesource.com/321116
Commit-Ready: Dan Shi <dshi@google.com>
Tested-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Fang Deng <fdeng@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
a28f62fb9fa0df4432d35766e22282e9e146e373 15-Jan-2016 Dan Shi <dshi@google.com> [autotest] Fix the logic to prefer devserver in local network.

The behavior was changed when the work to support selecting devserver in
restricted subnet is done. This change re-enable the feature so we can enable
this in the lab to reduce cross-lab traffic.

BUG=chromium:573011
TEST=unittest

Change-Id: Idc91297c1db3ea483b654d6457269f558d0903fe
Reviewed-on: https://chromium-review.googlesource.com/321868
Commit-Ready: Dan Shi <dshi@google.com>
Tested-by: Dan Shi <dshi@google.com>
Reviewed-by: Kevin Cheng <kevcheng@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
2bc5be69ce44e4d5b0582f833b04e141a0e4f502 08-Jan-2016 Dan Shi <dshi@google.com> [autotest] Only select from unrestricted subnet if host is not in restricted subnet.

Otherwise drone will keep calling check_health on devservers in restricted
subnet, which is a waste of time.

BUG=chromium:571651
TEST=unittest

Change-Id: I02c6ba2ff6d3afbaf9fab5d2dac1905479352e8e
Reviewed-on: https://chromium-review.googlesource.com/321170
Commit-Ready: Dan Shi <dshi@google.com>
Tested-by: Dan Shi <dshi@google.com>
Reviewed-by: Dan Shi <dshi@google.com>
/external/autotest/client/common_lib/cros/dev_server.py
216389cfb1c687a90581cfdd62cbff0535bb6c1f 22-Dec-2015 Dan Shi <dshi@google.com> [autotest] Locate devserver in restricted subnets

Add a new config (CROS/restricted_subnets) to specify a list of subnets that
requires devserver within the same subnet.

In devserver.ImageServer.resolve function, always pass in the hostname to check
subnet restriction. This does add a bit overhead to do IP lookup.

BUG=chromium:571651
TEST=local test

Change-Id: Id3e47c315523d6bc7d40e1c5fbedd1733147b8b9
Reviewed-on: https://chromium-review.googlesource.com/319612
Commit-Ready: Dan Shi <dshi@google.com>
Tested-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
ab999721eeb8ddddf42bfa49cc25da1233238535 04-Dec-2015 Dan Shi <dshi@google.com> [autotest] Enable provisioning Brillo device from autotest

This change adds the support of provision Brillo device from autotest. Methods
are added to adb_host, so machine_install knows how to provision a Brillo
device with given build, e.g., git_mnc-brillo-dev/dragonboard-userdebug/LATEST.

BUG=chromium:548881
TEST=moblab setup, set build to git_mnc-brillo-dev/dragonboard-userdebug/LATEST
http://dshi.mtv/afe/#tab_id=view_job&object_id=3066
also make sure the change doesn't break Android provision
http://dshi.mtv/afe/#tab_id=view_job&object_id=3027

Change-Id: I299349301067572e7f4ee6792e19da3853e657f3
Reviewed-on: https://chromium-review.googlesource.com/315775
Commit-Ready: Dan Shi <dshi@google.com>
Tested-by: Dan Shi <dshi@google.com>
Reviewed-by: Simran Basi <sbasi@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
ba943535dbec910f88b29e112039fe92ddac4d71 03-Dec-2015 Dan Shi <dshi@google.com> [autotest] Add support of LATEST for android build.

With this change, android build can have LATEST as build id.

BUG=chromium:548881
TEST=local
http://dshi/afe/#tab_id=view_job&object_id=3017

Change-Id: I54b416548be5003f592e68fa6cb2d91034e14a5b
Reviewed-on: https://chromium-review.googlesource.com/315680
Commit-Ready: Dan Shi <dshi@google.com>
Tested-by: Dan Shi <dshi@google.com>
Reviewed-by: Fang Deng <fdeng@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
5563b77f2029cfe97df9d719a87cc4a6f9fad5f1 12-Nov-2015 Dan Shi <dshi@chromium.org> [autotest] Update devserver to support stage request of Android build.

BUG=chromium:512668
TEST=local test script:
TARGET = 'shamu-userdebug'
BUILD_ID = '123'
BRANCH = 'master'

ARTIFACTS = ['bootloader_image', 'radio_image', 'zip_images']

devserver = dev_server.ImageServer('http://localhost:8082')
devserver.stage_artifacts(image='daisy-release/R47-7520.30.0',
artifacts=['control_files'])
devserver.trigger_download(image='daisy-release/R47-7520.30.0')
devserver.finish_download(image='daisy-release/R47-7520.30.0')

devserver = dev_server.AndroidBuildServer('http://localhost:8082')
devserver.trigger_download(target=TARGET, build_id=BUILD_ID,
branch=BRANCH)
devserver.finish_download(target=TARGET, build_id=BUILD_ID,
branch=BRANCH)

Change-Id: Ieb4bbbdbf0e6a8e38e99435b273dfc3c6845ac83
Reviewed-on: https://chromium-review.googlesource.com/312138
Commit-Ready: Dan Shi <dshi@chromium.org>
Tested-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Fang Deng <fdeng@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
76adb264a8e007b122f084cc8c0572daaa43751e 24-Sep-2015 Fang Deng <fdeng@chromium.org> [autotest] set symbolication timeout to 10mins and modify stats

Currently timeout is 20 which is too long.
A lot of builders have timed out.

Set timeout to 10mins and modify some stats.

TEST=Initialize a CrashServer and symbolicate a file.
BUG=chromium:532559,chromium:535671

Change-Id: I96c8f433d0ed3a63df6a44ff529dcac6523bd24f
Reviewed-on: https://chromium-review.googlesource.com/302144
Commit-Ready: Fang Deng <fdeng@chromium.org>
Tested-by: Fang Deng <fdeng@chromium.org>
Reviewed-by: Dan Shi <dshi@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
b9730f5ed07e99149d3b545cfc495e59e28e54a2 03-Sep-2015 Dan Shi <dshi@chromium.org> [autotest] Prefer devserver in the same subnet

BUG=chromium:421123
TEST=local run

Change-Id: I8b1235381ae772996f4322b21b2ef268dd9eba04
Reviewed-on: https://chromium-review.googlesource.com/297174
Commit-Ready: Dan Shi <dshi@chromium.org>
Tested-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Fang Deng <fdeng@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
cd756470f9583966c41a59ab1e25d773b701a119 26-Aug-2015 J. Richard Barnette <jrbarnette@chromium.org> [autotest] Remove obsolete "compatible_artifacts" handling.

This removes obsolete code related to handling different artifacts
generated by older version of the builders. The code was causing
devserver timeout error to be mis-reported.

N.B. This means it is no longer possible to stage builds from before
R40.

BUG=chromium:525186,chromium:433436
TEST=None

Change-Id: Ie4d7fc72b11dbfa68ada42f308afcba34867a907
Reviewed-on: https://chromium-review.googlesource.com/295750
Commit-Ready: Richard Barnette <jrbarnette@chromium.org>
Tested-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Richard Barnette <jrbarnette@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
61dd885dbfb51b8b1ed9f91ef8ad1cb7d863cb78 02-Jun-2015 Dan Shi <dshi@chromium.org> [autotest] Add a method to select least loaded devserver

This CL refactors how devserver_healthy is called and add a new method,
get_least_loaded_devserver to get the least loaded devserver to use.

The logic in _compare_load needs to be polished after we collect more
data in production devservers.

BUG=chromium:486278
TEST=unittest, setup local devserver, run script
import common
from autotest_lib.client.common_lib.cros import dev_server
print dev_server.get_least_loaded_devserver()

Change-Id: Iadd38e93a12fc5be93b25bf0e8a5836db864517c
Reviewed-on: https://chromium-review.googlesource.com/274873
Trybot-Ready: Dan Shi <dshi@chromium.org>
Tested-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
e749ff8acf12e19a31954c89e5bfbf294f9127bd 27-May-2015 Dan Shi <dshi@chromium.org> [autotest] Report all states collected by devserver's check_health call.

This will collect all performance data from devservers.

BUG=chromium:486278
TEST=local test run to access devserver

Change-Id: If3db95010b36c700dd52dd50f44eb0e971d26955
Reviewed-on: https://chromium-review.googlesource.com/273552
Trybot-Ready: Dan Shi <dshi@chromium.org>
Tested-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
848953993452a7d0d8cc913093a14e7f281a171b 30-Apr-2015 J. Richard Barnette <jrbarnette@chromium.org> [autotest] Update servo based on the stable_version table.

This switches servo_host updates to use the afe_stable_versions
database table to select the target version for update. This is
the same table used for selecting repair images for DUTs.

Previously, selection was done by asking the devserver for the
latest staged image for beaglebone_servo. This would fail if the
devserver had not already staged the image.

Additionally, this removes some outstanding dead code that was
previously used to find the URL for a test image for use in servo
repair.

BUG=None
TEST=Force the flow on a local autotest instance; see servo update

Change-Id: I5cc4395c9b173e1dace13eee97f337b7f0a58b26
Reviewed-on: https://chromium-review.googlesource.com/268543
Tested-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Fang Deng <fdeng@chromium.org>
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
1e1c41b1b4a1b97c0b7086b8430856ed45e064d3 05-Feb-2015 Gabe Black <gabeblack@chromium.org> graphite: Separate out configuration from the statsd classes.

The new version of the statsd classes should be created using an instance of
the new Statsd class which sets up some defaults without having to specify
them over and over. This makes it essentially compatible with the existing
usage in autotest, but will allow chromite to configure things differently and
avoid having side effects from importing the module or global state.

BUG=chromium:446291
TEST=Ran unit tests, ran stats_es_functionaltest.py, ran the
stats_mock_unittest, ran a butterfly-paladin tryjob with --hwtest, testing by
fdeng.
DEPLOY=apache,scheduler,host-scheduler

Change-Id: I1071813db197c0e5e035b4d8db615030386f1c1c
Reviewed-on: https://chromium-review.googlesource.com/246428
Reviewed-by: Fang Deng <fdeng@chromium.org>
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Gabe Black <gabeblack@chromium.org>
Tested-by: Gabe Black <gabeblack@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
25e7a922e3f5283539ed9a2f232299d02bc2df30 31-Oct-2014 Simran Basi <sbasi@google.com> [autotest] Support for new autotest artifacts.

This change updates dynamic_suites and provisioning to support the new
autotest build artifacts: control_files.tar and autotest_packages.tar.

In case these artifacts do not exist, it will then attempt to stage the
old autotest.tar artifact, thus still supporting older builds.

BUG=chromium:421122
TEST=Custom moblab build that successfully ran smoke suite against
2 images: 1 with autotest.tar and the other with the new artifacts.
CQ-DEPEND=CL:226822

Change-Id: I67898347d40c55a3ab1df89469bcc933b0b765b8
Reviewed-on: https://chromium-review.googlesource.com/231990
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Simran Basi <sbasi@chromium.org>
Tested-by: Simran Basi <sbasi@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
dc8c25da3dcb4a6d6356f1e950299b3b472a531b 18-Nov-2014 Fang Deng <fdeng@chromium.org> Revert "[autotest] Support for new autotest artifacts."

This reverts commit 747854e979de7b52f8a91883e8b5e1d78132e499.

can't figure out the root cause. reverting for now to save paladin builds. crbug.com/434168

Change-Id: Ie115b90a2a74a8daca28ecfe1e164b38320f74aa
Reviewed-on: https://chromium-review.googlesource.com/230345
Reviewed-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Fang Deng <fdeng@chromium.org>
Tested-by: Fang Deng <fdeng@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
747854e979de7b52f8a91883e8b5e1d78132e499 31-Oct-2014 Simran Basi <sbasi@google.com> [autotest] Support for new autotest artifacts.

This change updates dynamic_suites and provisioning to support the new
autotest build artifacts: control_files.tar and autotest_packages.tar.

In case these artifacts do not exist, it will then attempt to stage the
old autotest.tar artifact, thus still supporting older builds.

BUG=chromium:421122
TEST=Custom moblab build that successfully ran smoke suite against
2 images: 1 with autotest.tar and the other with the new artifacts.
CQ-DEPEND=CL:226822

Change-Id: If66026b46f77aaccaa83996e20a6749bad2b1625
Reviewed-on: https://chromium-review.googlesource.com/226823
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Simran Basi <sbasi@chromium.org>
Tested-by: Simran Basi <sbasi@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
e4cb9e23709425fcbb5faec40bfe824b818bd106 29-Aug-2014 Dan Shi <dshi@chromium.org> [autotest] Change devserver stats call to log data only for staging artifacts.

File names are different as they have build information included. Recording
stats for each file will create too many counters in graphite and lead to disk
space issue.

BUG=chromium:404475
TEST=local setup.
To verify metadata:
visit http://172.25.61.45:9200/_plugin/elastic-hammer/
update search url with index of local setup: dshi.mtv/_search
search for:
{
"query": {"bool": {"minimum_should_match": 1,
"should": [{"term": {"_type": "devserver"}}
]}},
"size": 10000,
"sort": [{"time_recorded": "asc"}]}

Confirm the data.

Change-Id: Ic11f76c3ef6fbf8cf5d25312d3285e1e9b87a178
Reviewed-on: https://chromium-review.googlesource.com/215640
Tested-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
2ce1ca5b31250354183250892f937465b8922c66 15-Aug-2014 Fang Deng <fdeng@chromium.org> [autotest] Add logging and stats in dynmaic suite and provisioning

Add logging/stats to suite:find_and_parse_tests,
dev_server:stage_artificats, and provision_AutoUpdate.py

BUG=chromium:404202
TEST=run suite.

Change-Id: I99414651b75c828f5d2ae6b3531c85845a3f91db
Reviewed-on: https://chromium-review.googlesource.com/212727
Reviewed-by: Prashanth B <beeps@chromium.org>
Commit-Queue: Fang Deng <fdeng@chromium.org>
Tested-by: Fang Deng <fdeng@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
da8c60af1e1e3ee97170c700d0b72991687e35a2 03-Jun-2014 Michael Liang <michaelliang@chromium.org> [autotest] Migrate graphite directory to client/common_lib/cros

This change allows us to report stats in client tests.
1. Change import paths for all files that import modules from graphite
2. Clean up some unused modules
Related CL: https://chromium-review.googlesource.com/#/c/202467/
BUG=chromium:237255
TEST=Ran scheduler locally, scheduled reboot jobs, verified stats such as monitor_db_cleanup.user_cleanup._cleanup were reported on chromeos-stats.
DEPLOY = apache, scheduler, host_scheduler
Change-Id: Iebfe3b8acc1c363a0b70ea555744e85d1367cb67
Reviewed-on: https://chromium-review.googlesource.com/202727
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Michael Liang <michaelliang@chromium.org>
Tested-by: Michael Liang <michaelliang@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
b7d2116ad62dd09c52791c8f27dfa61b9df186b9 22-May-2014 Simran Basi <sbasi@chromium.org> [autotest] Be able to specify LATEST build as target build.

This change allows the user to specify running a suite against
a build name in the format [builder]/LATEST. This allows for users
to just use the latest build without looking it up.

BUG=chromium:375354
TEST=Tested running from run_suite, and afe. Then created a recurring
job that worked as desired.
DEPLOY=apache
CQ-DEPEND=Iddc2ab2b0675591b53311ebe9e1692d0c0859377

Change-Id: I5c0cd9ad06bd016b539902248e193115704ae243
Reviewed-on: https://chromium-review.googlesource.com/200822
Reviewed-by: Simran Basi <sbasi@chromium.org>
Tested-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Simran Basi <sbasi@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
bd39ed3d82ed78ed86f7a79b6f0c4e857fa8170d 06-Mar-2014 Prashanth B <beeps@google.com> [autotest] Client devserver call to list contents of a staged image.

Usage:
ds = dev_server.ImageServer.resolve(1)
ds.list_image_dir('<board>-<branch>/R<Milestone>-<buildno>')

TEST=Executed the call against a local devserver.
BUG=chromium:348097, chromium:349044, chromium:349047
CQ-DEPEND=CL:189672
Change-Id: I8f60585253d61c779383f1a2ab71cfa8e3d0a0b6
Reviewed-on: https://chromium-review.googlesource.com/189674
Reviewed-by: Prashanth B <beeps@chromium.org>
Commit-Queue: Prashanth B <beeps@chromium.org>
Tested-by: Prashanth B <beeps@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
eada904504e23f528cd851f096326bb9850ee3fa 03-Dec-2013 Dan Shi <dshi@chromium.org> [autotest] beaglebone: devserver staging script

Script to stage servo image in all devservers.

This script should be executed in a server running autotest scheduler. It is
used to accomplish following tasks:
1. Get the latest staged build in devserver as the build that's currently used
by servo.
2. Stage the latest or a specific beaglebone-release build from Google Storage
in all devservers.

BUG=chromium:272351
TEST=in local setup of autotest and devserver. Confirm:
1. File chromiumos_base_image.bin and image.zip are staged in all devservers.
e.g., in folder images/beaglebone-release/R33-5045.0.0/
2. Test with trybot build trybot-beaglebone-release/R33-5060.0.0-b17
confirm update.gz is staged.
For a full coverage test, try following cases:
1. run site_utils/stage_servo_image.py, which stages the latest build
2. run |site_utils/stage_servo_image.py -i [some old build]|, confirm error
message about newer build found on devservers is shown.

CQ-DEPEND=CL:178850

Change-Id: I4eba5d5da1f985a14a5ff9a553430c073ffa4c46
Reviewed-on: https://chromium-review.googlesource.com/178534
Tested-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Chris Sosa <sosa@chromium.org>
Reviewed-by: Prashanth B <beeps@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
04cced6097195aec700976e9c7134cfacc2ba3cd 04-Nov-2013 Fang Deng <fdeng@chromium.org> [autotest] Do not use call_and_wait on devserver rpc call 'setup_telemetry'

Currently, client.common_lib.cros.dev_server:setup_telemetry is broken.
It calls call_and_wait with arg
artifacts=_ARTIFACTS_TO_BE_STAGED_FOR_IMAGE. However, setup_telemetry
rpc method won't stage all artifacts specified in the constant. The
client will wait forever for those artifacts to be staged.

Because setup_telemetry method in platform/dev/dev_server.py won't
respect 'async' arg, there is simply no need to wait for this call
as it will always be a synchronous call. So the fix would
be just not using call_and_wait for setup_telemetry.

BUG=chromium:314213
TEST=setup a local devserver, make sure nothing was staged,
run a test in telemetry_Benchmarks with atest;
confirm that the test does not stuck at setup_telemetry any more.
Run perf_v2 suite, confirm that setup_telemetry succeeds.

Change-Id: I203bbd41da5c8c692cb15302746ed344cba3023f
Reviewed-on: https://chromium-review.googlesource.com/175650
Reviewed-by: Chris Sosa <sosa@chromium.org>
Tested-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Fang Deng <fdeng@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
12c0a3cef9611d155d157651af7ca7afa9a90c96 03-Sep-2013 beeps <beeps@chromium.org> [autotest] Stage factory artifacts from the canary branch.

This cl enables us to stage artifacts form either the normal
release branch, or a canary branch, based on the type of artifact.

BUG=chromium:280889
TEST=Staged factory images from the canary branch.

Change-Id: If9a959d0847fecf4aded600974cf45e6e05c9ac5
Reviewed-on: https://chromium-review.googlesource.com/167884
Reviewed-by: Prashanth B <beeps@chromium.org>
Tested-by: Prashanth B <beeps@chromium.org>
Commit-Queue: Prashanth B <beeps@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
e539be0567f5159e11107580df4d55c292577e15 01-Aug-2013 beeps <beeps@chromium.org> [autotest] Teach the devserver to stage factory images.

To stage a factory image the client request needs to
be made with the url to the canary-channel.

CQ-DEPEND=CL:I51f710996e63b4a8d3a772688cdc5643a423b511
TEST=Staged images from the factory branch.
BUG=chromium:252503

Change-Id: I34e5a6db44ee3de0705115ea8c665aae7fb5b416
Reviewed-on: https://chromium-review.googlesource.com/64155
Reviewed-by: Dan Shi <dshi@chromium.org>
Tested-by: Prashanth Balasubramanian <beeps@chromium.org>
Commit-Queue: Prashanth Balasubramanian <beeps@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
15ebb06f3254603f77e1099bea66a2e7fa95e95c 19-Aug-2013 Dan Shi <dshi@chromium.org> [autotest] raise DevServerException in wait_for_artifacts_staged.

When devserver responds with HTTPError, the rpc call is failed with exception.
The exception should be raised to prevent caller waiting indefinitely. When the
caller runs into connection problem, URLError that's not HTTPError may be
thrown. In such case, caller should retry the devserver call.

This CL must be work together with the CL in devserver:
https://gerrit.chromium.org/gerrit/66239

BUG=chromium:273981
DEPLOY=must repo sync devserver first (CL: 66239)
TEST=test with three different cases:
1. run_suite for a legit build, e.g.,
./run_suite.py --build=lumpy-release/R30-4501.0.0 --board=lumpy --suite=dummy
wait for artifactions to download, suite job just be created successfully.
2. run_suite for a legit build, e.g.,
./run_suite.py --build=lumpy-release/R30-4501.0.0 --board=lumpy --suite=dummy
simulate connection problem by killing devserver right after downloading starts.
Restart devserver after 30 seconds or so. Artifacts should be successfully
downloaded and suite job should be created.
3. run_suite for a non-existing build, e.g.:
./run_suite.py --build=lumpy-release/R30-no_such_build --board=lumpy --suite=dummy
run_suite command should fail with following error in 5-10 seconds:
GSUtilError: Failed to list payloads. GSUTIL cmd gsutil ls gs://chromeos-image-archive/lumpy-release/R30-no_such_build/* failed with return code 1:
CommandException: One or more URIs matched no objects.

Change-Id: I7bcefce648d77cbd72b067a45f078df61d3e7082
Reviewed-on: https://gerrit.chromium.org/gerrit/66240
Reviewed-by: Dan Shi <dshi@chromium.org>
Tested-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
2f1ae9f1f2464ebbeb226a6669fa90ef211a179c 13-Aug-2013 Chris Sosa <sosa@chromium.org> Add new files logic to devserver and clean up AU E2E test to use it.

There's a couple nested things here:

1) Pipe files through the autotest-side of the devserver logic. I've gone ahead
and added 5 unittests to test this new workflow with various combos of
artifacts. The main thing here is the optional pass of archive_url + files to
the stage_artifacts methods. I've also added a convenience function to return
the url to any staged file on the devserver if you know the filename (which you
would if you are using files= API).

2) Use the new logic in AU E2E API and figure out the right URL's to pass to
stage artifacts from. This was a bit of refactoring to get right.
In order to allow me to serve any artifact I had to change a bit of of the
source version installation logic to allow me to update from any static url
(rather than one that must end in update.gz). This actually makes the
OmahaDevserver much more flexible and more Omaha-y.

BUG=chromium:267896
TEST=Unittests + tested with both an nmo + npo full run.

Change-Id: Ic490c7c0e7d76eb8c0416accefbe113b9c0bba32
Reviewed-on: https://gerrit.chromium.org/gerrit/65732
Reviewed-by: Chris Sosa <sosa@chromium.org>
Tested-by: Chris Sosa <sosa@chromium.org>
Commit-Queue: Chris Sosa <sosa@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
0c865033c1cd2614036e83d57bb03a2c04f2b7af 30-Jul-2013 beeps <beeps@chromium.org> [autotest] Don't swap devservers when verifying repo_url.

Since we're seeing more flake than we would like when
checking the health of a devserver we end up swapping
devservers and restaging contents, even when the initial
devserver is healthy. This leads to more load, which
further exacerbates the problem of flakyness, leading to
a vicious cycle. This cl restages autotest on the
job_repo_url devserver, and fails verification if the
devserver embedded in the url is down.

BUG=None
TEST=set job_repo_url and deleted directories on local
devserver, checked that we re-stage. Didn't start
up local devserver and checked that we received
an URLError. Killed the devserver in the middle
of a request and checked that the test Failed.

Change-Id: I18843d3047122b9ffea3b2b1305fb0332172b19d
Reviewed-on: https://gerrit.chromium.org/gerrit/63758
Tested-by: Prashanth Balasubramanian <beeps@chromium.org>
Reviewed-by: Alex Miller <milleral@chromium.org>
Commit-Queue: Prashanth Balasubramanian <beeps@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
e9be01e817f8ad0824bc37c0d930876c1b3c043b 09-Aug-2013 Dan Shi <dshi@chromium.org> [autotest] Fix setup_telemetry call

Fix the missing argument in call_and_wait.

BUG=none
TEST=./run_suite.py --build=trybot-lumpy-release/R30-4514.0.0-b1415 --board=lumpy --suite=push_to_prod -u 4

Change-Id: I543591768e8ce8b43d4fafaa642a99af37fecb15
Reviewed-on: https://gerrit.chromium.org/gerrit/65255
Reviewed-by: Alex Miller <milleral@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
Tested-by: Dan Shi <dshi@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
b81d24f51d4b7b8d4b7142fe7b0ba53bea988508 08-Aug-2013 Dan Shi <dshi@chromium.org> [autotest] fix missing constant _ARTIFACTS_TO_BE_STAGED_FOR_IMAGE_WITH_TELEMETRY

It should be replaced with _ARTIFACTS_TO_BE_STAGED_FOR_IMAGE_WITH_AUTOTEST

BUG=chromium:270148
TEST=none

Change-Id: I01c492c42c67ec181a9d73c67165de0b7d9111e5
Reviewed-on: https://gerrit.chromium.org/gerrit/65219
Reviewed-by: Alex Miller <milleral@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
Tested-by: Dan Shi <dshi@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
2e7b2ea7850bdd3ca92a5ef47f11d91a08c2b3db 06-Aug-2013 Dan Shi <dshi@chromium.org> [autotest] Use async poller to check if files are staged on devserver

In devserver rpc, stage, download, wait_for_status, set async=True, and poll
is_staged on devserver to check if artifacts are all staged. This approach is
implemented to avoid holding on cherrypy thread for too long.

BUG=chromium:266443
TEST=unittest, run_suite in local setup
DEPLOY=devserver, must push devserver change (cl 64310) first.

Change-Id: I9b38754bba197ba67d03eb3462e04672d0ad479c
Reviewed-on: https://gerrit.chromium.org/gerrit/64706
Tested-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
Reviewed-by: Dan Shi <dshi@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
6c00dde741fb42fed2d8bb25c30990557b288097 30-Jul-2013 Dan Shi <dshi@chromium.org> [autotest] re-throw exceptions properly in urlopen_socket_timeout

urlopen_socket_timeout call handles all urllib2.URLError but only re-throw
timeout exception. Change the code to re-throw other type of exception so caller
can have a better idea of what went wrong.

BUG=none
TEST=tested with local devserver

Change-Id: I8c60f75c1fc2b8bff11a710735ce4ff8800ec15f
Reviewed-on: https://gerrit.chromium.org/gerrit/63679
Commit-Queue: Dan Shi <dshi@chromium.org>
Reviewed-by: Dan Shi <dshi@chromium.org>
Tested-by: Dan Shi <dshi@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
c594c1c3ed039635462371af90562475fcee575e 10-Jul-2013 beeps <beeps@chromium.org> [autotest] Pipe suite_name through client devserver code.

To reduce the overhead involved with fetching control files
from the devserver we generate a mapping of suite->control
files at build time. This cl will ask the control file getter
to fetch the control files for a given suite instead of all
the control files associated with an image.

Also removing SUITE = None from some networking test control
files, and patched unittests.

BUG=chromium:252398
TEST=Ran a suite with many control files. Made sure the suite->
control file mapping falls back to the old code when a bad suite
name is specified. Compared the test objects created in the old
version vs those created with this change.
Ran unittests.

CQ-DEPEND=CL:I6b590c8c40a863e6744875a26ac228ffd4dd8794

Change-Id: I4274c78764ffaaabef1ed48cdb40f523e307938d
Reviewed-on: https://gerrit.chromium.org/gerrit/63182
Tested-by: Prashanth Balasubramanian <beeps@chromium.org>
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Prashanth Balasubramanian <beeps@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
49150a94e707dcbed7a46817bb58d5aae7d8fd28 25-Jul-2013 Dan Shi <dshi@chromium.org> [autotest] add stats for CrashServer.symbolicate_dump

Add a counter to measure how many times CrashServer.symbolicate_dump is called,
also add a timer to measure how long it takes. The counter should tell us if any
symbolicate calls are requested arount similar time.

BUG=chromium:217988
TEST=none
DEPLOY=none

Change-Id: I63be3a0c434112e0644c85c3e976ff26de07b9ac
Reviewed-on: https://gerrit.chromium.org/gerrit/63361
Tested-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
35476b958f33c6bada2932237a97f59b483b451d 24-Jul-2013 Prashanth Balasubramanian <beeps@chromium.org> Revert "[autotest] Pipe suite_name through client devserver code."

This reverts commit 5a80a9be9f8ff74675c8696b9c18dc6b19bb625c

Change-Id: Ic483b7a27b2603f533819cea3df33522596c5848
Reviewed-on: https://gerrit.chromium.org/gerrit/63171
Commit-Queue: Prashanth Balasubramanian <beeps@chromium.org>
Reviewed-by: Prashanth Balasubramanian <beeps@chromium.org>
Tested-by: Prashanth Balasubramanian <beeps@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
03eaad90a2fabae6f3253e9acde8859681237610 26-Jun-2013 joychen <joychen@google.com> Update devserver urls with /archive in autotest(2)

[autotest] Remove archive from the devserver url patterns.
Devserver is removing static/archive so that everything is served
straight from the static directory. Cutting out calls in autotest to
match.

BUG=chromium:214373
TEST=Ran a suite with a reimage job on a DUT with the old
style job_repo_url.

Change-Id: Id3e2e0559c82a5424ef50406c7360ce28ca9a337
Reviewed-on: https://gerrit.chromium.org/gerrit/62918
Reviewed-by: Prashanth Balasubramanian <beeps@chromium.org>
Commit-Queue: Joy Chen <joychen@chromium.org>
Reviewed-by: Joy Chen <joychen@chromium.org>
Tested-by: Joy Chen <joychen@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
5a80a9be9f8ff74675c8696b9c18dc6b19bb625c 10-Jul-2013 beeps <beeps@chromium.org> [autotest] Pipe suite_name through client devserver code.

To reduce the overhead involved with fetching control files
from the devserver we generate a mapping of suite->control
files at build time. This cl will ask the control file getter
to fetch the control files for a given suite instead of all
the control files associated with an image.

Also removing SUITE = None from some networking test control
files, and patched unittests.

BUG=chromium:252398
TEST=Ran a suite with many control files. Made sure the suite->
control file mapping falls back to the old code when a bad suite
name is specified. Compared the test objects created in the old
version vs those created with this change.
Ran unittests.

CQ-DEPEND=CL:I6b590c8c40a863e6744875a26ac228ffd4dd8794
Change-Id: Ic1d65672e8c65b728b9d783f60f3093c7d0d866b
Reviewed-on: https://gerrit.chromium.org/gerrit/61402
Commit-Queue: Prashanth Balasubramanian <beeps@chromium.org>
Reviewed-by: Prashanth Balasubramanian <beeps@chromium.org>
Tested-by: Prashanth Balasubramanian <beeps@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
458bf9ef5cf16bf88b306844a54d073c9cde5856 20-Jul-2013 Richard Barnette <jrbarnette@chromium.org> Revert "Update devserver urls with /archive in autotest."

This reverts commit f67d57a79eab93bf731f5bcf2a48fd04e3e08c68

This seems to be causing bugs in the test environment prior
to pushing to prod.

BUG=chromium:262279
TEST=see bug description

Change-Id: I79c300ad5647d283b862322e7ce29d8b7165766d
Reviewed-on: https://gerrit.chromium.org/gerrit/62708
Commit-Queue: Richard Barnette <jrbarnette@chromium.org>
Tested-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Joy Chen <joychen@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
f67d57a79eab93bf731f5bcf2a48fd04e3e08c68 26-Jun-2013 joychen <joychen@google.com> Update devserver urls with /archive in autotest.

Devserver is removing static/archive so that everything is served
straight from the static directory. Cutting out calls in autotest to
match.

BUG=chromium:214373
TEST=None

Change-Id: Iebc74469c93d0c95b676dcc45045dbcbd089efb5
Reviewed-on: https://gerrit.chromium.org/gerrit/60101
Commit-Queue: Joy Chen <joychen@chromium.org>
Reviewed-by: Joy Chen <joychen@chromium.org>
Tested-by: Joy Chen <joychen@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
60aec2423a2c2a14c46dc21757a6fd748931f428 26-Jun-2013 beeps <beeps@chromium.org> [autotest] Divorce the devserver retry requests and sigalarm.

All dev_server requests made from the client go through a retry
wrapper that uses sigalarm to expedite unresponsive calls. Since
we cannot register our signal handlers with mod wsgi, this cl
modifies the urlopen call to manage an internal socket timeout
instead. In order to continue retrying on other URLErrors, the
urlopen wrapper in site_utils converts socket timeouts to
TimeoutExceptions.

TEST=
1. Added a bad devserver, tried to run_suite on a build that hashed
to said devserver with and without this change and verified that
it only worked in the latter case.
2. Sigalarm still fires when devserver's resolve is called outside wsgi,
and urlopen has a larger timeout.
3. Non TimeoutException URLErrors retry.
4. dev_server_unittest pass.
BUG=chromium:246209

Change-Id: Iec9ef39982f4c8b92e260cf27bf454e624bccfa9
Reviewed-on: https://gerrit.chromium.org/gerrit/58659
Tested-by: Prashanth Balasubramanian <beeps@chromium.org>
Reviewed-by: Alex Miller <milleral@chromium.org>
Commit-Queue: Prashanth Balasubramanian <beeps@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
567c8f02d80562bc9a5df0477ef0424edb7cdcd5 19-Jun-2013 Dan Shi <dshi@chromium.org> [Autotest] Add an option in global config to skip devserver health check

An option, skip_devserver_health_check, is added under [CROS] section to skip
devserver health check. User can set this option to True in shadow config so
that a devserver can run in a local setup with limited disk space.

BUG=chromium:242297
TEST=tested with run_suite with localhost setup as devserver.
DEPLOY=none

Change-Id: Ie3247120d0c17cf4332655050c8a533fc2f9cd64
Reviewed-on: https://gerrit.chromium.org/gerrit/59271
Tested-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Chris Sosa <sosa@chromium.org>
Reviewed-by: Alex Miller <milleral@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
b76e0ee685a5808687d4ce657b39fb4c08647dea 23-May-2013 Chris Sosa <sosa@chromium.org> Add new test that has shorter reboot/down timeouts and increase others.

This CL adds platform_RebootAfterUpdate which specifically tests shorter
reboot timeouts. I originally planned on setting down_timeout and
reboot_timeout and passing these to reboot but it seemed to be actually a
worse solution for 2 reasons:

1) Since these are client errors, they are reinterpreted as AutoservRunErrors.
I'd literally have to parse the ssh commands to figure whether it was associated
with either not shutting down in time or rebooting in time.
2) Even with (1) wait_for_reboot in client/common_lib/hosts/base_classes.py
waits using the down_timeout + reboot_timeout -- sort of. It doesn't subtract
the time used waiting for wait_down from the total time in wait_up. This is
pretty bad logic and isn't really that useful to set a good timeout.

Given these, it doesn't make much sense to separate down/reboot errors (which I
originally planned to do). So all I could do was sort of pass this one aggregate
value (which wait_for_reboot doesn't allow). So given all these, I decided that
allowing the long timeouts and measuring the time was the best solution for
our problem.

BUG=chromium:242384
TEST=Ran it -- manually changed time outs to see failures correctly.

Change-Id: I52103c8f835cea326bf7efbeb126d9f145988a43
Reviewed-on: https://gerrit.chromium.org/gerrit/56348
Reviewed-by: Chris Sosa <sosa@chromium.org>
Tested-by: Chris Sosa <sosa@chromium.org>
Commit-Queue: Chris Sosa <sosa@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
6a868ddbc4ab2028c616e632c3126fb768f4ca51 26-Apr-2013 Dan Shi <dshi@chromium.org> [Autotest] Add free disk space check in devserver

Add a new method, check_health, to collect the health status of devserver.
Initially, collect the free disk space. The method is extendable to return more
information. The caller will determine if the devserver is ready for staging
image based on the returned health status. stats about each dev server's health
condition is also collected from caller.

BUG=chromium:219330
TEST=tested with run_suite in local machine, which runs as devserver as well.
DEPLOY=apache

Change-Id: I4828e6f75b3e3d177c74839333f3eaadce2e9aa7
Reviewed-on: https://gerrit.chromium.org/gerrit/49251
Reviewed-by: Alex Miller <milleral@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
Tested-by: Dan Shi <dshi@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
9e9a4a18fcbdf996d3a88a8b35e462e377ff0d2f 24-Apr-2013 beeps <beeps@chromium.org> [autotest] Devserver resolution shouldn't fail unless all devservers are down.

In trying to resolve a build to a devserver instance we go through the list of
devservers looking for one to use. This leads to a devserer_up call which
checks to see if it can access the index of the devserver, and waits for about
six seconds for a reply. If this call throws a TimeoutException we fail, we
should instead catch it and move on to the next devserver since if it's a real
devserver problem Nagios will send out mail.

TEST=Called the method in both the pass and fail case and checked behavior.
BUG=chromium:235164

Change-Id: Id9c52f3fdf47d464b6f9e77355e5049b1dc46d44
Reviewed-on: https://gerrit.chromium.org/gerrit/49088
Reviewed-by: Chris Sosa <sosa@chromium.org>
Commit-Queue: Prashanth Balasubramanian <beeps@chromium.org>
Tested-by: Prashanth Balasubramanian <beeps@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
a2def28164c6edea190d34c649cef9d835c051e9 11-Apr-2013 Chris Sosa <sosa@chromium.org> Fix bugs in nmo testing.

After adding logs I found two issues that are related to my changes
to modifying the dev_server code. This fixes the url returned
in dev_server.py for delta payloads to be correct.

I've also added additional logging to make this easier to find in the
future.

FYI: The reason why this was only sometimes failing was that the delta
payload url being returned ended up being the full payload url. So if
the image was staged on the same devserver as was chosen for the
devserver delta payload, it would pass, otherwise it would fail.

BUG=chromium:226159
TEST=Verified on a local dut through the test.

Change-Id: Ib3f00dc43f806f05d500062972b506954b8b7002
Reviewed-on: https://gerrit.chromium.org/gerrit/47914
Reviewed-by: Chris Sosa <sosa@chromium.org>
Tested-by: Chris Sosa <sosa@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
31b2e3196a0507f439ba130bdc53dea83288a5d9 05-Apr-2013 J. Richard Barnette <jrbarnette@chromium.org> Unify platform_InstallTestImage with servo repair.

The body of platform_InstallTestImage was an almost verbatim copy
of SiteHost._servo_repair(). Replaced that body with a call to the
host method.

BUG=None
TEST=run_remote_tests platform_InstallTestImage

Change-Id: Ie2dc1fb2adcba544cd472f4f1052a440e21b988e
Reviewed-on: https://gerrit.chromium.org/gerrit/47389
Commit-Queue: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Richard Barnette <jrbarnette@chromium.org>
Tested-by: Richard Barnette <jrbarnette@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
b09376dcd012137a2e7f02aa0c1f18e26738baa1 18-Mar-2013 Chris Sosa <sosa@chromium.org> Fix bugs with custom url pattern matching for EndToEnd test.

This CL both gets rid of 3 extra variables (cleaning up a TODO) and also
allows this test to be run against trybots and non-canary builders.

Unlike canary builders, trybot's and other builders do not follow a
${board}-release pattern. Instead they can be pretty much

<anything>/<version>. The devserver deals with this by just using the
archive_url (which embeds the build) to find/store the payload. We leverage
this and only use image to base our url's.

Note this also allows us to run AU Test on paladin's and chrome pfq's.

BUG=chromium:217855
TEST=Pylint + unittests + local autotest setup with endToEnd test.

Change-Id: I765d90bfabb4933e35b3de914f82b817c05982cb
Reviewed-on: https://gerrit.chromium.org/gerrit/45759
Commit-Queue: Chris Sosa <sosa@chromium.org>
Reviewed-by: Chris Sosa <sosa@chromium.org>
Tested-by: Chris Sosa <sosa@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
833814b1b821a6454642a276b8ea85d5ae710b8d 29-Jan-2013 Simran Basi <sbasi@chromium.org> Telemetry on Autotest: Telemetry runner code and simple test

This change adds the code to have the drones and server side tests
able to execute the telemetry code installed on the devserver.

The telemetry runner will determine the correct devserver for the
DUT's specific build, tell the devserver to setup telemetry and given
a benchmark and page_set it will execute a telemetry benchmark and
return a TelemetryResultObject to represent the results of telemetry's
execution.

The stdio of the telemetry output is also parsed and used to generate
perf key-value pairs to be uploaded into the autotest results db.

BUG=chromium-os:37412, chromium-os:38997
TEST=tested the telemetry side with my local AFE and devserver and
tested the pyauto proxy portion in the actual lab.

Change-Id: Ia8da3f991b498e1ee63fbfffbfaaa628e3785bf3
Reviewed-on: https://gerrit.chromium.org/gerrit/43679
Commit-Queue: Simran Basi <sbasi@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Tested-by: Simran Basi <sbasi@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
ae6acd9b3e1c6f56310cdb4e05737122b7b2818c 07-Feb-2013 Chris Sosa <sosa@chromium.org> Override devserver_up's default timeout to return immediately in au test.

Since we control this devserver, we want to control how long to wait for
the devserver to be up. This CL slightly refactors the devserver api
to allow callers to override the retry.retry timeout in some rpc's.

Also fixed the pylintrc to skip **kwargs, *args, and cls.

BUG=None
TEST=Pylint so far + unittests

Change-Id: Ie454e81be8940bdc7b8530bd682dab80910b8b83
Reviewed-on: https://gerrit.chromium.org/gerrit/43480
Tested-by: Chris Sosa <sosa@chromium.org>
Reviewed-by: Brian Harring <ferringb@chromium.org>
Reviewed-by: Prashanth Balasubramanian <beeps@chromium.org>
Reviewed-by: Alex Miller <milleral@chromium.org>
Commit-Queue: Chris Sosa <sosa@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
7d7b9ebfdde3c557e9d82e7af3f7e128533852b3 15-Feb-2013 Chris Sosa <sosa@chromium.org> Small refactoring / cleanup to allow autotest to continue working with devserver API.

Not much to this CL -- lots of small changes which make it continue
to work, and in some places work much better/faster.

1) Add a method to use the stage RPC directly (not cleaning up
trigger/finish_download in this CL
2) I removed the implicit download of the firmware tarball from the
spatial locality map -- it always errors out and pretty much hogs our
devserver logs. Instead I stage the firmware tarball when we need it
and fail that call as necessary. This is much more correct than what we have
now.
3) Removed test image RPC from dev_server and changed its one callspot to use
(1)
4) Fixed the bug in symoblicating crashes -- before there was a race to whoever
last staged symbols got to symoblicate crashes -- not anymore as we store the symbols
per build in a subdir for that build.
5) Cleaned up autoupdate test to stage only what it needs.

Consult I201dcdaa8a14024247 for information about artifact naming.

BUG=chromium-os:38427
TEST=In progress + unittests
CQ-DEPEND=I201dcdaa8a14024247

Change-Id: I64e35f3bdeac9e0cd5e7589df913c14ce6691f7a
Reviewed-on: https://gerrit.chromium.org/gerrit/43350
Reviewed-by: Chris Sosa <sosa@chromium.org>
Tested-by: Chris Sosa <sosa@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
6c55bdb98e967675456a71a0971b81058536cac8 19-Jan-2013 Chris Sosa <sosa@chromium.org> Start omaha server instance on devserver payloads are staged onto.

This took a lot to get right as ssh is very finicky at shutting down
a remote process in Linux. After talking with Richard/Simran, it seems
like autotest already has a convention of tracking pids so I'm using
that. In order to do this, I need to pipe my output elsewhere so I'm
using /tmp/devserver.${port} which should be unique per DUT (meaning
it can't be used more than once by this test.

Otherwise I've switched all calls to use SSHHost.run rather than
popen making it much easier to do the right thing with my ssh cmds.

BUG=chromium-os:37975
TEST=Ran it locally -- had to set up my own ssh keys to talk to my
own machine i.e. set CROS->devserver as my local hostname.

Change-Id: Ifce5310c20f39ac4cdfa278841875188b09fc360
Reviewed-on: https://gerrit.chromium.org/gerrit/41675
Tested-by: Chris Sosa <sosa@chromium.org>
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Commit-Queue: Chris Sosa <sosa@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
35f6105800ed49c38fd65681b94d972cfd32ed21 08-Jan-2013 Alex Miller <milleral@chromium.org> [autotest] Do not have 1-minute timeout on devserver calls.

Default timeout duration is 1 minute. This is probably not right.

TEST=None
BUG=chromium-os:37676

Change-Id: I71643fe3de02de7b13140b799a4530b2e1ed7b3a
Reviewed-on: https://gerrit.chromium.org/gerrit/40601
Tested-by: Alex Miller <milleral@chromium.org>
Reviewed-by: Chris Sosa <sosa@chromium.org>
Commit-Queue: Alex Miller <milleral@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
0ed760cbee4ce839988585003d445465bb0b95b9 06-Nov-2012 Gilad Arnold <garnold@chromium.org> autotest: end-to-end autoupdate test

This implements an autotest test for the Chrome OS autoupdate
functionality. It receives a single test configuration and performs an
end-to-end test of the update process specified by it, which includes:

* Initializing the DUT to a known start state (source version)

* Spawning a local Omaha/devserver instance.

* Triggering an update process.

* Tracking the progress of the update.

* Rebooting the DUT after the update.

* Ensuring that the update has succeeded (target version running).

BUG=chromium-os:33760,chromium-os:33761,chromium-os:33762
TEST=Running with run_remote_tests.sh works

Change-Id: I215e8214d3239f525f062a27a4d97806451c9736
Reviewed-on: https://gerrit.chromium.org/gerrit/37432
Commit-Ready: Gilad Arnold <garnold@chromium.org>
Reviewed-by: Gilad Arnold <garnold@chromium.org>
Tested-by: Gilad Arnold <garnold@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
3948b4931a7c6cb3bed29e05f151a0bdfb63c47b 17-Oct-2012 Chris Sosa <sosa@chromium.org> Fix devserver_up call to catch URLErrors that are raised.

I made a booboo and thought that @remote_devserver_call would return
a DevserverException when it couldn't connect to the device. This isn't true
(on purpose) so I catch this when running make_call.

BUG=chromium-os:35415
TEST=New unittest that failed before change.

Change-Id: I5c30877a4f7911f1510cf568a14ec5083ee96ee8
Reviewed-on: https://gerrit.chromium.org/gerrit/35844
Tested-by: Chris Sosa <sosa@chromium.org>
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Commit-Ready: Chris Sosa <sosa@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
8906ab1e576797069b5b928e646f06da015fbfd0 24-Jul-2012 Chris Masone <cmasone@chromium.org> [autotest] Honor DEPENDENCIES information when running dynamic_suites.

A test control file can specify a list of DEPENDENCIES, which are really just
the set of labels a host needs to have in order for that test to be scheduled
on it. In the case of a dynamic_suite, many tests in the suite may have
DEPENDENCIES specified. They may overlap, they may not.

This is a first-pass at a very naive mechanism for finding and reimaging a
set of machines that can satisfy all the test DEPENDENCIES in a given suite.
Per-test-per-suite DEPENDENCIES information is generated at build time, and
then fetched and consumed during suite preparation. If info exists for the
suite being run, the infrastructure will look for a host that matches each
unique DEPENDENCY list. If it can find such a set that is <= the suite
sharding factor, the infrastructure will schedule those machines for
reimaging. If not, the suite will fail.

If the suite to be run has no DEPENDENCIES, meta_host scheduling will be used
as today.

BUG=chromium-os:22060
TEST=unit
TEST=run_suite.py with a suite (like 'dummy') that uses DEPENDENCIES
TEST=run_suite.py with a suite that doesn't use DEPENDENCIES
TEST=run_suite.py with a suite that uses DEPENDENCIES; shut down a non-essential machine during reimage. Suite should complete.
TEST=Same as above, but for a suite that doesn't use DEPENDENCIES
TEST=run_suite.py with a suite that uses DEPENDENCIES; shut down an essential machine (e.g. the only one with the 'bluetooth' label) during reimage. Suite should not complete.

Change-Id: I64b2e39a79a05c92748f34ffa4acf7667b0476a6
Reviewed-on: https://gerrit.chromium.org/gerrit/27526
Commit-Ready: Chris Masone <cmasone@chromium.org>
Reviewed-by: Chris Masone <cmasone@chromium.org>
Tested-by: Chris Masone <cmasone@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
accb5ceaac066025592808247bfa9c97966bba97 31-Aug-2012 Chris Sosa <sosa@chromium.org> Support graceful failover/aggregation for autotest devserver pool.

This change implements the graceful failover/aggregation designed in
crosbug.com/26451. This required a large re-work of how other modules
interact with the dev_server module. Instead of always relying on being
able to pass in the "build" to any dev_server RPC, a caller must start
by using the static resolve method that resolves a build to an ip addr
and creating a DevServer instance from that hostname.

i.e.

ds = Devserver.create()
ds.trigger_download('build')

turns into

ds = ImageServer.resolve('build')
ds.trigger_download()

In order to guarantee the re-use of the same hostname throughout the life-cycle
of a suite run, we keep the hostname as part of the suitespec and also parse
the package_url based on the image_url -- not the build.

As noted, I've also separated the imaging / crash components into their
own DevServer subclasses to simplify the interaction and modified
callers.

BUG=chromium-os:26451
TEST=Unittests so far + local autotest setup with running bvt suite on one
host in the lab + ran suite_enumerator with/without -l

Change-Id: I4760e4033bb347259e3bb30f2fac31f321654c6f
Reviewed-on: https://gerrit.chromium.org/gerrit/32006
Commit-Ready: Chris Sosa <sosa@chromium.org>
Reviewed-by: Chris Sosa <sosa@chromium.org>
Tested-by: Chris Sosa <sosa@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
dea299380ad7a21f8331fd2db046e20c5f755834 02-Aug-2012 J. Richard Barnette <jrbarnette@chromium.org> Use a default image URL in platform_InstallTestImage.

If the target host has a known board type, use the default URL for
the image based on the board type. This allows hosts to be installed
automatically from the web interface.

BUG=chromium-os:33290
TEST=run_remote_tests, using a private_host_attributes file for the target

Change-Id: I58a611cdee4e404a1b2bd02c22f5bb63fa1aabfd
Reviewed-on: https://gerrit.chromium.org/gerrit/28997
Tested-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Commit-Ready: Richard Barnette <jrbarnette@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
24b3a027291ff030bb74482818106fa91bcc911d 31-Jul-2012 Chris Sosa <sosa@chromium.org> Reland "Use dev_server to determine the url of the devserver."

Condition for deploying required a scheduler restart. Original
change was fine.

This reverts commit 5ca9d1b7a16b80d71acdbf01bd5b8019b13e3165.

BUG=chromium-os:26451
TEST=Unittests + local run with prod devservers

Change-Id: I9acd6bce50a8919f85ca3ed65420ad235e8501e7
Reviewed-on: https://gerrit.chromium.org/gerrit/28855
Reviewed-by: Chris Sosa <sosa@chromium.org>
Tested-by: Chris Sosa <sosa@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
5ca9d1b7a16b80d71acdbf01bd5b8019b13e3165 30-Jul-2012 Chris Sosa <sosa@chromium.org> Revert "Use dev_server to determine the url of the devserver."

Started seeing this on the autotest server:

Traceback (most recent call last):
File "/usr/local/autotest/server/server_job.py", line 539, in run
self._execute_code(server_control_file, namespace)
File "/usr/local/autotest/server/server_job.py", line 1022, in _execute_code
execfile(code_file, namespace, namespace)
File "/usr/local/autotest/results/426888-chromeos-test/chromeos-rack8c-host2/
control.srv", line 31, in <module>
repo_url = dynamic_suite._package_url_pattern() % image_name

This reverts commit dfe7042a31f726cd637530cfc28ecaaec08256ba.

Change-Id: Icf560ac11f7f5567b7dbe4e32a13584ccdda06c7
Reviewed-on: https://gerrit.chromium.org/gerrit/28724
Reviewed-by: Chris Sosa <sosa@chromium.org>
Tested-by: Chris Sosa <sosa@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
dfe7042a31f726cd637530cfc28ecaaec08256ba 26-Jul-2012 Chris Sosa <sosa@chromium.org> Use dev_server to determine the url of the devserver.

The dynamic_suite code assumed the devserver variable in the autotest
config was a string that represented the url for the devserver.
This is no longer a correct assumption as it can be a list of devservers.
This change modifies the dynamic_suite logic to use a DevServer instance
to tell it the URL it should use for the devserver.

BUG=chromium-os:26451
TEST=Unittests

Change-Id: I0bc63029630d9e884a23b16ccdf75da1756e6518
Reviewed-on: https://gerrit.chromium.org/gerrit/28447
Reviewed-by: Chris Masone <cmasone@chromium.org>
Tested-by: Chris Sosa <sosa@chromium.org>
Commit-Ready: Chris Sosa <sosa@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
6f109089b5046b01ab518de7171fb2c4dee396e6 18-Jul-2012 Chris Masone <cmasone@chromium.org> [autotest] Add retrying logic to devserver calls, autoupdate 'test'

Have the autoupdate 'test' control file that we use to reimage devices
use a RetryingAFE, so that it doesn't die in the event of frontend
restarts.

Also, re-use the same retrying logic to have the DevServer class retry
calls in the event of URLErrors. We don't do this for
DevServerExceptions, because that would cause us to retry for things
like attempts to trigger a download with a malformed image name, which
we don't want to do.

BUG=chromium-os:31418
TEST=unit
TEST=run_suite using a local dev server that's not running initially.
TEST=It should keep trying until you start the dev server, at which point things should progress normally.

Change-Id: Id8c7b07ce19d528ae35e7df6a7359234329869df
Reviewed-on: https://gerrit.chromium.org/gerrit/28101
Tested-by: Chris Masone <cmasone@chromium.org>
Reviewed-by: Chris Sosa <sosa@chromium.org>
Commit-Ready: Chris Masone <cmasone@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
761341eafc11de617743ca8126a3b1a3b9f9c430 19-Jul-2012 Alex Miller <milleral@chromium.org> [autotest] Offload crash handling to a dedicated server.

Add the ability to specify a dedicated devserver to handle
symbolicate_dump rpc calls to offload work from the main dev_server.

BUG=chromium-os:32710
TEST=unit

Change-Id: I11313a92a96709bbc0feeeaea1c755adb92a4290
Reviewed-on: https://gerrit.chromium.org/gerrit/27940
Tested-by: Alex Miller <milleral@chromium.org>
Reviewed-by: Chris Sosa <sosa@chromium.org>
Commit-Ready: Alex Miller <milleral@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
e327ec8297a558d1976a622810be75c826258572 30-Jun-2012 Chris Sosa <sosa@chromium.org> Allow and use specification of multiple devservers.

This change modifies the devserver logic to accept a comma-delimited list
of devservers in the global_config.ini. We then take the hash of the image arg
when determining which devserver to use.

BUG=chromium-os:26451
TEST=Unittests so far

Change-Id: I6f01b944a618c186ebd923a2ef4d065196ad169f
Reviewed-on: https://gerrit.chromium.org/gerrit/26496
Commit-Ready: Chris Sosa <sosa@chromium.org>
Reviewed-by: Chris Sosa <sosa@chromium.org>
Tested-by: Chris Sosa <sosa@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
aa5ebbe1765264a6239e05bcd36b35d3230ac579 26-Jun-2012 Chris Masone <cmasone@chromium.org> [autotest] Tolerate UTF-32 in dev server error messages

It seems that the Apache instance we intend to wrap the dev server in
returns error messages in UTF-32, though the dev server we use today
returns ASCII. Tolerate either.

BUG=chromium-os:32081
TEST=run_suite against local dev server
TEST=run_suite against locally-running web server that sends back UTF-32

Change-Id: Ia4cea3eaeeed90242026ccbd658bfa6483fc418d
Reviewed-on: https://gerrit.chromium.org/gerrit/26075
Reviewed-by: Chris Masone <cmasone@chromium.org>
Tested-by: Chris Masone <cmasone@chromium.org>
Commit-Ready: Chris Masone <cmasone@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
acbf4e0ff5a0f556597a83e8d4b255d5fdd1819f 22-May-2012 Chris Masone <cmasone@chromium.org> [autotest] Handle HTTPErrors in symbolicate_dump

When constructing a urllib2.HTTPError, passing None for the final arg
causes read() to not get defined on the object. Weird. So, make
sure that we pass a file-like object containing error strings there.

BUG=chromium-os:31151
TEST=run a test that leaves a crash dump and make the dev server throw 500s'

Change-Id: If9b84276609c9dfd292925e84ccab88fb6ad41a1
Reviewed-on: https://gerrit.chromium.org/gerrit/23172
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Commit-Ready: Chris Masone <cmasone@chromium.org>
Tested-by: Chris Masone <cmasone@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
bafbbb0ff97fa5a5e07a47003d1ea57566f2186d 16-May-2012 Chris Masone <cmasone@chromium.org> [autotest] Re-land "Add logic to symbolicate dumps using dev server, stage symbols on-demand"

re-run /usr/local/autotest/utils/build_externals.py before trying to use this
outside the chroot.

When trying to symbolicate a crash dump using the dev server, we first
have to stage debug symbols on-demand. We can't stage them up front,
sadly. So, block and stage them on-demand, doing so iff we actually get
crashes during a test.

This reverts commit 77e95f2dd7a025966aa2cdf19cfb5f0bfcf2608f.

BUG=chromium-os:29730,chromium-os:30400
TEST=run_remote_tests a test that generates a crash dump; symbolication should
TEST=occur locally.
TEST=set up a dev server that does not have symbols staged, and run a test
TEST=that generates a crash dump. The symbols should get staged and the dump
TEST=symbolicated. Now run another. The symbols should be re-used.
STATUS=Fixed

Change-Id: I85c9f16ec054210ebacf22c4713d9ba65927551b
Reviewed-on: https://gerrit.chromium.org/gerrit/22866
Tested-by: Chris Masone <cmasone@chromium.org>
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Commit-Ready: Chris Masone <cmasone@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
f70650c57a8f44f80055f5d136c4808a79d67c4d 16-May-2012 Chris Masone <cmasone@chromium.org> [autotest] Re-land "Put dev server error strings in exceptions"

We've been ignoring error messages coming back from the dev server.
Strip out the HTML that comes in (some of) them and wrap them
in exceptions.

This reverts commit 0dcf3a4c80b66d7598f3d7391bc1f9bf5492d8e2.

BUG=chromium-os:30279
TEST=unit
TEST=use atest suite create to run a suite that doesn't exist on build that doesn't exist.

Change-Id: I9d684d8b1c959d17c4b17082ef6892112b860493
Reviewed-on: https://gerrit.chromium.org/gerrit/22847
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Tested-by: Chris Masone <cmasone@chromium.org>
Commit-Ready: Chris Masone <cmasone@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
0dcf3a4c80b66d7598f3d7391bc1f9bf5492d8e2 16-May-2012 Chris Sosa <sosa@chromium.org> Revert "[autotest] Put dev server error strings in exceptions"

This change exposed a bug in the old test_scheduler. The old test_scheduler
didn't do anything to suppress exceptions and aborts if the devserver returns
any exceptions.

This change is good as is and can be safely re-reverted once the old test_scheduler is robustified a little.

This reverts commit c807fcb56c57dbac1b193d92a90d59388c0cefe4

BUG=chromium-os:30915
TEST=Ran on autotest server with test_scheduler.

Change-Id: I22b9bbf87552c411c82b2d4e67fdbc33576eacb6
Reviewed-on: https://gerrit.chromium.org/gerrit/22776
Reviewed-by: Chris Sosa <sosa@chromium.org>
Tested-by: Chris Sosa <sosa@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
77e95f2dd7a025966aa2cdf19cfb5f0bfcf2608f 16-May-2012 Chris Masone <cmasone@chromium.org> Revert "[autotest] Add logic to symbolicate dumps using dev server, staging symbols on-demand"

An important change that this depends on got reverted because it broke a test that is of marginal utility, so this will start breaking stuff again if we leave it in.

This reverts commit d1294ffd263104fd0b7473c38c646fc9d26b8ea9

Change-Id: Ibcc1193db2e1a5907ddaf11fd6ab1b803dbdf303
Reviewed-on: https://gerrit.chromium.org/gerrit/22766
Commit-Ready: Chris Masone <cmasone@chromium.org>
Reviewed-by: Chris Masone <cmasone@chromium.org>
Tested-by: Chris Masone <cmasone@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
06e087cf2830e677cde88e32e4f42d0bc0547e68 08-May-2012 Chris Masone <cmasone@chromium.org> [autotest] Add logic to symbolicate dumps using dev server, staging symbols on-demand

re-run /usr/local/autotest/utils/build_externals.py before trying to use this
outside the chroot.

When trying to symbolicate a crash dump using the dev server, we first
have to stage debug symbols on-demand. We can't stage them up front,
sadly. So, block and stage them on-demand, doing so iff we actually get
crashes during a test.

BUG=chromium-os:29730,chromium-os:30400
TEST=run_remote_tests a test that generates a crash dump; symbolication should
TEST=occur locally.
TEST=set up a dev server that does not have symbols staged, and run a test
TEST=that generates a crash dump. The symbols should get staged and the dump
TEST=symbolicated. Now run another. The symbols should be re-used.

This reverts commit 01310f99e1aa2862af087583a0815c3271fc10c1.
This reverts commit 52c7fb7c3b8c6fcc81b62a391d3cd8b9443b25d1.

Change-Id: I5385d5213f21bc03c9301d4f9f88ed06d30d8e1f
Reviewed-on: https://gerrit.chromium.org/gerrit/22501
Commit-Ready: Chris Masone <cmasone@chromium.org>
Reviewed-by: Chris Masone <cmasone@chromium.org>
Tested-by: Chris Masone <cmasone@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
5117e3eb723645e48e004589123af8fb5e0787cf 10-May-2012 Chris Masone <cmasone@chromium.org> [autotest] Put dev server error strings in exceptions

We've been ignoring error messages coming back from the dev server.
Strip out the HTML that comes in (some of) them and wrap them
in exceptions.

BUG=chromium-os:30279
TEST=unit
TEST=use atest suite create to run a suite that doesn't exist on build that doesn't exist.

Change-Id: I86b149610bf54334cdca12836a078ab874c818b2
Reviewed-on: https://gerrit.chromium.org/gerrit/22315
Commit-Ready: Chris Masone <cmasone@chromium.org>
Reviewed-by: Chris Masone <cmasone@chromium.org>
Tested-by: Chris Masone <cmasone@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
01310f99e1aa2862af087583a0815c3271fc10c1 08-May-2012 Scott Zawalski <scottz@chromium.org> Revert "[autotest] Re-land "Add logic to symbolicate dumps using dev server""

Cherrypy is hanging when we send multiple POSTs using request to it in a short
amount of time

BUG=30650

This reverts commit 043bf47e7946366517757cdb981de99eb81adcb6

Change-Id: I6f91b730289d14502eeffcf672ccb36494d5f847
Reviewed-on: https://gerrit.chromium.org/gerrit/22115
Commit-Ready: Scott Zawalski <scottz@chromium.org>
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Tested-by: Scott Zawalski <scottz@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
52c7fb7c3b8c6fcc81b62a391d3cd8b9443b25d1 07-May-2012 Chris Masone <cmasone@chromium.org> Revert "[autotest] Stage debug symbols on-demand before symbolicating minidumps."

CherryPy is weaksauce and falls over if too many POSTs come in at the same time. Reverting until we can make the devserver handle the load.

This reverts commit a3fb82d1b85dad68b4e9dbbe6b2f60c4ea12adb2

Change-Id: I1d2b547031a6db8d48e00547b0029111574e901d
Reviewed-on: https://gerrit.chromium.org/gerrit/21997
Commit-Ready: Chris Masone <cmasone@chromium.org>
Reviewed-by: Chris Masone <cmasone@chromium.org>
Tested-by: Chris Masone <cmasone@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
0a939871cbc398f110f5cc846f84e8541b8540f8 03-May-2012 Chris Masone <cmasone@chromium.org> [autotest] Stage debug symbols on-demand before symbolicating minidumps.

We can't stage debug symbols up front, sadly. So, stage them on-demand iff
we actually get crashes during a test. Do so in a blocking fashion.

BUG=chromium-os:30400
TEST=set up a dev server that does not have symbols staged, and run a test
TEST=that generates a crash dump. The symbols should get staged and the dump
TEST=symbolicated. Now run another. The symbols should be re-used.

CQ-DEPEND=Ie460526396d2b9999137142c723b87793bc23aaa

Change-Id: I02b69ed2db7b22a557d94a2005fc1cef901eefcd
Reviewed-on: https://gerrit.chromium.org/gerrit/21826
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Commit-Ready: Chris Masone <cmasone@chromium.org>
Tested-by: Chris Masone <cmasone@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
4245a73ec638b602bd4546637b41c74f39158e14 30-Apr-2012 Chris Masone <cmasone@chromium.org> [autotest] Re-land "Add logic to symbolicate dumps using dev server"

re-run /usr/local/autotest/utils/build_externals.py before trying to use this
outside the chroot.

BUG=chromium-os:29730
TEST=point local autoserv at a local dev server, run a test that causes a crash; it should symbolicate the crash

This reverts commit 8972fa982c3ec369737f1e45f52a3b350c1824e7.

Change-Id: I080fb56b0d2508ccd90b4140826d7020efc81982
Reviewed-on: https://gerrit.chromium.org/gerrit/21493
Tested-by: Chris Masone <cmasone@chromium.org>
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Commit-Ready: Chris Masone <cmasone@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
8972fa982c3ec369737f1e45f52a3b350c1824e7 30-Apr-2012 Chris Masone <cmasone@chromium.org> Revert "[autotest] Add logic to symbolicate dumps using dev server"

Somehow, import logging is failing...?

This reverts commit 98bc0046b399518cb1217469b9306a4ab7a4b928

Change-Id: Iedab8e712494a730b797b0490dcabf5c6578e192
Reviewed-on: https://gerrit.chromium.org/gerrit/21478
Commit-Ready: Chris Masone <cmasone@chromium.org>
Reviewed-by: Chris Masone <cmasone@chromium.org>
Tested-by: Chris Masone <cmasone@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
d6a4718a15d37306f31a65ad8d6b9023b4a69b42 25-Apr-2012 Chris Masone <cmasone@chromium.org> [autotest] Add logic to symbolicate dumps using dev server

re-run /usr/local/autotest/utils/build_externals.py before trying to use this
outside the chroot.

BUG=chromium-os:29730
TEST=point local autoserv at a local dev server, run a test that causes a crash; it should symbolicate the crash into a .txt named after the minidump file.

CQ-DEPEND=Ib9d5b0abb73567fd8450db50d3e50d27bdc169f4

Change-Id: Id0d1fd6b4f00e560dd5683d87140ed943697f185
Reviewed-on: https://gerrit.chromium.org/gerrit/21277
Commit-Ready: Chris Masone <cmasone@chromium.org>
Reviewed-by: Chris Masone <cmasone@chromium.org>
Tested-by: Chris Masone <cmasone@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
6b288c89dfcc112f605854bd5563b70ee5d74006 30-Mar-2012 Chris Sosa <sosa@chromium.org> Modify dynamic suite pathway to allow some artifacts to be staged in the bgnd.

This change modifies the site_rpc_interface->dynamic_suite.py workflow to allow
autotest to stage larger build components while imaging devices. It uses
the download/wait_for_status interface provided by the devserver to
accomplish it.

BUG=chromium-os:27285
TEST=All effected unittests pass. Staging larger test on local devserver in
parallel.

Change-Id: Iae2a20762c565b8e6d440c9a501c1f99c73129ed
Reviewed-on: https://gerrit.chromium.org/gerrit/19328
Tested-by: Chris Sosa <sosa@chromium.org>
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Tested-by: Scott Zawalski <scottz@chromium.org>
Commit-Ready: Scott Zawalski <scottz@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
9d7955fef7bb93d7df3d45a13ff52e70619520dc 21-Mar-2012 Scott Zawalski <scottz@google.com> Update downloader and test_scheduler to utilize new staging.

We have hit a problem where the downloader code no longer properly
stages images for R19. This CL fixes that.

Add an option to downloader to stage images via the new devserver pathway.
IFF a board in the json config has 'release' in the name (Currently used to
signify TOT) and also has a download_devserver variable set to true. All other
boards with release in the name are skipped, these are currently downloaded
via the HWTest stage. All -RXX releases are processed in the old way still.

Update test_scheduler to get its control files from the dev server for all TOT
images. The new style puts the autotest files under /autotest/ instead of
/server/.

Fixed a bug where passing None to the dev server is interpreted as "None" and
causes issues because it is a valid string.

TEST=Local runs of downloader to ensure the pathway is being executed with the proper
variables.
BUG=chromium-os:28085

Change-Id: I9c5dc9e3a10ad02b9c6246beece11b8f31b0e5dd
Reviewed-on: https://gerrit.chromium.org/gerrit/18656
Commit-Ready: Scott Zawalski <scottz@chromium.org>
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Tested-by: Scott Zawalski <scottz@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
136dc4b37934664f9c4fdf71365e12881bf73827 20-Mar-2012 Scott Zawalski <scottz@google.com> Add a get_latest_build call to the cros/dev_server code.

Add in a pathway to ask the devserver what the latest build is
for a given target.

Update the old test_scheduler to use this pathway so that it no longer
relies on LATEST files.

TEST=unittests.
BUG=chromium-os:28085
CQ-DEPEND=I2b99a0e47a963fc528c1ad4bac49e340a24a9480

Change-Id: Ia5ef5b222a325af8e5074266fcbf90840da9dc12
Reviewed-on: https://gerrit.chromium.org/gerrit/18564
Commit-Ready: Scott Zawalski <scottz@chromium.org>
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Tested-by: Scott Zawalski <scottz@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
859fdecd96355279570007849d8bc46ff71874d7 30-Jan-2012 Chris Masone <cmasone@chromium.org> [autotest] Add create_suite_job() site RPC.

Add create_suite_job() RPC to autotest using the site_rpc mechanism.
Also add support to atest to call this RPC.

The purpose of this RPC is to stage a build on the dev server (if
necessary), image N machines with it, and then run the desired
suite on those machines.

BUG=chromium-os:25573
TEST=atest suite create -b x86-mario -i x86-mario-release/R19-1675.0.0-a1-b1588 test_suites/control.dummy

Change-Id: I09288fe6ffc675e5b111f7f59e349015f52ebb8e
Reviewed-on: https://gerrit.chromium.org/gerrit/15279
Tested-by: Chris Masone <cmasone@chromium.org>
Commit-Ready: Scott Zawalski <scottz@chromium.org>
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Tested-by: Scott Zawalski <scottz@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
fef2138b049ad06ba451308b29b06122b27277a9 17-Jan-2012 Chris Masone <cmasone@chromium.org> [autotest] Add ControlFileGetter impl that uses DevServer

Also, update a few files used in local testing.

BUG=chromium-os:24985
TEST=control_file_getter_unittest.py

Change-Id: Ic619b5c5f2ecbe0401400f2f2e6553cf50212c38
Reviewed-on: https://gerrit.chromium.org/gerrit/14323
Tested-by: Chris Masone <cmasone@chromium.org>
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Commit-Ready: Chris Masone <cmasone@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
8b764254b6b682e15802e3f3cb6ee22d56f237f1 17-Jan-2012 Chris Masone <cmasone@chromium.org> [autotest] Expand DevServer class to support control-file related calls

Add the ability to call the control_files/ endpoint on the dev server.

BUG=chromium-os:24985
TEST=dev_server_unittest.py

Change-Id: If20e7c6d1f13813cd10511c34d64e9ffddfb37f4
Reviewed-on: https://gerrit.chromium.org/gerrit/14302
Commit-Ready: Chris Masone <cmasone@chromium.org>
Reviewed-by: Chris Masone <cmasone@chromium.org>
Tested-by: Chris Masone <cmasone@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py
fb08a7736999b2be1349781fb35bdce5ae35d1ee 13-Jan-2012 Chris Masone <cmasone@chromium.org> [autotest] Add DevServer class that uses the new HTTP API

Instead of working from the existing class that SSH's to the dev
server and runs shell commands, create a new class to work against the
new HTTP api for the dev server. I do this because it's not clear
that we will want to continue using the interface of the existing
class, as it was built to do very different work than what we'd like
to do with the dev server now.

BUG=chromium-os:24985
TEST=dev_server_unittest.py

Change-Id: Ic776d72309bac30eb8362868370fd9f36616c3ba
Reviewed-on: https://gerrit.chromium.org/gerrit/14005
Commit-Ready: Chris Masone <cmasone@chromium.org>
Reviewed-by: Chris Masone <cmasone@chromium.org>
Tested-by: Chris Masone <cmasone@chromium.org>
/external/autotest/client/common_lib/cros/dev_server.py