History log of /external/autotest/site_utils/suite_scheduler/suite_scheduler.py
Revision Date Author Comments (<<< Hide modified files) (Show modified files >>>)
5e2efb71ffebead22aa4f0744ad843ee79814b43 07-Feb-2017 Dan Shi <dshi@google.com> [autotest] Use the metrics_mock object in case chromite is not set up.

BUG=chromium:688166
TEST=unittest

Change-Id: Ic0077cb2dba75a8d820f229060f3f70f507850a1
Reviewed-on: https://chromium-review.googlesource.com/438754
Commit-Ready: Dan Shi <dshi@google.com>
Tested-by: Dan Shi <dshi@google.com>
Reviewed-by: Dan Shi <dshi@google.com>
/external/autotest/site_utils/suite_scheduler/suite_scheduler.py
ad6d879832021e4d83960adc074a36a948a72171 29-Nov-2016 Ilja H. Friedel <ihf@chromium.org> autotest: reorganize arc-cts suites.

In particular add one extra suite per Chrome OS branch.

BUG=b:33186907
TEST=None.

Change-Id: I0904645821e8e190b6e6e4b3b94002f9770611f2
Reviewed-on: https://chromium-review.googlesource.com/414568
Commit-Ready: Ilja H. Friedel <ihf@chromium.org>
Tested-by: Ilja H. Friedel <ihf@chromium.org>
Reviewed-by: Haixia Shi <hshi@chromium.org>
/external/autotest/site_utils/suite_scheduler/suite_scheduler.py
abd3b05597bfa2e5244209b7f5146900a58290a9 03-Oct-2016 Paul Hobbs <phobbs@google.com> Revert "Revert "Revert "[autotest] Set up ts_mon with indirect=True"""

It turns out that even with the upgraded oauth2client, we're still not seeing metrics.

This reverts commit abaaf9a979b52dbd48f33dc147203d631a046443.

Change-Id: I5836ae76e0224d3554a9710ee32a62808bac8652
Reviewed-on: https://chromium-review.googlesource.com/391869
Reviewed-by: David Riley <davidriley@chromium.org>
Tested-by: Paul Hobbs <phobbs@google.com>
/external/autotest/site_utils/suite_scheduler/suite_scheduler.py
abaaf9a979b52dbd48f33dc147203d631a046443 01-Oct-2016 Paul Hobbs <phobbs@google.com> Revert "Revert "[autotest] Set up ts_mon with indirect=True""

This reverts commit 02af0131ff73c73294b0f080a853fe52e7889299.

Change-Id: Ibc86bfc489b9c8a0fd918333c1a3ef55038193f9
Reviewed-on: https://chromium-review.googlesource.com/391129
Reviewed-by: David Riley <davidriley@chromium.org>
Tested-by: Paul Hobbs <phobbs@google.com>
/external/autotest/site_utils/suite_scheduler/suite_scheduler.py
02af0131ff73c73294b0f080a853fe52e7889299 30-Sep-2016 Paul Hobbs <phobbs@google.com> Revert "[autotest] Set up ts_mon with indirect=True"

This reverts commit a065866598b020e05191cfd4fc0f4e7ced069640.

Change-Id: I248ec1e0d2355fea9acb0db7232fd26af695929f
Reviewed-on: https://chromium-review.googlesource.com/391124
Reviewed-by: David Riley <davidriley@chromium.org>
Tested-by: Paul Hobbs <phobbs@google.com>
/external/autotest/site_utils/suite_scheduler/suite_scheduler.py
a065866598b020e05191cfd4fc0f4e7ced069640 20-Sep-2016 Paul Hobbs <phobbs@google.com> [autotest] Set up ts_mon with indirect=True

The scheduler should use indirect=True to support the reset_after=True
flag for global metrics.

BUG=chromium:648312
TEST=None

Change-Id: I4187c84361a51cdc37ca3827493d848e7dcfa3a3
Reviewed-on: https://chromium-review.googlesource.com/387197
Commit-Ready: Paul Hobbs <phobbs@google.com>
Tested-by: Paul Hobbs <phobbs@google.com>
Reviewed-by: David Riley <davidriley@chromium.org>
Reviewed-by: Paul Hobbs <phobbs@google.com>
/external/autotest/site_utils/suite_scheduler/suite_scheduler.py
926a0e8d4ef323c1d2ce557c7be48203890d3ea1 13-Jul-2016 Dan Shi <dshi@google.com> [autotest] Reports suite created by suite scheduler to Monarch

BUG=None
TEST=unittest
DEPLOY=suite_scheduler

Change-Id: Iecd09a54d25e077fe572a86eb0ffb3e4734c8a38
Reviewed-on: https://chromium-review.googlesource.com/360242
Commit-Ready: Dan Shi <dshi@google.com>
Tested-by: Dan Shi <dshi@google.com>
Reviewed-by: Aviv Keshet <akeshet@chromium.org>
/external/autotest/site_utils/suite_scheduler/suite_scheduler.py
2121a3382d70bfb60284a830f341d78d6d555519 25-Feb-2016 Dan Shi <dshi@google.com> [autotest] Support scheduling Launch Control builds from suite scheduler.

This change adds the support of scheduling Launch Control builds from
suite scheduler.

For task defined like:
[brollo_test]
run_on: nightly
hour: 17
suite: dummy
os_type: brillo
branches: git_mnc-brillo-dev
targets: dragonboard-userdebug
pool: suites

Suite scheduler will try to schedule the latest build as of
git_mnc-brillo-dev/dragonboard-userdebug/LATEST
for dragonbard daily at 5PM, to run a duts in suites pool.

BUG=chromium:585628
TEST=unittest, local suite scheduler run:
/usr/local/autotest/site_utils/suite_scheduler/suite_scheduler.py \
-d /usr/local/autotest/logs -f /usr/local/autotest/ss_test.ini \
-e nightly -i git_mnc-brillo-dev/dragonboard-userdebug/2636090 \
-r /tmp/_autotmp_0pjXWQ_suite_scheduler -o brillo

Also test existing cros build can be scheduled:
/usr/local/autotest/site_utils/suite_scheduler/suite_scheduler.py \
-d /usr/local/autotest/logs -f /usr/local/autotest/ss_test.ini \
-e nightly -i veyron_jerry-release/R49-7834.3.0 \
-r /tmp/_autotmp_0pjXWQ_suite_scheduler

Change-Id: I6dc8ecb95ff0d353a1e8a96f98183945216f9a50
Reviewed-on: https://chromium-review.googlesource.com/329362
Commit-Ready: Dan Shi <dshi@google.com>
Tested-by: Dan Shi <dshi@google.com>
Reviewed-by: Dan Shi <dshi@google.com>
/external/autotest/site_utils/suite_scheduler/suite_scheduler.py
15d42311147778148e0a80b703e789abc6bbd12d 16-Dec-2015 Dan Shi <dshi@google.com> [autotest] Allow suite scheduler to process boards in parallel

suite scheduler processes each board in a single threaded fashion. Each board
takes about 2-10 mins to process for a single event (new_build, nightly or
weekly). We have over 100 boards in the lab, that leads to nightly run takes
almost 8 hours to finish, and weekly run takes more than 36 hours.

This CL changes suite scheduler driver to process boards in parallel, to
reduce the run time needed to handle each event.

BUG=chromium:560951
TEST=local run, confirm suite can be created.
/usr/local/autotest/site_utils/suite_scheduler/suite_scheduler.py -b \
-d /usr/local/autotest/logs -f /usr/local/autotest/t_suite_scheduler.ini \
-r /tmp/_autotmp_P5J9Zj_suite_scheduler
unittest

Change-Id: I3d7eea2b6449d3c4ea4a8c191ff980064e0301a6
Reviewed-on: https://chromium-review.googlesource.com/318503
Commit-Ready: Dan Shi <dshi@google.com>
Tested-by: Dan Shi <dshi@google.com>
Reviewed-by: Simran Basi <sbasi@chromium.org>
/external/autotest/site_utils/suite_scheduler/suite_scheduler.py
a54a5feea7cdd1ce015928fccddb973ecd93f48e 07-Aug-2015 Fang Deng <fdeng@chromium.org> [autotest] Specify which afe server to talk to for suite scheduler

Add a setting "suite_scheduler_afe" to [SERVER]
If set, suite scheduler will talk to this afe server.
otherwise, it will default to [SERVER]:hostname

TEST=Ran suite_scheduler with and without the setting in shadow config.
BUG=chromium:513062

Change-Id: I88e179a2d8306356e7e5316345f524b4a9982141
Reviewed-on: https://chromium-review.googlesource.com/291456
Reviewed-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Fang Deng <fdeng@chromium.org>
Tested-by: Fang Deng <fdeng@chromium.org>
/external/autotest/site_utils/suite_scheduler/suite_scheduler.py
f08814a436cdff50ea26571001e72a09e3d57aaf 03-Aug-2015 Fang Deng <fdeng@chromium.org> [autotest] suite-scheduler sanity check should not rely on django

Resubmit of CL:288396
This reverts commit babd70ccd5866fe5eb875e9c725a129d3c881c2c.
It also contains a fix for importing django.

TEST=Test on a ganeti
/usr/local/autotest/site_utils/suite_scheduler/suite_scheduler.py -b -d
/usr/local/autotest/logs -f /usr/local/autotest/suite_scheduler.ini
TEST=Test on a geneti
/usr/local/autotest/site_utils/suite_scheduler/suite_scheduler.py --sanity
(remove site-packages)
BUG=chromium:474671

Change-Id: I155750de1d05b6e3f2625a4ae210622f748975f7
Reviewed-on: https://chromium-review.googlesource.com/290251
Tested-by: Fang Deng <fdeng@chromium.org>
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Fang Deng <fdeng@chromium.org>
/external/autotest/site_utils/suite_scheduler/suite_scheduler.py
babd70ccd5866fe5eb875e9c725a129d3c881c2c 03-Aug-2015 Dan Shi <dshi@chromium.org> CL 288396 is causing suite scheduler to fail to start due to error:

$ /usr/local/autotest/site_utils/suite_scheduler/suite_scheduler.py -b -d /usr/local/autotest/logs -f /usr/local/autotest/suite_scheduler.ini
Traceback (most recent call last):
File "/usr/local/autotest/site_utils/suite_scheduler/suite_scheduler.py", line 48, in <module>
from autotest_lib.site_utils import server_manager_utils
File "/usr/local/autotest/site_utils/server_manager_utils.py", line 19, in <module>
from autotest_lib.frontend.server import models as server_models
File "/usr/local/autotest/frontend/server/models.py", line 8, in <module>
from django.db import models as dbmodels
File "/usr/local/autotest/site-packages/django/db/__init__.py", line 11, in <module>
if settings.DATABASES and DEFAULT_DB_ALIAS not in settings.DATABASES:
File "/usr/local/autotest/site-packages/django/conf/__init__.py", line 53, in __getattr__
self._setup(name)
File "/usr/local/autotest/site-packages/django/conf/__init__.py", line 46, in _setup
% (desc, ENVIRONMENT_VARIABLE))
django.core.exceptions.ImproperlyConfigured: Requested setting DATABASES, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.

Change-Id: I41232fa2bb8380f1fddcf664f55a2285d196e6e8
Reviewed-on: https://chromium-review.googlesource.com/290271
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
Tested-by: Dan Shi <dshi@chromium.org>
/external/autotest/site_utils/suite_scheduler/suite_scheduler.py
a25e0d48f2c15b7c4d430b671aaee9cb56c8fb1e 24-Jul-2015 Dan Shi <dshi@chromium.org> [autotest] Support scheduling suite jobs need firmware update in suite_scheduler.

Sample entry in suite_scheduler.ini will be like:

This should create faft_ec suite when new build created on firmware branch,
tot CrOS build is used. Test is from ChromeOS branch.
Test command:
/usr/local/autotest/site_utils/suite_scheduler/suite_scheduler.py -b \
-d /usr/local/autotest/logs -f /usr/local/autotest/test_suite_scheduler.ini \
-r /tmp/_autotmp_0Wmh0i_suite_scheduler --e new_build \
-i veyron_jerry-firmware/R41-6588.106.0

[faft]
run_on: new_build
suite: faft_ec
branch_specs: firmware
cros_build_spec: tot
test_source: cros
boards: veyron_jerry
pool: faft_test

This should create faft_ec suite when new build created on R44 branch,
tot firmware build on firmware branch is used. Test is from firmware branch.
Test command:
/usr/local/autotest/site_utils/suite_scheduler/suite_scheduler.py -b \
-d /usr/local/autotest/logs -f /usr/local/autotest/test_suite_scheduler.ini \
-r /tmp/_autotmp_0Wmh0i_suite_scheduler --e new_build \
-i veyron_jerry-release/R44-7077.90.0

[faft2]
run_on: new_build
suite: faft_ec
branch_specs: ==tot-2
firmware_rw_build_spec: firmware
test_source: firmware
boards: veyron_jerry
pool: faft_test

BUG=chromium:513409
TEST=local run suite_scheduler command as mentioned in commit message.

Change-Id: I8976c5af6ff771cd0ca409e55ffbbe1634c4cf4d
Reviewed-on: https://chromium-review.googlesource.com/288900
Trybot-Ready: Dan Shi <dshi@chromium.org>
Tested-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Wai-Hong Tam <waihong@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Reviewed-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
/external/autotest/site_utils/suite_scheduler/suite_scheduler.py
a0db389036c2c0c3245a243001a515a29caab158 25-Jul-2015 Fang Deng <fdeng@chromium.org> [autotest] suite-scheduler sanity check should not rely on django

CL:223811 makes suite-scheduler depend on django. This
breaks the sanity check. This CL removes the dependency.

TEST=./site_utils/suite_scheduler/suite_scheduler.py --sanity
TEST= Run scheduler
BUG=chromium:474671

Change-Id: Ie9375ffc4b340949f62d6a4efe46625175ce06ba
Reviewed-on: https://chromium-review.googlesource.com/288396
Reviewed-by: Fang Deng <fdeng@chromium.org>
Tested-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Fang Deng <fdeng@chromium.org>
/external/autotest/site_utils/suite_scheduler/suite_scheduler.py
2324514d4a15579ffd4cf7c1395492272ea98599 22-Jan-2015 Dan Shi <dshi@chromium.org> [autotest] Prevent making gsutil call when running suite_scheduler.py --sanity

"suite_scheduler.py --sanity" command is called in pre-submit hook to validate
the format of suite_scheduler.ini. The dependency of gsutil causes repo upload
failure if gsutil is not installed. This CL removes such dependency.

BUG=chromium:450689
TEST=uninstall gsutil, run "suite_scheduler.py --sanity", confirm the test
passes. Run "suite_scheduler.py", confirm the script failed as gsutil is not
found.

"suite_scheduler --sanity command check

Change-Id: I5db91d60a52d425e712a7f9a55ea1f4262b3c8f7
Reviewed-on: https://chromium-review.googlesource.com/242651
Tested-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
/external/autotest/site_utils/suite_scheduler/suite_scheduler.py
b9144a457ac2be505eac18f61e75d68751b3cea0 02-Dec-2014 Dan Shi <dshi@chromium.org> [autotest] Check server database before services start

Add code in scheduler, host scheduler and suite scheduler, to check if the
server has required role in server database, if use_server_db is set to True
in global config.

scheduler will also get the drone list from server database. Drone manager will
load drone attributes like max_processes from server database.

BUG=chromium:422535
CQ-DEPEND=CL:232003
TEST=setup local server database, add servers for scheduler, host scheduler and
drone. Check following conditions:
1. Disable use_server_db, make sure all services can start and run tests.
2. Enable use_server_db, make sure all services can start and run tests.
3. Add a new server to be configured as primary of scheduler, host scheduler
and suite scheduler, try to start each service in a different server.
Confirm that the service can't start.

Change-Id: I5105f6de7ed959c76ed6b920240f1ba5898bebd6
Reviewed-on: https://chromium-review.googlesource.com/232525
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
Trybot-Ready: Dan Shi <dshi@chromium.org>
Tested-by: Dan Shi <dshi@chromium.org>
/external/autotest/site_utils/suite_scheduler/suite_scheduler.py
f571aa6b964930e4cd7ea0569105bebe06b97070 14-Oct-2014 Prashanth Balasubramanian <beeps@google.com> [autotest] Teach suite scheduler about Tot versions.

In an attempt to get suite scheduler out of the way so the team can
focus on higher priority demands over the next few months, this cl
drops all suites to run tot nightly and older branches weekly by
introducing a couple of new suite scheduler idioms.

TEST=Ran suite scheduler and checked that nightly suites are
scheduled against localhost.
BUG=chromium:417131,chromium:423847
DEPLOY=suite_scheduler

Change-Id: Ib3f01ac1e6c710ec969cf777d2c872e74a93058a
Reviewed-on: https://chromium-review.googlesource.com/223811
Reviewed-by: Dan Shi <dshi@chromium.org>
Tested-by: Prashanth B <beeps@chromium.org>
Commit-Queue: Prashanth B <beeps@chromium.org>
/external/autotest/site_utils/suite_scheduler/suite_scheduler.py
9a1987acc36506ceba0397ce64aeca1b39eb5b39 22-Aug-2013 Alex Miller <milleral@chromium.org> [autotest] Death to the tryjobs.

This removes reimager.py from dynamic suite, and follows through with
all the consequences of doing so:
* reimage_and_run has had the reimager completely dropped from it
* All reimaging-related parameters in SuiteSpec have been removed
* Any unittests for reimaging code were removed
* The suite scheduler sanity check for `num` was removed
* run_suite no longer reports on reimage times, as there are none
* skip_reimage has been removed, including in suite control files
* The dependencies_info file is no longer needed, so therefore:
* suite_preprocessor is being stripped down until it's removed in
the ebuilds

BUG=chromium:250586
TEST=unit, manual
Built trybot-lumpy-paladin/R31-4609.0.0-b1136 with this patch, and ran
various suites on it. Dummy ran as expected. Network3g_tmobile was all
TEST_NA as expected. BVT scheduled and ran tests. All seemed fine.
Also ran lumpy-release/R29-4319.78.0 to confirm backwards compatibility.

Change-Id: Ia75400f906eff5f3234a2a97ab82e1e014229e7c
Reviewed-on: https://chromium-review.googlesource.com/66607
Reviewed-by: Aviv Keshet <akeshet@chromium.org>
Tested-by: Alexander Miller <milleral@chromium.org>
Commit-Queue: Alexander Miller <milleral@chromium.org>
/external/autotest/site_utils/suite_scheduler/suite_scheduler.py
3ccd77da5cbbd540b09256b0b4d67f98cd795ea6 08-Aug-2013 Alex Miller <milleral@chromium.org> [autotest] Verify that suite control files exist in sanity check.

If someone adds a suite to suite_scheduler, it should probably actually
exist. This is pretty easy to check just by looking at the autotest
directory, so let's make sure someone doesn't accidentally upload a
silly CL.

BUG=chromium:220432
TEST=Ran, no output, and return value 0
Modified suite_scheduler.ini to have an invalid suite name, ran sanity
check, and the sanity check failed and printed out the invalid suite
name.

Change-Id: I2bbfb4e0563525a32523167d86a6fbb8a773ac98
Reviewed-on: https://gerrit.chromium.org/gerrit/65159
Reviewed-by: Prashanth Balasubramanian <beeps@chromium.org>
Commit-Queue: Alex Miller <milleral@chromium.org>
Tested-by: Alex Miller <milleral@chromium.org>
/external/autotest/site_utils/suite_scheduler/suite_scheduler.py
3197b39f82eb92afff33c7d44b805afe120c7627 26-Jun-2013 Fang Deng <fdeng@chromium.org> [autotest] Suite scheduler files bugs for exceptions raised in create_suite_job

This CL is to make suite scheduler file bugs using our bug filer for
several types of exceptions raised by create_suite_job. The exceptions are:
- ControlFileNotFound
- NoControlFileList
- ControlFileEmpty
- ControlFileMalformed

To enable bug filing, use '-b' option of suite_scheduler.py
Bug will be assgined to Lab sheriff.

This CL changes the private method _create_bug_report to public in
server/cros/dynamic_suite/reporting.py, so that we can file a general
bug that is not a test failure. After crbug.com/254256 is fixed,
the bug filing logic in deduping_scheduler.py should be modified.

This CL also moves ParseBuildName() from base_event.py to site_utils,
to avoid import loop. Callers of this methods are updated.

BUG=chromium:242569
TEST=deduping_scheduler_unittest.py;driver_unittest.py;base_event_unittest.py;
ran on chromeos-lab1.hot and confirm bug are properly filed.
DEPLOY=suite_scheduler

Change-Id: Ia57a2e625b7b39dcfe51892c208613c927f3a54e
Reviewed-on: https://gerrit.chromium.org/gerrit/60158
Commit-Queue: Fang Deng <fdeng@chromium.org>
Reviewed-by: Fang Deng <fdeng@chromium.org>
Tested-by: Fang Deng <fdeng@chromium.org>
/external/autotest/site_utils/suite_scheduler/suite_scheduler.py
57631dc37a9f25cda25af804c1bf50ef55b1c22f 22-Feb-2013 Dan Shi <dshi@chromium.org> [autotest] Add comma back in urls, minor fix in docstring

Add comma back in assignment statement to urls to make it a list.
Minor typo fix in suite_scheduler docstring.

BUG=None
TEST=run_sutie against lumpy with smoke suite, and run suite_scheduler
agaist a test suite.

Change-Id: I04696407cb44848de14160fc00c7ce7583210b31
Reviewed-on: https://gerrit.chromium.org/gerrit/43796
Commit-Queue: Dan Shi <dshi@chromium.org>
Reviewed-by: Dan Shi <dshi@chromium.org>
Tested-by: Dan Shi <dshi@chromium.org>
/external/autotest/site_utils/suite_scheduler/suite_scheduler.py
9979b5a5e449662ce2047151735b7ba7bb206ca5 02-Nov-2012 Alex Miller <milleral@chromium.org> [autotest] Add dependencies analysis to |suite_scheduler --sanity|

It's quite easy to accidentally make a suite suddenly unable to properly
run once tests start getting added that have DEPENDENCIES in them. Any
time that the number of different machine configurations exceeds the
number of machines allocated to the suite, some tests will never be able
to get run.

Thankfully, we can do roughly the same analysis statically over the
codebase that dynamic_suites does at runtime so that we can generate
instructions on what suites are currently crippled and provide
instructions on how to fix them.

BUG=chromium-os:35828
TEST=There already existed violating suites
$ ./site_utils/suite_scheduler/suite_scheduler.py --sanity
WARNING:root:Increase KernelDailyRegression to |num: 2|
WARNING:root:Increase NightlyRegression to |num: 2|
$ echo $?
1

Change-Id: I5dc7a4f7a12b0a5877b1d4437d2a7562de2d6dc5
Reviewed-on: https://gerrit.chromium.org/gerrit/40455
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Commit-Queue: Alex Miller <milleral@chromium.org>
Tested-by: Alex Miller <milleral@chromium.org>
/external/autotest/site_utils/suite_scheduler/suite_scheduler.py
bf8775a9336fd5da1afae117cd5796936625e101 10-Sep-2012 Chris Masone <cmasone@chromium.org> [autotest] Implement suite_scheduler.py --list_events

Oversight. Implement --list_events (-l) to list the event keywords
supported by suite_scheduler.py

BUG=chromium-os:32908
TEST=use the switch
TEST=unit

Change-Id: I569fc848d75770f31bb536bda3f69508dfbcf3c9
Reviewed-on: https://gerrit.chromium.org/gerrit/32766
Reviewed-by: Alex Miller <milleral@chromium.org>
Commit-Ready: Chris Masone <cmasone@chromium.org>
Tested-by: Chris Masone <cmasone@chromium.org>
/external/autotest/site_utils/suite_scheduler/suite_scheduler.py
420c71e4a80b41438d323f600657a0f5175c494c 28-Aug-2012 Alex Miller <milleral@chromium.org> Revert "[autotest] Refactor email sending out of suite_scheduler."

This reverts commit db9540abf2c7db97389e45643f81e02c97cc1281

Change-Id: I232cabeb770f7f5d0fdd55586e0705efd139d93a
Reviewed-on: https://gerrit.chromium.org/gerrit/31615
Commit-Ready: Alex Miller <milleral@chromium.org>
Reviewed-by: Alex Miller <milleral@chromium.org>
Tested-by: Alex Miller <milleral@chromium.org>
/external/autotest/site_utils/suite_scheduler/suite_scheduler.py
db9540abf2c7db97389e45643f81e02c97cc1281 24-Aug-2012 Alex Miller <milleral@chromium.org> [autotest] Refactor email sending out of suite_scheduler.

Sending warnings/errors as emails is a useful thing to be able to do
outside of suite_scheduler. This exposes the ability to create a logging
handler from a config file and a subject line that sends emails for a
certain severity level of logging messages.

BUG=chromium-os:32925
TEST=pylint, ran unittests
TEST=ran suite_scheduler.py and verified it sends emails

Change-Id: I4a678779ea06989d0f86a89017b4a6753afe257e
Reviewed-on: https://gerrit.chromium.org/gerrit/30348
Commit-Ready: Alex Miller <milleral@chromium.org>
Reviewed-by: Alex Miller <milleral@chromium.org>
Tested-by: Alex Miller <milleral@chromium.org>
/external/autotest/site_utils/suite_scheduler/suite_scheduler.py
b493555db2d43e79d96e793cae9d1ffb822dd6c1 14-Aug-2012 Chris Masone <cmasone@chromium.org> [autotest] Move dynamic suite code into its own module

No big rewrite, just moving files around

BUG=chromium-os:30266
TEST=unit
TEST=successful run_suite.py run
TEST=suite_enumerator.py, suite_preprocessor.py, and suite_scheduler.py

Change-Id: I2fbcae0332104ce3adcb10a1e90ce94cb209aca8
Reviewed-on: https://gerrit.chromium.org/gerrit/30267
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Commit-Ready: Chris Masone <cmasone@chromium.org>
Tested-by: Chris Masone <cmasone@chromium.org>
/external/autotest/site_utils/suite_scheduler/suite_scheduler.py
bb535e25277092c0f5b4fa9ecd4851ed4426e494 12-Jul-2012 Alex Miller <milleral@chromium.org> [autotest] Add sanity checking option to suite_scheduler.

Invoking suite_scheduler with the |-t| option will cause it to parse the
config file and validate it for missing required entries, bad |run_on|
keywords, and malformed |branch_specs| entries and unkown entries in
sections.

BUG=chromium-os:31126
TEST=run broken configs with -t
STATUS=fixed

Change-Id: Ie8ccde6713a76313e83979ee0869ed6741cb23c6
Reviewed-on: https://gerrit.chromium.org/gerrit/27247
Commit-Ready: Alex Miller <milleral@chromium.org>
Reviewed-by: Alex Miller <milleral@chromium.org>
Tested-by: Alex Miller <milleral@chromium.org>
/external/autotest/site_utils/suite_scheduler/suite_scheduler.py
c15c6b4f2be888ff8631d43571d697ccf287efd1 09-Jul-2012 Scott Zawalski <scottz@google.com> Fix a few issues with some recent changes to suite_scheduler.

EnumeratorException was not fully referenced causing crashes.
In deduping_scheduler a function definition was missing self.

Start printing tracebacks in the error messages logged in emailed to make it
easier to track down issues like the above.

TEST=Unittests
BUG=None

Change-Id: Ieee88f9a18debedb1dafdd92fd811d271e4f3816
Reviewed-on: https://gerrit.chromium.org/gerrit/26943
Reviewed-by: Chris Masone <cmasone@chromium.org>
Commit-Ready: Scott Zawalski <scottz@chromium.org>
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Tested-by: Scott Zawalski <scottz@chromium.org>
/external/autotest/site_utils/suite_scheduler/suite_scheduler.py
5f656f087d5efef832fe435c73a2c52df97a9616 19-Jun-2012 Chris Masone <cmasone@chromium.org> [autotest] Ensure suite_scheduler error mail is sent only for appropriate level

ERROR log messages from the suite_scheduler were being processed
by both the WARNING and ERROR SMTP log handlers, as is normal.
We don't want that, though, so ensure that ERROR messages are ignored
by the WARNING handler.

BUG=chromium-os:31378
TEST=run suite_scheduler, pkill git process. Check that only ERROR mail was sent.
STATUS=Fixed

Change-Id: I9dc3269aef371afceae1107274f8cac4327f0cfa
Reviewed-on: https://gerrit.chromium.org/gerrit/25567
Commit-Ready: Chris Masone <cmasone@chromium.org>
Reviewed-by: Chris Masone <cmasone@chromium.org>
Tested-by: Chris Masone <cmasone@chromium.org>
Reviewed-by: Scott Zawalski <scottz@chromium.org>
/external/autotest/site_utils/suite_scheduler/suite_scheduler.py
c60b61796336b0e6232377973bcb70058b5ccf12 18-Jun-2012 Chris Masone <cmasone@chromium.org> [autotest] Ensure that fatal exceptions in suite_scheduler generate email

Logging them at the ERROR level will do the trick.

BUG=chromium-os:31747
TEST=force git clone of manifest-versions.git to fail; ensure appropriate logging.
STATUS=Fixed

Change-Id: I0daa016eb1b46ded4695951303e779788c393317
Reviewed-on: https://gerrit.chromium.org/gerrit/25541
Tested-by: Chris Masone <cmasone@chromium.org>
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Commit-Ready: Chris Masone <cmasone@chromium.org>
/external/autotest/site_utils/suite_scheduler/suite_scheduler.py
8020951d8fde3225704f50480fc4c7c1f20938c1 17-May-2012 Chris Masone <cmasone@chromium.org> [autotest] Add SMTP log handler for suite_scheduler errors

Initially only send mail on ERROR level logging.

Mails are initially sent to chromeos-lab-admins@, from chromeos-autotest@
These are configurable via global_config.ini

BUG=chromium-os:30972
TEST=Add a logging.error() call, ensure that mail gets sent.

Change-Id: Iddb8f203952d39fa47c88aa636f19868bcba8828
Reviewed-on: https://gerrit.chromium.org/gerrit/22947
Tested-by: Chris Masone <cmasone@chromium.org>
Reviewed-by: Chris Sosa <sosa@chromium.org>
Commit-Ready: Chris Masone <cmasone@chromium.org>
/external/autotest/site_utils/suite_scheduler/suite_scheduler.py
4d0533eae8099da0a86c1faf294e2a6bd816bd80 14-May-2012 Chris Masone <cmasone@chromium.org> [autotest] Split suite_scheduler logging into multiple files

Log DEBUG and INFO messages to appropriately-suffixed log files.
INFO and higher still go to stdout as well.

BUG=chromium-os:30741
TEST=run scheduler, tail new .INFO log and watch for messages

Change-Id: I819b927a7c36ce419b5509ef958dc580cdab6c02
Reviewed-on: https://gerrit.chromium.org/gerrit/22620
Tested-by: Chris Masone <cmasone@chromium.org>
Reviewed-by: Chris Sosa <sosa@chromium.org>
Commit-Ready: Chris Masone <cmasone@chromium.org>
/external/autotest/site_utils/suite_scheduler/suite_scheduler.py
855d86f838faccb4ad112a222412e29c047b7d3e 07-May-2012 Chris Masone <cmasone@chromium.org> [autotest] Reload scheduler config file every time we loop

The suite scheduler uses a config file to tell it what tasks to perform.
Until now, that config was read at startup and then never again. Now,
the scheduler will re-read this config every time it's about to go through
and decide which events to handle, updating task lists and event parameters
prior to using them.

BUG=chromium-os:30521
TEST=unit
TEST=run the scheduler. Change the config file as it's running and observe
TEST=that the changes are honored. An easy change is to add a task configured
TEST=to run_on an event, and then set that event to always_handle.

Change-Id: Idfa9085503f97cd9f099161fd2d4d43ed139c26f
Reviewed-on: https://gerrit.chromium.org/gerrit/22070
Tested-by: Chris Masone <cmasone@chromium.org>
Reviewed-by: Chris Sosa <sosa@chromium.org>
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Commit-Ready: Chris Masone <cmasone@chromium.org>
/external/autotest/site_utils/suite_scheduler/suite_scheduler.py
a7c843fc4a2ba497910ad1f107652e4512be07c1 01-May-2012 Chris Masone <cmasone@chromium.org> [autotest] Doc update for suite_scheduler.py

Make docstring in suite_scheduler.py better reflect real life.

BUG=None
TEST=None

Change-Id: Ibef34f34712f4adddcacadc38d0f95d428e3d40e
Reviewed-on: https://gerrit.chromium.org/gerrit/21500
Commit-Ready: Chris Masone <cmasone@chromium.org>
Tested-by: Chris Masone <cmasone@chromium.org>
Reviewed-by: Scott Zawalski <scottz@chromium.org>
/external/autotest/site_utils/suite_scheduler/suite_scheduler.py
ccfa7d6f43984f9ff36f1b9037205b40dd49ae52 27-Apr-2012 Chris Masone <cmasone@chromium.org> [autotest] Decrease RPC timeout in suite_scheduler.py

We'd like to fail faster.

BUG=chromium-os:30140
TEST=run the scheduler, make sure it starts up.

Change-Id: I9cdd1efb3276a6e508b3827d900ce923c47bbefa
Reviewed-on: https://gerrit.chromium.org/gerrit/21354
Tested-by: Chris Masone <cmasone@chromium.org>
Reviewed-by: Chris Sosa <sosa@chromium.org>
Commit-Ready: Chris Masone <cmasone@chromium.org>
/external/autotest/site_utils/suite_scheduler/suite_scheduler.py
93f51d4da152538007d5b44a2dc9d2bbb1fe3429 18-Apr-2012 Chris Masone <cmasone@chromium.org> [autotest] Add BuildEvent and derived NewBuild event class

The NewBuild event will allow us to Run() tasks whenever a new build appears
for a supported board.

BUG=chromium-os:27586
TEST=unit

Change-Id: I3432ecde38397fde6e0d7d675bf302a796c117f5
Reviewed-on: https://gerrit.chromium.org/gerrit/20524
Tested-by: Chris Masone <cmasone@chromium.org>
Reviewed-by: Chris Sosa <sosa@chromium.org>
Commit-Ready: Chris Masone <cmasone@chromium.org>
/external/autotest/site_utils/suite_scheduler/suite_scheduler.py
83af70c9fbb25953187e24d0b04c4506bce50693 18-Apr-2012 Chris Masone <cmasone@chromium.org> [autotest] Fix up parsing of 'should_handle' in config file

Also, add some more debug-level logging.

BUG=chromium-os:27586
TEST=unit
TEST=./suite_scheduler.py -f suite_scheduler_test.ini --log_dir .

Change-Id: Ic5e1b90f248ff8a01923948061378c494e913cfb
Reviewed-on: https://gerrit.chromium.org/gerrit/20503
Commit-Ready: Chris Masone <cmasone@chromium.org>
Reviewed-by: Chris Masone <cmasone@chromium.org>
Tested-by: Chris Masone <cmasone@chromium.org>
/external/autotest/site_utils/suite_scheduler/suite_scheduler.py
9273c0ddfcdd07a298c2145270d7bf8cb9865e23 13-Apr-2012 Chris Masone <cmasone@chromium.org> [autotest] Fix path-globbing bug in ManifestVersions

As of this CL, I'm able to do an end-to-end test and see suites
get scheduled on my local AFE.

Also added a mechanism to config an event to fire every time the
scheduler loops back around, for testing purposes.

BUG=chromium-os:27586
TEST=unit
TEST=create a config with an event that will run_on=nightly, and a [nightly_params] section with always_fire: True

Change-Id: Ib09e3eeac06ef3eae341434aa465b6bd1d9c8afb
Reviewed-on: https://gerrit.chromium.org/gerrit/20251
Commit-Ready: Chris Masone <cmasone@chromium.org>
Reviewed-by: Chris Masone <cmasone@chromium.org>
Tested-by: Chris Masone <cmasone@chromium.org>
/external/autotest/site_utils/suite_scheduler/suite_scheduler.py
67f06d6003bb11f03f925810272f595d79dce44a 13-Apr-2012 Chris Masone <cmasone@chromium.org> [autotest] Enable forcing an event to fire and schedule suites

We need to ability to make the scheduler force an event, so we can backfill
test results, if need be. This is also handy for testing stuff.

Also, fix a bug in comparing build branch names to branch specifications
encoded in Tasks. Before, every Task would run on every release branch,
even if the Task said it should only run on factory and/or firmware.

BUG=chromium-os:27586
TEST=unit
TEST=./suite_scheduler.py -f suite_scheduler_test.ini -e nightly -i x86-mario-release/R20-2130.0.0-a1-b1905

Change-Id: I58217c358d8b85faf49a940d01bca184dc8590e0
Reviewed-on: https://gerrit.chromium.org/gerrit/20124
Commit-Ready: Chris Masone <cmasone@chromium.org>
Reviewed-by: Chris Masone <cmasone@chromium.org>
Tested-by: Chris Masone <cmasone@chromium.org>
/external/autotest/site_utils/suite_scheduler/suite_scheduler.py
10374501fd69934492499f26f025705d5e8dc2d9 13-Apr-2012 Chris Masone <cmasone@chromium.org> [autotest] Add handlers for HUP, TERM and INT so suite_scheduler.py exits clean

BUG=chromium-os:27586
TEST=run suite_scheduler.py, send SIGTERM, watch it exit(0)

Change-Id: Idd207c58829ac6913a88e2cc49139011de3bd55a
Reviewed-on: https://gerrit.chromium.org/gerrit/20128
Tested-by: Chris Masone <cmasone@chromium.org>
Reviewed-by: Chris Sosa <sosa@chromium.org>
Commit-Ready: Chris Masone <cmasone@chromium.org>
/external/autotest/site_utils/suite_scheduler/suite_scheduler.py
2d61ca263d3e571b19604ea4c9d7b55d83c28405 03-Apr-2012 Chris Masone <cmasone@chromium.org> [autotest] Begin adding looping logic to new suite scheduler

driver.Driver will hold the main loop logic for the scheduler. As the
code in event.py started to get too big, I broke it into base_event.py
and timed_event.py. Also, add ForgivingConfigParser, which just wraps
SafeConfigParser to return None on get*() calls.

BUG=chromium-os:27586
TEST=unit tests

Change-Id: I6e69830f176f533a9d448ac4372da68c8c70a14b
Reviewed-on: https://gerrit.chromium.org/gerrit/19488
Tested-by: Chris Masone <cmasone@chromium.org>
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Commit-Ready: Chris Masone <cmasone@chromium.org>
/external/autotest/site_utils/suite_scheduler/suite_scheduler.py