• Home
  • History
  • Annotate
  • only in /external/autotest/frontend/
History log of /external/autotest/frontend/
Revision Date Author Comments (<<< Hide modified files) (Show modified files >>>)
a4b55dd2b1387e41544c64c4793992b906272234 28-Feb-2018 Keith Haddow <haddowk@chromium.org> [moblab] Change the persistent id for moblab from mac to serial.

As next gen moblab have two nics we can no longer use thei
ethernet device type to find a consistent mac address.

Use the device serial number in place of the mac address.

CQ-DEPEND=CL:944613
BUG=chromium:814891
TEST=build and tested on moblab

Change-Id: I76284984c287054146a336a8972e206f4fe6b224
Reviewed-on: https://chromium-review.googlesource.com/944807
Commit-Ready: Keith Haddow <haddowk@chromium.org>
Tested-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Keith Haddow <haddowk@chromium.org>
fe/moblab_rpc_interface.py
lient/src/autotest/moblab/rpc/VersionInfo.java
lient/src/autotest/moblab/wizard/ConfigWizard.java
2e315cf2389fd1d9642a17694054f76e30f0240a 02-Mar-2018 Prathmesh Prabhu <pprabhu@chromium.org> Revert "rpc: Fetch from readonly db during shard heartbeat"

This reverts commit 3c75d9252ec132f724e72d025939a82fa64c7d14.

Reason for revert: Shard hearbeat failure in prod.

Original change's description:
> rpc: Fetch from readonly db during shard heartbeat
>
> Fall back to master if readonly isn't available (mostly during tests)
>
> BUG=chromium:810965
> TEST=unit tests
>
> Change-Id: I5442d7b31a79908e12d09a60bed3f42645422ebc
> Reviewed-on: https://chromium-review.googlesource.com/938384
> Commit-Ready: Jacob Kopczynski <jkop@chromium.org>
> Tested-by: Jacob Kopczynski <jkop@chromium.org>
> Reviewed-by: Xixuan Wu <xixuan@chromium.org>

BUG=chromium:810965
BUG=chromium:818271

Change-Id: I2c428cf91a6a57c2a96c98c89938f683120ed77b
Reviewed-on: https://chromium-review.googlesource.com/946750
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
Tested-by: Prathmesh Prabhu <pprabhu@chromium.org>
fe/models.py
c00f6032e3770919a50ab7ba3a6b71c7108e4399 22-Feb-2018 Xixuan Wu <xixuan@chromium.org> autotest: Refuse to set/delete static attribute.

This CL also verifies that class tko_test won't be broken by any new
changes about static attributes.

BUG=chromium:792309
TEST=Ran unittest.

Change-Id: Ie4bf605b162af20a2714667b75661547e70b616a
Reviewed-on: https://chromium-review.googlesource.com/933735
Commit-Ready: Xixuan Wu <xixuan@chromium.org>
Tested-by: Xixuan Wu <xixuan@chromium.org>
Reviewed-by: Shuqian Zhao <shuqianz@chromium.org>
fe/frontend_test_utils.py
fe/model_logic.py
fe/models.py
fe/models_test.py
fe/rpc_interface_unittest.py
ko/models_test.py
2975f40890696fbf63ba9cf1d93f6fdd40a10953 22-Feb-2018 Xixuan Wu <xixuan@chromium.org> autotest: Make RPC 'get_host_attributes' respect static attributes

BUG=chromium:792309
TEST=Ran unittest. Call RPC 'get_host_attributes' locally.

Change-Id: Iae7dccfc0c63481f07c6b4122774e7f5e02f2bd0
Reviewed-on: https://chromium-review.googlesource.com/929889
Commit-Ready: Xixuan Wu <xixuan@chromium.org>
Tested-by: Xixuan Wu <xixuan@chromium.org>
Reviewed-by: Shuqian Zhao <shuqianz@chromium.org>
fe/rpc_interface.py
fe/rpc_interface_unittest.py
91ecef5829e0587b802247d12f2b266408c7c3ae 22-Feb-2018 Xixuan Wu <xixuan@chromium.org> autotest: Make RPC 'get_hosts_by_attribute' respect static attributes.

BUG=chromium:792309
TEST=Ran unittest. Ran 'dummy_Pass' with local autotest.
Call RPC 'get_hosts_by_attribute' locally.

Change-Id: I61ad23fdb124300d93d70fe3d1dafc757fe03298
Reviewed-on: https://chromium-review.googlesource.com/930177
Commit-Ready: Xixuan Wu <xixuan@chromium.org>
Tested-by: Xixuan Wu <xixuan@chromium.org>
Reviewed-by: Shuqian Zhao <shuqianz@chromium.org>
fe/rpc_interface.py
fe/rpc_interface_unittest.py
bd9d5e4759d00aca07a807864bf8bba29aed37e4 20-Feb-2018 Xixuan Wu <xixuan@chromium.org> autotest: Make 'get_hosts' respect static attributes & labels.

BUG=chromium:792309
TEST=Ran unittest. Call RPC 'get_hosts' locally.

Change-Id: I1fd5cbe59781fd3c17816c5e386cffc05635c0ff
Reviewed-on: https://chromium-review.googlesource.com/929609
Commit-Ready: Xixuan Wu <xixuan@chromium.org>
Tested-by: Xixuan Wu <xixuan@chromium.org>
Reviewed-by: Shuqian Zhao <shuqianz@chromium.org>
fe/rpc_interface.py
fe/rpc_interface_unittest.py
fe/rpc_utils.py
3c75d9252ec132f724e72d025939a82fa64c7d14 22-Feb-2018 Jacob Kopczynski <jkop@google.com> rpc: Fetch from readonly db during shard heartbeat

Fall back to master if readonly isn't available (mostly during tests)

BUG=chromium:810965
TEST=unit tests

Change-Id: I5442d7b31a79908e12d09a60bed3f42645422ebc
Reviewed-on: https://chromium-review.googlesource.com/938384
Commit-Ready: Jacob Kopczynski <jkop@chromium.org>
Tested-by: Jacob Kopczynski <jkop@chromium.org>
Reviewed-by: Xixuan Wu <xixuan@chromium.org>
fe/models.py
f5321f056459ea0073df92983a1daebe425875d4 24-Feb-2018 Matt Mallett <mattmallett@chromium.org> [moblab] Run dut ssh test in parallel

The Manage DUTs page becomes slow with many DUTs connected, as
it checks each connected DUT for SSH connectivity in sequence.
This change has the checks run in parallel subprocesses and
provides a good speedup.

Also updated some messaging on UI for DUTs that couldn't be
reached by SSH, to make it clear what the problem is.

BUG=chromium:815275,chromium:814425
TEST=Added unit test for parallel test execution logic, and tested
functionality on local device with 24 connected DUTs.

Change-Id: Id087f02e9a5bf20e06cd536642164c356c747b74
Reviewed-on: https://chromium-review.googlesource.com/935881
Commit-Ready: Matt Mallett <mattmallett@chromium.org>
Tested-by: Matt Mallett <mattmallett@chromium.org>
Reviewed-by: Keith Haddow <haddowk@chromium.org>
fe/moblab_rpc_interface.py
fe/moblab_rpc_interface_unittest.py
lient/src/autotest/moblab/DutManagementView.java
903087bcc53368167631aa3c84b96c8bf648cca0 21-Feb-2018 Xixuan Wu <xixuan@chromium.org> autotest: Add django model class for table afe_static_host_attributes.

BUG=chromium:792309
TEST=Ran 'dummy_pass' in local autotest.

Change-Id: Ib289113990ef80d473a5c74874e2f11c77b9429a
Reviewed-on: https://chromium-review.googlesource.com/927747
Commit-Ready: Xixuan Wu <xixuan@chromium.org>
Tested-by: Xixuan Wu <xixuan@chromium.org>
Reviewed-by: Shuqian Zhao <shuqianz@chromium.org>
fe/models.py
a8dcf051840febbcf5f8003b18c187b44811c2ef 14-Feb-2018 Aviv Keshet <akeshet@chromium.org> autotest: afe models: make host deletion logs more verbose

Host deletion happens on shards when moving a board from one shard to
another. It is surprisingly slow, and impossible from the logs to
determine why. Add logging to make it easier to debug.

BUG=None
TEST=None

Change-Id: I7bd7552a87d591ce01ba2c6d7ec0d66fff6018f5
Reviewed-on: https://chromium-review.googlesource.com/917622
Commit-Ready: Aviv Keshet <akeshet@chromium.org>
Tested-by: Aviv Keshet <akeshet@chromium.org>
Reviewed-by: Allen Li <ayatane@chromium.org>
fe/models.py
b5b377277110180ea986e12c0b2e3b7c64e1102c 22-Feb-2018 Keith Haddow <haddowk@chromium.org> [moblab] Change the default wifi password to ''

Currnetly if there is no wifi password configured we are passing
the string None as the password change to empty string.

TEST=built fizz moblab
BUG=chromium:814467

Change-Id: I6d3b99ae636cb7c183283b45bd6bc4122508955b
Reviewed-on: https://chromium-review.googlesource.com/932543
Commit-Ready: Keith Haddow <haddowk@chromium.org>
Tested-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Matt Mallett <mattmallett@chromium.org>
fe/moblab_rpc_interface.py
ce956a406effa42c3f5afd9dffb2dc5db8aca8e5 22-Feb-2018 Aviv Keshet <akeshet@chromium.org> autotest: fix rpc_interface so unittests doesn't hang

BUG=None
TEST=None

Change-Id: I13ce7c26d60dff5c51c5998eda882f5e04a36dd9
Reviewed-on: https://chromium-review.googlesource.com/932343
Commit-Ready: Aviv Keshet <akeshet@chromium.org>
Tested-by: Aviv Keshet <akeshet@chromium.org>
Reviewed-by: Xixuan Wu <xixuan@chromium.org>
fe/rpc_interface.py
b16b58f261af2e017964f3b4b756daa87232ec8c 15-Feb-2018 Xixuan Wu <xixuan@chromium.org> autotest: add new Table afe_static_host_attributes

BUG=chromium:792309
TEST=Ran 'migrate.py sync 127' locally.

Change-Id: I6fbd9073c5205d8b178f28065d8041ae06f3c222
Reviewed-on: https://chromium-review.googlesource.com/922450
Commit-Ready: Xixuan Wu <xixuan@chromium.org>
Tested-by: Xixuan Wu <xixuan@chromium.org>
Reviewed-by: Shuqian Zhao <shuqianz@chromium.org>
igrations/127_add_afe_static_host_attributes.py
fc284adf9505e4882e8f2c01ee1fd52e3501892b 14-Feb-2018 Allen Li <ayatane@chromium.org> [autotest] Handle non-string types when logging host status change

BUG=chromium:812041
TEST=None

Change-Id: Ie007e98e112d331a8be283e1cccfa20163b963b0
Reviewed-on: https://chromium-review.googlesource.com/918964
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Paul Hobbs <phobbs@google.com>
fe/models.py
8a97cf84d3bfa7bcd5a4a45af8c2dbabd61daed9 08-Feb-2018 Matt Mallett <mattmallett@chromium.org> [moblab] More helpful error when RPC to fetch builds fails

BUG=chromium:666954
TEST=Local device. Used a fresh flash with no boto key installed
to prompt the error.

Change-Id: I4f6c080f1a31bc75a2a103e57303e76fd5994814
Reviewed-on: https://chromium-review.googlesource.com/909337
Commit-Ready: Matt Mallett <mattmallett@chromium.org>
Tested-by: Matt Mallett <mattmallett@chromium.org>
Reviewed-by: Keith Haddow <haddowk@chromium.org>
fe/moblab_rpc_interface.py
a9ab98d31018389691bbe9540cb7b147daf2c8ff 09-Feb-2018 Aviv Keshet <akeshet@chromium.org> autotest: make shard deletion rpc more reliable with iterative deletion

BUG=chromium:810570
TEST=Tested interactively

Change-Id: I22ebd89895e1a59aeab36171338b33513b476f58
Reviewed-on: https://chromium-review.googlesource.com/910326
Commit-Ready: Aviv Keshet <akeshet@chromium.org>
Tested-by: Aviv Keshet <akeshet@chromium.org>
Reviewed-by: Dan Shi <dshi@google.com>
fe/rpc_interface.py
77545a3bff0ee92466f6158855153fb6d40da21d 01-Feb-2018 Matt Mallett <mattmallett@chromium.org> [moblab] Remove orphaned attributes

Remove attributes associated with a DUT when it is deleted.
Also check for and remove attributes on a new DUT, so that
new DUTs aren't picking up orphaned attributes from previously
deleted DUTs.

BUG=chromium:783994
TEST=local moblab device

Change-Id: If244470c64226bb8c60a1edaf0da092f40261906
Reviewed-on: https://chromium-review.googlesource.com/896581
Commit-Ready: Keith Haddow <haddowk@chromium.org>
Tested-by: Matt Mallett <mattmallett@chromium.org>
Reviewed-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Keith Haddow <haddowk@chromium.org>
fe/models.py
c01e9167879b150e93919847d4000c42ba79f688 07-Feb-2018 Matt Mallett <mattmallett@chromium.org> [moblab] Add help text tooltip for boto key

BUG=chromium:630798
TEXT=test on local device

Change-Id: I8a93d4756a34d04442e0f651c1305bed6ca5e4da
Reviewed-on: https://chromium-review.googlesource.com/907544
Commit-Ready: Keith Haddow <haddowk@chromium.org>
Tested-by: Matt Mallett <mattmallett@chromium.org>
Reviewed-by: Keith Haddow <haddowk@chromium.org>
lient/src/autotest/moblab/wizard/CloudStorageCard.java
1674447ddd7815b1ac463a37084afbbcc8ea86e1 08-Feb-2018 Matt Mallett <mattmallett@chromium.org> [moblab] Show update errors on UI

If a user requests an update from the UI, and the update fails,
surface the error to the UI.

BUG=chromium:806311
TEST=test on local moblab, forced an update error by setting the
omaha url to an unresolvable address

Change-Id: I425a8013a1df1e09f9a8e8b821f5afb76496eb58
Reviewed-on: https://chromium-review.googlesource.com/909330
Commit-Ready: Matt Mallett <mattmallett@chromium.org>
Tested-by: Matt Mallett <mattmallett@chromium.org>
Reviewed-by: Keith Haddow <haddowk@chromium.org>
fe/moblab_rpc_interface.py
fe/moblab_rpc_interface_unittest.py
3b5a7067d8e5ef20ad3723236a724506e5675c98 05-Feb-2018 Matt Mallett <mattmallett@chromium.org> [moblab] Check if dut has test image by testing ssh connection

Attempt to SSH into connected duts. If SSH failed, it's likely the
DUT doesn't have a test image. Show a message on the UI to let users
know.

BUG=chromium:616373
TEST=local moblab with one test image DUT and one non-test image DUT

Change-Id: I3f9e82b04d7c47a19522f8fc627bcb06f28c7d02
Reviewed-on: https://chromium-review.googlesource.com/902166
Commit-Ready: Keith Haddow <haddowk@chromium.org>
Tested-by: Matt Mallett <mattmallett@chromium.org>
Reviewed-by: Keith Haddow <haddowk@chromium.org>
fe/moblab_rpc_interface.py
fe/moblab_rpc_interface_unittest.py
lient/src/autotest/moblab/DutManagementView.java
lient/src/autotest/moblab/rpc/ConnectedDutInfo.java
c70f8d638d90c1c20aab7dd94b5b310589bff0bf 07-Feb-2018 Matt Mallett <mattmallett@chromium.org> [moblab] Remove faft tests from UI

BUG=chromium:809606
TEST=test on local device

Change-Id: I62636dfe55d107e670f52175b23bc95d4804f38f
Reviewed-on: https://chromium-review.googlesource.com/907183
Commit-Ready: Keith Haddow <haddowk@chromium.org>
Tested-by: Matt Mallett <mattmallett@chromium.org>
Reviewed-by: Keith Haddow <haddowk@chromium.org>
lient/src/autotest/moblab/SuiteRunnerView.java
fb63402501587586ac5d9e8546c04a93147c4fec 02-Feb-2018 Matt Mallett <mattmallett@chromium.org> [moblab] fix None in wifi config wizard

BUG=chromium:808507
TEST=test on local moblab

Change-Id: I923ea6fcca56e4985e0da18c28e7adb886163c20
Reviewed-on: https://chromium-review.googlesource.com/900046
Commit-Ready: Matt Mallett <mattmallett@chromium.org>
Tested-by: Matt Mallett <mattmallett@chromium.org>
Reviewed-by: Keith Haddow <haddowk@chromium.org>
fe/moblab_rpc_interface.py
b879766de2fe6fcc9beb9dc76587a20fe2c9499d 02-Feb-2018 Matt Mallett <mattmallett@chromium.org> [moblab] correctly display update progress on UI

Wait for the update engine to finish checking for an update before
returning from the RPC. Also improve messaging on the UI to make
update progress easy to follow.

BUG=chromium:613747
TEST=local moblab tested by updating from R64-10176.62.0 to
R64010176.65.0

Change-Id: I16910cc1365dc65980e269da7bbcabc28e28af43
Reviewed-on: https://chromium-review.googlesource.com/898254
Commit-Ready: Matt Mallett <mattmallett@chromium.org>
Tested-by: Matt Mallett <mattmallett@chromium.org>
Reviewed-by: Keith Haddow <haddowk@chromium.org>
fe/moblab_rpc_interface.py
fe/moblab_rpc_interface_unittest.py
lient/src/autotest/moblab/rpc/VersionInfo.java
lient/src/autotest/moblab/wizard/ConfigWizard.java
1fc27d37218eb3ace5e5dad394b974a808d81d4d 27-Jan-2018 Allen Li <ayatane@chromium.org> [autotest] Add drone field to JobHandoff model

BUG=chromium:806128
TEST=None

Change-Id: I814bc4aeb0ea100a5cd524091a9905f2172c6b54
Reviewed-on: https://chromium-review.googlesource.com/894739
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Allen Li <ayatane@chromium.org>
fe/models.py
c223505b3ed3e90c57c5bd4fb98799c9e8fa0078 31-Jan-2018 Matt Mallett <mattmallett@chromium.org> [moblab] Use the is_reboot_needed value correctly

Use the return code from --is_reboot_needed correctly
to cause the moblab device to update when a new
version is downloaded and ready to install

BUG=chromium:613747
TEST=updated local moblab device from 10176.62.0 to 10176.65.0 via UI

Change-Id: I2204b3fcf2a2602b72350d380d3112a47a48a2ad
Reviewed-on: https://chromium-review.googlesource.com/895905
Commit-Ready: Matt Mallett <mattmallett@chromium.org>
Tested-by: Matt Mallett <mattmallett@chromium.org>
Reviewed-by: Keith Haddow <haddowk@chromium.org>
fe/moblab_rpc_interface.py
fe/moblab_rpc_interface_unittest.py
lient/src/autotest/moblab/wizard/ConfigWizard.java
31c1eec1ffaa0aae641db219eedfe53e0dd9bd30 31-Jan-2018 Matt Mallett <mattmallett@chromium.org> [moblab] Select model in run_suite page

Users can select the model in the board dropdown, for cases where
model is different from board and we want to target a specific
model for testing

BUG=b:69108684
TEST=local moblab test with a bruce and veyron_minnie device

Change-Id: I0da0eee86c3a043258df28c12df0644a53c0c771
Reviewed-on: https://chromium-review.googlesource.com/894640
Commit-Ready: Matt Mallett <mattmallett@chromium.org>
Tested-by: Matt Mallett <mattmallett@chromium.org>
Reviewed-by: Keith Haddow <haddowk@chromium.org>
fe/moblab_rpc_interface.py
fe/moblab_rpc_interface_unittest.py
lient/src/autotest/moblab/SuiteRunnerView.java
lient/src/autotest/moblab/rpc/ConnectedBoard.java
lient/src/autotest/moblab/rpc/MoblabRpcCallbacks.java
lient/src/autotest/moblab/rpc/MoblabRpcHelper.java
c224008dfbba6fd18db74253805fa338cb6d8017 31-Jan-2018 Allen Li <ayatane@chromium.org> Revert "[autotest] Add drone field to JobHandoff model"

This reverts commit 2e4dd10387d69bb1115260a55115d817da869247.

Reason for revert: <INSERT REASONING HERE>

Original change's description:
> [autotest] Add drone field to JobHandoff model
>
> BUG=chromium:806128
> TEST=None
>
> Change-Id: I5873942c20a1d54eb8f9a20a1aa95f8a8de9a054
> Reviewed-on: https://chromium-review.googlesource.com/890104
> Commit-Ready: Allen Li <ayatane@chromium.org>
> Tested-by: Allen Li <ayatane@chromium.org>
> Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>

Bug: chromium:806128
Change-Id: I1a0026b354c805f3df594c915ac56d19aa61d676
Reviewed-on: https://chromium-review.googlesource.com/894823
Reviewed-by: Jacob Kopczynski <jkop@chromium.org>
Tested-by: Jacob Kopczynski <jkop@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Commit-Queue: Allen Li <ayatane@chromium.org>
Commit-Queue: Jacob Kopczynski <jkop@chromium.org>
Trybot-Ready: Allen Li <ayatane@chromium.org>
fe/models.py
2e4dd10387d69bb1115260a55115d817da869247 27-Jan-2018 Allen Li <ayatane@chromium.org> [autotest] Add drone field to JobHandoff model

BUG=chromium:806128
TEST=None

Change-Id: I5873942c20a1d54eb8f9a20a1aa95f8a8de9a054
Reviewed-on: https://chromium-review.googlesource.com/890104
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
fe/models.py
fd293cbaf6bfcc2ff332bf42097b7d2e8cae200e 30-Jan-2018 Xixuan Wu <xixuan@chromium.org> autotest: Fix dummy_fail.dependency test.

BUG=chromium:807074
TEST=Ran dummy_fail:dependency in local autotest.
Ran unittest.

Change-Id: I41e64d704af67a339d3f8c133c6d5e575d63a4c9
Reviewed-on: https://chromium-review.googlesource.com/892321
Commit-Ready: Shuqian Zhao <shuqianz@chromium.org>
Tested-by: Xixuan Wu <xixuan@chromium.org>
Reviewed-by: Shuqian Zhao <shuqianz@chromium.org>
fe/rpc_utils.py
2c69cf6cb2db6c7d15724886e3084d992a572692 27-Jan-2018 Allen Li <ayatane@chromium.org> [autotest] Add drone column to job_handoff table

BUG=chromium:806128
TEST=Run migration locally back and forth

Change-Id: Iceae4163f743efb1998a578263b2521d64488721
Reviewed-on: https://chromium-review.googlesource.com/890103
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Allen Li <ayatane@chromium.org>
igrations/126_add_job_handoff_drone_column.py
4379e9309115337fd49d58140ad1a70c9790e69f 17-Jan-2018 Xixuan Wu <xixuan@chromium.org> autotest: Make 'assign_to_shard' respect static labels.

BUG=chromium:792309
TEST=Ran unittest.

Change-Id: I681b9f947ab7e80d1d307c679dc0a5aa8911fb6b
Reviewed-on: https://chromium-review.googlesource.com/871047
Commit-Ready: Xixuan Wu <xixuan@chromium.org>
Tested-by: Xixuan Wu <xixuan@chromium.org>
Reviewed-by: Xixuan Wu <xixuan@chromium.org>
fe/models.py
fe/rpc_interface_unittest.py
ab2b93671ed54c266768b4b95fce8907e7c68b0c 17-Jan-2018 Xixuan Wu <xixuan@chromium.org> autotest: Let moblab_interface respect static labels.

BUG=chromium:792309
TEST=Ran unittest.

Change-Id: I7ae072a44bf31ae6bfe4be1d429bfcc1d99e9ab2
Reviewed-on: https://chromium-review.googlesource.com/869141
Commit-Ready: Xixuan Wu <xixuan@chromium.org>
Tested-by: Xixuan Wu <xixuan@chromium.org>
Reviewed-by: Shuqian Zhao <shuqianz@chromium.org>
fe/moblab_rpc_interface.py
2dc10cea03bf146ce5a516a0d5ffc1e6de470c46 16-Jan-2018 Xixuan Wu <xixuan@chromium.org> autotest: Make job dependencies respect RESPECT_STATIC_LABEL

BUG=chromium:792309
TEST=Ran unittest.

Change-Id: I22356aee23892661d382dc3473ac685fc90d72f7
Reviewed-on: https://chromium-review.googlesource.com/869131
Commit-Ready: Xixuan Wu <xixuan@chromium.org>
Tested-by: Xixuan Wu <xixuan@chromium.org>
Reviewed-by: Shuqian Zhao <shuqianz@chromium.org>
fe/rpc_interface_unittest.py
fe/rpc_utils.py
26d17d7687f9109b2e509f2bf97724796627253b 26-Jan-2018 Aviv Keshet <akeshet@chromium.org> autotest: rpc_interface: make delete_shard simpler

The current delete_shard behavior contains a lot of complicated, slow,
and not very reliable code to try to babysit the state of DUTs during
the shard deletion transition. This code is slow and prone to
breakage, and the value it provides is dubious because DUTs will end up
with eventually correctness anyway.

Delete all the complicated logic, and just make delete_shard delete the
shard.

BUG=chromium:806131, chromium:806129
TEST=None

Change-Id: If8d877bce01c0dbd824746f62160331b7a886cdd
Reviewed-on: https://chromium-review.googlesource.com/888086
Commit-Ready: Aviv Keshet <akeshet@chromium.org>
Tested-by: Aviv Keshet <akeshet@chromium.org>
Reviewed-by: Dan Shi <dshi@google.com>
fe/rpc_interface.py
fe/rpc_interface_unittest.py
20b323f23f56e15e506d3849f48e37ee3b86f4b8 12-Jan-2018 Xixuan Wu <xixuan@chromium.org> autotest: Modify RPC remove_board_from_shard

BUG=chromium:792309
TEST=Ran unittest.

Change-Id: I0b1f37648501ae06d39818c79646be4cebd6e699
Reviewed-on: https://chromium-review.googlesource.com/865638
Commit-Ready: Xixuan Wu <xixuan@chromium.org>
Tested-by: Xixuan Wu <xixuan@chromium.org>
Reviewed-by: Shuqian Zhao <shuqianz@chromium.org>
fe/rpc_interface.py
fe/rpc_interface_unittest.py
b0f8dc76b36d2b04e08024e98add237597e59d35 26-Jan-2018 Matt Mallett <mattmallett@chromium.org> [moblab] Add update button

BUG=chromium:613747
TEST=Tested features on a local moblab

Change-Id: Id512a96e57c5449ace561b8a551b2570ab637468
Reviewed-on: https://chromium-review.googlesource.com/887794
Commit-Ready: Matt Mallett <mattmallett@chromium.org>
Tested-by: Matt Mallett <mattmallett@chromium.org>
Reviewed-by: Keith Haddow <haddowk@chromium.org>
fe/moblab_rpc_interface.py
fe/moblab_rpc_interface_unittest.py
lient/src/autotest/moblab/rpc/MoblabRpcHelper.java
lient/src/autotest/moblab/rpc/VersionInfo.java
lient/src/autotest/moblab/wizard/ConfigWizard.java
040fc8b28c188e8f909a837f8317f10077f05ba9 12-Jan-2018 Xixuan Wu <xixuan@chromium.org> autotest: Modify RPC 'label_remove_hosts' & 'host_remove_labels'

BUG=chromium:792309
TEST=Ran unittest.
Call RPC locally with local Mysql.

Change-Id: I73a054f104c136293d57f6bba1659b0f0d700382
Reviewed-on: https://chromium-review.googlesource.com/865555
Commit-Ready: Xixuan Wu <xixuan@chromium.org>
Tested-by: Xixuan Wu <xixuan@chromium.org>
Reviewed-by: Shuqian Zhao <shuqianz@chromium.org>
fe/models.py
fe/rpc_interface.py
fe/rpc_interface_unittest.py
79cbf49949a471baeffb67f47e745c12396ff7fb 12-Jan-2018 Xixuan Wu <xixuan@chromium.org> autotest: Make 'get_labels' respect static labels.

BUG=chromium:792309
TEST=Ran unittest.
Ran RPC get_labels directly with local mysql.

Change-Id: I0b0763fe0d6f9c9b3eef41b12c8a3422897e2f3c
Reviewed-on: https://chromium-review.googlesource.com/862843
Commit-Ready: Xixuan Wu <xixuan@chromium.org>
Tested-by: Xixuan Wu <xixuan@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
fe/rpc_interface.py
fe/rpc_interface_unittest.py
8c7113a941511fcf77a05602142e7525dea7bae3 10-Jan-2018 Xixuan Wu <xixuan@chromium.org> autotest: Use Host.get_hosts_with_labels to find hosts in scheduler.

BUG=chromium:792309
TEST=Ran scheduling suites with local autotest.
Ran unittest.

Change-Id: I444f703dd4a9114073d7908999d21344ed590032
Reviewed-on: https://chromium-review.googlesource.com/858487
Commit-Ready: Xixuan Wu <xixuan@chromium.org>
Tested-by: Xixuan Wu <xixuan@chromium.org>
Reviewed-by: Xixuan Wu <xixuan@chromium.org>
fe/models.py
fe/rpc_utils.py
e7706031001503f1840d0b62b227c42bf59df102 22-Dec-2017 Xixuan Wu <xixuan@chromium.org> autotest: Modify RPCs about add labels to hosts.

BUG=chromium:792309
TEST=1. Ran unittest.
2. Ran RPC 'label_add_hosts' and 'host_add_labels' locally.
3. Ran ./cli/atest label add <label_name> -m <host_name>

Change-Id: I5329ec381c3b466cc6ad8687c8e5fa15cf91d525
Reviewed-on: https://chromium-review.googlesource.com/841531
Commit-Ready: Xixuan Wu <xixuan@chromium.org>
Tested-by: Xixuan Wu <xixuan@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
fe/models.py
fe/rpc_interface.py
fe/rpc_interface_unittest.py
9d9549f0878678f3ded9320a6225c295b6f72511 13-Jan-2018 Xixuan Wu <xixuan@chromium.org> autotest: Fix get_host_query by first calling non-sliced queries.

If a django QuerySet is a sliced QuerySet, which means it's filtered by
queries like 'query_limit=30', it cannot be filtered by other queries again,
due to 'Cannot filter a query once a slice has been taken':

https://docs.djangoproject.com/en/1.8/ref/models/querysets/.

This CL changes get_host_query to first call get_hosts_with_labels,
which doesn't include sliced queries, then call passed-in filter_data.

Also this CL changes that if a label doesn't exist in db, no host will
be matched.

BUG=chromium:801301
TEST=Ran unittest. Test local AFE to verify it works.

Change-Id: I9b18262b7bbee1d993f21b19636eee2070f6715c
Reviewed-on: https://chromium-review.googlesource.com/865958
Commit-Ready: Xixuan Wu <xixuan@chromium.org>
Tested-by: Xixuan Wu <xixuan@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
fe/models.py
fe/rpc_utils.py
c820ea892efc9d6fdf60b0ee7bb519ab400ebbb1 21-Dec-2017 Xixuan Wu <xixuan@chromium.org> autotest: Don't modify static labels in afe_labels.

BUG=chromium:792309
TEST=1. Ran unittest.
2. Calls RPC locally.

Change-Id: I8f087c596d7b03332155e95c880ac9629c867bcd
Reviewed-on: https://chromium-review.googlesource.com/841143
Commit-Ready: Xixuan Wu <xixuan@chromium.org>
Tested-by: Xixuan Wu <xixuan@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
fe/rpc_interface.py
fe/rpc_interface_unittest.py
76cd05821a91c15e76dac6a6609d0516539e7c42 21-Dec-2017 Xixuan Wu <xixuan@chromium.org> autotest: Don't delete static labels in afe_labels.

BUG=chromium:792309
TEST=1. Ran unittest.
2. Calls RPC locally.

Change-Id: I1e96f9628da8888ad0d14ff5d6446a0a9f6d12da
Reviewed-on: https://chromium-review.googlesource.com/841102
Commit-Ready: Xixuan Wu <xixuan@chromium.org>
Tested-by: Xixuan Wu <xixuan@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
fe/frontend_test_utils.py
fe/models.py
fe/rpc_interface.py
fe/rpc_interface_unittest.py
bc3e0c89c3f42c3e59b33857d5ecd5622526460a 18-Dec-2017 Xixuan Wu <xixuan@chromium.org> autotest: Add endpoints in afe.models.Host to coperate with static
labels.

BUG=chromium:792309
TEST=1. Ran unittest;
2. Ran with local autotest & migrated local mysql;
3. Ran ./cli/atest host list.

Change-Id: I55bd9d528602b3f300a324dfed9b931fc08f7286
Reviewed-on: https://chromium-review.googlesource.com/833207
Commit-Ready: Xixuan Wu <xixuan@chromium.org>
Tested-by: Xixuan Wu <xixuan@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
fe/models.py
fe/rpc_interface.py
fe/rpc_utils.py
cc12f3e61d8396a66d667eed71a9d0ff3ff28176 03-Jan-2018 Xixuan Wu <xixuan@chromium.org> autotest: Add db class in afe.models.

BUG=chromium:792309
TEST=Ran autotest locally with locally migrated db.
Ran unittest.

Change-Id: Ie3f98b5f4473123e7de13de6bf159d51c8772cce
Reviewed-on: https://chromium-review.googlesource.com/848192
Commit-Ready: Xixuan Wu <xixuan@chromium.org>
Tested-by: Xixuan Wu <xixuan@chromium.org>
Reviewed-by: Xixuan Wu <xixuan@chromium.org>
fe/models.py
b8b2e5916a157639b0f089cfc71f9faf46adfecc 15-Dec-2017 Allen Li <ayatane@chromium.org> [autotest] Add GATHERING handling

BUG=chromium:794779
TEST=None

Change-Id: I5a5bd1f6d8c755e95b1ffd8e7740fb53a35700a3
Reviewed-on: https://chromium-review.googlesource.com/830648
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
fe/models.py
536a63d130d13ffb7335b4e3c76e3df4ea4ccaff 06-Jan-2018 Prathmesh Prabhu <pprabhu@chromium.org> Enable moblab-generic-vm-pre-cq for infrastructure code.

BUG=chromium:725620
TEST=None. This CL only comes in effect once landed.

Change-Id: I4d6e6bd0d4f27dcc3ed1c9f30b39e1a9084170de
Reviewed-on: https://chromium-review.googlesource.com/853321
Tested-by: Prathmesh Prabhu <pprabhu@chromium.org>
Reviewed-by: Aviv Keshet <akeshet@chromium.org>
/external/autotest/COMMIT-QUEUE-moblab-pre-cq-only.ini
d5d752ec634e73f79334d6aabae9944669eb8e17 03-Jan-2018 Xixuan Wu <xixuan@chromium.org> autotest: add new table afe_replaced_labels.

BUG=chromium:791808
TEST=Ran 'migrate.py sync 125' locally.

Change-Id: I0bccba03cecbe7800f2beffb89e790c4380b6123
Reviewed-on: https://chromium-review.googlesource.com/847901
Commit-Ready: Xixuan Wu <xixuan@chromium.org>
Tested-by: Xixuan Wu <xixuan@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
igrations/125_add_replaced_labels_tables.py
ec9941ab88718a2432849b57b95e13f3ee5af5b3 02-Jan-2018 Xixuan Wu <xixuan@chromium.org> autotest: change migration 124 to no-op migration.

BUG=chromium:796210
TEST=None

Change-Id: Ieec22518dd798dd6cc03cb17900348ec3fa44aa5
Reviewed-on: https://chromium-review.googlesource.com/847961
Commit-Ready: Xixuan Wu <xixuan@chromium.org>
Tested-by: Xixuan Wu <xixuan@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
igrations/124_add_replaced_column_to_afe_labels.py
5ded661fb3a7e678f1db6906f2de13d2338b1e04 02-Jan-2018 Xixuan Wu <xixuan@chromium.org> Revert "Reland "autotest: add replaced_by_static_label column to afe_labels""

This reverts commit 3657b8518b77ace90ac109b5da6ad31bcd5778cf.

Reason for revert: crbug.com/796210

Original change's description:
> Reland "autotest: add replaced_by_static_label column to afe_labels"
>
> This is a reland of a69b15cdde11dbbdd96be9fd95eb2b3c7c45d76c
> Original change's description:
> > autotest: add replaced_by_static_label column to afe_labels
> >
> > Add one new column, replaced, to afe_labels, which means the value of
> > this
> > label is replaced by the corresponding one in afe_static_labels. This CL
> > is one step to migrate the local afe static labels data to inventory
> > service.
> >
> > BUG=chromium:791808
> > TEST=None
> >
> > Change-Id: Ic5173534094b53d86fbc4c5a1d93aec5d94642fb
> > Reviewed-on: https://chromium-review.googlesource.com/816074
> > Commit-Ready: Xixuan Wu <xixuan@chromium.org>
> > Tested-by: Xixuan Wu <xixuan@chromium.org>
> > Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
>
> Bug: chromium:791808
> Change-Id: I0aeed7da1e6e4cad3a25582d5c001acb3fe83f04

Bug: chromium:791808
Change-Id: Ife7605f814c7cabeb82d42f1d59f7a1bfe06b775
Reviewed-on: https://chromium-review.googlesource.com/847955
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
Tested-by: Xixuan Wu <xixuan@chromium.org>
Commit-Queue: Xixuan Wu <xixuan@chromium.org>
igrations/124_add_replaced_column_to_afe_labels.py
aec4a067cbdaf178ce3221da514d02e3fbafe38d 21-Dec-2017 Keith Haddow <haddowk@chromium.org> [moblab] Fix broken UI, deleted wrong suite name

BUG=chromium:772192
TEST=local build

Change-Id: I31eb909f14bfc64162dc3b27c606db1b7afe7b98
lient/src/autotest/moblab/SuiteRunnerView.java
e6c686186ecb9992cb69505cb7853c46d12be8df 21-Dec-2017 Xixuan Wu <xixuan@chromium.org> Revert "autotest: Add dbclass in afe.models."

This reverts commit 9f112777d0b6298288debfa2c443a42f82c9919a.

Reason for revert: <INSERT REASONING HERE>

Original change's description:
> autotest: Add dbclass in afe.models.
>
> BUG=chromium:792309
> TEST=Ran unittest. Ran with local autotest & migrated local mysql.
>
> Change-Id: I993f56f1e9ca355923511657d605cc69bcf3e91b

Bug: chromium:792309
Change-Id: I03bff0908c3b67f09b42e3df9616c917891e7ca9
Reviewed-on: https://chromium-review.googlesource.com/840784
Tested-by: Xixuan Wu <xixuan@chromium.org>
Commit-Queue: Xixuan Wu <xixuan@chromium.org>
Reviewed-by: Xixuan Wu <xixuan@chromium.org>
fe/models.py
fe/models_test.py
9f112777d0b6298288debfa2c443a42f82c9919a 13-Dec-2017 Xixuan Wu <xixuan@chromium.org> autotest: Add dbclass in afe.models.

BUG=chromium:792309
TEST=Ran unittest. Ran with local autotest & migrated local mysql.

Change-Id: I993f56f1e9ca355923511657d605cc69bcf3e91b
fe/models.py
fe/models_test.py
3657b8518b77ace90ac109b5da6ad31bcd5778cf 13-Dec-2017 shuqian zhao <shuqianz@google.com> Reland "autotest: add replaced_by_static_label column to afe_labels"

This is a reland of a69b15cdde11dbbdd96be9fd95eb2b3c7c45d76c
Original change's description:
> autotest: add replaced_by_static_label column to afe_labels
>
> Add one new column, replaced, to afe_labels, which means the value of
> this
> label is replaced by the corresponding one in afe_static_labels. This CL
> is one step to migrate the local afe static labels data to inventory
> service.
>
> BUG=chromium:791808
> TEST=None
>
> Change-Id: Ic5173534094b53d86fbc4c5a1d93aec5d94642fb
> Reviewed-on: https://chromium-review.googlesource.com/816074
> Commit-Ready: Xixuan Wu <xixuan@chromium.org>
> Tested-by: Xixuan Wu <xixuan@chromium.org>
> Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>

Bug: chromium:791808
Change-Id: I0aeed7da1e6e4cad3a25582d5c001acb3fe83f04
igrations/124_add_replaced_column_to_afe_labels.py
2892ff6fa7657b9a81aaae00445b38e8937ec4b0 19-Dec-2017 Prathmesh Prabhu <pprabhu@chromium.org> autotest: Respect url protocol in RPC client library.

BUG=chromium:795874
TEST=Manually run atest and dut-status commands against http and https
servers.

Change-Id: I4b609101f3d48cd46c766c04d72ac9a6a3299f8b
fe/rpc_client_lib.py
18f5649b7d5e1e6f173ceb5f793306a7c09d8099 14-Nov-2017 Shuhei Takahashi <nya@chromium.org> Add configs to support Stainless in autotest.

When Stainless related configs are set, it overrides wmatrix links.

BUG=b:69277911
TEST=AFE UI: Run AFE locally
TEST=run_suite: NOT TESTED

Change-Id: Ie002c8efa8487a0e69ea0830e959a7edcbd14029
fe/rpc_interface.py
fe/rpc_utils.py
lient/src/autotest/afe/AfeClient.java
lient/src/autotest/public/AfeClient.html
b8280cef9d858373ab2e9851dc2e2d08183c05f7 19-Dec-2017 Richard Barnette <jrbarnette@chromium.org> Revert "autotest: add replaced_by_static_label column to afe_labels"

This reverts commit a69b15cdde11dbbdd96be9fd95eb2b3c7c45d76c.

This breaks moblab

BUG=chromium:796210
TEST=None

Change-Id: Ieb01bce545bb0b4bb14505e6f766c592f7fc7a90
Reviewed-on: https://chromium-review.googlesource.com/834649
Reviewed-by: Richard Barnette <jrbarnette@google.com>
Tested-by: Richard Barnette <jrbarnette@google.com>
igrations/124_add_replaced_column_to_afe_labels.py
a69b15cdde11dbbdd96be9fd95eb2b3c7c45d76c 13-Dec-2017 shuqian zhao <shuqianz@google.com> autotest: add replaced_by_static_label column to afe_labels

Add one new column, replaced, to afe_labels, which means the value of
this
label is replaced by the corresponding one in afe_static_labels. This CL
is one step to migrate the local afe static labels data to inventory
service.

BUG=chromium:791808
TEST=None

Change-Id: Ic5173534094b53d86fbc4c5a1d93aec5d94642fb
Reviewed-on: https://chromium-review.googlesource.com/816074
Commit-Ready: Xixuan Wu <xixuan@chromium.org>
Tested-by: Xixuan Wu <xixuan@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
igrations/124_add_replaced_column_to_afe_labels.py
10d0c41923a6c0e69d34bd55bc83141053ebb9e4 13-Dec-2017 shuqian zhao <shuqianz@google.com> autotest: add two new tables and one new column for afe_labels migration

Add two new tables: afe_static_labels and afe_static_hosts_labels. Add
one new column, replaced, to afe_labels, which means the value of this
label is replaced by the corresponding one in afe_static_labels. This CL
is one step to migrate the local afe static labels data to inventory
service.

BUG=chromium:791808
TEST=run the following command on a test ganeti server:
$AUTOTEST_DIR/database/migrate.py sync --debug
The new changes are in the database.

Change-Id: Ida6409c122206edc58e2a137c4aa02c1e07ab16f
Reviewed-on: https://chromium-review.googlesource.com/816054
Commit-Ready: Xixuan Wu <xixuan@chromium.org>
Tested-by: Xixuan Wu <xixuan@chromium.org>
Reviewed-by: Xixuan Wu <xixuan@chromium.org>
igrations/123_add_static_labels_tables.py
51b0c60155e1415a49398a5995dd02a1f2a32dc9 09-Dec-2017 Xixuan Wu <xixuan@chromium.org> autotest: remove support for 'only_if_needed_labels'.

BUG=chromium:792309
TEST=Ran unittest.

Change-Id: I4b8a09161c507ef70aab80409858d0730648c596
Reviewed-on: https://chromium-review.googlesource.com/820101
Commit-Ready: Xixuan Wu <xixuan@chromium.org>
Tested-by: Xixuan Wu <xixuan@chromium.org>
Reviewed-by: Xixuan Wu <xixuan@chromium.org>
fe/rpc_interface.py
fe/rpc_interface_unittest.py
fe/rpc_utils.py
593b3a99377543946bd90eea79eb4fa7fa4ce07f 15-Dec-2017 Prathmesh Prabhu <pprabhu@chromium.org> cli: Improve error message on missing SSO credentials.

With use_sso_client, users AFE client tools need to obtain loas
credentials to access uberproxy services. Improve the error message in
this case instructing the user to obtain the required credentials.

BUG=b:32303896
TEST=./cli/atest host list -w https://ubercautotest.corp.google.com
w/ and w/o loas credentials.

Change-Id: Iec84f806afc3e328d2562f106c4a8f22720f5b3a
Reviewed-on: https://chromium-review.googlesource.com/828278
Commit-Ready: Prathmesh Prabhu <pprabhu@chromium.org>
Tested-by: Prathmesh Prabhu <pprabhu@chromium.org>
Reviewed-by: Aviv Keshet <akeshet@chromium.org>
fe/json_rpc/proxy.py
02163aa9d8cb6b98fe1d3b0c3d94902f0f0b35e1 13-Dec-2017 Prathmesh Prabhu <pprabhu@chromium.org> Support uberproxy AFE service from RPC clients

This CL is adding support for hitting the uberproxy AFE service(s) from
autotest CLI tools (atest, dut-status, balance_pool, run_suite,
test_that should all work).

To use this, add to your shadow_config.ini
[CLIENT]
use_sso_client: True

and target one of the uberproxy services ***using https***

Once this has been proven, the intention is to turn it on by default
everywhere outside the lab. At that point, it will become impossible to
directly hit any of the servers without going via uberproxy.

BUG=b:32303896
TEST=./cli/atest host list \
-w https://ubercautotest.corp.google.com \
-b 'board:kevin'

Change-Id: I7deb459c49cc4eae919da1ac29b6e39287db6aab
Reviewed-on: https://chromium-review.googlesource.com/823648
Commit-Ready: Prathmesh Prabhu <pprabhu@chromium.org>
Tested-by: Prathmesh Prabhu <pprabhu@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
fe/json_rpc/proxy.py
93e646cbcab150b4ae11945592ff969e464ad31a 08-Dec-2017 Xixuan Wu <xixuan@chromium.org> autotest: remove dependencies of suite_scheduler.

BUG=chromium:793179
TEST=Ran unittest.

Change-Id: I633a1ec2ccc352abd4da20c8b7667b0eb6d3305e
Reviewed-on: https://chromium-review.googlesource.com/816080
Commit-Ready: Xixuan Wu <xixuan@chromium.org>
Tested-by: Xixuan Wu <xixuan@chromium.org>
Reviewed-by: Dan Shi <dshi@google.com>
erver/models.py
e714771fca6ae01879d9df2bcf49c60bcd7897b7 07-Dec-2017 Shuqian Zhao <shuqianz@chromium.org> Revert "autotest: add replaced_by_static_label column to afe_labels"

This reverts commit d29baf9818784ac5f7fa69c2087f4f7ba9fcd27a.

Reason for revert: <INSERT REASONING HERE>

Original change's description:
> autotest: add replaced_by_static_label column to afe_labels
>
> Add one new column, replaced, to afe_labels, which means the value of this
> label is replaced by the corresponding one in afe_static_labels. This CL
> is one step to migrate the local afe static labels data to inventory
> service.
>
> BUG=chromium:791808
> TEST=None
>
> Change-Id: I809a262667dfe97595cc575f373610c86d4df571
> Reviewed-on: https://chromium-review.googlesource.com/809327
> Tested-by: Shuqian Zhao <shuqianz@chromium.org>
> Reviewed-by: Dan Shi <dshi@google.com>
> Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>

Bug: chromium:791808
Change-Id: I1895342b6fc3611c959519ba1fbac6dce6256379
Reviewed-on: https://chromium-review.googlesource.com/814616
Reviewed-by: Shuqian Zhao <shuqianz@chromium.org>
Tested-by: Shuqian Zhao <shuqianz@chromium.org>
igrations/124_add_replaced_column_to_afe_labels.py
6c9d62475d817883c10c61083b0210f0b0e093a5 07-Dec-2017 Shuqian Zhao <shuqianz@chromium.org> Revert "autotest: add two new tables and one new column for afe_labels migration"

This reverts commit 0162d8105f6eb3d40ae63098069b496a27f5cc59.

Reason for revert: <INSERT REASONING HERE>

Original change's description:
> autotest: add two new tables and one new column for afe_labels migration
>
> Add two new tables: afe_static_labels and afe_static_hosts_labels. Add
> one new column, replaced, to afe_labels, which means the value of this
> label is replaced by the corresponding one in afe_static_labels. This CL
> is one step to migrate the local afe static labels data to inventory
> service.
>
> BUG=chromium:791808
> TEST=run the following command on a test ganeti server:
> $AUTOTEST_DIR/database/migrate.py sync --debug
> The new changes are in the database.
>
> Change-Id: Ie82d764926d2372924588a000b3b5715f872bccf
> Reviewed-on: https://chromium-review.googlesource.com/807528
> Reviewed-by: Dan Shi <dshi@google.com>
> Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
> Tested-by: Shuqian Zhao <shuqianz@chromium.org>

Bug: chromium:791808
Change-Id: I4533f2ac08dd5f201de03590d82136e685a31abc
Reviewed-on: https://chromium-review.googlesource.com/814617
Tested-by: Shuqian Zhao <shuqianz@chromium.org>
Reviewed-by: Shuqian Zhao <shuqianz@chromium.org>
igrations/123_add_static_labels_tables.py
bff32b3302fdf5e296ecda38d96d7d8783440de7 07-Dec-2017 Paul Hobbs <phobbs@google.com> Revert "[autotest] Set read isolation level"

This reverts commit 29d75f0c0d704a9d579f795670f01acb66939592.

Reason for revert: crbug.com/792985

Original change's description:
> [autotest] Set read isolation level
>
> MariaDB/MySQL has the default isolation level of REPEATABLE-READ
>
> That means that if Django runs a SELECT statement, it starts a
> transaction and future SELECT statements will get stale data until the
> transaction is closed (e.g., you make a DML).
>
> See also https://stackoverflow.com/questions/3346124/
>
> BUG=chromium:787233
> TEST=None
>
> Change-Id: I34ee30d2c61c9c79b8103cff76b3f9a2e04dcf2d
> Reviewed-on: https://chromium-review.googlesource.com/783556
> Commit-Ready: ChromeOS CL Exonerator Bot <chromiumos-cl-exonerator@appspot.gserviceaccount.com>
> Tested-by: Allen Li <ayatane@chromium.org>
> Reviewed-by: Allen Li <ayatane@chromium.org>

Bug: chromium:787233
Change-Id: I63c070641119902f6273ad0c2bf8ac4e9ebce678
Reviewed-on: https://chromium-review.googlesource.com/814914
Reviewed-by: Paul Hobbs <phobbs@google.com>
Tested-by: Paul Hobbs <phobbs@google.com>
atabase_settings_helper.py
etup_test_environment.py
91f58e141f3773acf16ae4af3173366b299f76bb 29-Nov-2017 Xixuan Wu <xixuan@chromium.org> autotest: remove backup role for servers.

Also previously unittest is not running. This CL fixes it.

BUG=chromium:789784
TEST=Ran unittest.

Change-Id: Ic380c2832e53f782d3bc888f7d503e41349868ff
Reviewed-on: https://chromium-review.googlesource.com/797219
Commit-Ready: ChromeOS CL Exonerator Bot <chromiumos-cl-exonerator@appspot.gserviceaccount.com>
Tested-by: Xixuan Wu <xixuan@chromium.org>
Reviewed-by: Aviv Keshet <akeshet@chromium.org>
erver/models.py
d29baf9818784ac5f7fa69c2087f4f7ba9fcd27a 05-Dec-2017 Shuqian Zhao <shuqianz@chromium.org> autotest: add replaced_by_static_label column to afe_labels

Add one new column, replaced, to afe_labels, which means the value of this
label is replaced by the corresponding one in afe_static_labels. This CL
is one step to migrate the local afe static labels data to inventory
service.

BUG=chromium:791808
TEST=None

Change-Id: I809a262667dfe97595cc575f373610c86d4df571
Reviewed-on: https://chromium-review.googlesource.com/809327
Tested-by: Shuqian Zhao <shuqianz@chromium.org>
Reviewed-by: Dan Shi <dshi@google.com>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
igrations/124_add_replaced_column_to_afe_labels.py
0162d8105f6eb3d40ae63098069b496a27f5cc59 05-Dec-2017 Shuqian Zhao <shuqianz@chromium.org> autotest: add two new tables and one new column for afe_labels migration

Add two new tables: afe_static_labels and afe_static_hosts_labels. Add
one new column, replaced, to afe_labels, which means the value of this
label is replaced by the corresponding one in afe_static_labels. This CL
is one step to migrate the local afe static labels data to inventory
service.

BUG=chromium:791808
TEST=run the following command on a test ganeti server:
$AUTOTEST_DIR/database/migrate.py sync --debug
The new changes are in the database.

Change-Id: Ie82d764926d2372924588a000b3b5715f872bccf
Reviewed-on: https://chromium-review.googlesource.com/807528
Reviewed-by: Dan Shi <dshi@google.com>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
Tested-by: Shuqian Zhao <shuqianz@chromium.org>
igrations/123_add_static_labels_tables.py
04afc8f3b08c565b8988383c02689d547e688823 28-Nov-2017 Allen Li <ayatane@chromium.org> [autotest] Deprecate num parameter

This isnt used; see the bug.

BUG=chromium:782048
TEST=None
CQ-DEPEND=CL:791394

Change-Id: I0557d71362445691fc5ac02840fdaa75215a0c9e
Reviewed-on: https://chromium-review.googlesource.com/791999
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Allen Li <ayatane@chromium.org>
fe/rpc_interface.py
fe/rpc_interface_unittest.py
29d75f0c0d704a9d579f795670f01acb66939592 22-Nov-2017 Allen Li <ayatane@chromium.org> [autotest] Set read isolation level

MariaDB/MySQL has the default isolation level of REPEATABLE-READ

That means that if Django runs a SELECT statement, it starts a
transaction and future SELECT statements will get stale data until the
transaction is closed (e.g., you make a DML).

See also https://stackoverflow.com/questions/3346124/

BUG=chromium:787233
TEST=None

Change-Id: I34ee30d2c61c9c79b8103cff76b3f9a2e04dcf2d
Reviewed-on: https://chromium-review.googlesource.com/783556
Commit-Ready: ChromeOS CL Exonerator Bot <chromiumos-cl-exonerator@appspot.gserviceaccount.com>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Allen Li <ayatane@chromium.org>
atabase_settings_helper.py
etup_test_environment.py
035b47cc28d0e79654bb7a838ffae0d4156ac9c2 21-Nov-2017 Allen Li <ayatane@chromium.org> [autotest] Remove is_available() on wrong Host class

BUG=chromium:787233
TEST=None

Change-Id: I96f20b3224bafa2bc7b7a696de9faf1ef8544af6
Reviewed-on: https://chromium-review.googlesource.com/780879
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Allen Li <ayatane@chromium.org>
fe/models.py
fef16aed95e2984945e42240cb8de45aadabb421 21-Nov-2017 Allen Li <ayatane@chromium.org> [autotest] Add new fields to JobHandoff model

Migration has to run before schema change.

Fix the existing job field also. The database side is fine, but
Django needs to be informed of the constraints and primary key.

BUG=chromium:748234
TEST=Start monitor_db and it doesnt crash immediately
TEST=Start job_aborter and it doesnt crash immediately

Change-Id: I5243fa0abdf91e700c0439c713c5adbb9feef0f7
Reviewed-on: https://chromium-review.googlesource.com/780767
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Dan Shi <dshi@google.com>
fe/models.py
220e2bad928ed3f633fc9c18a26f998ec143f4b7 21-Nov-2017 Allen Li <ayatane@chromium.org> [autotest] Add new columns to JobHandoff

Migration has to run before schema change.

Make existing rows be completed. Nothing is using JobHandoff except
for testing, but old handoffs should not trigger job_aborter.

BUG=chromium:748234
TEST=/usr/local/autotest/database/migrate.py sync

Change-Id: If95c7bc84130b2c6cb141df5d1424490caa5caf1
Reviewed-on: https://chromium-review.googlesource.com/780766
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
Reviewed-by: Dan Shi <dshi@google.com>
igrations/122_add_job_handoff_completed_columns.py
cd36ae0a377f228c69f58d7c5c854ee1a7abf94b 10-Nov-2017 Xixuan Wu <xixuan@chromium.org> autotest: Enable passing in parameters from create job page.

BUG=chromium:782501
TEST=Ran client job & suite job on local autotest. Verify that fast is
injected into control file.
Ran unittest.

Change-Id: I061694392fb3b789a5135ad2af241f2b0886123a
Reviewed-on: https://chromium-review.googlesource.com/764431
Commit-Ready: Xixuan Wu <xixuan@chromium.org>
Tested-by: Xixuan Wu <xixuan@chromium.org>
Reviewed-by: Dan Shi <dshi@google.com>
fe/rpc_interface.py
ea45e551aaa86cf172f2a5074e03679b12192ed9 10-Nov-2017 Keith Haddow <haddowk@chromium.org> [moblab] Remove obsolite test suites from the UI

CTS (M) and the precondition suite are no longer used, remove
from the UI

BUG=chromium:772192
TEST=trybot job

Change-Id: I8b13a4663baa2bae3bab797a593cbac7ad468b0a
Reviewed-on: https://chromium-review.googlesource.com/764467
Commit-Ready: Keith Haddow <haddowk@chromium.org>
Tested-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Michael Tang <ntang@chromium.org>
lient/src/autotest/moblab/SuiteRunnerView.java
de5ecce46055de0b70c4a2816bb31522bef15c24 07-Nov-2017 Allen Li <ayatane@chromium.org> [autotest] Detect number of DUTs in provision suite

This commit makes provision suite schedule tests for all duts for the
given board/pool, with a num_max as a hard fallback API.

BUG=chromium:672348
TEST=None

Change-Id: I4827d6dbfb2f340223b3669391928a98fcbd67e4
Reviewed-on: https://chromium-review.googlesource.com/756369
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Allen Li <ayatane@chromium.org>
fe/models.py
c0ea883be96eab2c068d9af3fffe539c0234a964 30-Oct-2017 Aviv Keshet <akeshet@chromium.org> autotest: add archive_url column to afe_stable_versions

BUG=chromium:780568
TEST=None

Change-Id: I5036d0bd044f6b0dd095720e838c07fd150d884d
Reviewed-on: https://chromium-review.googlesource.com/749057
Commit-Ready: Aviv Keshet <akeshet@chromium.org>
Tested-by: Aviv Keshet <akeshet@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
igrations/121_update_afe_stable_versions_table_add_archive_url.py
74a56111af9eccd95ab8dafd1d114a6c04e247ec 03-Nov-2017 Paul Hobbs <phobbs@google.com> Revert "[autotest] Fix constraints in hqe_by_insert_time"

This reverts commit e1207af9ea77dfe241c14639a773449f6d07ecf5.

Reason for revert: Speculative fix for crbug.com/781302

Original change's description:
> [autotest] Fix constraints in hqe_by_insert_time
>
> get_host_queue_entries_by_insert_time was showing some strange behavior because
> it was making a wrong assumptions about the gap between insert-time and
> start-time being small.
>
> - Renamed to "get_host_queue_entries_by_start_time"
> - Changed the API to be based on started_on constraints.
> - Changed the function to infer only correct insert-time constraints. This
> means we cannot assume any lower bound on the insert-time, because the gap
> between insert and start times may be arbitrarily large.
> - Added more unit test coverage.
>
> BUG=chromium:748209
> TEST=unit tests pass.
>
> Change-Id: I60133e7a5f6431f35349c17f0644bc04e04cfcbd
> Reviewed-on: https://chromium-review.googlesource.com/685735
> Commit-Ready: Paul Hobbs <phobbs@google.com>
> Tested-by: Paul Hobbs <phobbs@google.com>
> Reviewed-by: Paul Hobbs <phobbs@google.com>

Bug: chromium:748209
Change-Id: I6a4e3c030527aafdd43f1218f514fbd3bbe6bbc6
Reviewed-on: https://chromium-review.googlesource.com/753998
Commit-Ready: Paul Hobbs <phobbs@google.com>
Tested-by: Paul Hobbs <phobbs@google.com>
Reviewed-by: Allen Li <ayatane@chromium.org>
fe/rpc_interface.py
fe/rpc_interface_unittest.py
33694eefcaef49ae451b1a695f77d70c22b8035e 30-Oct-2017 Aviv Keshet <akeshet@chromium.org> autotest: add extra_child_dependencies to create_suite_job argument

This threads a new argument from the create_suite_job rpc into the suite
control file, which will soon allow dynamic_suite to consume and use an
additional list of child job dependencies.

BUG=chromium:776892
TEST=On local afe:
- Verify run_suite without --model argument works.
- Verify run_suite with --model, with no model:XXX label DUTs hangs.
- Verify run_suite with --model, with model:XXX DUTs works.

Change-Id: I5116cc38834562087243b6e67023e130e060c49e
Reviewed-on: https://chromium-review.googlesource.com/744339
Commit-Ready: Prathmesh Prabhu <pprabhu@chromium.org>
Tested-by: Prathmesh Prabhu <pprabhu@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
fe/rpc_interface.py
e1207af9ea77dfe241c14639a773449f6d07ecf5 07-Sep-2017 Paul Hobbs <phobbs@google.com> [autotest] Fix constraints in hqe_by_insert_time

get_host_queue_entries_by_insert_time was showing some strange behavior because
it was making a wrong assumptions about the gap between insert-time and
start-time being small.

- Renamed to "get_host_queue_entries_by_start_time"
- Changed the API to be based on started_on constraints.
- Changed the function to infer only correct insert-time constraints. This
means we cannot assume any lower bound on the insert-time, because the gap
between insert and start times may be arbitrarily large.
- Added more unit test coverage.

BUG=chromium:748209
TEST=unit tests pass.

Change-Id: I60133e7a5f6431f35349c17f0644bc04e04cfcbd
Reviewed-on: https://chromium-review.googlesource.com/685735
Commit-Ready: Paul Hobbs <phobbs@google.com>
Tested-by: Paul Hobbs <phobbs@google.com>
Reviewed-by: Paul Hobbs <phobbs@google.com>
fe/rpc_interface.py
fe/rpc_interface_unittest.py
c1f0a91c4a28ccd1664f52fc560a05020c213b7b 20-Oct-2017 Allen Li <ayatane@chromium.org> [autotest] Add table to track job handoff

BUG=chromium:748234
TEST=CQ
TEST=Run migration and run a dummy_Pass and monitor_db locally

Change-Id: I4cd67211efd09acb397453aa75a4ed5724e06611
Reviewed-on: https://chromium-review.googlesource.com/729780
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Dan Shi <dshi@google.com>
fe/models.py
12291941077c62e997512e0daea3d9aed6fd337d 18-Oct-2017 Allen Li <ayatane@chromium.org> [autotest] Add migration for table to track job handoff

BUG=chromium:748234
TEST=Run SQL command against local database

Change-Id: Ib69c78885952e83e353118595543463971d47035
Reviewed-on: https://chromium-review.googlesource.com/724679
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Allen Li <ayatane@chromium.org>
igrations/120_add_job_handoff_table.py
1e3b52e60ea3e764af2281e43b8ab7e2b567103a 22-Sep-2017 Sida Liu <sidal@chromium.org> [moblab] Add support for avl process bug id and part id.

1) Add optional bug id textbox and part id text box in "Run Suite" view
when hardware_memoryqual or hardware_storagequal suite is selected.

2) Support two optional arguments in run_suite RPC call originalted from
moblab.

3) Modified control.hardware_memoryqual to write bug_id and part_id to
keyval file.

TEST=unittest and manual test on moblab

BUG=None

Change-Id: I284252ed792ee12f4a503bb089488fb76fdff3bb
Reviewed-on: https://chromium-review.googlesource.com/679257
Commit-Ready: Sida Liu <sidal@chromium.org>
Tested-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Sida Liu <sidal@chromium.org>
Reviewed-by: Keith Haddow <haddowk@chromium.org>
fe/moblab_rpc_interface.py
lient/src/autotest/moblab/SuiteRunnerView.java
lient/src/autotest/moblab/rpc/MoblabRpcHelper.java
lient/src/autotest/public/moblabsetup.css
62c08670618cddce55b9f128d326251e065c7c01 19-Sep-2017 Keith Haddow <haddowk@chromium.org> [moblab] Add some more options that are needed for retry to work

Turns out you need to set the max_retries, and wait for results
flags for the retries to actually work.

BUG=chromium:766235,chromium:675041
TEST=local moblab runs

Change-Id: I780abc1e316f6b77bc67802c8d7d21f560503ef7
Reviewed-on: https://chromium-review.googlesource.com/671879
Commit-Ready: Keith Haddow <haddowk@chromium.org>
Tested-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Michael Tang <ntang@chromium.org>
fe/moblab_rpc_interface.py
9d1db417848092bdb5add994421ea43a15d38a14 18-Sep-2017 Keith Haddow <haddowk@chromium.org> [moblab] Turn on job retries when creating the suite.

Job retries defaults to false so the settings in the control files
were being ignored, switch on retries.

BUG=chromium:766235
TEST=local moblab tests

Change-Id: I81fccb328c7dcc68cdfc06326e9527044ade5ac2
Reviewed-on: https://chromium-review.googlesource.com/671629
Commit-Ready: Keith Haddow <haddowk@chromium.org>
Tested-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Michael Tang <ntang@chromium.org>
fe/moblab_rpc_interface.py
145a8f473c98d407fbce900f8bb4aa2fcc7fb90b 15-Sep-2017 Keith Haddow <haddowk@chromium.org> [moblab] Move the suite args in the moblab run_suite to new format.

As requested by ayatane to be compatible with other changes.

TEST=tested on local moblab
BUG=chromium:668711

Change-Id: I14e1a93bdb0a0a1c50f37097b0d49fa4548c7d18
Reviewed-on: https://chromium-review.googlesource.com/669745
Commit-Ready: Keith Haddow <haddowk@chromium.org>
Tested-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Keith Haddow <haddowk@chromium.org>
fe/moblab_rpc_interface.py
db8b8aeefe1d45ab8dcba81c0eb267ec873a0e63 15-Sep-2017 Allen Li <ayatane@chromium.org> [autotest] Handle CTS/GTS suite_args when empty

CTS/GTS depend on the suite_args string being actually empty, or they
will start filtering tests based on whether their names appear
anywhere in the string.

So '' vs '{}' is the different between running all tests or no tests.

New CTS/GTS tests are passed a list of test names explicitly, so wont
suffer from this problem.

BUG=chromium:760675
TEST=Run on moblab

Change-Id: I8f58de70b6510f6ea84f598001440a7432f9634d
Reviewed-on: https://chromium-review.googlesource.com/668711
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Keith Haddow <haddowk@chromium.org>
fe/rpc_interface.py
f2becd326c01c09e09699e6f7394c91018ba1935 11-Sep-2017 Prathmesh Prabhu <pprabhu@chromium.org> autotest: ui: Drop support for editing control file for job.

BUG=chromium:753607
TEST=local AFE.

Change-Id: I3d2e89710150bb8920a4e8c5587d69e64fa46494
Reviewed-on: https://chromium-review.googlesource.com/660687
Commit-Ready: Prathmesh Prabhu <pprabhu@chromium.org>
Tested-by: Prathmesh Prabhu <pprabhu@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
lient/src/autotest/afe/create/CreateJobViewPresenter.java
db83fcee3fcb045508c0d9b7bcf6c19fddcdeda9 08-Sep-2017 Prathmesh Prabhu <pprabhu@chromium.org> autotest: ui: Fix glitch in default endDate on job list page

The default end date was being set to today - 6 days.

BUG=None
TEST=Local AFE.

Change-Id: Ifeb9c33c5d2b1124b35662c5b69f00e9ab9066ca
Reviewed-on: https://chromium-review.googlesource.com/657891
Commit-Ready: Prathmesh Prabhu <pprabhu@chromium.org>
Tested-by: Prathmesh Prabhu <pprabhu@chromium.org>
Reviewed-by: Paul Hobbs <phobbs@google.com>
lient/src/autotest/common/table/DatetimeSegmentFilter.java
df0a3a3a2b3ce70cd9aea93fac3448efd176e646 30-Jun-2017 Rohit Makasana <rohitbm@chromium.org> Enabling provisioning for ARC TH CQ tests.

This CL:
- adds provisioning config for ARC TH.
- adds CrOS provisioning into cheets update test.
- adds a suite config for arc-presubmit.

BUG=b:26895109
TEST=Tested on Moblab by running a suite.

site_utils/run_suite.py
-b veyron_minnie
-i 'veyron_minnie-release/R61-9765.43.0'
--cheets_build 'git_nyc-mr1-arc-m61/cheets_arm-usr/4295536'
-p suites
-s dummy_server
--max_runtime_mins 20

Test scenarios:

--> No CrOS update, update cheets image
--> Update CrOS image, no cheets update
--> Update CrOS image, update cheets image
--> Update CrOS image, update cheets image (but CrOS image comes with the
test cheets image, so action
needed)
--> No CrOS update, no cheets update
--> Update a DUT with -cheetsth label to a normal cros-version: label
using non-cheets provisioning.
--> Update a DUT with a normal cros-version: label to -cheetsth using
--cheets_build provisioning.

Change-Id: I99a8586dc651fde08b8c9e1bef5d3459f8fa2d19
Reviewed-on: https://chromium-review.googlesource.com/557934
Commit-Ready: Rohit Makasana <rohitbm@chromium.org>
Tested-by: Rohit Makasana <rohitbm@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Reviewed-by: Dan Shi <dshi@google.com>
fe/rpc_interface.py
65c5f363d2d0fc4b42539ef5fce462fab17b25a7 03-Sep-2017 Dan Shi <dshi@google.com> [autotest] add index for tko_jobs table on queued_time and finished_time

BUG=None
TEST=database/migrate.py sync

Change-Id: I2f009c8a4ddf23fce2f9a05a4c706df65fc0e4a6
Reviewed-on: https://chromium-review.googlesource.com/648637
Commit-Ready: Dan Shi <dshi@google.com>
Tested-by: Dan Shi <dshi@google.com>
Reviewed-by: Paul Hobbs <phobbs@google.com>
igrations/119_add_tko_jobs_indices.py
3a9c64664f6e30e40d1807f169cc8e0e584bd9f7 01-Sep-2017 Paul Hobbs <phobbs@google.com> migrations: Add HQE indices

Add various indices for optimizing slow queries that scan the HQE table.

BUG=chromium:748099
TEST=Applied locally

Change-Id: If869d2e8727f7beac4af1c554cbf528f815e88d8
Reviewed-on: https://chromium-review.googlesource.com/648193
Reviewed-by: Dan Shi <dshi@google.com>
Tested-by: Paul Hobbs <phobbs@google.com>
Commit-Queue: Paul Hobbs <phobbs@google.com>
igrations/118_add_hqe_indices.py
c983d5a0c3719a0edba5665708d3acab1359d12a 21-Aug-2017 Xixuan Wu <xixuan@chromium.org> autotest: add a simple RPC interface for getting lab health indicators.

This CL is a first step to build a full RPC call that returns a full
list of lab health indicators so that callers can make reasonable
desicions based on lab's health.

BUG=chromium:757482
TEST=None

Change-Id: I76ff7b12331faa374350439a89d232ce63f06c39
Reviewed-on: https://chromium-review.googlesource.com/623917
Commit-Ready: Xixuan Wu <xixuan@chromium.org>
Tested-by: Xixuan Wu <xixuan@chromium.org>
Reviewed-by: Xixuan Wu <xixuan@chromium.org>
fe/rpc_interface.py
a58c18bf8bd47c8ce525d3186d1f2f8bf6d59c89 27-Aug-2017 Paul Hobbs <phobbs@google.com> [autotest] ui: Add date constraint on page load

Add a 2-week constraint on the date range as soon
as the page loads, instead of only after the first
change to the endDate.

Factor out the value change handler code which
updates the start date constraint upon changing
the end date.

BUG=chromium:753394
TEST=tested manually on chromeos-autotest.hot

Change-Id: Id164aa623afa59560114b9a97365d4ff4a83aeb1
Reviewed-on: https://chromium-review.googlesource.com/636515
Reviewed-by: Paul Hobbs <phobbs@google.com>
Commit-Queue: Paul Hobbs <phobbs@google.com>
Tested-by: Paul Hobbs <phobbs@google.com>
lient/src/autotest/common/table/DatetimeSegmentFilter.java
3c0d6cba982b138e228c8f912028cbbc2db7b20e 22-Aug-2017 Allen Li <ayatane@chromium.org> [autotest] Clean up new suite_args API

This is a scrub of the repo, making sure the API is correct at all
interfaces and backward compatibility is maintained at the control
file interface.

In detail, the changes are:

AFE create_suite_job RPC made less defensive. An invalid argument
will cause the RPC call to error instead of replacing suite_args with
an empty dict, which probably would not have produced the desired
behavior anyway, so erroring out is better.

Inject suite_args into the control file. This allows new Autotest to
run old tests.

Make the relevant new tests support being run by old Autotest.

The referenced bug contains a diagram listing all of the interface
points between the new and old API and the compatibility between them.

BUG=chromium:758427
TEST=site_utils/test_that.py -b reks -i reks-release/R60-9592.71.0 -w localhost :lab: dummy_Pass
TEST=site_utils/run_suite.py -b reks -i reks-release/R60-9592.71.0 -p suites -s dummy

Change-Id: I352491b4ad72818ce288144a0380e441a3f65b40
Reviewed-on: https://chromium-review.googlesource.com/631236
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Allen Li <ayatane@chromium.org>
fe/rpc_interface.py
c4474c25e029e6359fe8b91953b1517e16fceba4 22-Aug-2017 Keith Haddow <haddowk@chromium.org> [moblab] Add new suite to support the USB camera qualification.

Add a new suite to the UI, to allow third party labs to easily
run the camera qualification tests.

BUG=chromium:757863
TEST=Manual test on moblab.

Change-Id: If7606fb6543dac2042b3d89a25d3687ff7b52820
Reviewed-on: https://chromium-review.googlesource.com/626939
Commit-Ready: Keith Haddow <haddowk@chromium.org>
Tested-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Michael Tang <ntang@chromium.org>
lient/src/autotest/moblab/SuiteRunnerView.java
4442d6fb2c7fa950a49aad4c38402003dc4eb484 17-Aug-2017 Paul Hobbs <phobbs@google.com> [autotest] Ui: Add 2 week date restriction.

Add a 2 week date restriction to the datetime-local input box for JobListView and HostDetailView.

WIP; Needs manual testing still.

BUG=chromium:753394
TEST=None

Change-Id: If4197c795687b0886cf167443c5a4dd35c8a7461
Reviewed-on: https://chromium-review.googlesource.com/617928
Commit-Ready: Paul Hobbs <phobbs@google.com>
Tested-by: Paul Hobbs <phobbs@google.com>
Reviewed-by: Paul Hobbs <phobbs@google.com>
lient/src/autotest/common/table/DatetimeSegmentFilter.java
lient/src/autotest/common/ui/DateTimeBox.java
309825bdeca9a9aadf71b5a596dbbba37982f5af 17-Aug-2017 Prathmesh Prabhu <pprabhu@chromium.org> autotest: Move 500.html to the templates subdirectory.

So that it can actually be served.

BUG=chromium:756263
TEST=hot-fixed cautotest.

Change-Id: I4cbf1519bbb41cb279de7d884daffc4dd1a6efb5
Reviewed-on: https://chromium-review.googlesource.com/617987
Commit-Ready: Prathmesh Prabhu <pprabhu@chromium.org>
Tested-by: Prathmesh Prabhu <pprabhu@chromium.org>
Reviewed-by: Ningning Xia <nxia@chromium.org>
fe/500.html
fe/templates/500.html
f28a816878ea69fc2f7fb84d51b319eff4f218a2 20-May-2017 Paul Hobbs <phobbs@google.com> [autotest] Add RPC method to request arguments for RPCs.

This is purely for informative purposes.

BUG=chromium:754522
TEST=None

Change-Id: I427f7d318432482d2899ac0a19869631f2db4a7d
Reviewed-on: https://chromium-review.googlesource.com/612599
Commit-Ready: Paul Hobbs <phobbs@google.com>
Tested-by: Paul Hobbs <phobbs@google.com>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
fe/json_rpc/proxy.py
e21ed7eb31a8983d7e03a3bd6f96aa86cfd5e33e 11-Jul-2017 Allen Li <ayatane@chromium.org> [autotest] Factor out suite_args evaluation (reland)

The intent is to:

1. Standardize how suite_args is handled (currently it is a special
snowflake only used by control.suite). This will be used to pass
additional arguments to provision suite and potentially other
suites.
2. Pass suite_args as a dict instead of as a string to be evaluated.
This is how test_args is passed currently. This change needs to be
made also in run_suite.py, but compatibility code needs to be kept
during the transition.

I feel like channeling Richard and jumping off a bridge.

BUG=chromium:672348
BUG=chromium:749971
TEST=None

Change-Id: Ic67f21366cc0731e4fd73a7e29135be2bb454229
Reviewed-on: https://chromium-review.googlesource.com/591885
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
fe/rpc_interface.py
8f01aa669d04798a28f5bea1df700327bc7d3901 10-Aug-2017 Prathmesh Prabhu <pprabhu@chromium.org> [autotest] Update shard state before master for modify_host

This is a testing-the-waters CL for the following idea: Any time there
is an rpc_fanout from the master AFE to the shards, the shard state must
be complete before the master state is modified.
This way, in case of a partial update, master state is left untouched.
Any corrective action assumes that the master state is correct in case
of master-shard desync, so that shard's partial updates will be
discarded.

Currently, we update master state before the shard in a few places. If
an update fails midway, this causes us to assume that the update was
successful, but the shards lag behind.

BUG=chromium:732999
TEST=modify hosts on master-shard on the staging servers.

Change-Id: Ia8920550f75e10e51f1c86eef347346c88e08e5e
Reviewed-on: https://chromium-review.googlesource.com/608580
Commit-Ready: Prathmesh Prabhu <pprabhu@chromium.org>
Tested-by: Prathmesh Prabhu <pprabhu@chromium.org>
Reviewed-by: Richard Barnette <jrbarnette@google.com>
fe/rpc_interface.py
b940817305207a493402e72b05601aad0038e8ee 04-Aug-2017 Keith Haddow <haddowk@chromium.org> [moblab] Provide default for getting the test args from config.

If a user AU's and does not go through the config wizard again
the wifi test args are missing from the config.

Provide a default value in the get request.

BUG=chromium:752558
TEST=unit tests and locally on moblab.

Change-Id: If1904a289df8effd54b62158f7310a17c0a6fe09
Reviewed-on: https://chromium-review.googlesource.com/602677
Commit-Ready: Keith Haddow <haddowk@chromium.org>
Tested-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Michael Tang <ntang@chromium.org>
fe/moblab_rpc_interface.py
0e9214d43571df4dbf721c9db05413ca26f30fa2 04-Aug-2017 Allen Li <ayatane@chromium.org> [autotest] Remove rpc_hostname feature from shards

This was only added for puppylab.

BUG=None
TEST=None

Change-Id: I95b2e595b8ee341ccc7fc8923313683708ece7a9
Reviewed-on: https://chromium-review.googlesource.com/602455
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
fe/models.py
fe/rpc_interface.py
fe/rpc_utils.py
0248035d3d970e9d32334610aeba11f853390858 17-Jul-2017 Allen Li <ayatane@chromium.org> [autotest] Remove puppylab code

grep returns no results

BUG=None
TEST=Run local dummy test

Change-Id: I5fc78f750e0eaaedd5a06ddb1d730c587520b25f
Reviewed-on: https://chromium-review.googlesource.com/576875
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
fe/models.py
3dabea54fc7623877abcbfc7792e1256881d013b 26-Jul-2017 Prathmesh Prabhu <pprabhu@chromium.org> [autotest] pass-through all host attribute updates to master

This brings attribute update behaviour on par with label updates. If a
set_host_attribute RPC is made to the shard directly, it passes it to
the master, which updates both its own DB and fans out the RPC back to
the relevant shards.

BUG=chromium:749200
TEST=(sanity) Test run on local AFE; run repair on a test_push shard.

Change-Id: I745e2b29e075d4077957169ffa2412c4f4916ea9
Reviewed-on: https://chromium-review.googlesource.com/587417
Commit-Ready: Prathmesh Prabhu <pprabhu@chromium.org>
Tested-by: Prathmesh Prabhu <pprabhu@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
fe/rpc_interface.py
0b963f997a900c662e5d3f95e7eb69bcc7986419 26-Jul-2017 Prathmesh Prabhu <pprabhu@chromium.org> [autotest] Add internal RPC for setting host attribute

This is preparation for forwarding all host attribute updates through
the master.

BUG=chromium:749200
TEST=local AFE sanity. New RPC is unused at this point.

Change-Id: I68a1e6138a5fb68238e3d42ee136a3847fee44ac
Reviewed-on: https://chromium-review.googlesource.com/587416
Commit-Ready: Prathmesh Prabhu <pprabhu@chromium.org>
Tested-by: Prathmesh Prabhu <pprabhu@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
Reviewed-by: Paul Hobbs <phobbs@google.com>
Reviewed-by: Richard Barnette <jrbarnette@google.com>
fe/rpc_interface.py
94dd4622cdad3a271375d0a0d11d9ce162b44bb9 29-Jul-2017 Keith Haddow <haddowk@chromium.org> [moblab] Fix issue where wifi password could not be reset to None

If you set the password, then went back and tried to remove the
password it was never updated.

BUG=chromium:748702
TEST=local moblab testing

Change-Id: Id33e8c495ccff9395cf5037cf76246834a53ed20
Reviewed-on: https://chromium-review.googlesource.com/592850
Commit-Ready: Keith Haddow <haddowk@chromium.org>
Tested-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Michael Tang <ntang@chromium.org>
lient/src/autotest/moblab/rpc/WifiInfo.java
9a98eaa02bed4e26d760b83870a363c339387065 31-Jul-2017 Keith Haddow <haddowk@chromium.org> [moblab] Increase the limit of builds provided for the UI

Feedback from users say they can not always see the builds they
require.

BUG=chromium:750719
TEST=unittest, moblab local testing

Change-Id: I3b66b2fce038b7ffe0db7ee730dc65601a421f77
Reviewed-on: https://chromium-review.googlesource.com/594349
Commit-Ready: Keith Haddow <haddowk@chromium.org>
Tested-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Michael Tang <ntang@chromium.org>
fe/moblab_rpc_interface.py
4544e330b7667ed71ed5c92d90831401b044f5a5 29-Jul-2017 Keith Haddow <haddowk@chromium.org> [moblab] Fix issue with pool checkbox not being enabled.

BUG=chromium:732505
TEST=local moblab test

Change-Id: I20cda7b73eca81a9d804122d1aa931f8cd3766ca
Reviewed-on: https://chromium-review.googlesource.com/592083
Commit-Ready: Keith Haddow <haddowk@chromium.org>
Tested-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Michael Tang <ntang@chromium.org>
lient/src/autotest/moblab/DutManagementView.java
5069755ba0d0b8ff58cbd80531fde69daf45d2fa 28-Jul-2017 Keith Haddow <haddowk@chromium.org> [moblab] Enable the precondition suite on the UI.

Add the precondition suite to the UI and also pass the wifi test
parameters along to all suite runs.

BUG=chromium:748702
TEST=local moblab testing

Change-Id: Id0cba8dbf5067fd5ef28d6057afa0e6663630524
Reviewed-on: https://chromium-review.googlesource.com/592072
Commit-Ready: Keith Haddow <haddowk@chromium.org>
Tested-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Michael Tang <ntang@chromium.org>
fe/moblab_rpc_interface.py
lient/src/autotest/moblab/SuiteRunnerView.java
lient/src/autotest/moblab/wizard/WifiCard.java
9bf04b6bd7dbd4c2a765e5bce3bac46bd0511564 28-Jul-2017 Aviv Keshet <akeshet@chromium.org> Revert "[autotest] Factor out suite_args evaluation."

This reverts commit 91f1ddd486eaf78fbfbd3469091f0023b98609f0.

BUG=chromium:749971, chromium:672348
TEST=None

Change-Id: I02bf08d33e09c9c27422aecefce404d7826728fa
Reviewed-on: https://chromium-review.googlesource.com/590935
Reviewed-by: Aviv Keshet <akeshet@chromium.org>
Tested-by: Aviv Keshet <akeshet@chromium.org>
fe/rpc_interface.py
a11346cafeb0cfa252afc3de096c9bb7ee192cd9 28-Jul-2017 Aviv Keshet <akeshet@chromium.org> Revert "[autotest] Add num control to provision suite"

This reverts commit a4656a265443f474efde7a31a02d302e8d956f91.

BUG=chromium:749971, chromium:672348
TEST=None

Change-Id: I409f8220a8923b9c5cc1127afcd0a2b46f0c61af
Reviewed-on: https://chromium-review.googlesource.com/590934
Reviewed-by: Aviv Keshet <akeshet@chromium.org>
Tested-by: Aviv Keshet <akeshet@chromium.org>
fe/rpc_interface.py
a4656a265443f474efde7a31a02d302e8d956f91 10-Jul-2017 Allen Li <ayatane@chromium.org> [autotest] Add num control to provision suite

BUG=chromium:672348
TEST=None

Change-Id: Id8e782cd3c087d79c819649ab7e5c6d63a67519e
Reviewed-on: https://chromium-review.googlesource.com/565333
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
fe/rpc_interface.py
91f1ddd486eaf78fbfbd3469091f0023b98609f0 11-Jul-2017 Allen Li <ayatane@chromium.org> [autotest] Factor out suite_args evaluation.

The intent is to:

1. Standardize how suite_args is handled (currently it is a special
snowflake only used by control.suite). This will be used to pass
additional arguments to provision suite and potentially other
suites.
2. Pass suite_args as a dict instead of as a string to be evaluated.
This is how test_args is passed currently. This change needs to be
made also in run_suite.py, but compatibility code needs to be kept
during the transition.

I feel like channeling Richard and jumping off a bridge.

BUG=chromium:672348
TEST=None

Change-Id: I85e0bea532d90733a3d391bf328c1d2e09bc79c7
Reviewed-on: https://chromium-review.googlesource.com/567328
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
fe/rpc_interface.py
bbc45d971e3442c58d230829fdb6734b4a619324 25-Jul-2017 Keith Haddow <haddowk@chromium.org> [moblab] Add UI option to specify AP name/password DUTs shouls use.

Certain CTS tests require the DUT to connect to a wifi AP.

During the setup of the moblab the user can specify the AP name
and password and that will be passed to all tests for it to
(optionally) connect to the wifi AP.

Example UI changes at http://100.96.59.141/moblab_setup

BUG=chromium:748702
TEST=local testing on moblab

Change-Id: I48d2b3e6790995e54adaad98ab6c3b1f64c06afd
Reviewed-on: https://chromium-review.googlesource.com/584861
Commit-Ready: Keith Haddow <haddowk@chromium.org>
Tested-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Michael Tang <ntang@chromium.org>
fe/moblab_rpc_interface.py
lient/src/autotest/moblab/ConfigWizardView.java
lient/src/autotest/moblab/rpc/MoblabRpcCallbacks.java
lient/src/autotest/moblab/rpc/MoblabRpcHelper.java
lient/src/autotest/moblab/rpc/WifiInfo.java
lient/src/autotest/moblab/wizard/WifiCard.java
49f089396b0ffbe152fb734cffb84f9de038c426 06-Jul-2017 Keith Haddow <haddowk@chromium.org> [moblab] Add UI functionality to Set/Update host attributes.

Ui demo at moblab 100.96.59.141

BUG=chromium:732505
TEST=unit test and local moblab testing.

Change-Id: Icdc3e532b50fbf9b1a2440f423a00bd74a1ac232
Reviewed-on: https://chromium-review.googlesource.com/583711
Commit-Ready: Keith Haddow <haddowk@chromium.org>
Tested-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Michael Tang <ntang@chromium.org>
fe/moblab_rpc_interface.py
lient/src/autotest/moblab/DutManagementView.java
lient/src/autotest/moblab/rpc/MoblabRpcHelper.java
2abe495c0b9c59a4044688e461a0eeecb7c23a7a 24-Jul-2017 Aviv Keshet <akeshet@chromium.org> autotest: drop unused label_lockout_table

BUG=chromium:717811
TEST=None

Change-Id: I1d4848ca87e1eb37fc1df4c3ce55808631b1bc62
Reviewed-on: https://chromium-review.googlesource.com/583513
Commit-Ready: Aviv Keshet <akeshet@chromium.org>
Tested-by: Aviv Keshet <akeshet@chromium.org>
Reviewed-by: Dan Shi <dshi@google.com>
igrations/117_drop_label_lockout_table.py
28a8cd6f9a6b68c186b45017106a1921e7e911aa 15-Jul-2017 Justin TerAvest <teravest@chromium.org> Remove make_superuser.py

I couldn't find any references to this file in the Chrome OS repo and it
hasn't been touched since 2009.

Change-Id: I27867d91ea249be09ef22832e053359cf89917af
Reviewed-on: https://chromium-review.googlesource.com/572795
Commit-Ready: Justin TerAvest <teravest@chromium.org>
Tested-by: Justin TerAvest <teravest@chromium.org>
Reviewed-by: Dan Erat <derat@chromium.org>
ake_superuser.py
1698430f931102957e318dde7af13d37e789c59c 15-Jul-2017 Justin TerAvest <teravest@chromium.org> Remove old afe frontend_unittest code.

There's no frontend.afe file to be found to kick off testing, so this
seems useless.

Change-Id: I60b5c9af3a4b0783159d5b10cdd1d1770a732f79
Reviewed-on: https://chromium-review.googlesource.com/572794
Commit-Ready: Justin TerAvest <teravest@chromium.org>
Tested-by: Justin TerAvest <teravest@chromium.org>
Reviewed-by: Dan Erat <derat@chromium.org>
fe/test.py
rontend_unittest.py
4d84cac78c05cd1130e26b5f60b99f20ad98756d 19-Jul-2017 Aviv Keshet <akeshet@chromium.org> autotest: delete host_history.py, all callers (depends on autotest_es)

Note: this also deprecates a afe rpc named "get_host_history" which
depends indirectly on autotest_es. This is not to be confused with
lib/status_history:get_host_history which is a separate afe+tko-backed
host history fetcher and is still in use.

BUG=chromium:738508
TEST=None

Change-Id: I6c70188ac1ee6c0c8c097c3f58665e4afc4613bf
Reviewed-on: https://chromium-review.googlesource.com/575575
Commit-Ready: Aviv Keshet <akeshet@chromium.org>
Tested-by: Aviv Keshet <akeshet@chromium.org>
Reviewed-by: Dan Shi <dshi@google.com>
fe/rpc_interface.py
b5843689572959e3cc96955028211e09a276ccd5 18-Jul-2017 Aviv Keshet <akeshet@chromium.org> autotest: remove frontend/health

None of this code has been touched in 4 years, it looks dead, and it
depends on code that is being deleted.

BUG=chromium:745894
TEST=None

Change-Id: I0cd065bd242d014b283d850b4295ff57f0f5ed3c
Reviewed-on: https://chromium-review.googlesource.com/576377
Commit-Ready: Aviv Keshet <akeshet@chromium.org>
Tested-by: Aviv Keshet <akeshet@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
ealth/__init__.py
ealth/check_test_health.py
ealth/check_test_health_unittest.py
ealth/common.py
ealth/complete_failures.py
ealth/complete_failures_functional_test.py
ealth/complete_failures_unittest.py
ealth/manual_check_passing_experimental.py
ealth/passing_experimental.py
ealth/passing_experimental_functional_test.py
ealth/passing_experimental_unittest.py
ealth/utils.py
ealth/utils_unittest.py
271fe6928d374131d0481d5527e801656d153f2b 18-Jul-2017 Aviv Keshet <akeshet@chromium.org> autotest: delete autotest_es from frontend/afe/models

BUG=chromium:738508
TEST=None

Change-Id: I86ee60d6bf35224ddc6ce06aab953b31198907ce
Reviewed-on: https://chromium-review.googlesource.com/576550
Commit-Ready: Aviv Keshet <akeshet@chromium.org>
Tested-by: Aviv Keshet <akeshet@chromium.org>
Reviewed-by: Dan Shi <dshi@google.com>
fe/models.py
ba8adaf2506d4bf2b9b09bf54f2490ced31dd16c 05-Jul-2017 Prathmesh Prabhu <pprabhu@chromium.org> [autotest] Drop remaining references to autotest_stats

We are neither looking at the graphite dashboard anymore, not
maintaining it really. OTOH, the autotest_stats module costs ~.25
seconds on module load time, hitting us in places where it matters.

So, drop all uses, also removing the false sense of security.

BUG=chromium:739466
TEST=unittests.

Change-Id: I0e4118653d929a5d55f4ff62b96e9c66e855efd7
Reviewed-on: https://chromium-review.googlesource.com/559832
Commit-Ready: Prathmesh Prabhu <pprabhu@chromium.org>
Tested-by: Prathmesh Prabhu <pprabhu@chromium.org>
Reviewed-by: Aviv Keshet <akeshet@chromium.org>
fe/json_rpc/serviceHandler.py
fe/model_logic.py
fe/rpc_interface.py
fe/rpc_utils.py
ae36b767aadf5aa7656aeb4f955da67564b920f9 01-Jul-2017 Paul Hobbs <phobbs@google.com> ui: Sort by created_on when filtering created_on

The JobStartingDatetimeFilter puts a constraint on created_on (even if not
specificed, because if blank, it still uses a setStartTimeToPlaceHolderValue()
regardless). When the query has uses ORDER BY id, the InnoDB query optimizer
uses the primary key index depsite having a constraint on created_on which may
cause it to scan through an unbounded number of rows before actually finding
rows that satisfy the created_on constraint. By switching to ORDER BY
created_on, the query optimizer switches to using the created_on index, as it
should.

BUG=chromium:685424
TEST=None

Change-Id: Id38e054170264d8697cecc5d8607c8f5ee61f761
Reviewed-on: https://chromium-review.googlesource.com/558669
Commit-Ready: Paul Hobbs <phobbs@google.com>
Tested-by: Paul Hobbs <phobbs@google.com>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
lient/src/autotest/afe/JobTable.java
2ad43677bc8865a608375a653c4b32d805d2dbd9 28-Jun-2017 Keith Haddow <haddowk@chromium.org> [autotest] Remove the admin links from the UI

Admin pages have been removed from the code for a few weeks,
to avoid user confusion we should not have a link that goes to an
error page.

BUG=chromium:723867
TEST=Manual

Change-Id: I3d1715c9005caebc0181e2d19eba9f5952162ae8
Reviewed-on: https://chromium-review.googlesource.com/552748
Commit-Ready: Keith Haddow <haddowk@chromium.org>
Tested-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Michael Tang <ntang@chromium.org>
lient/src/autotest/public/AfeClient.html
lient/src/autotest/public/TkoClient.html
5e7d4a7f4e222b31fbc10cdb50a877c7ea6a98b5 15-Jun-2017 Chris Ching <chingcodes@chromium.org> rpc_interface: add get_db_test rpc

get_db_test does a quick db connection, and returns ether 'Success' or
'Failure'

BUG=chromium:715386
TEST=./rpc_interface_unittest.py

Change-Id: I9e2cd553369ff39d26a21b640d655584878db648
Reviewed-on: https://chromium-review.googlesource.com/537213
Commit-Ready: Chris Ching <chingcodes@chromium.org>
Tested-by: Chris Ching <chingcodes@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
fe/rpc_interface.py
fe/rpc_interface_unittest.py
69de2fa6a83330b47b681615214af6f690ca2a07 07-Jun-2017 Paul Hobbs <phobbs@google.com> [autotest] Fix HQE index table model

Its insert_time field should be a DateTimeField, not a DateField.
Also, it previously had the wrong table name in its metadata.

BUG=chromium:704997
TEST=tests pass, tested on chromeos-autotest.hot

Change-Id: I3124acba3a44e0efe8207ed107a935e47e194acf
Reviewed-on: https://chromium-review.googlesource.com/527610
Commit-Ready: Paul Hobbs <phobbs@google.com>
Tested-by: Paul Hobbs <phobbs@google.com>
Reviewed-by: Dan Shi <dshi@google.com>
fe/models.py
c5ec0606943d3aa1232adf5e75eaebbdd2f48952 02-Jun-2017 Keith Haddow <haddowk@chromium.org> [moblab] Only download gsutil when needed.

Originally I got the gsutil cmd every time the module loaded, this
turns out to be a bad thing since it is possible to be doing an
expensive operation early on in the boot up process.

TEST=local moblab, unittests
BUG=chromium:723870

Change-Id: Ia7460051dde8baa21f1e8a4b22b5e5ae040efd05
Reviewed-on: https://chromium-review.googlesource.com/521739
Commit-Ready: Keith Haddow <haddowk@chromium.org>
Tested-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Jinsong Mu <jinsong@google.com>
Reviewed-by: Keith Haddow <haddowk@chromium.org>
fe/moblab_rpc_interface.py
fe/moblab_rpc_interface_unittest.py
c4273f917c12e19751dc0846d25f96f4a54c7b49 02-Jun-2017 Keith Haddow <haddowk@chromium.org> [moblab] Fix bug in moblab run suite RPC.

If the RPC is called with no suite_args the code path is incorrect
and results in no tests running / error message being recieved.

BUG=chromium:728846
TEST=Unit test passes, local testing on moblab.

Change-Id: Ibc83a38c0cb232ea15c126fdafc4e2fb3d2ffb0c
Reviewed-on: https://chromium-review.googlesource.com/521754
Commit-Ready: Keith Haddow <haddowk@chromium.org>
Tested-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Jinsong Mu <jinsong@google.com>
Reviewed-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Michael Tang <ntang@chromium.org>
Reviewed-by: Dan Shi <dshi@google.com>
fe/moblab_rpc_interface.py
9857f8754fddad14c578fc8f5615ff233c8612cc 24-May-2017 Keith Haddow <haddowk@chromium.org> [moblab] Add new featutre to run suite to limit the tests run.

When partners re-run a CTS suite they often just want to run a
subset of the jobs. The UI now lets you specify a list of test
names. The suite will only run the tests that are in the suite
and in the specified list.

Demo on http://100.107.3.16/moblab_setup

TEST=local moblab
BUG=chromium:725601

Change-Id: I7708d2f4765370bff130e224dd0d33854202f6fd
Reviewed-on: https://chromium-review.googlesource.com/514247
Commit-Ready: Keith Haddow <haddowk@chromium.org>
Tested-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Jinsong Mu <jinsong@google.com>
Reviewed-by: Keith Haddow <haddowk@chromium.org>
fe/moblab_rpc_interface.py
lient/src/autotest/moblab/SuiteRunnerView.java
lient/src/autotest/moblab/rpc/MoblabRpcHelper.java
lient/src/autotest/public/moblabsetup.css
3a3045652c43b07e48b226dd5d785117160ff1ad 26-May-2017 Prathmesh Prabhu <pprabhu@chromium.org> [autoest] Add a server role 'sentinel'.

This role represents the sentinel server where a few administrative
daemons run.

BUG=chromium:726796
TEST=atest server modify SERVER_HOSTNAME -r sentinel

Change-Id: I5c6ea6851196ec64b073ed5798f2c015e9cb48b7
Reviewed-on: https://chromium-review.googlesource.com/517269
Commit-Ready: Prathmesh Prabhu <pprabhu@chromium.org>
Tested-by: Prathmesh Prabhu <pprabhu@chromium.org>
Reviewed-by: Aviv Keshet <akeshet@chromium.org>
erver/models.py
6a156ff6e5abf85434454bc852b7b739479c82ba 27-May-2017 Aviv Keshet <akeshet@chromium.org> autotest: add label lockout table

BUG=chromium:717811
TEST=None

Change-Id: I2ad7c415228daa854bd04467be097efb06861fd0
Reviewed-on: https://chromium-review.googlesource.com/517392
Commit-Ready: Aviv Keshet <akeshet@chromium.org>
Tested-by: Aviv Keshet <akeshet@chromium.org>
Reviewed-by: Dan Shi <dshi@google.com>
igrations/116_add_label_lockout_table.py
341165b4e1110da9151d7e74ca419abc380403bd 26-May-2017 Paul Hobbs <phobbs@google.com> [autotest] Add rpc "host_queue_entries_by_insert_time"

This new RPC is optimized to use the new index table for hqe insert times. It
puts constraints on the id based on the start_time and end_time arguments, which
will reduce DB load by preventing these queries from scanning the entire table.

BUG=chromium:704997
TEST=The new test passes.

Change-Id: Idc9d3554830a04f3bf55af670082a280fd5ee926
Reviewed-on: https://chromium-review.googlesource.com/516605
Commit-Ready: Paul Hobbs <phobbs@google.com>
Tested-by: Paul Hobbs <phobbs@google.com>
Reviewed-by: Paul Hobbs <phobbs@google.com>
fe/rpc_interface.py
fe/rpc_interface_unittest.py
f31d36c9d6773dfa468838b0af0e0581acf08731 06-Apr-2017 Prathmesh Prabhu <pprabhu@chromium.org> [autotest] Convert more DB queries to be case insensitive

See https://chromium-review.googlesource.com/c/386306/ for an example of
where changing our django queries to be case insensitive brings down the
query execution time a lot.

This CL changes some more of these queries to be faster.

BUG=chromium:708679
TEST=Locally test that each of the DB query goes from being a BINARY
match to a text match

Change-Id: I27bffb7ab2206725eed08db74d3f0d99a8b374fe
Reviewed-on: https://chromium-review.googlesource.com/470286
Commit-Ready: Prathmesh Prabhu <pprabhu@chromium.org>
Tested-by: Prathmesh Prabhu <pprabhu@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
fe/rpc_interface.py
4621bf1162cc8830c9e63621c0bef264bba28095 12-May-2017 Paul Hobbs <phobbs@google.com> [autotest] Add a model for the hqe index table.

Added a model to afe/models.py for the new table
host_queue_entry_start_time_index.

BUG=chromium:704997
TEST=None

Change-Id: Ie3f320d27e48dd0612eeff35b6dc79828010c95f
Reviewed-on: https://chromium-review.googlesource.com/508471
Commit-Ready: Paul Hobbs <phobbs@google.com>
Tested-by: Paul Hobbs <phobbs@google.com>
Reviewed-by: Aviv Keshet <akeshet@chromium.org>
fe/models.py
807a6b79632ecd25a0ffe0ad8402c01131cdfba5 12-May-2017 Paul Hobbs <phobbs@google.com> [autotest] Add migrations for hqe index table

BUG=chromium:704997
TEST=Applied patches locally, tested that event works.

Change-Id: I836c63db94956fa56beb79f0480cbfbc568258ce
Reviewed-on: https://chromium-review.googlesource.com/505175
Commit-Ready: Paul Hobbs <phobbs@google.com>
Tested-by: Paul Hobbs <phobbs@google.com>
Reviewed-by: Aviv Keshet <akeshet@chromium.org>
igrations/114_add_hqe_start_times.py
igrations/115_add_hqe_index_updating_event.py
015e223b2f231ca059dc567d530f199a9e2c1ccf 12-May-2017 Aviv Keshet <akeshet@chromium.org> autotest: delete a bunch of graph-related tko gwt code

BUG=None
TEST=./utils/compile_gwt_clients -a succeeds

Change-Id: I376620eb724d3e45922ced2da46efb31de7f8b44
Reviewed-on: https://chromium-review.googlesource.com/505249
Commit-Ready: Aviv Keshet <akeshet@chromium.org>
Tested-by: Aviv Keshet <akeshet@chromium.org>
Reviewed-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Reviewed-by: Dan Shi <dshi@google.com>
lient/src/autotest/public/EmbeddedTkoClientTest.html
lient/src/autotest/tko/DynamicGraphingFrontend.java
lient/src/autotest/tko/EmbeddedTkoClient.java
lient/src/autotest/tko/ExistingGraphsFrontend.java
lient/src/autotest/tko/GraphingFrontend.java
lient/src/autotest/tko/GraphingView.java
lient/src/autotest/tko/MachineQualHistogram.java
lient/src/autotest/tko/MachineQualHistogramFrontend.java
lient/src/autotest/tko/MetricsPlot.java
lient/src/autotest/tko/MetricsPlotFrontend.java
lient/src/autotest/tko/Plot.java
lient/src/autotest/tko/TkoClient.java
ko/graphing_utils.py
ko/models.py
ko/views.py
cda7ca909d4178c2a350ff674f10212f84f7b430 24-May-2017 Paul Hobbs <phobbs@google.com> [autotest] Add 500.html

Add a 500.html page. We get a lot of noisey logs from the "TemplateDoesNotExist:
500.html" error which occurs any time we have an exception.

BUG=chromium:725720
TEST=None

Change-Id: I123c8bb577cd88ed8ae36ca5a3bb10112590cf2a
Reviewed-on: https://chromium-review.googlesource.com/513608
Commit-Ready: Paul Hobbs <phobbs@google.com>
Tested-by: Paul Hobbs <phobbs@google.com>
Reviewed-by: Dan Shi <dshi@google.com>
fe/500.html
21e33a5f9bc55e6e57ae2db156f5780714d5865c 18-May-2017 Keith Haddow <haddowk@chromium.org> [moblab] Use credentials file from the bucket if available.

If the partner bucket has a credentials file - use that file and
enable the cloud notifications.

Fix issue with the command to restart after the wizard, in some
cases restart was not a good option as the service had not started
stop and start solves that issue.

Cleanup, only validate the boto key one time, since it is now a
more expensive operation.

TEST=unit tests, manually tested running GTS on local moblab
BUG=chromium:723863

Change-Id: I88e6d6901227fa667299a7c973d4f9276aceaf96
Reviewed-on: https://chromium-review.googlesource.com/508157
Commit-Ready: Keith Haddow <haddowk@chromium.org>
Tested-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Michael Tang <ntang@chromium.org>
fe/moblab_rpc_interface.py
fe/moblab_rpc_interface_unittest.py
6576f46ad4b02ec082a9361760fd45e3d76ce279 18-May-2017 Allen Li <ayatane@chromium.org> [autotest] Fix references to client.common_lib.site_utils

BUG=None
TEST=None

Change-Id: I889cd224c66509627cb6bc8677ead2ef3c3c6a87
Reviewed-on: https://chromium-review.googlesource.com/508167
Reviewed-by: Aviv Keshet <akeshet@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Commit-Queue: Allen Li <ayatane@chromium.org>
fe/moblab_rpc_interface.py
82d149426a730f9f0efcae9256a911c30dd57969 12-May-2017 Aviv Keshet <akeshet@chromium.org> autotest: deprecate graph-related tko rpcs

These rpcs, which I believe are not actually used, create a lot of
import time dependencies, including on numpy (which is emitting warnings
into our error logs) and many other modules not needed in our apache rpc
handlers.

BUG=None
TEST=None

Change-Id: Icf102d2e3cd29bcb2a6d789c66ec329ac05775fc
Reviewed-on: https://chromium-review.googlesource.com/505089
Commit-Ready: Aviv Keshet <akeshet@chromium.org>
Tested-by: Aviv Keshet <akeshet@chromium.org>
Reviewed-by: Allen Li <ayatane@chromium.org>
ko/rpc_interface.py
3102fd49b27f975966a4dadef540b2e5c6de4926 10-May-2017 Keith Haddow <haddowk@chromium.org> [moblab] Sort the build numbers correctly when generating build list.

Build numbers were being sorted by standard cmp - change to a custom
comparison that gets the order correct.

BUG=chromium:681915
TEST=unit tests and local moblab testing

Change-Id: I3d1ae480420aa8ff9b1c96bec161106f0b0aca4b
Reviewed-on: https://chromium-review.googlesource.com/502048
Commit-Ready: Keith Haddow <haddowk@chromium.org>
Tested-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Michael Tang <ntang@chromium.org>
fe/moblab_rpc_interface.py
fe/moblab_rpc_interface_unittest.py
a5752e2a70694f17133bbbad3d9a4a82df124a73 08-May-2017 Aviv Keshet <akeshet@chromium.org> autotest: null out self-cancelleing afe migrations 110-113

BUG=chromium:719628
TEST=None

Change-Id: Ia30c98654688482e0d4a0e3a19d9025cb24f216e
Reviewed-on: https://chromium-review.googlesource.com/498711
Commit-Ready: Aviv Keshet <akeshet@chromium.org>
Tested-by: Aviv Keshet <akeshet@chromium.org>
Reviewed-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Dan Shi <dshi@google.com>
igrations/110_remove_synch_id.py
igrations/111_add_back_synch_id_temporarily.py
igrations/112_remove_synch_id.py
igrations/113_add_back_sync_id_temporarily.py
igrations/old_110_remove_synch_id.ignore
igrations/old_111_add_back_synch_id_temporarily.ignore
350d7890fe5081f3e99684366fd80386cae271a8 28-Apr-2017 Keith Haddow <haddowk@chromium.org> [moblab] Call new upstart command that correctly restarts apache.

restart moblab-apache-init does not cleanly or correctly restart
apache, call a new upstart config that calls the correct command
to restart apache.

Replace the use of the boto library with a gsutil command.

Use the same path to the gsutil command that the devserver uses.

BUG=chromium:714294
TEST=locally on moblab, unittests

Change-Id: I0befe734d2c6130dadcfdaa5984aec23bba07807
Reviewed-on: https://chromium-review.googlesource.com/497169
Commit-Ready: Keith Haddow <haddowk@chromium.org>
Tested-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Michael Tang <ntang@chromium.org>
fe/moblab_rpc_interface.py
fe/moblab_rpc_interface_unittest.py
5ed7e63aeec90dc0c6b4229a66bf161410aa86a6 04-Feb-2017 Allen Li <ayatane@chromium.org> [autotest] Merge client/common_lib/site_utils

BUG=chromium:672727
TEST=None

Change-Id: I15859c0e05e99d244eec5b470a984ced0c874f19
Reviewed-on: https://chromium-review.googlesource.com/437727
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Allen Li <ayatane@chromium.org>
fe/models.py
6dec0e1ddc726a0e5e2c0a8b2d8ebefd00ca32f0 18-Apr-2017 Aviv Keshet <akeshet@chromium.org> autotest: make master silently ignore wrong-shard job updates

BUG=chromium:711852
TEST=rpc_interface_unittest; rpc_utils_unittest; overhaul of a unittest
to correctly test the wrong-job-in-heartbeat behavior

Change-Id: I5ebaa659da89f1dbcf484e1ef1f0c7ffef177544
Reviewed-on: https://chromium-review.googlesource.com/479550
Reviewed-by: Aviv Keshet <akeshet@chromium.org>
Tested-by: Aviv Keshet <akeshet@chromium.org>
Commit-Queue: Aviv Keshet <akeshet@chromium.org>
fe/frontend_test_utils.py
fe/models.py
fe/rpc_interface_unittest.py
fe/rpc_utils.py
fe/rpc_utils_unittest.py
0a0029f7f35d7987b856b80e096649961b1f216a 17-Apr-2017 Aviv Keshet <akeshet@chromium.org> autotest: stop using django.urls.defaults (deprecated module)

BUG=chromium:711808
TEST=None

Change-Id: If33c59af9eadbf796ca2ef3ca343c79e4e30a031
Reviewed-on: https://chromium-review.googlesource.com/479586
Tested-by: Aviv Keshet <akeshet@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
Commit-Queue: Aviv Keshet <akeshet@chromium.org>
fe/urls.py
ko/urls.py
rls.py
rls_common.py
ee480ade8d9c29938a3cc1dfa7a7d0bf1dfd2c4d 17-Apr-2017 Aviv Keshet <akeshet@chromium.org> autotest: raw_post_data -> body (address django DeprecationWarning)

BUG=chromium:711808
TEST=None

Change-Id: I66e4b5c39c7cbfceeed61a148cbf9943c4076854
Reviewed-on: https://chromium-review.googlesource.com/479694
Commit-Ready: Aviv Keshet <akeshet@chromium.org>
Tested-by: Aviv Keshet <akeshet@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
fe/rpc_handler.py
fe/views.py
920a2d1fb62a56abdfd58f90df1d7d457abbc55c 20-Mar-2017 Michael Tang <ntang@google.com> Revert "autotest: throw Auto Refresh checkbox into the abyss".

We will add the auto refresh back to Moblab environtment only.

This reverts commit 04ec30490d877f5eddfd1b747b507f754798c985.

BUG=chromium:700418
TEST=manually.

Change-Id: Ib6069e35da927f1d71cdc08abda183fbefd0c3b0
Reviewed-on: https://chromium-review.googlesource.com/456634
Commit-Ready: Michael Tang <ntang@chromium.org>
Tested-by: Michael Tang <ntang@chromium.org>
Reviewed-by: Aviv Keshet <akeshet@chromium.org>
Reviewed-by: Keith Haddow <haddowk@chromium.org>
lient/src/autotest/afe/AfeClient.java
lient/src/autotest/common/ui/CustomTabPanel.java
lient/src/autotest/common/ui/TabView.java
lient/src/autotest/moblab/MoblabSetupClient.java
bbffcbef0ef7d62f76b0f43edde91fdc57678ca4 05-Apr-2017 Prathmesh Prabhu <pprabhu@chromium.org> [autotest] Include jobs from today in lists

Both on the host detail view and the job list, we would like to see the
jobs from the current day. So, set the default end date to be tomorrow.

BUG=chromium:708243
TEST=Manually check UI on local AFE.

Change-Id: If9959c12fa01fac15ce026ba9ea0e31c95bad56f
Reviewed-on: https://chromium-review.googlesource.com/468387
Commit-Ready: Prathmesh Prabhu <pprabhu@chromium.org>
Tested-by: Prathmesh Prabhu <pprabhu@chromium.org>
Reviewed-by: Xixuan Wu <xixuan@chromium.org>
lient/src/autotest/common/table/DatetimeSegmentFilter.java
224b2205aa7d271c67f7c8041d9f86cba80a1928 04-Apr-2017 Aviv Keshet <akeshet@chromium.org> autotest: add a remove_boards_from_shard RPC

BUG=chromium:704445
TEST=unittests added, and they pass

Change-Id: I1dbbc68cbb29a341e9d35d1bdb322f1bb398da74
Reviewed-on: https://chromium-review.googlesource.com/467990
Commit-Ready: Aviv Keshet <akeshet@chromium.org>
Tested-by: Aviv Keshet <akeshet@chromium.org>
Reviewed-by: Dan Shi <dshi@google.com>
fe/rpc_interface.py
fe/rpc_interface_unittest.py
bf963c188531f899edded62641118140a97f21dd 04-Apr-2017 Aviv Keshet <akeshet@chromium.org> autotest: fix race condition between heartbeat and label removal

BUG=chromium:704445
TEST=New test added. Test verified to fail prior to fix.

Change-Id: Ifdc49e1937d1a3580a8689d05ea135afc7a4d445
Reviewed-on: https://chromium-review.googlesource.com/468086
Commit-Ready: Aviv Keshet <akeshet@chromium.org>
Tested-by: Aviv Keshet <akeshet@chromium.org>
Reviewed-by: Aviv Keshet <akeshet@chromium.org>
fe/models.py
fe/rpc_interface_unittest.py
6f6f8996580dc94cfe2357aa54ebef09ba255ba9 29-Mar-2017 xixuan <xixuan@chromium.org> autotest: add timeout for reboot_dut_for_shard_deletion.

BUG=chromium:705166
TEST=Run unittest.

Change-Id: If63cd5e0a4f158a71ab1dce27ee1d30cfedebf46
Reviewed-on: https://chromium-review.googlesource.com/462335
Commit-Ready: Xixuan Wu <xixuan@chromium.org>
Tested-by: Xixuan Wu <xixuan@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>
fe/rpc_interface.py
ab026ee3192440424f71b25cef1144a3698bc3c5 29-Mar-2017 Prathmesh Prabhu <pprabhu@chromium.org> ui: Add a job creation date filter to job list

We regularly get hammered by queries for jobs matching a certain name.
Almost no one wants to look at all jobs since the start of time. So, add
a job created filter that defaults to the last week.

BUG=chromium:685424
TEST=manual testing on local afe; verified that the resulting db queries
limit the job list search within the bounds given.

Change-Id: I5f2229b81e7511fb717409a7657d44f887b58fa2
Reviewed-on: https://chromium-review.googlesource.com/462120
Commit-Ready: Prathmesh Prabhu <pprabhu@chromium.org>
Tested-by: Prathmesh Prabhu <pprabhu@chromium.org>
Reviewed-by: Xixuan Wu <xixuan@chromium.org>
lient/src/autotest/afe/JobListView.java
lient/src/autotest/public/AfeClient.html
cc430e2b9401f7f72440ce95bbcd1e920cf19982 29-Mar-2017 Prathmesh Prabhu <pprabhu@chromium.org> ui: Notify listeners when DateTimeSegmentFilter value changes

This is a bugfix. Nothing was depending on this notification being sent
yet, but a future CL adds a listener who wasn't getting notifications on
change.

BUG=chromium:685424
TEST=Manual testing on local afe.

Change-Id: I121b35bf742a5003b3105466a97e28e5fa9c5e51
Reviewed-on: https://chromium-review.googlesource.com/462119
Commit-Ready: Prathmesh Prabhu <pprabhu@chromium.org>
Tested-by: Prathmesh Prabhu <pprabhu@chromium.org>
Reviewed-by: Xixuan Wu <xixuan@chromium.org>
lient/src/autotest/common/table/DatetimeSegmentFilter.java
484217188039b02e25716a88901667b62a53653e 29-Mar-2017 Prathmesh Prabhu <pprabhu@chromium.org> ui: Change the default date range to be last week

In a future CL, this will let us use the default range for the initial
query.

BUG=chromium:685424
TEST=Local afe manual testing.

Change-Id: Ic589db689cce7e64240cd3a0612376cb73c50112
Reviewed-on: https://chromium-review.googlesource.com/461356
Commit-Ready: Prathmesh Prabhu <pprabhu@chromium.org>
Tested-by: Prathmesh Prabhu <pprabhu@chromium.org>
Reviewed-by: Xixuan Wu <xixuan@chromium.org>
lient/src/autotest/common/table/DatetimeSegmentFilter.java
4be5a16025db8deb035cea08eec92ad42661831c 23-Mar-2017 Aviv Keshet <akeshet@chromium.org> [autotest] determine incorrect hosts during heartbeat rpc on master

BUG=chromium:704445
TEST=new unittest added, passes

Change-Id: I46fbec68a3c1559d9f382a5245d5e59ed6481a15
Reviewed-on: https://chromium-review.googlesource.com/457810
Commit-Ready: Aviv Keshet <akeshet@chromium.org>
Tested-by: Aviv Keshet <akeshet@chromium.org>
Reviewed-by: Dan Shi <dshi@google.com>
fe/models.py
fe/rpc_interface.py
fe/rpc_interface_unittest.py
fe/rpc_utils.py
6120a15cce48735e54128ec779cea820ceae12ae 23-Mar-2017 Aviv Keshet <akeshet@chromium.org> [autotest] allow creation of empty shards

BUG=None
TEST=shard_unittest; rpc_interface_unittest

Change-Id: I0eda2bd67ca42cdd8717d16d93a7e228c1439ec2
Reviewed-on: https://chromium-review.googlesource.com/457848
Commit-Ready: Aviv Keshet <akeshet@chromium.org>
Tested-by: Aviv Keshet <akeshet@chromium.org>
Reviewed-by: Dan Shi <dshi@google.com>
fe/rpc_interface.py
fe/rpc_interface_unittest.py
fddab4a620978bdb74c72299325fbe441e36977a 02-Feb-2017 Richard Barnette <jrbarnette@chromium.org> [autotest] Delete support for parameterized jobs.

This deletes the `create_parameterized_job()` RPC endpoint and the
associated (unused!) "parameterized_jobs" config setting. It also
garbage collects code made unreferenced by these deletions.

BUG=chromium:679903
TEST=unit tests, and run push_to_prod suite in a local instance

Change-Id: I0c02db8dcd4e8b0a8ac9649723eefffd31c95d82
Reviewed-on: https://chromium-review.googlesource.com/451838
Commit-Ready: Richard Barnette <jrbarnette@chromium.org>
Tested-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Dan Shi <dshi@google.com>
fe/frontend_test_utils.py
fe/model_attributes.py
fe/models.py
fe/models_test.py
fe/rpc_interface.py
fe/rpc_interface_unittest.py
fe/rpc_utils.py
a5ae9a5d1f39f495065c02a2202136182fd7b52c 20-Mar-2017 Keith Haddow <haddowk@chromium.org> [moblab] Add the CTS N suite into the suite runner UI

New suite for running the N version of CTS

TEST=None
BUG=chromium:702008

Change-Id: Ia4e3fbf16368d32de21d212008ae913181ae87c5
Reviewed-on: https://chromium-review.googlesource.com/457407
Commit-Ready: Keith Haddow <haddowk@chromium.org>
Tested-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Jinsong Mu <jinsong@google.com>
Reviewed-by: Keith Haddow <haddowk@chromium.org>
lient/src/autotest/moblab/SuiteRunnerView.java
5d84dc7f52f71992f4ac2719bc099d55f815c37f 17-Mar-2017 Aviv Keshet <akeshet@chromium.org> Revert "Adds the auto refresh checkbox back to the UI for moblab."

This reverts commit ed09d8e93668acfbe638d5a8c07879466ca82de6.

Change-Id: I8b35fa4e4ae31e25c986b5d26929dce3b9efc28a
Reviewed-on: https://chromium-review.googlesource.com/456594
Commit-Ready: Aviv Keshet <akeshet@chromium.org>
Tested-by: Aviv Keshet <akeshet@chromium.org>
Reviewed-by: Aviv Keshet <akeshet@chromium.org>
lient/src/autotest/afe/AfeClient.java
lient/src/autotest/common/StaticDataRepository.java
lient/src/autotest/common/ui/CustomTabPanel.java
lient/src/autotest/common/ui/TabView.java
lient/src/autotest/tko/TkoClient.java
204eac11dbaf4b5659ebefdb50a536c821e9bb3e 31-Jan-2017 Allen Li <ayatane@chromium.org> [autotest] [atomic] Remove atomic groups RPC

BUG=chromium:681906
TEST=Run unittest suite

Change-Id: I7b46a4390000a3ed6500b2db0f893b7dfaa04ce6
Reviewed-on: https://chromium-review.googlesource.com/435566
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Richard Barnette <jrbarnette@google.com>
fe/admin.py
fe/doctests/001_rpc_test.txt
fe/frontend_test_utils.py
fe/moblab_rpc_interface.py
fe/models.py
fe/rpc_interface.py
fe/rpc_interface_unittest.py
fe/rpc_utils.py
ed09d8e93668acfbe638d5a8c07879466ca82de6 11-Mar-2017 Michael Tang <ntang@google.com> Adds the auto refresh checkbox back to the UI for moblab.

We will allow autofresh for Moblab UI, and all existing autotest UI
except the Job List tab.

This partially reverts commit 04ec30490d877f5eddfd1b747b507f754798c985.

BUG=chromium:700418
TEST=manually.

Change-Id: I0ef9bf65520684ff857c8442081f04ddd1e0c824
Reviewed-on: https://chromium-review.googlesource.com/453276
Commit-Ready: Michael Tang <ntang@chromium.org>
Tested-by: Michael Tang <ntang@chromium.org>
Reviewed-by: Aviv Keshet <akeshet@chromium.org>
Reviewed-by: Dan Shi <dshi@google.com>
lient/src/autotest/afe/AfeClient.java
lient/src/autotest/common/StaticDataRepository.java
lient/src/autotest/common/ui/CustomTabPanel.java
lient/src/autotest/common/ui/TabView.java
lient/src/autotest/tko/TkoClient.java
b28def33556c59638a885cdc5710c9594c8fcd76 07-Mar-2017 Richard Barnette <jrbarnette@chromium.org> [autotest] Remove the 'parameterized_jobs' global config setting.

This removes the 'parameterized_jobs' setting from global_config.ini,
and the associated code that uses or tests the setting.

BUG=chromium:679903
TEST=unit tests; test the 'Create Job' button in the AFE UI.

Change-Id: I0846c0bc0f369d48cec7f71346c04961f8f52365
Reviewed-on: https://chromium-review.googlesource.com/450829
Commit-Ready: Richard Barnette <jrbarnette@chromium.org>
Tested-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Michael Tang <ntang@chromium.org>
fe/frontend_test_utils.py
fe/models.py
fe/models_test.py
fe/rpc_interface.py
rontend_unittest.py
ed0da862e2553395b2e42566d273f9bd9f7e52a2 06-Mar-2017 Shuqian Zhao <shuqianz@chromium.org> [autotest] teach run_suite to pass test_args all the way to each test

Add a test_args option in run_suite, which is a dict of args that could
be used by the test in the run time. Teach run_suite.py to pass this
option all the way to the test itself.

BUG=chromium:689691
TEST=unittest and testing on a test server.

Change-Id: I7bfbc447d078d37aa6c6f725cf9d8c4064fad5e7
Reviewed-on: https://chromium-review.googlesource.com/446779
Commit-Ready: Shuqian Zhao <shuqianz@chromium.org>
Tested-by: Shuqian Zhao <shuqianz@chromium.org>
Reviewed-by: Ilja H. Friedel <ihf@chromium.org>
Reviewed-by: Aviv Keshet <akeshet@chromium.org>
fe/rpc_interface.py
c6fd83859297c962536dac9e4330df0023ab2b2a 09-Mar-2017 Keith Haddow <haddowk@chromium.org> [moblab] Add moblab id and mac address to the UI

These details are required for the linking of account to devices.

TEST=None
BUG=chromium:679769

Change-Id: Id23294814be6c135c2dbbe3e3e480457c47f5894
Reviewed-on: https://chromium-review.googlesource.com/452002
Commit-Ready: Keith Haddow <haddowk@chromium.org>
Tested-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Stephen Lin <stephenlin@google.com>
Reviewed-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Michael Tang <ntang@chromium.org>
fe/moblab_rpc_interface.py
lient/src/autotest/moblab/rpc/VersionInfo.java
lient/src/autotest/moblab/wizard/ConfigWizard.java
04ec30490d877f5eddfd1b747b507f754798c985 09-Mar-2017 Aviv Keshet <akeshet@chromium.org> autotest: throw Auto Refresh checkbox into the abyss

BUG=chromium:700174
TEST=None

Change-Id: I39bd7d48f043f1504a2322ec4ce49c2812b74d2c
Reviewed-on: https://chromium-review.googlesource.com/452742
Commit-Ready: Aviv Keshet <akeshet@chromium.org>
Tested-by: Aviv Keshet <akeshet@chromium.org>
Reviewed-by: Ningning Xia <nxia@chromium.org>
Reviewed-by: Dan Shi <dshi@google.com>
lient/src/autotest/common/ui/CustomTabPanel.java
lient/src/autotest/common/ui/TabView.java
2a56e2ed79a5e737ce2e830807f8778d862e6585 07-Mar-2017 Richard Barnette <jrbarnette@chromium.org> [autotest] Remove UI dependencies on parameterized jobs.

This removes the (small) UI dependencies on the 'parameterized_jobs'
data field returned by the 'get_static_data' RPC.

BUG=chromium:679903
TEST=Test the 'Create Job' button in the AFE UI.

Change-Id: I859301f509ad32fd1a84fd4addd126383b591fd8
Reviewed-on: https://chromium-review.googlesource.com/450870
Commit-Ready: Richard Barnette <jrbarnette@chromium.org>
Tested-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Michael Tang <ntang@chromium.org>
Reviewed-by: Dan Shi <dshi@google.com>
lient/src/autotest/afe/create/CreateJobViewPresenter.java
541debad7b2f90f8add9b4ae69b1489ba6752a0d 02-Mar-2017 Aviv Keshet <akeshet@chromium.org> autotest: web front end (JobListView) ban "standalone" radio button

There's some evidence that the query for standalone jobs is particularly
expensive, often the source of forever-hung queries in prod.

Eliminate the UI element for it.

BUG=chromium:626198
TEST=None

Change-Id: I9d46e74201cf01585d320fa0be3b7fb8f4965fbd
Reviewed-on: https://chromium-review.googlesource.com/448703
Commit-Ready: Aviv Keshet <akeshet@chromium.org>
Tested-by: Aviv Keshet <akeshet@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
lient/src/autotest/afe/JobListView.java
4cefffc8b537472a853e55967926c658639159c2 07-Mar-2017 Prathmesh Prabhu <pprabhu@chromium.org> [autotest] Fix unittest failure

Introduced by two reverts interacting badly. Not my day.

BUG=None
Test=utils/unittest_suite.py

Change-Id: I7e8b91ef01f499b7ba678310255c17de2ca55c2a
Reviewed-on: https://chromium-review.googlesource.com/450779
Tested-by: Prathmesh Prabhu <pprabhu@chromium.org>
Reviewed-by: Ningning Xia <nxia@chromium.org>
fe/models_test.py
2d3884ade80ee5a8e2316ddc0996e7a1a12d58da 06-Mar-2017 Prathmesh Prabhu <pprabhu@chromium.org> Revert "[autotest] Remove synch_id from database"

This reverts commit c744f5fa4992f996768a4857c28fcc8045ea9df1.

Reason for revert: We don't want lab downtime today. We decided
to batch up some outstanding DB changes and do it in one fell
swoop. I'm just the executor...

Original change's description:
> [autotest] Remove synch_id from database
>
> BUG=chromium:687781
> TEST=Run migrate.py sync back and forth
>
> Change-Id: Ieed2052c9aa6958cf3fb6081ee310499c233d112
> Reviewed-on: https://chromium-review.googlesource.com/449972
> Reviewed-by: Allen Li <ayatane@chromium.org>
> Commit-Queue: Allen Li <ayatane@chromium.org>
> Tested-by: Allen Li <ayatane@chromium.org>
>

TBR=pprabhu@chromium.org,xixuan@chromium.org,ayatane@chromium.org
BUG=chromium:687781
BUG=chromium:691084

Change-Id: Ia822a826f8f63dbe32c435adf9e4aea2045a6c3f
Reviewed-on: https://chromium-review.googlesource.com/450837
Reviewed-by: Ningning Xia <nxia@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
Commit-Queue: Ningning Xia <nxia@chromium.org>
Tested-by: Ningning Xia <nxia@chromium.org>
fe/doctests/001_rpc_test.txt
fe/models_test.py
fe/rdb_model_extensions.py
igrations/113_add_back_sync_id_temporarily.py
f616e8fb22fcefa94ba18a6f2f8223fde642790f 06-Mar-2017 Richard Barnette <jrbarnette@chromium.org> Revert "[autotest] Delete support for parameterized jobs."

This reverts commit 0ed607598403c8e757b64b8c92f02f6b13a3eaee.

BUG=chromium:698773
TEST=click "Create Job" in a local autotest instance

Change-Id: I7d9f073d81f8a003cc79945e9608c78e9465dfb1
Reviewed-on: https://chromium-review.googlesource.com/449967
Commit-Queue: Richard Barnette <jrbarnette@chromium.org>
Tested-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Dan Shi <dshi@google.com>
fe/frontend_test_utils.py
fe/model_attributes.py
fe/models.py
fe/models_test.py
fe/rpc_interface.py
fe/rpc_interface_unittest.py
fe/rpc_utils.py
rontend_unittest.py
c744f5fa4992f996768a4857c28fcc8045ea9df1 11-Feb-2017 Allen Li <ayatane@chromium.org> [autotest] Remove synch_id from database

BUG=chromium:687781
TEST=Run migrate.py sync back and forth

Change-Id: Ieed2052c9aa6958cf3fb6081ee310499c233d112
Reviewed-on: https://chromium-review.googlesource.com/449972
Reviewed-by: Allen Li <ayatane@chromium.org>
Commit-Queue: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
fe/doctests/001_rpc_test.txt
fe/models_test.py
fe/rdb_model_extensions.py
igrations/112_remove_synch_id.py
0ed607598403c8e757b64b8c92f02f6b13a3eaee 02-Feb-2017 Richard Barnette <jrbarnette@chromium.org> [autotest] Delete support for parameterized jobs.

This deletes the `create_parameterized_job()` RPC endpoint and the
associated (unused!) "parameterized_jobs" config setting. It also
garbage collects code made unreferenced by these deletions.

BUG=chromium:679903
TEST=unit tests, and run push_to_prod suite in a local instance

Change-Id: I8ed683d0c8ffa879d0e10319b0a736133d56327e
Reviewed-on: https://chromium-review.googlesource.com/436407
Commit-Ready: Richard Barnette <jrbarnette@chromium.org>
Tested-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Richard Barnette <jrbarnette@google.com>
fe/frontend_test_utils.py
fe/model_attributes.py
fe/models.py
fe/models_test.py
fe/rpc_interface.py
fe/rpc_interface_unittest.py
fe/rpc_utils.py
rontend_unittest.py
076a655d23cfd631b73bb668de46a236c1c16497 28-Feb-2017 Prathmesh Prabhu <pprabhu@chromium.org> Revert "[autotest] Remove synch_id from database"

This reverts commit eb97ee7fffb4a2ef3155b07f2aa71f04d574b642.
But not quite. We never revert DB migration scripts, especially if
they've been run on any servers. So, this CL adds a new migration (111)
to revert the migration (110) that was already applied to the test
servers.
To reland, remove synch_id once again via a new migration.

Reason for revert:
This CL requires lab downtime for push-to-prod because host_scheduler
can not be cleanly restarted across this CL.

To push this CL to lab, one must:
- close the lab.
- migrate DB to the new schema (host_scheduler starts failing)
- push the CL to lab (host_scheduler recovers)
- open the lab.

Now is not the time to do this dance, so reverting with the hope to
reland at a better time.

BUG=chromium:696691
TEST=test_push host_scheduler can start with tip-of-tree code and with
tip-of-prod after applying both migration 110 and 111.

Change-Id: If2f5428f100af400d498eded9dfb08d6f22aa4dc
Reviewed-on: https://chromium-review.googlesource.com/447200
Tested-by: Prathmesh Prabhu <pprabhu@chromium.org>
Reviewed-by: Dan Shi <dshi@google.com>
fe/doctests/001_rpc_test.txt
fe/models_test.py
fe/rdb_model_extensions.py
igrations/111_add_back_synch_id_temporarily.py
eb97ee7fffb4a2ef3155b07f2aa71f04d574b642 11-Feb-2017 Allen Li <ayatane@chromium.org> [autotest] Remove synch_id from database

BUG=chromium:687781
TEST=Run migrate.py sync back and forth

Change-Id: Ib066b19d90b18ae9926e8aec2c871b963e6a2465
Reviewed-on: https://chromium-review.googlesource.com/441107
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Dan Shi <dshi@google.com>
fe/doctests/001_rpc_test.txt
fe/models_test.py
fe/rdb_model_extensions.py
igrations/110_remove_synch_id.py
da1118db50fa006d1377a344262425126ddd9105 14-Feb-2017 Shuqian Zhao <shuqianz@chromium.org> [autotest] Teach afe rpcs to accept job_keyvals arg to pass general job keyvals

Add a job_keyval variable to afe rpcs, to pass a general dictionary of
keyvals to suite job. This dict of keyval can then be used by tko/parse
for any suite result operations or analysis.

BUG=chromium:690167
TEST=unittest and test on a testing server.

Change-Id: I8a348fee601257345b71ff0a573b18d0f2b9550d
Reviewed-on: https://chromium-review.googlesource.com/441151
Commit-Ready: Shuqian Zhao <shuqianz@chromium.org>
Tested-by: Shuqian Zhao <shuqianz@chromium.org>
Reviewed-by: Shuqian Zhao <shuqianz@chromium.org>
fe/rpc_interface.py
5d94b2b13f90e8016b548d64b353c0b179498466 23-Feb-2017 Dan Shi <dshi@google.com> [autotest] log rpc server name in the metadata logged to metadb

BUG=None
TEST=local test

Change-Id: Ic63b93ac5f8d19e8f00ffb2527e2ac8b9ca3878f
Reviewed-on: https://chromium-review.googlesource.com/446727
Commit-Ready: Dan Shi <dshi@google.com>
Tested-by: Dan Shi <dshi@google.com>
Reviewed-by: Kevin Cheng <kevcheng@chromium.org>
fe/json_rpc/serviceHandler.py
7009bf7a60e83646f02df7cc413794d942715494 06-Feb-2017 Allen Li <ayatane@chromium.org> [autotest] Add tests verifying Django behavior

BUG=None
TEST=None

Change-Id: Ia195910ab834c9f49a51de7545f8bb5faf6082b8
Reviewed-on: https://chromium-review.googlesource.com/437617
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
fe/rpc_utils_unittest.py
127977d03c7782c682ba124b363b75d849e12798 06-Feb-2017 Allen Li <ayatane@chromium.org> [autotest] Fix awkward Django usage

Django model fields are usually accessed by attribute.
get_object_dict is like using __dict__ to get a dict representation.

BUG=None
TEST=None

Change-Id: I02a18364a525ef934abca715f6b0b8f22363270a
Reviewed-on: https://chromium-review.googlesource.com/437822
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
fe/rpc_utils.py
a62507136ba9e68c9fb03339b2f111bfd5deccd5 06-Feb-2017 Allen Li <ayatane@chromium.org> [autotest] Clean up get_sample_dut (conservatively)

Caused by https://chromium-review.googlesource.com/#/c/418356/, see
original discussion there.

BUG=None
TEST=None

Change-Id: Id30b7f95d1de6f8bfb453b84aee5a9867b4be441
Reviewed-on: https://chromium-review.googlesource.com/422475
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
fe/rpc_utils.py
e4c08273f039d97349999818e25fedbb13cdcb4b 02-Feb-2017 Allen Li <ayatane@chromium.org> [autotest] Remove synch_id concept

BUG=chromium:687781
TEST=None

Change-Id: Ibbef8f7b58d7bb26dcbbe85df308b9e0c4b86bd9
Reviewed-on: https://chromium-review.googlesource.com/435711
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Allen Li <ayatane@chromium.org>
fe/models.py
fe/rdb_model_extensions.py
12b9bc1142bd9ac5cbe6187358d721471820ca93 15-Dec-2016 Allen Li <ayatane@chromium.org> [autotest] Remove unused exception variable

BUG=None
TEST=None

Change-Id: I4b2115dcc28c26861e6cddac3ed425169b7b0e18
Reviewed-on: https://chromium-review.googlesource.com/420515
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
fe/rpc_utils.py
afb7d3290bdc1a32f9f36e81597c4086e1510b77 15-Dec-2016 Allen Li <ayatane@chromium.org> [autotest] Make gather_unique_dicts private

BUG=None
TEST=None

Change-Id: I418f374f7129f25fb6d218b929e6a64411555ab7
Reviewed-on: https://chromium-review.googlesource.com/420514
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
fe/rpc_utils.py
07478c0069ee41c31d60ca2b84ae6d193980a24f 15-Dec-2016 Allen Li <ayatane@chromium.org> [autotest] Simplify gather_unique_dicts logic

BUG=None
TEST=None

Change-Id: I0b658ab95b2a405db83d731f2ae37dc7ff5ba64d
Reviewed-on: https://chromium-review.googlesource.com/420513
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
fe/rpc_utils.py
8239c35c1dddc79a213b1c01a1b8dca6f8cf853a 15-Dec-2016 Allen Li <ayatane@chromium.org> [autotest] Simplify logic on check_for_duplicate_hosts()

Of note, Django object compare and equal if they have the same
id (they represent the same database tuple).

BUG=None
TEST=None

Change-Id: Iaad49743253323dfcd7244f7ae62b9be46bf28a0
Reviewed-on: https://chromium-review.googlesource.com/420507
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
fe/rpc_utils.py
672f40a930ed4d2c53f5584cf2ca47b77c2a100c 04-Feb-2017 xixuan <xixuan@chromium.org> autotest: Remove RPCs related to users & Remove UserPreferences page.

This CL:
1. Remove RPCs related to users, including add/delete/modify_user.
2. Remove UserPreferences page.

BUG=chromium:688622
TEST=Run local AFE, kick off a verify job.
Run frontend_unittest & rpc_interface_unittest.

Change-Id: Ifa048635418240d6d9145e1ebf4ab2b601483bd4
Reviewed-on: https://chromium-review.googlesource.com/437728
Commit-Ready: Xixuan Wu <xixuan@chromium.org>
Tested-by: Xixuan Wu <xixuan@chromium.org>
Reviewed-by: Richard Barnette <jrbarnette@google.com>
fe/doctests/001_rpc_test.txt
fe/rpc_interface.py
lient/src/autotest/afe/AfeClient.java
lient/src/autotest/afe/UserPreferencesView.java
lient/src/autotest/public/AfeClient.html
a69662ed90334bd5064e4c324081097f7698f491 04-Feb-2017 xixuan <xixuan@chromium.org> autotest: Remove RPCs related to test.

This CL removes RPCs: add/delete/modify_test.

BUG=chromium:688620
TEST=Run frontend_unittest & rpc_interface_unittest.

Change-Id: I38bb8f410b2d7ff37f61a38cfa68ad526dc516b2
Reviewed-on: https://chromium-review.googlesource.com/437817
Commit-Ready: Xixuan Wu <xixuan@chromium.org>
Tested-by: Xixuan Wu <xixuan@chromium.org>
Reviewed-by: Richard Barnette <jrbarnette@google.com>
fe/doctests/001_rpc_test.txt
fe/doctests/003_misc_rpc_features.txt
fe/rpc_interface.py
6cd838f23983c501ed9b109931771f944cdf2bdc 03-Feb-2017 Dan Shi <dshi@google.com> [autotest] Create a generic mock class for handling import error from chromite

This change allows one to do following:
try:
from chromite.lib import metrics
except ImportError:
metrics = utils.metrics_mock

Then the code to call metrics can stay the same, and don't need to handle the
case where chromite is not installed through build_externals.

BUG=chromium:688166
TEST=unittest

Change-Id: I1ca58c8223b1a0d81bc8bddce567b863b5743e73
Reviewed-on: https://chromium-review.googlesource.com/436409
Commit-Ready: Dan Shi <dshi@google.com>
Tested-by: Dan Shi <dshi@google.com>
Reviewed-by: Dan Shi <dshi@google.com>
fe/rpc_interface.py
583215128a773aad0fedf3030e5fdf91f1e0f380 03-Feb-2017 Dan Shi <dshi@google.com> [autotest] Fix a bug in TKO query with custom fields.

BUG=chromium:441017
TEST=local check, unittest

Change-Id: Ia379411d7714f0b5e4318efad4ac1ec1681f957b
Reviewed-on: https://chromium-review.googlesource.com/437544
Commit-Ready: Dan Shi <dshi@google.com>
Tested-by: Dan Shi <dshi@google.com>
Reviewed-by: Richard Barnette <jrbarnette@google.com>
fe/model_logic.py
9d29c26beb33a13422479e98b3604b2a4eacf80b 01-Feb-2017 xixuan <xixuan@chromium.org> autotest: Remove Reinstall-related codes.

This CL removes all codes related to host 'Reinstall'.

BUG=chromium:615914
TEST=Run local afe, verify that there's no Reinstall on 'Host List' and
'View Host'. Kick off a verify job, and verify it runs well.

Change-Id: Id51a9ba5a7668a42cfdb2ccdd670c1f866f88526
Reviewed-on: https://chromium-review.googlesource.com/435487
Commit-Ready: Xixuan Wu <xixuan@chromium.org>
Tested-by: Xixuan Wu <xixuan@chromium.org>
Reviewed-by: Richard Barnette <jrbarnette@google.com>
lient/src/autotest/afe/AfeUtils.java
lient/src/autotest/afe/HostDetailView.java
lient/src/autotest/afe/HostListView.java
lient/src/autotest/public/AfeClient.html
046a14f0acb61fc08ca77e842adef3ff650af00d 02-Feb-2017 Allen Li <ayatane@chromium.org> [autotest] Simplify direct_afe proxy logic

BUG=chromium:672727
TEST=None

Change-Id: Idcc9627e5c53a0348ceb58b00b13afc000d8f16e
Reviewed-on: https://chromium-review.googlesource.com/436564
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
fe/direct_afe.py
cdd00f20ac7607ff89a97e83e2483a4c8feddb7b 02-Feb-2017 Allen Li <ayatane@chromium.org> [autotest] Remove site_rpc_interface

BUG=chromium:672727
TEST=Run unittests

Change-Id: I8535b121ba7f2317cccec981bf2a8a36ca2639cf
Reviewed-on: https://chromium-review.googlesource.com/435850
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
Reviewed-by: Allen Li <ayatane@chromium.org>
fe/control_file.py
fe/direct_afe.py
fe/rpc_interface.py
fe/rpc_interface_unittest.py
fe/site_rpc_interface.py
fe/site_rpc_interface_unittest.py
fe/views.py
dfb4e30592ddeaceb270df3fbab3ff3faefa5396 02-Feb-2017 Richard Barnette <jrbarnette@chromium.org> [autotest] Correctly spell "specifying" in the AFE GWT code.

BUG=chromium:687710
TEST=Look at the AFE in a local Autotest instance

Change-Id: I3137bc6464c8b6012191e698f13139b384e0a083
Reviewed-on: https://chromium-review.googlesource.com/435892
Commit-Ready: Richard Barnette <jrbarnette@chromium.org>
Tested-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Shuqian Zhao <shuqianz@chromium.org>
lient/src/autotest/afe/HostSelectorDisplay.java
4de4e74d1eeedf605e272530c780f2632a62181c 30-Jan-2017 xixuan <xixuan@chromium.org> autotest: Remove backend dead codes related to recurring jobs.

This CL removes all backend dead codes related to recurring jobs.

BUG=chromium:681913
TEST=Run local AFE, run a repair job for a DUT.
Run unittest. Run 'python models.test'.

Change-Id: I95d4ba9bdf47741cf82ca05808985a181e3ce467
Reviewed-on: https://chromium-review.googlesource.com/434838
Commit-Ready: Xixuan Wu <xixuan@chromium.org>
Tested-by: Xixuan Wu <xixuan@chromium.org>
Reviewed-by: Xixuan Wu <xixuan@chromium.org>
fe/models.py
fe/rpc_interface.py
d8220fa92c8777d84f51f0eed55ae7827a6120c4 02-Feb-2017 Dan Shi <dshi@google.com> [autotest] Update TKO db schema

1. Add index for tko_tests.finished_time
2. Update tko_test_attributes id key to bigint.

BUG=chromium:617703
TEST=local run migrate.py sync

Change-Id: I9167f2fcea2a99ac2bf6e9757e1d05b1fbe67104
Reviewed-on: https://chromium-review.googlesource.com/436485
Commit-Ready: Dan Shi <dshi@google.com>
Tested-by: Dan Shi <dshi@google.com>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
igrations/108_index_tko_tests_finished_time.py
igrations/109_change_tko_test_attributes_id_data_type.py
0b469fc935dbca1f0fd3cfbcd477cb7b49cebde5 30-Jan-2017 Allen Li <ayatane@chromium.org> [autotest] [atomic] Remove atomic group from AFE frontend

BUG=chromium:681906
TEST=Run unittest suite

Change-Id: I59e231974d6a82ac6011206e21f0b791224a1162
Reviewed-on: https://chromium-review.googlesource.com/434095
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Richard Barnette <jrbarnette@google.com>
lient/src/autotest/afe/AfeUtils.java
lient/src/autotest/afe/HostSelector.java
lient/src/autotest/afe/HostTableDecorator.java
51c7052e7caaef996d8d36682f9ee4a0234a54d5 30-Jan-2017 xixuan <xixuan@chromium.org> autotest: Remove frontend dead code about recurring jobs.

This CL removes frontend codes about recurring jobs.

BUG=chromium:681913
TEST=Run local AFE, verify that recurring tabs and buttons are removed.

Change-Id: Ifd9ba556eb3ddb84771fada5d374c09709fa2adc
Reviewed-on: https://chromium-review.googlesource.com/434109
Commit-Ready: Xixuan Wu <xixuan@chromium.org>
Tested-by: Xixuan Wu <xixuan@chromium.org>
Reviewed-by: Dan Shi <dshi@google.com>
lient/src/autotest/afe/AfeClient.java
lient/src/autotest/afe/JobDetailView.java
lient/src/autotest/afe/RecurringTable.java
lient/src/autotest/afe/RecurringView.java
lient/src/autotest/public/AfeClient.html
6c2b70aa871b5740f31e1c2ec65b8c0b2f3a1d98 26-Jan-2017 Richard Barnette <jrbarnette@chromium.org> [autotest] Use provisioning in place of parameterized jobs.

Formerly, when the create_job() RPC was passed an image for testing,
the handling was to create a special parameterized job that would
invoke autoserv with the `--image` option, so that the build would
be installed directly. However, the specified `image` parameter
couldn't be passed to shards, so the feature could only work for
DUTs assigned to the autotest master.

This changes handling such that when an image is specified at job
creation time, the image is converted to a `cros-version` label
dependency. This allows the specified build information to be
passed to shards. Additionally, after this change, the code
relating to parameterized jobs becomes dead (along with the related
image specification/installation code), enabling additional code
cleanup.

BUG=chromium:628694
TEST=create jobs with images via web UI and CLI in a local instance

Change-Id: Ic01c7bdbbc6ded537a9f4808844b2d7d32663494
Reviewed-on: https://chromium-review.googlesource.com/433364
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Allen Li <ayatane@chromium.org>
fe/rpc_interface.py
f3ed9cbca7eeee7dfa2200d88852ed0fb68cd617 23-Jan-2017 Dan Shi <dshi@google.com> [autotest] Add index to tko_jobs started_time

BUG=chromium:683965
TEST=local verify

Change-Id: I0a456964cfb210d03bb4b679f01a17313c0f3451
Reviewed-on: https://chromium-review.googlesource.com/431115
Commit-Ready: Dan Shi <dshi@google.com>
Tested-by: Dan Shi <dshi@google.com>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
igrations/107_index_tko_jobs_started_time.py
120ecfab95cb590370e8e3245b966aaabfda678b 23-Jan-2017 Dan Shi <dshi@google.com> [autotest] Modify tko_job_keyvals table to allow DELETE CASCADE in FK

BUG=None
TEST=local verify

Change-Id: I0187efa203acc22e08a2ef7a9cc124b0a165129b
Reviewed-on: https://chromium-review.googlesource.com/431314
Commit-Ready: Dan Shi <dshi@google.com>
Tested-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
igrations/106_update_test_job_keyvals_foreign_keys.py
8fd206e60be86ffbbc53e472b65a6a053b05400a 19-Jan-2017 Dan Shi <dshi@google.com> [autotest] Modify tko_test_labels_tests table to allow DELETE CASCADE in FK

That way we can delete a record in tko_tests table and have everything related
cleaned up.

BUG=None
TEST=run database/migrate.py in local instance
DEPLOY=migrate

Change-Id: If04139ac6922b6e0526614844ad1d9d11c1d09d9
Reviewed-on: https://chromium-review.googlesource.com/430814
Commit-Ready: Dan Shi <dshi@google.com>
Tested-by: Dan Shi <dshi@google.com>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
igrations/105_update_test_label_foreign_keys.py
224aa55cdae0640c4d1704255f5eaba7516fb919 15-Dec-2016 Allen Li <ayatane@chromium.org> [autotest] Move synch_count over max check into check function

check_atomic_group_create_job is called by create_job_common is called by
create_job.

BUG=None
TEST=None

Change-Id: I616a8725e95263f5d4276b4fde773bc720366895
Reviewed-on: https://chromium-review.googlesource.com/430209
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Allen Li <ayatane@chromium.org>
fe/rpc_utils.py
dbc0866bbf9e7f3c84e604e1fba4d7e627b3a9f6 15-Dec-2016 Allen Li <ayatane@chromium.org> [autotest] Move host_objects logic together

BUG=None
TEST=None

Change-Id: I73e6d8775c2885b2757681bfc53c125df2358006
Reviewed-on: https://chromium-review.googlesource.com/430208
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Allen Li <ayatane@chromium.org>
fe/rpc_utils.py
3d4e61115dd2a730e9bbe3e679fb32feac8a241a 28-Dec-2016 Allen Li <ayatane@chromium.org> [autotest] Simplify bucket_hosts_by_shard with defaultdict

Beside making the code cleaner, this is slightly more performant since
we dont create and throw away an empty list for setdefault() in every
loop.

BUG=None
TEST=None

Change-Id: I05aa81c47cbc604a1378794dd214ee58629a0c3e
Reviewed-on: https://chromium-review.googlesource.com/430207
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Allen Li <ayatane@chromium.org>
fe/rpc_utils.py
acb97924f18636f2c6bf70f7fe12c48c6ddb3c92 14-Dec-2016 Allen Li <ayatane@chromium.org> [autotest] Restructure _allowed_hosts_for_master_job() logic

BUG=None
TEST=Unit tests

Change-Id: I4573bf69abbef91cdbb96960865bb8cc0232becf
Reviewed-on: https://chromium-review.googlesource.com/430206
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Allen Li <ayatane@chromium.org>
fe/rpc_utils.py
9fb3a19af830e2f67e24a42bbe07ee17cc77e764 15-Dec-2016 Allen Li <ayatane@chromium.org> [autotest] Add tests for _allowed_hosts_for_master_job

BUG=None
TEST=Run tests

Change-Id: I844e91746e94467e0ef2e19481fd2626575f2eff
Reviewed-on: https://chromium-review.googlesource.com/420353
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
fe/rpc_utils_unittest.py
e62031989da33b8fd90d23083dee3a92b2ae040f 14-Dec-2016 Allen Li <ayatane@chromium.org> [autotest] Remove redundant iterable guards (squashed)

The default values are iterable, we dont need this guard.

BUG=None
TEST=None

!Change-Id: I8910e44f45fcdbff5ab03cccef28f3a117756aea

[autotest] Replace or chain with any()

BUG=None
TEST=None

!Change-Id: I647d42164b4c2b872c286dd35f511f9dd7cc6583

[autotest] Inline server_type temp var

BUG=None
TEST=None

!Change-Id: Ic7e57c70c84c8d4c72e741093c8822de05b6f54b

[autotest] Use dict comprehension for atomic_groups_by_name

BUG=None
TEST=None

!Change-Id: I90fc74d3f7d2b4d526cdc60be6d09644a556b439

[autotest] Inline owner temp var

BUG=None
TEST=None

!Change-Id: I4f1d5789b62380334886d2c1ca0d16327ec3f918

[autotest] Extract _get_atomic_group_name_from_labels

BUG=None
TEST=None

!Change-Id: I352c0ae8e355d6dfcca95799a6d1cda2f8f712f2

[autotest] Extract _validate_host_job_sharding

BUG=None
TEST=None

!Change-Id: Iae60e34a2b8e0ff5ae47a33c6b2fead66a5bb749

[autotest] Extract _allowed_hosts_for_master_job

BUG=None
TEST=None

!Change-Id: Ie271ad881e8a5564899408f4898008a134b68176

[autotest] Refactor check in _validate_host_job_sharding

BUG=None
TEST=None

!Change-Id: I5efcc07a2ecaee91ba1d844167f4e43c58b1a8b3

Change-Id: I2da7d4b2e15da4d93b1f8e4b85d365f3bf8fc055
Reviewed-on: https://chromium-review.googlesource.com/424932
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Allen Li <ayatane@chromium.org>
fe/rpc_utils.py
81996a8c7af4f0556755bf5534c2ba308a60aca4 14-Dec-2016 Allen Li <ayatane@chromium.org> [autotest] Nuke get_create_job_common_args

BUG=None
TEST=None

Change-Id: I689206c076461db297e72659c60bf27c875eadb6
Reviewed-on: https://chromium-review.googlesource.com/419693
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Allen Li <ayatane@chromium.org>
fe/rpc_interface.py
fe/rpc_utils.py
a5cfb97ee793d849acf0efa4369b43d6f91a9c4d 28-Dec-2016 Allen Li <ayatane@chromium.org> [autotest] Add get_hostname_addr_map() (squashed)

BUG=chromium:666022
TEST=None

!Change-Id: I9b3385fd8fba91f67fe968bb7bf85d39792e8c16

[autotest] Add hostname property to devserver

BUG=None
TEST=None

!Change-Id: I7fcd9b92d3c086ea9a8e6a4749f65b65a827855f

[autotest] Extract get_hostname() function

BUG=None
TEST=None

!Change-Id: I45f7a8e4f3f57beb77df6766fbe5747ebe295613

[autotest] Replace get_server_name with get_hostname

BUG=None
TEST=None

!Change-Id: I75ebfe05ccd1f4d9209d68bc0e5f2db691558a74

[autotest] Add _reverse_lookup_from_config()

BUG=chromium:666022
TEST=None

!Change-Id: I1ea54296cc5a7a67981338c1c3d4057b065fa279

[autotest] Attempt to look up devserver hostname for metrics

BUG=chromium:666022
TEST=None

!Change-Id: I3dec24d4874de66466a373c97de5364c4b1ba557

[autotest] Make get_section_as_dict() ordered

The ordering information is preserved in the config file, so may as
well preserve it here.

BUG=chromium:666022
TEST=None

!Change-Id: I46b9fa47fa87b789a4e63563797a4327ac308bef

[autotest] Clarify _reverse_lookup behavior for duplicates

BUG=chromium:666022
TEST=None

!Change-Id: I839ba7b55005b6137fa03887854ea59dfee49dfa

Change-Id: I0aeb05d16d8fc8b98987dc232da8928ddd9fdb9f
Reviewed-on: https://chromium-review.googlesource.com/424913
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Allen Li <ayatane@chromium.org>
fe/site_rpc_interface.py
fe/site_rpc_interface_unittest.py
47034eec9d1d1e3bfb6b4fbd1b754d136a263a92 14-Dec-2016 Allen Li <ayatane@chromium.org> [autotest] Expand locals in create_parameterized_job()

BUG=None
TEST=None

Change-Id: I637ad87081f861fe8ce8ab7a0d89d2665a727b11
Reviewed-on: https://chromium-review.googlesource.com/419692
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Allen Li <ayatane@chromium.org>
fe/rpc_interface.py
7975299832ecce3b2bd9670e53f9ab06fe560d29 14-Dec-2016 Allen Li <ayatane@chromium.org> [autotest] Reconcile create_job first common call args

The sole purpose of get_create_job_common_args() is pruning locals()
of invalid keyword arguments to create_job_common(). After expanding
locals() in place, this commit reconciles the keyworg arguments passed
to those that are valid for create_job_common().

BUG=None
TEST=None

Change-Id: Ibc72564f570261cee72d18386a6c6b3d1326fb76
Reviewed-on: https://chromium-review.googlesource.com/419809
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Allen Li <ayatane@chromium.org>
fe/rpc_interface.py
8e17eb1f4335a17a550c7cce72e795a3bc187959 14-Dec-2016 Allen Li <ayatane@chromium.org> [autotest] Expand locals()

BUG=None
TEST=None

Change-Id: I37b9193fd5308eeedf1b20c823b608641d56bc70
Reviewed-on: https://chromium-review.googlesource.com/420177
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Allen Li <ayatane@chromium.org>
fe/rpc_interface.py
ac199b6a6e3ee10061630dcb438ed5f137c25e03 14-Dec-2016 Allen Li <ayatane@chromium.org> [autotest] Remove unused parameter is_cloning

BUG=None
TEST=None

Change-Id: I3578fe0105b129bc02d5fdbada096270c0c9eb0e
Reviewed-on: https://chromium-review.googlesource.com/419691
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
fe/rpc_interface.py
a59b1269e959c830ecc700da88c4e1aa2c2cd83f 14-Dec-2016 Allen Li <ayatane@chromium.org> [autotest] Fix control_file name conflicts

BUG=None
TEST=None

Change-Id: Ie96bd20ef87a6eb75eaedef73fc285ce652fcdce
Reviewed-on: https://chromium-review.googlesource.com/419690
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
fe/rpc_interface.py
41e47c1ee16bc129b5b7381a5b438f66407c8a96 14-Dec-2016 Allen Li <ayatane@chromium.org> [autotest] Format create_parameterized_job parameter list

BUG=None
TEST=None

Change-Id: I1d48101f9f7877671cd2ced9a66b31ca8a72b669
Reviewed-on: https://chromium-review.googlesource.com/419689
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
fe/rpc_interface.py
7c918d841487f0597d0207657de09c054d348963 14-Dec-2016 Allen Li <ayatane@chromium.org> [autotest] Expand create_job second common call locals()

Expand locals() usage against valid parameters for create_job_common

BUG=None
TEST=None

Change-Id: I9a61c344ebccea45f2182ca3dee7a973bd0ee764
Reviewed-on: https://chromium-review.googlesource.com/419810
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Allen Li <ayatane@chromium.org>
fe/rpc_interface.py
1453fdfb9fba4a8f6aa5f81fec0bf9484e228235 14-Dec-2016 Allen Li <ayatane@chromium.org> [autotest] Format create_job_common parameter list

BUG=None
TEST=None

Change-Id: Ia04129f8590ecef0ec94488024e66b5d105b726b
Reviewed-on: https://chromium-review.googlesource.com/419808
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
fe/rpc_utils.py
a06ea083f536d8bf755be9356dba66a369bb9ab2 13-Jan-2017 Don Garrett <dgarrett@chromium.org> Revert "[autotest] Use provisioning in place of parameterized jobs."

This reverts commit ca14cc94db12a404c840ebb4b8e8dffe66245e3c.

BUG=chromium:680656

Change-Id: Ic596fb29e15c4a9309e099c83ab36723131d8743
Reviewed-on: https://chromium-review.googlesource.com/427999
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Don Garrett <dgarrett@chromium.org>
Tested-by: Don Garrett <dgarrett@chromium.org>
fe/rpc_interface.py
352b86a33c78c0ad35f55c14ffba11896d27556c 14-Dec-2016 Allen Li <ayatane@chromium.org> [autotest] Remove old priority hack

This is from 2013, Im assuming the priorities have all been migrated
over.

The reason for doing this is that the sole purpose of this function is
to remove invalid keywords from passing locals() to create_job_common,
so this is a step toward removing locals() and this function.

BUG=None
TEST=CQ

Change-Id: Ia5b527b509863bbe12ff047628c23a988dbc32c9
Reviewed-on: https://chromium-review.googlesource.com/420178
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
fe/rpc_interface_unittest.py
fe/rpc_utils.py
fe/site_rpc_interface_unittest.py
ca14cc94db12a404c840ebb4b8e8dffe66245e3c 22-Dec-2016 Richard Barnette <jrbarnette@chromium.org> [autotest] Use provisioning in place of parameterized jobs.

Formerly, when the create_job() RPC was passed an image for testing,
the handling was to create a special parameterized job that would
invoke autoserv with the `--image` option, so that the build would
be installed directly. However, the specified `image` parameter
couldn't be passed to shards, so the feature could only work for
DUTs assigned to the autotest master.

This changes handling such that when an image is specified at job
creation time, the image is converted to a `cros-version` label
dependency. This allows the specified build information to be
passed to shards. Additionally, after this change, the code
relating to parameterized jobs becomes dead (along with the related
image specification/installation code), enabling additional code
cleanup.

BUG=chromium:628694
TEST=create jobs with images via web UI and CLI in a local instance

Change-Id: Id1fb5444698bb776ca5367d00e74ad3dbbd1a88c
Reviewed-on: https://chromium-review.googlesource.com/423389
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Allen Li <ayatane@chromium.org>
fe/rpc_interface.py
ab8d379fb3746951da2db86449db0fa2b726c6d7 13-Dec-2016 Allen Li <ayatane@chromium.org> [autotest] Simplify _to_kwargs implementation

BUG=None
TEST=Run unittests

Change-Id: I9c5d03b7693d4cfbb9c6213b42ba252c75c595f5
Reviewed-on: https://chromium-review.googlesource.com/419087
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Allen Li <ayatane@chromium.org>
fe/rpc_utils.py
fe/rpc_utils_unittest.py
ce51f3729a7c1f48f01123d264c862debcd82a36 13-Dec-2016 Allen Li <ayatane@chromium.org> [autotest] Clean up route_rpc_to_master docstring

BUG=None
TEST=None

Change-Id: I577f9119dad3d5aa27c02331efd04d1e042dff9a
Reviewed-on: https://chromium-review.googlesource.com/419086
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
fe/rpc_utils.py
416c405c6bbdd736d1f424661805d38336843332 13-Dec-2016 Allen Li <ayatane@chromium.org> [autotest] Extract _to_kwargs function

BUG=None
TEST=None

Change-Id: Ib01ec5dee7f95f5d056a592cd2c335273cd1703c
Reviewed-on: https://chromium-review.googlesource.com/419085
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
fe/rpc_utils.py
54121d03789b5d10b1bb1116c547c5ee120829da 13-Dec-2016 Allen Li <ayatane@chromium.org> [autotest] Move check out of decorated wrapper

This check can and should be done during decoration time, not when the
decorated function is called.

BUG=None
TEST=None

Change-Id: I1112b23f7aea66c3dd8c2f5244c9ebcd4f4630f6
Reviewed-on: https://chromium-review.googlesource.com/419084
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Allen Li <ayatane@chromium.org>
fe/rpc_utils.py
8af9da0cf32d999413c00cdf141372f2d881e339 13-Dec-2016 Allen Li <ayatane@chromium.org> [autotest] Format create_job parameter list

BUG=None
TEST=None

Change-Id: I436f789200ff801c7f6048ef17a44b11fcb82623
Reviewed-on: https://chromium-review.googlesource.com/419083
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
fe/rpc_interface.py
c3d702b878761d209140f86afd5e3025e1a5f380 21-Dec-2016 Dan Shi <dshi@chromium.org> Revert "Clean up get_sample_dut()"

This reverts commit 73d77fa0919bee236f676e7d47908abbc6b22332.

Change-Id: Iba4d38be7422c071b83a39c97acf1bf8ba3619bb
Reviewed-on: https://chromium-review.googlesource.com/422498
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
Tested-by: Dan Shi <dshi@chromium.org>
fe/rpc_utils.py
7fdff33c26b5f5a4a8cfb05a7c788a3f9c373bb4 09-Dec-2016 Allen Li <ayatane@chromium.org> [autotest] Move control file code together

BUG=chromium:672348
TEST=None

Change-Id: I5c3c77786d1be0818b61cc9655f914b295771118
Reviewed-on: https://chromium-review.googlesource.com/418496
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Richard Barnette <jrbarnette@google.com>
fe/site_rpc_interface.py
d60106ad6c8a781ca145aad635deb03214c6a96e 09-Dec-2016 Allen Li <ayatane@chromium.org> [autotest] Remove redundant call

suite_name has already been canonicalized

BUG=chromium:672348
TEST=None

Change-Id: I44ab1ddc4da9e5660761a219d6edd6109d24db7b
Reviewed-on: https://chromium-review.googlesource.com/418495
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Richard Barnette <jrbarnette@google.com>
fe/site_rpc_interface.py
73d77fa0919bee236f676e7d47908abbc6b22332 09-Dec-2016 Allen Li <ayatane@chromium.org> Clean up get_sample_dut()

BUG=chromium:672348
TEST=None

Change-Id: Ie1970edab5fa028e996c8649ab8d45c2a7c23549
Reviewed-on: https://chromium-review.googlesource.com/418356
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Richard Barnette <jrbarnette@google.com>
fe/rpc_utils.py
3d43e60ee3167b0ef610652fc1c7ff38ebc57c62 09-Dec-2016 Allen Li <ayatane@chromium.org> Move prefer local devserver check

BUG=chromium:672348
TEST=None

Change-Id: I12725fbee9118a7d6ed19dd5b6f0fdbfa0e94cda
Reviewed-on: https://chromium-review.googlesource.com/418250
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Allen Li <ayatane@chromium.org>
fe/rpc_utils.py
fe/site_rpc_interface.py
f9d0e71a9a4449bed9eb7cd1ddf1155cd69edc2d 08-Dec-2016 Allen Li <ayatane@chromium.org> Rename _get_control_file_contents_by_name function

BUG=chromium:672348
TEST=None

Change-Id: I9b8994d69494d87f1136bdb82f5ce3e5fe2b5b3b
Reviewed-on: https://chromium-review.googlesource.com/418249
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Richard Barnette <jrbarnette@google.com>
fe/site_rpc_interface.py
653078deea86edaf8604068bcff8ea8533e1daa9 08-Dec-2016 Allen Li <ayatane@chromium.org> Extract _get_control_file_by_suite method

BUG=chromium:672348
TEST=None

Change-Id: I4b9a8eb749f0dc038d9d2a12fd59d7792c342161
Reviewed-on: https://chromium-review.googlesource.com/418248
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Allen Li <ayatane@chromium.org>
fe/site_rpc_interface.py
bb4ef4d1e84c7a3eb3a197e2aafa08eab43210c1 08-Dec-2016 Allen Li <ayatane@chromium.org> Make separate branch for control file logic

This bit is doing something different from the surround code. Saving
a single boolean test is not worth confounding the two.

BUG=chromium:672348
TEST=None

Change-Id: I528e054019567768b8d1de844d0a94c8dc681ffc
Reviewed-on: https://chromium-review.googlesource.com/418247
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Richard Barnette <jrbarnette@google.com>
fe/site_rpc_interface.py
72ad82c60a1222daeead2feb5b90e6bf9ab64ffe 08-Dec-2016 Allen Li <ayatane@chromium.org> Flip branch test to match previous branch

BUG=chromium:672348
TEST=None

Change-Id: I6e85d088c6a3b2f955e1fd0090ca5dad5d497dbb
Reviewed-on: https://chromium-review.googlesource.com/418246
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Richard Barnette <jrbarnette@google.com>
fe/site_rpc_interface.py
b7408f40ba9e50ee3843ed9c91691e8864507d78 08-Dec-2016 Allen Li <ayatane@chromium.org> Move temp variable closer to use site

BUG=chromium:672348
TEST=None

Change-Id: Ib3fef08f550a3991adda0d537622e2d7fe197710
Reviewed-on: https://chromium-review.googlesource.com/418245
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Richard Barnette <jrbarnette@google.com>
fe/site_rpc_interface.py
f48bcacd13e213aa013a702cbc3eaf4b3de8ba17 08-Dec-2016 Allen Li <ayatane@chromium.org> Replace dangerous mutable default for builds

BUG=chromium:672348
TEST=None

Change-Id: I4dbdb055781645773faf4f8b2a0b8c03666171a1
Reviewed-on: https://chromium-review.googlesource.com/418244
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Richard Barnette <jrbarnette@google.com>
fe/site_rpc_interface.py
7c2883ec06b4c1f1e3b774e02ecb462401c6c28a 08-Dec-2016 Allen Li <ayatane@chromium.org> Format argument list

BUG=chromium:672348
TEST=None

Change-Id: I0ff001b5aa605347073ff1bed66cb7b9b02e8abb
Reviewed-on: https://chromium-review.googlesource.com/418243
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Richard Barnette <jrbarnette@google.com>
fe/site_rpc_interface.py
f0e2700e68a64577dc9c2fe26f7b5d26a7ce2768 23-Nov-2016 Prathmesh Prabhu <pprabhu@chromium.org> autotest: Log devserver name on artifact staging failure.

Creating suites involves staging artifacts on devservers. Sometimes
staging fails for weird reasons. To debug these failures, it is
necessary to know the devserver which was used. So, log it.

BUG=chromium:653362
TEST=unittests, run suite job on moblab.

Change-Id: I80809f7943209aa5017ee266497147eae5f2a967
Reviewed-on: https://chromium-review.googlesource.com/414206
Commit-Ready: Prathmesh Prabhu <pprabhu@chromium.org>
Tested-by: Prathmesh Prabhu <pprabhu@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
Reviewed-by: Aseda Aboagye <aaboagye@chromium.org>
Reviewed-by: Xixuan Wu <xixuan@chromium.org>
fe/site_rpc_interface.py
14cac44542bc6a403cb7a3585eb3ebae00227992 21-Nov-2016 Aviv Keshet <akeshet@chromium.org> autotest: move or eliminate a bunch of autotest_stats stats

BUG=chromium:667171
TEST=unittests; guado_moblab tryjob + test run

Change-Id: I815d2d748a2daa3bc869ead5d09f722245b7317b
Reviewed-on: https://chromium-review.googlesource.com/413103
Reviewed-by: Aviv Keshet <akeshet@chromium.org>
Commit-Queue: Aviv Keshet <akeshet@chromium.org>
Tested-by: Aviv Keshet <akeshet@chromium.org>
fe/json_rpc/serviceHandler.py
fe/model_logic.py
fe/rpc_interface.py
fe/rpc_utils.py
fe/site_rpc_interface.py
be00dc8d039d47be84bb77d652245cbd5900e055 18-Nov-2016 Keith Haddow <haddowk@chromium.org> moblab: change link text from to just Moblab since we do more

TEST=manual
BUG=none

Change-Id: If15de56d3baaaad7ef4e928f2e0c49284edacf70
Reviewed-on: https://chromium-review.googlesource.com/412274
Commit-Ready: Keith Haddow <haddowk@chromium.org>
Tested-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Jinsong Mu <jinsong@google.com>
Reviewed-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Michael Tang <ntang@chromium.org>
lient/src/autotest/public/AfeClient.html
e24cf0efc0f8fd1f8ec8af307508a5518e53e3b7 18-Nov-2016 Keith Haddow <haddowk@chromium.org> moblab: remove results option from the config wizard.

Most people can use the default /results directory easily, the
advanced config options allow people to change if neccesary.

TEST=manual
BUG=none

Change-Id: Ic078805825d9c3f5b94938194247b8eb2a5ff79f
Reviewed-on: https://chromium-review.googlesource.com/412865
Commit-Ready: Keith Haddow <haddowk@chromium.org>
Tested-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Jinsong Mu <jinsong@google.com>
Reviewed-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Michael Tang <ntang@chromium.org>
lient/src/autotest/moblab/wizard/CloudStorageCard.java
510ec0b873c30c5a565065f4321a37ee39056d86 18-Nov-2016 Keith Haddow <haddowk@chromium.org> moblab: Add FAFT to the run suite options.

Add FAFT as an option for the run suite UI.

Improve the selection of builds returned from GCS to the
last 20 of the last three milestones.

Fix bug with bvt-inline shown twice.

BUG=chromium:659413
TEST=installed on a 56 build and manually tested.

Change-Id: I3f0637c7a1793df8e84d761c9520d357211e3e0b
Reviewed-on: https://chromium-review.googlesource.com/412352
Commit-Ready: Keith Haddow <haddowk@chromium.org>
Tested-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Michael Tang <ntang@chromium.org>
fe/moblab_rpc_interface.py
lient/src/autotest/moblab/SuiteRunnerView.java
lient/src/autotest/moblab/rpc/MoblabRpcCallbacks.java
lient/src/autotest/moblab/rpc/MoblabRpcHelper.java
7e2b0da6e67376e3176a14cdada317638bab609e 15-Nov-2016 Keith Haddow <haddowk@chromium.org> moblab: change suite runner UI to dropdown and fix bugs.

Fixed bug in the manage DUT page that caused extra whitespace on
every refresh.

Fixed issue where pool label was being applied by UI but not required
by the backend test runner.

Added RPC to get the list of pools and changed the suite running UI
to a drop down and use these pool labels.

TEST=manually deployed to local moblab on 56 and tested.
BUG=chromium:659416

Change-Id: Iedeafa522b192e8c22c0c7e6361f8236eec48308
Reviewed-on: https://chromium-review.googlesource.com/411368
Commit-Ready: Keith Haddow <haddowk@chromium.org>
Tested-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Michael Tang <ntang@chromium.org>
fe/moblab_rpc_interface.py
lient/src/autotest/moblab/DutManagementView.java
lient/src/autotest/moblab/SuiteRunnerView.java
lient/src/autotest/moblab/rpc/MoblabRpcCallbacks.java
lient/src/autotest/moblab/rpc/MoblabRpcHelper.java
c7ab4aaa26de244b35a0bde8239fa9cab3040f67 11-Nov-2016 Keith Haddow <haddowk@chromium.org> Add gts suite to the suite runner list of options.

BUG=none
TEST=deployed on local test machine in R56 and manual test

Change-Id: I514baf3a99814bcfdb7c3212cd947c44c2efadc9
Reviewed-on: https://chromium-review.googlesource.com/410133
Commit-Ready: Keith Haddow <haddowk@chromium.org>
Tested-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Michael Tang <ntang@chromium.org>
lient/src/autotest/moblab/SuiteRunnerView.java
b5b8b4f981036971c619b26956c8847140eabd35 02-Nov-2016 Dan Shi <dshi@google.com> [autotest] Loosens board list check when adding new board labels.

testbed may have a mix of different boards. This change loosens the board list
check to allow any board labels ending with -number to be added to a host.

BUG=chromium:660271
TEST=unittest

Change-Id: If5f8487472f1092b5ebc24e3df90c7965aa382d0
Reviewed-on: https://chromium-review.googlesource.com/407068
Commit-Ready: Dan Shi <dshi@google.com>
Tested-by: Dan Shi <dshi@google.com>
Reviewed-by: Dan Shi <dshi@google.com>
fe/models.py
fe/models_test.py
fe/rpc_interface.py
fe/rpc_interface_unittest.py
8ef84175d610e8e32128946f46a4e897d042445d 29-Oct-2016 Keith Haddow <haddowk@chromium.org> autotest: New simplified interface to run suites for moblab partners.

New UI available at http://100.96.58.155/moblab_setup/#tab_id=suite_run

BUG=chromium:656131
TEST=build local version, installed on local mobab, tested manually.

Change-Id: Ieac969295af2b01fa25bcda04c6f06d2e338be26
Reviewed-on: https://chromium-review.googlesource.com/404949
Commit-Ready: Keith Haddow <haddowk@chromium.org>
Tested-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Michael Tang <ntang@chromium.org>
fe/moblab_rpc_interface.py
lient/src/autotest/MoblabSetupClient.gwt.xml
lient/src/autotest/moblab/MoblabSetupClient.java
lient/src/autotest/moblab/SuiteRunnerView.java
lient/src/autotest/moblab/rpc/MoblabRpcCallbacks.java
lient/src/autotest/moblab/rpc/MoblabRpcHelper.java
lient/src/autotest/public/MoblabSetupClient.html
lient/src/autotest/public/moblabsetup.css
4a3deb8609497ce4812584728d5289e34759fe72 28-Oct-2016 Dan Shi <dshi@google.com> [autotest] Allow atest host create to add multiple board label for testbed

testbed may have multiple board labels, e.g.,
board:angler-1, board:angler-2, board:angler-3

autotest code was changed earlier to prevent a dut to have multiple board
labels, which may corrupt shard database.
This change allows testbed to be able to add multiple board labels.

BUG=chromium:660271
TEST=local add testbed, unittest

Change-Id: I198cafef999c7064e6459985402e3248a57470b5
Reviewed-on: https://chromium-review.googlesource.com/404535
Tested-by: Dan Shi <dshi@google.com>
Reviewed-by: Kris Rambish <krisr@chromium.org>
Commit-Queue: Dan Shi <dshi@google.com>
fe/rpc_interface.py
fe/rpc_interface_unittest.py
40e182b10320d3a8596a7a4eb87d2ec6b981bc71 11-Oct-2016 Shuqian Zhao <shuqianz@chromium.org> [autotest] Ensure one host only has one board label

Ensure one host only has one board label with it, otherwise shard-client
will crash

BUG=chromium:654100
TEST=Test in local autotest

Change-Id: I4e9035d331d03554612ba46040d118a6807d950c
Reviewed-on: https://chromium-review.googlesource.com/396530
Commit-Ready: Shuqian Zhao <shuqianz@chromium.org>
Tested-by: Shuqian Zhao <shuqianz@chromium.org>
Reviewed-by: Dan Shi <dshi@google.com>
fe/models.py
fe/models_test.py
fe/rpc_interface.py
6a34cafd6e9b9d2609308a80a5292ca60e22611e 22-Oct-2016 Michael Tang <ntang@google.com> Adds UI to upload all Moblab credentials files.

BUG=chromium:658462
TEST=manual.

Change-Id: I53fff080f524c8cb793911227d18125e45da00fd
Reviewed-on: https://chromium-review.googlesource.com/401282
Reviewed-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Michael Tang <ntang@chromium.org>
Commit-Queue: Michael Tang <ntang@chromium.org>
Tested-by: Michael Tang <ntang@chromium.org>
Trybot-Ready: Michael Tang <ntang@chromium.org>
fe/moblab_rpc_interface.py
lient/src/autotest/common/ui/ContextMenu.java
lient/src/autotest/moblab/BotoKeyView.java
lient/src/autotest/moblab/CredentialsUploadView.java
lient/src/autotest/moblab/KeyUploadView.java
lient/src/autotest/moblab/LaunchControlKeyView.java
lient/src/autotest/moblab/MoblabSetupClient.java
lient/src/autotest/public/MoblabSetupClient.html
c28d3955905faa1ac8585f35a46a29ce550b9c03 18-Oct-2016 Dan Shi <dshi@google.com> [autotest] Add Sponge url to AFE job view is available

BUG=chromium:642144
TEST=local AFE

Change-Id: I86268175e6329f2c446d0e2f1afb0a61c2059d7b
Reviewed-on: https://chromium-review.googlesource.com/400006
Commit-Ready: Dan Shi <dshi@google.com>
Tested-by: Dan Shi <dshi@google.com>
Reviewed-by: Shuqian Zhao <shuqianz@chromium.org>
Reviewed-by: Dan Shi <dshi@google.com>
lient/src/autotest/afe/AfeUtils.java
lient/src/autotest/afe/JobDetailView.java
lient/src/autotest/public/AfeClient.html
63cc4470e13cf7dab9edb7e18c0c39e9cd65ba47 07-Oct-2016 Keith Haddow <haddowk@chromium.org> Add new DUT Manage page to make it easier to add/remove/label DUT's

BUG=chromium:653727
TEST=local

Change-Id: Id554af22b672ecd1bb9f8eeb89bf9b0a4206a7a0
Reviewed-on: https://chromium-review.googlesource.com/396463
Commit-Ready: Keith Haddow <haddowk@chromium.org>
Tested-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Michael Tang <ntang@chromium.org>
fe/moblab_rpc_interface.py
lient/src/autotest/MoblabSetupClient.gwt.xml
lient/src/autotest/moblab/DutManagementView.java
lient/src/autotest/moblab/MoblabSetupClient.java
lient/src/autotest/moblab/rpc/ConnectedDutInfo.java
lient/src/autotest/moblab/rpc/MoblabRpcCallbacks.java
lient/src/autotest/moblab/rpc/MoblabRpcHelper.java
lient/src/autotest/public/MoblabSetupClient.html
lient/src/autotest/public/moblabsetup.css
8345c081f3c63ee44c12e16fea1d773324d0007f 30-Sep-2016 Keith Haddow <haddowk@chromium.org> Change configuration commit to restart services, not reboot.

Rebooting the device is very intrusive and slow - just restart the
impacted services.

BUG=chromium:651877
TEST=local

Change-Id: Iff21d3da56dcc56e301226cc0cb3a48ed1e03b8e
Reviewed-on: https://chromium-review.googlesource.com/391043
Reviewed-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Reviewed-by: Michael Tang <ntang@chromium.org>
Tested-by: Keith Haddow <haddowk@chromium.org>
fe/moblab_rpc_interface.py
lient/src/autotest/moblab/ConfigSettingsView.java
lient/src/autotest/moblab/wizard/ConfigEditMode.java
e6f2457c4a82285c9a6df5f624cc0ae6ba565409 29-Sep-2016 Keith Haddow <haddowk@chromium.org> mobab: Fix validation of the cloud config wizard.

Ensure that buckets end in /
trim whitespace from keys and buckets

BUG=chromium::631233
TEST=local

Change-Id: Iba21a1953fcabc1223c60f942732323d8ce17d18
Reviewed-on: https://chromium-review.googlesource.com/391213
Commit-Ready: Keith Haddow <haddowk@chromium.org>
Tested-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Michael Tang <ntang@chromium.org>
lient/src/autotest/moblab/rpc/CloudStorageInfo.java
lient/src/autotest/moblab/wizard/CloudStorageCard.java
f127867fbdac97c8cafc34602a4db031e1cf8b44 29-Sep-2016 Keith Haddow <haddowk@chromium.org> moblab: Add version information to the setup place to aid support.

Knowing the version number is important to be able to support
partner issues when running moblab.

BUG=chromium:631233
TEST=local

Change-Id: I63f205fc02505e963988a5a5ae0eeb5441ead7df
Reviewed-on: https://chromium-review.googlesource.com/391131
Commit-Ready: Keith Haddow <haddowk@chromium.org>
Tested-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Michael Tang <ntang@chromium.org>
fe/moblab_rpc_interface.py
lient/src/autotest/moblab/rpc/MoblabRpcCallbacks.java
lient/src/autotest/moblab/rpc/MoblabRpcHelper.java
lient/src/autotest/moblab/rpc/VersionInfo.java
lient/src/autotest/moblab/wizard/ConfigWizard.java
88d4de483f2edd6d2692de1fce8a1232e021b339 28-Sep-2016 Keith Haddow <haddowk@chromium.org> moblab: Fix typo in code

Fix spelling mistake

BUG=chromium:631233
TEST=local

Change-Id: I6b0716828d0b360fd6a69348cba432c82b043986
Reviewed-on: https://chromium-review.googlesource.com/390393
Commit-Ready: Keith Haddow <haddowk@chromium.org>
Tested-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Michael Tang <ntang@chromium.org>
lient/src/autotest/moblab/wizard/NetworkInfoCard.java
19521983b5d77a557889850ead4447d07595ee14 22-Sep-2016 Kevin Cheng <kevcheng@chromium.org> [autotest] Add in host attr and locking rpcs/methods.

Add in get_hosts_by_attributes(attribute, value) RPC which returns a list
of hosts that share the same value for the specified host attribute.

Added a {lock,unlock}_hosts method on the AFE.

BUG=chromium:599533
TEST=locally on moblab
afe = frontend_wrappers.RetryingAFE(timeout_min=5, delay_sec=10)
print afe.get_hosts_by_attribute(
attribute='servo_host',
value='100.107.151.237')

Change-Id: Id288697ebd2f11dad04ba6a477a35b45e39f9830
Reviewed-on: https://chromium-review.googlesource.com/388674
Commit-Ready: Kevin Cheng <kevcheng@chromium.org>
Tested-by: Kevin Cheng <kevcheng@chromium.org>
Reviewed-by: Dan Shi <dshi@google.com>
fe/rpc_interface.py
6b89f1c14c65c2a3a130cc1a063143b38ebc3b94 18-Sep-2016 Dan Shi <dshi@google.com> [autotest] Optimize shard heartbeat query

Replace the sub query with join to make heartbeat query go faster.
Refer to:
https://bugs.chromium.org/p/chromium/issues/detail?id=641290#c51

BUG=chromium:641290
TEST=unittest, local setup

Change-Id: If41230af31da778ff425778fd7c8a69dd8a08d56
Reviewed-on: https://chromium-review.googlesource.com/386386
Tested-by: Dan Shi <dshi@google.com>
Reviewed-by: Aviv Keshet <akeshet@chromium.org>
fe/models.py
4596c37c47355f7c4d8acf9ee5e625cd0dfcf79d 14-Sep-2016 Dan Shi <dshi@google.com> [autotest] Add index to afe_jobs name field.

This index should make the suite scheduler run faster when seaching for
existing suite job.

BUG=None
TEST=local test
./database/migrate.py sync
confirm name_index is created
./database/migrate.py sync 103
confirm name_index is removed

DEPLOY=migrate

Change-Id: I1851a700bd9ff5df4be7a90dbae169161359b497
Reviewed-on: https://chromium-review.googlesource.com/385536
Commit-Ready: Dan Shi <dshi@google.com>
Tested-by: Dan Shi <dshi@google.com>
Reviewed-by: Aviv Keshet <akeshet@chromium.org>
igrations/104_index_job_name.py
ba232a36e058b58188a662647eeaa52c2b76a945 26-Aug-2016 xixuan <xixuan@chromium.org> Autotest: Move moblab-related RPC to moblab_rpc_interface.py.

This CL moves all RPCs that can only be run on moblab to moblab_rpc_interface.
Also move related unittests to moblab_rpc_interface_unittest.

BUG=chromium:641549
TEST=cros flash to a local moblab, upload boto key, then run a suite on it.

Change-Id: I8246282a4c7250350dd9a4e9043eaeba293ccb34
Reviewed-on: https://chromium-review.googlesource.com/377418
Commit-Ready: Xixuan Wu <xixuan@chromium.org>
Tested-by: Xixuan Wu <xixuan@chromium.org>
Reviewed-by: Fang Deng <fdeng@chromium.org>
Reviewed-by: Dan Shi <dshi@google.com>
fe/moblab_rpc_interface.py
fe/moblab_rpc_interface_unittest.py
fe/rpc_utils.py
fe/site_rpc_interface.py
fe/site_rpc_interface_unittest.py
fe/views.py
7ee95860dabcd3ca1c6cb054f59203534fcd5132 31-Aug-2016 Aviv Keshet <akeshet@chromium.org> autotest: refactor status_history into a lib outside of site_utils

CQ-DEPEND=CL:378599

BUG=chromium:641093
TEST=Ran dut_status command, verified working.

Change-Id: I9f60e2dff24683e66af00c7ce1c8a825b239e782
Reviewed-on: https://chromium-review.googlesource.com/378435
Commit-Ready: Aviv Keshet <akeshet@chromium.org>
Tested-by: Aviv Keshet <akeshet@chromium.org>
Reviewed-by: Richard Barnette <jrbarnette@google.com>
fe/rpc_interface.py
543835d098916b1615e8fb3b12c4e2f6403a2ea4 22-Aug-2016 xixuan <xixuan@google.com> Autotest: Add an Rpc Validator to limit RPCs that can only be called by master.

Some RPCs cannot be called by caller except master AFE, or it will lead to db
inconsistency between shard and master.

This CL fixes this problem by adding an RPC validator to filter such calls from
RPC handling side. The validor will refuse any calls whose caller is not master
AFE but the target calling method is RPCs in shard.

BUG=chromium:635288
TEST=Testing with a cbf master and a cbf shard: shard works, jobs can be
picked up, scheduled and worked. Check master/shard AFEs, can't see any missing
contents.

Change-Id: Iad52e6173a1f785e4d48ca5287a5c1dd2fce17ab
Reviewed-on: https://chromium-review.googlesource.com/374139
Commit-Ready: Xixuan Wu <xixuan@chromium.org>
Tested-by: Xixuan Wu <xixuan@chromium.org>
Reviewed-by: Xixuan Wu <xixuan@chromium.org>
fe/rpc_handler.py
61f5d31b6f6173069e1ad5ce28c739666398e237 06-Aug-2016 Shuqian Zhao <shuqianz@chromium.org> [autotest]only accept hosts when both hosts and meta_hosts given for a clone job

When cloning a job via autotest frontend, if user chooses the default
clone action, which is 'Reuse any similar hosts' and then later still
select specific hosts in the next tab, the host scheduler will get
confused about the case and would try to schedule two same jobs to one
host. In this case, the host scheduler will crash because of duplicate entries
for one job in the database. The fix is to only accept the hosts
variables under this situation.

BUG=chromium:611064
TEST=unittest

Change-Id: Id945a9f8f2b45271c903f1ec038366da47f5d436
Reviewed-on: https://chromium-review.googlesource.com/366795
Commit-Ready: Dan Shi <dshi@chromium.org>
Tested-by: Shuqian Zhao <shuqianz@chromium.org>
Reviewed-by: Fang Deng <fdeng@chromium.org>
Reviewed-by: Dan Shi <dshi@google.com>
fe/rpc_interface.py
303d2669b78eb5a2ebd6c41dce80ca18d06a19cf 02-Jul-2016 Laurence Goodby <lgoodby@google.com> [autotest] Scheduler fix for SYNC_COUNT > 1.

Details in go/autotest-sync-count-fix

BUG=chromium:621257
TEST=Run included tests.

Change-Id: If08df8fb04771a321dbdf2122b885935e7ef3b41
Reviewed-on: https://chromium-review.googlesource.com/358092
Commit-Ready: Laurence Goodby <lgoodby@chromium.org>
Tested-by: Laurence Goodby <lgoodby@chromium.org>
Reviewed-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Richard Barnette <jrbarnette@google.com>
Reviewed-by: Dan Shi <dshi@google.com>
fe/models.py
68139fbee1d206d1da2df7049e5ac816a7f5157d 04-Aug-2016 Shuqian Zhao <shuqianz@chromium.org> [autotest] support adding board labels to shard

This CL will support RPC to add board labels to an exist shard.

BUG=chromium:628387
TEST=Test rpc in local autotest

Change-Id: I4be4263142ccb46c34c7def1cbfd04ff87fe5b2d
Reviewed-on: https://chromium-review.googlesource.com/366033
Commit-Ready: Shuqian Zhao <shuqianz@chromium.org>
Tested-by: Shuqian Zhao <shuqianz@chromium.org>
Reviewed-by: Shuqian Zhao <shuqianz@chromium.org>
fe/site_rpc_interface.py
fe/site_rpc_interface_unittest.py
ca73e2d627469b8d01b78613fa28e5e2ccf53d5b 25-Jul-2016 Shuqian Zhao <shuqianz@chromium.org> [autotest] change the status.log link to root link of all test results

The status.log link is broken recently. Therefore, change it point to
the parent link of status.log, which is the link to all logs of a given
test.

BUG=chromium:629310
TEST=None

Change-Id: Ia12996e5178388efb5551bb330d9ea4ebbad0507
Reviewed-on: https://chromium-review.googlesource.com/362881
Commit-Ready: Shuqian Zhao <shuqianz@chromium.org>
Tested-by: Shuqian Zhao <shuqianz@chromium.org>
Reviewed-by: Dan Shi <dshi@google.com>
lient/src/autotest/afe/JobDetailView.java
78ae5c7d98752d93984b2091e892bc6b2591e252 22-Jul-2016 Michael Tang <ntang@google.com> Fixed the type "secrete".

BUG=chromium:630482
TEST=Manual.

Change-Id: I06071b08641334838e2cf17c8a3292b463ceef96
Reviewed-on: https://chromium-review.googlesource.com/362447
Commit-Ready: Michael Tang <ntang@chromium.org>
Tested-by: Michael Tang <ntang@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Reviewed-by: Michael Tang <ntang@chromium.org>
lient/src/autotest/moblab/wizard/CloudStorageCard.java
971b324350a9746a524c498464db0252e02b7d45 21-Jul-2016 Justin Giorgi <jgiorgi@google.com> [autotest] Fixed 500 status when adding hosts.

Corrected variable name in afe/admin. Corrected redirect in urls_common.

BUG=chromium:629868
TEST=Verified locally and on moblab.

Change-Id: Ied07656af56a2922ab7e208e357635ac1e352a31
Reviewed-on: https://chromium-review.googlesource.com/362095
Commit-Ready: Justin Giorgi <jgiorgi@google.com>
Tested-by: Justin Giorgi <jgiorgi@google.com>
Reviewed-by: Justin Giorgi <jgiorgi@google.com>
fe/admin.py
rls_common.py
8e33b4ee0ecb8b9165ac34620e874ed6e1281b79 21-May-2016 Richard Barnette <jrbarnette@chromium.org> [autotest] Drop RPC support for specifying a test kernel.

Our legacy Autotest code included support for specifying a specific
kernel to be tested when creating a job. That support wasn't
appropriate for Chrome OS, and didn't work.

All existing user level interfaces for specifying a kernel (CLI and
AFE) have been changed to drop this feature. This change removes
support for the feature in the RPC calls. Lower-level layers still
contain support that is unused, which will be removed in subsequent
CLs.

BUG=chromium:266704
TEST=Look at control file from AFE job creation page; run atest job create

Change-Id: Ifb0c6888f550f6cda98195b489ba8076cdf47e6b
Reviewed-on: https://chromium-review.googlesource.com/346570
Commit-Ready: Richard Barnette <jrbarnette@chromium.org>
Tested-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Richard Barnette <jrbarnette@google.com>
fe/control_file.py
fe/doctests/001_rpc_test.txt
fe/doctests/003_misc_rpc_features.txt
fe/rpc_interface.py
fe/rpc_utils.py
67ad67d564fd605dc0ac3671b1620a601cfc6c90 29-Jun-2016 Justin Giorgi <jgiorgi@google.com> [autotest] Removed copy of unittest library.

All tests still pass with the standard unittest library.

BUG=None
TEST=Executed utils/unittest_suite.py locally

Change-Id: I86f7e2884282bca0fcb240fe2f70c4adb71f8ddf
Reviewed-on: https://chromium-review.googlesource.com/357330
Commit-Ready: Justin Giorgi <jgiorgi@google.com>
Tested-by: Justin Giorgi <jgiorgi@google.com>
Reviewed-by: Richard Barnette <jrbarnette@google.com>
Reviewed-by: Simran Basi <sbasi@chromium.org>
fe/rpc_interface_unittest.py
0a30d467b58126b10589cad8be448ebe4f5a530b 27-Jun-2016 Michael Tang <ntang@google.com> Honor the results_storage_server parameter in moblab.

By default, the results_storage_server in global_config.ini is empty
now. Both lab and moblab could set the value and take effect. If not
set, the lab takes the original default value, and moblab takes the
image bucket uri.

BUG=chromium:621212
TEST=unit test.

Change-Id: Ifd712b53bcffa76af17db8cd4d87947323291b08
Reviewed-on: https://chromium-review.googlesource.com/356126
Commit-Ready: Michael Tang <ntang@chromium.org>
Tested-by: Michael Tang <ntang@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Reviewed-by: Michael Tang <ntang@chromium.org>
fe/site_rpc_interface.py
fe/site_rpc_interface_unittest.py
lient/src/autotest/moblab/rpc/CloudStorageInfo.java
lient/src/autotest/moblab/wizard/CloudStorageCard.java
lient/src/autotest/public/afeclient.css
16bba56968e78f167f8fe69439dc684f28f1a57f 22-Jun-2016 Justin Giorgi <jgiorgi@google.com> Cleaned up host mod and host create logic.

Merged site_host.py into host.py. Created base class for shared logic
between mod and create actions. Also merged tests to run the same test
cases against both actions.

CQ-DEPEND=CL:354552

BUG=None
TEST=Unit tests verified locally.

Change-Id: If14786ef5f58a415406efd7a2f24b96ca3ea1a74
Reviewed-on: https://chromium-review.googlesource.com/355733
Commit-Ready: Justin Giorgi <jgiorgi@google.com>
Tested-by: Justin Giorgi <jgiorgi@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
fe/admin.py
84a2ecf3422e6b7f52ffd4628e4a8d47dbe53b4e 08-Jun-2016 Michael Tang <ntang@google.com> Enabled create test job out of private test build.

We now allow user to have a test source build containing tests not
present in the moblab and its database. The user could fetch those tests
and select them to create jobs.

BUG=chromium:603774
TEST=manually tests and some unit test.

Change-Id: I4aecb922b006a0743ef4cb223f555fac7343c6c0
Reviewed-on: https://chromium-review.googlesource.com/351480
Commit-Ready: Michael Tang <ntang@chromium.org>
Tested-by: Michael Tang <ntang@chromium.org>
Reviewed-by: Michael Tang <ntang@chromium.org>
Reviewed-by: Dan Shi <dshi@google.com>
fe/control_file.py
fe/rpc_interface.py
fe/rpc_utils.py
fe/rpc_utils_unittest.py
fe/site_rpc_interface.py
lient/src/autotest/afe/create/CreateJobViewDisplay.java
lient/src/autotest/afe/create/CreateJobViewPresenter.java
lient/src/autotest/public/AfeClient.html
6dc174e9fc08f7ff6e9cd545bf20607d9a7ceb09 01-Jun-2016 Michael Tang <ntang@google.com> Shows the image when clone a suite job or its child job.

BUG=chromium:485813
TEST=unit test and manual test.

Change-Id: I044991928513cea2c06eb6db8b31eeba6ea29289
Reviewed-on: https://chromium-review.googlesource.com/348482
Commit-Ready: Michael Tang <ntang@chromium.org>
Tested-by: Michael Tang <ntang@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Reviewed-by: Michael Tang <ntang@chromium.org>
fe/rpc_interface.py
fe/rpc_interface_unittest.py
fe/site_rpc_interface.py
lient/src/autotest/afe/AfeClient.java
lient/src/autotest/afe/create/CreateJobViewPresenter.java
lient/src/autotest/afe/create/CreateJobViewTab.java
d19a1df337f79ba28ddc6f9d573d0ffca7363c7b 27-May-2016 Richard Barnette <jrbarnette@google.com> Revert "[autotest] Loop over urllib2 reads of the RPC responses."

This reverts commit 9344c8078ad968e6f062d3ec4849be2f31739b92.

This change is debug code that's no longer needed.

BUG=chromium:606071
TEST=None

Change-Id: I0af49ccf71af11b17cfcbe56be4c069ae582c482
Reviewed-on: https://chromium-review.googlesource.com/347517
Commit-Ready: Richard Barnette <jrbarnette@google.com>
Tested-by: Richard Barnette <jrbarnette@google.com>
Reviewed-by: Simran Basi <sbasi@chromium.org>
fe/json_rpc/proxy.py
7560f33803b73b26555805a66a88822458cf6bc0 21-May-2016 Michael Tang <ntang@google.com> Open wmatrix and moblab links into new tabs.

BUG=chromium:613825
TEST=manually tested.

Change-Id: I34df82a8ed314c35fc337a5e58f4c62716707e42
Reviewed-on: https://chromium-review.googlesource.com/346580
Commit-Ready: Michael Tang <ntang@chromium.org>
Tested-by: Michael Tang <ntang@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Reviewed-by: Michael Tang <ntang@chromium.org>
lient/src/autotest/public/AfeClient.html
e318d50bd515a25047b66c223ef8c20d24692c58 19-May-2016 Richard Barnette <jrbarnette@chromium.org> [autotest] Drop support for specifying kernel from AFE.

The AFE's create_job view included two fields for specifying a
kernel and kernel command line arguments. Neither of those fields
actually worked, and neither is relevant to Chrome OS test jobs.

This deletes the GWT code associated with displaying and processing
those fileds. The underlying RPC features for processing the data
will be removed in subsequent CLs.

BUG=None
TEST=Create a job in a local AFE

Change-Id: Ibe7e1ff8cbc8fcaa311feac27996a18229ec5f15
Reviewed-on: https://chromium-review.googlesource.com/346112
Commit-Ready: Richard Barnette <jrbarnette@chromium.org>
Tested-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Dan Shi <dshi@google.com>
lient/src/autotest/afe/create/CreateJobViewDisplay.java
lient/src/autotest/afe/create/CreateJobViewPresenter.java
e5653cf6ce7fa70458eb2e5ddf4bd784a0240983 14-May-2016 Michael Tang <ntang@google.com> Fixed the moblab setup save.

BUG=chromium:611939
TEST=manual tested

Change-Id: If0f8d93463a1968f2d40e77d1990c61191c3bb43
Reviewed-on: https://chromium-review.googlesource.com/344638
Commit-Ready: Michael Tang <ntang@chromium.org>
Tested-by: Michael Tang <ntang@chromium.org>
Reviewed-by: Stephen Lin <stephenlin@google.com>
Reviewed-by: Michael Tang <ntang@chromium.org>
lient/src/autotest/moblab/ConfigSettingsView.java
54a5b675ec32d9fe5ba5593fc9ef8ceb055c0a61 12-May-2016 Shuqian Zhao <shuqianz@chromium.org> Revert "[autotest] Remove create_parameterized_test() RPC, et seq."

This reverts commit 7983b635dd5c4db5106bf5092174d716a3d1dd63.

Change-Id: I966beaf4b9660d234bd197357da2ebf3b970bfd2
Reviewed-on: https://chromium-review.googlesource.com/344350
Reviewed-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Shuqian Zhao <shuqianz@chromium.org>
Commit-Queue: Shuqian Zhao <shuqianz@chromium.org>
Tested-by: Shuqian Zhao <shuqianz@chromium.org>
fe/model_attributes.py
fe/models.py
fe/models_test.py
fe/rpc_interface.py
fe/rpc_interface_unittest.py
7983b635dd5c4db5106bf5092174d716a3d1dd63 09-May-2016 Richard Barnette <jrbarnette@chromium.org> [autotest] Remove create_parameterized_test() RPC, et seq.

The RPC function create_parameterized_test() is dead, so remove
it. Also delete other code made dead by the deletion.

BUG=None
TEST=utils/unittest_suite.py

Change-Id: I26b77b07f75256d7ca5348a3cc5a17cff3d3db6c
Reviewed-on: https://chromium-review.googlesource.com/343399
Commit-Ready: Richard Barnette <jrbarnette@chromium.org>
Tested-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Xixuan Wu <xixuan@chromium.org>
fe/model_attributes.py
fe/models.py
fe/models_test.py
fe/rpc_interface.py
fe/rpc_interface_unittest.py
9344c8078ad968e6f062d3ec4849be2f31739b92 05-May-2016 Simran Basi <sbasi@google.com> [autotest] Loop over urllib2 reads of the RPC responses.

Recently some json RPC responses are being cut off at 4096 bytes.
In an attempt to solve this problem, we will try to do multiple
reads of the urlfile until no data is returned.

BUG=chromium:606071
TEST=suite support still works in moblab.

Change-Id: Iea04d48599deb21716b23cadff324fd3816118ad
Reviewed-on: https://chromium-review.googlesource.com/342841
Reviewed-by: Simran Basi <sbasi@chromium.org>
Tested-by: Simran Basi <sbasi@chromium.org>
fe/json_rpc/proxy.py
c05c9efcfdd6a720edd0dfeb6f92284e264b303f 25-Mar-2016 Michael Tang <ntang@google.com> Reboot the device after wizard configuration.

BUG=chromium:596536
TEST=Manually test.

Change-Id: I01e6e9fb31a77c0132f92620463d25bf363e0605
Reviewed-on: https://chromium-review.googlesource.com/335332
Commit-Ready: Michael Tang <ntang@chromium.org>
Tested-by: Michael Tang <ntang@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Reviewed-by: Michael Tang <ntang@chromium.org>
fe/site_rpc_interface.py
lient/src/autotest/moblab/rpc/MoblabRpcHelper.java
lient/src/autotest/moblab/wizard/ConfigEditMode.java
lient/src/autotest/moblab/wizard/ConfigWizard.java
32e72883bcfea53b036c3b303254bfd388d1f3b8 23-Apr-2016 Richard Barnette <jrbarnette@chromium.org> [autotest] Delete now dead code from model_logic.py.

One method from a class in frontend.afe.model_logic was made
obsolete by the deletion of the RESTful AFE interface. So,
remove the dead code.

BUG=None
TEST=git grep for the method name

Change-Id: I5d1ac7c5368758a405e2b78b421002fb80844723
Reviewed-on: https://chromium-review.googlesource.com/340449
Commit-Ready: Richard Barnette <jrbarnette@chromium.org>
Tested-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Aviv Keshet <akeshet@chromium.org>
Reviewed-by: Dan Shi <dshi@google.com>
fe/model_logic.py
2b585ac05825a2b879c6bdc3b4a2752ecfc87d3b 23-Apr-2016 Richard Barnette <jrbarnette@chromium.org> [autotest] Delete the unused RESTful AFE&TKO interfaces.

The deleted code was inherited from upstream in an unfinished state,
and is currently unused.

BUG=None
TEST=push to prod suite in a local instance

Change-Id: I74e767dcbf313e67551bedb15a8923edbc6f2e0e
Reviewed-on: https://chromium-review.googlesource.com/340336
Commit-Ready: Richard Barnette <jrbarnette@chromium.org>
Tested-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Aviv Keshet <akeshet@chromium.org>
Reviewed-by: Dan Shi <dshi@chromium.org>
fe/resources.py
fe/resources_test.py
fe/urls.py
ettings.py
hared/__init__.py
hared/common.py
hared/exceptions.py
hared/json_html_formatter.py
hared/query_lib.py
hared/resource_lib.py
hared/resource_test_utils.py
hared/rest_client.py
tatic/jsonview.css
ko/resources.py
ko/resources_test_fixme.py
ko/urls.py
rls.py
bdfc57d71f70af729d94e547752aec1f0c41a8fd 14-Apr-2016 Kevin Cheng <kevcheng@chromium.org> [autotest] Update LabelRetriever to create labels everywhere.

This change does one major thing:
1. Update the rpc_interface host_add_labels() to create labels
if they don't exist on the master first, then to the shards.

With these changes, we should be creating labels on the master AFE
and propagating those changes to the shards properly.

BUG=chromium:603420
TEST=Locally on moblab and on test server with shard, verified label
gets created on test server and shard with same label ids. Also all
unittests pass.

Change-Id: I3fd88ae3cca32702f8b537f673d6db2941e3e7f7
Reviewed-on: https://chromium-review.googlesource.com/338904
Commit-Ready: Kevin Cheng <kevcheng@chromium.org>
Tested-by: Kevin Cheng <kevcheng@chromium.org>
Reviewed-by: Dan Shi <dshi@google.com>
fe/rpc_interface.py
0f7755d1e9ceca25f95ebb01dd05f040f154cba3 18-Apr-2016 xixuan <xixuan@google.com> Autotest: Make autotest call the new devserver API list_suite_controls

Related to CL:338931, this CL is to change the code in autotest to call
the API: list_suite_controls, to get the control contents of all control
files in one call to devserver.

BUG=chromium:602562
TEST=Ran dev_server_unittest, control_file_getter_unittest,
suite_unittest.py, site_rpc_interface_unittest for
ENABLE_CONTROLS_IN_BATCH=False/True;
Ran test_suite:dummy in local autotest instance using local devserver
for ENABLE_CONTROLS_IN_BATCH=False/True.

Change-Id: I075fad07b79827a5d78b7de371bba845fcdf09b1
Reviewed-on: https://chromium-review.googlesource.com/339480
Commit-Ready: Dan Shi <dshi@google.com>
Tested-by: Xixuan Wu <xixuan@chromium.org>
Reviewed-by: Dan Shi <dshi@google.com>
fe/site_rpc_interface.py
340efe3d29035ecdb28c7c758489fc83b08d31fb 16-Apr-2016 Michael Tang <ntang@google.com> Adds a flag to ignore invalid test control files in test image.

BUG=chromium:603302
TEST=Manually test the case on device.

Change-Id: I1d2ac46283f45e2c701a8c569f20b6ffdcf114c1
Reviewed-on: https://chromium-review.googlesource.com/339360
Commit-Ready: Michael Tang <ntang@chromium.org>
Tested-by: Michael Tang <ntang@chromium.org>
Reviewed-by: Michael Tang <ntang@chromium.org>
fe/site_rpc_interface.py
lient/src/autotest/afe/create/CreateJobViewDisplay.java
lient/src/autotest/afe/create/CreateJobViewPresenter.java
lient/src/autotest/public/AfeClient.html
5e8fa1873a5cc8fd4a2129c3ff39c0f37054a0d0 15-Apr-2016 Dan Shi <dshi@google.com> [autotest] Allow create_suite_job RPC to pick a devserver in desired subnet.

This change allows create_suite_job RPC to pick a devserver in the same subnet
of the duts may run tests. The assumption is that duts in the same pool and
board are in the same subnet, which should be true once ACL is enforced.

BUG=chromium:577363
TEST=run in local setup
update global config:
prefer_local_devserver: True
add couple devservers in different subnet
call run_suite, confirm the desired desverver is picked

Change-Id: I3435c13a7c5b25a429609780e0d8471a066e501d
Reviewed-on: https://chromium-review.googlesource.com/339250
Commit-Ready: Dan Shi <dshi@google.com>
Tested-by: Dan Shi <dshi@google.com>
Reviewed-by: Dan Shi <dshi@google.com>
fe/rpc_utils.py
fe/site_rpc_interface.py
6dc22d1af4335067e6f67e2d396082d77010dad1 07-Apr-2016 Dan Shi <dshi@google.com> [autotest] Keep build arg for suite control files that need it.

Some suite control files, e.g., control.au, still uses variable `suite`
to stage image inside the control file. Therefore, the variable still
needs to be injected into the control file.

BUG=chromium:496782
TEST=push to prod test

Change-Id: I73f80f251d6b8ce7307d6fd1178b2ed786e3bd1b
Reviewed-on: https://chromium-review.googlesource.com/337691
Commit-Ready: Dan Shi <dshi@chromium.org>
Tested-by: Dan Shi <dshi@google.com>
Reviewed-by: Richard Barnette <jrbarnette@chromium.org>
fe/site_rpc_interface.py
17660a5d98cfdd8c4ea0fa3d34eb0e77aec8f514 06-Apr-2016 Aviv Keshet <akeshet@chromium.org> Revert "[autotest] limit the size of the get_num_jobs query"

BUG=chromium:599267

This reverts commit b07a09d8fe0705e3f87af42bc6505a7e214f1467.

Change-Id: I8b36c08aab330b079a7316fbc7ef933f71c00523
Reviewed-on: https://chromium-review.googlesource.com/337403
Commit-Ready: Aviv Keshet <akeshet@chromium.org>
Tested-by: Aviv Keshet <akeshet@chromium.org>
Reviewed-by: Dan Shi <dshi@google.com>
fe/rpc_interface.py
5984d78dfcf0071f7de11107c4233e288baee86e 06-Apr-2016 Dan Shi <dshi@google.com> [autotest] Remove build arg in rpc create_suite_job

R45 is not tested any more, it's safe to drop this argument.

BUG=chromium:496782
TEST=unittest, local run_suite

Change-Id: Ifb2ec985dbe0a1d3592ce3eb6637732936133921
Reviewed-on: https://chromium-review.googlesource.com/337333
Commit-Ready: Dan Shi <dshi@google.com>
Tested-by: Dan Shi <dshi@google.com>
Reviewed-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Fang Deng <fdeng@chromium.org>
fe/site_rpc_interface.py
fe/site_rpc_interface_unittest.py
b07a09d8fe0705e3f87af42bc6505a7e214f1467 02-Apr-2016 Aviv Keshet <akeshet@chromium.org> [autotest] limit the size of the get_num_jobs query

get_num_jobs query has the potential to be very expensive, and is
emitted by the web afe. This modification puts a cap on the number of
jobs that will be inspected for the count, and thus a cap on the count.

BUG=chromium:599267
TEST=local autotest

Change-Id: I8e7f3a6711807732a0ac3de93860ee40dac6b53c
Reviewed-on: https://chromium-review.googlesource.com/336807
Commit-Ready: Fang Deng <fdeng@chromium.org>
Tested-by: Fang Deng <fdeng@chromium.org>
Reviewed-by: Aviv Keshet <akeshet@chromium.org>
fe/rpc_interface.py
03cb93fd2444eebe5bd992f773f6d1d73763c7d7 23-Mar-2016 xixuan <xixuan@google.com> Autotest: reboot DUTs when they are moved from shard to master.

A special task REPAIR is triggered to force rebooting on the shard DUTs
when a shard is deleted, in order to make sure the DUTs are still of
ready status and own testing logs in master DB. However, the REPAIR job
won't support reboot in a short time.

This CL triggers a reboot test with highest priority on all DUTs that will
be moved from shard to master. The procedure for deleting a shard is:
1. unlock all related DUTs of this shard.
2. delete any shard information in master DB.
3. trigger a reboot test with highest priority, to make sure that this
test runs firstly after the DUTs are unlocked.
4. unlock these DUTs.

BUG=chromium:499865
TEST=Configer a cbf master and a cbf shard. Set several tasks on master, one
is running, others are pending. Ran 'atest shard delete ***' on the master
to make sure:
* The DUTs beloging to the shard are locked.
* The shard is deleted
* A reboot with highest priority is triggered.
* The DUTs are unlocked.
* The reboot test is ran first, all other pending tasks are queued still.
* After reboot test is finished, other pending tasks will continue
to run on the master.
Ran site_rpc_interface_unittest locally.

Change-Id: I2b348e520c0f67bec5b4b1c89c75ad41e86c72a2
Reviewed-on: https://chromium-review.googlesource.com/334434
Commit-Ready: Xixuan Wu <xixuan@chromium.org>
Tested-by: Xixuan Wu <xixuan@chromium.org>
Reviewed-by: Fang Deng <fdeng@chromium.org>
Reviewed-by: Xixuan Wu <xixuan@chromium.org>
fe/site_rpc_interface.py
fe/site_rpc_interface_unittest.py
a3322c7befae5c20da28ac30098838d8c893840f 30-Mar-2016 Kevin Cheng <kevcheng@chromium.org> [autotest] Fix up AFE to display tables correctly Chrome R50

BUG=chromium:597819
TEST=locally on moblab

Change-Id: Ibc645e2ce287c7a7860adfc2baee3ba838203912
Reviewed-on: https://chromium-review.googlesource.com/335635
Reviewed-by: Aviv Keshet <akeshet@chromium.org>
Tested-by: Aviv Keshet <akeshet@chromium.org>
lient/src/autotest/public/afeclient.css
d27059144bb2f6339f5d5aa6c762574fd94200ca 21-Mar-2016 Michael Tang <ntang@google.com> Integrate the wizard into the Moblab UI.

Also adds updated based on some comments from change 334012.

BUG=chromium:596536
TEST=Manually test.

Change-Id: I70d04718bcec81c326ae9b2aab4550214e795558
Reviewed-on: https://chromium-review.googlesource.com/333993
Commit-Ready: Michael Tang <ntang@chromium.org>
Tested-by: Michael Tang <ntang@chromium.org>
Reviewed-by: Stephen Lin <stephenlin@google.com>
Reviewed-by: Michael Tang <ntang@chromium.org>
lient/src/autotest/common/Utils.java
lient/src/autotest/moblab/ConfigSettingsView.java
lient/src/autotest/moblab/ConfigWizardView.java
lient/src/autotest/moblab/MoblabSetupClient.java
lient/src/autotest/moblab/rpc/CloudStorageInfo.java
lient/src/autotest/moblab/rpc/JsonRpcEntity.java
lient/src/autotest/moblab/rpc/MoblabRpcHelper.java
lient/src/autotest/moblab/rpc/OperationStatus.java
lient/src/autotest/public/MoblabSetupClient.html
159ca506308ec48cf31cfa7dbb908143ed58f071 21-Mar-2016 Michael Tang <ntang@google.com> Moblab configuration wizard UI widget.

BUG=chromium:596536
TEST=Unit Tests.

Change-Id: I5d31fd30876b32d40980bbce462764ef26da5520
Reviewed-on: https://chromium-review.googlesource.com/334012
Commit-Ready: Michael Tang <ntang@chromium.org>
Tested-by: Michael Tang <ntang@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Reviewed-by: Michael Tang <ntang@chromium.org>
Reviewed-by: Dan Shi <dshi@google.com>
lient/src/autotest/common/Utils.java
lient/src/autotest/moblab/rpc/CloudStorageInfo.java
lient/src/autotest/moblab/rpc/JsonRpcEntity.java
lient/src/autotest/moblab/rpc/MoblabRpcCallbacks.java
lient/src/autotest/moblab/rpc/MoblabRpcHelper.java
lient/src/autotest/moblab/rpc/NetworkInfo.java
lient/src/autotest/moblab/rpc/OperationStatus.java
lient/src/autotest/moblab/wizard/CloudStorageCard.java
lient/src/autotest/moblab/wizard/ConfigEditMode.java
lient/src/autotest/moblab/wizard/ConfigViewMode.java
lient/src/autotest/moblab/wizard/ConfigWizard.java
lient/src/autotest/moblab/wizard/FlexWizardCard.java
lient/src/autotest/moblab/wizard/NetworkInfoCard.java
lient/src/autotest/moblab/wizard/WizardCard.java
lient/src/autotest/public/afeclient.css
lient/test/autotest/moblab/MoblabSetupClientTest.gwt.xml
lient/test/autotest/moblab/MoblabTest.java
lient/test/autotest/moblab/rpc/CloudStorageInfoTest.java
lient/test/autotest/moblab/rpc/NetworkInfoTest.java
lient/test/autotest/moblab/wizard/ConfigWizardTest.java
9afc74b7f9f49a1de4f714aff88f53bebcac0f4b 21-Mar-2016 Michael Tang <ntang@google.com> Backend RPC handlers for Moblab OOBE related calls.

BUG=chromium:596536

TEST=Unit tested.

Change-Id: Iaa113708138f6a3fe88f44057d66a8faf240b04a
Reviewed-on: https://chromium-review.googlesource.com/334191
Commit-Ready: Michael Tang <ntang@chromium.org>
Tested-by: Michael Tang <ntang@chromium.org>
Reviewed-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Michael Tang <ntang@chromium.org>
fe/site_rpc_interface.py
fe/site_rpc_interface_unittest.py
6450e148ba84c878490b4ea166218430fd393ddd 11-Mar-2016 Dan Shi <dshi@google.com> [autotest] Support run_suite with suite package and SSP for Brillo.

This change allows one to run run_suite command without using
--run_prod_code for Brillo build.
create_suite_job will use test_suites and control_files packages from
the Brillo build to create suite job and its test jobs.

Server-side packaging is also supported for newer builds that have
autotest_server_package artifact build.

BUG=chromium:584705
TEST=run in local instance, unittest, verify in moblab

Change-Id: Ia96ca4de919b178302580c23f911bb6445016285
Reviewed-on: https://chromium-review.googlesource.com/332431
Commit-Ready: Dan Shi <dshi@google.com>
Tested-by: Dan Shi <dshi@google.com>
Reviewed-by: Dan Shi <dshi@google.com>
fe/site_rpc_interface.py
fe/site_rpc_interface_unittest.py
275169775cab95fd5081cb3a0e9e832f5eb6c7c4 16-Mar-2016 Dan Shi <dshi@google.com> [autotest] Hide ab-version and testbed-version labels from the hosts filter

BUG=None
TEST=local instance

Change-Id: Iec350cbcb50dae3af7fdc093bb5fe9f05427fb1f
Reviewed-on: https://chromium-review.googlesource.com/333102
Commit-Ready: Dan Shi <dshi@google.com>
Tested-by: Dan Shi <dshi@google.com>
Reviewed-by: Kevin Cheng <kevcheng@chromium.org>
fe/rpc_interface.py
f770c456af6aec782e5fe2bdfe2b253a5b901f86 04-Mar-2016 Dan Shi <dshi@google.com> [autotest] Allow suite scheduler to set time out for RPC.

When an RPC server is under heavy load, long RPC calls like
create_suite_job might time out. However the call might still
successfully create suite job even though the caller received an
socket.timeout exception and retry the call. That leads to multiple
suites be created.

This change allows the create_suite_job call to pass in a minimum value
of timeout, to reduce the flake of the RPC. The same change is applied
to get_jobs and get_hostnames call, which may take longer than the
default 6s timeout.

BUG=chromium:591538
TEST=local suite schedule run, unittest, local AFE

Change-Id: If7b888fe6ca80f5c7e705026e06b883cef0bfdc4
Reviewed-on: https://chromium-review.googlesource.com/330463
Commit-Ready: Dan Shi <dshi@chromium.org>
Tested-by: Dan Shi <dshi@google.com>
Reviewed-by: Simran Basi <sbasi@chromium.org>
fe/json_rpc/proxy.py
b49bb8bf984bd1859b149ee48963ad01952975da 02-Mar-2016 Dan Shi <dshi@google.com> [autotest] Support firmware_ro_build_spec in suite scheduler.

This change allows user to specify firmware_ro_build_spec in suite
scheduler ini file. Suite scheduler will create suite jobs based on the
given RO firmware build spec.

BUG=chromium:575394
TEST=local run
/usr/local/autotest/site_utils/suite_scheduler/suite_scheduler.py \
-d /usr/local/autotest/logs -f /usr/local/autotest/ss_test.ini \
-e nightly -i veyron_jerry-release/R49-7834.3.0 \
-r /tmp/_autotmp_0pjXWQ_suite_scheduler

test task:
[FaftEcNightly]
run_on: nightly
suite: dummy
branch_specs: ==tot-2
firmware_rw_build_spec: firmware
firmware_ro_build_spec: firmware
test_source: cros

Change-Id: Ib453ccee6c95a10570c692f0280513c8d78e3173
Reviewed-on: https://chromium-review.googlesource.com/329931
Commit-Ready: Dan Shi <dshi@google.com>
Tested-by: Dan Shi <dshi@google.com>
Reviewed-by: Dan Shi <dshi@google.com>
fe/site_rpc_interface.py
2121a3382d70bfb60284a830f341d78d6d555519 25-Feb-2016 Dan Shi <dshi@google.com> [autotest] Support scheduling Launch Control builds from suite scheduler.

This change adds the support of scheduling Launch Control builds from
suite scheduler.

For task defined like:
[brollo_test]
run_on: nightly
hour: 17
suite: dummy
os_type: brillo
branches: git_mnc-brillo-dev
targets: dragonboard-userdebug
pool: suites

Suite scheduler will try to schedule the latest build as of
git_mnc-brillo-dev/dragonboard-userdebug/LATEST
for dragonbard daily at 5PM, to run a duts in suites pool.

BUG=chromium:585628
TEST=unittest, local suite scheduler run:
/usr/local/autotest/site_utils/suite_scheduler/suite_scheduler.py \
-d /usr/local/autotest/logs -f /usr/local/autotest/ss_test.ini \
-e nightly -i git_mnc-brillo-dev/dragonboard-userdebug/2636090 \
-r /tmp/_autotmp_0pjXWQ_suite_scheduler -o brillo

Also test existing cros build can be scheduled:
/usr/local/autotest/site_utils/suite_scheduler/suite_scheduler.py \
-d /usr/local/autotest/logs -f /usr/local/autotest/ss_test.ini \
-e nightly -i veyron_jerry-release/R49-7834.3.0 \
-r /tmp/_autotmp_0pjXWQ_suite_scheduler

Change-Id: I6dc8ecb95ff0d353a1e8a96f98183945216f9a50
Reviewed-on: https://chromium-review.googlesource.com/329362
Commit-Ready: Dan Shi <dshi@google.com>
Tested-by: Dan Shi <dshi@google.com>
Reviewed-by: Dan Shi <dshi@google.com>
fe/site_rpc_interface.py
059261aaf4455fa6ab5e9d4369bb0650143ee467 22-Feb-2016 Dan Shi <dshi@google.com> [auotest] Add a new arg delay_minutes to delay suite job creating test jobs.

This allow provision jobs to be delayed after a suite job is created. It
helps the suite scheduler to distribute devserver load over the time.
Otherwise, suite jobs created together will kick off provision jobs
together and add a peak load on devservers.

Next change will be on suite_scheduler to add delay like 5 minutes apart
for suite jobs created in a single event handling, e.g., nightly.

BUG=chromium:582353
TEST=local run, unittest
./site_utils/run_suite.py -b veyron_jerry -i \
veyron_jerry-release/R50-7871.0.0 -s dummy --delay_minutes 5

Change-Id: I6aa9404dc33ba2f6b90556d6f783038d86aa402b
Reviewed-on: https://chromium-review.googlesource.com/328784
Commit-Ready: Dan Shi <dshi@google.com>
Tested-by: Dan Shi <dshi@google.com>
Reviewed-by: Dan Shi <dshi@google.com>
fe/site_rpc_interface.py
f5adc7c0ab9a055904cdd203c548dadb517a8c33 23-Feb-2016 Shuqian Zhao <shuqianz@chromium.org> [autotest] Temporarily disable the foreign key check when setup database

Recently mysql-server package has been upgraded from 5.5 to 5.6. The
foreign key checking is more strict in 5.6. As of 5.6.7, the server
prohibits changes to foreign key columns with the potential to cause
loss of referential integrity. In autotest database setup, it fails when
altering foreign key column. Temporarily disable the foreign key check
when we need to alter foreign key columns.

BUG=chromium:588954
TEST=Test on a testing instance to run the database setup script.

Change-Id: Iab9a0af9de2b1c2da18bd57c6e4710a106c8c96f
Reviewed-on: https://chromium-review.googlesource.com/329130
Commit-Ready: Shuqian Zhao <shuqianz@chromium.org>
Tested-by: Shuqian Zhao <shuqianz@chromium.org>
Reviewed-by: Dan Shi <dshi@google.com>
igrations/058_drone_management.py
8b0d65e134832b50b350c64c90f94db0b3dd11fc 12-Feb-2016 Michael Tang <ntang@google.com> Adds links to Google Cloud Storage for a few configuration parameters.

Remove white spaces at end of lines.

BUG=chromium:585140
TEST=Manually tested by pushing to moblab devices.

Change-Id: I46f09d4be824b40887a264c8eaad041c219f3cbc
Reviewed-on: https://chromium-review.googlesource.com/327374
Commit-Ready: Michael Tang <ntang@chromium.org>
Tested-by: Michael Tang <ntang@chromium.org>
Reviewed-by: Michael Tang <ntang@chromium.org>
Reviewed-by: Dan Shi <dshi@google.com>
lient/src/autotest/moblab/ConfigSettingsView.java
86fe9c9e667bc9b9d78040b4c3de2f60aac7d34c 10-Feb-2016 Simran Basi <sbasi@google.com> [autotest] Fix naming for test suites with --run_prod_code

BUG=chromium:585625
TEST=moblab suite run

Change-Id: I8e3b7e31d0d007008d727d41006803fb20c3d45f
Reviewed-on: https://chromium-review.googlesource.com/326893
Commit-Ready: Simran Basi <sbasi@chromium.org>
Tested-by: Simran Basi <sbasi@chromium.org>
Reviewed-by: Dan Shi <dshi@chromium.org>
fe/site_rpc_interface.py
beb2bb20028027fa31c2790e54619c82f4d51422 04-Feb-2016 Simran Basi <sbasi@google.com> [autotest] Support adb_host repair

Implements a basic repair function for adb_host which
runs provision_AndroidUpdate if the device is reachable
via fastboot. The repair image will be retrieved from
global_config or the DB if it is set.

BUG=b:26879108
TEST=local moblab repair.

Change-Id: I26a962c1bf458a8b02ad66efbc2c2bda99530320
Reviewed-on: https://chromium-review.googlesource.com/325519
Commit-Ready: Simran Basi <sbasi@chromium.org>
Tested-by: Simran Basi <sbasi@chromium.org>
Reviewed-by: Kevin Cheng <kevcheng@chromium.org>
fe/site_rpc_interface.py
5ace6f2e17f808efbee863a7d7dbe1bc52527a19 07-Jan-2016 Simran Basi <sbasi@google.com> [autotest] Support Android/Brillo suite runs.

This change does 3 key things:

1) Enables suite runs to use the production code currently
deployed on the lab servers. This disables this requirement
and now run_suite.py can be passed --run_prod_code. This means
that it will use the control files on the drone and SSP has
been disabled.
2) Refactors out the AFE interactions for applying version labels
out of the host objects. These utilities are in a new module
called afe_utils. The afe_utils:host_in_lab function
determines if a host is running in an lab environment and if
it is in the AFE.
The introduction of afe_utils enables further cleanup of the
cros_host object.
3) Enables provisioning for Android/Brillo devices. Added a new
site-test and provision attribute. Run_suite will determine
if an Android/Brillo build is being used and will set the
correct version prefix. Then the scheduler/provision code
will find an appropriate device and kick off the
provision_AndroidUpdate test to install the build prior to
test runs.

BUG=chromium:574566
TEST=On my moblab:
./run_suite.py --build=git_mnc-brillo-dev/dragonboard-userdebug/2558576 \
--board=brillo-dragonboard --suite_name=brillo-bvt --run_prod_code \
--pool=''
./site_utils/run_suite.py --board=peppy --build=peppy-release/LATEST \
--suite_name=dummy --pool=''
test_that 100.96.51.40 dummy_PassServer
DEPLOY=apache

Change-Id: I5a81e6bd989188b7c0621e2a76752a39033f9782
Reviewed-on: https://chromium-review.googlesource.com/323781
Commit-Ready: Simran Basi <sbasi@chromium.org>
Tested-by: Simran Basi <sbasi@chromium.org>
Reviewed-by: Christopher Wiley <wiley@chromium.org>
Reviewed-by: Fang Deng <fdeng@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
fe/site_rpc_interface.py
4c0d2904152d2c8f3fc748f53cd9fd26f6aeab5d 13-Jan-2016 Shuqian Zhao <shuqianz@chromium.org> [autotest] Forcefully modify host locking status via cli

When host locking status is inconsistent between masterDB and shardDB for some
unknown rpc reasons, we may need to forcefully lock the host both in
masterDB and shardDB gaurantee the lab reliability.

BUG=chromium:568193
TEST=Test locally. Test in puppetlab to lock an already locked host and
unlock an already unlocked host.

Change-Id: I83f53da0a2e76dedfc8e58deb609073efec5f369
Reviewed-on: https://chromium-review.googlesource.com/321792
Commit-Ready: Shuqian Zhao <shuqianz@chromium.org>
Tested-by: Shuqian Zhao <shuqianz@chromium.org>
Reviewed-by: Shuqian Zhao <shuqianz@chromium.org>
fe/doctests/001_rpc_test.txt
fe/rpc_interface.py
fe/rpc_utils.py
8e2c2d0cbfb7ac2f1226a59cbbeaffb00288b8df 07-Jan-2016 MK Ryu <mkryu@google.com> [autotest] Change DB update order.

Originally, we update shard DBs first and then the master DB, and
use the shard DB as ground truth when inconsistency between the
master DB and a shard DB is detected. This incurs a tricky bug
as explained in crbug.com/574966.

We should change the order. Update the master DB first and then
shard DBs, and use the master DB as ground truth for fixing
DB inconsistency.

BUG=chromium:574966
TEST=Test updated RPCs in puppylab.

Change-Id: Ic785f9f3c677bea997d4637266c8c223618070d9
Reviewed-on: https://chromium-review.googlesource.com/320599
Commit-Ready: Mungyung Ryu <mkryu@google.com>
Tested-by: Mungyung Ryu <mkryu@google.com>
Reviewed-by: Dan Shi <dshi@chromium.org>
fe/rpc_interface.py
fe/rpc_interface_unittest.py
fe/rpc_utils.py
aec9901aa25aecfc58d8e1c9b867c624109d7fe2 07-Jan-2016 Dan Shi <dshi@google.com> [autotest] Support uploading a service key file to moblab

Add RPC to accept an uploaded key file and save it to
/home/moblab/.launch_control_key
Add a new tab in moblab setup page to allow user to upload a service key file.

BUG=chromium:570705
TEST=verify with a local moblab build

Change-Id: Ib54a6b707cb93bb5220e29ed6feeb854f5f4abe8
Reviewed-on: https://chromium-review.googlesource.com/321140
Commit-Ready: Dan Shi <dshi@google.com>
Tested-by: Dan Shi <dshi@google.com>
Reviewed-by: Simran Basi <sbasi@chromium.org>
fe/site_rpc_interface.py
fe/site_rpc_interface_unittest.py
lient/src/autotest/moblab/BotoKeyView.java
lient/src/autotest/moblab/KeyUploadView.java
lient/src/autotest/moblab/LaunchControlKeyView.java
lient/src/autotest/moblab/MoblabSetupClient.java
lient/src/autotest/public/MoblabSetupClient.html
9316171a2181b950cad906b8bb9a3ebc2d9a7840 21-Dec-2015 MK Ryu <mkryu@google.com> [autotest] Bug fix for modify_hosts RPC.

'update_data' is an argument of 'modify_hosts', and it is updated within
'modify_hosts'. That means the original variable passed as 'update_data'
is updated, and it seems to cause an error.
To be safe, it's better to make a copy for 'update_data' and updates the
copied one.

BUG=chromium:535651
TEST=Try lock/unlock multiple hosts.

Change-Id: I65e62158df55ff0b9d6d942e1ba9ab11730413dc
Reviewed-on: https://chromium-review.googlesource.com/319189
Reviewed-by: Dan Shi <dshi@chromium.org>
Tested-by: Mungyung Ryu <mkryu@google.com>
fe/rpc_interface.py
d53e1492e481c3e9d07ba790577914ba20d7631a 15-Dec-2015 MK Ryu <mkryu@google.com> [autotest] Consistent lock_time for a host

In the old code, 'lock_time' field for a host is recorded
when django saves the modified host record. It can sometimes
generate a slight lag (a couple seconds) for lock_time
between the master and a shard DB. This CL makes it always
consistent.

BUG=chromium:535651
TEST=Puppylab. Try locking a sharded host with a test code that
delays the message (modify_host_local) forwarding to a shard.

Change-Id: I33885f659c7511ae136493228380b459dae001d5
Reviewed-on: https://chromium-review.googlesource.com/318387
Commit-Ready: Mungyung Ryu <mkryu@google.com>
Tested-by: Mungyung Ryu <mkryu@google.com>
Reviewed-by: Mungyung Ryu <mkryu@google.com>
fe/models.py
fe/rpc_interface.py
fe/rpc_interface_unittest.py
37df54d11d7598f6b7192b5a397d6e9ca05bf8e3 14-Dec-2015 Dan Shi <dshi@google.com> [autotest] Add a currently running job/task of a host on cautotest AFE

Add a new argument include_current_job in get_hosts RPC. If it's set to True,
include the currently running job/task info in the host details. It's default
to False, since the call is used for other callers, and the extra db query will
add unnecessary load to the caller, e.g., shard heartbeat.

BUG=chromium:549193
TEST=local verify
host with no running job/task: http://dshi.mtv/afe/#tab_id=view_host&object_id=1
host with running job/task: http://dshi.mtv/afe/#tab_id=view_host&object_id=8

DEPLOY=afe, apache

Change-Id: Ifc86db3486efafe9a2f3f17f00ea0640f3f1e0ba
Reviewed-on: https://chromium-review.googlesource.com/318260
Commit-Ready: Dan Shi <dshi@google.com>
Tested-by: Dan Shi <dshi@google.com>
Reviewed-by: Fang Deng <fdeng@chromium.org>
fe/rpc_interface.py
lient/src/autotest/afe/HostDetailView.java
lient/src/autotest/common/table/RpcDataSource.java
lient/src/autotest/public/AfeClient.html
0cb2a3b1d2d86d70da06a3f45be9297139e48207 11-Dec-2015 Fang Deng <fdeng@chromium.org> [autotest] Move get_global_afe_hostname to server/site_utils

Move get_global_afe_hostname to server/site_utils so that
other code can import it without django env setup hassle.

TEST=Run unittest, run shard_client
BUG=chromium:568873

Change-Id: I574d3dc00e2f79af0283487ec9cac6fb8bfd9fd4
Reviewed-on: https://chromium-review.googlesource.com/317276
Commit-Ready: Fang Deng <fdeng@chromium.org>
Tested-by: Fang Deng <fdeng@chromium.org>
Reviewed-by: Dan Shi <dshi@chromium.org>
fe/rpc_interface_unittest.py
fe/rpc_utils.py
6157e8e83719c618d48b7a36bbc9f80c39c381f5 08-Dec-2015 Simran Basi <sbasi@google.com> [autotest] Support using *-LATEST builds for single job runs.

We support build names like peppy-release/LATEST for suites but not
single jobs from the AFE. This CL introduces this support for single
job runs.

BUG=chromium:431060
TEST=moblab.
DEPLOY=apache

Change-Id: I86ce34a79508d34bacbcfa4de655f13d1f14648c
Reviewed-on: https://chromium-review.googlesource.com/316683
Commit-Ready: Simran Basi <sbasi@chromium.org>
Tested-by: Simran Basi <sbasi@chromium.org>
Reviewed-by: Dan Shi <dshi@google.com>
fe/rpc_interface.py
c6645c5789a6c8e15290ccba0d643b7b8ad173d6 18-Nov-2015 Dan Shi <dshi@google.com> [autotest] Increase hostname input box's width for autotest web frontend

BUG=chromium:557468
TEST=local afe
DEPLOY=afe

Change-Id: Ieb3ce566effde13d119ca5cb9f7e83e42e258de7
Reviewed-on: https://chromium-review.googlesource.com/312989
Commit-Ready: Dan Shi <dshi@google.com>
Tested-by: Dan Shi <dshi@chromium.org>
Reviewed-by: danny chan <dchan@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
lient/src/autotest/afe/HostDetailView.java
2c9af7ba03810b2b9b32a3d23218d42acbdea38d 23-Oct-2015 MK Ryu <mkryu@google.com> [autotest] Bug fix for route_rpc_to_master decorator.

frontend.afe.rpc_interface:create_job() has a parameter, "args",
and the parameters was considered to be "**kwargs" in the
frontend.afe.rpc_utils:route_rpc_to_master decorator.

BUG=chromium:545572
TEST=frontend.afe.rpc_interface_unittest,
frontend.afe.site_rpc_interface_unittest,
Create a job in the local AFE.

Change-Id: I5e1b0091c2cdd20bd1820faa99fc1e04bf9f9ec4
Reviewed-on: https://chromium-review.googlesource.com/308265
Commit-Ready: Mungyung Ryu <mkryu@google.com>
Tested-by: Mungyung Ryu <mkryu@google.com>
Reviewed-by: Simran Basi <sbasi@chromium.org>
fe/rpc_utils.py
7051fe4264ccfe54de963cbd89339e2348975b91 20-Oct-2015 Fang Deng <fdeng@chromium.org> [autotest] Handle race condition on label creation.

If two suite runs in parallel we may run into a race condition when
trying to create the cros-version label. This CL handles the race
condition.

BUG=chromium:542928
TEST=Test locally with an autotest instance. Force the code to exeucte.

Change-Id: If59cd00e66525203e1d954fbdea99476c060e05c
Reviewed-on: https://chromium-review.googlesource.com/307590
Commit-Ready: Fang Deng <fdeng@chromium.org>
Tested-by: Fang Deng <fdeng@chromium.org>
Reviewed-by: Paul Hobbs <phobbs@google.com>
fe/rpc_utils.py
7c8f23e9ac9b8895dbdb70b4fa6325ddb1ef04c5 15-Oct-2015 Ilja H. Friedel <ihf@chromium.org> Delete graphics labels.

Graphics tests will not rely on labels anymore. The code is brittle,
autotest codebase is not available on DUT during label generation (407951).
DEPENDENCIES have not enough features (348189). Instead graphics tests
will run everywhere and decide how much testing they are going to do on
that particular DUT.

BUG=chromium:348189, chromium:407951
TEST=grep.

Change-Id: I97d5561ba9314b733fec53195b981e3e43b7b0bd
Reviewed-on: https://chromium-review.googlesource.com/305841
Commit-Ready: Ilja Friedel <ihf@chromium.org>
Tested-by: Ilja Friedel <ihf@chromium.org>
Reviewed-by: Stéphane Marchesin <marcheu@chromium.org>
Reviewed-by: Haixia Shi <hshi@chromium.org>
fe/models_test.py
01984f506453723cb63b2d39947017c62ba06047 13-Oct-2015 Simran Basi <sbasi@google.com> Brillo Tier 1 PTS: Add job monitoring and result processing.

Now after the PTS sequence is launched, the tier1 kickoff script
will monitor the PTS job and its child jobs until either they
complete or the host goes into a bad state.

Then the results are processed via the same ResultCollector
we use for running suites.

Also fixed the get_jobs RPC to work properly when finished=False,
before it ignored this value when set to false.

BUG=b:24774860
TEST=Ran against a local moblab. PTS is kicked off, monitored and
results are properly fetched. Links outputted by the ResultCollector
also work as expected.

Change-Id: Ie179b54aa5b380fdd78b803e8e4c922003805759
Reviewed-on: https://chromium-review.googlesource.com/305263
Commit-Ready: Simran Basi <sbasi@chromium.org>
Tested-by: Simran Basi <sbasi@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
fe/rpc_interface.py
fe/rpc_utils.py
53cc141a0b2ba102f773127184c67dcdf3dce523 08-Oct-2015 MK Ryu <mkryu@google.com> [autotest] Make explicit that lock_reason of AbstractHostModel can be NULL.

lock_reason of AbstractHostModel didn't specify null property explicitly.
The effect of it is MySQL has NULL property for lock_reason column, but
Sqlite doesn't have NULL property for the column.
simple_heartbeat_client.py uses in-memory sqlite, and it produces error
like below.

django.db.utils.IntegrityError: NOT NULL constraint failed: afe_hosts.lock_reason

This means a host entry got from prod database has NULL lock_reason,
and it violates integrity for sqlite DB.
We should make it explicit that lock_reason column can be NULL.

BUG=None
TEST=Run simple_heartbeat_client.py in local machine.

Change-Id: Ic2bdb35cff17cf8c1789db78c7166a404610827b
Reviewed-on: https://chromium-review.googlesource.com/304577
Commit-Ready: Mungyung Ryu <mkryu@google.com>
Tested-by: Mungyung Ryu <mkryu@google.com>
Reviewed-by: Matthew Sartori <msartori@chromium.org>
Reviewed-by: Dan Shi <dshi@chromium.org>
fe/rdb_model_extensions.py
08cfdcd888a34f7bd0145e88ad7151a5e1e619c6 24-Sep-2015 Simran Basi <sbasi@google.com> autotest: Fix Create Job Test Source Build tooltip text.

BUG=None
TEST=None

Change-Id: I1e03214f165098c6943cc2e6d6586aee426996b0
Reviewed-on: https://chromium-review.googlesource.com/302157
Commit-Ready: Simran Basi <sbasi@chromium.org>
Tested-by: Simran Basi <sbasi@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
lient/src/autotest/afe/create/CreateJobViewDisplay.java
e1604a4a63be13492c1aaecab144799722837cee 24-Sep-2015 Dan Shi <dshi@chromium.org> [autotest] Update tooltip for "Require server-side packaging"

BUG=None
TEST=local afe

Change-Id: I71c11041148088af70928c8286e96e4fa921df62
Reviewed-on: https://chromium-review.googlesource.com/302401
Commit-Ready: Dan Shi <dshi@chromium.org>
Tested-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
lient/src/autotest/afe/create/CreateJobViewDisplay.java
0a9c82ef2af5e8a12921b14ea35aa4d0e146332b 18-Sep-2015 MK Ryu <mkryu@google.com> [autotest] Forward user when forwaring/routing RPCs.

User account information is passed to RPC server via
HTTP header. We can pass the user info to the header when
creating server.frontend.RpcClient object. We should forward
the user info when forwaring/routing RPCs not to lose the
user account info in the receiving side.

BUG=chromium:533159
TEST=puppylab. Lock a sharded host with root account and check
the host is locked by root in the master DB and the shard DB.

Change-Id: I185e49f028c8fdfdf630855412a6e370d5684491
Reviewed-on: https://chromium-review.googlesource.com/300594
Commit-Ready: Mungyung Ryu <mkryu@google.com>
Tested-by: Mungyung Ryu <mkryu@google.com>
Reviewed-by: Fang Deng <fdeng@chromium.org>
Reviewed-by: Dan Shi <dshi@chromium.org>
fe/rpc_interface_unittest.py
fe/rpc_utils.py
fdee1399893c2e84d7608a35d26da8535c7fe643 16-Sep-2015 Matthew Sartori <msartori@google.com> autotest: Mob* Monitor link in AFE.

This CL adds a link in the AFE to the Mob* Monitor.

BUG=None
TEST=Tested the link on a moblab and from different machines.

Change-Id: I089341504bccbff7aac9c33c31a652a5bdffeea2
Reviewed-on: https://chromium-review.googlesource.com/300100
Commit-Ready: Matthew Sartori <msartori@chromium.org>
Tested-by: Matthew Sartori <msartori@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
lient/src/autotest/afe/AfeClient.java
lient/src/autotest/public/AfeClient.html
3388961364ed423bb3f7e9addcf83283e9e8b2ef 04-Sep-2015 MK Ryu <mkryu@google.com> [autotest] Route modify_host RPC to master when called on a shard.

Users mistakenly lock/unlock on a shard AFE page.
It incurs database inconsistency between master DB and the shard DB.
This CL makes modify_host / modify_hosts RPCs routed to the master
if they are called on a shard.

BUG=chromium:444600
TEST=Puppylab and unittest.

Change-Id: I97d0cfcd85fe8a399224028a4c5fdbe32910fb88
Reviewed-on: https://chromium-review.googlesource.com/297595
Commit-Ready: Mungyung Ryu <mkryu@google.com>
Tested-by: Mungyung Ryu <mkryu@google.com>
Reviewed-by: Fang Deng <fdeng@chromium.org>
fe/rpc_interface.py
fe/rpc_interface_unittest.py
6f5eadb58a7c29aab578764e2f3887f320d20194 04-Sep-2015 MK Ryu <mkryu@google.com> [autotest] Bug fix for route_rpc_to_master decorator

Detail is in the comment in code.

BUG=chromium:444600
TEST=rpc_interface_unittest, site_rpc_interface_unittest.

Change-Id: If64632504f2c7826a451004378b2d4bd995126f1
Reviewed-on: https://chromium-review.googlesource.com/297591
Commit-Ready: Mungyung Ryu <mkryu@google.com>
Tested-by: Mungyung Ryu <mkryu@google.com>
Reviewed-by: Mungyung Ryu <mkryu@google.com>
fe/rpc_utils.py
3b1134936931ffcf964d37c15484793f1f464aec 29-Aug-2015 Dan Shi <dshi@chromium.org> [autotest] Allow to clone a job created by suite job to run with SSP.

Job created by suite job does not have image associated, instead, it relies on
label dependency to force image installation. The clone of such job doesn't
have image attribute either, thus the clone can't run with SSP.

This CL adds some check on label dependency. If the job has a cros-version
label, allow the clone to run with SSP.

BUG=chromium:526279
TEST=local

DEPLOY=afe

Change-Id: I9a96871ef828dece4126d2933933cae356233f85
Reviewed-on: https://chromium-review.googlesource.com/296080
Commit-Ready: Dan Shi <dshi@chromium.org>
Tested-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Matthew Sartori <msartori@chromium.org>
lient/src/autotest/afe/create/CreateJobViewPresenter.java
b1dc824f0b75908f0f051fcdef559e9c0ec21c5d 27-Aug-2015 MK Ryu <mkryu@google.com> [autotest] Change to read global_afe_hostname from different config key.

global_afe_hostname now is in [SERVER] section instead of [SHARD].

BUG=chromium:524678
TEST=None

Change-Id: Ic10e100925e99e2b062fde618e6489664322c201
Reviewed-on: https://chromium-review.googlesource.com/295687
Commit-Ready: Mungyung Ryu <mkryu@google.com>
Tested-by: Mungyung Ryu <mkryu@google.com>
Reviewed-by: Dan Shi <dshi@chromium.org>
fe/rpc_utils.py
a34e3b187b13e5b128916675bcae5fe1757dd5f2 22-Aug-2015 MK Ryu <mkryu@google.com> [autotest] Add an option for label_cleaner

When -s option is enabled, label_cleaner checks if the server
where the tool is running on is 'primary' server.
If not, it just returns.

BUG=chromium:380365
TEST=Run the tool with/without -s option in local machine.
DEPLOY=apache

Change-Id: I0eb2c4463f94e51a96c81d3c02baae41fca025cd
Reviewed-on: https://chromium-review.googlesource.com/294878
Reviewed-by: Mungyung Ryu <mkryu@google.com>
Commit-Queue: Mungyung Ryu <mkryu@google.com>
Tested-by: Mungyung Ryu <mkryu@google.com>
fe/site_rpc_interface.py
e0b08e6170b57f90262726eb7f04e059cb47419c 11-Aug-2015 Hsinyu Chao <hychao@chromium.org> Revert "autotest: move constants to common_lib"

This reverts commit a4ac7a8f67d61f8a616724f9426dbc0fce71ce69.

BUG=chromium:519353

Change-Id: Iec2873ad22b5bb90e46a9e5b323f9626a2c07d9b
Reviewed-on: https://chromium-review.googlesource.com/292347
Reviewed-by: Hsinyu Chao <hychao@chromium.org>
Commit-Queue: Hsinyu Chao <hychao@chromium.org>
Tested-by: Hsinyu Chao <hychao@chromium.org>
Reviewed-by: Sonny Rao <sonnyrao@chromium.org>
Commit-Queue: Sonny Rao <sonnyrao@chromium.org>
Tested-by: Sonny Rao <sonnyrao@chromium.org>
pache_auth.py
a4ac7a8f67d61f8a616724f9426dbc0fce71ce69 06-Aug-2015 Eric Caruso <ejcaruso@chromium.org> autotest: move constants to common_lib

This file gets imported from lots of server-side tests and
infrastructure so it shouldn't be hiding in client/cros. This
moves the file and fixes all imports.

BUG=chromium:485834
TEST=run many tests, make sure there are no problems importing
constants

Change-Id: I5c2b2a0bef6ebc998e62f40162a6ec9cc4436480
Reviewed-on: https://chromium-review.googlesource.com/291302
Trybot-Ready: Eric Caruso <ejcaruso@chromium.org>
Commit-Queue: Eric Caruso <ejcaruso@chromium.org>
Tested-by: Eric Caruso <ejcaruso@chromium.org>
Reviewed-by: Chris Sosa <sosa@google.com>
pache_auth.py
65351d6f17ba32b942a18c42ca149dd154eebce7 03-Aug-2015 Dan Shi <dshi@chromium.org> [autotest] Hide firmware labels from AFE

fwrw-version and fwro-version are the version label just like cros-version,
they don't need to be listed in Host List tab's label listbox.

BUG=None
TEST=local afe

Change-Id: Icb98d432df11bb1baced12a1763c3efd2aa2b0c0
Reviewed-on: https://chromium-review.googlesource.com/290351
Trybot-Ready: Dan Shi <dshi@chromium.org>
Tested-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
fe/rpc_interface.py
5aa2504a6947eda0943689ca55ed6bd4dce8444f 29-Jul-2015 MK Ryu <mkryu@google.com> [autotest] Support repair_hosts RPC in sharding

BUG=chromium:482192
TEST=Use testing master and shard machines.
Run Reverify and Repair tasks on both master DUTs
and shard DUTs via frontend web ui.
DEPLOY=apache

Change-Id: Iac88eb72e0fcc8993a6bccbf53a91575ce8623b2
Reviewed-on: https://chromium-review.googlesource.com/289304
Trybot-Ready: Mungyung Ryu <mkryu@google.com>
Tested-by: Mungyung Ryu <mkryu@google.com>
Reviewed-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Mungyung Ryu <mkryu@google.com>
fe/rpc_interface.py
70647cafbd061a7754ac304fd9dc067f2b6dbab4 17-Jul-2015 Dan Shi <dshi@chromium.org> [autotest] Save parent job id, build, board and suite info to tko_jobs.

parent job id is passed in through autoserv commandline. autoserv saves the
value to keyval file in results folder. The parser job then reads the parent
job id from keyval file.

build, board and suite info are parsed from job name. The label column in
tko_jobs is essentially the job name. However, that column has a size limit of
100 characters, thus the name could be truncated. This CL parse the actual job
name to get the build, board and suite info and save to tko_jobs table.

BUG=chromium:509770,chromium:509901
TEST=local test
CQ-DEPEND=CL:285026

Change-Id: I06b073b052a9d07ffd36308b1682a7bc12699898
Reviewed-on: https://chromium-review.googlesource.com/286265
Tested-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Mungyung Ryu <mkryu@google.com>
Commit-Queue: Dan Shi <dshi@chromium.org>
fe/site_rpc_interface.py
07a109f19cd7363fb5440c72e870802392b7ce24 22-Jul-2015 MK Ryu <mkryu@google.com> [autotest] Sync host's status

For hosts in shards, their status is never synced with master.
Users should go to shard AFE to see the host's status.
This CL syncs the host status via shard heartbeat.

BUG=chromium:510996
TEST=puppylab. Change a host status in stumpy shard manually,
and check if it is synced to the master.
DEPLOY=apache,shard_client

Change-Id: Ic2bdce6aeeea868cbb000a79b6301e4848514be7
Reviewed-on: https://chromium-review.googlesource.com/287615
Reviewed-by: Mungyung Ryu <mkryu@google.com>
Commit-Queue: Mungyung Ryu <mkryu@google.com>
Tested-by: Mungyung Ryu <mkryu@google.com>
fe/site_rpc_interface.py
fe/site_rpc_interface_unittest.py
2a5297bf96c1d3eec4c71931c9cebba0737fef36 24-Jul-2015 Dan Shi <dshi@chromium.org> [autotest] Fix a typo in create_parameterized_job RPC

The RPC is not used any where, so no damage is done so far.

BUG=None
TEST=None

Change-Id: I9f616f61c0af2dc7c064bd346ceb121787145af0
Reviewed-on: https://chromium-review.googlesource.com/288042
Commit-Queue: Dan Shi <dshi@chromium.org>
Trybot-Ready: Dan Shi <dshi@chromium.org>
Tested-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Laurence Goodby <lgoodby@chromium.org>
fe/rpc_interface.py
a09a37a6208ad09b0dbcd2ad0da5ad4c41c24144 23-Jul-2015 Fang Deng <fdeng@chromium.org> [autotest] Add a server role 'afe'

This role represents our rpc server.

TEST=atest server create -r afe SERVER_HOSTNAME
BUG=chromium:513062

Change-Id: I840b546c35f73d96b136802ef859fe363d02c1ec
Reviewed-on: https://chromium-review.googlesource.com/287785
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Fang Deng <fdeng@chromium.org>
Tested-by: Fang Deng <fdeng@chromium.org>
erver/models.py
5dfcc89c498c0efd647afe7baef14334346b42d1 17-Jul-2015 MK Ryu <mkryu@google.com> [autotest] Create a shard with multiple board labels

'atest shard create' allows only one label for a shard.
Since our sharding architecture allows multiple board labels for
a shard, update atest tool accordingly.

Old code only lists all shards with 'atest shard list' even
in the case a user passes 'shards' parameter.
'shards' parameter didn't work. This CL corrects the feature.

In addition, old code doesn't have '--label' option for
'atest shard list', but there is a code to parse the option.
Removed the parsing code.

BUG=chromium:489914
TEST=Test 'atest' tool on local machine.
DEPLOY=apache

Change-Id: Ia18374d9b8b9d44ead63c98cc13fb47a84ddf70f
Reviewed-on: https://chromium-review.googlesource.com/286194
Trybot-Ready: Mungyung Ryu <mkryu@google.com>
Tested-by: Mungyung Ryu <mkryu@google.com>
Reviewed-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Mungyung Ryu <mkryu@google.com>
fe/site_rpc_interface.py
fe/site_rpc_interface_unittest.py
8a420d66388011e875eea58012de6ca462493e4c 13-Jul-2015 Dan Shi <dshi@chromium.org> [autotest] Add 5 columns to tko_jobs table.

Parent job id is used in wmatrix. For wmatrix to query only tko tables, we need
to include the parent job id in tko_jobs table.

GoldenEye gets the information of build, build_version, board and suite from afe
tables. This CL adds those 4 columns to tko_jobs table.

build_version is the version part of value build, e.g., R45-7111.0.0
build_version is added to help GoldenEye's query performance.

BUG=chromium:509770,chromium:509901
TEST=local test
./database/migrate.py sync
./database/migrate.py sync 102

DEPLOY=migrate

Change-Id: I5dc8c41bde79aad72a0d1a57534ea4b08521a250
Reviewed-on: https://chromium-review.googlesource.com/285026
Trybot-Ready: Dan Shi <dshi@chromium.org>
Tested-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Moises Osorio <moisesosorio@google.com>
Reviewed-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
igrations/103_add_5_columns_to_tko_jobs.py
5c7ef30e9d5ad3fd49a57b3ca08520875b2cce6d 11-Jul-2015 MK Ryu <mkryu@google.com> [autotest] Forward the job created by specifying host label to shard

On AFE, we can create a job by specifying a host label.
Such jobs are not currently forwarded to a shard.
By creating a record in afe_jobs_dependency_labels table for the job,
we can make it forwarded.

BUG=chromium:508770
TEST=Puppylab. Create a job with a meta_host from AFE and check if
the job is forwarded to the shard.
DEPLOY=apache

Change-Id: I86388abb7f685f8c6950364ebed95e6b4081f143
Reviewed-on: https://chromium-review.googlesource.com/284924
Reviewed-by: Mungyung Ryu <mkryu@google.com>
Commit-Queue: Mungyung Ryu <mkryu@google.com>
Tested-by: Mungyung Ryu <mkryu@google.com>
Reviewed-by: Fang Deng <fdeng@chromium.org>
fe/models.py
fe/rpc_utils.py
2e3e6059f397ba791a2f9c836ed38bb0e3dfe34b 10-Jul-2015 MK Ryu <mkryu@google.com> [autotest] Frontend job is not forwared to its shard

frontend.afe.models.Job.assign_to_shard() had a bug.

BUG=chromium:508697
TEST=puppylab. Create a frontend job to a shard host.

Change-Id: I5218af68dff3531202c25b1f960766b8f810970b
Reviewed-on: https://chromium-review.googlesource.com/284674
Trybot-Ready: Mungyung Ryu <mkryu@google.com>
Tested-by: Mungyung Ryu <mkryu@google.com>
Reviewed-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Mungyung Ryu <mkryu@google.com>
fe/models.py
c08c18f84994aef06a282dada77bcd6e6ace8d38 08-Jul-2015 J. Richard Barnette <jrbarnette@chromium.org> [autotest] Delete dead code relating to obsolete web interfaces.

Deletes things frontend/croschart, site_utils/experimental, and
things that seemed to mention them.

BUG=None
TEST=Cherry-picked to chromeos-autotest.cbf, and ran push to prod test

Change-Id: I15954bab6a93ff8e33a72938f8d8b239078308b2
Reviewed-on: https://chromium-review.googlesource.com/284591
Reviewed-by: Don Garrett <dgarrett@chromium.org>
Tested-by: Richard Barnette <jrbarnette@chromium.org>
Commit-Queue: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Fang Deng <fdeng@chromium.org>
roschart/DEPLOYMENT_README
roschart/__init__.py
roschart/charterrors.py
roschart/chartmodels.py
roschart/chartutils.py
roschart/chartviews.py
roschart/common.py
roschart/croschart_defaults.json
roschart/deploy_new_croschart.sh
roschart/labtest/__init__.py
roschart/labtest/models.py
roschart/perfchart/__init__.py
roschart/perfchart/models.py
roschart/releasereport/__init__.py
roschart/releasereport/crosrelease_defaults.json
roschart/releasereport/models.py
roschart/reportviews.py
roschart/templates/base.html
roschart/templates/chartplatforms/plot_chartplatforms.html
roschart/templates/chartreport/plot_chartreport.html
roschart/templates/labtest/plot_labtestreport.html
roschart/templates/perfchart/plot_chart.html
roschart/templates/perfchart/plot_chart_base.html
roschart/templates/perfchart/plot_chartdiff.html
roschart/templates/plot_syntax.html
roschart/templates/plot_unavailable.html
roschart/templates/releasereport/plot_releasereport.html
roschart/templates/testreport/plot_testreport.html
roschart/testreport/__init__.py
roschart/testreport/models.py
roschart/urls.py
roschart/validators.py
roschart/views.py
erf-dashboard/common.py
erf-dashboard/croschart_defaults.json
erf-dashboard/doc/default.html
erf-dashboard/extract_perf.py
erf-dashboard/generate_perf_graphs.py
erf-dashboard/templates/branch_platform_overview.html
erf-dashboard/templates/compare_by_overview.html
erf-dashboard/templates/config.js
erf-dashboard/templates/overview_base.html
ettings.py
emplates/404.html
emplates/500.html
emplates/emails/performance_regressed.html
emplates/emails/tests_base.html
emplates/emails/tests_failure.html
emplates/emails/tests_success.html
emplates/tables/base.html
emplates/tables/configs/header.html
emplates/tables/configs/test_config.html
emplates/tables/details/details.html
emplates/tables/details/header.html
emplates/tables/details/index.html
emplates/tables/details/perf_builds.html
emplates/tables/details/perf_index.html
emplates/tables/details/test_header.html
emplates/tables/monitoring.html
emplates/tables/performance/build_performance.html
emplates/tables/summary/bvt_timing.html
emplates/tables/summary/emails.html
emplates/tables/summary/header.html
emplates/tables/summary/index.html
emplates/tables/summary/kernel_header.html
emplates/tables/summary/kernel_index.html
emplates/tables/summary/table_index.html
emplates/tables/summary/waterfall_index.html
emplates/tables/summary/waterfall_kernel.html
rls.py
e019aae6c55e7e31acaf291370c6a48d5792cd79 07-Jul-2015 MK Ryu <mkryu@google.com> [autotest] Keep platform bit for a label in shards

When we add a label to a host that is in a shard,
the platform bit of the label should be forwarded as well.

BUG=chromium:502337
TEST=puppylab. Add a platform label to a host in stumpy shard.
DEPLOY=apache

Change-Id: I12c1dfc7d9a46cda5982d687a2fd160d187582a0
Reviewed-on: https://chromium-review.googlesource.com/283943
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Mungyung Ryu <mkryu@google.com>
Tested-by: Mungyung Ryu <mkryu@google.com>
fe/rpc_interface.py
73be986fc3bc0c44f6cf0e3f5ce42869d946c9f9 06-Jul-2015 MK Ryu <mkryu@google.com> [autotest] We do not need to create cros-version label when provisioning

We add a cros-version label to a host when provision finishes.
We do not need to create the label in the provision special task.

BUG=chromium:506399
TEST=./cli/atest job create -d cros-version:nyan_big-release/R43-6872.0.0
-b board:nyan_big --test=dummy_Pass dummy_Pass
DEPLOY=apache

Change-Id: Ie042bcdf6a35ade0ee3570743cd3a8a43d96140b
Reviewed-on: https://chromium-review.googlesource.com/283540
Tested-by: Mungyung Ryu <mkryu@google.com>
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Mungyung Ryu <mkryu@google.com>
fe/rpc_utils.py
fb5e3a861d4ff455276ebbefe6dc04e1e89c22dc 01-Jul-2015 MK Ryu <mkryu@google.com> [autotest] cros-version mismatch

add_cros_version_labels_and_job_repo_url() sends a RPC, 'add_label',
to AFE where the code is running (that could be a shard node).
This results in adding the label only to shard DB and the master DB
doesn't have the label.

BUG=chromium:503440
TEST=Test provision on shard testing machines.
DEPLOY=apache

Change-Id: Iedf46190c3c998677b43642f86cb72a55ed4fb27
Reviewed-on: https://chromium-review.googlesource.com/283100
Trybot-Ready: Mungyung Ryu <mkryu@google.com>
Tested-by: Mungyung Ryu <mkryu@google.com>
Reviewed-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Mungyung Ryu <mkryu@google.com>
fe/rpc_utils.py
f6ab8a720a22094c3c9c89593ce05f1ce6835a70 06-Jul-2015 MK Ryu <mkryu@google.com> [autotest] Make route_rpc_to_master decorator allow positional arguments

With the help of inspect.getcallargs(), we can call a decorated function
with positional arguments.

BUG=chromium:506316
TEST=Clone a job in dummy suite that was run before on local machine.
DEPLOY=apache

Change-Id: I6828f465dfb341f84ec7c7721432977bde4630b6
Reviewed-on: https://chromium-review.googlesource.com/283491
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Mungyung Ryu <mkryu@google.com>
Tested-by: Mungyung Ryu <mkryu@google.com>
fe/rpc_utils.py
e782e855a366eb4c9e51540c6e4d3440caf82772 30-Jun-2015 MK Ryu <mkryu@google.com> [autotest] Include client's IP in RPC logging

Having the RPC requester's IP address in the rpc log and in the
metaDB will help us a lot to debug lab fire.

BUG=chromium:505568
TEST=Request a RPC to local apache from a remote machine.
DEPLOY=apache

Change-Id: I283f55d78eb29ff59cce6e487661adaf84dea84a
Reviewed-on: https://chromium-review.googlesource.com/282574
Reviewed-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Paul Hobbs <phobbs@google.com>
Commit-Queue: Mungyung Ryu <mkryu@google.com>
Tested-by: Mungyung Ryu <mkryu@google.com>
fe/json_rpc/serviceHandler.py
fe/rpc_handler.py
78f281ab42f1afb28977e13a9c1be107b897db0f 29-Jun-2015 J. Richard Barnette <jrbarnette@chromium.org> [autotest] Fix get_host_diagnosis_interval() for shards.

The call detected "host has a shard", but didn't detect "... and
I'm the shard for the host", leading to infinite RPC recursion.
Very Bad.

BUG=chromium:492662
TEST=test on the .cbf shard

Change-Id: I6dc2c615bae4fe8266b8fcb6bedeb7647024c83c
Reviewed-on: https://chromium-review.googlesource.com/282580
Reviewed-by: Dan Shi <dshi@chromium.org>
Tested-by: Richard Barnette <jrbarnette@chromium.org>
Commit-Queue: Richard Barnette <jrbarnette@chromium.org>
fe/rpc_interface.py
e301eb76e56ffbf958cceded3920808a29912510 25-Jun-2015 MK Ryu <mkryu@google.com> [autotest] Reroute some RPCs called to a shard to master

create_job and create_suite_job RPCs should not be handled by a shard.
They should be redirected to the master.

BUG=chromium:503188
TEST=puppylab. Submit a dummy suite in a stumpy shard.

Change-Id: Ibe432655c677024e9eee1f4aca118ef6db7951f7
Reviewed-on: https://chromium-review.googlesource.com/281818
Reviewed-by: Mungyung Ryu <mkryu@google.com>
Commit-Queue: Mungyung Ryu <mkryu@google.com>
Tested-by: Mungyung Ryu <mkryu@google.com>
fe/rpc_interface.py
fe/rpc_interface_unittest.py
fe/site_rpc_interface.py
fe/site_rpc_interface_unittest.py
38c2499bcfece570bfd5aa4d62155985ac6d5a11 25-Jun-2015 Dan Shi <dshi@chromium.org> [autotest] Default build to CrOS build in create_suite_job

The build meant to be default to CrOS build. Set it to test_source_build
introduced a bug when test_source_build is a firmware build.

BUG=None
TEST=Local

Change-Id: Ibb81820cd75b0a9fb6544092297f96fc9c04a744
Reviewed-on: https://chromium-review.googlesource.com/281925
Reviewed-by: Simran Basi <sbasi@chromium.org>
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
Trybot-Ready: Dan Shi <dshi@chromium.org>
Tested-by: Dan Shi <dshi@chromium.org>
fe/site_rpc_interface.py
1043809018daa39812a64805c04456ce7b706d08 24-Jun-2015 Matthew Sartori <msartori@chromium.org> [autotest] Fixes for 'Create Job' page when using 'Fetch Tests from Build'

'Fetch Tests from Build' correctly populated the test selector with
the tests supported by the provided image, however, when communicating
with the backend, default test information from the database was being
used. This resulted in the wrong control file being used for the
selected test. In addition, the get_tests_by_build rpc reponse was
missing information required by certain AFE functions, this resulted
in the auto-selection for hostless jobs to break when fetching tests
for an image.

This CL makes it so that when tests are fetched for a particular image,
client state describing the control file is used to find/build
the control file rather than pulling from the default test database.
It also corrects the get_tests_by_build rpc so that the missing
attributes are provided.

BUG=chromium:503385,chromium:503731
DEPLOY=afe
TEST=Tested changes on a moblab.

Change-Id: I54739a4bb6e96cd2517ea59dc0a937d0d81cc000
Reviewed-on: https://chromium-review.googlesource.com/281760
Tested-by: Matthew Sartori <msartori@chromium.org>
Reviewed-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Matthew Sartori <msartori@chromium.org>
fe/rpc_interface.py
fe/rpc_utils.py
fe/site_rpc_interface.py
lient/src/autotest/afe/TestSelector.java
lient/src/autotest/afe/create/CreateJobViewPresenter.java
0723bf5ed7ff506e15fea180547cb6a8ae9102eb 24-Jun-2015 Dan Shi <dshi@chromium.org> [autotest] Rename provision.FW_VERSION_PREFIX to provision.FW_RW_VERSION_PREFIX

To make it clear that the build is for updating RW firmware.

BUG=chromium:270258
TEST=local run_suite
./run_suite.py -b veyron_jerry -i trybot-veyron_jerry-paladin/R45-7122.0.0-b11 \
--firmware_rw_build veyron_jerry-firmware/R41-6588.106.0 --test_source_build \
veyron_jerry-firmware/R41-6588.106.0 -p suites -s dummy

Change-Id: I842f348bc1ba567c5622ac126e8ca35392506774
Reviewed-on: https://chromium-review.googlesource.com/281623
Trybot-Ready: Dan Shi <dshi@chromium.org>
Tested-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
fe/rpc_interface.py
8abbfd638af09a39bb65fc28e819fc46199a8281 23-Jun-2015 J. Richard Barnette <jrbarnette@chromium.org> [autotest] Add dut-status --diagnosis.

This adds a new --diagnosis option to the dut-status command that
automatically displays all logs in a sequence leading to a DUT
failure.

This change also plumbs in a new RPC call needed to support the
feature.

BUG=chromium:492662
TEST=Run the command with the new option using a local RPC server

Change-Id: Ibb7431199bd0f8bbfe06fb8fffcb59512b01ab5b
Reviewed-on: https://chromium-review.googlesource.com/281205
Tested-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Richard Barnette <jrbarnette@chromium.org>
fe/rpc_interface.py
d215dbe17369b0781b1a0ceb4a70b25f120670fd 19-Jun-2015 Dan Shi <dshi@chromium.org> [autotest] Add more build option in autotest create_job tab for FAFT

Add an Advanced Build Options panel for user to specify:
firmware build (RW), firmware RO build, and test source build.

Then create a suite job from AFE to update both CrOS and firmware.

BUG=chromium:493429
TEST=local test
http://dshi.mtv/afe/#tab_id=create_job
1. check AFE shows the Advanced Build Options panel
2. Confirm the builds specified in the options are used in the sutie job run.
3. Confirm non-suite job can't be created with firmware image specified.
sample cros build: veyron_jerry-release/R45-7201.0.0
sample firmware build: veyron_jerry-firmware/R41-6588.106.0
DEPLOY=afe,apache

Change-Id: I81e1f5f81aaa9dfb5fe7b9f2f3677b8fb437130c
Reviewed-on: https://chromium-review.googlesource.com/281021
Trybot-Ready: Dan Shi <dshi@chromium.org>
Tested-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
fe/rpc_interface.py
fe/site_rpc_interface.py
lient/src/autotest/afe/create/CreateJobViewDisplay.java
lient/src/autotest/afe/create/CreateJobViewPresenter.java
lient/src/autotest/public/AfeClient.html
8c554cf044be8993824bf7159cd0a4ac2fb2ec78 12-Jun-2015 MK Ryu <mkryu@google.com> [autotest] Forward modify_label and delete_label RPCs

BUG=chromium:499660
TEST=puppylab. Let test_host1 and test_host11 have a label l1.
test_host1 is in master and test_host11 is in stumpy shard.
Send modify_label and delete_label RPCs for the label l1,
and check master DB and stumpy shard's DB if they are
updated in the same way.
DEPLOY=apache

Change-Id: I1d7b0f06a4aeb0b2470a677cea12d195535d154f
Reviewed-on: https://chromium-review.googlesource.com/277143
Tested-by: Mungyung Ryu <mkryu@google.com>
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Mungyung Ryu <mkryu@google.com>
fe/rpc_interface.py
36cfd831af781eed114337efa5b90c103a49b502 10-Oct-2014 Dan Shi <dshi@chromium.org> [autotest] Support updating both firmware and CrOS from run_suite

Add firmware_build option for run_suite to pass in firmware build.
Save the build containing server-side test to job_keyvals.

Design doc:
https://docs.google.com/a/google.com/document/d/115aAHyZaatFDzuKWm61Sj2fImH4NDWikXqifvWKbbW0/edit#

BUG=chromium:270258
TEST=local run_suite:
make sure it works with older build:
./run_suite.py -b veyron_jerry -i trybot-veyron_jerry-paladin/R45-7086.0.0-b8 -p suites -s dummy_server

Call failed as --firmware_ro_build not supported yet.
./run_suite.py -b veyron_jerry -i trybot-veyron_jerry-paladin/R45-7122.0.0-b11 --firmware_build veyron_jerry-firmware/R41-6588.9.0 --test_source_build trybot-veyron_jerry-paladin/R45-7122.0.0-b11 --firmware_ro_build veyron_jerry-firmware/R41-6588.9.0 -p suites -s dummy_server

Call failed as the firmware build does not have test_suites package built:
./run_suite.py -b veyron_jerry -i trybot-veyron_jerry-paladin/R45-7122.0.0-b11 --firmware_build veyron_jerry-firmware/R41-6588.9.0 --test_source_build veyron_jerry-firmware/R41-6588.9.0 -p suites -s dummy_server

make sure it works with new build:
./run_suite.py -b veyron_jerry -i trybot-veyron_jerry-paladin/R45-7122.0.0-b11 --firmware_build veyron_jerry-firmware/R41-6588.9.0 --test_source_build trybot-veyron_jerry-paladin/R45-7122.0.0-b11 -p suites -s dummy_server

DEPLOY=apache,scheduler

Change-Id: I50d23a7e81e4d6224b3483111110f910eb407074
Reviewed-on: https://chromium-review.googlesource.com/272793
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
Trybot-Ready: Dan Shi <dshi@chromium.org>
Tested-by: Dan Shi <dshi@chromium.org>
fe/site_rpc_interface.py
9651ca5498fab442b4b3e819a0ac854c66b91e23 09-Jun-2015 MK Ryu <mkryu@google.com> [autotest] Use RetryingAFE for RPCs to shards

RPC call can fail by timeout time to time.
If a RPC sent to a shard from a master fails, DB inconsistency can happen.
Let's use RetryingAFE when sending RPCs to shards.

BUG=chromium:496452
TEST=puppylab. Try to add/remove labels for a sharded host.
It uses RPCs that calls RetryingAFE.
Unit Tests.
frontend.afe.rpc_interface_unittest
frontend.afe.site_rpc_interface_unittest
DEPLOY=apache

Change-Id: I23853c8c0f76d31bb10493759cafdbdc6c003b3f
Reviewed-on: https://chromium-review.googlesource.com/276248
Reviewed-by: Mungyung Ryu <mkryu@google.com>
Tested-by: Mungyung Ryu <mkryu@google.com>
Commit-Queue: Mungyung Ryu <mkryu@google.com>
fe/rpc_interface_unittest.py
fe/rpc_utils.py
fbb002cb354904aa7b318dced5baa0042d8dc357 08-Jun-2015 MK Ryu <mkryu@google.com> [autotest] Forward RPCs that update host labels to shards

Other than label_add_hosts and label_remove_hosts,
we have 2 more RPCs that updates host labels.
They are host_add_labels and host_remove_labels.
These RPCs should be forwarded to shards.

BUG=chromium:497887
TEST=puppylab. test_host13 is a host in a shard.
>>> import common
>>> from autotest_lib.server import frontend
>>> afe = frontend.AFE()
>>> h = afe.get_hosts(['test_host13'])[0]
>>> h.add_labels(['l1','l3'])
Adding labels ['l1', 'l3'] to host test_host13
>>> h.remove_labels(['l1'])
DEPLOY=apache

Change-Id: Ic37c026efa009eff3aa5276cbbef8c3e1d62e508
Reviewed-on: https://chromium-review.googlesource.com/276211
Reviewed-by: Mungyung Ryu <mkryu@google.com>
Commit-Queue: Mungyung Ryu <mkryu@google.com>
Tested-by: Mungyung Ryu <mkryu@google.com>
fe/doctests/001_rpc_test.txt
fe/rpc_interface.py
fe/rpc_interface_unittest.py
fe/rpc_utils.py
fe/site_rpc_interface.py
c6b879894cbfefe51bd003d1686a96e4f4d0e6de 07-Jun-2015 Fang Deng <fdeng@chromium.org> [autotest] Add database slave role to server db.

TEST=cli/atest create -r database_slave HOST
BUG=chromium:444260

Change-Id: I7df22456b60a5ddb70c6e6f6977e569e9898de53
Reviewed-on: https://chromium-review.googlesource.com/275737
Tested-by: Fang Deng <fdeng@chromium.org>
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Fang Deng <fdeng@chromium.org>
erver/models.py
22f385b0c3becac398cd398a6ba5bb675dc761c7 06-Jun-2015 Fang Deng <fdeng@chromium.org> [autotest] modification to 102 migration script

Merge two alter statement into one, which makes it
faster.

with 81664102 rows in tko_tests, it takes 9 hours 22mins

TEST=database/migrate.py sync
database/migrate.py sync 101
BUG=None
DEPLOY=migrate

Change-Id: Ic52ac126a6ba6175f7f0ad445be15821da988e5c
Reviewed-on: https://chromium-review.googlesource.com/275744
Tested-by: Fang Deng <fdeng@chromium.org>
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Fang Deng <fdeng@chromium.org>
igrations/102_change_invalidates_test_idx_fk_constraint.py
dc4dff39fb6f3bc81ba712872317b1a122313447 05-Jun-2015 Fang Deng <fdeng@chromium.org> [autotest] modify invalidates_test_idx fk constraint

The foreign key invalidates_test_idx is causing trouble
in db cleanup. If row1 points to row2 points to row3, they can only
be removed in the order of row3, row2, row1, which often not
the case in the db cleanup. This CL change the fk constraint
from ON DELETE NO ACTION to ON DELETE CASADE, such that
the db will handle the dependency.

TEST=database/migrate.py sync
database/migrate.py sync 101
BUG=None
DEPLOY=migrate

Change-Id: Ic208b87b95ded81612d50f0798aa43d4f611f798
Reviewed-on: https://chromium-review.googlesource.com/275470
Reviewed-by: Dan Shi <dshi@chromium.org>
Tested-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Fang Deng <fdeng@chromium.org>
igrations/102_change_invalidates_test_idx_fk_constraint.py
3e1de8b4baf97c1b50cfd6a2c5307c49bcfbbb50 28-May-2015 MK Ryu <mkryu@google.com> [autotest] RPC logging server

frontend.afe.rpc_handler is not logging RPC logs correctly
due to multi apache processes.
We should have a daemon that receives log records from the
apache processes and writes them in a chronological order.

BUG=chromium:490897
TEST=Run dummy suite on local machine.
DEPLOY=apache

Change-Id: If53667720922d0db248225d17b9e5a31d350f063
Reviewed-on: https://chromium-review.googlesource.com/274827
Reviewed-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Fang Deng <fdeng@chromium.org>
Tested-by: Mungyung Ryu <mkryu@google.com>
Commit-Queue: Mungyung Ryu <mkryu@google.com>
fe/rpc_handler.py
fe/rpcserver_logging.py
d96fb9b34a65661008adab39b5ca998844ec7543 20-May-2015 Matthew Sartori <msartori@chromium.org> [autotest] Display only tests supported by build

The 'Create Job' tab was updated so that an 'Update Tests'
button was placed next to the entry for the image URL.

With this change, pressing the 'Update Tests' button when
a valid build is specified updates the test selection to
only contain tests that are specified in that image.

BUG=chromium:486221
DEPLOY=afe
TEST=Tested changes on a panther moblab.

Change-Id: I9817c3939c9454e7e2b558fa525496cc201fa03a
Reviewed-on: https://chromium-review.googlesource.com/272285
Reviewed-by: Dan Shi <dshi@chromium.org>
Tested-by: Matthew Sartori <msartori@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Aviv Keshet <akeshet@chromium.org>
fe/site_rpc_interface.py
lient/src/autotest/afe/TestSelector.java
lient/src/autotest/afe/create/CreateJobViewDisplay.java
lient/src/autotest/afe/create/CreateJobViewPresenter.java
lient/src/autotest/public/AfeClient.html
9162112211afc15977a879857f37fce909bae3cf 02-Jun-2015 Dan Shi <dshi@chromium.org> [autotest] Add a reserve role for server db

BUG=chromium:495762
TEST=local

Change-Id: Icfd39fdecb86f89c0a25f13865676f592ae350a7
Reviewed-on: https://chromium-review.googlesource.com/274815
Commit-Queue: Dan Shi <dshi@chromium.org>
Trybot-Ready: Dan Shi <dshi@chromium.org>
Tested-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
erver/models.py
26f0c9397777f98bdfdf5e5b4e6374cbb9131264 29-May-2015 MK Ryu <mkryu@google.com> [autotest] Forward set_host_attribute RPC

set_host_attribute RPC should be forwarded to shards of affected
hosts to update not only master DB but also the shards' DB.

BUG=chromium:492822
TEST=puppylab. Run ./atest/host mod -a testattr=hello1 test_host11
on the master VM. test_host11 is a host of stumpy shard.
Check if its host attribute has been affected in both master DB
and stumpy shard DB.
DEPLOY=apache

Change-Id: I1fb940ce92df8ca9afe1986cccc517b09c96f43e
Reviewed-on: https://chromium-review.googlesource.com/273947
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Mungyung Ryu <mkryu@google.com>
Tested-by: Mungyung Ryu <mkryu@google.com>
fe/rpc_interface.py
fe/rpc_utils.py
2b79fd7627875033b2650c302e385712976aea7f 21-May-2015 Fang Deng <fdeng@chromium.org> [autotest] Optimize shard queries

The django filters are slow as it translates
to a sql that searches through entire afe_host_queue_entries
table in order to exclude completed jobs, non-aborted known jobs
and active jobs.

This CL optimizes the quries.

DEPLOY=apache
BUG=chromium:490485
TEST=1. Test in puppy lab. Shard sync jobs and frontend jobs correctly.
Aborting jobs works.
2. manually execute the resulting SQL and verify return value.
3. python
>>> shard =
models.Shard.objects.filter(hostname='chromeos-server14.mtv.corp.google.com')[0]
>>> q = models.Job.objects.filter(dependency_labels=shard.labels.all(),
hostqueueentry__isnull=False)
>>> q = models.Job._add_filters_for_shard_assignment(q, [333])
The above query takes 30 secs in prod.
4. unittest
With the raw sql, it takes 5 secs.
Change-Id: I95297f26a7f81b48e1ad18f7247b0e3d19751a9d
Reviewed-on: https://chromium-review.googlesource.com/272516
Tested-by: Fang Deng <fdeng@chromium.org>
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Fang Deng <fdeng@chromium.org>
fe/models.py
509516b07b7e23656763dd05b1176b355506da22 18-May-2015 MK Ryu <mkryu@google.com> [autotest] Breakdown shard_heartbeat RPC latency

The current latency of shard_heartbeat RPC on master is 1 - 2.5 mins.
The RPC is composed of a series of DB accesses.
The latency is slower than expected. We need to figure out what DB
query takes most of the time.

BUG=chromium:489357
TEST=Puppylab. Make the master send stats to testing statsd server.

Change-Id: Ibe45d0234e41b48eccb6abcf978c739dd45eb64a
Reviewed-on: https://chromium-review.googlesource.com/271796
Tested-by: Mungyung Ryu <mkryu@google.com>
Reviewed-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Mungyung Ryu <mkryu@google.com>
fe/rpc_utils.py
773a86ebaa2e4e4cce213b8553aeb4b30c1f3a9d 14-May-2015 Simran Basi <sbasi@google.com> moblab_setup page: Save config diff to shadow_config.ini

Updates the moblab_setup page to only save the changed settings
to the shadow_config and not the whole config which eliminated
our ability to AU config changes. Now that only the changed settings
are saved we can properly toggle on/off features remotely.

BUG=chromium:487415
TEST=local moblab setup and unittests.

Change-Id: Ie75de09d196673f14908519a228e9bc0b18c0095
Reviewed-on: https://chromium-review.googlesource.com/271171
Reviewed-by: Simran Basi <sbasi@chromium.org>
Tested-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Simran Basi <sbasi@chromium.org>
fe/site_rpc_interface.py
fe/site_rpc_interface_unittest.py
8f8cdb425db94473f50579e2123f61831fac7322 12-May-2015 MK Ryu <mkryu@google.com> [autotest] Getting mixed HQEs and special tasks fail on shard AFE

BUG=chromium:486926
TEST=Prepare shard testing cluster. Run a dummy suite on a DUT
that is sharded, and check if I can get mixed HQEs and special tasks
on the shard AFE.

Change-Id: I66449e07d6f5ae4c5e39ee87d4608b8eae9777f3
Reviewed-on: https://chromium-review.googlesource.com/270342
Reviewed-by: Dan Shi <dshi@chromium.org>
Tested-by: Mungyung Ryu <mkryu@google.com>
Commit-Queue: Mungyung Ryu <mkryu@google.com>
fe/models.py
67900ba0ac175e65461867f95321e2761cb7847c 15-May-2015 Simran Basi <sbasi@google.com> [autotest] Relabel "suite jobs" in the AFE to parent jobs.

The suite job radio button is really just displaying parent jobs.
Since sequences will also have parent jobs lets rename this.

BUG=None
TEST=local moblab.
DEPLOY=afe

Change-Id: Ibe89ae8736a0ad2a2d3a0d12e166e65fcc1caaa4
Reviewed-on: https://chromium-review.googlesource.com/271344
Commit-Queue: Simran Basi <sbasi@chromium.org>
Tested-by: Simran Basi <sbasi@chromium.org>
Reviewed-by: Gwendal Grignou <gwendal@chromium.org>
Reviewed-by: Dan Shi <dshi@chromium.org>
lient/src/autotest/afe/JobListView.java
47656b155d1fe9532ff1335c1c9a4d7d1debac57 05-May-2015 Matthew Sartori <msartori@chromium.org> [autotest] Toggle display of 'Advanced Options' in 'Create Host' tab

The order of items on the Create Host tab has been reordered as
follows:

Job Name
Image URL/Build
test selection
advanced options
control file
host selector

The display of the advanced options can be toggled. The hostless
option under the advanced options panel is auto-selected if
the user chooses a test with name containing 'test_suites'.

BUG=chromium:483652
DEPLOY=afe
TEST=Tested by deploying changes to a moblab and verifying that
the advanced options display could be toggled and that the
hostless checkbox was auto-selected as desired.

Change-Id: I773c9189903e06b519f42a85464bdd2dc48554d2
Reviewed-on: https://chromium-review.googlesource.com/269259
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Matthew Sartori <msartori@chromium.org>
Tested-by: Matthew Sartori <msartori@chromium.org>
lient/src/autotest/afe/HostSelectorDisplay.java
lient/src/autotest/afe/TestSelectorDisplay.java
lient/src/autotest/afe/create/CreateJobViewDisplay.java
lient/src/autotest/afe/create/CreateJobViewPresenter.java
lient/src/autotest/public/AfeClient.html
lient/src/autotest/public/afeclient.css
6766de73dd6d43d38f4b5e194ad84da0f7509d3f 14-May-2015 MK Ryu <mkryu@google.com> [autotest] Forward RPCs related to stable version to master

Servo uses following 4 RPCs to get/set/delete stable version.
get_stable_version, get_all_stable_versions,
set_stable_version, and delete_stable_version.
When these RPCs are requested to a shard, then forward them
to the master, because only master DB has data in the
afe_stable_versions table (this table is not synced with shards).

BUG=chromium:487792
TEST=Puppylab. Try to request the 4 RPCs to both the master and
the stumpyshard and check the return.

Change-Id: I39724eab9701e1592a84ae71f101a5a93363986f
Reviewed-on: https://chromium-review.googlesource.com/270819
Tested-by: Mungyung Ryu <mkryu@google.com>
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Mungyung Ryu <mkryu@google.com>
fe/site_rpc_interface.py
638a6cf6935329fd726bade7a87a45a180218f29 08-May-2015 MK Ryu <mkryu@google.com> [autotest] Shard supports multi board labels

A single chromeos board can have more than one board label.
For example, stumpy DUTs can have either "board:stumpy" or
"board:stumpy_freon".
DUTs with "board:stumpy" and those with "board:stumpy_freon" can
be swapped each other by DUT management scripts.
Current sharding design requires those DUTs be in the same shard.
Therefore, one shard should support multi board labels.

BUG=chromium:485817
TEST=Prepare DUTs with "board:stumpy" and DUTs with "board:stumpy_freon".
Create a shard that has both board labels.
Run dummy suite and bvt-cq suite in puppylab and in testing machines
using real DUTs.

Change-Id: I6a7d04b1afd9b23fa4ea4e21358d50d31e82388f
Reviewed-on: https://chromium-review.googlesource.com/270114
Tested-by: Mungyung Ryu <mkryu@google.com>
Reviewed-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Mungyung Ryu <mkryu@google.com>
fe/models.py
0c1a37dd9b1237fe8d43c7f911ce601104806339 30-Apr-2015 MK Ryu <mkryu@google.com> [autotest] Support sharding for getting HQEs and special tasks

Following 2 RPCs that are used by AFE java client are improved
to support sharding.

get_host_queue_entries_and_special_tasks
get_num_host_queue_entries_and_special_tasks

BUG=chromium:462819
DEPLOY=afe,apache
TEST=puppylab. Get HQEs and special tasks for a lumpy host
which is sharded from the master AFE.

Change-Id: I68c0d1a70fb6b61034c755d3a2b7f26475994bb0
Reviewed-on: https://chromium-review.googlesource.com/268523
Reviewed-by: Mungyung Ryu <mkryu@google.com>
Commit-Queue: Mungyung Ryu <mkryu@google.com>
Tested-by: Mungyung Ryu <mkryu@google.com>
fe/models.py
fe/rpc_interface.py
fe/rpc_interface_unittest.py
fe/rpc_utils.py
lient/src/autotest/afe/HostDetailView.java
4d7e6e6eedc3a021c98a289ce156c279e0d4a737 01-May-2015 J. Richard Barnette <jrbarnette@chromium.org> [autotest] Fix the RPC interface for get_status_task().

The interface failed to provide termination when the call chain
reached the shard that owned the target host. This splits the
API into two parts: `get_status_task()` and `get_host_status_task()`.
The former actually gets the status task from the local shard
(without forwarding). The latter merely converts to a call to
`get_status_task()` targeted at the proper shard.

BUG=chromium:483485
TEST=test using local instance pointed at production database

Change-Id: I64411cab8a8b3f4bae0bf378c459099f32545a93
Reviewed-on: https://chromium-review.googlesource.com/268790
Reviewed-by: Mungyung Ryu <mkryu@google.com>
Tested-by: Richard Barnette <jrbarnette@chromium.org>
Commit-Queue: Richard Barnette <jrbarnette@chromium.org>
fe/rpc_interface.py
6818633834ad52c3de153235639ea9299a6e9a6d 28-Apr-2015 Matthew Sartori <msartori@chromium.org> [autotest] Require lock reason to lock device

When locking a device, a locking reason now must be provided.
This applies to both adding a new device, and modifying an
existing device from both the web frontend and the atest
command-line tool.

BUG=chromium:336805
DEPLOY=migrate
TEST=Tested adding locked/unlocked devices and locking/unlocking devices
from both the web frontend and using the 'atest host ...' command-line tools.

Change-Id: I3a8cd8891a2999f026dd709ae8a79e2b8cbc251a
Reviewed-on: https://chromium-review.googlesource.com/267595
Tested-by: Matthew Sartori <msartori@chromium.org>
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Matthew Sartori <msartori@chromium.org>
fe/admin.py
fe/doctests/001_rpc_test.txt
fe/models.py
fe/models_test.py
fe/rdb_model_extensions.py
fe/rpc_interface.py
fe/rpc_interface_unittest.py
fe/rpc_utils.py
lient/src/autotest/afe/AfeUtils.java
lient/src/autotest/afe/HostDetailView.java
lient/src/autotest/afe/HostListView.java
lient/src/autotest/afe/HostTable.java
lient/src/autotest/public/AfeClient.html
igrations/101_add_lock_reason_afe_hosts.py
06a4b52b21f663027da8a2b64c552b63224896d0 25-Apr-2015 MK Ryu <mkryu@google.com> [autotest] Exclude jobs that don't have HQE in heartbeat response

For a heartbeat response, a master included jobs
that don't have HQEs. HQE records can be accidently removed.
This CL makes autotest tolerable to such an accident.

BUG=chromium:479766
DEPLOY=apache
TEST=puppylab

Change-Id: Id1c75c9cd869e91552749f8fc7ea3bb14b810bb1
Reviewed-on: https://chromium-review.googlesource.com/267290
Reviewed-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Mungyung Ryu <mkryu@google.com>
Tested-by: Mungyung Ryu <mkryu@google.com>
fe/models.py
39255faa9eebb1b64da7b21fc5874414ad6bad83 15-Apr-2015 J. Richard Barnette <jrbarnette@chromium.org> [autotest] Create a new get_status_task RPC to improve dut_status.

Previously, status was determined by fetching all jobs in a given
time range, and then searching back through the returned list for
a job that indicated host status. This had two drawbacks:
* If the job indicating status was earlier than the start time
of the query, no status could be determined.
* The extra data returned in the query slowed it down.

This adds a new RPC that finds the task through a single database
query searching for the specific necessary condition. This should
largely eliminate the '--' status, and is notably faster (possibly 4
times faster, or more).

This also changes the rules for what constitues a diagnostic job or
task, by changing the diagnosis associated with test jobs. This
is done because
* It makes determining status possible with a query to a single
table.
* The experience has been that test jobs don't indicate a
diagnosis anyway.

This is the first of two parts; this change creates the server
side of the new RPC. This change must be pushed to prod before
the follow-on change for the client side.

BUG=chromium:463632
TEST=Run dut_status using a local instance pointed at the prod database

Change-Id: Ifc61387f0cdc46ceb32062663c6a73c5d4bcfb4f
Reviewed-on: https://chromium-review.googlesource.com/265783
Reviewed-by: Richard Barnette <jrbarnette@chromium.org>
Commit-Queue: Richard Barnette <jrbarnette@chromium.org>
Tested-by: Richard Barnette <jrbarnette@chromium.org>
fe/rpc_interface.py
1e10e92049f4b2724ea43186083e995e418f9802 17-Apr-2015 Simran Basi <sbasi@google.com> [autotest] Add --offload_failures_only flag for Suites

Adds a new run_suite flag: --offload_failures_only and the required
piping to set it as a job keyval for each subjob of a suite.

Updates the tko_parser to determine if a test was successful or not
and should it be offloaded or not.

Introduces the concept of gs_offloader_instructions json file for
each test. This allows us to improve the intelligence of
gs_offloader by allowing tests to supply information to gs_offloader.

BUG=chromium:471330
TEST=local moblab setup. Ensured that only failed jobs were offloaded.
DEPLOY=gs_offloader,apache

Change-Id: I2310d710f1dce39a70766f6408f6a3c94f493e0d
Reviewed-on: https://chromium-review.googlesource.com/266090
Reviewed-by: Simran Basi <sbasi@chromium.org>
Tested-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Simran Basi <sbasi@chromium.org>
fe/site_rpc_interface.py
fdfcd664b69e585d47b518ab41ae6621ab50c0de 14-Apr-2015 J. Richard Barnette <jrbarnette@chromium.org> [autotest] Drop compatibility for the old dut-status.

This is the last of three stages to update the RPC API for special
tasks. This change drops the compatibility behaviors from the
get_special_tasks() RPC call.

This change cannot be pushed to prod until all old versions of
dut-status have been updated.

BUG=None
TEST=Test dut-status on a local instance using the prod database

Change-Id: I2680f2541c363975518b0fbe33f9bc312236ec8c
Reviewed-on: https://chromium-review.googlesource.com/265575
Reviewed-by: Fang Deng <fdeng@chromium.org>
Tested-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Mungyung Ryu <mkryu@google.com>
Commit-Queue: Richard Barnette <jrbarnette@chromium.org>
fe/rpc_interface.py
fe/rpc_utils.py
b5164d69f540b9f64413d43fb462aa3da73c58e0 13-Apr-2015 J. Richard Barnette <jrbarnette@chromium.org> [autotest] Create the get_host_special_tasks() RPC.

The existing get_special_tasks() RPC has two distinct callers:
Callers that only need tasks from the current shard, and callers
that need tasks for a specific host (and therefore, require the
RPC to be forwarded to the host's shard). This change splits that
function into two separate RPC calls: get_special_tasks() is used
for the local shard case, and get_host_special_tasks() is used for
single-host queries.

The only caller affected by this change is the status_history
module. To preserve compatibility with older versions of dut-status
this change is rolling out in three stages. This change provides
temporary support for both old and new versions of dut-status until
all the old versions of dut-status can be updated.

BUG=None
TEST=Test dut-status on a local instance using the prod database

Change-Id: I06f239d011d20f5a6357c35121951f9ecf34f61e
Reviewed-on: https://chromium-review.googlesource.com/265486
Tested-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Richard Barnette <jrbarnette@chromium.org>
fe/rpc_interface.py
fe/rpc_utils.py
86a4758dc7a958fda9e8476d07a08804b763dc0c 20-Mar-2015 Dan Shi <dshi@chromium.org> [autotest] In AFE, default require_ssp to false if its value is NULL in afe_job

AFE is compiled from java code, it can't convert null to a proper boolean value.
For afe_job object with require_ssp set to NULL, we will default it to show as
false in AFE.

BUG=chromium:467075
TEST=create an afe_job entry with require_ssp set to NULL, view the job in AFE,
confirm Require server-side packaging showing as false. Try to clone the job,
confirm the "Require server-side packaging" setting in create job page is
unchecked.

Change-Id: I849d66382b233e5ffcfaf69d0214f29dbc9a3e40
Reviewed-on: https://chromium-review.googlesource.com/261360
Trybot-Ready: Dan Shi <dshi@chromium.org>
Tested-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
lient/src/autotest/afe/JobDetailView.java
lient/src/autotest/afe/create/CreateJobViewPresenter.java
df4d423700d566bd73fe1859740c9c6b9ed3bca2 06-Mar-2015 MK Ryu <mkryu@google.com> [autotest] Allow fields of SERIALIZATION_LINKS_TO_KEEP to be serialized

In the logic of ModelExtensions, fields in SERIALIZATION_LINKS_TO_KEEP
are not considered to be related fields.
These fields can be included in serialization without any break of
logic.
This fix allows syncing back host_id of afe_host_queue_entries
from shard to master via heartbeat.

BUG=chromium:461637
DEPLOY=apache,scheduler,shard-client
TEST=puppylab. Run a dummy suite on a lumpy host (in shard)
and check master DB for the suite jobs' HQE after completion.

Change-Id: I6369f88106d9489385f0d7b78f10a3b0bf5f4314
Reviewed-on: https://chromium-review.googlesource.com/257031
Tested-by: Mungyung Ryu <mkryu@google.com>
Reviewed-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Mungyung Ryu <mkryu@google.com>
fe/model_logic.py
2d107567ded9b824b26bfd82e63c004f5f71314e 25-Feb-2015 MK Ryu <mkryu@google.com> [autotest] Consult appropriate shard AFE for special tasks

Special tasks are updated in the local DB of a shard.
RPCs hitting a master AFE that are getting special tasks
should consult an appropriate shard AFE.

BUG=chromium:461637
DEPLOY=apache
TEST=puppylab.
./site_utils/dut_status.py test_host41, where test_host41 is the dut
in master.
./site_utils/dut_status.py test_host51, where test_host51 is the dut
in lumpyshard.

Change-Id: I118ed5f4f20eba14c2e4c2c056e217c31f334aac
Reviewed-on: https://chromium-review.googlesource.com/254420
Reviewed-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Mungyung Ryu <mkryu@google.com>
Tested-by: Mungyung Ryu <mkryu@google.com>
fe/rpc_interface.py
fe/rpc_utils.py
c6e36907cbd16271d6cee207144b34d25b344068 05-Mar-2015 Dan Shi <dshi@chromium.org> [autotest] Add an option in frontend to force a new job requires server-side packaging

BUG=chromium:453624
TEST=rebuild AFE, try to create new jobs with require ssp enabled/disabled.
DEPLOY=afe
CQ-DEPEND=CL:249841

Change-Id: I1692c9af794b57fca976fd289fbd3606eb19e504
Reviewed-on: https://chromium-review.googlesource.com/256404
Reviewed-by: Dan Shi <dshi@chromium.org>
Tested-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
lient/src/autotest/afe/JobDetailView.java
lient/src/autotest/afe/create/CreateJobViewDisplay.java
lient/src/autotest/afe/create/CreateJobViewPresenter.java
lient/src/autotest/public/AfeClient.html
ec1d47d85cc83f30631518d8fbb6406036a3ac39 13-Feb-2015 Dan Shi <dshi@chromium.org> [autotest] Add support for scheduler to honor require_ssp attribute in control file

This CL adds changes to pipe require_ssp attribute in control file to autoserv
command. Following are the work flow:

1. The control file parser stores require_ssp attribute value in afe_jobs table.
2. QueueTask compiles command line list, --require-ssp option will be added to
the command line list if following conditions are met:
a. AUTOSERV/enable_ssp_container in global config is True
b. The test is a server-side test
c. require_ssp for the job entry is None or True.
3. When agent_task tries to call run method to run the command, it will check if
there is any drone supporting server-side packaging first. If no drone is found,
the agent task will will run the command in a drone without using server-side
packaging. A warning will be posted in the autoserv log.
4. If a drone without SSP supported is assigned to a test requires SSP, the test
will be run without ssp.

BUG=chromium:453624
TEST=unittest, local test:
set AUTOSERV/enable_ssp_container to True in shadow config;
Create a job for dummy_PassServer in AFE, check require SSP, confirm the job
succeeds but with a warning in the autoserv log.

Create a job for dummy_PassServer_nossp in AFE, uncheck require SSP, confirm
the job passes without warning in the autoserv log.

set AUTOSERV/enable_ssp_container to False in shadow config, restart scheduler.
Create a job for dummy_PassServer in AFE, check require SSP, confirm the job
succeeds without warning in the autoserv log.

also run test_that in local chroot to make sure test_that is not affected.

DEPLOY=apache,scheduler, db migrate must be done before push this CL to prod.

Change-Id: I02f3d137186676ae570e8380d975a1bcd9ffbb94
Reviewed-on: https://chromium-review.googlesource.com/249841
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
Trybot-Ready: Dan Shi <dshi@chromium.org>
Tested-by: Dan Shi <dshi@chromium.org>
fe/rpc_interface.py
fe/rpc_utils.py
d346fc992ad4d4d909b69c82242a082d96fda0d3 05-Mar-2015 Dan Shi <dshi@chromium.org> [autotest] Add a dummy database migration as a placeholder for the reverted CL

As mentioned in the db migrate comment:
This database migration does nothing. It works as a replacement for a reverted
migrate CL: https://chromium-review.googlesource.com/#/c/253760
For routine database migration revert, a new CL should be added to add
DOWN_SQL of the reverted migration as UP_SQL. However, in that CL's case,
DOWN_SQL is not available.
The dummy migration avoid the requirement to manually downgrade migrate_info
in each database.

BUG=None
TEST=./database/migrate.py sync

Change-Id: Ifd3ac2b51a2bd068e503f311c71237ce7014e03c
Reviewed-on: https://chromium-review.googlesource.com/256227
Commit-Queue: Dan Shi <dshi@chromium.org>
Trybot-Ready: Dan Shi <dshi@chromium.org>
Tested-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Fang Deng <fdeng@chromium.org>
igrations/100_dummy_migrate.py
cf027c67cbe849fe39dc876bde3057fd3677f997 04-Mar-2015 MK Ryu <mkryu@google.com> [autotest] Forward label_remove_hosts RPC to master

Master should coordinate label removal from hosts just
like label addition to hosts.

BUG=chromium:463977
DEPLOY=apache
TEST=puppylab.
afe = frontend.AFE(server='localhost:8004')
kwargs = {
'id': 'newlabel',
'hosts': ['test_host5','test_host51','test_host52','test_host53']
}
afe.run('label_remove_hosts', **kwargs)

Test on shard testing nodes, testing-shard-master.fra and
testing-shard-1.cbf.

Change-Id: I31798b7e01dd23ff281ed773f78295063dda6191
Reviewed-on: https://chromium-review.googlesource.com/255952
Reviewed-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Mungyung Ryu <mkryu@google.com>
Tested-by: Mungyung Ryu <mkryu@google.com>
fe/rpc_interface.py
0739d68fcbd84d02839a85ce3c594e774e82c602 26-Feb-2015 Simran Basi <sbasi@google.com> Moblab: Servo Support via Host Attributes.

Moblab devices will look for servo_args inside a host's host
attributes.

Host attributes can be set via the CLI or the Web Frontend.

CLI to see attributes:
atest host stat <host>

CLI to set attribute:
atest host mod --attribute <attribute> --value <value> <host>

Updated the afe's management system so that it properly configures
the admin interface to allow editing of host attributes.

BUG=chromium:394544
TEST=local moblab setup. Tested both CLI and AFE host attribute
manipulation. Launched Servod, and ensure servo host is only
created when the attribute is applied to the host.

Change-Id: Ie3cccab31aa7518435ef0abc6ce206363406c272
Reviewed-on: https://chromium-review.googlesource.com/255550
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Simran Basi <sbasi@chromium.org>
Tested-by: Simran Basi <sbasi@chromium.org>
fe/management.py
211c5acb05537a3a0835e60a3c42c12aa5e08e8e 03-Mar-2015 Simran Basi <sbasi@chromium.org> Revert "Moblab: Servo Support via Host Attributes."

This reverts commit 71f190bc31be36311c1c59d762a13a28754d7ea9.

Reverting as the db migration does not work on systems that have not touched the afe at all.

Change-Id: I0c6875ec17f9362b7e646471b6ae5d06c70b770d
Reviewed-on: https://chromium-review.googlesource.com/255350
Reviewed-by: Simran Basi <sbasi@chromium.org>
Tested-by: Simran Basi <sbasi@chromium.org>
igrations/100_insert_ignore_auth_group_permissions.py
f37059917de290d2983a7ca490884a8f694fb23e 17-Dec-2014 Fang Deng <fdeng@chromium.org> [autotest] Sync suite job keyvals to shards.

As we roll out shard, we need a way to communicate suite-level
metadata to shards. One way to do it is to store suite-level
metadata as job keyvals and sync them to shard, which is what
this CL is about.

Currently, only the keyval 'suite_min_duts' is needed by a shard.
In the future, we can pipe more keyvals to shards as needed like
file_bugs and retry if we were to move bug filing and retry
out of dynamic suites.

CQ-DEPEND=CL:254270
DEPLOY=apache,shard_client
BUG=chromium:432653
TEST=Setup a testing cluster with on master and one shard.
python
>> import common
>> from autotest_lib.server import frontend
>> afe = frontend.AFE(server='localhost:8001')
>> afe.run('shard_heartbeat', shard_hostname='localhost:8004')
Confirm the returned packet has 'suite_keyvals'
TEST=Setup a testing cluster with on master and one shard.
- Create a suite job and confirm suite keyvals are synced to shard.
- Confirm keyvals known to shard won't get synced multiple times.
TEST=unittest
TEST=Run dummy suite with run_suite, confirm everything still works.

Change-Id: I5d7816c2b3f6dfa374a297c09d597573f762c374
Reviewed-on: https://chromium-review.googlesource.com/254230
Tested-by: Fang Deng <fdeng@chromium.org>
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Fang Deng <fdeng@chromium.org>
fe/rpc_utils.py
fe/site_rpc_interface.py
936e6e41b276a66459e1256cbf52d50725f4220f 17-Dec-2014 Fang Deng <fdeng@chromium.org> [autotest] Drop foreign key constraint on job_id in afe_job_keyvals

As we roll out shard, we will sync suite-level metadata stored
as suite job keyvals to shards. We need to drop
the foreign constraint on job_id in afe_job_keyvals so that
we can sync only the keyvals without syncing the suite jobs.

BUG=chromium:432653
TEST=database/migrate.py sync; database/migrate.py sync 98
TEST=Confirm Job and JobKeyval still works. The following
will work if job 63 exists on the local db:
models.JobKeyval.objects.filter(job_id=63)[1].job
DEPLOY=migrate

Change-Id: If24aeae6734918cbb356c3904cc891453b0f61f8
Reviewed-on: https://chromium-review.googlesource.com/254270
Tested-by: Fang Deng <fdeng@chromium.org>
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
igrations/099_drop_afe_job_keyvals_foreign_key.py
71f190bc31be36311c1c59d762a13a28754d7ea9 26-Feb-2015 Simran Basi <sbasi@google.com> Moblab: Servo Support via Host Attributes.

Moblab devices will look for servo_args inside a host's host
attributes.

Host attributes can be set via the CLI or the Web Frontend.

CLI to see attributes:
atest host stat <host>

CLI to set attribute:
atest host mod --attribute <attribute> --value <value> <host>

For the Web Frontend, we require a DB change to make the host
attributes panel visible in the admin interface.

BUG=chromium:394544
TEST=local moblab setup. Tested both CLI and AFE host attribute
manipulation. Launched Servod, and ensure servo host is only
created when the attribute is applied to the host.
DEPLOY=migrate

Change-Id: I9fb3e8ab6d386104c7e9ed555eaa110a0ae72b02
Reviewed-on: https://chromium-review.googlesource.com/253760
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Simran Basi <sbasi@chromium.org>
Tested-by: Simran Basi <sbasi@chromium.org>
igrations/100_insert_ignore_auth_group_permissions.py
c9e1714424621786322d0e6ea48ac167bf35ce16 19-Feb-2015 Dan Shi <dshi@chromium.org> [autotest] Add a new field require_ssp in afe_jobs table

When a job is created, data in control file is parsed and saved in afe_jobs.
scheduler picks up the database row and tries to compile the commandline for
autoserv to run. require_ssp (require server-side packaging) is a new field
will be added to control file to force to use/do not use ssp when running
a server side test.

If the value is True or None(default), server-side packaging will be applied
to run server side test, if global config AUTOSERV/enable_ssp_container is
set to True.

Since the attribute can only be retrieved when parsing control file, it needs
to be stored in afe_jobs table. Therefore, the new field is added to the table.

require_ssp is nullable and has no default value, so the sql command should
only take several minutes to finish in production database.

BUG=chromium:453624
TEST=local run migrate.py sync and migrate.py sync 97
run scheduler locally and run a dummy suite, confirm scheduler and other
components work without issue (refer to bug 459523).

DEPLOY=migrate, db must be migrated after this CL is pushed to prod. Otherwise
scheduler will fail.

Change-Id: I7c65150b7ee3309310fdbed42b5079e564845a65
Reviewed-on: https://chromium-review.googlesource.com/251340
Trybot-Ready: Dan Shi <dshi@chromium.org>
Tested-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
fe/doctests/001_rpc_test.txt
fe/frontend_test_utils.py
fe/models.py
fe/models_test.py
igrations/098_add_require_ssp.py
84573e181e1bf732c274b22850b89d5fa681d196 19-Feb-2015 MK Ryu <mkryu@google.com> [autotest] Improve clarity of RPC failure for shards

frontend/afe/rpc_utils.fanout_rpc is the function that forwards
RPC to shards.
Add additional message which RPC failed on which shard for more
clarity.

BUG=chromium:448273
TEST=puppylab. Test 2 cases. One is to try cli/atest label add
-m test_host51 newlabel -w localhost:8001 when lumpyshard is
unreachable. This raises URLError exception.
The another is to try same atest command when test_host51 is
removed from lumpyshard. This raises DoesNotExist exception.
DEPLOY=apache

Change-Id: Ie58c3360bb4fa920bd807eca4545b294d179697a
Reviewed-on: https://chromium-review.googlesource.com/250953
Reviewed-by: Mungyung Ryu <mkryu@google.com>
Tested-by: Mungyung Ryu <mkryu@google.com>
Commit-Queue: Mungyung Ryu <mkryu@google.com>
fe/rpc_utils.py
ff36159d40fc7e7b5124f4950b6a18103aebd922 03-Feb-2015 Fang Deng <fdeng@chromium.org> [autotest] Sync based on key instead of id for attribute-like models

HostAttribute and JobKeyval are updated on both shard and
master. Both table should be synced based on "key" instead
auto-incremental id.

BUG=chromium:453122
DEPLOY=shard_cient, apache
TEST=python
>>import common
>>from autotest_lib.frontend import setup_django_environment
>>from autotest_lib.frontend.afe import models
>>m = models.Host.objects.get(id=20)
>>s = m.serialize()
>>s['hostattribute_set'] = A LIST OF NEW ATTRIBUTE RECORDS
>>m = models.Host.deserialize(s)
>>m.hostattribute_set.all()[0].__dict__

>>j = models.Job.objects.get(id=35)
>>s = j.serialize()
>>s['jobkeyval_set'] = A LIST OF NEW JOBKEYVAL RECORDS
>>m = models.Job.deserialize(s)
>>m.jobkeyval_set.all()[0].__dict__

TEST=Run a shard, schedule a dummy suite, ensure everything still
works.

Change-Id: I9c6450f93f06510477312bd02b119fd96b70857c
Reviewed-on: https://chromium-review.googlesource.com/245560
Reviewed-by: Fang Deng <fdeng@chromium.org>
Tested-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Fang Deng <fdeng@chromium.org>
fe/model_logic.py
fe/models.py
fe/models_test.py
60007cb606bfa60ad5e90ac356cc6284a5af2f46 18-Feb-2015 Simran Basi <sbasi@chromium.org> Revert "[autotest] Add a new field require_ssp in afe_jobs table"

This reverts commit e17ca03ed8d26c17261e47abd3f41fc5bbc98768.

Change-Id: I3300de55bfe3e0266095a3e43094f9ba0fb0f79d
Reviewed-on: https://chromium-review.googlesource.com/250691
Tested-by: Simran Basi <sbasi@chromium.org>
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
Trybot-Ready: Dan Shi <dshi@chromium.org>
igrations/098_add_require_ssp.py
e17ca03ed8d26c17261e47abd3f41fc5bbc98768 12-Feb-2015 Dan Shi <dshi@chromium.org> [autotest] Add a new field require_ssp in afe_jobs table

When a job is created, data in control file is parsed and saved in afe_jobs.
scheduler picks up the database row and tries to compile the commandline for
autoserv to run. require_ssp (require server-side packaging) is a new field
will be added to control file to force to use/do not use ssp when running
a server side test.

If the value is True or None(default), server-side packaging will be applied
to run server side test, if global config AUTOSERV/enable_ssp_container is
set to True.

Since the attribute can only be retrieved when parsing control file, it needs
to be stored in afe_jobs table. Therefore, the new field is added to the table.

require_ssp is nullable and has no default value, so the sql command should
only take several minutes to finish in production database.

BUG=chromium:453624
TEST=local run migrate.py sync and migrate.py sync 97

Change-Id: I509ae60f50c116df5ee76b2d302c35bb75551adc
Reviewed-on: https://chromium-review.googlesource.com/249170
Trybot-Ready: Dan Shi <dshi@chromium.org>
Tested-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
igrations/098_add_require_ssp.py
9c5fbbe29ab7ca7f8b69687de08a55c518378ae7 12-Feb-2015 MK Ryu <mkryu@google.com> [autotest] Send labels to shards preserving their ids.

We were allowing deviations between a shard and the master
by not specifying the label id for forwarded label create
rpcs. This cl breaks down the forwarding into 2 phases,
the first to create the label and the second to add the label
to the host, and sends the label id as part of the packet for
label creation.

TEST=atest label add -m host_not_on_shard new_label1
atest label add -m host_on_shard new_label2
checked that new_label2 have the same id on the master and shard.
BUG=chromium:448367
DEPLOY=apache

Change-Id: I29c99ca3fa25a0a3cb2cd91b759db009ee346799
Reviewed-on: https://chromium-review.googlesource.com/248835
Tested-by: Mungyung Ryu <mkryu@google.com>
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Mungyung Ryu <mkryu@google.com>
fe/rpc_interface.py
fe/rpc_utils.py
5cfd96aca6204b6fd193ab2e15a24808756e6198 31-Jan-2015 MK Ryu <mkryu@google.com> [autotest] Monitor the latency of heartbeat serialize/deserialize

Measure the latency for serializing/deserializing objects in heartbeat
between master and shards, and send the data to statsd server.

Serializing/deserializing Host object could become an overhead in the
future due to its dependency on other objects (epecially, AclGroup and
User), therefore, we should keep an eye on it.

BUG=chromium:448608
TEST=puppylab
DEPLOY=apache

Change-Id: Ic81233de3489d66e1f8287614c09d8b8db944982
Reviewed-on: https://chromium-review.googlesource.com/244849
Reviewed-by: Mungyung Ryu <mkryu@google.com>
Commit-Queue: Mungyung Ryu <mkryu@google.com>
Tested-by: Mungyung Ryu <mkryu@google.com>
fe/model_logic.py
1e1c41b1b4a1b97c0b7086b8430856ed45e064d3 05-Feb-2015 Gabe Black <gabeblack@chromium.org> graphite: Separate out configuration from the statsd classes.

The new version of the statsd classes should be created using an instance of
the new Statsd class which sets up some defaults without having to specify
them over and over. This makes it essentially compatible with the existing
usage in autotest, but will allow chromite to configure things differently and
avoid having side effects from importing the module or global state.

BUG=chromium:446291
TEST=Ran unit tests, ran stats_es_functionaltest.py, ran the
stats_mock_unittest, ran a butterfly-paladin tryjob with --hwtest, testing by
fdeng.
DEPLOY=apache,scheduler,host-scheduler

Change-Id: I1071813db197c0e5e035b4d8db615030386f1c1c
Reviewed-on: https://chromium-review.googlesource.com/246428
Reviewed-by: Fang Deng <fdeng@chromium.org>
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Gabe Black <gabeblack@chromium.org>
Tested-by: Gabe Black <gabeblack@chromium.org>
fe/json_rpc/serviceHandler.py
fe/rpc_interface.py
fe/site_rpc_interface.py
b72f4fbcf1583da27f09f4abb9d8162530bf4559 21-Jan-2015 Gabe Black <gabeblack@chromium.org> graphite: Reorganize the elastic search code so we can put it in chromite.

This change reorganizes the elastic search integration code so that it's
separate from the code that, for instance, reads config information from the
autotest global config. That way, it can be moved from chromite without
breaking any dependencies.

BUG=chromium:446291
TEST=Ran stats_es_functionaltest.py. Ran unit tests. Ran a butterfly-paladin
tryjob with --hwtest.

Change-Id: I0dbf135c4f1732d633e5fc9d5edb9e1f4f7199d5
Reviewed-on: https://chromium-review.googlesource.com/242701
Reviewed-by: Dan Shi <dshi@chromium.org>
Tested-by: Gabe Black <gabeblack@chromium.org>
Commit-Queue: Gabe Black <gabeblack@chromium.org>
fe/models.py
d893df09d301e00e59e75266b7af3d1c4b9a4bcf 05-Feb-2015 Mungyung Ryu <mkryu@google.com> Revert "[autotest] Send labels to shards preserving their ids."

This reverts commit 744898f7e2fe4a3d90659d2183119b0d1fddcb28.

Change-Id: I8c8611b1b3095223e527fb465c2030e743ab6152
Reviewed-on: https://chromium-review.googlesource.com/246274
Reviewed-by: Mungyung Ryu <mkryu@google.com>
Commit-Queue: Mungyung Ryu <mkryu@google.com>
Tested-by: Mungyung Ryu <mkryu@google.com>
fe/rpc_interface.py
fe/rpc_utils.py
744898f7e2fe4a3d90659d2183119b0d1fddcb28 13-Jan-2015 Prashanth Balasubramanian <beeps@google.com> [autotest] Send labels to shards preserving their ids.

We were allowing deviations between a shard and the master
by not specifying the label id for forwarded label create
rpcs. This cl breaks down the forwarding into 2 phases,
the first to create the label and the second to add the label
to the host, and sends the label id as part of the packet for
label creation.

TEST=atest label add -m host_not_on_shard new_label1
atest label add -m host_on_shard new_label2
checked that new_label2 have the same id on the master and shard.
BUG=chromium:448367
DEPLOY=apache

Change-Id: I9773def43b94ad77e85bfeadcb69bbddf9e02460
Reviewed-on: https://chromium-review.googlesource.com/240348
Tested-by: Mungyung Ryu <mkryu@google.com>
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Mungyung Ryu <mkryu@google.com>
fe/rpc_interface.py
fe/rpc_utils.py
8c98ac10beaa08bfb975c412b0b3bda23178763a 23-Dec-2014 Prashanth Balasubramanian <beeps@google.com> [autotest] Send frontend jobs to shards.

Frontend jobs on hosts that are on the shard are disallowed
currently, because the host-scheduler on master currently
ignore jobs based on meta-host, but frontend jobs have no
meta-host. This CL have the following changes:
- Make host-scheduler ignore frontend jobs that are supposed
to be picked by shard.
- Send such frontend jobs in heartbeat.
- Allow creation of frontend jobs in rpc.

TEST=Test the follows:
- Create a job on a host on shard from AFE frontend.
Observe it runs on shards and completes on master.
- Create a job on two hosts (one host on shard, the other on master)
from AFE frontend. Make sure exception is railed with correct
message.
- Run a normal dummy suite on shard, make sure normal flow still
works. Heartbeat contains the right information.
- Run a normal dummy suite on master, make sure it works.
BUG=chromium:444790
DEPLOY=apache, host-scheduler

Change-Id: Ibca3d36cb59fed695233ffdc89506364c402cc37
Reviewed-on: https://chromium-review.googlesource.com/240396
Reviewed-by: Mungyung Ryu <mkryu@google.com>
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Fang Deng <fdeng@chromium.org>
Tested-by: Fang Deng <fdeng@chromium.org>
fe/models.py
fe/models_test.py
fe/rpc_interface.py
fe/rpc_utils.py
25e1fd446390578299d7b7da0f84914a0728c5fc 19-Dec-2014 Dan Shi <dshi@chromium.org> [autotest] Add atest stable_version command to manage stable versions.

Add a decorator require_confirmation to atest, any delete action will prompt
for confirmation, use option --no-confirmation to skip that. It's applicable
to command like atest label delete, atest host delete.

Add 3 actions for topic stable_version:
list: Show version of a given board or list all boards and their stable
versions if --board option is not specified.

$ ./atest stable_version list
==============================
board | version
------------------------------
DEFAULT | R41-4687.0.0
peppy | R40-4555.0.0
==============================

modify: Set the stable version of a given board to the given value.

$ ./atest stable_version modify -b peppy -i R40-4515.0.0
Stable version for board peppy is changed from R40-4555.0.0.0 to R40-4515.0.0.

delete: Delete the stable version of a given board. So its stable version will
use the value for board `DEFAULT`.

$ ./atest stable_version delete -b peppy
Are you sure to delete stable version for board peppy? After this action is
done, stable version for board peppy will be R41.0.0.0
Continue? [y/N] y
Stable version for board peppy is deleted.

DEPLOY=apache
BUG=chromium:436656
TEST=local setup, unittest

Change-Id: I31047740a4886854aa653b1bf0f16c5f5c7a3f14
Reviewed-on: https://chromium-review.googlesource.com/236951
Tested-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
Trybot-Ready: Dan Shi <dshi@chromium.org>
fe/site_rpc_interface.py
d7bb4f16fde2f29efac95b0f7b8122b05ea4bf03 06-Jan-2015 Dan Shi <dshi@chromium.org> [autotest] Add an RPC to get a list of servers from server DB based on given role and status

BUG=chromium:446333
TEST=local rpc runs
DEPLOY=apache

Change-Id: I39b2c1c8c81b47b492d5eaa642ace234ff86c6a5
Reviewed-on: https://chromium-review.googlesource.com/238760
Trybot-Ready: Dan Shi <dshi@chromium.org>
Tested-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Fang Deng <fdeng@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
fe/site_rpc_interface.py
erver/models.py
0697ccd2453b9eafc087e1e6a7bdb289b81b45ed 05-Jan-2015 Dan Shi <dshi@chromium.org> [autotest] Use ping to verify server exists when creating server in server db.

Devservers can't use reverse lookup to loopup the hostname with IP address.

BUG=chromium:444625
TEST=add devserver to server db.

Change-Id: I387dab27f77e18348413488d6dc4bd915b42cacc
Reviewed-on: https://chromium-review.googlesource.com/238430
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
Trybot-Ready: Dan Shi <dshi@chromium.org>
Tested-by: Dan Shi <dshi@chromium.org>
erver/models.py
443f1955bd3f201be9d695ec891cb8bddb6e4699 02-Jan-2015 Fang Deng <fdeng@chromium.org> [autotest] Set limit on how many retries happen in a suite

In catastrophic cases where all the tests fail, having a lot of
retries can hurt the lab. This CL allows set a ceiling on
the number of retries that happen.

BUG=chromium:426177
TEST=Add JOB_RETRIES=2 to dummy_Fail. Run dummy suite with
1) --retry='True' -> retried twice
3) --retry='True' and --max_retries=0 -> No retry
2) --retry='True' and --max_retries=1 -> only 1 retry is run
3) --retry='True' and --max_retries=3 -> retried twice
4) --retry='False' and --max_retries=1 --> invalid option
DEPLOY=apache

Change-Id: I93af1dfe0744a21d45500e420a9e13b4c5680685
Reviewed-on: https://chromium-review.googlesource.com/238421
Reviewed-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Fang Deng <fdeng@chromium.org>
Trybot-Ready: Fang Deng <fdeng@chromium.org>
Tested-by: Fang Deng <fdeng@chromium.org>
fe/site_rpc_interface.py
bf0414d47628c2bdbfae12bf7d421791f04c6535 05-Jan-2015 Dan Shi <dshi@chromium.org> [autotest] Add new roles to server database: shard, golo_proxy

BUG=chromium:444625
TEST=None

Change-Id: Ifb696d4dfd723258c5c1de0601967b3a349d3c7a
Reviewed-on: https://chromium-review.googlesource.com/238391
Commit-Queue: Dan Shi <dshi@chromium.org>
Trybot-Ready: Dan Shi <dshi@chromium.org>
Tested-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Fang Deng <fdeng@chromium.org>
erver/models.py
63b0e4558330e57db5ed11f5a925c4769802c502 19-Dec-2014 Fang Deng <fdeng@chromium.org> [autotest] Optimize queries in abort_suites

abort_suites is called by buildbots.
The way it currently aborts suite is very slow.
This CL makes it faster by aborting the suite jobs
given a build name and a suite name, and rely on autotest
scheduler to aborts the subjobs.

TEST=Scheduler synchronous/asynchronous suites. Abort
them when the suite status is Queued/Running/Complete.
Confirm any Queued/Running subjobs are aborted.
BUG=chromium:444151
DEPLOY=apache

Change-Id: I182913e0dad08bcba8a4c05b904ba7a2d2a6c61b
Reviewed-on: https://chromium-review.googlesource.com/236943
Reviewed-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Fang Deng <fdeng@chromium.org>
Trybot-Ready: Fang Deng <fdeng@chromium.org>
Tested-by: Fang Deng <fdeng@chromium.org>
fe/doctests/001_rpc_test.txt
fe/rpc_interface.py
86248502517d42d6c036a66a57e3a24d4a6dc3b8 19-Dec-2014 Fang Deng <fdeng@chromium.org> [autotest] Serialize foreign key we don't want to follow

In some case, we don't want to follow a relationship when
serialize and de-serialize an object. However, we may
want to serialize the value of the foreign key value.

For example, we follow Host.hostattribute_set relationship to
serialize HostAttribute objects associated with a host. We
don't want to follow HostAttribute.host_id to go back to Host.

This CL allows a model to claim foreign keys that will be
serialized but will not be followed.

BUG=None
TEST=In python
>>> import common
>>> from autotest_lib.frontend import setup_django_environment
>>> from autotest_lib.frontend.afe import models
>>> models.HostAttribute.objects.filter(host_id=20)[0].serialize()
{'host_id': 20L, u'id': 203L, 'value': u'2271', 'attribute':
u'need_crash_log'}
>>> models.HostAttribute.deserialize({'host_id': 20L, u'id': 251L,
'value': u'2271', 'attribute': u'need_crash_log'})
<HostAttribute: HostAttribute object>
TEST=Test shard heartbeat works when host attributes present.
DEPLOY=apache, shard_client

Change-Id: I8e0fbb1b7eb037f0968c1599334446301ddbbcb0
Reviewed-on: https://chromium-review.googlesource.com/236708
Reviewed-by: Fang Deng <fdeng@chromium.org>
Tested-by: Fang Deng <fdeng@chromium.org>
Reviewed-by: Prashanth B <beeps@chromium.org>
Tested-by: Prashanth B <beeps@chromium.org>
Commit-Queue: Prashanth B <beeps@chromium.org>
Trybot-Ready: Prashanth B <beeps@chromium.org>
fe/model_logic.py
fe/models.py
6964fa53798525f39cd490e26481e8d923815406 18-Dec-2014 Dan Shi <dshi@chromium.org> [autotest] Use stable version from afe_stable_versions table

Add a util to get stable version from afe_stable_versions table. If the table
is not configured, return the global config value CROS.stable_cros_version.

Add an RPC get_stable_version(board) to get stable version for given board.
If board is not specified, default stable version will be returned.

Update calls in moblab_RunSuite and cros_host to get stable version through
the new RPC.

BUG=chromium:436656
DEPLOY=apache
CQ-DEPEND=CL:236571
TEST=local rpc test, run repair, confirm repair uses stable version from
afe_stable_versions table.

Change-Id: I6a22a4b471060297403cc1ae7fcbf96bec6563d9
Reviewed-on: https://chromium-review.googlesource.com/236582
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
Trybot-Ready: Dan Shi <dshi@chromium.org>
Tested-by: Dan Shi <dshi@chromium.org>
fe/models.py
fe/site_rpc_interface.py
6d49ac7fdf327a8e9e86a9b6111be2f28ebb792b 18-Dec-2014 Dan Shi <dshi@chromium.org> [autotest] Add afe_stable_versions table to store stable versions for each board.

As a sample setup:
+----+---------+----------------------------+
| id | board | version |
+----+---------+----------------------------+
| 1 | lumpy | R41-6531.0.0 |
| 2 | DEFAULT | R41-6550.0.0 |
+----+---------+----------------------------+

For lumpy, the stable build is stable_build_pattern % (lumpy, 'R41-6531.0.0').
For any other board, the stable build is
stable_build_pattern % (board, 'R41-6550.0.0')

BUG=chromium:436656
DEPLOY=migrate
TEST=local db:
database/migrate.py sync
database/migrate.py sync 96

Change-Id: Id3dbed3dd846d8ab6697b7261dc375727290c760
Reviewed-on: https://chromium-review.googlesource.com/236571
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
Trybot-Ready: Dan Shi <dshi@chromium.org>
Tested-by: Dan Shi <dshi@chromium.org>
igrations/097_add_stable_versions.py
82cf714cb9824278f368a7863e7a69318e1544d4 18-Dec-2014 Dan Shi <dshi@chromium.org> Should have used 097


Revert "[autotest] Add afe_stable_versions table to store stable versions for each board."

This reverts commit d08fcc6cb47a1be4a90a8ec4f4e3eb8cd81695b3.

Change-Id: I75edf2793d45c481c8664d2c978dac3aa1b82610
Reviewed-on: https://chromium-review.googlesource.com/236610
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
Tested-by: Dan Shi <dshi@chromium.org>
igrations/098_add_stable_versions.py
d08fcc6cb47a1be4a90a8ec4f4e3eb8cd81695b3 17-Dec-2014 Dan Shi <dshi@chromium.org> [autotest] Add afe_stable_versions table to store stable versions for each board.

As a sample setup:
+----+---------+----------------------------+
| id | board | version |
+----+---------+----------------------------+
| 1 | lumpy | R41-6531.0.0 |
| 2 | DEFAULT | R41-6550.0.0 |
+----+---------+----------------------------+

For lumpy, the stable build is stable_build_pattern % (lumpy, 'R41-6531.0.0').
For any other board, the stable build is
stable_build_pattern % (board, 'R41-6550.0.0')

BUG=chromium:436656
TEST=local db:
database/migrate.py sync
database/migrate.py sync 97

Change-Id: I94fc61e1ca18ab1cafd77bebd710a6af40f6c15e
Reviewed-on: https://chromium-review.googlesource.com/236414
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
Trybot-Ready: Dan Shi <dshi@chromium.org>
Tested-by: Dan Shi <dshi@chromium.org>
igrations/098_add_stable_versions.py
4098123fc6d2adcaf12ed6d19203228fed3b0b22 17-Dec-2014 Prashanth Balasubramanian <beeps@google.com> [autotest] Redirect reverification of hosts.

All reverifications sent to the master through the
frontend or otherwise will get redirected to hosts
on shards.

TEST=Reverified hosts through the frontend of the master.
BUG=chromium:423225
DEPLOY=apache

Change-Id: I66e242d4987a35e7aefd204d280371d3bdfb68a2
Reviewed-on: https://chromium-review.googlesource.com/236172
Reviewed-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Prashanth B <beeps@chromium.org>
Commit-Queue: Prashanth B <beeps@chromium.org>
Trybot-Ready: Prashanth B <beeps@chromium.org>
Tested-by: Prashanth B <beeps@chromium.org>
fe/rpc_interface.py
9d33e7c57959d91d49e802ce198a01215c7c5868 16-Dec-2014 Dan Shi <dshi@chromium.org> [autotest] Log RPC parameters to metaDB

This is added to troubleshooting the cause of long RPC with logging the details
of the request.

BUG=chromium:442840
TEST=local rpc call, confirm metadata is logged:
http://172.25.61.45:9200/_plugin/elastic-hammer/
search for:
{
"query": {"bool": {"minimum_should_match": 1,
"should": [{"term": {"_type": "rpc"}}]}},

"size": 100,
"sort": [{"time_recorded": "desc"}]}

expect hits like:

_source{}
params[]
id: 1527
is_complete: true
subname: "dshi_mtv.rpc"
time_recorded: 1418767090.919642
method: "get_special_tasks"
value: 0.012034893035888672
id: "jsonrpc"

Change-Id: Id0f32fcbb621341efb5d17a07348ea887f58c6f7
Reviewed-on: https://chromium-review.googlesource.com/236074
Trybot-Ready: Dan Shi <dshi@chromium.org>
Tested-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Prashanth B <beeps@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
fe/json_rpc/serviceHandler.py
0a03f9d6232427110cca384ffd8451507bc9d47b 16-Dec-2014 Dan Shi <dshi@chromium.org> [autotest] enable running special task from AFE on shard

Only raise error if the special task is queued from master AFE for a host
belongs to a shard.

BUG=None
TEST=vms
DEPLOY=apache

Change-Id: I917a2727c6c7b115c4765b2fccfc71ed8db7589f
Reviewed-on: https://chromium-review.googlesource.com/236020
Reviewed-by: Prashanth B <beeps@chromium.org>
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
Trybot-Ready: Dan Shi <dshi@chromium.org>
Tested-by: Dan Shi <dshi@chromium.org>
fe/rpc_interface.py
af5166418be945356238768cfc8226f3c2aa7138 13-Dec-2014 Prashanth Balasubramanian <beeps@google.com> [autotest] Delete shard user if it differs from master.

For most models the shard starts off with an empty database
and copies over records from the master. There is one exception,
afe_users, this is because users are proactively created when
someone navigates to the frontend. So even though we start
off with a clean afe_users table the `debug_user` will be
inserted the first time anyone visits the shard frontend. The id
of this record can be different from the id on the master.

To avoid this race, before saving any user record for the first
time check if a matching user already exists (matching id or login)
and simply delete that user before saving the new one.

TEST=Ran suites on shard.
BUG=chromium:423225
DEPLOY=apache

Change-Id: Ibe1d452bbba037c88e6525c749f27a2f1f5ca2c3
Reviewed-on: https://chromium-review.googlesource.com/235533
Tested-by: Prashanth B <beeps@chromium.org>
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Prashanth B <beeps@chromium.org>
Trybot-Ready: Prashanth B <beeps@chromium.org>
fe/model_logic.py
fe/models.py
56f1ba77e4b8b4c13d5bc72b0ebaeabda9f9d0bc 04-Dec-2014 Dan Shi <dshi@chromium.org> [autotest] Add actions to take when add/remove role from server

This CL adds actions needed to make adding/removing role from server effective.
For example, when server database is enabled and a new drone is added, scheduler
needs to be restarted.

BUG=chromium:424778
CQ-DEPEND=CL:232003
TEST=unittest, setup local server database, manually test follow cases
(CL 232525 is needed for drone test)
add scheduler: Confirm scheduler service is started in the server.
remove scheduler: Confirm scheduler service is stopped in the server.
add drone: Confirm scheduler service is restarted.
remove drone: Confirm scheduler service is restarted

Change-Id: I14d6bb15d68a9b94fa3ab5b0bcc202469b253c89
Reviewed-on: https://chromium-review.googlesource.com/233181
Tested-by: Dan Shi <dshi@chromium.org>
Trybot-Ready: Dan Shi <dshi@chromium.org>
Reviewed-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
erver/models.py
6aafd21388d8c37e3660156aaddbeb22135c0875 10-Dec-2014 Fang Deng <fdeng@chromium.org> Revert "[autotest] higher_is_better in tko_iteration_perf_value allows null"

This reverts commit 8386e7db1a9a60a4c1b7805b99f2662850a74a9c.

There was a workaround here https://chromium-review.googlesource.com/#/c/234250/

Hold back the db alteration for now.

Change-Id: If3ed332ffc82e1e0c5a565fa26910a5139fa14cf
Reviewed-on: https://chromium-review.googlesource.com/234260
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Fang Deng <fdeng@chromium.org>
Trybot-Ready: Fang Deng <fdeng@chromium.org>
Tested-by: Fang Deng <fdeng@chromium.org>
igrations/097_tko_iteration_perf_value_higher_is_better_allows_null.py
8ea0dd3d328b6c0d7cba5e365cdf068938f7b839 05-Dec-2014 Dan Shi <dshi@chromium.org> [autotest] Include a link to the shard in job page on master AFE

BUG=chromium:431783
TEST=setup local cluster in vagrant, make the code change, rebuild afe, and
confirm the job page has the link to shard.
DEPLOY=apache

Change-Id: I85dc0e1d42a4e2425711509e224b6fe37eeb788f
Reviewed-on: https://chromium-review.googlesource.com/233600
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
Trybot-Ready: Dan Shi <dshi@chromium.org>
Tested-by: Dan Shi <dshi@chromium.org>
lient/src/autotest/afe/JobDetailView.java
lient/src/autotest/public/AfeClient.html
2dc7a1026c14859ebb8d5a1477e77bdd4602c93b 03-Dec-2014 Moises Osorio <moisesosorio@chromium.org> Add new fast endpoint method to get test status counts.
This method deprecates get_tests_summary and get_tests_summary_with_wildcards
in TKO, by outperforming them and making use of more efficient Django queries.
The new method works by first getting the IDs of the Jobs matching a search
criteria and then counting the number of tests ran by such jobs with a passing
or a failing status, returning a summary that can be easily digested by the
caller.

BUG=chromium:437096
TEST=Manually on local setup of Autotest
DEPLOY=apache

Change-Id: I3a053c660774a0aa58eadbdf7d328304d539f61d
Reviewed-on: https://chromium-review.googlesource.com/232805
Reviewed-by: Prashanth B <beeps@chromium.org>
Commit-Queue: Moises Osorio <moisesosorio@google.com>
Tested-by: Moises Osorio <moisesosorio@google.com>
fe/rpc_interface.py
cbc0121831c8677920ab90ee612b6f0aa98d94b9 26-Nov-2014 Fang Deng <fdeng@chromium.org> [autotest] Drop suite_min_duts keyval when creating suite job

This is part III of making host scheduler support a suite_min_duts
requirement per suite.

With this CL, the suite_min_duts is piped from run_suite
to the rpc method create_suite_job and dropped as a job keyval.

DEPLOY=apache
TEST=Run run_suite, confirm 'suite_min_duts' is dropped
as a job keyval of the suite in the db.
BUG=chromium:432652

Change-Id: I3193aba8c0938c3e8d7b3854c19b42c3f14e2829
Reviewed-on: https://chromium-review.googlesource.com/232570
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Fang Deng <fdeng@chromium.org>
Tested-by: Fang Deng <fdeng@chromium.org>
fe/site_rpc_interface.py
722c3e77029c6533c17ae3591563be2caae6ea69 02-Dec-2014 Dan Shi <dshi@chromium.org> [autotest] Do not enable autocommit for server database

server database may not be used by Autotest. For moblab, server database is
not initialized, thus any attempt to connect to it will fail.

For server_manager script, it can always call save to push the changes to
database if needed.

BUG=chromium:430976
TEST=local start scheduler

Change-Id: I9b17a770bc0cb04e778dbf704dc63915be9fcb98
Reviewed-on: https://chromium-review.googlesource.com/232761
Commit-Queue: Dan Shi <dshi@chromium.org>
Tested-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
etup_django_environment.py
144e98d83803cba1240efc9a3f101e6686550641 29-Nov-2014 Prashanth Balasubramanian <beeps@google.com> [autotest] Don't pass the default through db config getter.

This prevents the exception and uses the default even when
there is an equivalent section in the base config.

TEST=Scheduler doesn't crash.
BUG=chromium:430976

Change-Id: Ib6e10275e31054473232bc6d17c13e3f3d4358a0
Reviewed-on: https://chromium-review.googlesource.com/232280
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Prashanth B <beeps@chromium.org>
Tested-by: Prashanth B <beeps@chromium.org>
atabase_settings_helper.py
6edaaf9f5d10cda21d1c7f2401eb1d0bef2192fa 25-Nov-2014 Prashanth Balasubramanian <beeps@google.com> [autotest] Disallow frontend jobs on hosts given to shards.

TEST=Ran jobs and checked exceptions.
Ran jobs on non-shard hosts.
BUG=chromium:431789
DEPLOY=apache
Change-Id: Ide385ed1db135a7e98ab0385df1f9a64d97bd631
Reviewed-on: https://chromium-review.googlesource.com/231735
Tested-by: Prashanth B <beeps@chromium.org>
Reviewed-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Prashanth B <beeps@chromium.org>
fe/models.py
fe/rpc_interface.py
fe/rpc_utils.py
9a535c9f3144690cf85d88a51e07b98c21454159 24-Nov-2014 Dan Shi <dshi@chromium.org> [autotest] Add server database to django model.

Server database is added to store information about servers running in an
Autotest instance. This is the first CL to implement it. Design doc:
go/chromeos-lab-serverdb-design

django model uses db_router to rout database calls to different database
connections:
local: All AFE table calls.
global: All TKO table calls.
readonly: Calls from web frontend.

This CL adds another router for all calls to server database to `server`.

DEPLOY=migrate_server_db
CQ-DEPEND=CL:230814
BUG=chromium:424778
TEST=unitest, part of the test is done by atest code that'll be in another cl.
Test server database migration:
./database/migrate.py sync 0 -d AUTOTEST_SERVER_DB
./database/migrate.py sync 1 -d AUTOTEST_SERVER_DB
./database/migrate.py sync 2 -d AUTOTEST_SERVER_DB
./database/migrate.py sync 3 -d AUTOTEST_SERVER_DB
python frontend/health/utils_unittest.py

Change-Id: I84be386c8f5b7efd53ae1ecbd6293eae4326f19f
Reviewed-on: https://chromium-review.googlesource.com/231671
Tested-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
atabase_settings_helper.py
b_router.py
erver/__init__.py
erver/common.py
erver/models.py
ettings.py
etup_django_environment.py
etup_test_environment.py
75be1d3f881ef4f4f9cffe0c38fc3139338d8f84 26-Nov-2014 Prashanth Balasubramanian <beeps@google.com> [autotest] Wrap heartbeat operations in transactions.

Serializing/deserializing records in and out of the database
needs to be atomic or the scheduler will get confused. Examples
are cases where a job with a metahost is deemed hostless because
of the deserialization delay, or a complete job is thought to be
running because of the delay in setting the status.

TEST=Ran suites across shards.
BUG=chromium:423225
DEPLOY=apache

Change-Id: I058babf3b70399eeb6baa4bf331f962ed2dc4e00
Reviewed-on: https://chromium-review.googlesource.com/232000
Reviewed-by: Prashanth B <beeps@chromium.org>
Tested-by: Prashanth B <beeps@chromium.org>
Commit-Queue: Prashanth B <beeps@chromium.org>
fe/model_logic.py
6a4cf3797efec66575cc388735259e8791d03be1 25-Nov-2014 Prashanth Balasubramanian <beeps@google.com> [autotest] Include the port from the global_config, default to ''

On vms we have an extra global_db_port setting. It is left undefined
in normal shadow_configs.

TEST=Ran suites.
BUG=None
DEPLOY=apache

Change-Id: I25230b05512516b60102812d8337d602219c9d95
Reviewed-on: https://chromium-review.googlesource.com/231726
Tested-by: Prashanth B <beeps@chromium.org>
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Prashanth B <beeps@chromium.org>
atabase_settings_helper.py
5949b4af7a872aeb58e7ad29090812d648725ed5 23-Nov-2014 Prashanth Balasubramanian <beeps@google.com> [Autotest] Allow the syncing of labels added/removed.

This allows us to perform label add/remove actions
on the master and have it pass down to the shards those hosts
are on by introducing a generic decorator.

The cl also fixes some logging issues with the shard-client,
and a bug with propogating the invalid bit of a host.

TEST=Added/removed labels.
BUG=chromium:431786
DEPLOY=apache

Change-Id: Ic176054e04a5508bb3e967ecc902422628446d32
Reviewed-on: https://chromium-review.googlesource.com/231558
Reviewed-by: Prashanth B <beeps@chromium.org>
Tested-by: Prashanth B <beeps@chromium.org>
Commit-Queue: Prashanth B <beeps@chromium.org>
fe/doctests/001_rpc_test.txt
fe/models.py
fe/rpc_interface.py
fe/rpc_interface_unittest.py
fe/rpc_utils.py
8a764d1fb6cd99d08cda6400bd35162fa3ac5fe0 15-Oct-2014 Jakob Juelich <jakobjuelich@chromium.org> [autotest] Use global database for tko models in django v3.

This adds the global database to django.
A django database router is added to determine to which database
should be used for which models. All tko models are always taken or
written from or into the global database while all other objects
remain unchanged.

BUG=chromium:422637
TEST=Ran suites, syncdb, restart apache, ran a suite.
Confirm global database connection is closed after each attempt to get
test view: https://x20web.corp.google.com/~beeps/log/db_routers.html

Change-Id: Idf6933d1d112bbc5a2896fa61afd03f6604dafb5
Reviewed-on: https://chromium-review.googlesource.com/223501
Tested-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Prashanth B <beeps@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
b_router.py
ettings.py
etup_django_environment.py
etup_test_environment.py
abe3bb7d47369bc734ca5a68b0919db0d944e370 20-Nov-2014 Prashanth Balasubramanian <beeps@google.com> [Autotest] Time the staging of control files.

To debug some apache slowness this cl times the staging/parsing of
control files. This operation is the main suspect for the slowness
of the create_suite_job rpc.

TEST=Executed the rpc.
BUG=None
DEPLOY=apache

Change-Id: I3c3892054a33a52dd31d467567b09862395dc91b
Reviewed-on: https://chromium-review.googlesource.com/231079
Reviewed-by: Prashanth B <beeps@chromium.org>
Commit-Queue: Prashanth B <beeps@chromium.org>
Tested-by: Prashanth B <beeps@chromium.org>
fe/site_rpc_interface.py
fe/site_rpc_interface_unittest.py
8386e7db1a9a60a4c1b7805b99f2662850a74a9c 10-Nov-2014 Fang Deng <fdeng@chromium.org> [autotest] higher_is_better in tko_iteration_perf_value allows null

Allow higher_is_better to be set to None when sending
perf data to dashboard. Perf dashboard will automatically
detect whether "higher is better" or "lower is better" using
its defaults for the units.

TEST=Run a perf test that outputs higher_is_better=None
sync up: migrate.py sync
sync down: migrate.py sync 96
DEPLOY= migrate
BUG=None

Change-Id: I51502525661d369b14b8f5667ad12d4316db0d41
Reviewed-on: https://chromium-review.googlesource.com/228780
Reviewed-by: Ilja Friedel <ihf@chromium.org>
Tested-by: Fang Deng <fdeng@chromium.org>
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Fang Deng <fdeng@chromium.org>
igrations/097_tko_iteration_perf_value_higher_is_better_allows_null.py
05e94c488c55bf764f27e9c112d1dda6f03d2d7d 10-Nov-2014 Prashanth Balasubramanian <beeps@google.com> [autotest] AFE modifications to show a link to the shard a host is on.

Directly embeds the url to the host on the shard, in the page for the
host on the master.

TEST=Ran afe.
BUG=chromium:423225,chromium:425347
DEPLOY=afe,apache

Change-Id: I0f6b0a350b375f78c616fe19cbc7699a5dfde0b9
Reviewed-on: https://chromium-review.googlesource.com/228747
Tested-by: Prashanth B <beeps@chromium.org>
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Prashanth B <beeps@chromium.org>
lient/src/autotest/afe/HostDetailView.java
lient/src/autotest/public/AfeClient.html
de87dea6a5a82b4ce1ddc5ce881ddbc70b54fdda 10-Nov-2014 Prashanth Balasubramanian <beeps@google.com> [autotest] Include the time created as a uid in task results.

Since tasks will run directly on shards there is some chance
that a shard will failover and its successor clobbers the
special task logs of the dead shard in google storage.

To avoid this scenario, include the creation time of the
task as a uid in the path to the logs. Unless one of the
shards is living in the past these timestamps should never
repeat. Moreover, we take measures against clock drift (ntpd)
and if a shard has the wrong system time configuration all its
jobs will timeout anyway, since each job has a TTL based off
time_created which is set on the master.

TEST=Ran suites.
BUG=chromium:423225,chromium:425347
DEPLOY=scheduler,apache

Change-Id: Ia23ac8fd721f53fbb9b475c8eb9f8d25e4fd1c2f
Reviewed-on: https://chromium-review.googlesource.com/228781
Tested-by: Prashanth B <beeps@chromium.org>
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Prashanth B <beeps@chromium.org>
fe/models.py
042c1479b8df4f2d87586815d7ad0646cc39abeb 23-Oct-2014 Fang Deng <fdeng@chromium.org> [autotest] record autoserv run time in metadata db.

Measure autoserv run time and send it to metadata db.
The job id (or special task id) is extracted from the
result directory path using regular expression.

TEST=1)locally run autoserv. 2) Run run_suite.
Made sure that all the special tasks have been excercised.
3) run test_that
BUG=chromium:426105,chromium:422581

Change-Id: I16b119dbcf9b163967abed09a303d86568ea1394
Reviewed-on: https://chromium-review.googlesource.com/225236
Reviewed-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Fang Deng <fdeng@chromium.org>
Tested-by: Fang Deng <fdeng@chromium.org>
fe/rdb_model_extensions.py
e1dd2d1ebc59c8a0d5eec339ae1c33fdb0873367 24-Oct-2014 Rohit Makasana <rohitbm@chromium.org> Moving ChromeOS telemetry tests from ChromiumPerf to ChromeOSPerf master

So we don't have to maintain multiple masters for ChromeOS.
And ChromiumPerf can be only used by the chomium.perf buildbot.

TEST=None
BUG=None

Change-Id: Idd84453ad95961223240843f0067c5845f826f24
Reviewed-on: https://chromium-review.googlesource.com/225443
Reviewed-by: Quinten Yearsley <qyearsley@google.com>
Commit-Queue: Rohit Makasana <rohitbm@chromium.org>
Tested-by: Rohit Makasana <rohitbm@chromium.org>
erf-dashboard/croschart_defaults.json
6285f6acffd9ed3176cdc89544944a87340ac0b1 09-May-2014 Prashanth B <beeps@google.com> [autotest] Retry when staging a control files fails in run-suite.

The cl also adds some stats for reporting run_suite failures, and
sneaks in some autoserv testing changes.

TEST=Ran run_suite against a bad devserver, raised the exception.
BUG=chromium:418928, chromium:359741, chromium:371644

Change-Id: If49a3b96c053432cb26a01a02e160176be37c037
Reviewed-on: https://chromium-review.googlesource.com/220973
Reviewed-by: Prashanth B <beeps@chromium.org>
Tested-by: Prashanth B <beeps@chromium.org>
Commit-Queue: Prashanth B <beeps@chromium.org>
fe/site_rpc_interface.py
fe/site_rpc_interface_unittest.py
02e6129187328bd3edd96dae8062e6a19ab7936c 17-Oct-2014 Jakob Juelich <jakobjuelich@chromium.org> [autotest] Allow job updates from shard without shard_id.

In the following scenario currently an exception is raised:
A job was synced from the master to a shard. It was then aborted
on the master, while the shard was executing it. The abortion
sets the shard_id of the job back to null (that happens always
when a job is set to a completed status).
The there was no heartbeat between the abortion and the completion
of the job on the shard.

The next heartbeat will find the completed job on the shard and will
try uploading it to the master. The sanity check if the job is a
assigned to the shard it was uploaded from will fail, as the shard
id is set to null on the master.

This changes this behavior: The sanity check will not fail if
the shard doesn't have a shard_id set.

BUG=None
DEPLOY=afe
TEST=Ran Suites.

Change-Id: Ifd8eafd3898e7fadab6e52e48bc033acd5ae1dd5
Reviewed-on: https://chromium-review.googlesource.com/224110
Tested-by: Jakob Jülich <jakobjuelich@chromium.org>
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Jakob Jülich <jakobjuelich@chromium.org>
fe/models.py
fe/site_rpc_interface_unittest.py
1b52574752be108a743d3b33561c34324f8538e7 30-Sep-2014 Jakob Juelich <jakobjuelich@chromium.org> [autotest] Retransmit hosts and jobs if heartbeat failed on client.

If the client fails while performing a heartbeat, the master will
have already set the shard_id and therefore won't ever send jobs or
hosts again.

This changes this behavior: The shard client sends ids of hosts and
incomplete jobs it already has in the heartbeat.
Objects with id's that weren't sent, will be returned by the
heartbeat regardless of already having a shard_id set.

BUG=None
DEPLOY=apache
TEST=Ran suites

Change-Id: I46bbb13a81886476ec48c6f879f123290769b659
Reviewed-on: https://chromium-review.googlesource.com/220692
Tested-by: Jakob Jülich <jakobjuelich@chromium.org>
Reviewed-by: Prashanth B <beeps@chromium.org>
Commit-Queue: Jakob Jülich <jakobjuelich@chromium.org>
fe/models.py
fe/rpc_utils.py
fe/site_rpc_interface.py
fe/site_rpc_interface_unittest.py
c5fcff6e13beb99165930240c1f008a07974b136 15-Oct-2014 Jakob Juelich <jakobjuelich@chromium.org> [autotest] Simplify access to readonly_connection

After refactoring the way readonly_connection works in
CL:223393 some uses of it can be shortened.

BUG=None
DEPLOY=apache,scheduler,host_scheduler,shard_client
TEST=Ran suites, syncdb, apache restart and dummy suite.

Change-Id: I461345ecb362bd54740417f0016a64072581a513
Reviewed-on: https://chromium-review.googlesource.com/223394
Tested-by: Jakob Jülich <jakobjuelich@chromium.org>
Reviewed-by: Prashanth B <beeps@chromium.org>
Commit-Queue: Jakob Jülich <jakobjuelich@chromium.org>
roschart/labtest/models.py
roschart/perfchart/models.py
roschart/releasereport/models.py
rontend_unittest.py
ko/graphing_utils.py
ko/models.py
ko/rpc_interface.py
89db1e40495e2f2d863c293f1b5c15044ac4d707 10-Oct-2014 Jakob Juelich <jakobjuelich@chromium.org> [autotest] Fail on adding second shard for a board.

One board should only be assigned to one shard.

This ensures an exception is raised, if multiple shards are
assigned to one label.

BUG=None
DEPLOY=apache,migrate,scheduler,host_scheduler,shard_client
TEST=Ran Suites.

Change-Id: I86e7f4b77d7334885e2e12de2f4e0708f0a30384
Reviewed-on: https://chromium-review.googlesource.com/222440
Tested-by: Jakob Jülich <jakobjuelich@chromium.org>
Reviewed-by: Prashanth B <beeps@chromium.org>
Commit-Queue: Jakob Jülich <jakobjuelich@chromium.org>
igrations/096_only_one_shard_per_label.py
7bef8416138f662c39326bdb6f8a9b548c3d4b45 15-Oct-2014 Jakob Juelich <jakobjuelich@chromium.org> [autotest] Refactor readonly_connection

The way readonly connections were used used
some monkey patching and manual connection handling.

Code gets easier to maintain if Django manages all the
database connections and no monkeypatching is done.
This is what this CL achieves.

BUG=chromium:422637
DEPLOY=apache,scheduler,host_scheduler,shard_client
TEST=Ran suites.

Change-Id: I2761af10f5172bb3558e7dbd7c1f1875f7a7cb51
Reviewed-on: https://chromium-review.googlesource.com/223393
Tested-by: Jakob Jülich <jakobjuelich@chromium.org>
Reviewed-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Jakob Jülich <jakobjuelich@chromium.org>
fe/model_logic.py
fe/readonly_connection.py
ettings.py
etup_django_lite_environment.py
etup_test_environment.py
852ec0d282735523b140e9d42c2ef3e36d1baf24 16-Sep-2014 Jakob Juelich <jakobjuelich@chromium.org> [autotest] Re-sync aborted jobs to shards

If a job is aborted on the master, the shards need to know, so
they can abort the job.

With this commit jobs that are already synced but have been aborted
in the meantime will be re-synced.

BUG=None
TEST=Ran suites

Change-Id: I077400047ba51b19151035f0f58c585959ceba7f
Reviewed-on: https://chromium-review.googlesource.com/218390
Tested-by: Jakob Jülich <jakobjuelich@chromium.org>
Reviewed-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Jakob Jülich <jakobjuelich@chromium.org>
fe/models.py
fe/site_rpc_interface_unittest.py
934f0dca520ebf8eeebaa7befa484408281e0f91 15-Oct-2014 Jakob Juelich <jakobjuelich@chromium.org> [autotest] Move database configuration to separate file.

As we're about to add more databases the logic for parsing the
database configurations gets more complicated.

This centralizes the configuration parsing in one dedicated file.

BUG=chromium:419435
DEPLOY=apache,scheduler,host_scheduler,shard_client
TEST=Ran suites, tried syncdb and apache restart

Change-Id: I486b72f30ccd0f1e44316927b4fe3d5c93eacbe2
Reviewed-on: https://chromium-review.googlesource.com/223377
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Jakob Jülich <jakobjuelich@chromium.org>
Tested-by: Jakob Jülich <jakobjuelich@chromium.org>
atabase_settings_helper.py
ettings.py
a0acfbcb08a96fbe13eeb68203e119f71f1ce56c 15-Oct-2014 Dan Shi <dshi@chromium.org> [autotest] log the user that lock/unlock a dut to metaDB

BUG=None
TEST=local setup, lock and unlock couple times, then check metaDB to see the
data.
http://172.25.61.45:9200/_plugin/elastic-hammer/
with query:
{"query": {"bool": {"minimum_should_match": 1,
"should": [{"term": {"_type": "lock_history"}}]}},
"size": 100,
"sort": [{"time_recorded": "desc"}]}
Confirm response has entries like:
changed_by: "debug_user"
hostname: "172.27.213.193"
locked: true
time_recorded: 1413327364.848333

Change-Id: I643d5aefa37494170000729e2083dad799e1b8f5
Reviewed-on: https://chromium-review.googlesource.com/223371
Commit-Queue: Dan Shi <dshi@chromium.org>
Tested-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Fang Deng <fdeng@chromium.org>
fe/model_logic.py
fe/models.py
8b110eeff624e9724c1b717394d0828b3371221f 16-Sep-2014 Jakob Juelich <jakobjuelich@chromium.org> [autotest] Manage shards with atest

This adds functionality to list, create and delete shards to atest.

BUG=None
TEST=Ran suites, manual test of create/list/delete.

Change-Id: I0771d0c1b46c7c6890819822204f42ac2211b104
Reviewed-on: https://chromium-review.googlesource.com/218295
Tested-by: Jakob Jülich <jakobjuelich@chromium.org>
Reviewed-by: Prashanth B <beeps@chromium.org>
Commit-Queue: Jakob Jülich <jakobjuelich@chromium.org>
fe/site_rpc_interface.py
3270e189d5da845d034231a6eb75c5bd4e37ac97 13-Oct-2014 Jakob Juelich <jakobjuelich@chromium.org> Revert "[autotest] Always use tko tables on master from shards."

This reverts commit e2bf396b7d3870f7565b2b10018087ab7fa6256e.

BUG=422637
CQ-DEPEND=CL:223065

Change-Id: I0513889a3acfc3a189843e79030da9bbd551999d
Reviewed-on: https://chromium-review.googlesource.com/223072
Reviewed-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Jakob Jülich <jakobjuelich@chromium.org>
Tested-by: Jakob Jülich <jakobjuelich@chromium.org>
b_router.py
ettings.py
etup_test_environment.py
a204088d40d4fd1ecfd8f219f723b72d8ebe391c 13-Oct-2014 Jakob Juelich <jakobjuelich@chromium.org> Revert "[autotest] Always use tko tables on master from shards."

This reverts commit 8aff81d0334c05c5e7b8d6c3f1dcdc2b1ef21c29.

BUG=422637
CQ-DEPEND=CL:223072

Change-Id: I897828cd665b265c2518c08cbe4c55ea6edb64b6
Reviewed-on: https://chromium-review.googlesource.com/223065
Reviewed-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Jakob Jülich <jakobjuelich@chromium.org>
Tested-by: Jakob Jülich <jakobjuelich@chromium.org>
atabase_settings_helper.py
ettings.py
8aff81d0334c05c5e7b8d6c3f1dcdc2b1ef21c29 02-Oct-2014 Jakob Juelich <jakobjuelich@chromium.org> [autotest] Always use tko tables on master from shards.

tko/db.py was changed to always use settings for the global database.
This added configuration overhead for non-shard machines. This adds
the normal, old settings as a failover.

Also, this adds the global database to django.
A django database router is added to determine to which database
should be used for which models. All tko models are always taken or
written from or into the global database while all other objects
remain unchanged.

This is a re-submit of CL:221018 and contains a fix for the import
error on the builders.

BUG=chromium:419435
TEST=Ran suites and manually tried rpcs.
DEPLOY=apache

Change-Id: Id470ae189231384eda8aa3e78aa97f8dde2548e1
Reviewed-on: https://chromium-review.googlesource.com/222286
Reviewed-by: Prashanth B <beeps@chromium.org>
Reviewed-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Jakob Jülich <jakobjuelich@chromium.org>
Tested-by: Jakob Jülich <jakobjuelich@chromium.org>
Reviewed-by: Dan Shi <dshi@chromium.org>
atabase_settings_helper.py
ettings.py
7a2734784ed059b3d8d5a09cb2dd6edab47e411e 08-Oct-2014 Michael Spang <spang@chromium.org> Revert "[autotest] Don't use db query timeout in django settings."

To fix broken AUTests.

This reverts commit 87c1551f9a6919e5f28b20f26fe6873aaecb4e02.

BUG=chromium:419435
TEST=none

Change-Id: I2dafdb33981fee032fc626ba41e995c8311e7eeb
Reviewed-on: https://chromium-review.googlesource.com/221499
Reviewed-by: Michael Spang <spang@chromium.org>
Commit-Queue: Michael Spang <spang@chromium.org>
Tested-by: Michael Spang <spang@chromium.org>
ettings.py
87c1551f9a6919e5f28b20f26fe6873aaecb4e02 08-Oct-2014 Jakob Juelich <jakobjuelich@chromium.org> [autotest] Don't use db query timeout in django settings.

CL:221018 introduced taking over django's database settings for
tko/db.py. Unfortunately timeouts have different meanings for
tko/db.py and django. While they are a maximum query runtime for
the first, they are a cache invalidation timeout for the latter.

This fixes this.

BUG=None
DEPLOY=tko,apache
TEST=Ran suites, tried syncdb, tried restarting afe.

Change-Id: I6b7b805da030d323ac96683b66d04ca446902965
Reviewed-on: https://chromium-review.googlesource.com/222042
Reviewed-by: Prashanth B <beeps@chromium.org>
Commit-Queue: Jakob Jülich <jakobjuelich@chromium.org>
Tested-by: Jakob Jülich <jakobjuelich@chromium.org>
ettings.py
e2bf396b7d3870f7565b2b10018087ab7fa6256e 02-Oct-2014 Jakob Juelich <jakobjuelich@chromium.org> [autotest] Always use tko tables on master from shards.

tko/db.py was changed to always use settings for the global database.
This added configuration overhead for non-shard machines. This adds
the normal, old settings as a failover.

Also, this adds the global database to django.
A django database router is added to determine to which database
should be used for which models. All tko models are always taken or
written from or into the global database while all other objects
remain unchanged.

BUG=chromium:419435
TEST=Ran suites and manually tried rpcs.
DEPLOY=apache,tko

Change-Id: I675b064ffdcb0440cd33835b9a83e3858f826ca7
Reviewed-on: https://chromium-review.googlesource.com/221018
Tested-by: Jakob Jülich <jakobjuelich@chromium.org>
Reviewed-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Jakob Jülich <jakobjuelich@chromium.org>
b_router.py
ettings.py
etup_test_environment.py
50e91f7aa792af0a5f25ab718027de59963f4790 01-Oct-2014 Jakob Juelich <jakobjuelich@chromium.org> [autotest] Forward calls to modify_host(s) to shards.

If a host gets locked after it was assigned to a shard, right now
only the master will know about it.

With this commit, calls to modify_host and modify_hosts will be
forwarded to the relevant shards, so locking of hosts will propagate.

BUG=None
DEPLOY=apache
TEST=Ran suites and called rpcs manually, also with failing shard rpcs.

Change-Id: I90ca34a4cefbdf55acd47ee6e8df872527b27285
Reviewed-on: https://chromium-review.googlesource.com/220850
Tested-by: Jakob Jülich <jakobjuelich@chromium.org>
Reviewed-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Jakob Jülich <jakobjuelich@chromium.org>
fe/doctests/001_rpc_test.txt
fe/rpc_interface.py
fe/rpc_interface_unittest.py
fe/rpc_utils.py
acf359283e28219cc42ff3fe82b8a53f6a2d90f3 03-Oct-2014 MK Ryu <mkryu@google.com> [autotest] Add a new RPC interface, get_host_attribute.

Retrieve a host's attribute from afe_host_attributes table.
When you need to get only attribute info of a host, this RPC call is
lighter than using get_hosts RPC call.

DEPLOY=apache
BUG=chromium:215160
TEST=AUTOTEST_ROOT$ python
>>import common
>>from autotest_lib.server.cros.dynamic_suite import tools, frontend_wrappers
>>afe = frontend_wrappers.RetryingAFE(timeout_min=5, delay_sec=10)
>>afe.get_host_attribute('need_crash_logs', hostname=<host ip>)

Change-Id: I86bc17091c078823c270206d80bd572f2fbbeb95
Reviewed-on: https://chromium-review.googlesource.com/221510
Reviewed-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Mungyung Ryu <mkryu@google.com>
Tested-by: Mungyung Ryu <mkryu@google.com>
fe/models.py
fe/rpc_interface.py
4afcd934f31ddf1f9df8b7de20072e14783f8f25 06-Oct-2014 Richard Barnette <jrbarnette@chromium.org> Revert "Dead code removal."

Alas, there are references to gpu_hang in suite_scheduler.ini,
which is causing a flood of e-mail complaints.

This reverts commit ed6c6c8ab133e04996003dbfadab046fd8fb8fc4.

Change-Id: I96dbf79288ce293e0fc7fb7c31d2c18d560c418a
Reviewed-on: https://chromium-review.googlesource.com/221699
Reviewed-by: Richard Barnette <jrbarnette@chromium.org>
Tested-by: Richard Barnette <jrbarnette@chromium.org>
erf-dashboard/common.py
erf-dashboard/croschart_defaults.json
erf-dashboard/doc/default.html
erf-dashboard/extract_perf.py
erf-dashboard/generate_perf_graphs.py
erf-dashboard/templates/branch_platform_overview.html
erf-dashboard/templates/compare_by_overview.html
erf-dashboard/templates/config.js
erf-dashboard/templates/overview_base.html
ed6c6c8ab133e04996003dbfadab046fd8fb8fc4 01-Oct-2014 Ilja H. Friedel <ihf@chromium.org> Dead code removal.

BUG=None.
TEST=None.

Change-Id: I3d3c21a4cabea2e5c38418f15d82be3b60c06ec2
Reviewed-on: https://chromium-review.googlesource.com/220690
Reviewed-by: Fang Deng <fdeng@chromium.org>
Tested-by: Ilja Friedel <ihf@chromium.org>
Commit-Queue: Ilja Friedel <ihf@chromium.org>
erf-dashboard/common.py
erf-dashboard/croschart_defaults.json
erf-dashboard/doc/default.html
erf-dashboard/extract_perf.py
erf-dashboard/generate_perf_graphs.py
erf-dashboard/templates/branch_platform_overview.html
erf-dashboard/templates/compare_by_overview.html
erf-dashboard/templates/config.js
erf-dashboard/templates/overview_base.html
f865d33cc3b0584f10b2f6f4de414a6d7f9e3000 29-Sep-2014 Jakob Juelich <jakobjuelich@chromium.org> [autotest] Allow updating selected model attributes

Right now, objects that have been persisted on the client once,
will never be overwritten again with new data, that comes from
the master.

In order to sync i.e. aborted bits from the master to the slave,
this needs to change.

This introduces a new variable SERIALIZATION_LOCAL_LINKS_TO_UPDATE,
which objects can overwrite with a set of names of attributes,
that should be overwritten with data that's sent from the master.

BUG=None
DEPLOY=shard_client
TEST=Ran suites.

Change-Id: I6b461475116db0acd2de7b8fe628aed791e7c418
Reviewed-on: https://chromium-review.googlesource.com/220331
Tested-by: Jakob Jülich <jakobjuelich@chromium.org>
Reviewed-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Jakob Jülich <jakobjuelich@chromium.org>
fe/model_logic.py
fe/models.py
fe/models_test.py
8421d5905ab0aed8689c2eea6be8d9c4042ce618 18-Sep-2014 Jakob Juelich <jakobjuelich@chromium.org> [autotest] Sync completed jobs back to master

Records of jobs that are completed should be send back to the master
upon a heartbeat.

This changes the heartbeat to include these things.

BUG=None
CQ-DEPEND=CL:212725
DEPLOY=apache,afe
TEST=Ran suites, tried out manually

Change-Id: I25daed310dc1bad064bd0c4819a43d18ba04606d
Reviewed-on: https://chromium-review.googlesource.com/218730
Tested-by: Jakob Jülich <jakobjuelich@chromium.org>
Reviewed-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Jakob Jülich <jakobjuelich@chromium.org>
fe/models.py
ab666c82867d0707f1d2cb6eaa4a077e351394bb 24-Sep-2014 Moises Osorio <moisesosorio@chromium.org> Add new endpoint to get the tests summary allowing wildcards to match up job names

BUG=None
TEST=Manually on local setup of Autotest
Change-Id: I6225e01fd0b68cca9918fa9575257bd87b25732f
Reviewed-on: https://chromium-review.googlesource.com/219850
Reviewed-by: Fang Deng <fdeng@chromium.org>
Reviewed-by: Nick Matthijssen <nickam@chromium.org>
Tested-by: Jesus Moises Osorio <moisesosorio@chromium.org>
Commit-Queue: Jesus Moises Osorio <moisesosorio@chromium.org>
ko/rpc_interface.py
a94efe60bc94c9aa10ecfe40bddf97518985c7c2 19-Sep-2014 Jakob Juelich <jakobjuelich@chromium.org> [autotest] Endpoints for syncing jobs and hqes back to master

The status of jobs that were completed on a shard should be visible
in the master AFE, so a human can easily see which jobs have been run in one
central place.

This adds the functionality to the AFE to retrieve newer versions of
jobs and hqes from shards.

This is a newer version of CL:213901 which was reverted because of
http://crbug.com/418022.

BUG=418022
DEPLOY=apache, afe
TEST=Ran Suites

Change-Id: I3c9a35c78585a43d193874decbb7b16216b2c69a
Reviewed-on: https://chromium-review.googlesource.com/220141
Reviewed-by: Fang Deng <fdeng@chromium.org>
Tested-by: Jakob Jülich <jakobjuelich@chromium.org>
Commit-Queue: Jakob Jülich <jakobjuelich@chromium.org>
fe/json_rpc/proxy.py
fe/model_logic.py
fe/models.py
fe/rpc_utils.py
fe/site_rpc_interface.py
fe/site_rpc_interface_unittest.py
e5482e5614e47a1eda238902eaa83e16a0f5475d 26-Sep-2014 Owen Lin <owenlin@chromium.org> Revert "[autotest] Endpoints for syncing jobs and hqes back to master"

This reverts commit 1b22ff2921c65b45e3727a8aaa852325f0f0cfb4.

BUG=418022

Change-Id: I30473d21d16911f620ee7f06917c13e6017f20fb
Reviewed-on: https://chromium-review.googlesource.com/219918
Reviewed-by: Owen Lin <owenlin@chromium.org>
Commit-Queue: Owen Lin <owenlin@chromium.org>
Tested-by: Owen Lin <owenlin@chromium.org>
fe/json_rpc/proxy.py
fe/model_logic.py
fe/models.py
fe/rpc_utils.py
fe/site_rpc_interface.py
fe/site_rpc_interface_unittest.py
1b22ff2921c65b45e3727a8aaa852325f0f0cfb4 19-Sep-2014 Jakob Juelich <jakobjuelich@chromium.org> [autotest] Endpoints for syncing jobs and hqes back to master

The status of jobs that were completed on a shard should be visible
in the master AFE, so a human can easily see which jobs have been run in one
central place.

This adds the functionality to the AFE to retrieve newer versions of
jobs and hqes from shards.

BUG=None
DEPLOY=apache, afe
TEST=Ran Suites

Change-Id: I04c3401921a3f54bae6070639af805904952bf6a
Reviewed-on: https://chromium-review.googlesource.com/213901
Reviewed-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Jakob Jülich <jakobjuelich@chromium.org>
Tested-by: Jakob Jülich <jakobjuelich@chromium.org>
fe/json_rpc/proxy.py
fe/model_logic.py
fe/models.py
fe/rpc_utils.py
fe/site_rpc_interface.py
fe/site_rpc_interface_unittest.py
82b7d1c262ff19d01ed0bc8c02911d02fbc2a148 16-Sep-2014 Jakob Juelich <jakobjuelich@chromium.org> [autotest] Add AFE endpoints for shard management

This adds endpoints to the AFE to create, delete and list shards.

Especially sensitive is the delete operation: All hosts and jobs
are claimed back from the shard to the master.
Hosts will be rebooted and reverified using repair. This is to ensure
no processes from the shards are still running on the DUTs, as they
could potentially interfer with upcoming tests.

BUG=None
DEPLOY=apache
TEST=Ran suites

Change-Id: Idf887452d62f37ae62771d42b7661c09e72d9a08
Reviewed-on: https://chromium-review.googlesource.com/218294
Reviewed-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Jakob Jülich <jakobjuelich@chromium.org>
Tested-by: Jakob Jülich <jakobjuelich@chromium.org>
fe/site_rpc_interface.py
fe/site_rpc_interface_unittest.py
116ff0fc2674082fe475afd64ce4dec998ed71b7 18-Sep-2014 Jakob Juelich <jakobjuelich@chromium.org> [autotest] Deserializer: Allow updating objects

Currently the state of an object is never changed by the
deserializer.

After a shard executed a job, the master should be informed
about it's result so one can see if a job has been executed or not
in one central place for all jobs.

To update a job's status on the master, this is necessary.

Doing this, a bug of the previous implementation became apparent:
Deserializing a host object while no AclGroups were present yet
would create an Everyone-AclGroup in it's save() method. This is
dangerous because this way the states of master and shards might
diverge, specifically AclGroups with same names and different ids
might occur, which will then, upon dumping the Everyone-AclGroup from
the master, will cause exceptions to be raised.

Therefore just not running them prevents errors.

BUG=None
DEPLOY=scheduler,apache,host_scheduler
TEST=Ran suites, tested with real data on shard

Change-Id: Ie8ccd3205ac0efd5bbba4402da5de4c2615b495c
Reviewed-on: https://chromium-review.googlesource.com/218806
Reviewed-by: Prashanth B <beeps@chromium.org>
Commit-Queue: Jakob Jülich <jakobjuelich@chromium.org>
Tested-by: Jakob Jülich <jakobjuelich@chromium.org>
fe/model_logic.py
fe/models.py
fe/models_test.py
3b27dbc2358aef655e050a92510ff8e9e080bf81 04-Sep-2014 Jakob Juelich <jakobjuelich@chromium.org> [autotest] Fetching jobs from global AFE to moblab

To scale autotest with sharding jobs and hosts need to be fetched by
the shards. This adds the fetching-functionality on the shard side.

TEST=Ran suites and tried scheduling dummy on global and fetching it
DEPLOY=apache

Change-Id: I54ac73e07027e36982e18eb606cdf5958c8c6c89
Reviewed-on: https://chromium-review.googlesource.com/212725
Reviewed-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Jakob Jülich <jakobjuelich@chromium.org>
Tested-by: Jakob Jülich <jakobjuelich@chromium.org>
fe/site_rpc_interface_unittest.py
7745757652d220bb746fedfdb6158d8033c288d6 23-Sep-2014 Jakob Juelich <jakobjuelich@chromium.org> [autotest] Don't automatically create shards in heartbeats

Shards are added and assigned to labels using atest. If they
already exist this can not be done. Therefore they must not
automatically be created on their first heartbeat. Also it makes
more sense to fail instead of silently accepting unknown heartbeats.

BUG=None
DEPLOY=apache
TEST=Ran suites.

Change-Id: Id485ad956c83c16dc96916ba58e25c6f91028dd8
Reviewed-on: https://chromium-review.googlesource.com/219423
Reviewed-by: Alex Miller <milleral@chromium.org>
Commit-Queue: Jakob Jülich <jakobjuelich@chromium.org>
Tested-by: Jakob Jülich <jakobjuelich@chromium.org>
fe/rpc_utils.py
fe/site_rpc_interface_unittest.py
59cfe5497d7f3be2bc4ade144d649a131e88448e 03-Sep-2014 Jakob Juelich <jakobjuelich@chromium.org> [autotest] Add heartbeat AFE endpoint to shard Autotest

To improve the workload the autotest setup can handle, jobs should
be executed by multiple shards. They are sharded by type of board.

This a HTTP accessible endpoint to AFE. This assigns jobs and hosts
to moblabs and returns them as records to insert into a shard's
local database.

TEST=Ran suites and tried manually retrieving jobs
DEPLOY=scheduler,apache,host-scheduler

Change-Id: I20ac41364e4c67e17883729181af798e18d8093e
Reviewed-on: https://chromium-review.googlesource.com/217968
Reviewed-by: Jakob Jülich <jakobjuelich@chromium.org>
Reviewed-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Jakob Jülich <jakobjuelich@chromium.org>
Tested-by: Jakob Jülich <jakobjuelich@chromium.org>
fe/json_rpc/serviceHandler.py
fe/json_rpc/serviceHandler_unittest.py
fe/models.py
fe/rpc_utils.py
fe/site_rpc_interface.py
fe/site_rpc_interface_unittest.py
2dda22e6957717396c50c8bc20843c78a3109b7d 17-Sep-2014 Moises Osorio <moisesosorio@chromium.org> Add an endpoint that summarizes the number of passed and failed tests per suite.

BUG=None
TEST=Manually on local setup of Autotest
DEPLOY=apache
Change-Id: I84ac81ba8b386c1317d9809d88dcc61860550386
Reviewed-on: https://chromium-review.googlesource.com/218320
Reviewed-by: Nick Matthijssen <nickam@chromium.org>
Reviewed-by: Fang Deng <fdeng@chromium.org>
Tested-by: Jesus Moises Osorio <moisesosorio@chromium.org>
Commit-Queue: Aviv Keshet <akeshet@chromium.org>
fe/rpc_utils.py
ko/rpc_interface.py
dfea368e5c830b1d7950ced5ee7b191e3b141ca3 11-Aug-2014 Dan Shi <dshi@chromium.org> [autotest] Add RPC get_host_history to get host history.

The RPC takes in arguments like board, pool or a list of hosts, and return the
history of specified hosts. The history is a dictionary of each host and its
history of status. For example:
{'172.22.33.51': [{'status': 'Resetting'
'start_time': '2014-08-07 10:02:16',
'end_time': '2014-08-07 10:03:16',
'log_url': 'http://autotest/reset-546546/debug',
'task_id': 19441991},
{'status': 'Running'
'start_time': '2014-08-07 10:03:18',
'end_time': '2014-08-07 10:13:00',
'log_url': 'http://autotest/reset-546546/debug',
'job_id': 14995562}
]
}

Add a client util module time_utils to handle common task to do time string
and epoch time conversion.

BUG=chromium:394451
TEST=local setup
from autotest_lib.client.common_lib import global_config
from autotest_lib.server.cros.dynamic_suite import frontend_wrappers
instance_server = global_config.global_config.get_config_value(
'SERVER', 'hostname', type=str)
afe = frontend_wrappers.RetryingAFE(
server=instance_server, timeout_min=60, delay_sec=0)
result = afe.run('get_host_history', board='lumpy', pool='bvt',
start_time='2014-08-05 14:00:00',
end_time='2014-08-06 14:00:00')
import pprint
pprint.pprint(result)
rebuild local afe, test run_suite and create job from afe.

Change-Id: I66ecc5bb69a4a1fdb33437520288ed08e562ed00
Reviewed-on: https://chromium-review.googlesource.com/211772
Tested-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Prashanth B <beeps@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
fe/site_rpc_interface.py
f88fa938050d0e2b662a2b32ef7e931e01a4d8fe 04-Sep-2014 Jakob Juelich <jakobjuelich@chromium.org> [autotest] Deserialize records and persist them

To send records from the master to shards it's necessary to serialize
them. This adds the deserialization functionality.

TEST=Ran suites.
DEPLOY=apache

Change-Id: I2806b0cfe1e4fbfba5d89e6d422800c7637ed4e9
Reviewed-on: https://chromium-review.googlesource.com/216355
Reviewed-by: Prashanth B <beeps@chromium.org>
Tested-by: Jakob Jülich <jakobjuelich@chromium.org>
Commit-Queue: Jakob Jülich <jakobjuelich@chromium.org>
fe/model_logic.py
fe/models.py
fe/models_test.py
fe/site_rpc_interface_unittest.py
102e35297e35f0389c3ca66f1fa3b052da37696d 11-Sep-2014 Simran Basi <sbasi@chromium.org> [autotest] Allow piping of max_runtime_mins to dynamic suites.

The AFE allows users to set the max runtime of a job but this is
not properly passed into dynamic_suites which already has support
for this variable.

This change pipes in max_runtime_mins from create_suite_job to the
suite control file from which dyanmic_suites may use.

Also updates the storagequal suite to pass max_runtime_mins to
dynamic suites.

BUG=chromium:395865
TEST=local setup, verified subjobs had max_runtime_mins value set
by the suite job.
DEPLOY=apache

Change-Id: I285e6153f3f420f4eefe5d4c79f4212a4df30f58
Reviewed-on: https://chromium-review.googlesource.com/217780
Tested-by: Simran Basi <sbasi@chromium.org>
Reviewed-by: Gwendal Grignou <gwendal@chromium.org>
Commit-Queue: Simran Basi <sbasi@chromium.org>
fe/site_rpc_interface.py
23b2895d2c3819a0191deab727327bcb894a0f99 13-Sep-2014 Fang Deng <fdeng@chromium.org> This is suspect to break our afe "debug" link. I am going to revert it for now.

Revert "[autotest] Add heartbeat AFE endpoint to shard Autotest"

This reverts commit 1e10d745c65ecafa79bd4f0b4f0b743bd5f1eff3.

Change-Id: I81fd554a910c8c5b7537b34ec13301fcf15be3fd
Reviewed-on: https://chromium-review.googlesource.com/217980
Reviewed-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Fang Deng <fdeng@chromium.org>
Tested-by: Fang Deng <fdeng@chromium.org>
fe/json_rpc/serviceHandler.py
fe/json_rpc/serviceHandler_unittest.py
fe/models.py
fe/rpc_utils.py
fe/site_rpc_interface.py
fe/site_rpc_interface_unittest.py
1e10d745c65ecafa79bd4f0b4f0b743bd5f1eff3 03-Sep-2014 Jakob Juelich <jakobjuelich@chromium.org> [autotest] Add heartbeat AFE endpoint to shard Autotest

To improve the workload the autotest setup can handle, jobs should
be executed by multiple shards. They are sharded by type of board.

This a HTTP accessible endpoint to AFE. This assigns jobs and hosts
to moblabs and returns them as records to insert into a shard's
local database.

TEST=Ran suites and tried manually retrieving jobs
DEPLOY=scheduler,apache,host-scheduler

Change-Id: Ie1dfa7bb295f685c56bb1354675d83f82d933c65
Reviewed-on: https://chromium-review.googlesource.com/211214
Reviewed-by: Alex Miller <milleral@chromium.org>
Tested-by: Jakob Jülich <jakobjuelich@chromium.org>
Commit-Queue: Jakob Jülich <jakobjuelich@chromium.org>
fe/json_rpc/serviceHandler.py
fe/json_rpc/serviceHandler_unittest.py
fe/models.py
fe/rpc_utils.py
fe/site_rpc_interface.py
fe/site_rpc_interface_unittest.py
3bb7c8095d7c66d955462673dd3df5565c1b4a96 03-Sep-2014 Jakob Juelich <jakobjuelich@chromium.org> [autotest] Add serialization functionality including dependencies

In order to send hosts and jobs to shards it's necessary to
serialize them including their dependencies.

This changelist adds functionality to find an object's dependencies
and to serialize it into lists and dicts, so it can be converted
to json.

TEST=Ran suites.
DEPLOY=apache

Change-Id: Icc158de1f23c7483ebc22b6ca3482b7ffccdd5c4
Reviewed-on: https://chromium-review.googlesource.com/215998
Tested-by: Jakob Jülich <jakobjuelich@chromium.org>
Reviewed-by: Alex Miller <milleral@chromium.org>
Commit-Queue: Jakob Jülich <jakobjuelich@chromium.org>
fe/model_logic.py
fe/models.py
fe/models_test.py
de2b9a93674cb762230a83b1c41cc373073e4fdb 03-Sep-2014 Jakob Juelich <jakobjuelich@chromium.org> [autotest] Add shard_id to hosts

To assign hosts to shards and avoid reassigning them again and again
all the time, somehow it must be stored if a host was already synced
to a shard.

This changelist adds a column in afe_hosts to reference the shard
it has been synced to.

TEST=Ran suites and manually tried db migrations and running dummy.
DEPLOY=scheduler,apache,host-scheduler,migrate

Change-Id: Ide181a221f363fe96130611da7732baec9680a4f
Reviewed-on: https://chromium-review.googlesource.com/215997
Reviewed-by: Alex Miller <milleral@chromium.org>
Commit-Queue: Jakob Jülich <jakobjuelich@chromium.org>
Tested-by: Jakob Jülich <jakobjuelich@chromium.org>
fe/doctests/001_rpc_test.txt
fe/models.py
igrations/095_shards_hostnames_and_hosts.py
e4cb9e23709425fcbb5faec40bfe824b818bd106 29-Aug-2014 Dan Shi <dshi@chromium.org> [autotest] Change devserver stats call to log data only for staging artifacts.

File names are different as they have build information included. Recording
stats for each file will create too many counters in graphite and lead to disk
space issue.

BUG=chromium:404475
TEST=local setup.
To verify metadata:
visit http://172.25.61.45:9200/_plugin/elastic-hammer/
update search url with index of local setup: dshi.mtv/_search
search for:
{
"query": {"bool": {"minimum_should_match": 1,
"should": [{"term": {"_type": "devserver"}}
]}},
"size": 10000,
"sort": [{"time_recorded": "asc"}]}

Confirm the data.

Change-Id: Ic11f76c3ef6fbf8cf5d25312d3285e1e9b87a178
Reviewed-on: https://chromium-review.googlesource.com/215640
Tested-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
fe/model_logic.py
193905e68043c4386ed1d3f21a74d88de4e9a9b7 26-Jul-2014 Dan Shi <dshi@chromium.org> [autotest] Add job history support.

Add a module to be used to collect job history. For example, all special tasks
executed before and after the test job.
Add an rpc get_job_history to afe to get dictionary of the job and its special
tasks' start/end time.

Update AFE to add a button to show job history.

BUG=chromium:394445
TEST=local setup
./site_utils/job_history.py --job_id=4113
http://dshi.mtv.corp.google.com/afe/#tab_id=view_job&object_id=4113

Change-Id: I0da2e77aaf7e2a38158efd6a64d4f924cf0c803d
Reviewed-on: https://chromium-review.googlesource.com/210091
Tested-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
fe/site_rpc_interface.py
lient/src/autotest/afe/AfeUtils.java
lient/src/autotest/afe/JobDetailView.java
lient/src/autotest/public/AfeClient.html
92c06334ea5e9baab4289ad70135e3f17ed9f872 25-Aug-2014 Jakob Jülich <jakobjuelich@chromium.org> Revert "Revert "[autotest] Add shard tables and models to Autotest""

This reverts commit c1a9d91f65d8fa5225ab673266d165e759c645ab.

Change-Id: Ic6e01cbf06279e19b87926a076548731677024fa
Reviewed-on: https://chromium-review.googlesource.com/214027
Reviewed-by: Alex Miller <milleral@chromium.org>
Tested-by: Jakob Jülich <jakobjuelich@chromium.org>
Commit-Queue: Jakob Jülich <jakobjuelich@chromium.org>
fe/doctests/001_rpc_test.txt
fe/models.py
igrations/094_add_shards.py
9fffe4f5092dab01ca7aff8043db49b3d308575f 15-Aug-2014 Jakob Juelich <jakobjuelich@chromium.org> [autotest] Make rpc_interface_unittest to use django test

The rpc_interface_unittest avoided importing rpc_utils as they need
Django to be set up, otherwise the import would fail. To do that
it contained a lot of code for mocks.

This simplifies this by using the Django unittests.

TEST=Ran suites.
DEPLOY=afe,apache

Change-Id: I63d4b9f2f098b7e10ff5520dcae217da7147cc62
Reviewed-on: https://chromium-review.googlesource.com/212508
Reviewed-by: Alex Miller <milleral@chromium.org>
Tested-by: Jakob Jülich <jakobjuelich@chromium.org>
Commit-Queue: Jakob Jülich <jakobjuelich@chromium.org>
fe/site_rpc_interface.py
fe/site_rpc_interface_unittest.py
c1a9d91f65d8fa5225ab673266d165e759c645ab 22-Aug-2014 Prashanth B <beeps@chromium.org> Revert "[autotest] Add shard tables and models to Autotest"

Afe jobs has 7 million rows and adding a column might take upwards of 20 minutes. I would like to avoid the downtime
during the day, and we need a push asap for a servo fix
that's affecting cq. Since this isn't critical, I'm moving
it out of the way till we have space for the migrate.

This reverts commit 962e3413dce0a6ea2fb0bed87323655f1641e57c.

Change-Id: I053c59cdab314aa7bab5db37b3a4d7cada1f1c7d
Reviewed-on: https://chromium-review.googlesource.com/213791
Reviewed-by: Prashanth B <beeps@chromium.org>
Commit-Queue: Prashanth B <beeps@chromium.org>
Tested-by: Prashanth B <beeps@chromium.org>
fe/doctests/001_rpc_test.txt
fe/models.py
igrations/094_add_shards.py
962e3413dce0a6ea2fb0bed87323655f1641e57c 15-Aug-2014 Jakob Juelich <jakobjuelich@chromium.org> [autotest] Add shard tables and models to Autotest

To improve the workload the autotest setup can handle, jobs should
be executed by multiple shards. They are sharded by type of board.

This is to enable autotest to manage shard entities.

TEST=Ran suites and dummy suite. Tried db migrations up and down.
DEPLOY=afe,apache,tko,scheduler

Change-Id: I9a62f98b5b8eaf111bb53a4f3e3499c4e5a647df
Reviewed-on: https://chromium-review.googlesource.com/212662
Reviewed-by: Alex Miller <milleral@chromium.org>
Tested-by: Jakob Jülich <jakobjuelich@chromium.org>
Commit-Queue: Jakob Jülich <jakobjuelich@chromium.org>
fe/doctests/001_rpc_test.txt
fe/models.py
igrations/094_add_shards.py
71206ef92012e5a5c36bbd1ae47e9176be04036d 13-Aug-2014 Simran Basi <sbasi@chromium.org> [MobLab] Moblab Setup page.

* Adds a new page located at /moblab_setup
* 2 are on this page: 1 to upload a Boto Key and 1 to edit the config values.
* Link to /moblab_setup is visible only on a moblab system.
* RPC's that execute moblab_setup's actions are gated by a decorator that
limits them to only run on a moblab_system.
* Unittests for new RPC's.
* Editting the config values, writes the full config to shadow_config.ini and
reboots the system so changes takes effort.
* Resetting the config values, makes shadow_config.ini an empty file and
reboots the system so it is restored.
* Uploading the boto key uses shutil.copyfile to write in the new boto file's
contents to the boto file location.

BUG=chromium:396694
TEST=unittests, Uploaded a boto key and successfully ran a suite, editted
config and ensured changes were written into shadow_config, & reset config
and ensured that the default settings were restored.
DEPLOY=afe,apache
CQ-DEPEND=CL:212322
CQ-DEPEND=CL:212323
CQ-DEPEND=CL:212295

Change-Id: Ie354a2df310393045f3116e93004f58ea671de36
Reviewed-on: https://chromium-review.googlesource.com/209685
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Simran Basi <sbasi@chromium.org>
Tested-by: Simran Basi <sbasi@chromium.org>
fe/rpc_interface.py
fe/site_rpc_interface.py
fe/site_rpc_interface_unittest.py
fe/urls.py
fe/views.py
lient/src/autotest/MoblabSetupClient.gwt.xml
lient/src/autotest/afe/AfeClient.java
lient/src/autotest/moblab/BotoKeyView.java
lient/src/autotest/moblab/ConfigSettingsView.java
lient/src/autotest/moblab/MoblabSetupClient.java
lient/src/autotest/public/AfeClient.html
lient/src/autotest/public/MoblabSetupClient.html
a393617fd0f5bf6c64356afd869732288330ab24 07-Aug-2014 J. Richard Barnette <jrbarnette@chromium.org> [autotest] Fix a typo in the AFE home page.

BUG=None
TEST=None
DEPLOY=apache

Change-Id: I7840bc516b90526db23ae54070881217856b78ae
Reviewed-on: https://chromium-review.googlesource.com/211432
Commit-Queue: Richard Barnette <jrbarnette@chromium.org>
Tested-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
lient/src/autotest/public/AfeClient.html
57bc195c551d71c5268d7630aa6e872aaad242bf 23-Jul-2014 Jiaxi Luo <jiaxiluo@chromium.org> [autotest] Add time segmented filter on job table on View Host page.

Modified the frontend to provide a start time and an end time with
datetime pickers. The place holder values are set to midnight "today".
Please note that the place holder values are not default values.
They are equivalent to empty strings, but exist to avoid making users
have to fill in all six segments (year, month, day, hour, minute, am/pm).

On the rpc side, added start_time and end_time parameters to
get_host_queue_entries, get_num_host_queue_entries,
get_host_queue_entries_and_special_tasks,
and get_num_host_queue_entries_and_special_tasks.

A helper function inject_start_end_time_to_dict is used to inject start_time
and end_time to filter_datas.

BUG=chromium:362240
TEST=ran afe, passed rpc_interface_unittest and frontend_unittest
DEPLOY=afe, apache

Change-Id: I44d59124cc104cf55d2ac3838be983286e270195
Reviewed-on: https://chromium-review.googlesource.com/209389
Tested-by: Jiaxi Luo <jiaxiluo@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Jiaxi Luo <jiaxiluo@chromium.org>
fe/rpc_interface.py
fe/rpc_utils.py
lient/src/autotest/afe/HostDetailView.java
lient/src/autotest/common/table/DatetimeSegmentFilter.java
lient/src/autotest/common/ui/DateTimeBox.java
9d1b6d100cc3f15094784dd5d015885a33924e6b 23-Jul-2014 Jiaxi Luo <jiaxiluo@chromium.org> [autotest] Move RadioChooser from package common.ui to afe.

RadioChooser depends on IRadioChooser, which is in package afe.
This will have compilation error because compiling other gwt frontend
needs to rely on package common, but cannot rely on package afe.

Moving RadioChooser to package afe will solve this problem, since no other
gwt frontend other than afe is using it.

BUG=chromium:396694
TEST=ran afe, tko, moblab_setup
DEPLOY=afe,tko,apache

Change-Id: I61a668988843ae8290cdfa585dfb0b535b551b2c
Reviewed-on: https://chromium-review.googlesource.com/209672
Tested-by: Jiaxi Luo <jiaxiluo@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Jiaxi Luo <jiaxiluo@chromium.org>
lient/src/autotest/afe/AfeUtils.java
lient/src/autotest/afe/RadioChooser.java
lient/src/autotest/afe/RadioChooserDisplay.java
lient/src/autotest/afe/UserPreferencesView.java
lient/src/autotest/afe/create/CreateJobViewDisplay.java
lient/src/autotest/afe/create/CreateJobViewPresenter.java
lient/src/autotest/common/ui/RadioChooser.java
lient/src/autotest/common/ui/RadioChooserDisplay.java
8864e86fb4b25912837eb030c343ddc6fd6896cf 22-Jul-2014 Michael Liang <michaelliang@chromium.org> [autotest] Add logging of state 'locked' for model_logic

This will help us to determine whether abnormalities in statuses such as
repair failed are actual problems. If a host is locked, we typically
don't care about its status when doing analysis. I looked at rpc_interface.py,
seems like there are two calls modify_hosts and modify_host that call
update_object(host) only after checking for invalid modifications. So I
decided to log the data in rpc_interface.py

BUG=chromium:394451
TEST=On local afe added a host and via frontend locked and unlocked host.
Verified results in es db.

Change-Id: I0362f64d0c8270dc9373a421944e024ef0175ecd
Reviewed-on: https://chromium-review.googlesource.com/209268
Tested-by: Michael Liang <michaelliang@chromium.org>
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Michael Liang <michaelliang@chromium.org>
fe/model_logic.py
7a776a607e84e5101818142f67cae3d81c18dcd0 18-Jul-2014 Jiaxi Luo <jiaxiluo@chromium.org> [autotest] Add ToolTips to AFE Job list, Create Job, and User Preference pages.

As part of the UI redesign, ToolTips are adopted to drop more explanations to
many unclear fields.

BUG=chromium:394145
TEST=ran afe
DEPLOY=afe,apache

Change-Id: Id72d3ba5160948d4e84e804306550d1ae7d92b14
Reviewed-on: https://chromium-review.googlesource.com/208956
Tested-by: Jiaxi Luo <jiaxiluo@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Jiaxi Luo <jiaxiluo@chromium.org>
lient/src/autotest/afe/JobListView.java
lient/src/autotest/afe/TestSelectorDisplay.java
lient/src/autotest/afe/UserPreferencesView.java
lient/src/autotest/afe/create/CreateJobViewDisplay.java
lient/src/autotest/common/ui/ToolTip.java
lient/src/autotest/public/AfeClient.html
c685857cfadec7741e42ee91147f23555f1b9451 21-Jul-2014 Jiaxi Luo <jiaxiluo@chromium.org> [autotest] Add unqiue constraint on afe_users.login

Add unique constraint to avoid duplicate login names, which happened on moblab.

BUG=chromium:395262
TEST=ran migrate
DEPLOY=migrate

Change-Id: Idbf08957a0b181013e4bb7951c002cd078bcc44e
Reviewed-on: https://chromium-review.googlesource.com/209167
Tested-by: Jiaxi Luo <jiaxiluo@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Reviewed-by: Alex Miller <milleral@chromium.org>
Commit-Queue: Jiaxi Luo <jiaxiluo@chromium.org>
igrations/093_make_afe_users_login_unique.py
dd67bebb78d9b00e6b14106fef9fa711b97044cd 19-Jul-2014 Jiaxi Luo <jiaxiluo@chromium.org> [autotest] Enforce control file when creating a "standard" test.

Control file will only be enforced on creating non-suite
and non-parameterized jobs.

BUG=chromium:394989
TEST=ran afe
DEPLOY=apache

Change-Id: I6e263c5de77be832db13a5032312a9f4204af391
Reviewed-on: https://chromium-review.googlesource.com/209161
Tested-by: Jiaxi Luo <jiaxiluo@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Jiaxi Luo <jiaxiluo@chromium.org>
fe/rpc_interface.py
6c40be4575c9101d99027ce9a6ca71b7e4962f85 16-Jul-2014 Jiaxi Luo <jiaxiluo@chromium.org> [autotest] Disable action menu items when no table item is selected.

It's confusing to have action menu items clickable but actually having
no effect.

BUG=chromium:394442
TEST=ran afe
DEPLOY=afe,apache

Change-Id: If8dabf66f8a38e2d6a671123235d22cc7eb306c5
Reviewed-on: https://chromium-review.googlesource.com/208484
Tested-by: Jiaxi Luo <jiaxiluo@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Jiaxi Luo <jiaxiluo@chromium.org>
lient/src/autotest/afe/HostDetailView.java
lient/src/autotest/afe/HostListView.java
lient/src/autotest/afe/JobDetailView.java
lient/src/autotest/afe/JobListView.java
lient/src/autotest/afe/RecurringView.java
lient/src/autotest/common/ui/ContextMenu.java
d58f008bca8c2d70c3f36d93a9fc1f76f25b4d3f 18-Jul-2014 Jiaxi Luo <jiaxiluo@chromium.org> [autotest] New AFE UI migration.

Migrate AFE to the new UI design including:
* A sharper and more organized UI,
* Organize info on View Job and View Host pages into tables.

BUG=chromium:394145
TEST=ran afe
DEPLOY=afe,apache
Change-Id: I4deb61893888d06ed720e8eb19cdaa1de0270877
Reviewed-on: https://chromium-review.googlesource.com/208159
Tested-by: Jiaxi Luo <jiaxiluo@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Jiaxi Luo <jiaxiluo@chromium.org>
lient/src/autotest/afe/HostSelectorDisplay.java
lient/src/autotest/afe/HostTable.java
lient/src/autotest/afe/JobListView.java
lient/src/autotest/afe/TestSelectorDisplay.java
lient/src/autotest/afe/create/CreateJobViewDisplay.java
lient/src/autotest/common/table/DataTable.java
lient/src/autotest/common/table/SearchFilter.java
lient/src/autotest/common/table/TableDecorator.java
lient/src/autotest/common/ui/CustomTabPanel.java
lient/src/autotest/common/ui/TableActionsPanel.java
lient/src/autotest/public/AfeClient.html
lient/src/autotest/public/Open+Sans:300.woff
lient/src/autotest/public/Roboto+Bold:700.woff
lient/src/autotest/public/Roboto+Light:300.woff
lient/src/autotest/public/Roboto+Medium:500.woff
lient/src/autotest/public/Roboto+Regular:400.woff
lient/src/autotest/public/afeclient.css
d85d611e7560f28baf622c0f76b992e0a7fb2789 14-Jul-2014 Dan Shi <dshi@chromium.org> [autotest] Do not record time_finished if a special task is not started yet

BUG=chromium:387918
TEST=local setup

Change-Id: I4cd64792f16305e375981f01019b5485f43f7756
Reviewed-on: https://chromium-review.googlesource.com/207810
Tested-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
fe/models.py
d94d8dc25b6c3f7af965075474d1583cbd7e471e 16-Jul-2014 Jiaxi Luo <jiaxiluo@chromium.org> [autotest] Make AFE remember the Autorefresh state of each tab.

If one checks the Autorefresh on AFE, instead of making the Autorefresh
global to every tab, the AFE will remember the state of each tab.

BUG=chromium:394189
TEST=ran afe
DEPLOY=afe,apache

Change-Id: Ib8f136ff229f92424a1623b2525de579c5662547
Reviewed-on: https://chromium-review.googlesource.com/208208
Tested-by: Jiaxi Luo <jiaxiluo@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Jiaxi Luo <jiaxiluo@chromium.org>
lient/src/autotest/common/ui/CustomTabPanel.java
lient/src/autotest/common/ui/TabView.java
cb91d2eb70554b4f1e18ff5f5ec29a6b8cb3d9a8 30-Jun-2014 Jiaxi Luo <jiaxiluo@chromium.org> [autotest] Abort special tasks through AFE.

On View Host page, after checking the show special tasks checkbox,
the job table becomes a mixing of special tasks and regular jobs.
But only regular jobs are able to be aborted before.

I made the special tasks selectable as well and made the Abort action
apply to both regular and special tasks.

On the GWT side, the selected entries will be divided into special tasks
and regular jobs. Two rpc calls will be made separately to abort them.

BUG=chromium:290476
TEST=ran afe, abort regular jobs and special tasks separately and together
DEPLOY=afe,apache

Change-Id: I0a861552e67602a9f9a5f6ee096104b745e9fc5f
Reviewed-on: https://chromium-review.googlesource.com/206068
Tested-by: Jiaxi Luo <jiaxiluo@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Jiaxi Luo <jiaxiluo@chromium.org>
fe/rpc_utils.py
lient/src/autotest/afe/AfeUtils.java
lient/src/autotest/afe/HostDetailView.java
15cbf37cebd87112bb493d12330e123d1b007c23 02-Jul-2014 Jiaxi Luo <jiaxiluo@chromium.org> [autotest] filter suite and individual jobs on AFE.

Modified rpc_interface get_jobs and get_num_jobs to take in three extra
filters, suite, sub, and standalone. Add a rpc_util to add a where clause
to SQL, and add a unittest accordingly.

On GWT, add a group of radiobuttons as filters.

BUG=chromium:390345
TEST=ran afe, ran Job List, use filters
DEPLOY=afe,apache

Change-Id: Ibce875b47bef7e89bcc94972e23261290f0ac0e0
Reviewed-on: https://chromium-review.googlesource.com/206415
Tested-by: Jiaxi Luo <jiaxiluo@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Jiaxi Luo <jiaxiluo@chromium.org>
fe/rpc_interface.py
fe/rpc_interface_unittest.py
fe/rpc_utils.py
lient/src/autotest/afe/JobListView.java
lient/src/autotest/common/table/RadioButtonSetFilter.java
lient/src/autotest/public/AfeClient.html
421608e2ae505cdb9d57bc1d5bbd198bdb9b6243 07-Jul-2014 Jiaxi Luo <jiaxiluo@chromium.org> [autotest] WMatrix link on AFE.

RPC get_static_data will get wmatrix_url from config file and send to frontend.
If wmatrix_url is not present or set to empty, the WMatrix link will not show
up on AFE. Otherwise, it will on the top-right corner.

BUG=chromium:391851
TEST=edit config files, ran afe
DEPLOY=afe,apache

Change-Id: I29948ec59d203ac84d60e68d2c5a9b612b9a4cbc
Reviewed-on: https://chromium-review.googlesource.com/206896
Tested-by: Jiaxi Luo <jiaxiluo@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Jiaxi Luo <jiaxiluo@chromium.org>
fe/rpc_interface.py
fe/rpc_utils.py
lient/src/autotest/afe/AfeClient.java
lient/src/autotest/public/AfeClient.html
020cd19872b2e940d9ba36825bb0f473345c4a09 30-Jun-2014 Jiaxi Luo <jiaxiluo@chromium.org> [MobLab] Add/Edit/Delete host attributes through AFE admin interface.

MobLab needs to use host attributes to specify Servo and Wificells.

Add an inline table when editing host through AFE admin interface.
Also grant permission to add/edit/delete host attributes to users through
db migration.

BUG=chromium:389659
TEST=ran afe, add/edit/delete host attributes
DEPLOY=migrate,apache

Change-Id: I13004481fd7b378b856dfbbad5ed9195603f83f6
Reviewed-on: https://chromium-review.googlesource.com/206142
Tested-by: Jiaxi Luo <jiaxiluo@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Jiaxi Luo <jiaxiluo@chromium.org>
fe/admin.py
igrations/092_host_attributes_permissions.py
3bb15cfa61fd3f5ff766000ed1b84a27815875c8 01-Jul-2014 Jiaxi Luo <jiaxiluo@chromium.org> [autotest] get host by hostname through AFE url.

Originally hostname as id to fetch hosts and their jobs/special tasks on AFE.
HostId has been used instead as id to boost the speed fetching jobs
and special tasks (crbug.com/383589). This results in the lost of using
hostname as parameter in AFE url to fetch host.

I made Host View page to handle both hostname and hostid in its url.
One can use cautotest/afe/#tab_id=view_host&object_id=432 to fetch host by id
or use /afe/#tab_id=view_host&hostname=chromeos1-rack3-host3 to fetch host
by hostname. If both object_id and hostname is provided, hostname will be
neglected.

BUG=chromium:388014
TEST=ran afe, put a combination of view_host urls with object_id and hostname
DEPLOY=afe,apache

Change-Id: I60a6a1e0a5bb63f2c4326a7e73f05c1a05a6b4cb
Reviewed-on: https://chromium-review.googlesource.com/206171
Tested-by: Jiaxi Luo <jiaxiluo@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Jiaxi Luo <jiaxiluo@chromium.org>
lient/src/autotest/afe/HostDetailView.java
aac545784cea51dbe230c6c126b52987b7906fb5 04-Jun-2014 Jiaxi Luo <jiaxiluo@chromium.org> [autotest] AFE child jobs table on parent job.

Add a table to list all the child jobs of a parent job on Job View tab.
One can click on and navigate between child and parent.

BUG=chromium:379959
DEPLOY=afe,apache
TEST=ran afe, viewed a job, navigated between parent and child jobs

Change-Id: Id70c41c8f7cee40bd71a206e1f3e08a68efe054f
Reviewed-on: https://chromium-review.googlesource.com/202579
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Jiaxi Luo <jiaxiluo@chromium.org>
Tested-by: Jiaxi Luo <jiaxiluo@chromium.org>
fe/doctests/001_rpc_test.txt
fe/rpc_interface.py
fe/rpc_interface_unittest.py
lient/src/autotest/afe/JobDetailView.java
lient/src/autotest/afe/JobTable.java
lient/src/autotest/common/StatusSummary.java
lient/src/autotest/public/AfeClient.html
lient/src/autotest/tko/SpreadsheetDataProcessor.java
lient/src/autotest/tko/StatusSummary.java
lient/src/autotest/tko/TableView.java
ko/rpc_interface_unittest_fixme.py
51599038f08395067097dc265127cfbcf77c427d 24-Jun-2014 Fang Deng <fdeng@chromium.org> [autotest] Record ending time of HostQueueEntry

Add finished_on to afe_host_queue_entries table.
Update the models.
Set the value of finished_on when the hqe is set as completed.

TEST=Run a dummy suite, observe the value is properly set in database.
BUG=chromium:387917
DEPLOY=scheduler, migrate, apache

Change-Id: I462cc0ea508dac6bdc6960b8169199ceb1091086
Reviewed-on: https://chromium-review.googlesource.com/205287
Reviewed-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Fang Deng <fdeng@chromium.org>
Tested-by: Fang Deng <fdeng@chromium.org>
fe/doctests/001_rpc_test.txt
fe/models.py
igrations/091_add_hqe_finished_on.py
d07255493d6301c5fd58762073d70e2b8a520671 24-Jun-2014 Dan Shi <dshi@chromium.org> [autotest] Record ending time for special tasks in afe_special_tasks

Add a new field time_finished in afe_special_tasks table. The new field records
when a special task is finished.
In SpecialTask.finish method, set the value of time_finished.

BUG=chromium:387918
DEPLOY=migrate,apache
TEST=run a verify job in local setup, confirm the new field has a value of the
verify task.

Change-Id: Ic570bc8ec1c594587c8ec6ee51c31ec4ad140a6a
Reviewed-on: https://chromium-review.googlesource.com/205291
Tested-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
fe/models.py
igrations/090_add_field_time_finished_to_afe_special_tasks.py
79ce642210e581dc2b5cc1df1876e98dae38b286 14-Jun-2014 Jiaxi Luo <jiaxiluo@chromium.org> [autotest] Speed up jobs table on View Host tab on AFE.

AFE was using hostname as key to fetch host and related jobs, which is slow
since afe_host_queue_entries and afe_special_tasks tables are using host_id
instead of hostname as foreign key.

I changed to use host id instead, which largely reduced the latency.
Host table's click listeners on View Job and Host List tabs are also changed
to fetch host by id. The arguments of rpc interfaces
get_host_queue_entries_and_special_tasks and
get_num_host_queue_entries_and_special_tasks are also changed to accept host_id
instead of hostname.

BUG=chromium:383589
DEPLOY=afe,apache
TEST=ran afe, ran View Job and Host List tab and cliked host, ran View Host tab

Change-Id: I7e7d1dd195fec211b6eb867976397d1e643b91ce
Reviewed-on: https://chromium-review.googlesource.com/203843
Tested-by: Jiaxi Luo <jiaxiluo@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Jiaxi Luo <jiaxiluo@chromium.org>
fe/rpc_interface.py
fe/rpc_interface_unittest.py
lient/src/autotest/afe/AfeClient.java
lient/src/autotest/afe/HostDetailView.java
lient/src/autotest/afe/HostListView.java
lient/src/autotest/afe/JobDetailView.java
lient/src/autotest/public/AfeClient.html
90190c94bcf29f43601989f75b20cd228eef5428 18-Jun-2014 Jiaxi Luo <jiaxiluo@chromium.org> [autotest] Be able to pass in args when creating job from the AFE.

A number of tests require args to run. I added a textbox which takes a list of
comma seperated args. The client side will split and trim the input and send
it to RPC. I added a optional argument "args" to the rpc interface
"create_job". To ensure the RPC interface backward compatible,
I added the argument at the end.

BUG=chromium:374918
DEPLOY=afe,apache
TEST=ran afe, create job and type in args

Change-Id: I59df10d4d03dd23181623afdda29c3a683d3685e
Reviewed-on: https://chromium-review.googlesource.com/203011
Tested-by: Jiaxi Luo <jiaxiluo@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Jiaxi Luo <jiaxiluo@chromium.org>
fe/rpc_interface.py
lient/src/autotest/afe/create/CreateJobViewDisplay.java
lient/src/autotest/afe/create/CreateJobViewPresenter.java
lient/src/autotest/public/AfeClient.html
ef6f8a1a9c031e34495625e21af6f637e114a28f 20-Jun-2014 Jiaxi Luo <jiaxiluo@chromium.org> [autotest] Restore tko result database frontend.

A change to a shared file between afe and tko broke tko.

BUG=chromium:386768
TEST=ran tko
DEPLOY=tko (./utils/compile_gwt_clients.py -c autotest.TkoClient), apache

Change-Id: I5b39919758ab622c4b66dc3375593fa90ea0114f
Reviewed-on: https://chromium-review.googlesource.com/204748
Reviewed-by: Alex Miller <milleral@chromium.org>
Commit-Queue: Jiaxi Luo <jiaxiluo@chromium.org>
Tested-by: Jiaxi Luo <jiaxiluo@chromium.org>
ko/rpc_interface.py
3187459dfd0dccbf5c1a94461263c07245aece79 11-Jun-2014 Jiaxi Luo <jiaxiluo@chromium.org> [autotest] Stops rpc get_static_data from fetching cros- and fw-version labels.

Both the rpc interface get_static_data and afe pages loads slow as they need
to read all 60000+ labels. From a user friendly perspective, it's not easy to
find a desired label from this long list on afe.

This will leave only 188 labels. The afe page load time is reduced from 10s to
less than 1s, and the db query time was reduced from 2.37s to 0.07s (both
tested on my local machine with a dump prod db copy).

BUG=chromium:368308
DEPLOY=apache
TEST=ran afe, ran Create Job and Host List tab, selected label from label list

Change-Id: Ifd0e8bc91b2cce8e453f93eec94bb132c2442744
Reviewed-on: https://chromium-review.googlesource.com/203444
Tested-by: Jiaxi Luo <jiaxiluo@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Jiaxi Luo <jiaxiluo@chromium.org>
fe/rpc_interface.py
ad57645f1fd54b398a386fe184f5c47a91ddb1a3 10-Jun-2014 Jiaxi Luo <jiaxiluo@chromium.org> [autotest] Add test name filter on Create Job Tab on AFE.

Based on user feedback from partners, finding a specific test to run
by scrolling is really troublesome. Hence I added a textbox on the top of
test table to filter the test name.

BUG=chromium:382344
DEPLOY=afe,apache
TEST=ran afe, ran Create Job tab, typed in some text in the filter

Change-Id: I9583bc0c738aa135bf01a7e66706920883aadf08
Reviewed-on: https://chromium-review.googlesource.com/203088
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Jiaxi Luo <jiaxiluo@chromium.org>
Tested-by: Jiaxi Luo <jiaxiluo@chromium.org>
lient/src/autotest/afe/ITextBox.java
lient/src/autotest/afe/TestSelector.java
lient/src/autotest/afe/TestSelectorDisplay.java
9e049285fcdf1a964708774f46b84aae5c09837a 05-Jun-2014 Jiaxi Luo <jiaxiluo@chromium.org> [autotest] Allow autorefresh on AFE.

Add a checkbox (default unchecked) to autorefresh the current tab on AFE.
Particularly useful on View Job tab as users don't need to manully refresh
to see the result.

The autorefresh interval is set to 5 seconds. Refresh won't interfere with
any temporary form data from user.

BUG=chromium:359475
DEPLOY=afe,apache
TEST=ran afe, checked the checkbox and wait for five seconds

Change-Id: Icdd8e1c340ec985d2bae65455e30a46fd49c95de
Reviewed-on: https://chromium-review.googlesource.com/202626
Tested-by: Jiaxi Luo <jiaxiluo@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Jiaxi Luo <jiaxiluo@chromium.org>
lient/src/autotest/common/ui/CustomTabPanel.java
cd0a01d0017e6f4c8b5a83890021cefeae3ca64f 12-Jun-2014 Dan Shi <dshi@chromium.org> [autotest] Let log socket server use an unused port

BUG=chromium:383370
TEST=unittest

Change-Id: I56df36cb230d4f4f81f8ce1525b417a132c15aaf
Reviewed-on: https://chromium-review.googlesource.com/203542
Tested-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Jiaxi Luo <jiaxiluo@chromium.org>
Reviewed-by: Prashanth B <beeps@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
fe/json_rpc/serviceHandler.py
f04c41b02786afe6d68b44c6b8f2f35a259bd051 21-May-2014 Jiaxi Luo <jiaxiluo@chromium.org> [autotest] Index owner on afe_jobs table.

The query to get a list of jobs of an owner looks like

SELECT * FROM `afe_jobs` WHERE `afe_jobs`.`owner` = 'debug_user';

and the query to get the number of jobs looks like

SELECT COUNT(DISTINCT `afe_jobs`.`id`) FROM `afe_jobs`
WHERE `afe_jobs`.`owner` = 'debug_user';

These two queries take 25s and 17s at the worst case. The index will
drop the queries to negligible on a db filled with a backup from prod.

BUG=chromium:374403
TEST=Ran migrate, ran frontend
DEPLOY=migrate

Change-Id: Icfeb02c7aae1a6ed2f9a8b53ddd63f6430f92cfc
Reviewed-on: https://chromium-review.googlesource.com/200754
Tested-by: Jiaxi Luo <jiaxiluo@chromium.org>
Reviewed-by: Alex Miller <milleral@chromium.org>
Commit-Queue: Jiaxi Luo <jiaxiluo@chromium.org>
igrations/089_index_afe_jobs_on_owner.py
8b7d7e13b97842c18bbdee696cd257b75c8a27c3 29-May-2014 Jiaxi Luo <jiaxiluo@chromium.org> [AFE] Optimize GWT to load Create Job and Host List tabs much faster.

Originally, the Create Job and Host List tabs take one minute to load.
This is due to sub-optimized GWT instead of any DB query latency.

Specificly, GWT by default is using a javascript method add(option, before)
to add options to `select` DOM element. This method runs in O(n) since
it will perform a linear search on the existing options to find `before`
and add the `option` after `before`.

Instead, I override it to make GWT using appendChild(option), which simply
add option at the end and runs in O(1).

BUG=chromium:378888
DEPLOY=afe
TEST=ran afe, create job on specific hosts

Change-Id: Ia6353074495fe04dd89f4c05bd4d919a322e3025
Reviewed-on: https://chromium-review.googlesource.com/202120
Tested-by: Jiaxi Luo <jiaxiluo@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Jiaxi Luo <jiaxiluo@chromium.org>
lient/src/autotest/common/table/ListFilter.java
lient/src/autotest/common/ui/ExtendedListBox.java
da8c60af1e1e3ee97170c700d0b72991687e35a2 03-Jun-2014 Michael Liang <michaelliang@chromium.org> [autotest] Migrate graphite directory to client/common_lib/cros

This change allows us to report stats in client tests.
1. Change import paths for all files that import modules from graphite
2. Clean up some unused modules
Related CL: https://chromium-review.googlesource.com/#/c/202467/
BUG=chromium:237255
TEST=Ran scheduler locally, scheduled reboot jobs, verified stats such as monitor_db_cleanup.user_cleanup._cleanup were reported on chromeos-stats.
DEPLOY = apache, scheduler, host_scheduler
Change-Id: Iebfe3b8acc1c363a0b70ea555744e85d1367cb67
Reviewed-on: https://chromium-review.googlesource.com/202727
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Michael Liang <michaelliang@chromium.org>
Tested-by: Michael Liang <michaelliang@chromium.org>
fe/json_rpc/serviceHandler.py
0f5f0443e9286483365d6978ff432d8e4029d599 23-May-2014 Jiaxi Luo <jiaxiluo@chromium.org> [autotest] Timer on rpc handler

Leverage on statsd and graphite to monitor on rpc_interface
through rpc handler. This will give us a good idea of the
bottlenecks on DB latency and throughput, and reflect on
the changes made on DB schema.

There is already a counter at serviceHandler.py. The timer is added
in a similar manner.

BUG=None
TEST=Ran afe, ran cli, ran graphite
Change-Id: I4415aacc920db339b18d7208b841747843830997
Reviewed-on: https://chromium-review.googlesource.com/201381
Tested-by: Jiaxi Luo <jiaxiluo@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Jiaxi Luo <jiaxiluo@chromium.org>
fe/json_rpc/serviceHandler.py
972d8a5f4958dad85a48589ec99dd23ebc1320d2 03-Jun-2014 Jiaxi Luo <jiaxiluo@chromium.org> [autotest] AFE opens Parent Job in the same window.

AFE used to open Parent Job in a new window, which takes an extra
five seconds to load. I change it to load the parent job in the
existing window by default. User can still open it in a new window
if they want in the way they open any other links in a new window.

BUG=chromium:379920
DEPLOY=afe
TEST=ran afe and clicked on Parent Job Id

Change-Id: I153128c1d3fab66016ba5a633c36f5263d672d53
Reviewed-on: https://chromium-review.googlesource.com/202432
Tested-by: Jiaxi Luo <jiaxiluo@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Jiaxi Luo <jiaxiluo@chromium.org>
lient/src/autotest/public/AfeClient.html
ab5a1bf348f22cf0c995cf7d43bfab141d4c2e10 29-May-2014 Simran Basi <sbasi@chromium.org> [autotest]: Restore create_job parameterized job functionality.

Turns out the lab admin's use parameterized jobs to reimage devices
through the AFE. We deleted this logic as we thought it was not used.
This is restoring the same code we deleted.

Logic originally removed in:
https://chromium-review.googlesource.com/#/c/194349/

BUG=chromium:372640
TEST=Able to specify an image and kick off a non-suite job and have
the device be reimaged.
DEPLOY=apache

Change-Id: I7c2f0e752362257da3aadb5a3b167b1847840f09
Reviewed-on: https://chromium-review.googlesource.com/201963
Tested-by: Simran Basi <sbasi@chromium.org>
Reviewed-by: Jay Kim <yongjaek@chromium.org>
Commit-Queue: Jay Kim <yongjaek@chromium.org>
fe/rpc_interface.py
40f5df9a2ab68518abe094d1173153bad8e88f76 28-May-2014 Jiaxi Luo <jiaxiluo@chromium.org> [Moblab] Apache conf of Moblab django directory.

Make apache points to the correct directory of django on Moblab,
which is /usr/lib64/python2.7/site-packages/django.

Also change the static link on the nav bar of AFE from WIKI to
Chromium OS Testing, which now points to
http://www.chromium.org/chromium-os/testing.

BUG=chromium:378044
TEST=Ran afe admin on Moblab and local
DEPLOY=afe, apache

Change-Id: I6a75ddfae999005fd0381643dbd39554eaae86b4
Reviewed-on: https://chromium-review.googlesource.com/201775
Tested-by: Jiaxi Luo <jiaxiluo@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Jiaxi Luo <jiaxiluo@chromium.org>
lient/src/autotest/public/AfeClient.html
4ec9867f46deb969c154bebf2e64729d56c3a1d3 15-May-2014 Prashanth B <beeps@google.com> [autotest] Split host acquisition and job scheduling II.

This cl creates a stand-alone service capable of acquiring hosts for
new jobs. The host scheduler will be responsible for assigning a host to
a job and scheduling its first special tasks (to reset and provision the host).
There on after, the special tasks will either change the state of a host or
schedule more tasks against it (eg: repair), till the host is ready to
run the job associated with the Host Queue Entry to which it was
assigned. The job scheduler (monitor_db) will only run jobs, including the
special tasks created by the host scheduler.

Note that the host scheduler won't go live till we flip the
inline_host_acquisition flag in the shadow config, and restart both
services. The host scheduler is dead, long live the host scheduler.

TEST=Ran the schedulers, created suites. Unittests.
BUG=chromium:344613, chromium:366141, chromium:343945, chromium:343937
CQ-DEPEND=CL:199383
DEPLOY=scheduler, host-scheduler

Change-Id: I59a1e0f0d59f369e00750abec627b772e0419e06
Reviewed-on: https://chromium-review.googlesource.com/200029
Reviewed-by: Prashanth B <beeps@chromium.org>
Tested-by: Prashanth B <beeps@chromium.org>
Commit-Queue: Prashanth B <beeps@chromium.org>
fe/frontend_test_utils.py
c342f9fb42a17db4e26f8b265b3d094388d54102 20-May-2014 Jiaxi Luo <jiaxiluo@chromium.org> [afe] afe admin add host label auto detection

Add a checkbox when add/edit host from afe admin interface,
which will trigger label auto detection by piping to cli host create.
Add a helper function in host_create to fill in data without parsing
from command line.

TEST=Compared adding hosts through afe and cli, ran host_unittest.
BUG=chromium:371898
DEPLOY=apache
Change-Id: Id5307e68d55dc5120e6a8036888987d2819b4ad6
Reviewed-on: https://chromium-review.googlesource.com/199607
Tested-by: Jiaxi Luo <jiaxiluo@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Jiaxi Luo <jiaxiluo@chromium.org>
fe/admin.py
ettings.py
ad1841b1a07d40e3ca4110f9766b0bedee419c07 21-May-2014 Jiaxi Luo <jiaxiluo@chromium.org> [afe] "Job List" entry number not updated

On the "Job List" tab, when changing owners or changing from "All Jobs"
to "Running Jobs" etc., the number of jobs is not be updated.

TEST=Compile afe, ran frontend
BUG=chromium:375468
DEPOLY=afe

Change-Id: Ib4f8379740ca8df9854a25feb0054b9fef9c99e7
Reviewed-on: https://chromium-review.googlesource.com/200671
Tested-by: Jiaxi Luo <jiaxiluo@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Jiaxi Luo <jiaxiluo@chromium.org>
lient/src/autotest/common/table/DynamicTable.java
058860cf9d5107bd0d464e691f8df221d1b8d4b5 16-May-2014 Fang Deng <fdeng@chromium.org> [autotest] Add --retry option to run_suite

Pipe the retry configuration ->frontend->suite control files.
This allows us to enable/disable retry for each suite run.

The next step is to update all the suite control files to
pipe 'job_retry' to dynamic suite which will be done in CL:200308

TEST=unittest;
locally run a suite with and without the option, observe
tests were retried/not retried.
1) ./run_suite.py --build trybot-lumpy-release/R37-5858.0.0-b2105
--suite_name dummy --board lumpy --retry=True
2)./run_suite.py --build trybot-lumpy-release/R37-5858.0.0-b2105
--suite_name dummy --board lumpy
BUG=chromium:373929
DEPLOY=apache

Change-Id: I2c8c9caf36fd2132d75bd381e015d0adfddda34f
Reviewed-on: https://chromium-review.googlesource.com/200223
Reviewed-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Fang Deng <fdeng@chromium.org>
Tested-by: Fang Deng <fdeng@chromium.org>
fe/site_rpc_interface.py
b6ec8ae8b13902110f664cf51e3a1d9b43c9f8f3 23-Apr-2014 Simran Basi <sbasi@chromium.org> [autotest]: Run suites from AFE.

* Updated the AFE to allow for suite runs. When an image is provided
the create_job RPC will redirect to the create_suite_job RPC instead.

* Updated create_suite_job to allow for a control_file to be supplied.
create_suite_job will use this control file rather than what the
devserver stages when it creates the suite job.

* Removed the parameterized job code from create_job as it is not used
and broken.

* The create_job page now includes a 'Pool' text box to specify for
these jobs.

BUG=chromium:358579
TEST=Able to launch suite jobs from run_suite and AFE. And regular tests
kicked off via the AFE still work as well. rpc_interface and
site_rpc_interface unittests, suite_scheduler unittests.
DEPLOY=apache,afe,suite_scheduler

Change-Id: I53312419c32740e78fc07598babeb497c162ba81
Reviewed-on: https://chromium-review.googlesource.com/194349
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Simran Basi <sbasi@chromium.org>
Tested-by: Simran Basi <sbasi@chromium.org>
fe/rpc_interface.py
fe/rpc_utils.py
fe/site_rpc_interface.py
fe/site_rpc_interface_unittest.py
lient/src/autotest/afe/create/CreateJobViewDisplay.java
lient/src/autotest/afe/create/CreateJobViewPresenter.java
lient/src/autotest/public/AfeClient.html
6f46fd768fc4764b791cb42a7891ca539455fcb3 12-May-2014 Simran Basi <sbasi@chromium.org> [autotest]: Move 087_synch_count_not_null migration.

2 087_* migrations landed at the same time. One was cherry-picked
and pushed to prod so this change updates the numbering for the
second migration.

BUG=chromium:372536
TEST=None
DEPLOY=migrate

Change-Id: I97a8cbd51df842aa0b1a36468dd359857c907029
Reviewed-on: https://chromium-review.googlesource.com/199376
Reviewed-by: Dan Shi <dshi@chromium.org>
Tested-by: Simran Basi <sbasi@chromium.org>
igrations/087_synch_count_not_null.py
igrations/088_synch_count_not_null.py
23c2a5d7bdba7585e70ba94b4fa81277efa78940 08-Apr-2014 Fang Deng <fdeng@chromium.org> [autotest] add invalidate_test_idx and invalid to tko_test_view_2

Expose invalid and invalidate_test_idx in tko_test_view_2.
This is part of the changes to make run_suite understand
which tests are retries.

CQ-DEPEND=CL:197306
BUG=chromium:353169
TEST=unittests; integration test with CL:199173
1) Add "JOB_RETRIES=2" to dummy_Fail/control
Create a trybot image that has the change.
2) run_suite.py -b lumpy -s dummy -i
trybot-lumpy-release/R35-5653.0.0-b1948
3) Observe that dummy_Fail is retried twice and only the last
job's result was output by run_suite.
DEPLOY=apache, migrate

Change-Id: I5b9133e062c549f7ada7ae70ac0ade02b78e96f9
Reviewed-on: https://chromium-review.googlesource.com/198284
Reviewed-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Fang Deng <fdeng@chromium.org>
Tested-by: Fang Deng <fdeng@chromium.org>
igrations/087_add_fields_to_tko_test_view_2.py
ko/models.py
8d89b64d7edb2ea7d7f2d9c59ded2e2f09ed3ea8 03-May-2014 Simran Basi <sbasi@chromium.org> [autotest] models.Job set default synch_count to 0.

Currently when creating a suite job we pass in a synch_count of
None, pass synch_count validation, then create a Job entry in
the database. However the default value for synch_count is 1 and
that is what gets recorded in the database.

This breaks the ability to create a recurring job for a suite job
as the original job's synch_count does not match what was used to
create it. In order to address this, make the default value for
synch_count be 0.

BUG=chromium:369726
TEST=./database/migrate sync and ensured afe_jobs now shows not null
for synch_count. Created a regular job from the afe, kicked off
a suite via run_suite, created a recurring suite job and a recurring
regular job. frontend_unittest.py
DEPLOY=scheduler,apache,migrate

Change-Id: I31728f9f876c917a3acaa390c7b9ffc33042bbc2
Reviewed-on: https://chromium-review.googlesource.com/198102
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Simran Basi <sbasi@chromium.org>
Tested-by: Simran Basi <sbasi@chromium.org>
fe/doctests/001_rpc_test.txt
fe/models.py
igrations/087_synch_count_not_null.py
76a11d37d187d7cd49b7e71aa197f03a9036a3a9 09-May-2014 Jiaxi Luo <jiaxiluo@google.com> [autotest] Add message middleware to django

This will make django admin interface work.

BUG=chromium:371603

TEST=Ran frontend admin interface.

Change-Id: Ic564234494bdced39d4131c8cc10b5c37a85db3d
Reviewed-on: https://chromium-review.googlesource.com/198966
Reviewed-by: Alex Miller <milleral@chromium.org>
Tested-by: Jiaxi Luo <jiaxiluo@chromium.org>
Commit-Queue: Jiaxi Luo <jiaxiluo@chromium.org>
ettings.py
9ec668053c3cc508d3966e319e5b2bc644b272d6 28-Apr-2014 Fang Deng <fdeng@chromium.org> Revert "Revert "[autotest] TKO parser mark original tests as invalid""

This reverts commit dbd9037df0cbe1232fcb016a2c261f5994ae03a2.

Commit this cl again. Once this cl lands, deploy the db changes together with

https://chrome-internal-review.googlesource.com/#/c/161494/
https://chrome-internal-review.googlesource.com/#/c/161426/

DEPLOY=apache, migrate, wmatrix db changes should be pushed together (see commit messeage)

Change-Id: I47916708e7b49bbc2064370262e3f9e231f08efe
Reviewed-on: https://chromium-review.googlesource.com/197306
Reviewed-by: Dan Shi <dshi@chromium.org>
Tested-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Fang Deng <fdeng@chromium.org>
igrations/086_add_invalidates_test_idx_to_tko_tests.py
ko/models.py
dbd9037df0cbe1232fcb016a2c261f5994ae03a2 28-Apr-2014 Prashanth B <beeps@chromium.org> Revert "[autotest] TKO parser mark original tests as invalid"

This reverts commit 5ca36681271b3feab50dfabc67eb43b6669032c6.

Change-Id: I7e597d0f341f9046078e2f9319c30d1d2cb8f599
Reviewed-on: https://chromium-review.googlesource.com/197230
Reviewed-by: Prashanth B <beeps@chromium.org>
Commit-Queue: Prashanth B <beeps@chromium.org>
Tested-by: Prashanth B <beeps@chromium.org>
igrations/086_add_invalidates_test_idx_to_tko_tests.py
ko/models.py
5ca36681271b3feab50dfabc67eb43b6669032c6 28-Feb-2014 Fang Deng <fdeng@chromium.org> [autotest] TKO parser mark original tests as invalid

This is part of the test retry imeplementation.
When retry job finished, we mark the original tests as invalid
so that run_suite/tko frontend could get the latest
results by only retrieving "valid" tests.

Also add a column 'invalidates_test_idx' in tko_tests.
This field works as a pointer from retry to original tests.
It will be used by wmatrix.

Design doc: crbug.com/314759

CQ-DEPEND=CL:191002, CL:190585
TEST=1)Manually call create_job twice, drop a keyval
"original_retry_job_id" for the first job. Let the scheduler
pick up and run the two jobs. Observe that at the end, the rows
for the first job in tko_tests are marked as invalid.
And 'invalidates_test_idx' are updated correctly.
2) Integration test with crosreview.com/191002 and
crosreview.com/190585.
- Add "JOB_RETRIES=2" to dummy_Fail/control
and dummy_Pass/control.
Create a trybot image that has the change.
- run_suite.py -b lumpy -s dummy -i
trybot-lumpy-release/R35-5653.0.0-b1948
- Observe that dummy_Fail is retried twice, and
'invalid' and 'invalidates_job_id' are properly set
in tko_tests.
Observe that dummy_Pass is not retried.
- Test the same flow but dependencies are not satisfied
Observed no retry was scheduled
- Test the same flow but abandon dummy_Fail before it started
Observed no retry was scheduled
3) dummy suite runs ok with run_suite and test_that
4) Run database/migrate.py sync 86 (85)
Observed database sync'd properly.
BUG=chromium:347687
DEPLOY=migrate, apache

Change-Id: Ieed64a724c7330c50e8c022bd659b9d1ec781061
Reviewed-on: https://chromium-review.googlesource.com/188365
Reviewed-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Fang Deng <fdeng@chromium.org>
Tested-by: Fang Deng <fdeng@chromium.org>
igrations/086_add_invalidates_test_idx_to_tko_tests.py
ko/models.py
aa77200876b530a3b37e6be3ed1e2c0b7098c9dc 11-Apr-2014 Alex Miller <milleral@chromium.org> [autotest] Treat special task labels like "provisionable" labels.

As it stands, to use the special task labels, you need to add the label
to every host, since we still treat it as a normal/standard DEPENDENCY.

If any special task treats a label differently, then we shouldn't
require that label on the DUT, as we're using it as a marker that we
want some special handling of this test rather than an actual
dependency.

BUG=chromium:334418
DEPLOY=scheduler, apache
TEST=added a potato->dummy_PassServer mapping, saw dummy_PassServer run
in cleanup on job with DEPENDENCIES=potato where host didn't have
potato.

Change-Id: I34936a5a84a42897c8dfc0e2acd50195e0e74ec5
Reviewed-on: https://chromium-review.googlesource.com/194212
Reviewed-by: Alex Miller <milleral@chromium.org>
Commit-Queue: Alex Miller <milleral@chromium.org>
Tested-by: Alex Miller <milleral@chromium.org>
fe/rpc_utils.py
2c1a22a9f93bf50147cd4e6b10487d02768f8919 03-Apr-2014 Prashanth B <beeps@google.com> [autotest] Include parent_job_id as a tie breaker in the host request.

Teach the rdb to give hosts to requests from the same suite by including
parent_job_id in the acquire hosts request, and using it to order
requests of equal priority. This means that we will only group requests
from the same suite, which will result in an O(number of concurrent suites)
increase in database queries to find hosts, but it should lead to an
overall win since suites will timeout less often if they're allowed to
run till completion before hosts in the same pool are re-allocated to another
suite.

TEST=Ran suites, added unittests that failed before and passes now.
BUG=chromium:351861

Change-Id: Ia6e01f92484fb7bfd2e2e1e780cc4a16832fe901
Reviewed-on: https://chromium-review.googlesource.com/192965
Tested-by: Prashanth B <beeps@chromium.org>
Reviewed-by: Alex Miller <milleral@chromium.org>
Commit-Queue: Prashanth B <beeps@chromium.org>
fe/frontend_test_utils.py
85f4c36b14448d8fd707b34107c7688f13f258f2 08-Apr-2014 Simran Basi <sbasi@chromium.org> Moblab: Display 'moblab' user as default in AFE.

Currently there are 2 separate users for Moblab: 'default_user' for
kicking off jobs from the AFE and 'moblab' for kicking off jobs from
the run_suite command.

This change defaults the AFE to use 'moblab'. And allows jobs to be
aborted from the AFE.

BUG=chromium:358579
TEST=Moblab AFE defaults to 'moblab' user and is able to abort jobs
created by run_suite. Also verified get_current_board() still works
on a moblab.

Change-Id: Ic9a0a7bfc745a670796863629d070056e1dfd69f
Reviewed-on: https://chromium-review.googlesource.com/193657
Reviewed-by: Prashanth B <beeps@chromium.org>
Tested-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Simran Basi <sbasi@chromium.org>
pache_auth.py
489b91d72cd225e902081dbd3f9e47448fe867f6 15-Mar-2014 Prashanth B <beeps@google.com> [autotest] Establish a common interface for host representation.

This cl has the work needed to ensure that schema changes made on
the server trickled down into the client. If the same changes don't
reflect on the client, creating or saving a client host wrapper for
a given host will fail deterministically on the client side until
modules using the rdb_host are modified to reflect the changes.

1. rdb_hosts: A module containing the host heirarchy needed to
establish a dependence between the creation of the RDBServerHostWrapper
(which is serialized and returned to the client, which converts it
into an RDBClientHostWrapper) and the saving of the RDBClientHostWrapper
through and rdb update request.
2. rdb_requests: Contains the requests/request managers that were in
rdb_utils, because I plan to expand them in subsequent cls.
3. rdb_model_extensions: Contains model classes common
to both server and client that help in establishing
the common host model interface.
4. rdb integration tests.

TEST=Ran suites, unittests
BUG=chromium: 348176
DEPLOY=scheduler

Change-Id: I0bbab1dd184e505b1130ee73714e45ceb7bf4189
Reviewed-on: https://chromium-review.googlesource.com/191357
Commit-Queue: Prashanth B <beeps@chromium.org>
Tested-by: Prashanth B <beeps@chromium.org>
Reviewed-by: Dan Shi <dshi@chromium.org>
fe/doctests/001_rpc_test.txt
fe/frontend_test_utils.py
fe/model_logic.py
fe/models.py
fe/rdb_model_extensions.py
ec21225c11b0983b563c24bb71828f5bb6bbcaee 01-Mar-2014 Alex Miller <milleral@chromium.org> [autotest] Feed job labels from the scheduler to autoserv.

All job labels now get passed into reset, verify, cleanup, provision,
and repair. Everything bug provision can run standalone, and thus
passing the job labels needs to be optional for those.

BUG=chromium:334418
DEPLOY=scheduler
TEST=special tasks run fine, scheduler shows autoserv being invoked
correctly

Change-Id: I524cc8441e654685e50c935fb1d55cf87e8a2f73
Reviewed-on: https://chromium-review.googlesource.com/188455
Reviewed-by: Alex Miller <milleral@chromium.org>
Tested-by: Alex Miller <milleral@chromium.org>
Commit-Queue: Alex Miller <milleral@chromium.org>
fe/models.py
cc9fc70587d37775673e47b3dcb4d6ded0c6dcb4 02-Dec-2013 beeps <beeps@chromium.org> [autotest] RDB Refactor II + Request/Response API.

Scheduler Refactor:
1. Batched processing of jobs.
2. Rdb hits the database instead of going through host_scheduler.
3. Migration to add a leased column.The scheduler released hosts
every tick, back to the rdb.
4. Client rdb host that queue_entries use to track a host, instead
of a database model.

Establishes a basic request/response api for the rdb:
rdb_utils:
1. Requests: Assert the format and fields of some basic request types.
2. Helper client/server modules to communicate with the rdb.
rdb_lib:
1. Request managers for rdb methods:
a. Match request-response
b. Abstract the batching of requests.
2. JobQueryManager: Regulates database access for job information.
rdb:
1. QueryManagers: Regulate database access
2. RequestHandlers: Use query managers to get things done.
3. Dispatchers: Send incoming requests to the appropriate handlers.
Ignores wire formats.

TEST=unittests, functional verification.
BUG=chromium:314081, chromium:314083, chromium:314084
DEPLOY=scheduler, migrate

Change-Id: Id174c663c6e78295d365142751053eae4023116d
Reviewed-on: https://chromium-review.googlesource.com/183385
Reviewed-by: Prashanth B <beeps@chromium.org>
Commit-Queue: Prashanth B <beeps@chromium.org>
Tested-by: Prashanth B <beeps@chromium.org>
fe/doctests/001_rpc_test.txt
fe/model_logic.py
fe/models.py
igrations/085_lease_hosts.py
cfe5e0bfdf22ee352bfb68b588f17e7ddab5bd92 18-Jan-2014 Prathmesh Prabhu <pprabhu@chromium.org> [autotest] Fix django path reference error in afe admin interface.

We have moved on to django 1.5.1
An old path reference in the admin interface needs correction.

BUG=chromium:335847
DEPLOY=afe
TEST=Launched autotest frontend locally and loaded the admin interface.

Change-Id: I84319f65bd48bcfa6b3726aa3ced899e7b3db74d
Reviewed-on: https://chromium-review.googlesource.com/183129
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
Tested-by: Prathmesh Prabhu <pprabhu@chromium.org>
Commit-Queue: Prathmesh Prabhu <pprabhu@chromium.org>
fe/templates/admin/base.html
9512241978928631e38d59b5aca6467172fd404b 13-Nov-2013 Dan Shi <dshi@chromium.org> [autotest] Modify option no_wait for a suite job to fire and forget

no_wait is an option in run_suite for the script to exit without waiting for
the suite job to finish.

To prevent a suite job from holding on a process slot in drone for a long time.
The usage of no_wait option is expended so that the suite job can exit once all
test jobs are created, without waiting for all test jobs to finish.

related CL for test_suites:
https://chromium-review.googlesource.com/#/c/177309/

BUG=chromium:316990
TEST=trybot build: trybot-lumpy-paladin/R33-4997.0.0-b1385
then run_suite in local setup:
./run_suite.py --build=lumpy-release/R33-4933.0.0 --board=lumpy
--suite_name=dummy -p bvt --no_wait=True
Confirm the suite job is completed right after the test jobs are scheduled.

DEPLOY=apache

Change-Id: I039a50371f9c5ebe8c8e155a9aabfb37999be4e2
Reviewed-on: https://chromium-review.googlesource.com/176601
Tested-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Alex Miller <milleral@chromium.org>
Reviewed-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
fe/site_rpc_interface.py
7d8273bad1318c13698a162a6e5910bea060d167 06-Nov-2013 beeps <beeps@chromium.org> [autotest] RDB refactor I

Initial refactor for the rdb, implementes 1 in this schematic:
https://x20web.corp.google.com/~beeps/rdb_v1_midway.jpg

Also achieves the following:
- Don't process an hqe more than once, after having assigned a host to it.
- Don't assign a host to a queued, aborted hqe.
- Drop the metahost concept.
- Stop using labelmetahostscheduler to find hosts for non-metahost jobs.
- Include a database migration script for jobs that were still queued during
the scheduler restart, since they will now need a meta_host dependency.

This cl also doesn't support the schedulers ability to:
- Schedule an atomic group
* Consequently, also the ability to block a host even when the hqe using it is
no longer active.
- Schedule a metahost differently from a non-metahost
* Both metahosts and non-metahosts are just labels now
* Jobs which are already assigned hosts are still give precedence, though
- Schedule based on only_if_needed.

And fixes the unittests appropriately.

TEST=Ran suites, unittests. Restarted scheduler after applying these changes
and tested migration. Ran suite scheduler.
BUG=chromium:314082,chromium:314219,chromium:313680,chromium:315824,chromium:312333
DEPLOY=scheduler, migrate

Change-Id: I70c3c3c740e51581db88fe3ce5879c53d6e6511e
Reviewed-on: https://chromium-review.googlesource.com/175957
Reviewed-by: Alex Miller <milleral@chromium.org>
Commit-Queue: Prashanth B <beeps@chromium.org>
Tested-by: Prashanth B <beeps@chromium.org>
fe/frontend_test_utils.py
igrations/084_convert_metahost_to_label.py
7e60574f3baa0b3928623216a74a770f19d2c316 12-Nov-2013 Simran Basi <sbasi@chromium.org> Autotest: Update RPC, Scheduler and AFE to use timeout_mins.

Now that timeout_mins is in the database, we need to update the
infrastructure to display and utilize timeout_mins rather than
timeout.

The scheduler now aborts based off of timeout_mins.

The afe displays/creates jobs using timeout_mins rather than timeout.

The RPC's take in timeout and timeout_min depending on what is inputted
to the system. Note the value supplied for timeout_mins takes precedence
over timeout.

BUG=chromium:221264
TEST=Created job through AFE, run_suite with smoke suite, job_unittest
and rpc_unittests.
DEPLOY=apache, afe, scheduler

CQ-DEPEND=I09f8b6023a31dcef37e98c298dc0d63b37bc0a9e

Change-Id: I838cd1ff85a2ce6bd49fe96bf18fd82be608c758
Reviewed-on: https://chromium-review.googlesource.com/176605
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Simran Basi <sbasi@chromium.org>
Tested-by: Simran Basi <sbasi@chromium.org>
fe/resources.py
fe/rpc_interface.py
fe/rpc_utils.py
fe/site_rpc_interface.py
fe/site_rpc_interface_unittest.py
lient/src/autotest/afe/AfeUtils.java
lient/src/autotest/afe/JobDetailView.java
lient/src/autotest/afe/create/CreateJobViewPresenter.java
lient/src/autotest/public/AfeClient.html
f4ea9b54fac2efeea8cbc2a7c75b5bffbd07897d 19-Nov-2013 Simran Basi <sbasi@chromium.org> Autotest: Rename 082_job_add_timeout_mins.py

Another migration got '082' so making this one '083'.

BUG=chromium:221264
TEST=Did move on CBF server and ran database/migrate sync and verified
the column appeared in afe_jobs.
DEPLOY=migrate

Change-Id: I76e48230ccdde2df63f6d87832474a142ce54bc6
Reviewed-on: https://chromium-review.googlesource.com/177304
Reviewed-by: Alex Miller <milleral@chromium.org>
Commit-Queue: Simran Basi <sbasi@chromium.org>
Tested-by: Simran Basi <sbasi@chromium.org>
igrations/082_job_add_timeout_mins.py
igrations/083_job_add_timeout_mins.py
94dc003dcce29c296a071add2f397013045c240c 12-Nov-2013 Simran Basi <sbasi@chromium.org> [Autotest] Update Database timeout to timeout_mins.

This change adds a timeout_mins column to the afe_jobs table and updates
exisiting jobs to include this entry.

Also updates the django models that reflect afe_jobs.
BUG=chromium:221264
DEPLOY=migrate, scheduler
TEST=Ran migration, checked column is in db, scheduler_models_unittest.py,
ran smoke suite on local AFE.

Change-Id: I09f8b6023a31dcef37e98c298dc0d63b37bc0a9e
Reviewed-on: https://chromium-review.googlesource.com/173916
Reviewed-by: Simran Basi <sbasi@chromium.org>
Tested-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Simran Basi <sbasi@chromium.org>
fe/doctests/001_rpc_test.txt
fe/models.py
igrations/082_job_add_timeout_mins.py
7f24f0bfe69b2f36ba74350fba9ad547ac55571c 12-Nov-2013 Fang Deng <fdeng@chromium.org> [autotest] test can output graph name when calling output_perf_value

Previously, if we want to group several perf measurements together
to be displayed on the same graph, we need to specify the mapping
information in perf_dashbaord_config.json. However, some tests
like telemetry_Benchmarks output many metrics. We would end up
with a very huge config file if we specify them all. It would require
much efforts if those metrics change in the future.
This change is to allow test to output graph names dynamically.

1) output_perf_value now takes a new arg 'graph'. This value is
written to the json file perf_measurements.
2) a new column 'graph' is added to the database table
tko_iteration_perf_value.
3) the tko parser is modified to store the value of 'graph' from
perf_measurements to the database.
4) perf_uploader is modified to upload a perf measurement to the
specific graph if 'graph' is set.

Example:
Previously, if we wanted to specify group "metric1" and "metric2"
to the same graph "metricGroupA", and group "metric3" and "metric4"
to "metricGroupB", the json needed to look like:
{
“autotest_name”: “myTest”,
“master_name”: “myMaster”,
“dashboard_test_name”: “myCoolTest”
“graphs”: [
{
“graph_name”: “metricGroupA”,
“descriptions”: [“metric1”, “metric2”]
},
{
“graph_name”: “metricGroupB”,
“descriptions”: [“metric3”, “metric4”]
},

]
}
After this change list lands, the json file will not
take "graphs" configuration. It will look like.
{
“autotest_name”: “myTest”,
“master_name”: “myMaster”,
“dashboard_test_name”: “myCoolTest”
}
Meanwhile, the test should call
output_perf_value(description="metric1", units=.., higher_is_better=..,
graph='metricGroupA')
output_perf_value(description="metric2", units=.., higher_is_better=..,
graph='metricGroupA')
output_perf_value(description="metric3", units=.., higher_is_better=..,
graph='metricGroupB')
output_perf_value(description="metric4", units=.., higher_is_better=..,
graph='metricGroupB')

BUG=chromium:315274
TEST=1) Verified all unit tests passed: version_1_unittest.py,
perf_uploader_unittest.py
2) Modify perf_uploader.py to use staging perf dashboard server
_DASHBOARD_UPLOAD_URL =
'https://chrome-perf.googleplex.com/add_point'
Modify platform_BootPerfServer to output graph name.
Invoked platform_BootPerfServer from local AFE.
Verify that the perf_measurements file have the graph name.
Verify that the data was displayed correctly in the dashboard.
3) python migrate.py sync 82, to verify column 'graph' get created
python migrate.py sync 81, to verify column 'graph' is dropped
DEPLOY=migrate (/usr/local/autotest/database$ python migrate.py sync 82)

Change-Id: I0b746f4ab3162eb7ba4aefa12294b7a60e326513
Reviewed-on: https://chromium-review.googlesource.com/176575
Reviewed-by: Richard Barnette <jrbarnette@chromium.org>
Commit-Queue: Fang Deng <fdeng@chromium.org>
Tested-by: Fang Deng <fdeng@chromium.org>
igrations/082_add_tko_iteration_perf_value_graph_field.py
1b2493186b54a7707e9e3c731021049d49932c75 13-Nov-2013 Alex Miller <milleral@chromium.org> [autotest] Move rpc stats down the stack to cover all attempted RPCs.

This will log all RPCs, afe and tko, and even ones that error out or
don't make sense. This should let us get a realistic view of the RPC
load on our systems.

TEST=restarted apache, saw stats on chromeos-stats/
DEPLOY=apache
BUG=None

Change-Id: I8223347e86b358fa97c44f956c656a380edb3c48
Reviewed-on: https://chromium-review.googlesource.com/176591
Reviewed-by: Alex Miller <milleral@chromium.org>
Commit-Queue: Alex Miller <milleral@chromium.org>
Tested-by: Alex Miller <milleral@chromium.org>
fe/json_rpc/serviceHandler.py
fe/rpc_interface.py
fe/site_rpc_interface.py
faecbce48bd329dcefe23516b2cd21603f8e6fb5 29-Oct-2013 beeps <beeps@chromium.org> [autotest] Fix parent/child and multiple aborts.

The queue_entry_id is a primary key in afe_aborted_host_queue_entries, which
should precludes us from trying to abort the same job multiple times. It will
happen if we abort a parent job (which leads to the abortion of its children),
and then abort a child in that set before the scheduler sets the complete bit.

The complete bit is set when we abort the hqe, so depending on where the
scheduler is in the tick, this leaves a ~30 second window within which our
second child abort will lead to an IntegrityError. This window is actually much
longer if the HQE is already in PARSING, since we won't set the complete bit
till the epilog of the PostJobTask.

TEST=Aborted suites, then their children. Tried to abort complete jobs.
BUG=chromium:308010
DEPLOY=Apache

Change-Id: Ib112502e966f5575c523ac053e94d44bd92359d6
Reviewed-on: https://chromium-review.googlesource.com/174988
Tested-by: Prashanth B <beeps@chromium.org>
Reviewed-by: Alex Miller <milleral@chromium.org>
Commit-Queue: Prashanth B <beeps@chromium.org>
fe/rpc_interface.py
f0c752204cfcbb1ec0c84540cfcfbe364839a3b2 23-Oct-2013 Alex Miller <milleral@chromium.org> [autotest] Only stage test_suites in create_suite_job.

This reverts commit ecc0b7086ab66557b0009fc8d0765710fc593b41, with some
fixed unitttests.

TEST=unit
BUG=None

Change-Id: I250f3204fdd5fb0fef8a10fa6dcb27f9bd041994
Reviewed-on: https://chromium-review.googlesource.com/174246
Reviewed-by: Alex Miller <milleral@chromium.org>
Commit-Queue: Alex Miller <milleral@chromium.org>
Tested-by: Alex Miller <milleral@chromium.org>
fe/site_rpc_interface.py
fe/site_rpc_interface_unittest.py
ecc0b7086ab66557b0009fc8d0765710fc593b41 23-Oct-2013 Alex Miller <milleral@chromium.org> Revert "[autotest] Only stage test_suites in create_suite_job."

Yay for precq. boo for reading.

This reverts commit 3a171e0b406123c93687cb9938878d9c5d809a88.

Change-Id: I69bdea6340e8c25de459e4f0d1f871afa236c7b2
Reviewed-on: https://chromium-review.googlesource.com/174245
Reviewed-by: Alex Miller <milleral@chromium.org>
Commit-Queue: Alex Miller <milleral@chromium.org>
Tested-by: Alex Miller <milleral@chromium.org>
fe/site_rpc_interface.py
3a171e0b406123c93687cb9938878d9c5d809a88 22-Oct-2013 Alex Miller <milleral@chromium.org> [autotest] Only stage test_suites in create_suite_job.

Other pieces of the infrastructure will now stage their own artifacts,
so we no longer need to wait on the full payloads to download.

BUG=chromium:282541
DEPLOY=apache
TEST=scheduled a suite locally. ran.

Change-Id: I73272e31c156c22a935663596639303a9a7f0ab8
Reviewed-on: https://chromium-review.googlesource.com/174106
Tested-by: Alex Miller <milleral@chromium.org>
Reviewed-by: Prashanth B <beeps@chromium.org>
Commit-Queue: Alex Miller <milleral@chromium.org>
fe/site_rpc_interface.py
6e6981cde7dceb974d29dac1d7e0718b060ae9b6 28-Sep-2013 Aaron Gable <agable@google.com> Rewrite git/gerrit/gerrit-int urls to googlesource

BUG=chromium:281665,chromium:290413

TEST=None

Change-Id: Ic4e9a0fc3a98d4ee76021868fea695d8b73414b6
Reviewed-on: https://chromium-review.googlesource.com/171285
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Tested-by: Aaron Gable <agable@chromium.org>
Reviewed-by: Richard Barnette <jrbarnette@chromium.org>
Commit-Queue: Aaron Gable <agable@chromium.org>
emplates/emails/tests_failure.html
emplates/tables/configs/header.html
emplates/tables/details/details.html
emplates/tables/performance/build_performance.html
emplates/tables/summary/emails.html
7cd12315f095d45c697267df9a37427ed87f9ea3 25-Jul-2013 Aviv Keshet <akeshet@chromium.org> [autotest] add ability for test_that to run jobs in lab

This CL adds the ability for test_that to run jobs in the lab. It
accomplishes this thusly:

1) A new optional |suite_args| argument to afe.create_suite_job, whose
value gets injected into the suite control file in much the same way as
job args are injected into job control files.

2) Plumbing to thread #1 through from run_suite.py.

3) A new suite control file, control.test_that_wrapper, that is
responsible for kicking of test_that-initiated suites. It expects to see
a suite_args string giving the command line that test_that was run with,
and re-uses test_that's argument parser to figure out which tests (and
therefore which suite predicates) are needed.

4) A small refactor to dynamic_suite, so that it uses suite's
create_from_predicates rather than create_from_name.

5) A new optional |predicate| argument to dynamic_suite's
reimage_and_run, which if specified causes dynamic suite to use this
job-matching predicate rather than its default suite-name predicate.

6) Code path in test_that to pass :lab: runs on to run_suite.py.

So, when a developer runs, at their desk:
$ test_that -b board -i image :lab: dummy_Pass
Then test_that will, kick off a test_that_wrapper suite in the lab, using
run_suite, passing along test_that's args into the suite control file so
that it can figure out which tests to run:
$ run_suite -b board -i image -s test_that_wrapper --pool try-bot \
--suite_args="-b board -i image :lab: dummy_Pass"

This CL also fixes a bug in dynamic_suite/tools.py in which strings that
contained quotes were being incorrectly injected into control files in
inject_var.

CQ-DEPEND=CL:I4b1483b977d9032c7adf8117cf32e83c65eed007
BUG=chromium:262984
TEST=Unit tests pass.
Ran some test_that jobs against my local autotest instance, using
a trybot image that was build with this CL (so that
control.test_that_wrapper would be available).
$ test_that -b lumpy -i ${image} :lab: dummy_Pass
$ test_that -b lumpy -i ${image} :lab: e:login_.*
$ test_that -b lumpy -i ${image} :lab: suite:smoke dummy_Fail
DEPLOY=apache

Change-Id: Ifa73d7de7aac9c6efebd5f559708623804ad3691
Reviewed-on: https://chromium-review.googlesource.com/63378
Reviewed-by: Aviv Keshet <akeshet@chromium.org>
Tested-by: Aviv Keshet <akeshet@chromium.org>
Commit-Queue: Aviv Keshet <akeshet@chromium.org>
fe/site_rpc_interface.py
139690b24c630e518bff3076e32384a69d9bfe59 08-Sep-2013 Alex Miller <milleral@chromium.org> [autotest] Accept `timeout` as an arg for create_suite_job.

And thread it through to the suite control files.

BUG=chromium:221264
TEST=unit, run_suite

Change-Id: Ibefdab2193467623c2855719d3c2061da1855b64
Reviewed-on: https://chromium-review.googlesource.com/168566
Reviewed-by: Dan Shi <dshi@chromium.org>
Tested-by: Alex Miller <milleral@chromium.org>
Commit-Queue: Alex Miller <milleral@chromium.org>
fe/site_rpc_interface.py
fe/site_rpc_interface_unittest.py
b020606be3158d0650a6e94da316373e8e58e245 08-Sep-2013 Alex Miller <milleral@chromium.org> [autotest] `Clone job` is currently broken on the AFE. This unbreaks.

I missed a case where priority was still expected to be a string. :(

There's also a fix in here that if you clone a job with a higher
priority than what you're allowed to schedule, it's automatically bumped
down to the maximum of what you can schedule.

BUG=None
DEPLOY=afe
TEST=cloned job

Change-Id: Ie00531ea478379b753c6a3b3bbbb1cac82da040e
Reviewed-on: https://chromium-review.googlesource.com/168560
Reviewed-by: Alex Miller <milleral@chromium.org>
Commit-Queue: Alex Miller <milleral@chromium.org>
Tested-by: Alex Miller <milleral@chromium.org>
lient/src/autotest/afe/create/CreateJobViewPresenter.java
7d658cf6bade565c1098fd7b47075e96e7b542ca 05-Sep-2013 Alex Miller <milleral@chromium.org> [autotest] Thread priority from create_suite_job through to create_job.

This schedules a suite job itself at the same priority level of its
tests, and also injects the priority value into the suite control files.

The change to thread priority through from dynamic_suite into create_job
needs to happen concurrently, because the type of `priority` is changing
from a string to an int. This also means everything that passes in a
priority gets fixed up in this CL.

suite_scheduler has also been tweaked to schedule all tests at the
PostBuild priority (for now...).

This also does all of the work to get priority to show up by name
instead of by integral value. In the create_job pane, we disallow
anyone to schedule a job at a priority that would preempt major build
processes (anything coming from a waterfall). However, all old jobs are
shown at the integral value, because showing two seperate priority
scales at the same time would be incredibly confusing. (Especially
since URGENT would be incredibly low priority)

BUG=chromium:250583
DEPLOY=afe, apache, suite_scheduler
TEST=unit, run_suite with a priority level

Change-Id: I9ecf5ceed5c58bd8ee0815c6d15f4aba300082fe
Reviewed-on: https://chromium-review.googlesource.com/168143
Reviewed-by: Alex Miller <milleral@chromium.org>
Tested-by: Alex Miller <milleral@chromium.org>
Commit-Queue: Alex Miller <milleral@chromium.org>
fe/doctests/001_rpc_test.txt
fe/models.py
fe/resources.py
fe/rpc_interface.py
fe/rpc_interface_unittest.py
fe/rpc_utils.py
fe/site_rpc_interface.py
fe/site_rpc_interface_unittest.py
lient/src/autotest/afe/JobDetailView.java
lient/src/autotest/afe/JobTable.java
lient/src/autotest/afe/create/CreateJobViewDisplay.java
lient/src/autotest/afe/create/CreateJobViewPresenter.java
lient/src/autotest/common/StaticDataRepository.java
a776c8bae51a0b5472602bf66ebd903dc779a6ab 27-Aug-2013 Keyar Hood <keyar@chromium.org> [autotest] Create generic bug filer function.

This makes it easier to file simple generic bugs. The user doesn't need to
create the Bug, create the Reporter and then call report().

BUG=None
TEST=Unittests. Manual test for passing_experimental.py.

Change-Id: I6d27cfa6a72b8dd4d22d8301b1c3105b90363a5a
Reviewed-on: https://chromium-review.googlesource.com/67009
Reviewed-by: Prashanth Balasubramanian <beeps@chromium.org>
Tested-by: Keyar Hood <keyar@chromium.org>
Commit-Queue: Prashanth Balasubramanian <beeps@chromium.org>
ealth/passing_experimental.py
ealth/passing_experimental_functional_test.py
4a193695bd3038f0d44f039584fc5c5015b13404 21-Aug-2013 Alex Miller <milleral@chromium.org> [autotest] create_job should fail if the metahost cannot run the job

If one passes a specific list of hosts into `create_job`, then those
hosts are vetted to make sure they satisfy the dependencies of the job.

If one passes in a metahost, no checking is done. This is unfortunate,
as it means one needs to be very careful when scheduling jobs with
metahosts to make sure that it is actually possible for the job to run.
Instead, let's just do the checking in the RPC and raise an
easy-to-identify Exception if we do hit this case.

BUG=chromium:250586
DEPLOY=apache
TEST=unit, Scheduled a job with a metahost and DEPENDENCIES that did not
exist. Got an error.NoEligibleHostException. Both dummy and bvt still
run (testing provisionable labels and tests with unsatisfyable
dependencies).

Change-Id: I020be2607867fafd04194b9c17d3052b006f60e0
Reviewed-on: https://chromium-review.googlesource.com/66603
Tested-by: Alexander Miller <milleral@chromium.org>
Reviewed-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Prashanth Balasubramanian <beeps@chromium.org>
Commit-Queue: Alexander Miller <milleral@chromium.org>
fe/frontend_test_utils.py
fe/json_rpc/proxy.py
fe/resources_test.py
fe/rpc_interface_unittest.py
fe/rpc_utils.py
551b37e3f2c2236a882bddd8c842714f1ea6fa9e 22-Aug-2013 Keyar Hood <keyar@chromium.org> [autotest] passing_experimental.py now files bugs.

Instead of sending emails, passing_experimental.py will now file bugs.

manual_check_passing_experimetnal.py is used to test that
passing_experimental.py correctly submits bug reports that are deduped. It
creates a fake database and then calls passing_experimental.py. It requires
that the user then manually check that the bug is correctly submitted and
deduped.

BUG=chromium:212671
TEST=manual_check_passing_experimetnal.py submits issues using the
passing_experimental.py that are then manually checked. The unittests are also
used for testing.

Change-Id: I8b25fc7e409710dbd783145dec6e33de639c12d7
Reviewed-on: https://gerrit.chromium.org/gerrit/66573
Reviewed-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
Tested-by: Keyar Hood <keyar@chromium.org>
Commit-Queue: Keyar Hood <keyar@chromium.org>
ealth/manual_check_passing_experimental.py
ealth/passing_experimental.py
ealth/passing_experimental_functional_test.py
33ca53b02d284a1cf99f7d5de5b49cf35b72c6ac 22-Aug-2013 Keyar Hood <keyar@chromium.org> [autotest] Test health scripts will run again...

Do not pass script name to the argparse in test health scripts. It throws an
error as it expects the passed arguments to exclude the script name.

BUG=None
TEST=Manual testing.

Change-Id: I84187211873994f0951f77b7bfa27390015ae771
Reviewed-on: https://gerrit.chromium.org/gerrit/66576
Reviewed-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
Commit-Queue: Keyar Hood <keyar@chromium.org>
Tested-by: Keyar Hood <keyar@chromium.org>
ealth/check_test_health.py
ealth/complete_failures.py
ealth/passing_experimental.py
2ff7b41552b92f0a0fd688467a2209656b017b09 20-Aug-2013 Keyar Hood <keyar@chromium.org> [autotest]: Add --help to test health scripts.

Although these scripts have no interesting options, people will not be
suprised when they try to find out more and suddenly the script is bein ran.

BUG=chromium:276668
TEST=Manual testing.

Change-Id: I247c1e681a26314ffb74f1905575be4a1706deb8
Reviewed-on: https://gerrit.chromium.org/gerrit/66441
Reviewed-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
Commit-Queue: Keyar Hood <keyar@chromium.org>
Tested-by: Keyar Hood <keyar@chromium.org>
ealth/check_test_health.py
ealth/complete_failures.py
ealth/passing_experimental.py
488d70cb50caeeb0da4b6545b74a31ba659639d2 21-Aug-2013 Aviv Keshet <akeshet@chromium.org> [autotest] add a parent_job_id index to afe_jobs table

This CL adds a parent_job_id index to the afe_jobs table, which is
particularly important in light of the pending CL
I47dbc5f1a3348bdd5297ffbd2e26b82dc9d705f6.

BUG=chromium:276471
TEST=From the mysql query log, determined the underling SQL query being
used in get_jobs(parent_job_id=xxx, finished=True). Then did a
before/after comparison of the mysql EXPLAIN report.

mysql> explain extended SELECT DISTINCT id from `afe_jobs` WHERE
(`afe_jobs`.`parent_job_id` = 219 AND (id NOT IN (SELECT job_id FROM
afe_host_queue_entries WHERE not complete))) \G;
*************************** 1. row ***************************
id: 1
select_type: PRIMARY
table: afe_jobs
type: ALL
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: 232
filtered: 100.00
Extra: Using where
*************************** 2. row ***************************
id: 2
select_type: DEPENDENT SUBQUERY
table: afe_host_queue_entries
type: index_subquery
possible_keys: host_queue_entries_job_id_and_host_id
key: host_queue_entries_job_id_and_host_id
key_len: 5
ref: func
rows: 2
filtered: 100.00
Extra: Using where
2 rows in set, 1 warning (0.00 sec)

after:
*************************** 1. row ***************************
id: 1
select_type: PRIMARY
table: afe_jobs
type: ref
possible_keys: parent_job_id_index
key: parent_job_id_index
key_len: 5
ref: const
rows: 2
filtered: 100.00
Extra: Using where; Using index
*************************** 2. row ***************************
id: 2
select_type: DEPENDENT SUBQUERY
table: afe_host_queue_entries
type: index_subquery
possible_keys: host_queue_entries_job_id_and_host_id
key: host_queue_entries_job_id_and_host_id
key_len: 5
ref: func
rows: 2
filtered: 100.00
Extra: Using where

DEPLOY=migrate

Change-Id: I4b407fd0cc1a769ad0d4829bdffed87213e2da04
Reviewed-on: https://gerrit.chromium.org/gerrit/66464
Tested-by: Aviv Keshet <akeshet@chromium.org>
Reviewed-by: Alex Miller <milleral@chromium.org>
Commit-Queue: Aviv Keshet <akeshet@chromium.org>
igrations/081_index_afe_jobs_on_parent_id.py
708f1fbedbf992f18d6be7da9c310d800f0cd847 12-Aug-2013 Keyar Hood <keyar@chromium.org> Autotest: Clean up unneeded test-health storage.

The complete_failures.py script was still using file storage but was not doing
anything with it. This patch removes this storage as there is no longer plans to
store this information.

BUG=None
TEST=Unit tests.

Change-Id: I5fe092d6aebe133676c95d8fc5681881f9d1c894
Reviewed-on: https://gerrit.chromium.org/gerrit/65658
Reviewed-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
Tested-by: Keyar Hood <keyar@chromium.org>
Commit-Queue: Keyar Hood <keyar@chromium.org>
ealth/complete_failures.py
ealth/complete_failures_functional_test.py
ealth/complete_failures_unittest.py
cfc2c23e4c7ff35049b695b35c66725d85c4334a 15-Aug-2013 Ilja Friedel <ihf@chromium.org> ffmpeg_tests removal.

Chromium is removing this test binary. Hence CrOS is removing everything that
depends on it.

BUG=chromium:273278
TEST=Emerged chromeos-chrome and autotest* for Daisy.

Change-Id: I357fea769eb54e1cd5ba39b1c2c8596256edfa69
Reviewed-on: https://gerrit.chromium.org/gerrit/65908
Reviewed-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
Reviewed-by: Dale Curtis <dalecurtis@chromium.org>
Commit-Queue: Ilja H. Friedel <ihf@chromium.org>
Reviewed-by: Ilja H. Friedel <ihf@chromium.org>
Tested-by: Ilja H. Friedel <ihf@chromium.org>
roschart/croschart_defaults.json
erf-dashboard/croschart_defaults.json
f0e327d999927d6da3d06c71ef6c56876c0fd19d 09-Aug-2013 Keyar Hood <keyar@chromium.org> Autotest: Fix unittest file names so they are ran.

The automatic test finding script does not find files of the form
*_unittests.py. These have been renamed to be *_unittest.py.

BUG=None
TEST=Ran test ./utils/unittests_suite.py with --debug flag and verified tests
were ran.

Change-Id: Idfc6ee90a19138d486e9167620d6e4df14b84c76
Reviewed-on: https://gerrit.chromium.org/gerrit/65270
Reviewed-by: Keyar Hood <keyar@chromium.org>
Tested-by: Keyar Hood <keyar@chromium.org>
Commit-Queue: Keyar Hood <keyar@chromium.org>
ealth/passing_experimental_unittest.py
ealth/passing_experimental_unittests.py
ealth/utils_unittest.py
ealth/utils_unittests.py
01e1d9e97d6e0cbf7dc9bd3b7166491dd3b4a4ea 14-Aug-2013 Keyar Hood <keyar@chromium.org> Autotest: Create test health script runner.

The check_test_health.py script will be ran as a cron job. This will allow
people to add more analysis by just modifying this script.

Furthermore, this has a preperation step that allows scripts that prepare the
database to be ran before the analysis steps. In particular, this means we
moved the call to test_importer.py from passing_experimental.py to
check_test_health.py.

BUG=chromium:272534
TEST=Autotests.

Change-Id: I438bd69e1f8688ba7e94910c9db9c4cac35fffcf
Reviewed-on: https://gerrit.chromium.org/gerrit/65754
Reviewed-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
Commit-Queue: Keyar Hood <keyar@chromium.org>
Tested-by: Keyar Hood <keyar@chromium.org>
ealth/check_test_health.py
ealth/check_test_health_unittest.py
ealth/complete_failures.py
ealth/passing_experimental.py
ealth/passing_experimental_functional_test.py
1d2826b7cd5fea3bee9ac90b2b7a095816c6c1c5 13-Aug-2013 Simran Basi <sbasi@chromium.org> Autotest: Make parent job id clickable.

Creates a url to point to the parent id that will link you directly
to the parent id page.

Includes a special easter egg for jobs with no parent job id.

BUG=chromium:221584
TEST=viewed jobs locally.
DEPLOY=utils/compile_gwt_clients.py and restart apache.

Change-Id: I8e8b019fce2b184d610d2a400b8e4eb081aea66e
Reviewed-on: https://gerrit.chromium.org/gerrit/65642
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Commit-Queue: Simran Basi <sbasi@chromium.org>
Tested-by: Simran Basi <sbasi@chromium.org>
lient/src/autotest/afe/JobDetailView.java
lient/src/autotest/public/AfeClient.html
871291b33463733e015a36f888f7ee88ce48baaf 08-Aug-2013 Alex Miller <milleral@chromium.org> [autotest] Only pull labels that we need when creating a job.

Instead of building a giant dictionary of all the labels, and then
pulling the ones we need from it, we can just ask Django to turn
[list of label string names] into [list of label objects], thus avoiding
pulling all ~30k of labels.

This should save a couple seconds per `create_job` RPC in prod.

This also removes case-insensitivity from label scheduling, which is
something I wasn't aware that we have, and I don't think the scheduler
obeys, so all you crazy kids can't go scheduling against
`bOaRd:dAiSy_SpRiNg` anymore.

BUG=chromium:267105
DEPLOY=apache
TEST=atest job create still works

Change-Id: I6bee6a9744233d3f344764831c98d252ac9031b7
Reviewed-on: https://gerrit.chromium.org/gerrit/65156
Commit-Queue: Alex Miller <milleral@chromium.org>
Reviewed-by: Alex Miller <milleral@chromium.org>
Tested-by: Alex Miller <milleral@chromium.org>
fe/rpc_utils.py
0f48660cb3c4cce2fef87735f839b314d2139219 08-Aug-2013 Alex Miller <milleral@chromium.org> [autotest] Index finding HQEs to abort.

This query does a full table scan, and takes about a second in prod. We
can index that down to almost nothing.

BUG=chromium:270215
DEPLOY=migrate
TEST=ran sql locally

Change-Id: Ib816ef2ad667926c937f014c632cfed5aa8b7fd1
Reviewed-on: https://gerrit.chromium.org/gerrit/65228
Tested-by: Alex Miller <milleral@chromium.org>
Reviewed-by: Aviv Keshet <akeshet@chromium.org>
Commit-Queue: Alex Miller <milleral@chromium.org>
igrations/080_index_abort_host_queue_entries.py
8bb1f7d8766d202d220879642a7d5c40e2775e52 05-Aug-2013 beeps <beeps@chromium.org> [autotest] Teach the scheduler to abort SpecialTasks.

1. Database migration that adds a column to afe_special_tasks.
2. An rpc that modifies the db with the abort bit.
3. A scheduler method that polls the database for aborted jobs
and handles the agent.

TEST=Scheduled a job and aborted it's special tasks. Checked that
subsequent special tasks were scheduled.
Aborted Repair->host goes into repair fail.
Aborted (Reset, Verify, Cleanup)->Repair queued.
Checked that the Hqe is requeued through PreJobTask epilog.
Aborted SpecialTasks without hqes.
Aborted jobs normally through the frontend.

BUG=chromium:234223
DEPLOY=migrate, apache, scheduler

Change-Id: I1a47bc2d801486a8abdffb44091c59a8f5bdbefc
Reviewed-on: https://gerrit.chromium.org/gerrit/64753
Commit-Queue: Prashanth Balasubramanian <beeps@chromium.org>
Reviewed-by: Prashanth Balasubramanian <beeps@chromium.org>
Tested-by: Prashanth Balasubramanian <beeps@chromium.org>
fe/models.py
fe/rpc_interface.py
igrations/079_special_task_abort.py
dfff2fdc8477be3ba89fd915fde2afe8d3716624 28-May-2013 Alex Miller <milleral@chromium.org> [autotest] Add a provision special task.

We now insert a special task which calls |autoserv --provision| with the
host that the HQE is about to run on to provision the machine correctly
before the test runs. If the provisioning fails, the HQE will also be
marked as failed. No provisioning special task will be queued if no
provisioning needs to be done to the host before the job can/will run.

With *just* this CL, no provisioning tasks should actually get
scheduled, because the part of the scheduler that maps HQEs to hosts
hasn't been taught about provisioning yet. That will come in a later
CL.

Once this CL goes in, it should not be reverted. The scheduler will
become very unhappy if it sees special tasks in its database, but can't
find a corresponding AgentTask definition for them. One would need to
do manual database cleanup to revert this CL. However, since one can
disable provisioning by reverting the (future) scheduling change CL,
this shouldn't be an issue.

BUG=chromium:249437
DEPLOY=scheduler
TEST=lots:
* Ran a job on a host with a non-matching cros-version:* label, and
a provision special task was correctly created. It ran after Reset,
and correctly kicked off the HQE after it finished.
* Ran a job on a host with a matching cros-version:* label, and no
provision special task was created.
* Ran a job on a host with a non-matching cros-version:* label, and
modified Reset so that it would fail. When reset failed, it canceled
the provision task, and the HQE was still rescheduled.
* Ran a job on a host with a non-matching cros-version:* label, and
modified the cros-version provisioning test to throw an exception.
The provision special task aborted the HQE with the desired semantics
(see comments in the ProvisionTask class in monitor_db), and scheduled
a repair to run after its failure.
The provision failures were all deduped against each other when bug
filing was enabled. See
https://code.google.com/p/autotest-bug-filing-test/issues/detail?id=1678
* Successfully debugged an autoupdate/devserver issue from provision
logs, thus proving that sufficient information is collected for debug.

Change-Id: I96dbfc7b001b90e7dc09e1196c0901adf35ba4d8
Reviewed-on: https://gerrit.chromium.org/gerrit/58385
Reviewed-by: Prashanth Balasubramanian <beeps@chromium.org>
Tested-by: Alex Miller <milleral@chromium.org>
Commit-Queue: Prashanth Balasubramanian <beeps@chromium.org>
fe/models.py
fe/rpc_interface.py
e0424b5ea1e4baa8439c84a3897417aefd4a89d6 06-Aug-2013 Keyar Hood <keyar@chromium.org> Autotest: Fix incorrect tearDowns for unittests.

The tearDowns now correctly call the superclass' tearDown functions in
passing_experimental_unittests.py. Before they were calling the super class'
setUp.

BUG=None
TEST=Ran unittests again.

Change-Id: I3d8d2b18ca025678ddc7fb87391f9dc4c111891e
Reviewed-on: https://gerrit.chromium.org/gerrit/64717
Reviewed-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
Commit-Queue: Keyar Hood <keyar@chromium.org>
Tested-by: Keyar Hood <keyar@chromium.org>
ealth/passing_experimental_unittests.py
c68807ece645ebab7fa74af4825de0d60c7d4918 01-Aug-2013 Aviv Keshet <akeshet@chromium.org> [autotest] do not enforce satisfiabiliy of provisionable label

Provisionable labels should not already need to exist and be applied to
a host in order for a job to be validated (hosts with acquire the label
once they are provisioned).

Because of some strange database/transauction/consistency weirdness, I
was not able to use server.cros.provision.ensure_label_exists from
inside of create_new_job in rpc_utils. I will file a separate bug about
that mystery.

BUG=chromium:265660
TEST=In a fresh autotest database with no labels yet created:
`./atest job create -d carrot -b board:lumpy --test=dummy_Pass
dummy_Pass` -> ValidationError as desired.
`./atest job create -d cros-version:carrot -b board:lumpy --test=dummy_Pass
dummy_Pass` -> Job and label (cros-version:carrot) created.

DEPLOY=apache

Change-Id: I9b93811f401a82e68dbad7388b655ab1ffec2602
Reviewed-on: https://gerrit.chromium.org/gerrit/64242
Commit-Queue: Aviv Keshet <akeshet@chromium.org>
Reviewed-by: Aviv Keshet <akeshet@chromium.org>
Tested-by: Aviv Keshet <akeshet@chromium.org>
fe/rpc_utils.py
30d94bdad75c6aa6cfe81d365c6275be9235f531 23-Jul-2013 Keyar Hood <keyar@chromium.org> Autotest: Find long passing experimental tests.

This adds a script to that finds any autotest test that has been marked
as experimental but has been passing for at least 30 days. An email is sent
about these tests so that the appropriate action can be taken.

BUG=chromium:212671
TEST=Unittests. Manual testing by commenting out where emails are sent (so
as not to spam people). Also by commenting out the database updating call when
checking against the real database.

Change-Id: Ieae34adc2ef49458e50a5dd7d6f9851743e80a10
Reviewed-on: https://gerrit.chromium.org/gerrit/63479
Reviewed-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
Tested-by: Keyar Hood <keyar@chromium.org>
Commit-Queue: Keyar Hood <keyar@chromium.org>
ealth/passing_experimental.py
ealth/passing_experimental_functional_test.py
ealth/passing_experimental_unittests.py
ealth/utils.py
ealth/utils_unittests.py
a35647145cb53dc2b8b55ae2a0a62c01744831a1 30-Jul-2013 Keyar Hood <keyar@chromium.org> Autotest: Better complete_failures.py email info.

The number of tests failed out of the total number of tests is given. Also,
the results are sorted alphabetically.

BUG=chromium:265929
TEST=Unittests.

Change-Id: I45847ec8958d390ae12cd41134cc568ac84c13e2
Reviewed-on: https://gerrit.chromium.org/gerrit/63680
Reviewed-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Keyar Hood <keyar@chromium.org>
Tested-by: Keyar Hood <keyar@chromium.org>
ealth/complete_failures.py
ealth/complete_failures_functional_test.py
ealth/complete_failures_unittest.py
0f26dbaff69d0135c527ed87c1af1fd5fb56ad33 19-Jul-2013 Keyar Hood <keyar@chromium.org> Autotest: Refactor complete_failures slightly.

Move the get_last_pass_times functions to a common area as it will be useful
for future scripts. Also tests are looked up based on status name instead
of status id.

BUG=None
DEPLOY=None
TEST=Unittests and ran script manually to verify

Change-Id: I398237d0f11bef63492e10a1206387bb2058dc40
Reviewed-on: https://gerrit.chromium.org/gerrit/62570
Reviewed-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
Tested-by: Keyar Hood <keyar@chromium.org>
Commit-Queue: Keyar Hood <keyar@chromium.org>
ealth/complete_failures.py
ealth/complete_failures_unittest.py
ealth/utils.py
ealth/utils_unittests.py
2c72396a8c952b9bb0804c5bb5cac92b0b0f8bff 12-Jul-2013 Dennis Jeffrey <dennisjeffrey@chromium.org> Add a results database migration to create the tko_iteration_perf_value table.

This change is part of a series of CLs in which we're changing how
autotest stores and handles perf test data. Previously, we added a
new function for perf tests to output perf measurements to an output
file. In the current CL, we're adding a new results database table.
In a subsequent CL, this table will be used by the TKO parser to store
the outputted perf measurements.

BUG=chromium:258229
DEPLOY=migrate (/usr/local/autotest/database$ python migrate.py sync 78)
TEST="python migrate.py sync 78" to verify using a local autotest database
that the new table gets created.
"python migrate.py sync 77" to verify that the new table is dropped if
we migrate back down.

Change-Id: I210a42b3c3c5ec1b7aa26dd0caabda5f6e2e2bb9
Reviewed-on: https://gerrit.chromium.org/gerrit/61706
Reviewed-by: Simran Basi <sbasi@chromium.org>
Tested-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
Commit-Queue: Dennis Jeffrey <dennisjeffrey@chromium.org>
igrations/078_add_tko_iteration_perf_value.py
07e09aff0baf871b33e5479e337e5e3e0523b729 12-Apr-2013 Dan Shi <dshi@chromium.org> [Autotest] merge cleanup and verify

The objective of this CL is to merge cleanup and verify into a single job to
reduce run time of each test. In existing design, by default, a cleanup job is
scheduled after a test is finished, and a verify job is scheduled before a
test is started. By merging these two jobs together, we are seeing the total
run time of these two jobs is reduced from about 47s to 37s, around 10s saving.
That does not include the saving on scheduler to schedule two jobs, which may
take another 5-10s.

The design is to create a new special task, reset, which runs at the beginning
of a job by default. Verify task is changed to not to run by default before a
job starts. Cleanup job will only be run if a job is scheduled to reboot and
any test failed in that job.

BUG=chromium:220679
TEST=tested with run_suite in local machine
DEPLOY=afe,apache,scheduler,change all users' preference on reboot_after to
Never, sql: |update chromeos_autotest_db.afe_users set reboot_after=0|

Change-Id: Ia38baf6b73897b7e09fdf635eadedc752b5eba2f
Reviewed-on: https://gerrit.chromium.org/gerrit/48685
Commit-Queue: Dan Shi <dshi@chromium.org>
Reviewed-by: Dan Shi <dshi@chromium.org>
Tested-by: Dan Shi <dshi@chromium.org>
fe/doctests/001_rpc_test.txt
fe/models.py
fe/resources.py
fe/resources_test.py
fe/rpc_interface.py
fe/rpc_utils.py
lient/src/autotest/afe/HostDetailView.java
lient/src/autotest/afe/create/CreateJobViewDisplay.java
lient/src/autotest/afe/create/CreateJobViewPresenter.java
lient/src/autotest/public/AfeClient.html
igrations/077_add_run_reset.py
3d8076899b7ae59c6099934b2c797e34553f45bb 02-Jul-2013 Keyar Hood <keyar@chromium.org> Autotest: Minor refactor for complete_failures.py

Separate the storing logic from the reporting logic.

BUG=None
DEPLOY=None
TEST=Unittests plus ran script.

Change-Id: I4a5265c03bcbf3c0fcee5abafcfb34308bf283ff
Reviewed-on: https://gerrit.chromium.org/gerrit/60697
Reviewed-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
Tested-by: Keyar Hood <keyar@chromium.org>
Commit-Queue: Keyar Hood <keyar@chromium.org>
ealth/complete_failures.py
ealth/complete_failures_unittest.py
4d5f9d1ed2fe6ddc31f30ceede342d525f2e2616 29-Jun-2013 Keyar Hood <keyar@chromium.org> Autotest: Always email all the long failing tests.

We probably will not need to send emails that only show differences between
failures we already know and ones we just found out about. This change will
also make it a bit easier to separate out the storage logic from the reporting
logic.

BUG=chromium:255681
DEPLOY=none
TEST=Unittests

Change-Id: I72cb92496382e2949c964f79263e2b3025b8f697
Reviewed-on: https://gerrit.chromium.org/gerrit/60540
Reviewed-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
Tested-by: Keyar Hood <keyar@chromium.org>
Commit-Queue: Keyar Hood <keyar@chromium.org>
ealth/complete_failures.py
ealth/complete_failures_unittest.py
e7b118479a372b12619abc97124396240be191a1 26-Jun-2013 Keyar Hood <keyar@chromium.org> AUTOTEST: Check recently ran tests for health.

complete_failures.py will now only check tests that have been ran recently.

BUG=chromium:253599
DEPLOY=none
TEST=Unittests plus ran script and manually verified.

Change-Id: I475a16fd1f4d4fb2330eb94884eea4b3285ef965
Reviewed-on: https://gerrit.chromium.org/gerrit/60113
Reviewed-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
Commit-Queue: Keyar Hood <keyar@chromium.org>
Tested-by: Keyar Hood <keyar@chromium.org>
ealth/complete_failures.py
ealth/complete_failures_functional_test.py
ealth/complete_failures_unittest.py
97582a2097bfb27a41514e9eaa071de50d1f7be6 27-Jun-2013 Simran Basi <sbasi@chromium.org> Fix bad reference in models.py

Left behind a bug in models.py referencing an non-existant module.
Fixing to point to the correct functions.

BUG=chromium:255130
TEST=unittests

Change-Id: I4034669b74a72cbde9f4437c0ca4f7fed9bfefd7
Reviewed-on: https://gerrit.chromium.org/gerrit/60253
Reviewed-by: Aviv Keshet <akeshet@chromium.org>
Tested-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Simran Basi <sbasi@chromium.org>
fe/models.py
c1b26769845fe59168f2d9b7d6b1d37d93ea4bf1 26-Jun-2013 Simran Basi <sbasi@chromium.org> Fix duplicate entry Integrity Error when aborting from AFE

Currently aborting a suite job and its sub jobs from the afe causes
the AFE to spit out duplicate entry errors.

This is do the suite job aborting before a child job does and aborts
the child job. However the model object for that child job has not
been updated so we try to abort it as well and create a new
aborted host queue entry for it but it already exists.

Refactored the logic from abort to a class method that takes in a list
of HQE's to abort. This can be shared between abort and the rpc_interface
code.

BUG=chromium:242040
TEST=Started a suite, did a bulk abort and the error no longer occured.

Change-Id: I0d82506234bac5fea9307b5cd20c836cdf7ccb9c
Reviewed-on: https://gerrit.chromium.org/gerrit/60138
Commit-Queue: Simran Basi <sbasi@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Tested-by: Simran Basi <sbasi@chromium.org>
fe/models.py
fe/rpc_interface.py
005539dbc39ccba6b8c97771cda2dba0d8a163f3 24-Jun-2013 Keyar Hood <keyar@chromium.org> Autotest: Filter out invalid test names.

BUG=chromium:253597
DEPLOY=none
TEST=Unittests plus ran script and manually verified.

Change-Id: Ifcdc9934438d9701bfc62e5595bb6b25e4a1e590
Reviewed-on: https://gerrit.chromium.org/gerrit/59841
Commit-Queue: Keyar Hood <keyar@chromium.org>
Reviewed-by: Keyar Hood <keyar@chromium.org>
Tested-by: Keyar Hood <keyar@chromium.org>
ealth/complete_failures.py
ealth/complete_failures_functional_test.py
ealth/complete_failures_unittest.py
2d218a09789e34c907568a61f31d9cd70fcf556b 22-Jun-2013 Aviv Keshet <akeshet@chromium.org> [autotest] suppress ImportError on gviz_api and matplotlib

Suppressing these ImportErrors allows unit tests to run inside the
chroot without needing to install gviz_api and matplotlib in the chroot
(both of which do not have a readily-installable portage ebuild).

BUG=chromium:221254
TEST=autotest_lib.frontend.afe.resources_test passes inside the chroot,
after removing all build external packages from autotest repo (using
`git clean -fxd` -- use with caution)

Change-Id: I65aaedb82f5ebed4b7c23f28f2ccc2dd0e574e8d
Reviewed-on: https://gerrit.chromium.org/gerrit/59619
Tested-by: Aviv Keshet <akeshet@chromium.org>
Reviewed-by: Alex Miller <milleral@chromium.org>
Commit-Queue: Aviv Keshet <akeshet@chromium.org>
roschart/labtest/models.py
roschart/perfchart/models.py
roschart/releasereport/models.py
roschart/testreport/models.py
ko/graphing_utils.py
788e498ae7b6735857c83d88de7f5fffabbc7f8f 18-Jun-2013 Keyar Hood <keyar@chromium.org> Autotest: Add test-health in-memory DB tests.

BUG=chromium:251434
DEPLOY=none
TEST=Ran the tests being added.

Change-Id: I127e780ca7884407a1f07162b026c16b5cddcaa0
Reviewed-on: https://gerrit.chromium.org/gerrit/59096
Reviewed-by: Aviv Keshet <akeshet@chromium.org>
Tested-by: Keyar Hood <keyar@chromium.org>
Commit-Queue: Keyar Hood <keyar@chromium.org>
ealth/complete_failures_functional_test.py
ealth/complete_failures_unittest.py
90ac29a155461e8727c84b7734c26ba45c4741b4 21-Jun-2013 Dan Shi <dshi@chromium.org> [autotest] fix unittest with mocking devserver.url call

BUG=chromium:251879
TEST=run python site_rpc_interface_unittest.py
DEPLOY=none

Change-Id: I97e51b466efd90b49b3f968841385c97edbdf838
Reviewed-on: https://gerrit.chromium.org/gerrit/59577
Reviewed-by: Aviv Keshet <akeshet@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
Tested-by: Dan Shi <dshi@chromium.org>
fe/site_rpc_interface_unittest.py
b8a991101c4f72d9d5f2953ab6375cc15137b3ae 18-Jun-2013 Dan Shi <dshi@chromium.org> [Autotest] Add a global in control.srv generated by rpc interface for suite job

This CL is splitted from CL 58110 to get prod ready to run hwtest for it. In CL
58110, the code passes the selected devserver url from create_suite_job to
dynamic_suite as an argument in dynamic_suite.reimage_and_run method.

BUG=chromium:236540
TEST=tested with run_suite in local workstation
DEPLOY=apache

Change-Id: I3b95686af04e40444a5f482e7fd22a49d771b3f5
Reviewed-on: https://gerrit.chromium.org/gerrit/59091
Tested-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Alex Miller <milleral@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
fe/site_rpc_interface.py
f9a36511e83c4ebeaa6192ca7219895597284681 14-Jun-2013 Keyar Hood <keyar@chromium.org> Autotest: Test health script uses Django models.

Instead of doing raw SQL queries, the complete_failures.py script will
use Django models to check the database.

BUG=chromium:249571
DEPLY=none
TEST=Manually ran script.

Change-Id: I89b02bf4ad0c0951ae0e74b91bebdeff99ca0de4
Reviewed-on: https://gerrit.chromium.org/gerrit/58618
Reviewed-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
Commit-Queue: Keyar Hood <keyar@chromium.org>
Tested-by: Keyar Hood <keyar@chromium.org>
ealth/complete_failures.py
a86253c63422f7b970e1fad1c20ad704a6910741 13-Jun-2013 Dan Shi <dshi@chromium.org> [Autotest] User Preferences failed to be saved when drone set is disabled

This CL fixed the bug when saving User Preferences failed due to drone set is
disabled.

BUG=chromium:242317
TEST=tested in local workstation
DEPLOY=afe

Change-Id: I23facc9c461930397357caec1549e649ee6ffc7f
Reviewed-on: https://gerrit.chromium.org/gerrit/58569
Reviewed-by: Aviv Keshet <akeshet@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
Tested-by: Dan Shi <dshi@chromium.org>
lient/src/autotest/afe/UserPreferencesView.java
0420f636d1f192b5898fdd18780c69ef9032f8ac 05-Jun-2013 Keyar Hood <keyar@chromium.org> Autotest: Add Django readonly database access.

Add a settings_readonly and a setup_django_readonly_environment.py so readonly
access through Django to the database can be done.

BUG=chromium:248363
DEPLOY=none
TEST=Ran the Autotest frontend server and nothing exploded.

Change-Id: I60a8522200352afafd5a194080cd36b48656c188
Reviewed-on: https://gerrit.chromium.org/gerrit/58150
Reviewed-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Keyar Hood <keyar@chromium.org>
Tested-by: Keyar Hood <keyar@chromium.org>
ettings_readonly.py
etup_django_readonly_environment.py
1a3c8dd59f71086eb2f5d08b0f13df95428e60fa 30-May-2013 Keyar Hood <keyar@chromium.org> Autotest: Find long-failing tests.

Currently looks in the (unfiltered) database and finds all tests that
have not passed for the past 60 days.

This is a basic implementation so that further commit can be smaller in size.

BUG=chromium:247047
DEPLOY=none
TEST=There exist a set of unittests that test the core send-email or not logic.

Change-Id: I6893330cc595e308c5875408989dd40a9273f9ce
Reviewed-on: https://gerrit.chromium.org/gerrit/57237
Commit-Queue: Keyar Hood <keyar@chromium.org>
Reviewed-by: Keyar Hood <keyar@chromium.org>
Tested-by: Keyar Hood <keyar@chromium.org>
ealth/__init__.py
ealth/common.py
ealth/complete_failures.py
ealth/complete_failures_unittest.py
e19d303a6718cbd9995e27485554212b63a0a701 30-May-2013 beeps <beeps@chromium.org> [autotest] pylint ignores setters, getters and special modules.

Prior to this change pylint only ignored one special module,
common. This change makes it so we can specify a list of
modules to ignore.

BUG=chromium:239949
TEST=Successfully uploaded this cl
run_pylint on:
1. modules with getters and setters.
2. a change which uses setup_django_lite.
3. module with actual unused modules.
4. repeat within the chroot.

Change-Id: I2cbd2b654cfc78323b9dd840c1822b2a60b02e83
Reviewed-on: https://gerrit.chromium.org/gerrit/51002
Tested-by: Prashanth Balasubramanian <beeps@chromium.org>
Reviewed-by: Aviv Keshet <akeshet@chromium.org>
Commit-Queue: Prashanth Balasubramanian <beeps@chromium.org>
fe/rpc_interface_unittest.py
2b7f5e283fccd35f4e02116fea9f00b01d60cd89 22-May-2013 Dennis Jeffrey <dennisjeffrey@chromium.org> Upload chromeOS Telemetry perf test results for all platforms to perf dash.

Previously, we only uploaded chromeOS Telemetry perf test results for
the "lumpy" platform. Now, we upload results for all platforms.

By default, the perf dashboard considers platforms to be internal-only
unless they are explicitly whitelisted by an authorized user through the
dashboard UI. Internal-only test results can only be viewed by people
who are logged into the dashboard using an @google.com account.

BUG=chromium:237598
TEST=Verified that Telemetry perf results for all platforms can be uploaded
to a local instance of the chrome perf dashboard. Also verified that the
results are only displayed on the dashboard when logged in to an @google.com
account.

Change-Id: Icb1de793540ed13267e78e247c668517f9135405
Reviewed-on: https://gerrit.chromium.org/gerrit/56111
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Commit-Queue: Dennis Jeffrey <dennisjeffrey@chromium.org>
Tested-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
erf-dashboard/generate_perf_graphs.py
3986bac27734a8c3cd864d9ade775131a2261413 21-May-2013 Aviv Keshet <akeshet@chromium.org> [autotest] reduce verbosity of unittests that use django

Prior to this CL, running a unit test that uses a test django setup (for
instance with `utils/unittest_suite.py
autotest_lib.scheduler.monitor_db_unittest --debug`) results in a huge
amount of meaningless repetitions of messages like "No permission for
add_group", coming from repeated setups and teardowns of a test django
database without all of the prod admin groups set up. This CL silences
these messages behind a verbosity flag.

BUG=None
TEST=`utils/unittest_suite.py
autotest_lib.scheduler.monitor_db_unittest --debug`; no more "No
permission X" spam.

Change-Id: I64e0be1cb8e5864a22a0e7dd26aaa0d8a4c842cf
Reviewed-on: https://gerrit.chromium.org/gerrit/56096
Tested-by: Aviv Keshet <akeshet@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Aviv Keshet <akeshet@chromium.org>
fe/management.py
9658a95c995f3fbba9612276e61297a31bc18677 15-May-2013 Alex Miller <milleral@chromium.org> [autotest] Check host locking on batch most modifications.

If a batch host modify RPC would lock a host that is already locked,
fail hard and do not modify any of the hosts.

BUG=chromium:238025
TEST=restarted apache, modify_hosts now throws ValidationError
DEPLOY=apache

Change-Id: Ia12ba1346743c6b9608ca497ba60cb5cd25e829e
Reviewed-on: https://gerrit.chromium.org/gerrit/51227
Tested-by: Alex Miller <milleral@chromium.org>
Reviewed-by: Tan Gao <tgao@chromium.org>
Commit-Queue: Alex Miller <milleral@chromium.org>
fe/rpc_interface.py
a5745e9e7d05af07c82161a8f93cfb39accf4a31 09-May-2013 Scott Zawalski <scottz@google.com> Remove mod-python in favor of mod-wsgi.

mod-python has been deprecated for a while and our current distro of
Django doesn't support it anymore. Moving us to use wsgi completely.

Pull in bits from:
https://github.com/autotest/autotest/commit/ee3cced1afb6bc45216c7038472827ad4033c3f5

Pull in change to db/backends/afe/base.py to fixes readonly db
connection issues.

TEST=setup_dev_autotest.sh and moved our development server to
modpython.
BUG=chromium:217194

Change-Id: Ic3306700e07c7000b2451a576554810d8a6bf7f2
Reviewed-on: https://gerrit.chromium.org/gerrit/50656
Tested-by: Scott Zawalski <scottz@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Scott Zawalski <scottz@chromium.org>
Reviewed-by: Scott Zawalski <scottz@chromium.org>
b/backends/afe/base.py
rontend.wsgi
5ab8b691eb1e9b9ce0a44f886dad9655b9cfa376 07-May-2013 Aviv Keshet <akeshet@chromium.org> [autotest] add a directAFE class which bypasses rpc calls

This CL adds a directAFE object which inherits from frontend.AFE, but
which replaces the underlying RpcClient.run function (responsible for
turning AFE calls into RPC calls) into a function that just makese
direct calls to rpc_interface and site_rpc_interface, bypassing RPC
process. This is desirable in places like test_that, which use an
in-memory process-local sqlite database to back a local afe.

BUG=chromium:236471
TEST=unit tests

Change-Id: Ib89a28be9ebc6f5c4e6ff93a08721ac63ab8b377
Reviewed-on: https://gerrit.chromium.org/gerrit/50383
Commit-Queue: Aviv Keshet <akeshet@chromium.org>
Reviewed-by: Aviv Keshet <akeshet@chromium.org>
Tested-by: Aviv Keshet <akeshet@chromium.org>
fe/direct_afe.py
fe/direct_afe_unittest.py
fa1990038c71dd4abed91a145d7a186dd076a70f 09-May-2013 Aviv Keshet <akeshet@chromium.org> [autotest] Clarify django ImproperlyConfigured exception.

If a developer imports a django models or django models dependent module
before importing one of the django setup modules, they get a slightly
cryptic ImproperlyConfigured exception from Django which doesn't point
them in the direction of our existing django setup modules. This CL
catches the exception and rethrows it as a more descriptive one.

BUG=None
TEST=From interactive python session, import common, then import
direct_afe without first importing one of the the db setup modules. See
clearer exception message.

Change-Id: I397bbd1d3762b9cba0f4f6203c6b73a5d8de9212
Reviewed-on: https://gerrit.chromium.org/gerrit/50901
Commit-Queue: Aviv Keshet <akeshet@chromium.org>
Reviewed-by: Aviv Keshet <akeshet@chromium.org>
Tested-by: Aviv Keshet <akeshet@chromium.org>
fe/models.py
62cf0477206aa81c0163de63b3c86c26ed72eb79 05-May-2013 Aviv Keshet <akeshet@chromium.org> [autotest] add lightweight django setup for use in test_that

BUG=chromium:236471
TEST=new unit test, ensure that jobs can be added to django_lite db

Change-Id: I340934126f46e8158dd80b4727e6d54f856787dd
Reviewed-on: https://gerrit.chromium.org/gerrit/50210
Commit-Queue: Aviv Keshet <akeshet@chromium.org>
Reviewed-by: Aviv Keshet <akeshet@chromium.org>
Tested-by: Aviv Keshet <akeshet@chromium.org>
fe/management.py
jango_lite_unittest.py
ettings.py
ettings_lite.py
etup_django_lite_environment.py
3dd8beb386f7298ffe84d7410d00cce26973e170 14-May-2013 Aviv Keshet <akeshet@chromium.org> [autotest] make a consistent CONTROL_TYPE enum across the codebase

Prior to this CL, there were a multitude of duplicate defitions of the
two control files types (Client or Server), incluiding a difference
between the afe Job model (1 = Server) and the afe Test model (1 =
Client). This CL introduces enums CONTROL_TYPE and CONTROL_TYPE_NAMES to
control_data, to act as the central and consistent defition across the
codebase. In order to avoid needing to mangle the running Jobs table, we
have adopted the existing Job model convention (1 = Server); the Test
table will be re-written with the new consistent convention during test
import.

BUG=chromium:240643
TEST=All existing unit tests pass;
In local autotest without this patch applied, started a suite. Halfway
through suite, applied this patch, ran test importer, restarted apache,
and restarted scheduler. Suite finished successfully.
Verified manually that Client/Server type control files show up
correctly in afe Create Job view.
DEPLOY=scheduler
DEPLOY=apache
DEPLOY=test_importer

Change-Id: Ia5b2573e1d08d96b3826f2837903ef407dcae303
Reviewed-on: https://gerrit.chromium.org/gerrit/51191
Commit-Queue: Aviv Keshet <akeshet@chromium.org>
Reviewed-by: Aviv Keshet <akeshet@chromium.org>
Tested-by: Aviv Keshet <akeshet@chromium.org>
fe/model_attributes.py
fe/models.py
fe/models_test.py
fe/resources.py
fe/resources_test.py
fe/rpc_interface.py
fe/rpc_interface_unittest.py
fe/rpc_utils.py
1f23b6918dd601f469eea2975f3e8dbda6659b58 14-May-2013 Aviv Keshet <akeshet@chromium.org> [autotest] reenable django or simplejson requiring unit tests

For a refactor of control_type, a lot of the relevant unit test coverage
is currently blacklisted in utils/unittest_suite.py due to requiring
Django or simplejson.

This CL un-blacklists those tests. It also therefore
fixes a few tests which are failing or broken. In particular:

tko/rpc_interface_unittest was throwing
`DatabaseError: only a single result allowed for a SELECT that is
part of an expression` all over the place. I couldn't blacklist just
this test file, since it has the same filename as
afe/rpc_interface_unittest.py which we do not want to blacklist and with
the way blacklists work in utils/unittest_suite I couldn't blacklist
just one of them. Instead, I have renamed the broken test file to
rpc_interface_unittest_fixme.py. tko/resources_test imports this failing
module, so I had to rename that one too.

One test in resources_test was throwing the same excepiion as above, so
I commented it out. (test_keyval_filtering)

in monitor_db_unittest, the test
test_HostScheduler_get_host_atomic_group_id throws KeyErrors that seem
to be related to labels not being correctly set up or committed to the
test database in the test setup phase. After trying to blindly track
this down a bit, I realized I was in over my head and just commented out
this specific test).

monitor_db_functional_test was throwing
`DatabaseError: near "TRUNCATE": syntax error` all over the place, so I
blacklisted that test file in utils/unittest_suite.py

BUG=chromium:240643
TEST=utils/unittest_suite.py # All tests pass
Change-Id: I8fdbe048b04516548e96bd888ed74e9fc82a2d88
Reviewed-on: https://gerrit.chromium.org/gerrit/51190
Commit-Queue: Aviv Keshet <akeshet@chromium.org>
Reviewed-by: Aviv Keshet <akeshet@chromium.org>
Tested-by: Aviv Keshet <akeshet@chromium.org>
fe/doctests/001_rpc_test.txt
fe/resources_test.py
ko/resources_test.py
ko/resources_test_fixme.py
ko/rpc_interface_unittest.py
ko/rpc_interface_unittest_fixme.py
9de522853580e1c04c0a8ff1636878df7ba9789b 10-May-2013 Dennis Jeffrey <dennisjeffrey@chromium.org> Upload results for other Telemetry perf tests to chrome's new perf dashboard.

Previously, we were only uploading perf results for Telemetry's octane benchmark
to chrome's new perf dashboard. Now, we also upload results for:

dromaeo_domcoreattr
dromaeo_domcoremodify
dromaeo_domcorequery
dromaeo_domcoretraverse
kraken
robohornetpro
sunspider

BUG=chromium:237598
TEST=Verified that results for the above listed perf tests are successfully
uploaded to a local test instance of chrome's new perf dashboard.

Change-Id: I836a7cf3056e0f7c57e510f5fb2d8ebd02e64053
Reviewed-on: https://gerrit.chromium.org/gerrit/50751
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Commit-Queue: Dennis Jeffrey <dennisjeffrey@chromium.org>
Tested-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
erf-dashboard/croschart_defaults.json
erf-dashboard/generate_perf_graphs.py
05d3ecbc26987ce953ae51aa39b24111e682d819 03-May-2013 Dennis Jeffrey <dennisjeffrey@chromium.org> Perf dashboard: do not filter out perf values that are 0.

A bug exists in the script that extracts perf data from the autotest
results database to display on the perf dashboard. The effect of the
bug is that all perf keyvals that have a value of 0 are omitted from
being displayed on the perf dashboard. This is now fixed.

BUG=chromium:237171
TEST=Verified that perf keyvals previously missing from
platform_GesturesRegressionTest because they had values of 0,
are now properly displayed on a local instance of the perf
dashboard.

Change-Id: Icd6aebacdeaa84b303959a9d94c68f2671349388
Reviewed-on: https://gerrit.chromium.org/gerrit/49957
Reviewed-by: Richard Barnette <jrbarnette@chromium.org>
Commit-Queue: Dennis Jeffrey <dennisjeffrey@chromium.org>
Tested-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
erf-dashboard/extract_perf.py
4f34170dbb70080a56de88dc13859b3a7ffffb0a 25-Mar-2013 Alex Miller <milleral@chromium.org> [autotest] Use |json| instead of |simplejson|.

simplejson was moved into the library as of python 2.4. We're now at
python 2.6/2.7, and having to keep installing simplejson everywhere has
slowly turned into a hassle. Since we can just trivially drop/rename
it, let's do so.

The rpc_interface parts of this change have been vetted, the rest is
done just as a s/simplejson/json/ across the code, which afaik should be
a safe thing to do.

BUG=None
TEST=unit, run_suite
DEPLOY=apache

Change-Id: Iab35f71157e7e5cd3be5291c776b07eb3283be8e
Reviewed-on: https://gerrit.chromium.org/gerrit/46424
Tested-by: Aviv Keshet <akeshet@chromium.org>
Reviewed-by: Alex Miller <milleral@chromium.org>
Commit-Queue: Aviv Keshet <akeshet@chromium.org>
fe/json_rpc/proxy.py
fe/json_rpc/serviceHandler.py
erf-dashboard/extract_perf.py
erf-dashboard/generate_perf_graphs.py
hared/json_html_formatter.py
hared/resource_lib.py
hared/resource_test_utils.py
ko/graphing_utils.py
9316bb2dce97af8885d887100b55d8a76a83e182 01-May-2013 Dennis Jeffrey <dennisjeffrey@chromium.org> Modify test names according to chrome rules when uploading cros results to perf dashboard.

Chrome team follows a few rules for modifying test names before uploading
them to the perf dashboard. ChromeOS team now mirrors these rules so that
Telemetry perf data from chromeOS matches that from other platforms on
the perf dashboard.

BUG=None
TEST=Verified on a local instance of the perf dashboard that the new
naming convention is applied and the graphs still show up properly.

Change-Id: I5164cc58574835a50dbe15f342fe26d8274a0b87
Reviewed-on: https://gerrit.chromium.org/gerrit/49768
Reviewed-by: Richard Barnette <jrbarnette@chromium.org>
Commit-Queue: Dennis Jeffrey <dennisjeffrey@chromium.org>
Tested-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
erf-dashboard/generate_perf_graphs.py
6895db39e3ecfc17d06d5e8699325e42a2957b6a 01-May-2013 Alex Miller <milleral@chromium.org> [autotest] Index status on hqe table.

The query to get all Queued HQEs looks like

SELECT * FROM afe_host_queue_entries INNER JOIN afe_jobs ON
(job_id=afe_jobs.id) WHERE NOT complete AND NOT active AND
status="Queued" ORDER BY afe_jobs.priority DESC, meta_host, job_id;

This query can have some trouble, since the ORDER BY on columns across
tables means that MySQL has to make a temporary table, and then sort the
table. However, one can't avoid this just by index creation, so this is
left as an unsolved "problem" for now.

We currently have all `id` fields and `meta_host` indexed, with
`active`, `complete` and `status` being un-indexed. Messing with
indexes here showed that an index on `status` was most effective, and
dropped the query from 1.53s to 0.07s on a db filled with a backup from
prod.

BUG=chromium:236636
DEPLOY=migrate
TEST=ran migration, ran now-faster query

Change-Id: I1c251b7d1fe95da396ce1e4873447d4294f8385a
Reviewed-on: https://gerrit.chromium.org/gerrit/49712
Commit-Queue: Alex Miller <milleral@chromium.org>
Reviewed-by: Alex Miller <milleral@chromium.org>
Tested-by: Alex Miller <milleral@chromium.org>
igrations/076_index_pending_hqe.py
00ee98b61a2f02aa20dab735e949d4f79d64d023 01-May-2013 Dennis Jeffrey <dennisjeffrey@chromium.org> Change "http" to "https" in perf dashboard upload URL.

When uploading chromeOS perf data to Chrome team's live perf dashboard,
we need to upload with "https" instead of "http". When we use http,
we get an HTML 405: "Method Not Allowed" error.

BUG=None
TEST=Verified that we can upload successfully to the live perf
dashboard with this change.

Change-Id: If2eb6a7c0bc75d02e7a47101318e9674624536e3
Reviewed-on: https://gerrit.chromium.org/gerrit/49672
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Commit-Queue: Dennis Jeffrey <dennisjeffrey@chromium.org>
Tested-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
erf-dashboard/generate_perf_graphs.py
a1a2bdd00142e7e291e1874c6ba8055bc89e90dc 30-Apr-2013 Alex Miller <milleral@chromium.org> [autotest] Index the query for special tasks for the scheduler.

This query takes ~5 seconds to run, scans ~100,000 rows, and is
unindexed. This alone is about 45% of the scheduler tick.

Finding what indexes to create was done by taking the django query that's built
up, pasting it into a python interpreter, and calling
`queued_tasks.query.get_compiler('default').as_sql()` on it to get the SQL
Django generates. One can then run `EXPLAIN SELECT ... \G` to get MySQL to
show how many rows it looks at for each part of the query, and if it's using an
index to do so or not. In this case, we were scanning huge numbers of rows in
both `afe_special_tasks` and `afe_host_queue_entries`.

One can then create indexes for the columns that are being looked in the query.
In our case, trying to find special tasks that haven't been run yet required a
table scan to look at the `is_active` and `is_complete` columns, so those get
an index. The query that looks at `afe_host_queue_entries` looks at `host_id`
and `active` at the same time, so we need to index both of those together too.

On a local prod-like setup, these indexes took the query from 30s down to 0.03s.

BUG=chromium:236636
DEPLOY=migrate
TEST=Ran migration, ran queries.

Change-Id: I86fb21716f7aa2cb7e04a4f461581a64be2e9652
Reviewed-on: https://gerrit.chromium.org/gerrit/49711
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Commit-Queue: Alex Miller <milleral@chromium.org>
Reviewed-by: Alex Miller <milleral@chromium.org>
Tested-by: Alex Miller <milleral@chromium.org>
igrations/075_index_get_prioritized_special_tasks.py
9fece30f6e07dab030f3d1f1d12d3e7485541010 26-Mar-2013 Dennis Jeffrey <dennisjeffrey@chromium.org> Upload chromeOS Octane benchmark perf results to Chrome's new perf dashboard.

This CL contains 3 related changes:

1) modify croschart_defaults.json to use the new test name/perf key names
for the Telemetry Octane benchmark that's running on chromeOS.

2) modify extract_perf.py to also extract the chrome version number
associated with each perf result (which is available in the results
database).

3) modify generate_perf_graphs.py to actually upload the perf data for
a single platform/perf test (lumpy/Telemetry Octane benchmark) to Chrome's
live perf dashboard.

Once we verify things are working well on the live dashboard for this
single platform/test, we'll open it up to more platforms/tests. In a future
CL, we'll cache results that can't be uploaded to the live dashboard
(e.g., if the live dashboard is temporarily down), so that they can be
uploaded on a later invocation of the script.

BUG=chromium:220573
TEST=Verified that extract_perf.py and generate_perf_graphs.py will properly
upload only results for Telemetry Octane benchmark on lumpy, to a local instance
of Chrome's new perf dashboard (sending the perf data to the URL
"http://localhost:8080/add_point"). The CL now uses the actual URL
of the live dashboard (which can be viewed externally).

Change-Id: I98f28d2813ba4ccd52d342db1eebddcf3eba5a85
Reviewed-on: https://gerrit.chromium.org/gerrit/49257
Reviewed-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Commit-Queue: Dennis Jeffrey <dennisjeffrey@chromium.org>
Tested-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
erf-dashboard/croschart_defaults.json
erf-dashboard/extract_perf.py
erf-dashboard/generate_perf_graphs.py
dea67041f8d99fa3a385d7be919e4477cf642a34 23-Apr-2013 Alex Miller <milleral@chromium.org> [autotest] Reland Aborting a job aborts all child jobs.

Django has been upgraded to 1.5, so this is now safe to land.

When a HQE is aborted, we find all HQEs associated with a parent_id of
job that is being aborted, and abort those also.

This was implemented as "requesting a job to abort also requests all
child jobs to abort". All calls to abort a host queue entry end up in
the afe model's abort(). If one directly modifies the database, one can
still abort a singular job without also aborting its children, in case
anyone ever has a need to do so.

BUG=chromium:226826
DEPLOY=apache, scheduler
TEST=Kicked off two suites. Aborted first one immediately, and the
other suite was unaffected. Aborted the second suite after the first
job finished, and the completed job was unaffected. All other tests
(and suite job) were aborted. Confirmed in database that the host queue
entries rows were set correctly, and the appropriate rows in the
|afe_aborted_host_queue_entries| were still being inserted.
Modified a suite after it was made so that it was its own parent.
Confirmed that aborting the suite ran correctly.

Change-Id: I48303dfe5264d24183c66fa7fd2158a6b838cd51
Reviewed-on: https://gerrit.chromium.org/gerrit/48864
Tested-by: Alex Miller <milleral@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Alex Miller <milleral@chromium.org>
fe/models.py
bb903b59f543fedbfe5f0e68fb2f0b1961983392 23-Apr-2013 Alex Miller <milleral@chromium.org> [autotest] Fix issues from Django 1.3 -> 1.5 upgrade.

Some old code was left, so there was still a file using django 1.3 code
instead of django 1.5 code. This does not address the issue that we're
still using the mod_python handler from 1.3 to run django 1.5.

BUG=chromium:232388
DEPLOY=apache
TEST=AFE renders

Change-Id: I47d5195d56f9b0cce22b3620386c94422d80c780
Reviewed-on: https://gerrit.chromium.org/gerrit/48863
Tested-by: Alex Miller <milleral@chromium.org>
Reviewed-by: Prashanth Balasubramanian <beeps@chromium.org>
Commit-Queue: Alex Miller <milleral@chromium.org>
fe/feeds/feed.py
f9e613340d5adbd9fcf7cfe27deb30e761eb03dd 22-Apr-2013 Fang Deng <fdeng@chromium.org> [autotest] Update afe_sqlite backend after updating django to 1.5

In django 1.5, __init__ of DatabaseOperations of sqlite3 requires a
new arg |connection|. DatabaseWrapper of SQLite needs update.

BUG=None
Test=manually create a DatabaseWrapper of SQLite

Change-Id: Icd7c1f0281f4cb4cc297d1545037ef13f261e31b
Reviewed-on: https://gerrit.chromium.org/gerrit/48799
Reviewed-by: Alex Miller <milleral@chromium.org>
Commit-Queue: Fang Deng <fdeng@chromium.org>
Tested-by: Fang Deng <fdeng@chromium.org>
b/backends/afe_sqlite/base.py
47d61281a7b23a5e6d8c5835659b502f56108d07 17-Apr-2013 Alex Miller <milleral@chromium.org> [autotest] Bump django to 1.5.

This includes a mysql-python version bump to get rid of a warning
message, and a few fixes throughout the codebase to get things working
again under django 1.5.

TEST=AFE works, RPCs work, run_suite works, scheduler works
BUG=chromium:232388
DEPLOY=apache,scheduler

Change-Id: Ie0d7f5fd1203a25bdedb78d31778d8373aaf6ff5
Reviewed-on: https://gerrit.chromium.org/gerrit/48402
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Reviewed-by: Brian Harring <ferringb@chromium.org>
Tested-by: Alex Miller <milleral@chromium.org>
Commit-Queue: Alex Miller <milleral@chromium.org>
b/backends/afe/base.py
anage.py
ettings.py
etup_django_environment.py
04d3061503d9335175edada614021454d390ff7c 11-Apr-2013 Fang Deng <fdeng@chromium.org> [autotest] get_hosts returns empty result set when given non-existing labels

Previously rpc calls to get_hosts failed when unknown
labels were given. Failing calls were retried by RetryingAFE
and thus the flow got stalled. To fix it, we make
get_host return an empty result set instead of
raising DoesNotExist error on failure. As a result, the call
won't fail and we leave the rpc client to deal with
the empty result. The fix actually lies in
rpc_utils.get_host_query which is called by get_hosts.

BUG=chromium:222927
TEST=Test with a local setup. Confirmed get_hosts won't fail
and won't be retried.
DEPLOY=apache

Change-Id: I87fd1e245b2f898d863a80aff66f61355791aeea
Reviewed-on: https://gerrit.chromium.org/gerrit/47970
Reviewed-by: Alex Miller <milleral@chromium.org>
Tested-by: Fang Deng <fdeng@chromium.org>
Commit-Queue: Fang Deng <fdeng@chromium.org>
fe/rpc_utils.py
2304143dd6f6324c3057e8895341cc4fe56142f4 17-Apr-2013 Scott Zawalski <scottz@chromium.org> Revert "[autotest] Aborting a job aborts all child jobs."
https://code.google.com/p/chromium/issues/detail?id=232388

bulk_create unknown attribute.

This reverts commit f87c46e524d64bbe8b28058856faa2fb4acc7367

Change-Id: I1a4fcaefaeb2b9e27eea31494b29def1fe8f13d1
Reviewed-on: https://gerrit.chromium.org/gerrit/48363
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Tested-by: Scott Zawalski <scottz@chromium.org>
fe/models.py
f87c46e524d64bbe8b28058856faa2fb4acc7367 09-Apr-2013 Alex Miller <milleral@chromium.org> [autotest] Aborting a job aborts all child jobs.

When a HQE is aborted, we find all HQEs associated with a parent_id of
job that is being aborted, and abort those also.

This was implemented as "requesting a job to abort also requests all
child jobs to abort". All calls to abort a host queue entry end up in
the afe model's abort(). If one directly modifies the database, one can
still abort a singular job without also aborting its children, in case
anyone ever has a need to do so.

BUG=chromium:226826
DEPLOY=scheduler
TEST=Kicked off two suites. Aborted first one immediately, and the
other suite was unaffected. Aborted the second suite after the first
job finished, and the completed job was unaffected. All other tests
(and suite job) were aborted. Confirmed in database that the host queue
entries rows were set correctly, and the appropriate rows in the
|afe_aborted_host_queue_entries| were still being inserted.
Modified a suite after it was made so that it was its own parent.
Confirmed that aborting the suite ran correctly.

Change-Id: I7cea75b3917974210b2d975c9b36870129966dcc
Reviewed-on: https://gerrit.chromium.org/gerrit/47695
Commit-Queue: Alex Miller <milleral@chromium.org>
Reviewed-by: Alex Miller <milleral@chromium.org>
Tested-by: Alex Miller <milleral@chromium.org>
fe/models.py
19b9d18d896fcd654c19802ee3d12720dc91e2b2 02-Apr-2013 Todd Broch <tbroch@chromium.org> perf_dashboard: Add power_LoadTest one hour runs graphs to dashboard.

BUG=chromium:225867
TEST=jsonlint -q -c croschart_defaults.json passed

Change-Id: I52003fdaf910302623ec234c727785a02125fc19
Reviewed-on: https://gerrit.chromium.org/gerrit/47125
Reviewed-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
Tested-by: Todd Broch <tbroch@chromium.org>
Commit-Queue: Todd Broch <tbroch@chromium.org>
erf-dashboard/croschart_defaults.json
5c42c8f9339929eb17425da2a17bf068eb5dfec7 02-Apr-2013 Todd Broch <tbroch@chromium.org> perf_dashboard: add Power_Consumption system_pwr to graphs.

BUG=chromium:225867
TEST=jsonlint -c -q croschart_defaults.json passes

Change-Id: I0b0f18ab277ca0f15a215e202cc51fa51874e220
Reviewed-on: https://gerrit.chromium.org/gerrit/47132
Reviewed-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
Tested-by: Todd Broch <tbroch@chromium.org>
Commit-Queue: Todd Broch <tbroch@chromium.org>
erf-dashboard/croschart_defaults.json
2222f836aa426a5dba14945e801d3bde25d7bce6 27-Mar-2013 Chung-yih Wang <cywang@chromium.org> perf-dashboard: add master chromeos.touchpad

With this patch, we will make platform_GesturesRegressionTest listed in
the master chromeos.touchpad of chromium dashboard.

BUG=chromium:224047
TEST=should not break current cautotest dashboard

Change-Id: I791bcec75a4dcca2b94b907b1617d249318ac346
Reviewed-on: https://gerrit.chromium.org/gerrit/46606
Reviewed-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
Commit-Queue: Chung-yih Wang <cywang@chromium.org>
Tested-by: Chung-yih Wang <cywang@chromium.org>
erf-dashboard/croschart_defaults.json
de14d54df901fa5411c49dc4ef7e431027fa1c3d 23-Mar-2013 Dennis Jeffrey <dennisjeffrey@chromium.org> Add Telemetry Octane Benchmark overall score graph to chromeOS perf graphs.

This change ensures that on the existing chromeOS perf graphs ("EagleEye"),
we start plotting the overall score results for telemetry_OctaneBenchmark.

BUG=None
TEST=Verified that I can properly see the new Octane Benchmark overall
score graphs using a local EagleEye setup, with results taken from the
past 7 days.

Change-Id: I3e348c7e2d12fcc6c51cafd09ee833ad906879c8
Reviewed-on: https://gerrit.chromium.org/gerrit/46339
Reviewed-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
Tested-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
Commit-Queue: Dennis Jeffrey <dennisjeffrey@chromium.org>
erf-dashboard/croschart_defaults.json
b71f46fc60854df8f31dac7effa8d34c54bc7e1e 15-Mar-2013 Chung-yih Wang <cywang@chromium.org> perf_dashboard: add platform_GesturesRegressionTest graphs

Use wildcard to list all different keys from all platforms for
platform_GesturesRegression graphs.

BUG=chromium-os:37644
TEST=visit cautotest perf dashboard and see if
platform_GesturesRegressionTest is included once the new configuration
is synced to the cautotest server.
CQ-DEPEND=Id1c233f8db81f942f2fd91c7a2dc38bebedbfdb3

Change-Id: I1657b149fa032ed5b6b697cb54d8a113db6dd67f
Reviewed-on: https://gerrit.chromium.org/gerrit/45546
Commit-Queue: Chung-yih Wang <cywang@chromium.org>
Reviewed-by: Chung-yih Wang <cywang@chromium.org>
Tested-by: Chung-yih Wang <cywang@chromium.org>
erf-dashboard/croschart_defaults.json
4df3925d82e8155afd465ed0ec408e0796e9602e 19-Mar-2013 Dan Shi <dshi@chromium.org> [Autotest] Add stat to create_job and create_suite_job RPCs

Count rpc calls, including create_job and create_suite_job using
site_utils/graphite/stats.

BUG=chromium-os:196374
TEST=manually run in local machine

Change-Id: I632fb73a64f6360066ce6594761886d1f44f0607
Reviewed-on: https://gerrit.chromium.org/gerrit/45876
Reviewed-by: Alex Miller <milleral@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
Tested-by: Dan Shi <dshi@chromium.org>
fe/rpc_interface.py
fe/site_rpc_interface.py
900b0697f49f1c77978d48746f28b01c3c26e484 11-Mar-2013 Dennis Jeffrey <dennisjeffrey@chromium.org> Initial skeleton for uploading perf results to Chrome's perf dashboard.

We're beginning work on uploading chromeOS perf results to a new dashboard
that is being developed by the Chrome team. Our plan is to first get results
showing up in both EagleEye (the current chromeOS perf dashboard) and also
Chrome's perf dashboard. Eventually we will deprecate EagleEye in favor of
Chrome's perf dashboard.

This CL adds some skeleton code to the EagleEye perf graph generation scripts,
which is a no-op for now, but which will later be used to upload data to the
new dashboard. I plan to check in these changes across a series of smaller
CLs, rather than one large CL.

BUG=chromium-os:39456,chromium-os:38432
TEST=Verified that after these changes, extract_perf.py and
generate_perf_graphs.py still work as before, using perf data in the autotest
database from the past 7 days.

Change-Id: I0962ed64c55a2ba0dc4d616bf2f95a76b8e6b3da
Reviewed-on: https://gerrit.chromium.org/gerrit/45362
Commit-Queue: Dennis Jeffrey <dennisjeffrey@chromium.org>
Reviewed-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
Tested-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
erf-dashboard/croschart_defaults.json
erf-dashboard/generate_perf_graphs.py
eadbf7098c7cf810aaaf88791b1d59b786407e9f 14-Mar-2013 Scott Zawalski <scottz@google.com> Modify our support of --image to work with our devservers.

Add a pathway to the --image code in site_host to stage anything that
is passed to it via --image that does not start with http://.

Added code to properly update job_repo_url in site_host rather
than in the try-jobs we spawn off. This makes it so that repaired
images as well as installs via --install have the proper job_repo_url
if they exist in the AFE.

Updated AfeClient.html to show examples of what can be specified in
Image Url.

Update old way of setting REPO URL in site_autotest.py to only be
used if someone is specifying a HTTP URL themselves, otherwise use the
AFE.

TEST=ran autoserv:
./autoserv --image=stumpy-release/R27-3836.0.0 -m scottz-stumpy1 -s
tests/sleeptest/control
and
./autoserv
--image=http://172.22.50.205:8082/update/stumpy-paladin/R27-3837.0.0-rc2
-m scottz-stumpy1 -s tests/sleeptest/control
and with a bad image which fails after a short while with an error about
the build path.
./autoserv --image=x86-mario-release/R27-3834.dd0.0 -m scottz-mario1 -s
tests/sleeptest/control
Test on a non-afe host:
./autoserv --image=x86-mario-release/R27-3834.0.0 -m 172.29.151.133 -s
tests/sleeptest/control
Tested repair update as well just to make sure I didn't break anything.
./autoserv -R -m scottz-mario1 # with a modification to make verify
fail always.
Ran tests through the frontend.
BUG=chromium-os:38653

Change-Id: I4b620a4332c695f4c972a8f8c112b4bf1dd6ba9a
Reviewed-on: https://gerrit.chromium.org/gerrit/45419
Tested-by: Scott Zawalski <scottz@chromium.org>
Reviewed-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Commit-Queue: Scott Zawalski <scottz@chromium.org>
lient/src/autotest/public/AfeClient.html
f61bcd5875651547da04fe5c70ea9e8230578842 14-Mar-2013 Dennis Jeffrey <dennisjeffrey@chromium.org> Support wildcards in list of perf keys to plot on chromeOS perf graphs.

When generating chromeOS perf graphs, a JSON file specifies which perf keys
from which tests to plot on which graphs. That JSON file now supports the
use of wildcards, so you don't need to specify exact perf keys in the JSON
file for them to show up in perf graphs:

1) *metric: Matches perf keys that end with "metric".
2) metric*: Matches perf keys that start with "metric".
3) *metric*: Matches perf keys that contain "metric".
4) metric: Matches only the perf key "metric" (exact match).
5) *: Matches any perf key.

BUG=chromium:188811,chromium:196265
TEST=Verified that when generating perf graphs from the perf results in
the past 7 days (which includes certain perf keys that are now longer than
they used to be), the newly-longer perf key results are now properly
showing up in the graphs with the use of wildcards, whereas before this change,
they were not showing up properly.

Change-Id: Id1c233f8db81f942f2fd91c7a2dc38bebedbfdb3
Reviewed-on: https://gerrit.chromium.org/gerrit/45507
Reviewed-by: Richard Barnette <jrbarnette@chromium.org>
Commit-Queue: Dennis Jeffrey <dennisjeffrey@chromium.org>
Tested-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
erf-dashboard/croschart_defaults.json
erf-dashboard/generate_perf_graphs.py
ce46fa588d89bdfa44d630991201c3d821c688be 14-Mar-2013 Aviv Keshet <akeshet@chromium.org> [autotest] allow 0 test retries in create_job view

This CL fixes a parsing bug which was preventing jobs from being created
in the autotest afe create_job when the value 0 was specified for
test_retries.

Also, Java is verbose.

BUG=chromium:196247
TEST=compile frontend with utils/compile_gwt_clients.py. Then restart
apache. Then verify that able to create jobs with test_retry field set
to zero, and that leading or trailing spaces are ignored.

Change-Id: I46e4fc1d0e6cef8581a2941a6f9a343e4335aa6d
Reviewed-on: https://gerrit.chromium.org/gerrit/45439
Reviewed-by: Aviv Keshet <akeshet@chromium.org>
Tested-by: Aviv Keshet <akeshet@chromium.org>
Commit-Queue: Aviv Keshet <akeshet@chromium.org>
lient/src/autotest/afe/create/CreateJobViewPresenter.java
08a52c77f812036bc1ec58f172a4048c15d64322 05-Mar-2013 Aviv Keshet <akeshet@chromium.org> [autotest] add retries field to create job view

A job created in the afe's create job tab can now have its test_retry
value specified. If the field in the afe is left blank, the number of
retries used is taken from the RETRIES value that was parsed from the
control file at test-import time (or the maximum of this value ot
over all the selected jobs, if multiple jobs are selected).

BUG=chromium-os:37158
TEST=Ran './utils/compile_gwt_clients.py -c autotest.AfeClient' to
recompile frontend. Then, 'sudo /etc/init.d/apache2 restart'. Verified
that the above behavior holds, and that the clone job button populates the
retry field with the value from the job to be cloned.

Change-Id: I6f28a0d9e1eb8a39fae0d1669bef846c65c99198
Reviewed-on: https://gerrit.chromium.org/gerrit/45164
Tested-by: Aviv Keshet <akeshet@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Aviv Keshet <akeshet@chromium.org>
lient/src/autotest/afe/create/CreateJobViewDisplay.java
lient/src/autotest/afe/create/CreateJobViewPresenter.java
lient/src/autotest/public/AfeClient.html
9af96d3b0c1d3fc7e137a3535835391b9019b810 05-Mar-2013 Aviv Keshet <akeshet@chromium.org> [autotest] Add test_retry field to afe_autotests table

BUG=chromium-os:37158
TEST=database/migrate.py sync. Verified that new column is created, and
defaults to value 0 for all existing entries.

Change-Id: Iac5af51aa66edc00c5a00df035626e6470ae7d0c
Reviewed-on: https://gerrit.chromium.org/gerrit/44701
Reviewed-by: Aviv Keshet <akeshet@chromium.org>
Tested-by: Aviv Keshet <akeshet@chromium.org>
Commit-Queue: Scott Zawalski <scottz@chromium.org>
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Tested-by: Scott Zawalski <scottz@chromium.org>
fe/models.py
igrations/074_test_add_test_retry.py
0a419fe9a703c2ff31ea8fc1393f668e3efcde4c 02-Mar-2013 Aviv Keshet <akeshet@chromium.org> [autotest] Add test_retry view to autotest frontend job detail

BUG=chromium-os:37158
TEST=Ran './utils/compile_gwt_clients.py -c autotest.AfeClient' to
recompile frontend. Then, 'sudo /etc/init.d/apache2 restart'. Then set
the retry field manually for some job, and saw that this change was
visible as the value in job detail.

Change-Id: Ie5589356b5ab846ca75cefd11dbcf41bc718cef2
Reviewed-on: https://gerrit.chromium.org/gerrit/44428
Reviewed-by: Aviv Keshet <akeshet@chromium.org>
Tested-by: Aviv Keshet <akeshet@chromium.org>
Commit-Queue: Scott Zawalski <scottz@chromium.org>
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Tested-by: Scott Zawalski <scottz@chromium.org>
lient/src/autotest/afe/JobDetailView.java
lient/src/autotest/public/AfeClient.html
cd1ff9b72bd73c7308145750f07908b26c5d08da 01-Mar-2013 Aviv Keshet <akeshet@chromium.org> [autotest] Add test_retry field to afe_jobs table

Does what it says on the tin.

CQ-DEPEND=If8134fe263bb33ee5d52bc92e78faee05388b239

BUG=chromium-os:37158
TEST=database/migrate.py sync, ensure that jobs can still be viewed in
afe. Ensure that new column exists and is zeroed out for all jobs
already in db. Kick off a smoke suite and ensure that it runs.

Change-Id: I0790d60d7c86d9278507b46ae1579a7b72b20c5a
Reviewed-on: https://gerrit.chromium.org/gerrit/44427
Tested-by: Aviv Keshet <akeshet@chromium.org>
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Commit-Queue: Scott Zawalski <scottz@chromium.org>
Tested-by: Scott Zawalski <scottz@chromium.org>
fe/models.py
fe/rpc_interface.py
fe/rpc_interface_unittest.py
fe/rpc_utils.py
igrations/073_job_add_test_retry.py
8bd3e983eb1af3dcef85eb31a719bfaa197f05f5 05-Mar-2013 Dennis Jeffrey <dennisjeffrey@chromium.org> Increase max length of TKO model for tko_iteration_result.attribute to 256.

In https://gerrit.chromium.org/gerrit/#/c/44647/ , the max length of
a perf key in the autotest database is increased to 256 characters.
The current CL makes the corresponding change to the max length in the
corresponding TKO model.

Also added docstrings and removed one import that appears to be unused,
to satisfy pylint.

BUG=chromium-os:39494
TEST=None

Change-Id: I54d1089f225c0a92f0477fe4cc8508d008aa7977
Reviewed-on: https://gerrit.chromium.org/gerrit/44668
Reviewed-by: Alex Miller <milleral@chromium.org>
Commit-Queue: Dennis Jeffrey <dennisjeffrey@chromium.org>
Tested-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
ko/models.py
d65bbd14975db1ee5c25a6fac18216794d7070bd 05-Mar-2013 Dennis Jeffrey <dennisjeffrey@chromium.org> Increase character limit in autotest DB for perf keys from 30 to 256 characters.

This change is accomplished by adding a new migration file to the set of
autotest DB migrations (with this change, we're now at version 72).

This change is required for new Telemetry perf tests -- written by the chrome
team -- that we'll soon run on chromeOS. Some of these tests have perf keys
that are considerably longer than 30 characters, but are less than 256
characters.

Also removed an unused command-line option ("-a") from migrate.py,
the script that is used to perform the database migrations.

Also added a bunch of docstrings and removed some unused imports, to
satisfy pylint.

BUG=chromium-os:39494
TEST="python migrate.py sync", running on my local machine. Verified that
I can migrate up and down from version 71 to 72, and when doing so, the
column "tko_iteration_result.attribute" changes from varchar(30) to
varchar(256) as expected. Similarly, this change is reflected in the
views "tko_perf_view.iteration_key" and "tko_perf_view_2.iteration_key".

Change-Id: I4fa7524f6d732a5ff8f4950445d64d3f20a8c630
Reviewed-on: https://gerrit.chromium.org/gerrit/44647
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Commit-Queue: Dennis Jeffrey <dennisjeffrey@chromium.org>
Tested-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
igrations/072_expand_perf_key_fields.py
73dae55766b9c8b6922194901e334c4e52f67d14 25-Feb-2013 Simran Basi <sbasi@chromium.org> Autotest AFE: Replace Reinstall button with Repair.

It is sometimes useful to kick off a repair job from the AFE, added
a new repair button to the host info page.

BUG=chromium:39213
TEST=compiled locally and verified that when I click repair a repair
job is kicked off. Also added a new unittest and ensured the old
ones still worked.

Change-Id: I71533987eb353e52b2b7fd32725adf00df017bc9
Reviewed-on: https://gerrit.chromium.org/gerrit/43975
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Simran Basi <sbasi@chromium.org>
Tested-by: Simran Basi <sbasi@chromium.org>
fe/rpc_interface.py
fe/rpc_interface_unittest.py
lient/src/autotest/afe/AfeUtils.java
lient/src/autotest/afe/HostDetailView.java
lient/src/autotest/public/AfeClient.html
8a30538855bb98a4d80f795a9463b623a815cd17 28-Feb-2013 Dennis Jeffrey <dennisjeffrey@chromium.org> Make EagleEye input/output directories customizable via command-line options.

Two EagleEye (chromeOS perf dashboard) scripts previously used hard-coded
locations for their input/output directories. This CL makes those locations
customizable via command-line options to the scripts.

BUG=chromium-os:38150
TEST=Verified the scripts can read input files and write output files to
custom directories specified as command-line options to the scripts.

Change-Id: Icc535a61d2967e96fc2bf2af6f48be2cb44398d5
Reviewed-on: https://gerrit.chromium.org/gerrit/44324
Commit-Queue: Dennis Jeffrey <dennisjeffrey@chromium.org>
Reviewed-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
Tested-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
erf-dashboard/extract_perf.py
erf-dashboard/generate_perf_graphs.py
a713e2575a9ea59db59444deb29a951a395aec7e 01-Mar-2013 Alex Miller <milleral@chromium.org> [autotest] Do not allow unicode control files.

Our system does not properly handle non-ascii characters. Therefore,
rather than having the scheduler explode somewhere down the line, let's
just explicitly disallow control files that contain unicode characters.

BUG=chromium-os:39337
TEST=local run_suite

Change-Id: Id2dcee75087803f1bd4236872f75e71039ae034b
Reviewed-on: https://gerrit.chromium.org/gerrit/44401
Tested-by: Alex Miller <milleral@chromium.org>
Reviewed-by: Aviv Keshet <akeshet@chromium.org>
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Commit-Queue: Alex Miller <milleral@chromium.org>
fe/rpc_interface.py
fe/site_rpc_interface.py
89c44dd685440cfe70ffeb4b0ef4357a51da2521 26-Feb-2013 Scott Zawalski <scottz@google.com> Fix auto repair and the reinstall button from the AFE.

Set up an autorepair feature by way of the autoupdate test that will
automatically use a CROS.stable_repair_version from the global_config
to stage and install a particular version of Chrome OS.

This should recover ~10% of our repair failed devices a day.

Add code to automatically start update_engine IFF we are in repairing
mode.

BUG=chromium-os:39212,chromium-os:38151,chromium-os:34446,chromium-os:39213
TEST=local runs of control.repair and complete end to end runs from the
frontend. Tested reinstall with a device missing python and libc
problems.
Filled stateful on a DUT and ran repair end to end
server/autoserv -m scottz-mario1 -R

Change-Id: I18f428b4022cdb29040c3a0366c7e7b069229225
Reviewed-on: https://gerrit.chromium.org/gerrit/44053
Tested-by: Scott Zawalski <scottz@chromium.org>
Reviewed-by: Richard Barnette <jrbarnette@chromium.org>
Commit-Queue: Scott Zawalski <scottz@chromium.org>
Reviewed-by: Scott Zawalski <scottz@chromium.org>
lient/src/autotest/afe/AfeUtils.java
bbe98a25afa18dfc368d4f59269932d6a27d8fc1 13-Feb-2013 Dennis Jeffrey <dennisjeffrey@chromium.org> EagleEye perf graphs now support autotests whose names have changed.

The JSON file specifying which performance graphs to plot has now been
restructured: the file is now a list of objects representing each test
to plot. A test has a name, an optional list of "old_test_names" (in
case the name of the test has changed over time), and a list of graphs
to display that are associated with the test.

The scripts to extract perf values and generate perf graphs have been
adjusted to account for the changed JSON file format. In the event that
a test is associated with one or more "old_test_names", then any perf values
from an old test name are processed and treated as if they are simply
associated with the new test name. So a test that used to be called "X",
but is now called "Y", will show up in the graphs as test "Y", but will
implicitly include all results from both "X" and "Y" taken together.

BUG=chromium-os:38150
TEST=Verified locally that autotest desktopui_PyAutoPerf (which used to
be called desktopui_PyAutoPerfTests in the autotest database) shows up
properly in the perf graphs and contains the perf results from both
the old and new autotest names.

Change-Id: I5b41e176b5fa72f0156984b067cd14dc56f3d73b
Reviewed-on: https://gerrit.chromium.org/gerrit/43238
Reviewed-by: Richard Barnette <jrbarnette@chromium.org>
Commit-Queue: Dennis Jeffrey <dennisjeffrey@chromium.org>
Tested-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
erf-dashboard/croschart_defaults.json
erf-dashboard/extract_perf.py
erf-dashboard/generate_perf_graphs.py
18308922b8086ecc9aed084f02ddf831ae252bf7 20-Feb-2013 Aviv Keshet <akeshet@chromium.org> [autotest] Create dynamic_suite child jobs with parent_job_id

Child jobs of a dynamic_suite call are now created with appropriate
parent_job_id pointing back at the suite job.

BUG=chromium-os:38253
TEST=Kick of a suite with run_suite.py. Child jobs have correct parent
job id, as seen in job detail view.

Change-Id: Ie45aac1b741b1282ddf792812480bc1f4d35278a
Reviewed-on: https://gerrit.chromium.org/gerrit/43583
Reviewed-by: Alex Miller <milleral@chromium.org>
Tested-by: Aviv Keshet <akeshet@chromium.org>
Reviewed-by: Aviv Keshet <akeshet@chromium.org>
Commit-Queue: Aviv Keshet <akeshet@chromium.org>
fe/models.py
fe/rpc_utils.py
ec19f934f095d47d8800e478f5e0f270d409af8d 07-Feb-2013 Aviv Keshet <akeshet@chromium.org> [autotest] Add parent_job_id view to autotest frontend job detail

Adds a item to job detail views in autotest displaying the parent_job_id
for the job under study. Shows <null> as the parent id for jobs with no
parent.

BUG=None
TEST=Ran './utils/compile_gwt_clients.py -c autotest.AfeClient' to
recompile frontend. Then manually updated one job's parent id to an
integer value. For that job, the parent id is correctly displayed in job
detail view (under http://localhost/). For other parentless jobs, job id
displays as <null>

Change-Id: I51dbb02517e0b6d63c249e4afa23166aa791d471
Reviewed-on: https://gerrit.chromium.org/gerrit/42802
Commit-Queue: Aviv Keshet <akeshet@chromium.org>
Reviewed-by: Aviv Keshet <akeshet@chromium.org>
Tested-by: Aviv Keshet <akeshet@chromium.org>
lient/src/autotest/afe/JobDetailView.java
lient/src/autotest/public/AfeClient.html
0b9cfc93f17ec81535de54e3bae4f5f0e8d8595b 05-Feb-2013 Aviv Keshet <akeshet@chromium.org> [autotest] Add parent_job field to database and models

Adds a parent_job field to dbmodels, and a parent_job_id column to the
afe jobs table, allowing jobs to have some notion of referring to parent
jobs that may have kicked them off (for instance, in a dynamic suite --
to be implemented in future commit).

BUG=chromium-os:38253
TEST=Ran database/migrate.py sync . Ensuree that previous jobs can still
be viewed in the afe, and than a new parent_job_id column has been
created in afe jobs table (via mysql client). Ran a smoke suite
successfully.
CQ-DEPEND=CL:Ie45aac1b741b1282ddf792812480bc1f4d35278a

Change-Id: Iadfedc2a9e8f603ab223c87ed7c1510a46622216
Reviewed-on: https://gerrit.chromium.org/gerrit/42673
Commit-Queue: Aviv Keshet <akeshet@chromium.org>
Reviewed-by: Aviv Keshet <akeshet@chromium.org>
Tested-by: Aviv Keshet <akeshet@chromium.org>
fe/models.py
fe/rpc_interface.py
igrations/071_job_add_parent_job.py
7cb0f0f36242d7ac60afb4886bb19c636202180f 14-Feb-2013 Todd Broch <tbroch@chromium.org> Add power_BatteryCharge.full results to croschart.

BUG=none
TEST=none

Change-Id: I61253947573d5c7062d2870d0654fea48f62dc35
Reviewed-on: https://gerrit.chromium.org/gerrit/43325
Reviewed-by: Sameer Nanda <snanda@chromium.org>
Reviewed-by: Mike Truty <truty@chromium.org>
Commit-Queue: Todd Broch <tbroch@chromium.org>
Tested-by: Todd Broch <tbroch@chromium.org>
roschart/croschart_defaults.json
18c70b3d929b759ea9ba4e876517c6b8fd4d6921 15-Feb-2013 Chris Sosa <sosa@chromium.org> Some small fixes found while doing an e2e test with devserver changes.

1) Alex's change to not exit 0 totally now clobbers you shadow config
even if you say no.
2) Release config should be updated for R27.
3) Site RPC interface -- better logging.
4) Site_Suite didn't actually allow an int override -- kept on casting it
to a string.

BUG=None
TEST=Ran through devserver test using cli/atest suite create and setup
autotest with shadow config.

Change-Id: Ie7f61fe95c3a642455de33416367085e159c3947
Reviewed-on: https://gerrit.chromium.org/gerrit/43429
Commit-Queue: Chris Sosa <sosa@chromium.org>
Reviewed-by: Chris Sosa <sosa@chromium.org>
Tested-by: Chris Sosa <sosa@chromium.org>
fe/site_rpc_interface.py
7db38bab49bb85027c216bdf89e289d0f24dba16 13-Feb-2013 Dennis Jeffrey <dennisjeffrey@chromium.org> Fix presubmit warnings in autotest's frontend/afe/models.py.

BUG=chromium-os:38655
TEST=None

Change-Id: I7adfd21e3fc1265fc74bdfa141b515160d31d16e
Reviewed-on: https://gerrit.chromium.org/gerrit/43192
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Commit-Queue: Dennis Jeffrey <dennisjeffrey@chromium.org>
Tested-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
fe/models.py
b95ba5ad2d48c14cf7754b2872861125a4f73fa0 13-Nov-2012 Dennis Jeffrey <dennisjeffrey@chromium.org> Perf extraction/parsing scripts for new ChromeOS performance dashboard.

This CL contains 2 python scripts that: (1) extract perf keys from the
autotest DB; and (2) creates perf data files from the extracted data
that can be read in to display chrome-style perf graphs.

BUG=chromium-os:30753
TEST=Verified the scripts work on my local machine and on cautotest, and
that chrome-style graphs can be displayed from the created data files.

Change-Id: I7ef85c7aba62d00f9c226a1ce33904bf6788dc78
Reviewed-on: https://gerrit.chromium.org/gerrit/37871
Commit-Queue: Dennis Jeffrey <dennisjeffrey@chromium.org>
Reviewed-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
Tested-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
erf-dashboard/common.py
erf-dashboard/croschart_defaults.json
erf-dashboard/doc/default.html
erf-dashboard/extract_perf.py
erf-dashboard/generate_perf_graphs.py
erf-dashboard/templates/branch_platform_overview.html
erf-dashboard/templates/compare_by_overview.html
erf-dashboard/templates/config.js
erf-dashboard/templates/overview_base.html
ettings.py
d83ef449fc80c663ce90a3a07aa15d6c25ee08a1 17-Jan-2013 Aviv Keshet <akeshet@chromium.org> [autotest] Only integer or None as num argument in create_suite_job

Cleanup to reduce confusion and type casting back and forth between int
and string.

BUG=chromium-os:37936
TEST=unit test that non-integer or None num arguments throw appropriate
exception; unit test that integer arguments get passed along correctly;
ran a suite locally to ensure rpc call still working

Change-Id: Id8b8e0dd5a08db2ebec67cdba13b2b1d8eb0b149
Reviewed-on: https://gerrit.chromium.org/gerrit/41791
Reviewed-by: Aviv Keshet <akeshet@chromium.org>
Tested-by: Aviv Keshet <akeshet@chromium.org>
Commit-Queue: David James <davidjames@chromium.org>
fe/site_rpc_interface.py
fe/site_rpc_interface_unittest.py
9e30bb204a993933f5a8875498abb84b7e227e0a 01-Jan-2013 Alex Miller <milleral@chromium.org> [autotest] Change os.path.join use in common.py to be portable.

All uses of |os.path.join("x", "../..")| have been changed into
|os.path.join("x", "..", "..")|.

TEST=syntax check all common.py files, trybot passes HWTest
BUG=None

Change-Id: Ic05a2e375352eab4447ae5b180da750ccbc1b914
Reviewed-on: https://gerrit.chromium.org/gerrit/40312
Tested-by: Alex Miller <milleral@chromium.org>
Commit-Queue: Scott Zawalski <scottz@chromium.org>
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Commit-Queue: Alex Miller <milleral@chromium.org>
fe/json_rpc/common.py
de622cb37f581dbc62eb3c22165f3a2f65de9ae8 04-Dec-2012 Todd Broch <tbroch@chromium.org> Add power_LoadTest one hour results to croschart.

BUG=none
TEST=none

Change-Id: Iddcc22b6877ae0c98f758b99feb42c77b4da3a95
Reviewed-on: https://gerrit.chromium.org/gerrit/39211
Reviewed-by: Mike Truty <truty@chromium.org>
Commit-Ready: Todd Broch <tbroch@chromium.org>
Tested-by: Todd Broch <tbroch@chromium.org>
roschart/croschart_defaults.json
55e4351c8f8da46bd05ecf678d26d3249a0f9d94 19-Nov-2012 Dennis Jeffrey <dennisjeffrey@chromium.org> Modify "perf" links on autotest result dashboard BVT view to point to revamped perf graphs.

On the autotest results dashboard, default BVT view, the "perf" links are
now modified to point to the revamped perf graphs that are generated using
chrome-style perf graphs.

The "coming soon" link at the top of the BVT view is also removed, since the
change is no longer "coming soon", as it is now here.

BUG=chromium-os:30753
TEST=Verified changes on the new BVT view using a local autotest UI front-end
test environment that I set up on my local machine.

Change-Id: I9294731e21aefc77d413a9fc79a714b410d553ce
Reviewed-on: https://gerrit.chromium.org/gerrit/38326
Reviewed-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Chris Sosa <sosa@chromium.org>
Commit-Ready: Dennis Jeffrey <dennisjeffrey@chromium.org>
Tested-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
emplates/tables/summary/header.html
emplates/tables/summary/waterfall_index.html
34217022229b755bc1ee52f83665acba76bd5044 06-Nov-2012 Simran Basi <sbasi@chromium.org> Autotest: Increase run timeout granularity

This CL modifies all references to max_runtime_hrs to instead be
max_runtime_mins. This includes the django models, rpc interfaces, the
cleanup timeout code, and the frontend java views.

The frontend java code will need to be recompiled once this commits to
prevent the frontend from breaking.

The cleanup timeout pathway has been adjusted to find all timedout jobs
by minute, and has been changed to run every 5 minutes vs every hour as
before.

BUG=chromium-os:36067
TEST=Ran on my local afe, ensure that jobs can still be created correctly,
and jobs with short timeouts do indeed get aborted when expected.

Change-Id: Idfdeb3f1d4947d6b2e6b48127a31db535704e972
Reviewed-on: https://gerrit.chromium.org/gerrit/37827
Tested-by: Simran Basi <sbasi@chromium.org>
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Ready: Simran Basi <sbasi@chromium.org>
fe/doctests/001_rpc_test.txt
fe/models.py
fe/resources.py
fe/rpc_interface.py
fe/rpc_utils.py
lient/src/autotest/afe/AfeUtils.java
lient/src/autotest/afe/JobDetailView.java
lient/src/autotest/afe/create/CreateJobViewPresenter.java
lient/src/autotest/public/AfeClient.html
a8a0d75c2591ab872f695d7b8f22f2ca10493dab 08-Nov-2012 Scott Zawalski <scottz@chromium.org> Revert "Autotest: Increase run timeout granularity"

This causes autoupdate problems, see bug crosbug.com/36183

This reverts commit daffa57a6738c2a80caee1f311a527313ece1207

Change-Id: Iad65be9bf1b0ea329866bfdc764b0089c4f9b2a5
Reviewed-on: https://gerrit.chromium.org/gerrit/37657
Commit-Ready: Scott Zawalski <scottz@chromium.org>
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Tested-by: Scott Zawalski <scottz@chromium.org>
fe/doctests/001_rpc_test.txt
fe/models.py
fe/resources.py
fe/rpc_interface.py
fe/rpc_utils.py
lient/src/autotest/afe/AfeUtils.java
lient/src/autotest/afe/JobDetailView.java
lient/src/autotest/afe/create/CreateJobViewPresenter.java
lient/src/autotest/public/AfeClient.html
daffa57a6738c2a80caee1f311a527313ece1207 06-Nov-2012 Simran Basi <sbasi@chromium.org> Autotest: Increase run timeout granularity

This CL modifies all references to max_runtime_hrs to instead be
max_runtime_mins. This includes the django models, rpc interfaces, the
cleanup timeout code, and the frontend java views.

The frontend java code will need to be recompiled once this commits to
prevent the frontend from breaking.

The cleanup timeout pathway has been adjusted to find all timedout jobs
by minute, and has been changed to run every 5 minutes vs every hour as
before.

BUG=chromium-os:36067
TEST=Ran on my local afe, ensure that jobs can still be created correctly,
and jobs with short timeouts do indeed get aborted when expected.

Change-Id: Id7668bbd05a9b02c22e7c549fac232fae02fc728
Reviewed-on: https://gerrit.chromium.org/gerrit/37479
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Commit-Ready: Simran Basi <sbasi@chromium.org>
Tested-by: Simran Basi <sbasi@chromium.org>
fe/doctests/001_rpc_test.txt
fe/models.py
fe/resources.py
fe/rpc_interface.py
fe/rpc_utils.py
lient/src/autotest/afe/AfeUtils.java
lient/src/autotest/afe/JobDetailView.java
lient/src/autotest/afe/create/CreateJobViewPresenter.java
lient/src/autotest/public/AfeClient.html
e8b48fee8c5a2d0f129fac2bcbfe22dbf5d82868 06-Nov-2012 Simran Basi <sbasi@chromium.org> Autotest: Add max_runtime_mins Column to AFE_JOBS

As part of increasing the granularity of our job timeouts, we need
to add a new column to the afe_jobs table.

This migration will create a new column, max_runtime_mins, which will
be populated by the data in max_runtime_hrs * 60.

BUG=chromium-os:36067
TEST=Ran database/migrate.py sync on my local setup and verified that
a new column was added with the values from max_runtime_hrs
multipled by 60.

Change-Id: Ifeda0f6d5ea64b7a97047287c6102f4530f4c531
Reviewed-on: https://gerrit.chromium.org/gerrit/37567
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Commit-Ready: Simran Basi <sbasi@chromium.org>
Tested-by: Simran Basi <sbasi@chromium.org>
igrations/070_job_add_max_runtime_mins.py
353f538b7b38f8bb1f8f6ca1961912e7a137529a 18-Oct-2012 Scott Zawalski <scottz@google.com> Update Results Database link to point to /new_tko.

The /tko results database has been deprecated since before Chrome OS started
using Autotest. However, When you click on the link it spins forever and
never returns for the user but on the backend it kicks off a query that
hits the database pretty hard (See referenced bug).

BUG=chromium-os:34706
TEST=utils/compile_gwt_clients.py -a and checked the webpage.

Change-Id: Iae765de2391530b38a13427bfcff0ff327a4fd25
Reviewed-on: https://gerrit.chromium.org/gerrit/35980
Tested-by: Scott Zawalski <scottz@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Ready: Scott Zawalski <scottz@chromium.org>
Reviewed-by: Scott Zawalski <scottz@chromium.org>
lient/src/autotest/public/AfeClient.html
lient/src/autotest/public/TkoClient.html
5eb445e8d307b0afe68523dc09e9b1d9a0b265d3 13-Oct-2012 Mike Truty <truty@chromium.org> Present separate indicators of experimental test failures on dash pages.

Summary dash: If experimental bvt tests are run and any fail, the summary page
will show a secondary count of experimental pass/total.
If all experimental tests run pass or none are run, there is no
indication on the summary.
Popups elaborate on which failed tests were experimental.
Details pages: Experimental tests are marked as pink. They were not displayed
previously.

BUG=chromium-os:33884
TEST=http://truty.i/results/dashboard/waterfall_index.html
TEST=http://truty.i/results/dashboard/zgb/x86-zgb-r23/bvt.html

Change-Id: I64399590b3d0560169d699100aab01d259c0c807
Reviewed-on: https://gerrit.chromium.org/gerrit/35468
Reviewed-by: Danielle Drew <ddrew@chromium.org>
Tested-by: Danielle Drew <ddrew@chromium.org>
Commit-Ready: Danielle Drew <ddrew@chromium.org>
emplates/tables/base.html
emplates/tables/details/details.html
emplates/tables/summary/waterfall_index.html
emplates/tables/summary/waterfall_kernel.html
1d0490d1e91691858febe111ba6bc8f1b74742bc 12-Oct-2012 Mike Truty <truty@chromium.org> Clarify failure emails on 'experimental' tests.

This was complicated to reproduce. We run some tests as 'experimental'.
Generally, this implies the test is a bit flaky and we should not act
on it's results as aggressively (e.g. should not close the tree). This
tag is applied to some tests on some releases. To reproduce this needed
to find a failed, experimental test within the recent email window.
In the end I forced the behavior a bit.

Then, verified that internal 'experimental' flag is getting set properly
took a little time. Once this was all checked, the code to improve the
email was surprisingly simple.

BUG=chromium-os:35261
TEST=run_generate.py -j 300 -k -m -v
TEST=sample email: https://www/~truty/dashboard/2012-10-12_Experimental_Failure_Email.png

Change-Id: I80b06b7d7f2b5c70206aedf563fced43a87f5ce6
Reviewed-on: https://gerrit.chromium.org/gerrit/35434
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Tested-by: Mike Truty <truty@chromium.org>
Commit-Ready: Mike Truty <truty@chromium.org>
emplates/emails/tests_failure.html
c577f3e5fc270e16159d63300e1a2574ad311429 27-Sep-2012 Alex Miller <milleral@chromium.org> [autotest] Add a |file_bugs| flag to dynamic_suites.

A flag named |file_bugs| is now accepted as part of the create_suite_job
rpc call and |reimage_and_run| now takes an additional |file_bugs| flag
to indicate if bugs should be filed for test failures in the suite run.

TEST=local run_suite
BUG=chromium-os:29513

Change-Id: I7354ed463d22d6fe85abdfbf9fad1882e7bd9dc0
Reviewed-on: https://gerrit.chromium.org/gerrit/34476
Tested-by: Alex Miller <milleral@chromium.org>
Reviewed-by: Chris Masone <cmasone@chromium.org>
Commit-Ready: Alex Miller <milleral@chromium.org>
fe/site_rpc_interface.py
8906ab1e576797069b5b928e646f06da015fbfd0 24-Jul-2012 Chris Masone <cmasone@chromium.org> [autotest] Honor DEPENDENCIES information when running dynamic_suites.

A test control file can specify a list of DEPENDENCIES, which are really just
the set of labels a host needs to have in order for that test to be scheduled
on it. In the case of a dynamic_suite, many tests in the suite may have
DEPENDENCIES specified. They may overlap, they may not.

This is a first-pass at a very naive mechanism for finding and reimaging a
set of machines that can satisfy all the test DEPENDENCIES in a given suite.
Per-test-per-suite DEPENDENCIES information is generated at build time, and
then fetched and consumed during suite preparation. If info exists for the
suite being run, the infrastructure will look for a host that matches each
unique DEPENDENCY list. If it can find such a set that is <= the suite
sharding factor, the infrastructure will schedule those machines for
reimaging. If not, the suite will fail.

If the suite to be run has no DEPENDENCIES, meta_host scheduling will be used
as today.

BUG=chromium-os:22060
TEST=unit
TEST=run_suite.py with a suite (like 'dummy') that uses DEPENDENCIES
TEST=run_suite.py with a suite that doesn't use DEPENDENCIES
TEST=run_suite.py with a suite that uses DEPENDENCIES; shut down a non-essential machine during reimage. Suite should complete.
TEST=Same as above, but for a suite that doesn't use DEPENDENCIES
TEST=run_suite.py with a suite that uses DEPENDENCIES; shut down an essential machine (e.g. the only one with the 'bluetooth' label) during reimage. Suite should not complete.

Change-Id: I64b2e39a79a05c92748f34ffa4acf7667b0476a6
Reviewed-on: https://gerrit.chromium.org/gerrit/27526
Commit-Ready: Chris Masone <cmasone@chromium.org>
Reviewed-by: Chris Masone <cmasone@chromium.org>
Tested-by: Chris Masone <cmasone@chromium.org>
fe/site_rpc_interface.py
f2f35ed796bb96cf4c52174eb5caee5c6ae0198d 14-Sep-2012 Dennis Jeffrey <dennisjeffrey@chromium.org> Add a "coming soon" link to autotest dashboard for new perf graphs.

This change modifies the autotest dashboard UI to display a "COMING SOON"
link in the dashboard header (underneath the "last updated:" message).
The link points to a proof-of-concept UI for a revamped perf dashboard
that is meant to replace the existing chromeOS perf dashboard soon.
The new perf dashboard more closely follows the style used by the Chrome
perf graphs.

As soon as the transition is made from the current perf dashboard UI to
the new one, the COMING SOON link will be removed.

BUG=None
TEST=None

Change-Id: Iecfaf84b22dbab387610a997443b45d741e3cd34
Reviewed-on: https://gerrit.chromium.org/gerrit/33253
Reviewed-by: Nirnimesh <nirnimesh@chromium.org>
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Commit-Ready: Dennis Jeffrey <dennisjeffrey@chromium.org>
Tested-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
emplates/tables/summary/header.html
accb5ceaac066025592808247bfa9c97966bba97 31-Aug-2012 Chris Sosa <sosa@chromium.org> Support graceful failover/aggregation for autotest devserver pool.

This change implements the graceful failover/aggregation designed in
crosbug.com/26451. This required a large re-work of how other modules
interact with the dev_server module. Instead of always relying on being
able to pass in the "build" to any dev_server RPC, a caller must start
by using the static resolve method that resolves a build to an ip addr
and creating a DevServer instance from that hostname.

i.e.

ds = Devserver.create()
ds.trigger_download('build')

turns into

ds = ImageServer.resolve('build')
ds.trigger_download()

In order to guarantee the re-use of the same hostname throughout the life-cycle
of a suite run, we keep the hostname as part of the suitespec and also parse
the package_url based on the image_url -- not the build.

As noted, I've also separated the imaging / crash components into their
own DevServer subclasses to simplify the interaction and modified
callers.

BUG=chromium-os:26451
TEST=Unittests so far + local autotest setup with running bvt suite on one
host in the lab + ran suite_enumerator with/without -l

Change-Id: I4760e4033bb347259e3bb30f2fac31f321654c6f
Reviewed-on: https://gerrit.chromium.org/gerrit/32006
Commit-Ready: Chris Sosa <sosa@chromium.org>
Reviewed-by: Chris Sosa <sosa@chromium.org>
Tested-by: Chris Sosa <sosa@chromium.org>
fe/site_rpc_interface.py
fe/site_rpc_interface_unittest.py
44e4d6ce5a7c38451c25d35b307b35338606091f 15-Aug-2012 Chris Masone <cmasone@chromium.org> [autotest] Continuing refactor of dynamic_suite code; splitting monster files!

Split the biiiiig dynamic_suite.py and dynamic_suite_unittest.py into separate
files for Reimager and Suite. Adjust imports to compensate.

BUG=chromium-os:30266
TEST=unit
TEST=successful run_suite.py run
TEST=suite_enumerator.py, suite_preprocessor.py,

Change-Id: I0457053a7ae1422970138619a64edbcfbbc338bc
Reviewed-on: https://gerrit.chromium.org/gerrit/30457
Tested-by: Chris Masone <cmasone@chromium.org>
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Commit-Ready: Chris Masone <cmasone@chromium.org>
fe/site_rpc_interface.py
fe/site_rpc_interface_unittest.py
b493555db2d43e79d96e793cae9d1ffb822dd6c1 14-Aug-2012 Chris Masone <cmasone@chromium.org> [autotest] Move dynamic suite code into its own module

No big rewrite, just moving files around

BUG=chromium-os:30266
TEST=unit
TEST=successful run_suite.py run
TEST=suite_enumerator.py, suite_preprocessor.py, and suite_scheduler.py

Change-Id: I2fbcae0332104ce3adcb10a1e90ce94cb209aca8
Reviewed-on: https://gerrit.chromium.org/gerrit/30267
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Commit-Ready: Chris Masone <cmasone@chromium.org>
Tested-by: Chris Masone <cmasone@chromium.org>
fe/site_rpc_interface.py
fe/site_rpc_interface_unittest.py
a525071ce40f60eb989541f8b13b33c77da27235 11-Aug-2012 Mark Koudritsky <kamrik@chromium.org> Fix power_Consumption graphs with shorter keyval names.

After keval names were shortened in
https://gerrit.chromium.org/gerrit/#/c/29631/
to fit into VARCHAR(30) field in the DB, those names
also have to be fiexed in croschart config.

Also separated the package and system domains into two
separate graphs.

BUG=none
TEST=none

Change-Id: Icd43ad1de689fb9b2cc4d13deb2e176a4d7a88b7
Reviewed-on: https://gerrit.chromium.org/gerrit/29948
Reviewed-by: Mike Truty <truty@chromium.org>
Commit-Ready: Mark Koudritsky <kamrik@chromium.org>
Tested-by: Mark Koudritsky <kamrik@chromium.org>
roschart/croschart_defaults.json
46d0eb11086d97db0178aa3cf316c8eb6d23c88d 28-Jul-2012 Chris Masone <cmasone@chromium.org> [autotest] Add |num| argument to create_suite_job RPC

This CL does not make any suite control file honor the argument yet.
It also does not provide a way to pass this arg from run_suite.py.

This change should drop in safely, and be a no-op to the rest of the
infrastructure. As the other changes are more likely to impact
behavior in production, I wanted to get this in on its own so we don't
churn this API a bunch with reverts and re-lands.

BUG=chromium-os:33027
TEST=use atest create suite to create a suite with the new -n arg; check the job on the AFE to see that your value got injected into the suite control file.

Change-Id: I53431b565ec6bf664baaadeca8ddd279a500f381
Reviewed-on: https://gerrit.chromium.org/gerrit/28651
Tested-by: Chris Masone <cmasone@chromium.org>
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Commit-Ready: Chris Masone <cmasone@chromium.org>
fe/site_rpc_interface.py
fe/site_rpc_interface_unittest.py
35a8a3f744b5183b7c07bbc58c2129cfd89aea8b 20-Jul-2012 Mike Truty <truty@chromium.org> Prune a few unused parts of the dashboard to make it a bit more manageable.

Kept the 'old-style' pages around for a while but really no one is using
them. So, it's ok to delete them. Also, removed a timing dash that is
no longer used and a test_config view.

BUG=None
TEST=http://truty.i/results/dashboard/waterfall_index.html

Change-Id: I7f21491b12b45b2915bdd9026def8f6b6b45a237
Reviewed-on: https://gerrit.chromium.org/gerrit/28028
Reviewed-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
Tested-by: Mike Truty <truty@chromium.org>
Commit-Ready: Mike Truty <truty@chromium.org>
emplates/tables/summary/header.html
3deb216bb6a1948a09ca1bcd678842ab5c33a68d 19-Jul-2012 Mike Truty <truty@chromium.org> Add proper redirection to find test results on multiple servers.

The retrieve_logs.cgi script locates the results whether they
remain on the CAutotest server, a results server or have been
archived into gs.

BUG=chromium-os:32483
TEST=http://truty.i/results/dashboard/daisy/daisy-r22/bvt.html

Change-Id: I8f4b658d884aea41bc367b86a9526003c6528026
Reviewed-on: https://gerrit.chromium.org/gerrit/27853
Reviewed-by: Kees Cook <keescook@chromium.org>
Reviewed-by: Jim Hebert <jimhebert@chromium.org>
Tested-by: Mike Truty <truty@chromium.org>
Commit-Ready: Mike Truty <truty@chromium.org>
emplates/emails/tests_failure.html
emplates/tables/details/details.html
f40a66ec3ca11e4fea2b6aecb2e4227cee2b71b5 12-Jul-2012 Mike Truty <truty@chromium.org> Improve ability to find result logs in multiple places.

retrieve_logs.cgi finds the logs whether they reside on
a local server or remote in gs.

BUG=chromium-os:32483
TEST=http://truty.i/results/dashboard/alex/x86-alex-r21/security.html

Change-Id: Ifbd29f050abb3407ca685fba0d5328774a56d889
Reviewed-on: https://gerrit.chromium.org/gerrit/27294
Tested-by: Mike Truty <truty@chromium.org>
Reviewed-by: Nirnimesh <nirnimesh@chromium.org>
Commit-Ready: Mike Truty <truty@chromium.org>
emplates/tables/details/details.html
38428efca550c8decba134f8f4a1b08ff7619ac5 10-Jul-2012 Scott Zawalski <scottz@google.com> Add in random server fixes for the move to the new Atlantis machines.

Update settings.py to contain a SESSION_COOKIE_AGE of 1200 seconds. this is a
temporary hack to keep the number of django sessions down via the autotest
scheduler clean up process. Every new RPC seems to be creating a new
django_session entry and this needs to be futher evaluated.

Update global_config.ini to enable atlantis2 as a drone. This moves the
autoserv processes off of the main database server and opens the door for
further horizontal scaling as we need it later on.

Set the process limit for atlantis2 to 600 so we do not kill the machine.

Add in a dummy site_archive_results so that we do not copy results back to
atlantis1 AKA cautotest. This copy is unnecessary since we move all of our
results to Google Storage via gs_offloader.

TEST=Running live on the server now.
BUG=None

Change-Id: I77fc08c543092c21acd0cc238ea04c91527c9738
Reviewed-on: https://gerrit.chromium.org/gerrit/27057
Reviewed-by: Chris Sosa <sosa@chromium.org>
Reviewed-by: Chris Masone <cmasone@chromium.org>
Tested-by: Scott Zawalski <scottz@chromium.org>
Commit-Ready: Scott Zawalski <scottz@chromium.org>
Reviewed-by: Scott Zawalski <scottz@chromium.org>
ettings.py
8d6e6413016cb267edfb45f8eb2ff17bbede0967 28-Jun-2012 Chris Masone <cmasone@chromium.org> [autotest] Factor out some result-gathering code

In preparation for some reworking of the way we reimage devices,
factor out the code we use in the Suite class to wait for test results
so that we can re-use it in the Reimager class too.

BUG=chromium-os:22060
TEST=run_suite

Change-Id: I630d64e955c83dfd4e885f13a0500c62e30fe59c
Reviewed-on: https://gerrit.chromium.org/gerrit/26324
Tested-by: Chris Masone <cmasone@chromium.org>
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Commit-Ready: Chris Masone <cmasone@chromium.org>
fe/site_rpc_interface.py
8dd27e06d8e7031c65f358bf5e2aa474675d9047 26-Jun-2012 Chris Masone <cmasone@chromium.org> [autotest] Add more info to exceptions raised by suite_scheduler

Add build/board info to exceptions coming from the create_suite_job() RPC

BUG=chromium-os:31377
TEST=run_suite with a real suite/build
TEST=run_suite with a nonexistent suite
TEST=run_suite with a nonexistent build
STATUS=Fixed

Change-Id: Id862842dab03fd59b36addc8d12e5bae7a602139
Reviewed-on: https://gerrit.chromium.org/gerrit/26063
Commit-Ready: Chris Masone <cmasone@chromium.org>
Reviewed-by: Chris Masone <cmasone@chromium.org>
Tested-by: Chris Masone <cmasone@chromium.org>
fe/site_rpc_interface.py
45cdf2db6e3f76e721e1712a91f3bfbbd43a755b 21-Jun-2012 Todd Broch <tbroch@chromium.org> autotest: Add RAPL measurements to power_Idle chart.

BUG=chromium-os:31503
TEST=none
Change-Id: If40c0c963c0e8cbbfc7fae81d9affad9b99c5bbb
Reviewed-on: https://gerrit.chromium.org/gerrit/25853
Reviewed-by: Mark Koudritsky <kamrik@chromium.org>
Tested-by: Todd Broch <tbroch@chromium.org>
Commit-Ready: Todd Broch <tbroch@chromium.org>
roschart/croschart_defaults.json
6ef6e36c4823614290ff3b98d0e31de2a079f527 15-Jun-2012 Mark Koudritsky <kamrik@chromium.org> Add some values from power_Consumption to dashboard graphs.

This is a partial list of variables that seemed the "most interesting"
to me. More will follow. Look at the bug for discussion.

BUG=chromium-os:31886
TEST=none

Change-Id: I24ba3f9bab9d9f91a27d7e8766803fea6361100c
Reviewed-on: https://gerrit.chromium.org/gerrit/25402
Reviewed-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
Reviewed-by: Todd Broch <tbroch@chromium.org>
Commit-Ready: Mark Koudritsky <kamrik@chromium.org>
Tested-by: Mark Koudritsky <kamrik@chromium.org>
roschart/croschart_defaults.json
b6f82af084aa76bad383501019474854d0bd6b03 15-Jun-2012 Mike Truty <truty@chromium.org> Adjust perf charts for v2 suites job name style.

New suites use release/R21 instead of release-R21.

BUG=chromium-os:31846
TEST=http://truty.i/croschart/server/chart?testkey=platform_BootPerfServer,seconds_kernel_to_startup,seconds_kernel_to_startup_done,seconds_kernel_to_x_started,seconds_kernel_to_chrome_exec,seconds_kernel_to_chrome_main,seconds_kernel_to_login,seconds_kernel_to_network&interval=4,week&board=x86-alex-r21&system=alex&width=1200&height=800&updatecache=true

Change-Id: If974f3184b5791494f7b2280f3dc8715cf3031d3
Reviewed-on: https://gerrit.chromium.org/gerrit/25415
Reviewed-by: Richard Barnette <jrbarnette@chromium.org>
Tested-by: Mike Truty <truty@chromium.org>
Commit-Ready: Mike Truty <truty@chromium.org>
roschart/chartmodels.py
roschart/chartutils.py
71c5dbb61c0906596d47891346948350b51096d5 07-Jun-2012 Mike Truty <truty@chromium.org> Create a small report to track django_session table growth.

Need to also create a process to poll the count over time
and persist the poll results somewhere available for plotting.

BUG=chromium-os:31608
TEST=http://truty.i/results/dashboard/monitoring.html

Change-Id: I3325bd9e194bdb65b909a03fa4e874e46ad2b91a
Reviewed-on: https://gerrit.chromium.org/gerrit/24664
Reviewed-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
Reviewed-by: Chris Sosa <sosa@chromium.org>
Tested-by: Mike Truty <truty@chromium.org>
Commit-Ready: Mike Truty <truty@chromium.org>
emplates/tables/monitoring.html
715654f7bebe55b14d417f756d13b3018c846d1b 30-May-2012 Ilja H. Friedel <ihf@google.com> autotest: chart some audiovideo_FFMPEG results.

TEST=Passes jsonlint.com

Change-Id: I5f129cbc7202138336ca69d48f09ed987a4cd7b9
Reviewed-on: https://gerrit.chromium.org/gerrit/23952
Reviewed-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
Commit-Ready: Ilja H. Friedel <ihf@chromium.org>
Reviewed-by: Ilja H. Friedel <ihf@chromium.org>
Tested-by: Ilja H. Friedel <ihf@chromium.org>
roschart/croschart_defaults.json
aa10f8ed912a35f83aa9b561f3b7ccc90f655d7c 15-May-2012 Chris Masone <cmasone@chromium.org> [autotest] Gather dynamic suite constants into dynamic_suite.py

BUG=chromium-os:30351
TEST=unit tests, run_suite

Change-Id: Ia9a0d486a38aac3ab181e32563d2d3ed05b35d59
Reviewed-on: https://gerrit.chromium.org/gerrit/22875
Commit-Ready: Chris Masone <cmasone@chromium.org>
Reviewed-by: Chris Masone <cmasone@chromium.org>
Tested-by: Chris Masone <cmasone@chromium.org>
fe/site_rpc_interface.py
fe/site_rpc_interface_unittest.py
f70650c57a8f44f80055f5d136c4808a79d67c4d 16-May-2012 Chris Masone <cmasone@chromium.org> [autotest] Re-land "Put dev server error strings in exceptions"

We've been ignoring error messages coming back from the dev server.
Strip out the HTML that comes in (some of) them and wrap them
in exceptions.

This reverts commit 0dcf3a4c80b66d7598f3d7391bc1f9bf5492d8e2.

BUG=chromium-os:30279
TEST=unit
TEST=use atest suite create to run a suite that doesn't exist on build that doesn't exist.

Change-Id: I9d684d8b1c959d17c4b17082ef6892112b860493
Reviewed-on: https://gerrit.chromium.org/gerrit/22847
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Tested-by: Chris Masone <cmasone@chromium.org>
Commit-Ready: Chris Masone <cmasone@chromium.org>
fe/site_rpc_interface.py
0dcf3a4c80b66d7598f3d7391bc1f9bf5492d8e2 16-May-2012 Chris Sosa <sosa@chromium.org> Revert "[autotest] Put dev server error strings in exceptions"

This change exposed a bug in the old test_scheduler. The old test_scheduler
didn't do anything to suppress exceptions and aborts if the devserver returns
any exceptions.

This change is good as is and can be safely re-reverted once the old test_scheduler is robustified a little.

This reverts commit c807fcb56c57dbac1b193d92a90d59388c0cefe4

BUG=chromium-os:30915
TEST=Ran on autotest server with test_scheduler.

Change-Id: I22b9bbf87552c411c82b2d4e67fdbc33576eacb6
Reviewed-on: https://gerrit.chromium.org/gerrit/22776
Reviewed-by: Chris Sosa <sosa@chromium.org>
Tested-by: Chris Sosa <sosa@chromium.org>
fe/site_rpc_interface.py
e3e63a22ab0cc4e63a66ed35d367b110f82ff66a 15-May-2012 Chris Masone <cmasone@chromium.org> [autotest] Fix site_rpc_interface testStageBuildFail unittest

This was broken when I refactored some dev_server.DevServer code to raise
an exception instead of returning False.

BUG=None
TEST=unit

Change-Id: I6bdc2b63d108db384be6955fabe60218f959c9a0
Reviewed-on: https://gerrit.chromium.org/gerrit/22737
Tested-by: Chris Masone <cmasone@chromium.org>
Reviewed-by: Chris Sosa <sosa@chromium.org>
Commit-Ready: Chris Masone <cmasone@chromium.org>
fe/site_rpc_interface_unittest.py
5117e3eb723645e48e004589123af8fb5e0787cf 10-May-2012 Chris Masone <cmasone@chromium.org> [autotest] Put dev server error strings in exceptions

We've been ignoring error messages coming back from the dev server.
Strip out the HTML that comes in (some of) them and wrap them
in exceptions.

BUG=chromium-os:30279
TEST=unit
TEST=use atest suite create to run a suite that doesn't exist on build that doesn't exist.

Change-Id: I86b149610bf54334cdca12836a078ab874c818b2
Reviewed-on: https://gerrit.chromium.org/gerrit/22315
Commit-Ready: Chris Masone <cmasone@chromium.org>
Reviewed-by: Chris Masone <cmasone@chromium.org>
Tested-by: Chris Masone <cmasone@chromium.org>
fe/site_rpc_interface.py
ae4d278eaab0c998c11532332ba062729db95b53 10-May-2012 Dennis Jeffrey <dennisjeffrey@chromium.org> Remove autotests desktopui_V8Bench and desktopui_SunSpiderBench.

These autotests have been replaced by pyauto-based versions that are
implemented as part of desktopui_PyAutoPerfTests.

BUG=chromium-os:18457
TEST=None

Change-Id: Ic4411be7a2e185f608f44d6a03b23f402313bb87
Reviewed-on: https://gerrit.chromium.org/gerrit/22290
Reviewed-by: Mike Truty <truty@chromium.org>
Reviewed-by: Sonny Rao <sonnyrao@chromium.org>
Tested-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
Commit-Ready: Dennis Jeffrey <dennisjeffrey@chromium.org>
roschart/croschart_defaults.json
roschart/releasereport/crosrelease_defaults.json
f8b5306840870a8ebec3a5dc4b77c0bd79a69c5b 09-May-2012 Chris Masone <cmasone@chromium.org> [autotest] Make json_rpc raise meaningful exceptions

Add some dynamic suite exceptions to client/common_lib/error.py so
that different parts of the system can all reference them. Then,
enable the json_rpc code to detect these exceptions coming back
over the wire, re-instantiate them on the client side, and raise
them.

BUG=chromium-os:30279
TEST=unit
TEST=use atest suite create to try to create a suite for a build that doesn't exist
TEST=use atest suite create to try to run a suite that doesn't exist

Change-Id: I1b6d56a7e1bdb63cc893a07581efc8decc0407f4
Reviewed-on: https://gerrit.chromium.org/gerrit/22250
Tested-by: Chris Masone <cmasone@chromium.org>
Reviewed-by: Chris Sosa <sosa@chromium.org>
Commit-Ready: Chris Masone <cmasone@chromium.org>
fe/json_rpc/proxy.py
fe/site_rpc_interface.py
fe/site_rpc_interface_unittest.py
4264c4a21335eba7a54a4be8c977c50fd46be257 05-May-2012 Mike Truty <truty@chromium.org> Finish board matching in simplified dash_config.

Board config matching needed another layer of matching in
the config specific emailer on top of previously implemented
category and test matching.

Also fixed the utility that generates a view of the data
model for debugging issues related to the model data.

Also updated the emailer instructions page.

BUG=None
TEST=local dash email generation and cautotest dash email gen.
http://truty.i.corp.google.com/results/dashboard/emails.html

Change-Id: Ie1cd3bec6157fb8dfc00b06627292d0609a82553
Reviewed-on: https://gerrit.chromium.org/gerrit/21928
Reviewed-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
Commit-Ready: Mike Truty <truty@chromium.org>
Tested-by: Mike Truty <truty@chromium.org>
emplates/tables/summary/emails.html
5053b531d7f378d7adc3271c4bef9cfd5c047fbc 07-May-2012 Ilja H. Friedel <ihf@google.com> autotest: remove glbench clear_depth from chart.

It breaks plotting on SNB because it is so much bigger than the rest.

TEST=Validated with jsonlint.com

Change-Id: I34473ef0791118453b9001cd6b3ffec54616658e
Reviewed-on: https://gerrit.chromium.org/gerrit/21945
Reviewed-by: Stuart Abercrombie <sabercrombie@chromium.org>
Commit-Ready: Ilja H. Friedel <ihf@chromium.org>
Reviewed-by: Ilja H. Friedel <ihf@chromium.org>
Tested-by: Ilja H. Friedel <ihf@chromium.org>
roschart/croschart_defaults.json
49cbad1d29df10aaabfd7a55f98a7e10c8e1d034 03-May-2012 Nirnimesh <nirnimesh@chromium.org> Remove referneces to desktopui_PageCyclerTests

BUG=None
TEST=None

Change-Id: I082c44eb58aa9e0638301604c37fe367b3b6552c
Reviewed-on: https://gerrit.chromium.org/gerrit/21701
Reviewed-by: Sonny Rao <sonnyrao@chromium.org>
Commit-Ready: Nirnimesh <nirnimesh@chromium.org>
Tested-by: Nirnimesh <nirnimesh@chromium.org>
roschart/croschart_defaults.json
roschart/releasereport/crosrelease_defaults.json
d2f2af61ca9e182188ef21b3a74daec4f920d8f8 02-May-2012 Mike Truty <truty@chromium.org> Adjust bvt dash to accept xx_bvt suites as bvt.

Special-case kaen bvt suite to exclude unfixable tests.
Also, adjust summary link toward http://groups email away
from static file.

BUG=None
TEST=http://truty.i/results/dashboard/waterfall_index.html

Change-Id: Ic4fbc78454f6afe1095231cb0ffcc1cf55694b52
Reviewed-on: https://gerrit.chromium.org/gerrit/21606
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Commit-Ready: Mike Truty <truty@chromium.org>
Tested-by: Mike Truty <truty@chromium.org>
emplates/tables/summary/waterfall_index.html
57ff4a7253e8e8d9bf0b3e8e1aa66e3ecce1545b 02-May-2012 Ilja H. Friedel <ihf@google.com> autotest: fully hook up graphics_GLMark2.

It started running recently with the new scheduler (I think). Make sure it also
runs with all other graphics_* tests and the score gets plotted on the perf
dashboard.

BUG=none
TEST=Runs on my lumpy, score shows up, json validates.

Change-Id: I83c9c270f869b947ac011efdf24f19a262b7e356
Reviewed-on: https://gerrit.chromium.org/gerrit/21605
Commit-Ready: Ilja H. Friedel <ihf@chromium.org>
Reviewed-by: Ilja H. Friedel <ihf@chromium.org>
Tested-by: Ilja H. Friedel <ihf@chromium.org>
Reviewed-by: Chinyue Chen <chinyue@chromium.org>
roschart/croschart_defaults.json
a8066a9706027b55199b7eac573df09e20206e6b 02-May-2012 Chris Masone <cmasone@chromium.org> [autotest] Add more timing info for suites

Use job keyvals to keep track of the timing of certain
events that do not map nicely to the start/completion
of jobs. These include:
- Triggering of payload/build artifact downloads
- Completion of staging payloads
- Completion of staging build artifacts (debug symbols, etc)

BUG=chromium-os:30327
TEST=unit, run_suite and look at the glory of additional timestamps.

Change-Id: I2e8f0c148415bdbb2031616f057153b4bd9523ad
Reviewed-on: https://gerrit.chromium.org/gerrit/21578
Commit-Ready: Chris Masone <cmasone@chromium.org>
Reviewed-by: Chris Masone <cmasone@chromium.org>
Tested-by: Chris Masone <cmasone@chromium.org>
fe/site_rpc_interface.py
fe/site_rpc_interface_unittest.py
84182a4f5107d55732e76e92bcc5c713d84f276c 30-Mar-2012 Mike Truty <truty@chromium.org> Simplify test failure emails and send to Google group.

Setup a parallel email generator that does not use any config
file. Rather, send an email for every test failure on every
board to a Google Group. Test owners can then subscribe to
the group and setup a filter on either the test category
or the test naem.

Simplify test owner email registration format to include
the ability to specify groups as a regex and to supply an
optional test regex within a group regex to filter test failures.

BUG=chromium-os:28728
TEST=https://groups.google.com/a/google.com/forum/?fromgroups#!topicsearchin/chromeos-automated-test-failures/%22Autotest$20failure$20in$20pyauto_basic$20on$20x86-alex-r20$20(R20-2048.0.0-a1-b1855)%22

Change-Id: Ie115becd1b039e1dafc820ed8aed11ae82dae2b1
Reviewed-on: https://gerrit.chromium.org/gerrit/19346
Reviewed-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
Tested-by: Mike Truty <truty@chromium.org>
Commit-Ready: Mike Truty <truty@chromium.org>
emplates/emails/tests_base.html
emplates/tables/details/details.html
emplates/tables/summary/emails.html
ee8f9f224166f8d39a1581498b6718967d967804 30-Apr-2012 Zdenek Behan <zbehan@chromium.org> migrations: extend reason column for tko_tests

Apparently, old size isn't enough. Changed to 4096, which
is yet another bulgarian constant.

BUG=chromium-os:24392
TEST=/usr/local/autotest/database/migrate.py sync

Change-Id: Ie5a4a5e3e6400571fbc9b93108339b578dd1dee1
Reviewed-on: https://gerrit.chromium.org/gerrit/21475
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Commit-Ready: Zdenek Behan <zbehan@chromium.org>
Tested-by: Zdenek Behan <zbehan@chromium.org>
igrations/069_tko_tests_extend_reason_table.py
9572d08a3d5f27d5d171a5979712556e4ed94f5c 28-Apr-2012 Ilja H. Friedel <ihf@google.com> Autotest perf charting.

Add GPU and CPU memory plots for the new memory tests.
Also clean up some of the graphics plotting.

BUG=29815
TEST=Validated by http://jsonlint.com.

Change-Id: I2986b40216a79c870cec81a6ecec9d80b0cc22c3
Reviewed-on: https://gerrit.chromium.org/gerrit/21430
Reviewed-by: Sonny Rao <sonnyrao@chromium.org>
Tested-by: Ilja H. Friedel <ihf@chromium.org>
Commit-Ready: Ilja H. Friedel <ihf@chromium.org>
Reviewed-by: Ilja H. Friedel <ihf@chromium.org>
roschart/croschart_defaults.json
249697c413b290145a28499ffa5c647a27c5de02 25-Apr-2012 Dennis Jeffrey <dennisjeffrey@chromium.org> Adding pyauto-based page cycler test graphs to the autotest perf dashboard.

BUG=None
TEST=http://cros-reports/croschart/server/chart?testkey=desktopui_PyAutoPerfTests,milliseconds_Alexa_usFile_Page,milliseconds_MozFile_PageCycle,milliseconds_Moz2File_PageCycl&interval=4,week&board=lumpy-r20&system=lumpy

http://cros-reports/croschart/server/chart?testkey=desktopui_PyAutoPerfTests,milliseconds_DhtmlFile_PageCyc,milliseconds_MoreJSFile_PageCy,milliseconds_Intl2File_PageCyc,milliseconds_Intl1File_PageCyc&interval=4,week&board=lumpy-r20&system=lumpy

http://cros-reports/croschart/server/chart?testkey=desktopui_PyAutoPerfTests,milliseconds_BloatFile_PageCyc&interval=4,week&board=lumpy-r20&system=lumpy

Change-Id: Id461455bd392ca8af9c0ef11672ac2b69cc9336f
Reviewed-on: https://gerrit.chromium.org/gerrit/21069
Reviewed-by: Mike Truty <truty@chromium.org>
Tested-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
Commit-Ready: Dennis Jeffrey <dennisjeffrey@chromium.org>
roschart/croschart_defaults.json
47c9e64d54f772970ab5fd9d454650bf767944a0 25-Apr-2012 Chris Masone <cmasone@chromium.org> [autotest] Avoid races due to 'version' labels on devices in the lab.

At a high level, this change makes dynamic suites stop deleting the version
labels we use at the end of a run, and also makes the code charged with
creating that label tolerant of discovering that one already exists.

To do the latter, we needed to make the underlying RPC code return
appropriate RPC errors immediately, so that we can detect (and ignore)
them when we try to create a label and it already exists. Since autotest's
json_rpc just had one type of exception it could throw, we had to add a new
exception type to proxy.py and handle it appropriately.

BUG=chromium-os:29971
TEST=unit
TEST=pre-create a version label and run a suite. It should run, not error out.
TEST=Run a suite without the label existing. Should also run.

Change-Id: I71de5670439c6933d1523ae65f7bfb6bc66c4f88
Reviewed-on: https://gerrit.chromium.org/gerrit/21124
Reviewed-by: Chris Masone <cmasone@chromium.org>
Tested-by: Chris Masone <cmasone@chromium.org>
Commit-Ready: Chris Masone <cmasone@chromium.org>
fe/json_rpc/proxy.py
fdfc133c61df832c041ea6b506c08558f613bb99 16-Apr-2012 Mike Truty <truty@chromium.org> Improve perf chart usability: longer durations: 2weeks -> 4weeks.

BUG=chromium-os:29666

TEST=http://truty.i.corp.google.com/results/dashboard/mario/x86-mario-r20/perf_index.html

Change-Id: I7ded521c6e7b2764e8e76ab69bc9ba74c55a51a1
Reviewed-on: https://gerrit.chromium.org/gerrit/20508
Reviewed-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
Tested-by: Mike Truty <truty@chromium.org>
Commit-Ready: Mike Truty <truty@chromium.org>
emplates/tables/details/index.html
emplates/tables/details/perf_index.html
emplates/tables/summary/header.html
emplates/tables/summary/kernel_header.html
851249c4897e1b7c84e2bc941204ba019fc270ab 16-Apr-2012 Mike Truty <truty@chromium.org> Fix perfcharts to accept R##-ww.xx.yy jobname/build strings.

Perf chart axix labels are populated by parsing build strings.
Build strings change when jobnames changed recently. This
gets perf charts to show recent build data points.

Also, the perf chart diff view was failing with a javascript error
for the same reason. Fixed that parsing.

BUG=chromium-os:29438
TEST=http://truty.i/croschart/server/chartreport?board=x86-alex-r20&system=alex&interval=2,week
TEST=http://truty.i/croschart/server/chart?testkey=platform_BootPerfServer,seconds_kernel_to_startup,seconds_kernel_to_startup_done,seconds_kernel_to_x_started,seconds_kernel_to_chrome_exec,seconds_kernel_to_chrome_main,seconds_kernel_to_login,seconds_kernel_to_network&interval=2,week&board=x86-alex-r20&system=alex
TEST=http://truty.i/croschart/server/chartdiff?interval=2%2Cweek&board=x86-alex-r20&testkey=platform_BootPerfServer%2Cseconds_kernel_to_startup%2Cseconds_kernel_to_startup_done%2Cseconds_kernel_to_x_started%2Cseconds_kernel_to_chrome_exec%2Cseconds_kernel_to_chrome_main%2Cseconds_kernel_to_login%2Cseconds_kernel_to_network&system=alex

Change-Id: I77e9e7f085d5b3ed09e81a9f6f50538c2d5aa8ac
Reviewed-on: https://gerrit.chromium.org/gerrit/20310
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Tested-by: Mike Truty <truty@chromium.org>
Commit-Ready: Mike Truty <truty@chromium.org>
roschart/chartutils.py
roschart/templates/perfchart/plot_chartdiff.html
4b2c6f3d54838baacfba870fb6470eca1d1053c0 14-Apr-2012 Mike Truty <truty@chromium.org> Adjust job name parsing to avoid build sequence#'s.

Build sequence #'s were originally used to locate build logs.
Until a standard 'lookup' function is developed, the
build logs were located by updating BUILDBOT_URLS in dash_strings.

Since sequence#'s are no longer available, deprecate the ability
to lookup build logs.

In addition, Build#'s used to be longer and so the build sequence
numbers were used as short substitutes. Now since build#'s are
shorter and build sequence numbers are deprecated, simplify
the code that used sequence numbers.

BUG=chromium-os:29438
TEST=http://truty.i/results/dashboard/waterfall_index.html

Change-Id: I1864566489c55c21719516524727b5e106c26c1b
Reviewed-on: https://gerrit.chromium.org/gerrit/20272
Reviewed-by: Chris Sosa <sosa@chromium.org>
Tested-by: Mike Truty <truty@chromium.org>
Commit-Ready: Mike Truty <truty@chromium.org>
emplates/tables/summary/header.html
633148b83fa3180d78fbf6f0f49614df54900846 12-Apr-2012 Ilja H. Friedel <ihf@google.com> Remove comments from json.

The comments broke the perf plotting dashboard.
After removing it validates using http://jsonlint.com/

Change-Id: Icd14d83568582efca6843d3346557ad7a3b7ad57
Reviewed-on: https://gerrit.chromium.org/gerrit/20092
Reviewed-by: Chris Masone <cmasone@chromium.org>
Commit-Ready: Ilja H. Friedel <ihf@chromium.org>
Reviewed-by: Ilja H. Friedel <ihf@chromium.org>
Tested-by: Ilja H. Friedel <ihf@chromium.org>
roschart/croschart_defaults.json
7c769f2bc7a160e163aaa815628b1f78aa355a54 11-Apr-2012 Ilja H. Friedel <ihf@google.com> autotest: chart numbers from graphics_* tests.

This will allow us to keep track of changes with less effort.
Also start scheduling graphics_GLPerformance test once a day.

TEST=none

Change-Id: I07654336d60fc183f160cddeb08209ed37b41b77
Reviewed-on: https://gerrit.chromium.org/gerrit/19965
Reviewed-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
Tested-by: Ilja H. Friedel <ihf@chromium.org>
Commit-Ready: Ilja H. Friedel <ihf@chromium.org>
roschart/croschart_defaults.json
9ee6f4feb97e18b8da746ec76c3c397dfae53856 10-Apr-2012 Dennis Jeffrey <dennisjeffrey@chromium.org> Add back the MB_ScimarkGui* graph to the CrOS perf dashboard.

This graph was removed as one of the changes in

https://gerrit.chromium.org/gerrit/#change,19851

but it is still useful for at least a few people, so it is being
added back to the dashboard.

BUG=None
TEST=None

Change-Id: Ie512b166f0ae50f77a7eee3984c3ebfaa05f444d
Reviewed-on: https://gerrit.chromium.org/gerrit/19892
Reviewed-by: Mike Truty <truty@chromium.org>
Tested-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
Commit-Ready: Dennis Jeffrey <dennisjeffrey@chromium.org>
Reviewed-by: Ilja H. Friedel <ihf@chromium.org>
roschart/croschart_defaults.json
07b5261acb77cea32f1fe825e95be760f203b5f0 09-Apr-2012 Dennis Jeffrey <dennisjeffrey@chromium.org> Remove MB_ScimarkGui- graph from perf dashboard, and add graph line to FPS_Scroll for new Gmail scrolling test.

The MB_ScimarkGui graph shows results that are all over the place, and I don't think the graph is particularly
helpful. This change removes that graph from the default perf report on the autotest dashboard.

BUG=chromium-os:28890
TEST=http://cros-reports/croschart/server/chart?interval=2%2Cweek&board=x86-alex-r20&testkey=desktopui_PyAutoPerfTests%2CFPS_ScrollBlankPage%2CFPS_ScrollTextPage%2CFPS_ScrollGooglePlusPage%2CFPS_ScrollGmail&system=alex

Change-Id: I43c603935288731943d22655a4e3eafd9cb556c9
Reviewed-on: https://gerrit.chromium.org/gerrit/19851
Reviewed-by: Mike Truty <truty@chromium.org>
Tested-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
Commit-Ready: Dennis Jeffrey <dennisjeffrey@chromium.org>
roschart/croschart_defaults.json
347aaf4b6830f64709ba76217c2db702a7c1a98d 03-Apr-2012 Scott Zawalski <scottz@google.com> Remove need for KerberosLogin from RPC tools.

The initial move, which happened outside of Chrome OS, to KerberosLogin was to
add more security around jobs that ran in Google Data centers. Chrome OS's lab
is lot more isolated and all tests are run on limited hardware. The risk
of exposure here is a lot lower, as the Chrome OS lab has no access to customer
data.

Removing this requirement moves auth back to the previous getpass() pathway.
When people go to the frontend they will be properly identified by SSO.
When they run jobs via the CLI or RPC interfaces they will be identified by
getpass accordingly. This is spoofable, but the previous KerberosLogin was also
spoofable after auth.

This will allow people to use the RPC interface from the chroot and will
alleviate failures like the one referenced in this commit.

TEST=put on test machine and tested SSO login from the frontend and cli access.
frontend_unittest.py tests.
Checked apahce logs and verified requests are going through the /noauth/
pathway.
BUG=chromium-os:28847

Change-Id: I7fd32a3b3864ecb7c3a42a642c666f1673c6e1c6
Reviewed-on: https://gerrit.chromium.org/gerrit/19537
Tested-by: Scott Zawalski <scottz@chromium.org>
Reviewed-by: Chris Masone <cmasone@chromium.org>
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Commit-Ready: Scott Zawalski <scottz@chromium.org>
fe/doctests/001_rpc_test.txt
ko/urls.py
rls_common.py
6b288c89dfcc112f605854bd5563b70ee5d74006 30-Mar-2012 Chris Sosa <sosa@chromium.org> Modify dynamic suite pathway to allow some artifacts to be staged in the bgnd.

This change modifies the site_rpc_interface->dynamic_suite.py workflow to allow
autotest to stage larger build components while imaging devices. It uses
the download/wait_for_status interface provided by the devserver to
accomplish it.

BUG=chromium-os:27285
TEST=All effected unittests pass. Staging larger test on local devserver in
parallel.

Change-Id: Iae2a20762c565b8e6d440c9a501c1f99c73129ed
Reviewed-on: https://gerrit.chromium.org/gerrit/19328
Tested-by: Chris Sosa <sosa@chromium.org>
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Tested-by: Scott Zawalski <scottz@chromium.org>
Commit-Ready: Scott Zawalski <scottz@chromium.org>
fe/site_rpc_interface.py
fe/site_rpc_interface_unittest.py
8ce2adb4005356669580935ea4c70548a07c3801 24-Mar-2012 Mike Truty <truty@chromium.org> Adjust job naming parsing for another variation.

Autotest jobnames were not expected to change at all.
Board, release and suite are extracted from them and
key all other records. This aligns the dashboard jobbname
parsing with the current jobs.

New platform names have been introduced. This gets
the dash emails looking for failures on the new platforms.

In testing a cache directory limit was noticed. The cache
directory can now get pruned to hold not more than 60days
job artifacts (for crash collection).

BUG=chromium-os:28341
BUG=chromium-os:28343
TEST=http://truty.i/results/dashboard/alex/x86-alex-r19/security.html

Change-Id: Ie9d5016501c09ceffb7eb7faacd1cb304fe9b01e
Reviewed-on: https://gerrit.chromium.org/gerrit/19034
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Commit-Ready: Mike Truty <truty@chromium.org>
Tested-by: Mike Truty <truty@chromium.org>
emplates/tables/base.html
emplates/tables/details/details.html
1c876ec21d9da6e5d5371c29a06950696231e3f1 22-Mar-2012 Mike Truty <truty@chromium.org> Update perf dashboard to manage new job naming conventions.
Now accepts old (x86-mario-r19) and new (x86-mario-release-R19)
job naming in order to show perf data for old and new builds.

To see updated version of a chart on cros-reports:
-View perf chart page on cros-reports
-Click chart title to view single chart
-Change server part of url to http://truty.i
-Append &updatecache=true
-Refresh

BUG=chromium-os:27946
TEST=http://truty.i/croschart/server/chart?interval=2%2Cweek&board=x86-alex-r19&testkey=platform_BootPerfServer%2Cseconds_kernel_to_startup%2Cseconds_kernel_to_startup_done%2Cseconds_kernel_to_x_started%2Cseconds_kernel_to_chrome_exec%2Cseconds_kernel_to_chrome_main%2Cseconds_kernel_to_login%2Cseconds_kernel_to_network&system=ALEX&updatecache=true&width=1200&height=800

Change-Id: I18c217a7168fe4ee0d57ef37dd6e4743cf174658
Reviewed-on: https://gerrit.chromium.org/gerrit/18902
Reviewed-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
Commit-Ready: Mike Truty <truty@chromium.org>
Tested-by: Mike Truty <truty@chromium.org>
roschart/chartmodels.py
roschart/chartutils.py
6504247a12d5c30f3033847c4168df85f0c4510d 17-Mar-2012 Mike Truty <truty@chromium.org> Scrub platforms to unify old/new platform labels in reports.

Old-style platform labels were prefixed with netbook_ or
desktop_ while new platforms labels are not. This merges
old and new platforms with the same base name.
For example results from netbook_ALEX and ALEX will get
merged into 'alex'.

Also adjusted error links to Autotest queries to reflect
new TOT job naming.

BUG=chromium-os:27946
TEST=http://truty.i/results/dashboard/waterfall_index.html

Change-Id: I357ee34d1bb4721c10c2063148c214c778fe04b5
Reviewed-on: https://gerrit.chromium.org/gerrit/18413
Commit-Ready: Mike Truty <truty@chromium.org>
Reviewed-by: Mike Truty <truty@chromium.org>
Tested-by: Mike Truty <truty@chromium.org>
roschart/chartmodels.py
emplates/emails/performance_regressed.html
emplates/tables/details/header.html
emplates/tables/details/index.html
emplates/tables/details/perf_index.html
emplates/tables/performance/build_performance.html
emplates/tables/summary/bvt_timing.html
emplates/tables/summary/kernel_index.html
emplates/tables/summary/table_index.html
emplates/tables/summary/waterfall_index.html
emplates/tables/summary/waterfall_kernel.html
b1451a00453151e15c6699de862b329520f7d375 12-Mar-2012 Chris Masone <cmasone@chromium.org> [autotest] Move rpc_utils import in site_rpc_interface.py

In order to allow for unit tests of site_rpc_interface.py, we
need to avoid importing autotest_lib.frontend.afe.rpc_utils, as
it pulls in and tries to set up django. The current mechanism
seems to be a bit flaky, so simplify the logic and simply
put the import behind a layer of indirection that can be
mocked out during tests.

BUG=chromium-os:27074
TEST=unit
TEST=./cli/atest suite create -b x86-mario -i x86-mario-release/R19-1932.0.0-a1-b1751 dummy

Change-Id: I0b331128a317fcf64269964325e051c45aeef023
Reviewed-on: https://gerrit.chromium.org/gerrit/17843
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Commit-Ready: Chris Masone <cmasone@chromium.org>
Tested-by: Chris Masone <cmasone@chromium.org>
fe/site_rpc_interface.py
62579122694934d942a4844a615bc034bc2e8aa1 09-Mar-2012 Chris Masone <cmasone@chromium.org> [autotest] Pipe check_hosts parameter down into dynamic_suite

The builders need to schedule a test suite and then wait for it to
complete. They probably want to check to make sure the suite can be
scheduled, and error out if not. That's great. For suites that we
schedule once a day or asynchronously in some other way, we probably
don't want to bail just because the testbed has too many hosts that
need Repair or something. We want to fire and forget.

To support these two modes of operation, pipe a 'check_hosts' paramter
all the way from the create_suite_job() RPC, down through control
files, all the way to dynamic_suite.reimage_and_run(). Ensure that we
tolerate check_hosts being left unset in reimage_and_run(). Also,
make 'atest suite create' able to toggle this value.

BUG=chromium-os:27526
TEST=dynamic_suite_unittest.py, site_rpc_interface_unittest.py
TEST=./server/autoserv test_suites/dev_harness with check_hosts set both ways.
TEST=install the patch on an autotest instance, and re-run a test suite against the new interface. The suite should check for hosts and behave appropriately.

Change-Id: I10c3f42dbc37f26d7af3c40439ce212ebf74cfcd
Reviewed-on: https://gerrit.chromium.org/gerrit/17633
Tested-by: Chris Masone <cmasone@chromium.org>
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Commit-Ready: Chris Masone <cmasone@chromium.org>
fe/site_rpc_interface.py
fe/site_rpc_interface_unittest.py
ab4fd4c06f320ae2d710aef2fbd12201103a8c98 08-Mar-2012 Mike Truty <truty@chromium.org> Introduce a hostnames query parameter to scrub data from inferior machines.

This is an accommodation to enable some data scrubbing from machines that
confuse comparisons between unlike systems.

BUG=chromium-os:27346
TEST=http://truty.i/croschart/server/chart?interval=2%2Cweek&board=lumpy-r19&testkey=desktopui_PageCyclerTests%2CPageCyclerTest.Alexa_usFile%2CPageCyclerTest.MozFile%2CPageCyclerTest.Moz2File&system=LUMPY&hostnames=chromeos-rack5-host11,chromeos-rack5-host12

Change-Id: I0aaa65d47196d1e38ddc47cc1817c4403819cf15
Reviewed-on: https://gerrit.chromium.org/gerrit/17539
Reviewed-by: asharif <asharif@chromium.org>
Commit-Ready: Mike Truty <truty@chromium.org>
Tested-by: Mike Truty <truty@chromium.org>
roschart/perfchart/models.py
31b31da315a0e958bc39bdc03d4c5a6e4f7c5045 27-Jan-2012 Mike Truty <truty@chromium.org> Add test_name and system parameters to the testreport report.

The regular croschart/server/testreport shows all tests which takes a while
to return. Passing test_name='xxx'&system='yyy' greatly speeds the
query/report.

BUG=None
TEST=http://truty.i/croschart/server/testreport?interval=2,week&test_name=platform_BootPerfServer&system=ALEX

Change-Id: I67079fa95442faebabd4080f79953b363b4bc772
Reviewed-on: https://gerrit.chromium.org/gerrit/14956
Reviewed-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
Commit-Ready: Mike Truty <truty@chromium.org>
Tested-by: Mike Truty <truty@chromium.org>
roschart/chartmodels.py
roschart/testreport/models.py
65650173bd945da839c51a666d61be7f07b4d6d6 16-Feb-2012 Scott Zawalski <scottz@google.com> Add a "pool" option to Suite and Reimager

A pool is a set of machines with a specific label. The concept of pools is
needed to be able to ensure scheduling priority and to segrate our machine
population.

If no pool is set the previous behavior is followed. Jobs will be scheduled
against chromeos versions and reimager jobs will be scheduled against "board"

Modify VERSION_PREFIX to use a : instead of -.

Uodate run_suite to offer the optional --pool flag.
Update create_suite RPC interface to accept the pool option.
Update create_suite to assume all "suite names" are under
test_suites/control.<suite name>
TEST=Unittest and development server end to end runs.
BUG=chromium-os:26491

Change-Id: I3677c956bee195e20ad63b4e034fa7c6611b1ce1
Reviewed-on: https://gerrit.chromium.org/gerrit/16130
Reviewed-by: Chris Masone <cmasone@chromium.org>
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Tested-by: Scott Zawalski <scottz@chromium.org>
Commit-Ready: Frank Farzan <frankf@chromium.org>
fe/site_rpc_interface.py
859fdecd96355279570007849d8bc46ff71874d7 30-Jan-2012 Chris Masone <cmasone@chromium.org> [autotest] Add create_suite_job() site RPC.

Add create_suite_job() RPC to autotest using the site_rpc mechanism.
Also add support to atest to call this RPC.

The purpose of this RPC is to stage a build on the dev server (if
necessary), image N machines with it, and then run the desired
suite on those machines.

BUG=chromium-os:25573
TEST=atest suite create -b x86-mario -i x86-mario-release/R19-1675.0.0-a1-b1588 test_suites/control.dummy

Change-Id: I09288fe6ffc675e5b111f7f59e349015f52ebb8e
Reviewed-on: https://gerrit.chromium.org/gerrit/15279
Tested-by: Chris Masone <cmasone@chromium.org>
Commit-Ready: Scott Zawalski <scottz@chromium.org>
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Tested-by: Scott Zawalski <scottz@chromium.org>
fe/site_rpc_interface.py
fe/site_rpc_interface_unittest.py
5636fed542b7031c93f956664ebf694c2dfc5462 31-Jan-2012 Nirnimesh <nirnimesh@chromium.org> Remove obsolete test desktopui_ChromeFirstRender

BUG=chromium-os:20024
TEST=None

Change-Id: I6f6916c39a0c34a7c0dabf90bf37aad884b7c3f3
Reviewed-on: https://gerrit.chromium.org/gerrit/15111
Reviewed-by: Xiyuan Xia <xiyuan@chromium.org>
Commit-Ready: Nirnimesh <nirnimesh@chromium.org>
Tested-by: Nirnimesh <nirnimesh@chromium.org>
roschart/croschart_defaults.json
roschart/releasereport/crosrelease_defaults.json
28714735061550121ef816044f7fa00d5b00bb17 26-Jan-2012 Scott Zawalski <scottz@google.com> Move templates for the dashboard to the location
Django expects them to be in.

TEST=Ran dashboard setup without files under site_utils/dashboard/templates
BUG=None

Change-Id: Ic0ffa8a0c6fdc05ec5d25a0e9b9eda9cc71f87fc
Reviewed-on: https://gerrit.chromium.org/gerrit/14879
Reviewed-by: Mike Truty <truty@chromium.org>
Commit-Ready: Scott Zawalski <scottz@chromium.org>
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Tested-by: Scott Zawalski <scottz@chromium.org>
emplates/emails/performance_regressed.html
emplates/emails/tests_base.html
emplates/emails/tests_failure.html
emplates/emails/tests_success.html
emplates/tables/base.html
emplates/tables/configs/header.html
emplates/tables/configs/test_config.html
emplates/tables/details/details.html
emplates/tables/details/header.html
emplates/tables/details/index.html
emplates/tables/details/perf_builds.html
emplates/tables/details/perf_index.html
emplates/tables/details/test_header.html
emplates/tables/performance/build_performance.html
emplates/tables/summary/bvt_timing.html
emplates/tables/summary/emails.html
emplates/tables/summary/header.html
emplates/tables/summary/index.html
emplates/tables/summary/kernel_header.html
emplates/tables/summary/kernel_index.html
emplates/tables/summary/table_index.html
emplates/tables/summary/waterfall_index.html
emplates/tables/summary/waterfall_kernel.html
e8603507778a6a0257b12fea0fc96f9b0721a1d1 06-Jan-2012 Dennis Jeffrey <dennisjeffrey@chromium.org> Remove graph for old perf key "frames_YoutubeDroppedFrames" from the default perf report.

Also moved the two existing Youtube graphs together. This change does
not add any new graphs.

BUG=None
TEST=None

Change-Id: Iecda8d0574abe7a3dda7e80361d7f21f844758f9
Reviewed-on: https://gerrit.chromium.org/gerrit/13815
Reviewed-by: Rohit Makasana <rohitbm@chromium.org>
Tested-by: Rohit Makasana <rohitbm@chromium.org>
Tested-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
Commit-Ready: Dennis Jeffrey <dennisjeffrey@chromium.org>
roschart/croschart_defaults.json
9de1ee4a4a71486c28112408d4b58926740a70f1 08-Dec-2011 Dennis Jeffrey <dennisjeffrey@chromium.org> Enabling 2 perf graphs for an Angry Birds test in the autotest default perf report.

BUG=chromium-os:23230
TEST=http://cros-reports/croschart/server/chart?testkey=desktopui_PyAutoPerfTests,Fraction_AngryBirdsCpuBusy&interval=2,week&board=x86-alex-r17&system=ALEX
http://cros-reports/croschart/server/chart?testkey=desktopui_PyAutoPerfTests,MB_AngryBirdsV8HeapSize&interval=2,week&board=x86-alex-r17&system=ALEX

Change-Id: If5018887a027687cb3a204dfcc4b8fa6acd031cc
Reviewed-on: https://gerrit.chromium.org/gerrit/12653
Commit-Ready: Dennis Jeffrey <dennisjeffrey@chromium.org>
Reviewed-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
Tested-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
roschart/croschart_defaults.json
2a3925bc7b1582f1ef42c77b3914950ed0882238 05-Dec-2011 Rohit Makasana <rohitbm@chromium.org> Adding a performance test graph for various types of Youtube videos to default autotest per report.

BUG=None
TEST=http://cros-reports/croschart/server/chart?interval=2%2Cweek&board=x86-alex-r17&testkey=desktopui_PyAutoPerfTests%2Cframes_YoutubeDroppedFramesSlo%2Cframes_YoutubeDroppedFramesNor%2Cframes_YoutubeDroppedFramesFas&system=ALEX

Change-Id: I9b6850125f5f994b2a4c64cbffdf315721e4b8ef
Reviewed-on: https://gerrit.chromium.org/gerrit/12456
Reviewed-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
Reviewed-by: Mike Truty <truty@chromium.org>
Tested-by: Rohit Makasana <rohitbm@chromium.org>
Commit-Ready: Rohit Makasana <rohitbm@chromium.org>
Reviewed-by: Rohit Makasana <rohitbm@chromium.org>
roschart/croschart_defaults.json
5be0e9a2457064480d86c72a23480c3dcfb28cac 01-Dec-2011 Dennis Jeffrey <dennisjeffrey@chromium.org> Adding WebGL perf graphs to default autotest perf report.

BUG=None
TEST=http://cros-reports/croschart/server/chart?interval=2%2Cweek&board=x86-alex-r17&testkey=desktopui_PyAutoPerfTests%2Cfps_WebGLAquarium,fps_WebGLField,fps_WebGLSpaceRocks&system=ALEX

Change-Id: I7b4737ce537c0d2ab74814ee7f9b45d6a3d06b71
Reviewed-on: https://gerrit.chromium.org/gerrit/12303
Commit-Ready: Dennis Jeffrey <dennisjeffrey@chromium.org>
Reviewed-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
Tested-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
roschart/croschart_defaults.json
ae3cd6dab27fc618d4a20cd390c5029738cd53ea 30-Nov-2011 Rohit Makasana <rohitbm@chromium.org> Adding Netflix CPU and dropped frames performance test graphs to default autotest perf report

BUG=None
TEST=http://cros-reports/croschart/server/chart?interval=2%2Cweek&board=x86-alex-r17&testkey=desktopui_PyAutoPerfTests%2Cextrapolation_NetflixCPUExtrap&system=ALEX , http://cros-reports/croschart/server/chart?interval=2%2Cweek&board=x86-alex-r17&testkey=desktopui_PyAutoPerfTests%2Cframes_NetflixDroppedFrames&system=ALEX

Change-Id: Ibaed78894bade262aa3963aeedbb82ca371ba85b
Reviewed-on: https://gerrit.chromium.org/gerrit/12266
Reviewed-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
Reviewed-by: Mike Truty <truty@chromium.org>
Commit-Ready: Rohit Makasana <rohitbm@chromium.org>
Reviewed-by: Rohit Makasana <rohitbm@chromium.org>
Tested-by: Rohit Makasana <rohitbm@chromium.org>
roschart/croschart_defaults.json
0f3c87b3b676dea16916c1a2b56ef997d0128e78 30-Nov-2011 Mike Truty <truty@chromium.org> Create new report template to compare platforms.

BUG=None
TEST=http://truty.i/croschart/server/chartplatforms?platform=x86-alex-r16+x86-alex-r15,ALEX&platform=x86-zgb-r16+x86-zgb-r15,ACER_ZGB&from_build=0.15.1011.118&to_build=R16-1193.65.0&testkey=power_Resume,seconds_system_suspend&testkey=power_Resume,seconds_system_resume

Change-Id: Ifda399c0e12eb186ce46908ff942b2600baac6c1
Reviewed-on: https://gerrit.chromium.org/gerrit/12230
Commit-Ready: Mike Truty <truty@chromium.org>
Reviewed-by: Mike Truty <truty@chromium.org>
Tested-by: Mike Truty <truty@chromium.org>
roschart/reportviews.py
roschart/templates/chartplatforms/plot_chartplatforms.html
roschart/urls.py
roschart/validators.py
roschart/views.py
efa4655a640a30a276dc69b7c558d68bbd5f797b 22-Nov-2011 Mike Truty <truty@chromium.org> Extend the desktop_ fix to the releasereport.

Support for desktop_ added in addition to netbook_.

BUG=None
TEST=http://truty.i/croschart/server/releasereport?board=x86-alex-r15&board=x86-alex-r16&system=ALEX&from_build=0.15.1011.118&to_build=R16-1193.65.0

Change-Id: Ibcff50b14ed8ef7274e2d1a8407b44ce66169b20
Reviewed-on: https://gerrit.chromium.org/gerrit/12032
Reviewed-by: Danielle Drew <ddrew@chromium.org>
Tested-by: Danielle Drew <ddrew@chromium.org>
Commit-Ready: Danielle Drew <ddrew@chromium.org>
roschart/releasereport/models.py
80466afa17252c877d37d2b08acfb6a91ebb3e45 15-Nov-2011 Rohit Makasana <rohitbm@chromium.org> Adding Youtube CPU performance test graph to default autotest perf report.

Change-Id: Ia27e00a630cfbe4eb6c179384a0887648accd9cf
BUG=None
TEST=None
Reviewed-on: https://gerrit.chromium.org/gerrit/11713
Commit-Ready: Rohit Makasana <rohitbm@chromium.org>
Tested-by: Rohit Makasana <rohitbm@chromium.org>
Reviewed-by: Rohit Makasana <rohitbm@chromium.org>
roschart/croschart_defaults.json
b33e17b088e69b858b000c8f2876baf5b2ed3979 15-Nov-2011 Mike Truty <truty@chromium.org> Add perf charts for Stumpy.

Adjust for stumpy platform prefix.

BUG=None
TEST=http://truty.i/croschart/server/chartreport?board=stumpy-r17&system=STUMPY&interval=2,week

Change-Id: I04f599cd764caf04ead91a61b6540ba7340e352b
Reviewed-on: https://gerrit.chromium.org/gerrit/11644
Reviewed-by: Sameer Nanda <snanda@chromium.org>
Tested-by: Mike Truty <truty@chromium.org>
Commit-Ready: Mike Truty <truty@chromium.org>
roschart/chartmodels.py
28c3ae7390210742d2ea5827ff7ee1e4a0695e06 15-Nov-2011 Dennis Jeffrey <dennisjeffrey@chromium.org> Enable graphs for pyauto-based SunSpider/Dromaeo tests on default perf report.

BUG=None
TEST=http://cros-reports/croschart/server/chart?interval=2%2Cweek&board=x86-mario-r17&testkey=desktopui_PyAutoPerfTests%2Cms_SunSpider-total&system=MARIO_MP
http://cros-reports/croschart/server/chart?interval=2%2Cweek&board=x86-mario-r17&testkey=desktopui_PyAutoPerfTests%2Cms_SunSpider-3d,ms_SunSpider-access,ms_SunSpider-bitops,ms_SunSpider-controlflow,ms_SunSpider-crypto,ms_SunSpider-date,ms_SunSpider-math,ms_SunSpider-regexp,ms_SunSpider-string&system=MARIO_MP

http://cros-reports/croschart/server/chart?interval=2%2Cweek&board=x86-mario-r17&testkey=desktopui_PyAutoPerfTests%2CrunsPerSec_Dromaeo-total&system=MARIO_MP
http://cros-reports/croschart/server/chart?interval=2%2Cweek&board=x86-mario-r17&testkey=desktopui_PyAutoPerfTests%2CrunsPerSec_Dromaeo-RegularExpr,runsPerSec_Dromaeo-Arrays,runsPerSec_Dromaeo-Base64Encod,runsPerSec_Dromaeo-CodeEvaluat,runsPerSec_Dromaeo-Rotating3DC,runsPerSec_Dromaeo-Strings&system=MARIO_MP

Change-Id: If729f31932f76c7c755b161e0aa49aa0984071ca
Reviewed-on: https://gerrit.chromium.org/gerrit/11709
Reviewed-by: Mike Truty <truty@chromium.org>
Tested-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
Commit-Ready: Dennis Jeffrey <dennisjeffrey@chromium.org>
roschart/croschart_defaults.json
50aeaf2ddde38e66883f5fe475befaa5c35ab35f 27-Oct-2011 Dennis Jeffrey <dennisjeffrey@chromium.org> Enabling perf graph for individual v8 benchmark results from desktopui_PyAutoPerfTests.

BUG=chromium-os:18457
TEST=Sample graph: http://cros-reports/croschart/server/chart?interval=2%2Cweek&board=x86-alex-r17&testkey=desktopui_PyAutoPerfTests%2Cscore_V8Benchmark-Richards,score_V8Benchmark-DeltaBlue,score_V8Benchmark-Crypto,score_V8Benchmark-RayTrace,score_V8Benchmark-EarleyBoyer,score_V8Benchmark-RegExp,score_V8Benchmark-Splay&system=ALEX

Change-Id: I62a41c164d2fefd0c974154549271aa4ff5be298
Reviewed-on: https://gerrit.chromium.org/gerrit/10806
Reviewed-by: Mike Truty <truty@chromium.org>
Tested-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
Commit-Ready: Dennis Jeffrey <dennisjeffrey@chromium.org>
roschart/croschart_defaults.json
d156307a10cad9369989c7a42ef514006ec2b080 21-Oct-2011 Mike Truty <truty@chromium.org> Accommodate new version number convention.

BUG=chromium-os:21938
TEST=http://truty.i/croschart/server/releasereport?board=x86-alex-r16&system=Alex&from_build=R16-1131.0.0&to_build=R16-1169.0.0

Change-Id: Ie547691ce3e0e28dcc1d7e976d4f008389ed3572
Reviewed-on: http://gerrit.chromium.org/gerrit/10464
Reviewed-by: Danielle Drew <ddrew@chromium.org>
Tested-by: Danielle Drew <ddrew@chromium.org>
Commit-Ready: Danielle Drew <ddrew@chromium.org>
Tested-by: Mike Truty <truty@chromium.org>
Commit-Ready: Mike Truty <truty@chromium.org>
roschart/releasereport/models.py
556438fc1cd31874d1347b4fea2e18ae7c81a395 19-Oct-2011 Dennis Jeffrey <dennisjeffrey@chromium.org> Add pyauto-based flash performance test graphs to autotest default perf report.

Sample perf charts related to this CL:
http://cros-reports/croschart/server/chart?testkey=desktopui_PyAutoPerfTests,FPS_FlashGaming,FPS_FlashText&interval=2,week&board=x86-mario-r16&system=MARIO_MP
http://cros-reports/croschart/server/chart?testkey=desktopui_PyAutoPerfTests,MFLOPS_ScimarkGui-lu_factoriza,MFLOPS_ScimarkGui-fast_fourier,MFLOPS_ScimarkGui-jacobi_sor-M,MFLOPS_ScimarkGui-monte_carlo_,MFLOPS_ScimarkGui-sparse_matri&interval=2,week&board=x86-mario-r16&system=MARIO_MP
http://cros-reports/croschart/server/chart?testkey=desktopui_PyAutoPerfTests,MB_ScimarkGui-lu_factorization,MB_ScimarkGui-fast_fourier_tra,MB_ScimarkGui-jacobi_sor-Mem,MB_ScimarkGui-monte_carlo_inte,MB_ScimarkGui-sparse_matrix_mu&interval=2,week&board=x86-mario-r16&system=MARIO_MP

BUG=None
TEST=None

Change-Id: Ica4b3288fe2b86f06a7decdbd99fbf8637dfa404
Reviewed-on: http://gerrit.chromium.org/gerrit/10280
Reviewed-by: Mike Truty <truty@chromium.org>
Tested-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
Commit-Ready: Dennis Jeffrey <dennisjeffrey@chromium.org>
roschart/croschart_defaults.json
98f8ee7db9a64caba9fcc7e04930a6d268d12bd1 19-Oct-2011 Rohit Makasana <rohitbm@chromium.org> Adding Youtube performance test graph to default autotest perf report.

BUG=None
TEST=None

Change-Id: I185f9b1e6fc68d758f669681a6fb2ea98f3bb4b5
Reviewed-on: http://gerrit.chromium.org/gerrit/10276
Reviewed-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
Reviewed-by: Mike Truty <truty@chromium.org>
Tested-by: Rohit Makasana <rohitbm@chromium.org>
Commit-Ready: Rohit Makasana <rohitbm@chromium.org>
Reviewed-by: Rohit Makasana <rohitbm@chromium.org>
roschart/croschart_defaults.json
f3b5487de29a26093150cb7793b7f7c1a1c12cb7 04-Oct-2011 Mike Truty <truty@chromium.org> Handles both old (0.w.x.y) and new (Rw-x.y.z).

Coordinated change with crostools/version_extract.py as well.

BUG=chromium-os:21201

TEST=http://truty.i/croschart/server/chart?testkey=platform_BootPerfServer,seconds_kernel_to_startup,seconds_kernel_to_startup_done,seconds_kernel_to_x_started,seconds_kernel_to_chrome_exec,seconds_kernel_to_chrome_main,seconds_kernel_to_login,seconds_kernel_to_network&interval=2,week&board=x86-alex-r16&system=ALEX&updatecache=true

Change-Id: I546f86b51b21ad82afae713ba379b2e0dc89a657
Reviewed-on: http://gerrit.chromium.org/gerrit/8722
Tested-by: Mike Truty <truty@chromium.org>
Reviewed-by: Dale Curtis <dalecurtis@chromium.org>
Commit-Ready: Mike Truty <truty@chromium.org>
roschart/chartutils.py
roschart/validators.py
97cef153420a4d956b94af1497588e5ecfa10d2b 04-Oct-2011 Dennis Jeffrey <dennisjeffrey@chromium.org> Adding scrolling test results from desktopui_PyAutoPerfTests to default autotest perf report.

BUG=chromium-os:18185
TEST=None

Change-Id: Ifd55cd3b5961e58d11a9601b715ca05af18e42b2
Reviewed-on: http://gerrit.chromium.org/gerrit/8736
Reviewed-by: Mike Truty <truty@chromium.org>
Reviewed-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
Tested-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
Commit-Ready: Dennis Jeffrey <dennisjeffrey@chromium.org>
roschart/croschart_defaults.json
c108704e3fda06b5e71568ae46ee68a59fc3011e 26-Sep-2011 Mike Truty <truty@chromium.org> Improve triage of perf charts by enabling job view in addition to keyvals.

Sometimes triage requires investigation into data values while sometimes
it requires investigation into job data. Since a switch to store results
in gs it's hard to get back to see job data. This shows the keyvals as a
part of the job tree making both keyvals and job data easy to find.

BUG=chromium-os:7677

TEST=tested with new perfchart views

Change-Id: Ice721939bc3931dd4e4729b3fce5ef6b78023399
Reviewed-on: http://gerrit.chromium.org/gerrit/8289
Reviewed-by: Todd Broch <tbroch@chromium.org>
Tested-by: Mike Truty <truty@chromium.org>
Commit-Ready: Mike Truty <truty@chromium.org>
roschart/templates/perfchart/plot_chart.html
8adf78936c915df6fd1edb6c592f40a7ed8350a5 09-Sep-2011 Dale Curtis <dalecurtis@chromium.org> Autotest upstream merge.

Merged from d9d64b855363d214996b187380532d4cc9991d29 to
7bad38846fe9c74e42018131ce85aec2b5e6c7a9

BUG=none
TEST=emerge autotest, run bvt, smoke.

Change-Id: Ibe6462198e84e0d41fa160af086283cd712da4a6
Reviewed-on: http://gerrit.chromium.org/gerrit/7440
Tested-by: Dale Curtis <dalecurtis@chromium.org>
Reviewed-by: Dale Curtis <dalecurtis@chromium.org>
b/backends/afe_sqlite/validation.py
ettings.py
hared/resource_lib.py
ko/csv_encoder.py
62ddbd482b11e0582752211733a50422c283b9d5 13-Sep-2011 Dennis Jeffrey <dennisjeffrey@chromium.org> Adding more desktopui_PyAutoPerfTest results to the default autotest perf report.

BUG=chromium-os:18185
TEST=None

Change-Id: I1de108ac7cc7ee44abe84cc61bf17524574d2fc0
Reviewed-on: http://gerrit.chromium.org/gerrit/7579
Reviewed-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
Tested-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
Reviewed-by: Mike Truty <truty@chromium.org>
Tested-by: Mike Truty <truty@chromium.org>
roschart/croschart_defaults.json
f5814ba44d6060086f16ad81d4281c7ddd40189b 30-Aug-2011 Mike Truty <truty@chromium.org> Fix remaining deprecation warning from Django 1.3 upgrade.

Message that no longer emits:
django.template.loaders.filesystem.load_template_source' is deprecated; use
'django.template.loaders.filesystem.Loader

TEST=ran local dash gen that uses templates and no longer see warning message.

Change-Id: I32d09239554243f5bdde5f70cc17dcd459e94ea2
Reviewed-on: http://gerrit.chromium.org/gerrit/6876
Reviewed-by: Dale Curtis <dalecurtis@chromium.org>
Tested-by: Mike Truty <truty@chromium.org>
ettings.py
3536b98cf0f552eb9c4d8d6dc0403423166da9f5 29-Aug-2011 Mike Truty <truty@chromium.org> Fix django 1.3 DeprecationWarning for DATABASES.

Django 1.3 which we migrated to, modified the settings structure so that
the database settings are no longer a set of constants. They are replaced
with a dictionary of dictionaries where the 'default' dictionary contains
the desired settings. I've switched to that format to avoid the warning. Not
sure when those constants will actually stop working.

This is the actual message that no longer shows:
/usr/local/autotest/site-packages/django/db/__init__.py:19: DeprecationWarning:
settings.DATABASE_* is deprecated; use settings.DATABASES instead.

TEST=ran dashboard generation on local autotest

Change-Id: I647f2c353b67a0ec820895c7973bfffab84f3f87
Reviewed-on: http://gerrit.chromium.org/gerrit/6835
Reviewed-by: Lucas Meneghel Rodrigues <lookkas@gmail.com>
Tested-by: Lucas Meneghel Rodrigues <lookkas@gmail.com>
Reviewed-by: Dale Curtis <dalecurtis@chromium.org>
Tested-by: Mike Truty <truty@chromium.org>
fe/readonly_connection.py
fe/test.py
ettings.py
etup_test_environment.py
f22cec4b09dd5e74b8f4f6547049fe2433da367f 25-Aug-2011 Dennis Jeffrey <dennisjeffrey@chromium.org> Add desktopui_PyAutoPerfTests results to the default autotest perf report.

This change creates three new plots in the default performance report
on the autotest dashboard, representing results from the pyauto tests in
desktopui_PyAutoPerfTests.

BUG=chromium-os:18185
TEST=None

Change-Id: I0f27a38edb868aba5ed415d72e54d6843d522052
Reviewed-on: http://gerrit.chromium.org/gerrit/6618
Reviewed-by: Mike Truty <truty@chromium.org>
Tested-by: Dennis Jeffrey <dennisjeffrey@chromium.org>
roschart/croschart_defaults.json
32c760f98b4ffc4f807e12367fe99e6cbdd82d2b 20-Aug-2011 Mike Truty <truty@chromium.org> Fixup relocated templates from last refactoring.

Not noticed until deployment cleanup of stale files.

TEST=http://truty.i/croschart/server/chartreport?board=x86-mario-r15&system=MARIO_MP&from_date=2011-08-01&to_date=2011-08-20&width=320&height=240

Change-Id: I281380e2b8333373a558566e253d3257fa58f7be
Reviewed-on: http://gerrit.chromium.org/gerrit/6377
Reviewed-by: Dale Curtis <dalecurtis@chromium.org>
Tested-by: Mike Truty <truty@chromium.org>
roschart/templates/perfchart/plot_chart.html
roschart/templates/perfchart/plot_chartdiff.html
e79f566258f959c0b61d743323f62daac1432c60 12-Aug-2011 Mike Truty <truty@chromium.org> Split model and templates code into report specific parts for easier maint.

Building reports is complicated by the fact that chartmodels.py is huge
and confusing. This clarifies what needs to get done to add/remove
a report. This is a precursor to a techtalk on writing/updating reports.

TEST=http://truty.i/croschart/server/labtestreport?interval=5,week&updatecache=true
TEST=http://truty.i/croschart/server/testreport?interval=1,week

Change-Id: Id031d2e93448ffbc3d14763550e6a895a6af1f44
Reviewed-on: http://gerrit.chromium.org/gerrit/5827
Reviewed-by: Mike Truty <truty@chromium.org>
Tested-by: Mike Truty <truty@chromium.org>
roschart/chartmodels.py
roschart/chartutils.py
roschart/crosrelease_defaults.json
roschart/deploy_new_croschart.sh
roschart/labtest/__init__.py
roschart/labtest/models.py
roschart/perfchart/__init__.py
roschart/perfchart/models.py
roschart/releasereport/__init__.py
roschart/releasereport/crosrelease_defaults.json
roschart/releasereport/models.py
roschart/templates/base.html
roschart/templates/chartreport/plot_chartreport.html
roschart/templates/labtest/plot_labtestreport.html
roschart/templates/perfchart/plot_chart.html
roschart/templates/perfchart/plot_chart_base.html
roschart/templates/perfchart/plot_chartdiff.html
roschart/templates/plot_chart.html
roschart/templates/plot_chart_base.html
roschart/templates/plot_chartdiff.html
roschart/templates/plot_chartreport.html
roschart/templates/plot_labtestreport.html
roschart/templates/plot_releasereport.html
roschart/templates/plot_syntax.html
roschart/templates/plot_testreport.html
roschart/templates/plot_unavailable.html
roschart/templates/releasereport/plot_releasereport.html
roschart/templates/testreport/plot_testreport.html
roschart/testreport/__init__.py
roschart/testreport/models.py
roschart/views.py
ae942bb2b2111ae5eb03bf48b2df67ed44f3714c 06-Aug-2011 Mike Truty <truty@chromium.org> Adjust the last updated date to include the numeric day of the month.

TEST=http://truty.i/croschart/server/chartreport?board=x86-alex-r15&system=ALEX&interval=2,week

Change-Id: I4ba780d64a4b9b3d8d472c41bb10aa9b66c72e86
Reviewed-on: http://gerrit.chromium.org/gerrit/5441
Reviewed-by: Dale Curtis <dalecurtis@chromium.org>
Tested-by: Mike Truty <truty@chromium.org>
roschart/templates/plot_chartreport.html
456d3c115952bf1ae984770e226c5a50676b31c0 19-Jul-2011 Dale Curtis <dalecurtis@chromium.org> Upstream Autotest merge.

Merged from d9d64b855363d214996b187380532d4cc9991d29.

BUG=none
TEST=emerge autotest-tests, local server, run_remote_tests.

Change-Id: Id8cf1ef930bc0cd80347d77f2de65561be2a12a4
Reviewed-on: http://gerrit.chromium.org/gerrit/4664
Reviewed-by: Mike Truty <truty@chromium.org>
Reviewed-by: Scott Zawalski <scottz@chromium.org>
Tested-by: Dale Curtis <dalecurtis@chromium.org>
lient/src/autotest/afe/create/CreateJobViewPresenter.java
igrations/067_add_id_afe_acl_groups_hosts.py
igrations/068_add_id_afe_acl_groups_users.py
f1e47e36e05e20238df47324a68b3abd66718ff3 26-Jul-2011 Mike Truty <truty@chromium.org> Ignore invalid job_names and add lab_test user report.

An aggregate job_name was introduced that shows multiple build versions which
confuses the version# parsing. In addition, recently the job_name build hash
was abbreviated. Update the parsing code to accept the new hash and ignore the
multi-build job_name.

Introduces a graphical, per-user lab_test report to easily identify lab_test
users (or lab_test cron job creators).

TEST=http://truty.i/croschart/server/labtestreport?from_date=2011-01-01&to_date=2011-07-24
TEST=http://truty.i/croschart/server/releasereport?board=x86-mario-r14&system=MARIO_MP&from_build=0.14.774.0&to_build=0.14.804.0
TEST=http://cautotest/results/dashboard/netbook_MARIO_MP/x86-mario-r14/perf_index.html

Change-Id: I94398515a624d9e0f249f5299659f455678c715f
Reviewed-on: http://gerrit.chromium.org/gerrit/4689
Reviewed-by: Dale Curtis <dalecurtis@chromium.org>
Tested-by: Mike Truty <truty@chromium.org>
roschart/chartmodels.py
roschart/templates/plot_labtestreport.html
94cc5a822a6164657bfc66a2160d6777dfbe0297 20-Jul-2011 Mike Truty <truty@chromium.org> Create a kernel report detailing labtest usage.
Query for jobs in tko test results for jobs run through labtest.
Ordered by most recent to least recent.
Allows for internal= or from_date/to_date parameters.
Clicking on any row takes the user to that job in the Autotest result UI.

TEST=http://truty.i/croschart/server/labtestreport?interval=14,week
TEST=http://truty.i/croschart/server/labtestreport?from_date=2011-01-01&to_date=2011-07-18

Change-Id: Ibf68a93977a4bc230500f45d9052057d1fb00361
Reviewed-on: http://gerrit.chromium.org/gerrit/4372
Reviewed-by: Dale Curtis <dalecurtis@chromium.org>
Tested-by: Mike Truty <truty@chromium.org>
roschart/chartmodels.py
roschart/templates/plot_labtestreport.html
roschart/templates/plot_syntax.html
roschart/templates/plot_testreport.html
roschart/urls.py
roschart/views.py
2fb55a8fc47f2558c1a8c959111495c8590e2b18 19-Jul-2011 Mike Truty <truty@chromium.org> Add report to document tests.

This report shows what tests exist in the file system and what tests have been
run. The tests in the file system are discovered by the Autotest test_importer
and the tests that have been executed are found by there test results.

The final report serves 2 very useful purposes:
1. Shows frequency of tests actually running on each platform.
2. Shows tests not being run - to figure out where added coverage is available.

TEST=http://truty.i/croschart/server/testreport?interval=2,week

Change-Id: I5a5391e5e0c430fae728e830c7e5ef40d219277c
Reviewed-on: http://gerrit.chromium.org/gerrit/4281
Tested-by: Mike Truty <truty@chromium.org>
Reviewed-by: Mike Truty <truty@chromium.org>
roschart/chartmodels.py
roschart/templates/plot_syntax.html
roschart/templates/plot_testreport.html
roschart/urls.py
roschart/views.py
74a314b490ff542c4dd2ae4aa0d11c6394d92960 23-Jun-2011 Dale Curtis <dalecurtis@chromium.org> Upstream Autotest merge.

As titled, a merge up to 93fc426ca133e775eb495f34d138fc57d92fb55e.

- Removes a bunch of deprecated code.
- Moves several private utilities into the private repo.
- Couple changes ported upstream and resynced.

BUG=None
TEST=In progress... will spin up new Autotest server and use
run_remote_tests for bvt, regression, smoke.

Change-Id: Id3e2ad529bb7b05f148e5d98aea46bb9ea828200
Reviewed-on: http://gerrit.chromium.org/gerrit/3350
Tested-by: Dale Curtis <dalecurtis@chromium.org>
Reviewed-by: Dale Curtis <dalecurtis@chromium.org>
fe/admin.py
fe/model_logic.py
fe/readonly_connection.py
lient/TestPlannerClient-shell
lient/TestPlannerClient.launch
lient/src/autotest/TestPlannerClient.gwt.xml
lient/src/autotest/common/JsonRpcProxy.java
lient/src/autotest/planner/AutoprocessedTab.java
lient/src/autotest/planner/AutoprocessedTabDisplay.java
lient/src/autotest/planner/HistoryTab.java
lient/src/autotest/planner/HistoryTabDisplay.java
lient/src/autotest/planner/TestPlanSelector.java
lient/src/autotest/planner/TestPlanSelectorDisplay.java
lient/src/autotest/planner/TestPlannerClient.java
lient/src/autotest/planner/TestPlannerDisplay.java
lient/src/autotest/planner/TestPlannerPresenter.java
lient/src/autotest/planner/TestPlannerTab.java
lient/src/autotest/planner/TestPlannerTableDisplay.java
lient/src/autotest/planner/TestPlannerTableImpl.java
lient/src/autotest/planner/TestPlannerUtils.java
lient/src/autotest/planner/TestViewTab.java
lient/src/autotest/planner/TestViewTabDisplay.java
lient/src/autotest/planner/machine/MachineViewDisplay.java
lient/src/autotest/planner/machine/MachineViewPresenter.java
lient/src/autotest/planner/machine/MachineViewTab.java
lient/src/autotest/planner/machine/MachineViewTable.java
lient/src/autotest/planner/machine/MachineViewTableDisplay.java
lient/src/autotest/planner/machine/StatusSummary.java
lient/src/autotest/planner/overview/OverviewTab.java
lient/src/autotest/planner/overview/OverviewTabDisplay.java
lient/src/autotest/planner/overview/OverviewTabPresenter.java
lient/src/autotest/planner/overview/OverviewTable.java
lient/src/autotest/planner/overview/OverviewTableDisplay.java
lient/src/autotest/planner/resources/PlannerClientBundle.java
lient/src/autotest/planner/resources/close.png
lient/src/autotest/planner/test/TestViewDisplay.java
lient/src/autotest/planner/test/TestViewPresenter.java
lient/src/autotest/planner/test/TestViewTab.java
lient/src/autotest/planner/test/TestViewTable.java
lient/src/autotest/planner/triage/FailureTable.java
lient/src/autotest/planner/triage/FailureTableDisplay.java
lient/src/autotest/planner/triage/TriagePopup.java
lient/src/autotest/planner/triage/TriagePopupDisplay.java
lient/src/autotest/planner/triage/TriageViewDisplay.java
lient/src/autotest/planner/triage/TriageViewPresenter.java
lient/src/autotest/planner/triage/TriageViewTab.java
lient/src/autotest/public/TestPlannerClient.html
lient/src/autotest/public/header.png
b/__init__.py
b/backends/__init__.py
b/backends/afe/__init__.py
b/backends/afe/base.py
b/backends/afe/compiler.py
b/backends/afe/creation.py
b/backends/afe/introspection.py
b/backends/afe/validation.py
b/backends/afe_sqlite/__init__.py
b/backends/afe_sqlite/base.py
b/backends/afe_sqlite/compiler.py
b/backends/afe_sqlite/creation.py
b/backends/afe_sqlite/introspection.py
b/backends/afe_sqlite/validation.py
igrations/065_add_id_afe_hosts_labels.py
igrations/066_drop_planner.py
lanner/__init__.py
lanner/common.py
lanner/control_file.py
lanner/control_file_unittest.py
lanner/execution_engine.py
lanner/execution_engine_control.srv
lanner/execution_engine_unittest.py
lanner/failure_actions.py
lanner/model_attributes.py
lanner/models.py
lanner/models_test.py
lanner/planner_test_utils.py
lanner/rpc_interface.py
lanner/rpc_interface_unittest.py
lanner/rpc_utils.py
lanner/rpc_utils_unittest.py
lanner/set_atomic_group_control.srv
lanner/support.py
lanner/urls.py
lanner/views.py
ettings.py
etup_test_environment.py
ko/models.py
rls.py
bab6028694230dd763b27c85ad5f79ffc2c38af9 17-Jun-2011 Mike Truty <truty@google.com> Improve firmare metrics.

BUG=chromium-os:14925
TEST=http://truty.i/croschart/server/chartreport?board=x86-alex-r14&system=ALEX&interval=2,week#platform_BootPerfServer&updatecache=true
TEST=http://truty.i/croschart/server/chartreport?board=x86-alex-r14&system=ALEX&interval=2,week&testkey=platform_BootPerfServer,seconds_power_on_to_lf_start,seconds_power_on_to_lf_end,seconds_power_on_to_lk_start,seconds_power_on_to_lk_end,seconds_power_on_to_kernel

Change-Id: I3f87bdc1c25eeeacbb9bb980256b80a5ee6ab8bb
Reviewed-on: http://gerrit.chromium.org/gerrit/2851
Reviewed-by: Richard Barnette <jrbarnette@chromium.org>
Tested-by: Mike Truty <truty@chromium.org>
roschart/croschart_defaults.json
7d4e8b554cddddeb4d7d0500be9a5221eef8a9dc 13-Jun-2011 Mike Truty <truty@google.com> Abridge chart legends and improve chrome version readability.

BUG=CROSBUG:14738
BUG=CROSBUG:14746
TEST=http://truty.i/croschart/server/chartreport?board=x86-mario-r13&system=MARIO_MP&from_date=2011-05-01&to_date=2011-06-06&testkey=graphics_GLBench,mpixels_sec_fill_tex_bilinear&testkey=graphics_GLBench,mpixels_sec_yuv_shader_3&testkey=graphics_GLBench,mtexel_sec_texture_update_texi&testkey=graphics_GLBench,1280x768_fps_compositing&testkey=graphics_GLBench,mvtx_sec_attribute_fetch_shade

Change-Id: I9882a98f4718e9be5b3a01b418ac6cb4bcfb4f07
Reviewed-on: http://gerrit.chromium.org/gerrit/2524
Reviewed-by: Dale Curtis <dalecurtis@chromium.org>
Tested-by: Mike Truty <truty@chromium.org>
Reviewed-by: Mike Truty <truty@chromium.org>
roschart/chartmodels.py
roschart/chartutils.py
roschart/reportviews.py
roschart/templates/plot_chart.html
roschart/templates/plot_chart_base.html
roschart/templates/plot_chartdiff.html
72aea8d708cddcb8072e9efeb6de116dca2ec33f 07-Jun-2011 Mike Truty <truty@google.com> Add Chrome version and title/diff links to reports.

BUG=crosbug:14746
TEST=http://truty.i/croschart/server/chartreport?board=x86-mario-r12&system=MARIO_MP&from_build=0.12.394.0&to_build=0.12.432.0&testkey=desktopui_PageCyclerTests,PageCyclerTest.Alexa_usFile&testkey=platform_BootPerfServer,seconds_power_on_to_login&testkey=platform_BootPerfServer,seconds_kernel_to_startup&testkey=platform_BootPerfServer,seconds_kernel_to_x_started&testkey=desktopui_V8Bench,score_total&updatecache=true

Change-Id: I05e5deb297ae288a04d7f527f301162159c81469
Reviewed-on: http://gerrit.chromium.org/gerrit/2235
Reviewed-by: Dale Curtis <dalecurtis@chromium.org>
Tested-by: Mike Truty <truty@chromium.org>
roschart/chartmodels.py
roschart/chartviews.py
roschart/templates/plot_chart.html
roschart/templates/plot_chart_base.html
roschart/templates/plot_chartdiff.html
roschart/templates/plot_chartreport.html
roschart/templates/plot_syntax.html
roschart/validators.py
2421ca8394a79862129d9ed6bd3288e478dd4097 03-Jun-2011 Mike Truty <truty@google.com> Tighten release report back to two builds.

BUG=crosbug:14747
TEST=http://truty.i/croschart/server/releasereport?board=x86-mario-r12&system=MARIO_MP&from_build=0.12.433.57&to_build=0.12.433.90&updatecache=true

Change-Id: I701383f2f15936ae3528f0c49ec7a3d082a6efdc
Reviewed-on: http://gerrit.chromium.org/gerrit/2036
Reviewed-by: Dale Curtis <dalecurtis@chromium.org>
Tested-by: Mike Truty <truty@chromium.org>
roschart/chartmodels.py
roschart/validators.py
97cda94c7d2b3f878cd7609e6bd7eaeb7fde6a54 29-May-2011 Mike Truty <truty@google.com> Adjust release report metrics for kernel.

TEST=http://truty.i/croschart/server/releasereport?board=x86-mario-r12&system=MARIO_MP&from_build=0.12.394.0&to_build=0.12.432.0&updatecache=true

Change-Id: Ib7502aa001926e2e2f7ff05bbe9ffe8d277217ef
Reviewed-on: http://gerrit.chromium.org/gerrit/1792
Reviewed-by: <oritm@google.com>
Tested-by: <oritm@google.com>
Reviewed-by: Richard Barnette <jrbarnette@chromium.org>
Tested-by: Mike Truty <truty@chromium.org>
roschart/crosrelease_defaults.json
b9d5e18168737b47876ce567fd70785f8e5209f3 25-May-2011 Mike Truty <truty@google.com> Add support for multi-line charts and reports.

BUG=crosbug:14745
TEST=https://spreadsheets.google.com/a/google.com/spreadsheet/ccc?key=0Ahn3gnzB-A8mdFVMMnNjNW04LWp0RmFWSzdmZU1xZUE&hl=en_US#gid=0

Change-Id: I4123f3b55d18b2d86376f6f31782bb43261f9699
Reviewed-on: http://gerrit.chromium.org/gerrit/1573
Reviewed-by: Eric Li <ericli@chromium.org>
Tested-by: Mike Truty <truty@chromium.org>
roschart/chartmodels.py
roschart/chartutils.py
roschart/croschart_defaults.json
roschart/reportviews.py
roschart/templates/plot_chart_base.html
roschart/templates/plot_chartreport.html
roschart/validators.py
roschart/views.py
b31e2a4597a3902476a9180d497fe0827f5a4877 24-May-2011 Eric Li <ericli@chromium.org> Fix autotest web frontend readonly db connection.

Now it could really talk to a readonly database.

Change-Id: Ib45b01fe55ea04bd10665a087b0b7e00a806db1e
Reviewed-on: http://gerrit.chromium.org/gerrit/1459
Reviewed-by: Mike Truty <truty@chromium.org>
Reviewed-by: Eric Li <ericli@chromium.org>
Tested-by: Eric Li <ericli@chromium.org>
fe/readonly_connection.py
8858e5e79da197d0c542aede641fdca2e8c35093 13-May-2011 Mike Truty <truty@google.com> Add release report with image status indicators.

BUG=crosbug:14747
TEST=https://spreadsheets.google.com/a/google.com/ccc?key=0Ahn3gnzB-A8mdFVMMnNjNW04LWp0RmFWSzdmZU1xZUE&hl=en&pli=1#gid=0

Change-Id: I40e845cfad3d714c6a57a671eb1cfb88a646d5c1
Reviewed-on: http://gerrit.chromium.org/gerrit/842
Reviewed-by: Eric Li <ericli@chromium.org>
Tested-by: Mike Truty <truty@chromium.org>
roschart/DEPLOYMENT_README
roschart/chartmodels.py
roschart/crosrelease_defaults.json
roschart/templates/plot_releasereport.html
roschart/urls.py
roschart/views.py
0a99391317b5caca3d373d0bba931bcb62302566 17-May-2011 Eric Li <ericli@chromium.org> Merge autotest upstream from @5337 ~ @5360

Change-Id: Iad59314186db370f9219c3fb294b851f4e214e2e
Reviewed-on: http://gerrit.chromium.org/gerrit/1045
Reviewed-by: Eric Li <ericli@chromium.org>
Tested-by: Eric Li <ericli@chromium.org>
fe/models.py
fe/resources.py
hared/rest_client.py
6c0b70ba5fa83b57fc0bc2f6cf347b546c1bc9e8 17-May-2011 Eric Li <ericli@chromium.org> Revert "Merge autotest upstream from @5337 ~ @5360"

This reverts commit ce1b0620fddfb6efa33cc9441d9c91ce06b30de3.

Change-Id: Iab10c235ce76bfd45e7a2036ea2413729e34ae15
Reviewed-on: http://gerrit.chromium.org/gerrit/1033
Reviewed-by: Eric Li <ericli@chromium.org>
Tested-by: Eric Li <ericli@chromium.org>
fe/models.py
fe/resources.py
hared/rest_client.py
ce1b0620fddfb6efa33cc9441d9c91ce06b30de3 16-May-2011 Eric Li <ericli@chromium.org> Merge autotest upstream from @5337 ~ @5360

Change-Id: I2052380f1fb1ff952977914cb6f516a31c076aba
Reviewed-on: http://gerrit.chromium.org/gerrit/1031
Tested-by: Eric Li <ericli@chromium.org>
Reviewed-by: Eric Li <ericli@chromium.org>
fe/models.py
fe/resources.py
hared/rest_client.py
ff43b15c135231c415060a72f1f9ab98b6f23ae6 10-May-2011 Mike Truty <truty@google.com> One more unfolded refactoring.

BUG=crosbug:14044
TEST=https://spreadsheets.google.com/a/google.com/ccc?key=0Ahn3gnzB-A8mdFVMMnNjNW04LWp0RmFWSzdmZU1xZUE&hl=en&ndplr=1#gid=0

Change-Id: I5a1a0f25f5a3a6a217cf93d61a1be2744532ab20
Reviewed-on: http://gerrit.chromium.org/gerrit/635
Reviewed-by: Eric Li(李咏竹) <ericli@chromium.org>
Tested-by: Eric Li(李咏竹) <ericli@chromium.org>
Tested-by: Mike Truty <truty@google.com>
roschart/DEPLOYMENT_README
roschart/charterrors.py
roschart/chartmodels.py
roschart/chartviews.py
roschart/models.py
roschart/reportviews.py
roschart/templates/charts.html
roschart/templates/plot_chart.html
roschart/templates/plot_chart_base.html
roschart/templates/plot_chartdiff.html
roschart/templates/plot_chartreport.html
roschart/templates/plot_syntax.html
roschart/templates/plot_unavailable.html
roschart/urls.py
roschart/validators.py
roschart/views.py
d656d56473f50b9c1a9f5e2b2f5a9472181ee342 20-Apr-2011 Eric Li <ericli@chromium.org> Merge remote branch 'autotest-upstream/master' into autotest-merge

Merged to autotest upstream @5317.

BUG=none
TEST=
1. utils/unittest_suite.py
2. utils/packager.py --all upload -r /tmp/pkgs
3. utils/compile_gwt_clients.py

enter_chroot
emerge-x86-generic autotest-*
emerge-arm-generic autotest-*

run_remote_test suite_Smoke
run_remote_test bvt

Review URL: http://codereview.chromium.org/6883035

Change-Id: I03240d8652ffdfde4d6c8fbf474e2fad7f3cda10
lient/src/autotest/afe/create/CreateJobViewPresenter.java
lient/src/autotest/public/AfeClient.html
ddd44b22416bb1285c8bdf25267ba346d84da3f5 15-Apr-2011 Mike Truty <truty@google.com> Integrate dynamic charts into autotest frontend.

BUG=chromium-os:14044
TEST=http://truty.i/croschart/server/x86-mario-r12/MARIO_MP/0.12.306.0/0.12.389.0/platform_BootPerfServer/seconds_kernel_to_startup/1280/1024/

Review URL: http://codereview.chromium.org/6821082

Change-Id: Ide1bac507850a45a9a34ef9459a6b148e52e3305
roschart/__init__.py
roschart/common.py
roschart/croschart_defaults.json
roschart/models.py
roschart/templates/base.html
roschart/templates/charts.html
roschart/templates/plot_chart.html
roschart/templates/plot_unavailable.html
roschart/urls.py
roschart/views.py
ettings.py
rls.py
dd8a3833bf770dbdbd4544586656d9958774299b 19-Mar-2011 Eric Li <ericli@chromium.org> Add dashboard link from autotest afe ui.

Change-Id: I74dd11202e4209d38010a389a0afe5aa3fa67a90

BUG=
TEST=

Review URL: http://codereview.chromium.org/6688042
lient/src/autotest/public/AfeClient.html
8a12e800f05f6862fc39c6a45c9c57d9051b795e 17-Feb-2011 Eric Li <ericli@chromium.org> Merge remote branch 'cros/upstream' into master.

Merged to upstream autotest @5216~@5235.

Most of the stuff are from Olof's arm cross compiling and some misc changes from Dale Curtis.

As usual,the entire change list description is too big to enlist here. Please refer to upstream (http://autotest.kernel.org/browser) for more details.

BUG=none
TEST=1. emerge-arm-generic autotest autotest-deps autotest-tests
2. emerge-x86-generic autotest autotest-deps autotest-tests
3. run bvt from both src/third_party and inside emerged
chroot/..usr/local/autotest.
Seems fine. No regression found.

Review URL: http://codereview.chromium.org/6539001

Change-Id: I613929ec9c29849db5110ae8234bc68a80f4d2d5
fe/frontend_test_utils.py
fe/resources_test.py
rontend_unittest.py
d23bc19b7d23b3bde3d5bc43447445ca118bf786 09-Feb-2011 Eric Li <ericli@chromium.org> Add --image support for autotest web frontend.

An easy extension based on Paul's dean recent work.

Change-Id: I1117f779297b6f12a8abd3688718a0480548ccf2

BUG=none
TEST=Tested with the frontend web gui.
1. Create Job w/o image update url.
2. View Job details w/o image update url.
3. Clone Job w/o image update url.

Verified job with image parameter was properly saved in db.

What else I missed?

Review URL: http://codereview.chromium.org/6460003
fe/rpc_interface.py
lient/src/autotest/afe/JobDetailView.java
lient/src/autotest/afe/create/CreateJobViewDisplay.java
lient/src/autotest/afe/create/CreateJobViewPresenter.java
lient/src/autotest/public/AfeClient.html
861b2d54aec24228cdb3895dbc40062cb40cb2ad 04-Feb-2011 Eric Li <ericli@chromium.org> Merge remote branch 'cros/upstream' into master

Merged to upstream autotest @4749~@5215.

The entire change list description is too big to enlist here. Please refer to upstream (http://autotest.kernel.org/browser) for more details.

BUG=
TEST=emerged both x86 and arm build.
Tested emerged x86 build bvt against a chromeos device.

Review URL: http://codereview.chromium.org/6246035

Change-Id: I8455f2135c87c321c6efc232e2869dc8f675395e
fe/resources_test.py
lient/gwt_dir
lient/src/autotest/EmbeddedTkoClient.gwt.xml
igrations/062_drone_sets_unique.py
lanner/control_file_unittest.py
lanner/execution_engine_unittest.py
lanner/rpc_interface_unittest.py
lanner/rpc_utils_unittest.py
ko/common.py
ko/resources_test.py
ko/rpc_interface_unittest.py
5a8c6ad7e67d86090ed6c4ef216b82e45c0f9283 01-Feb-2011 Paul Pendlebury <pauldean@chromium.org> Add support for an --image flag to atest.

This enables creating jobs from the CLI that use autoserv to install a new OS image before running the requested test.

This was not added to the database like the reboot before/reboot after flags to maintain backward compatibility and avoid any changes to the database schema.

Also it is not working as a pure parameterized job as the current implementation is not 100% complete and would require more work to finish. And since mixed jobs are not allowed it would also mean moving existing control file jobs to parameterized jobs.

So the implementation of adding a parameterized id to control jobs and using a known test to hold the OS image path is the most straight forward of the options.

Change-Id: I77cdda0c50c222a4c594da2626a71fa55f5957cb

BUG=chromium-os:11486
TEST=Manual testing using atest cli to create jobs with --image parameters and verifying the value is passed to autoserv.

Review URL: http://codereview.chromium.org/6181003
fe/model_logic.py
fe/models.py
fe/rpc_interface.py
bdaab795cffa33f9a37995bb283047cb03699272 07-Jan-2011 Kenneth Waters <kwaters@chromium.org> Revert "Merge remote branch 'cros/upstream' into autotest-rebase"

This reverts commit 7edb30498d75a29a3287fe07070f2b51a116c5d4.

TEST=Built for tegra2_seaboard
BUG=10756

Review URL: http://codereview.chromium.org/6124004

Change-Id: Ifcd1a2279f5a62b99552a8f961c1430889a3a928
fe/resources_test.py
lient/gwt_dir
lient/src/autotest/EmbeddedTkoClient.gwt.xml
igrations/062_drone_sets_unique.py
lanner/control_file_unittest.py
lanner/execution_engine_unittest.py
lanner/rpc_interface_unittest.py
lanner/rpc_utils_unittest.py
ko/resources_test.py
ko/rpc_interface_unittest.py
7edb30498d75a29a3287fe07070f2b51a116c5d4 07-Jan-2011 Eric Li <ericli@chromium.org> Merge remote branch 'cros/upstream' into autotest-rebase

Merged to upstream trunk@5066, from trunk@4749.

There is no way I could enlist each individual CL from the upstream here since it will blow up the changelist description field.

BUG=
TEST=
Had patched this CL into a fresh cut client to avoid any side effect.
run_remote_test bvt from both emerged location and third_party/autotest/file.

Both test passed!

We should also keep any eye on this to see how it gets propagated into cautotest server.
TBR=dalecurtis

Change-Id: I72f2bc7a9de530178484aea1bfb5ace68bcad029
fe/resources_test.py
lient/gwt_dir
lient/src/autotest/EmbeddedTkoClient.gwt.xml
igrations/062_drone_sets_unique.py
lanner/control_file_unittest.py
lanner/execution_engine_unittest.py
lanner/rpc_interface_unittest.py
lanner/rpc_utils_unittest.py
ko/resources_test.py
ko/rpc_interface_unittest.py
e0493a4af57c1a73376a7bafaed542c01f588196 15-Nov-2010 Eric Li <ericli@chromium.org> Merge remote branch 'cros/upstream' into tempbranch

BUG=
TEST=

Review URL: http://codereview.chromium.org/4823005

Change-Id: I5d56f1c10d0fce7f9d7dc3ad727ea52dcb9b2d6c
fe/rpc_utils.py
igrations/064_add_jobs_and_tests_time_indices.py
6f27d4f22a1ba5063968b8c322fa0845f3279ade 29-Sep-2010 Eric Li <ericli@chromium.org> Merge remote branch 'cros/upstream' into tempbranch3

Merge to trunk@4817

BUG=
TEST=

Review URL: http://codereview.chromium.org/3554003

Change-Id: I83376bc7d28104ec2678e157eadbe7df7c05c0e0
lient/src/autotest/afe/AfeClient.java
lient/src/autotest/afe/AfeUtils.java
lient/src/autotest/afe/CheckBoxPanel.java
lient/src/autotest/afe/CheckBoxPanelDisplay.java
lient/src/autotest/afe/ClassFactory.java
lient/src/autotest/afe/ControlTypeSelect.java
lient/src/autotest/afe/ControlTypeSelectDisplay.java
lient/src/autotest/afe/CreateJobView.java
lient/src/autotest/afe/HostDetailView.java
lient/src/autotest/afe/HostListView.java
lient/src/autotest/afe/HostSelector.java
lient/src/autotest/afe/IButton.java
lient/src/autotest/afe/ICheckBox.java
lient/src/autotest/afe/IRadioButton.java
lient/src/autotest/afe/ITextArea.java
lient/src/autotest/afe/ITextBox.java
lient/src/autotest/afe/TestSelector.java
lient/src/autotest/afe/TestSelectorDisplay.java
lient/src/autotest/afe/UserPreferencesView.java
lient/src/autotest/afe/create/CreateJobViewDisplay.java
lient/src/autotest/afe/create/CreateJobViewPresenter.java
lient/src/autotest/afe/create/CreateJobViewTab.java
lient/src/autotest/common/table/DynamicTable.java
lient/src/autotest/common/ui/MultiListSelectPresenter.java
lient/src/autotest/common/ui/RadioChooser.java
lient/src/autotest/common/ui/RadioChooserDisplay.java
lient/src/autotest/common/ui/SimplifiedList.java
ko/rpc_interface.py
ko/rpc_interface_unittest.py
517d95a1ef4edb04da427763f86068a447d45ec7 29-Sep-2010 Benson Leung <bleung@chromium.org> Revert "Merge remote branch 'cros/upstream' into tempbranch2"

This reverts commit 25fc6d1f28e54c46689f12d3b93c2540ef45323a.

TBR=ericli@chromium.org

Review URL: http://codereview.chromium.org/3541002

Change-Id: Ib0165b19bfdf02264f8a6a74ddf3ae74c8c0f7df
lient/src/autotest/afe/AfeClient.java
lient/src/autotest/afe/AfeUtils.java
lient/src/autotest/afe/CheckBoxPanel.java
lient/src/autotest/afe/CheckBoxPanelDisplay.java
lient/src/autotest/afe/ClassFactory.java
lient/src/autotest/afe/ControlTypeSelect.java
lient/src/autotest/afe/ControlTypeSelectDisplay.java
lient/src/autotest/afe/CreateJobView.java
lient/src/autotest/afe/HostDetailView.java
lient/src/autotest/afe/HostListView.java
lient/src/autotest/afe/HostSelector.java
lient/src/autotest/afe/IButton.java
lient/src/autotest/afe/ICheckBox.java
lient/src/autotest/afe/IRadioButton.java
lient/src/autotest/afe/ITextArea.java
lient/src/autotest/afe/ITextBox.java
lient/src/autotest/afe/TestSelector.java
lient/src/autotest/afe/TestSelectorDisplay.java
lient/src/autotest/afe/UserPreferencesView.java
lient/src/autotest/afe/create/CreateJobViewDisplay.java
lient/src/autotest/afe/create/CreateJobViewPresenter.java
lient/src/autotest/afe/create/CreateJobViewTab.java
lient/src/autotest/common/table/DynamicTable.java
lient/src/autotest/common/ui/MultiListSelectPresenter.java
lient/src/autotest/common/ui/RadioChooser.java
lient/src/autotest/common/ui/RadioChooserDisplay.java
lient/src/autotest/common/ui/SimplifiedList.java
ko/rpc_interface.py
ko/rpc_interface_unittest.py
25fc6d1f28e54c46689f12d3b93c2540ef45323a 29-Sep-2010 Eric Li <ericli@chromium.org> Merge remote branch 'cros/upstream' into tempbranch2

Merged to trunk@4816.

BUG=
TEST=we will build a new autotest server instance, and keep cautotest running and then later do a cname switch.

Review URL: http://codereview.chromium.org/3511003

Change-Id: Iee5f52f45f28f84927d6c6f9a74edc370d40288a
lient/src/autotest/afe/AfeClient.java
lient/src/autotest/afe/AfeUtils.java
lient/src/autotest/afe/CheckBoxPanel.java
lient/src/autotest/afe/CheckBoxPanelDisplay.java
lient/src/autotest/afe/ClassFactory.java
lient/src/autotest/afe/ControlTypeSelect.java
lient/src/autotest/afe/ControlTypeSelectDisplay.java
lient/src/autotest/afe/CreateJobView.java
lient/src/autotest/afe/HostDetailView.java
lient/src/autotest/afe/HostListView.java
lient/src/autotest/afe/HostSelector.java
lient/src/autotest/afe/IButton.java
lient/src/autotest/afe/ICheckBox.java
lient/src/autotest/afe/IRadioButton.java
lient/src/autotest/afe/ITextArea.java
lient/src/autotest/afe/ITextBox.java
lient/src/autotest/afe/TestSelector.java
lient/src/autotest/afe/TestSelectorDisplay.java
lient/src/autotest/afe/UserPreferencesView.java
lient/src/autotest/afe/create/CreateJobViewDisplay.java
lient/src/autotest/afe/create/CreateJobViewPresenter.java
lient/src/autotest/afe/create/CreateJobViewTab.java
lient/src/autotest/common/table/DynamicTable.java
lient/src/autotest/common/ui/MultiListSelectPresenter.java
lient/src/autotest/common/ui/RadioChooser.java
lient/src/autotest/common/ui/RadioChooserDisplay.java
lient/src/autotest/common/ui/SimplifiedList.java
ko/rpc_interface.py
ko/rpc_interface_unittest.py
7f0e2c7ef169a6a8cab2e1394787973fbc081cd5 29-Jul-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Override parameterized_jobs setting for frontend_unittest as well

Signed-off-by: James Ren <jamesren@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@4737 592f7852-d20e-0410-864c-8624ca9c26a4
rontend_unittest.py
26dd6aa23eb2771acefa611c029c06f6ee6f1b11 17-Jul-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Prevent NullPointerException when clicking on a test in Create Job.

No functional effect, just keeps the exception from showing up in DevMode.

Signed-off-by: James Ren <jamesren@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@4721 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/common/table/DataTable.java
4a41e0145c2ede3605d19cdfacb779ca6aca1ccc 17-Jul-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Begin adding support for parameterized jobs.

This will allow test developers to specify certain parameters that a test
control file may take, so that users can then easily set those parameters
on job create. Enabling this feature removes the ability to edit the
control file directly on job creation.

Feature is currently INCOMPLETE. Do not attempt to use. This feature will
be committed in small pieces for the sake of having smaller code reviews.

Signed-off-by: James Ren <jamesren@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@4720 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/001_rpc_test.txt
fe/frontend_test_utils.py
fe/model_attributes.py
fe/models.py
fe/models_test.py
fe/rpc_interface.py
fe/rpc_interface_unittest.py
fe/rpc_utils.py
lient/src/autotest/afe/CreateJobView.java
igrations/063_parameterized_tests.py
d3b0c8c8c65d36943e721b7f4d0d45f76809e101 12-Jul-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> This file was missing from the patch.

Signed-off-by: James Ren <jamesren@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@4719 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/tko/TestSet.java
f2341ebc50501393e4ee8a4d15d27fe5d7a17b40 21-Jun-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Switch urllib2 with httplib2 in frontend/afe/views.py.
urllib2 was causing random 'Connection reset by peer' on GWT devmode; appears to
be a bug in the library, as httplib2 does not cause this error.

Signed-off-by: James Ren <jamesren@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@4655 592f7852-d20e-0410-864c-8624ca9c26a4
fe/views.py
e0ca68330b56a5fff67008d7226f784bbd0dc911 11-Jun-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Have embedded spreadsheets follow the passive view design pattern

Signed-off-by: James Ren <jamesren@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@4607 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/EmbeddedSpreadsheetClient.gwt.xml
lient/src/autotest/tko/EmbeddedSpreadsheetClient.java
lient/src/autotest/tko/HeaderField.java
lient/src/autotest/tko/TestGroupDataSource.java
lient/src/autotest/tko/embedded_spreadsheet/EmbeddedSpreadsheetClient.java
lient/src/autotest/tko/embedded_spreadsheet/EmbeddedSpreadsheetDisplay.java
lient/src/autotest/tko/embedded_spreadsheet/EmbeddedSpreadsheetPresenter.java
a397e604073b2c8d77aab5b97fe8966dcfd40a24 11-Jun-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Add a listener to the embedded spreadsheet, to open the full TKO results
interface upon click.

Signed-off-by: James Ren <jamesren@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@4596 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/tko/EmbeddedSpreadsheetClient.java
lient/src/autotest/tko/SpreadsheetDataProcessor.java
lient/src/autotest/tko/SpreadsheetView.java
lient/src/autotest/tko/TkoSpreadsheetUtils.java
8444cf4bb5e180cb9f1050c5908889331ae49253 11-Jun-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> From: Shaun Ruffell <sruffell@digium.com>

Eliminates the following exception when clicking the feeds link from the
frontend.

Traceback:
File "/usr/local/autotest/site-packages/django/core/handlers/base.py" in get_response
92. response = callback(request, *callback_args, **callback_kwargs)
File "/usr/local/autotest/frontend/afe/feeds/feed.py" in feed_view
30. feedgen = f(slug, request).get_feed(param)
File "/usr/local/autotest/frontend/afe/feeds/feed.py" in __init__
50. super(JobFeed, self).__init__(slug, request.path)
File "/usr/local/autotest/site-packages/django/contrib/syndication/feeds.py" in __init__
33. self.feed_url = self.feed_url or request.path

Exception Type: AttributeError at /afe/server/feeds/jobs/completed/
Exception Value: 'unicode' object has no attribute 'path'

Signed-off-by: Shaun Ruffell <sruffell@digium.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4595 592f7852-d20e-0410-864c-8624ca9c26a4
fe/feeds/feed.py
b41f6c9c6f565bd6861117c46c6c35edfbd108e2 10-Jun-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Handle changes in MySQL behavior between migrations

Signed-off-by: James Ren <jamesren@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@4590 592f7852-d20e-0410-864c-8624ca9c26a4
igrations/062_drone_sets_unique.py
543d9fb64c2978a8094ca52b9045f43a3b54ff4b 07-Jun-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Enforce that a drone may appear in at most one drone set. Also hide drones that
are already used from the drone set filter_horizontal widget.

Signed-off-by: James Ren <jamesren@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@4580 592f7852-d20e-0410-864c-8624ca9c26a4
fe/admin.py
igrations/062_drone_sets_unique.py
2ae97ccd4b63399e7b6ccecec84a6dcbb523c119 02-Jun-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Fix a race condition in new embedded spreadsheets. Also render a nicer page when
no results exist.
The problem occurred when the parent window finished loading and sends the
postMessage before the embedded frame finishes setting up its listener. The
message is then lost, and the frame remains blank. This way passes the job down
as part of the URL, so it's always available in the embedded page.

Signed-off-by: James Ren <jamesren@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@4575 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/JobDetailView.java
lient/src/autotest/tko/EmbeddedSpreadsheetClient.java
c277b540001d734a75bb72dc55b93abb6eebaa4e 01-Jun-2010 lmr <lmr@592f7852-d20e-0410-864c-8624ca9c26a4> Adding missing __init__.py file under frontend/migrations

That directory is supposed to be importable by the migration
script 061_drone_sets_permissions_proper.py, however the
__init__.py file is missing in the upstream tree (probably
because patch doesn't like to add 0-sized files in the
filesystem). Let's add it.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4571 592f7852-d20e-0410-864c-8624ca9c26a4
igrations/__init__.py
e84f1fab8c04cdcfc992f1c122e0c08c65843657 01-Jun-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Create embeddable spreadsheets, and use that on the AFE Job Detail tab.

Signed-off-by: James Ren <jamesren@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@4570 592f7852-d20e-0410-864c-8624ca9c26a4
lient/EmbeddedSpreadsheetClient.launch
lient/src/autotest/EmbeddedSpreadsheetClient.gwt.xml
lient/src/autotest/afe/JobDetailView.java
lient/src/autotest/common/Utils.java
lient/src/autotest/public/EmbeddedSpreadsheetClient.html
lient/src/autotest/public/afeclient.css
lient/src/autotest/tko/EmbeddedSpreadsheetClient.java
lient/src/autotest/tko/SpreadsheetDataProcessor.java
lient/src/autotest/tko/TestGroupDataSource.java
589cc78019a41fa293d7583a3c1b6dfca42da41b 28-May-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Code cleanup
Whitespace fixes, and addition of newlines at end of some files.
Use StringBuilder.append() rather than String += in loops.
Add equals() and hashCode() for class that implements Comparable.

Signed-off-by: James Ren <jamesren@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@4567 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/AfeUtils.java
lient/src/autotest/afe/HostDataSource.java
lient/src/autotest/afe/JobStatusDataSource.java
lient/src/autotest/common/table/CheckboxFilter.java
lient/src/autotest/common/table/FieldFilter.java
lient/src/autotest/common/table/Filter.java
lient/src/autotest/common/table/JSONObjectComparator.java
lient/src/autotest/common/table/JSONObjectSet.java
lient/src/autotest/common/table/ListFilter.java
lient/src/autotest/common/ui/SimpleDialog.java
lient/src/autotest/tko/StatusSummary.java
06e59f14d08f614c659331bb03a352bf488a7cfc 24-May-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Initial implementation of Test Planner Test View.

Trying again because generated patch was malformed.

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4556 592f7852-d20e-0410-864c-8624ca9c26a4
fe/frontend_test_utils.py
lient/src/autotest/common/JsonManipulator.java
lient/src/autotest/common/Utils.java
lient/src/autotest/planner/TestPlannerClient.java
lient/src/autotest/planner/TestPlannerTableDisplay.java
lient/src/autotest/planner/TestPlannerTableImpl.java
lient/src/autotest/planner/TestViewTab.java
lient/src/autotest/planner/TestViewTabDisplay.java
lient/src/autotest/planner/machine/MachineViewDisplay.java
lient/src/autotest/planner/machine/MachineViewPresenter.java
lient/src/autotest/planner/machine/MachineViewTable.java
lient/src/autotest/planner/machine/MachineViewTableDisplay.java
lient/src/autotest/planner/overview/OverviewTable.java
lient/src/autotest/planner/test/TestViewDisplay.java
lient/src/autotest/planner/test/TestViewPresenter.java
lient/src/autotest/planner/test/TestViewTab.java
lient/src/autotest/planner/test/TestViewTable.java
lient/src/autotest/public/TestPlannerClient.html
lanner/models.py
lanner/models_test.py
lanner/rpc_interface.py
lanner/rpc_interface_unittest.py
lanner/rpc_utils.py
lanner/rpc_utils_unittest.py
d56e8d2d80f3f877b97da601ba90ca5233743c2c 20-May-2010 lmr <lmr@592f7852-d20e-0410-864c-8624ca9c26a4> Removing the last hardcoded reference to python 2.4

Not sure how it was still there, nevertheless, remove it.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4532 592f7852-d20e-0410-864c-8624ca9c26a4
ko/csv_encoder_unittest.py
fefab4aeaf1cc8ae9837a860c8502b001eec0c99 18-May-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Add filter on locked_by user on AFE host lists

Trying again, so that there isn't a huge quoted patch in the patch notes.

Signed-off-by: James Ren <jamesren@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@4504 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/HostTableDecorator.java
lient/src/autotest/common/table/BooleanFilter.java
lient/src/autotest/common/table/ListFilter.java
2e48bcb728e151f5744d73f0eb8a274ee59fc461 18-May-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Initial implementation of Test Planner Overview tab

Signed-off-by: James Ren <jamesren@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@4503 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/TestPlannerClient.gwt.xml
lient/src/autotest/common/JsonManipulator.java
lient/src/autotest/planner/OverviewTab.java
lient/src/autotest/planner/OverviewTabDisplay.java
lient/src/autotest/planner/TestPlanSelector.java
lient/src/autotest/planner/TestPlannerClient.java
lient/src/autotest/planner/TestPlannerTab.java
lient/src/autotest/planner/overview/OverviewTab.java
lient/src/autotest/planner/overview/OverviewTabDisplay.java
lient/src/autotest/planner/overview/OverviewTabPresenter.java
lient/src/autotest/planner/overview/OverviewTable.java
lient/src/autotest/planner/overview/OverviewTableDisplay.java
lient/src/autotest/public/TestPlannerClient.html
lient/src/autotest/public/testplannerclient.css
lanner/planner_test_utils.py
lanner/rpc_interface.py
lanner/rpc_interface_unittest.py
lanner/rpc_utils.py
lanner/rpc_utils_unittest.py
f8c6307dc5abe8013a9824b1f45dcbe3f7a59f87 15-May-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Replace case-sensitive logic in create_job with case-insensitive logic. The
MySQL database performs case-insensitive searches by default.

Signed-off-by: James Ren <jamesren@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@4490 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpc_interface.py
53cd3240c81eb3c39c51a70c32b588fa16f64b3b 07-May-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Fix migration 59's handling of a fresh database. The migration did not handle
the case where the Django auth tables did not exist. The permissions will be
inserted into the table later upon the first run of .
Also add a migration to properly create the permissions entries in the existing
tables.

Signed-off-by: James Ren <jamesren@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@4486 592f7852-d20e-0410-864c-8624ca9c26a4
igrations/059_drone_sets_permissions.py
igrations/061_drone_sets_permissions_proper.py
55fc38afc375e326656f121770305376b6239659 07-May-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Add coloring to Test Planner machine view table cells.

Signed-off-by: James Ren <jamesren@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@4485 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/common/AbstractStatusSummary.java
lient/src/autotest/common/spreadsheet/Spreadsheet.java
lient/src/autotest/common/table/TableRenderer.java
lient/src/autotest/planner/machine/MachineViewTable.java
lient/src/autotest/planner/machine/MachineViewTableDisplay.java
lient/src/autotest/planner/machine/StatusSummary.java
lient/src/autotest/public/common.css
lient/src/autotest/tko/SpreadsheetDataProcessor.java
lient/src/autotest/tko/StatusSummary.java
lient/src/autotest/tko/TableView.java
lanner/rpc_interface.py
lanner/rpc_interface_unittest.py
8d3963567c33b24e78e1c8a2656fbb247aeaf167 06-May-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Handle missing 'content' parameter in TKO Spreadsheet View

Signed-off-by: James Ren <jamesren@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@4481 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/tko/ContentSelect.java
8a7f36583afe076a7198d1b34fe109aa491dc277 06-May-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Remove SimpleHyperlink and replace with Anchor, which is a built-in class that
does the same thing.

Signed-off-by: James Ren <jamesren@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@4480 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/CreateJobView.java
lient/src/autotest/afe/HostSelector.java
lient/src/autotest/common/ui/Paginator.java
lient/src/autotest/common/ui/SimpleHyperlink.java
lient/src/autotest/common/ui/TableActionsPanel.java
lient/src/autotest/common/ui/TableSelectionPanel.java
lient/src/autotest/common/ui/ToggleLink.java
lient/src/autotest/tko/CommonPanel.java
lient/src/autotest/tko/ContentSelect.java
lient/src/autotest/tko/DynamicGraphingFrontend.java
lient/src/autotest/tko/FilterSelector.java
lient/src/autotest/tko/FilterStringViewer.java
lient/src/autotest/tko/GraphingFrontend.java
lient/src/autotest/tko/MetricsPlot.java
lient/src/autotest/tko/ParameterizedFieldListDisplay.java
lient/src/autotest/tko/SeriesSelector.java
lient/src/autotest/tko/SpreadsheetView.java
lient/src/autotest/tko/TestLabelManager.java
lient/src/autotest/tko/WidgetList.java
9af703ae5afb1c21370200f95e4474aeafd13c9b 06-May-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Initial implementation of Test Planner Machine View. No coloring or
drilldowns yet.

Signed-off-by: James Ren <jamesren@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@4479 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/planner/MachineViewTab.java
lient/src/autotest/planner/MachineViewTabDisplay.java
lient/src/autotest/planner/TestPlannerClient.java
lient/src/autotest/planner/TestPlannerUtils.java
lient/src/autotest/planner/machine/MachineViewDisplay.java
lient/src/autotest/planner/machine/MachineViewPresenter.java
lient/src/autotest/planner/machine/MachineViewTab.java
lient/src/autotest/planner/machine/MachineViewTable.java
lient/src/autotest/planner/machine/MachineViewTableDisplay.java
lient/src/autotest/planner/triage/FailureTable.java
lient/src/autotest/planner/triage/FailureTableDisplay.java
lient/src/autotest/planner/triage/TriageViewDisplay.java
lient/src/autotest/planner/triage/TriageViewPresenter.java
lient/src/autotest/public/TestPlannerClient.html
lanner/planner_test_utils.py
lanner/rpc_interface.py
lanner/rpc_interface_unittest.py
daa8128fa8d6788ca2a4f551db6a222fa55c8912 06-May-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Fix Test Planner execution engine unittest

Signed-off-by: James Ren <jamesren@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@4478 592f7852-d20e-0410-864c-8624ca9c26a4
lanner/execution_engine_unittest.py
0c82cd6a562c9414076108073b8645a919ccebf8 06-May-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Remove usage of ImageBundle (deprecated in GWT 2.0)

Signed-off-by: James Ren <jamesren@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@4471 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/TestPlannerClient.gwt.xml
lient/src/autotest/planner/resources/PlannerClientBundle.java
lient/src/autotest/planner/resources/PlannerImageBundle.java
lient/src/autotest/planner/triage/TriagePopupDisplay.java
b33350f43a891fb9d571d46a5fc7351a78b9f3b7 06-May-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Set the Test Planner "set atomic group" job parameters directly instead of using
keyvals. This is to support cloning of the "set atomic group" job.

Signed-off-by: James Ren <jamesren@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@4470 592f7852-d20e-0410-864c-8624ca9c26a4
lanner/execution_engine.py
lanner/set_atomic_group_control.srv
9a6f5f6d6935af0a3c4218feb5dba7fdc63edce3 06-May-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Allow additional parameters to be specified with test plans. Also wrap
the control file for each test in the test plan, so that the Test
Planner can prefix each intended test with a verify_test, along with
any site-specific prefixes that may be required.

Allowing additional parameters gives the users the option to directly
specify the parameters for the verify_test, along with what
site-specific prefixes, if any, to attach.

Signed-off-by: James Ren <jamesren@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@4469 592f7852-d20e-0410-864c-8624ca9c26a4
igrations/060_add_planner_additional_parameters.py
lanner/control_file.py
lanner/control_file_unittest.py
lanner/execution_engine.py
lanner/model_attributes.py
lanner/models.py
lanner/models_test.py
lanner/rpc_interface.py
lanner/rpc_interface_unittest.py
lanner/rpc_utils.py
lanner/rpc_utils_unittest.py
lanner/support.py
5adb58ebbc02cfa61d26689046679c5eb72da287 05-May-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Ensure that drone set permissions are enabled in the admin interface.

Signed-off-by: James Ren <jamesren@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@4466 592f7852-d20e-0410-864c-8624ca9c26a4
igrations/059_drone_sets_permissions.py
6181f3eb0b96142ccf6b62cb514cd60161358dff 05-May-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Fix some Planner Triage View styling. Add "glass" behind the Triage
popup, which grays out the rest of the page until the user closes the
popup.

Signed-off-by: James Ren <jamesren@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@4465 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/planner/triage/TriagePopupDisplay.java
lient/src/autotest/planner/triage/TriageViewDisplay.java
lient/src/autotest/public/standard.css
4b7efc77814c0af1f35dd6fbb5619fcf472a1ca2 03-May-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Refactor Test Planner GWT code. Factoring out common functionality in
presenter, display, and tab classes for reuse with other Test Planner
tabs.

Signed-off-by: James Ren <jamesren@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@4459 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/planner/TestPlannerDisplay.java
lient/src/autotest/planner/TestPlannerPresenter.java
lient/src/autotest/planner/TestPlannerTab.java
lient/src/autotest/planner/triage/TriageViewDisplay.java
lient/src/autotest/planner/triage/TriageViewPresenter.java
lient/src/autotest/planner/triage/TriageViewTab.java
012d0325b33deeb81f06fa11f5498d2bc8146f57 30-Apr-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Move to GWT 2.0.3. Drop in new gwt-incubator to match.

Signed-off-by: James Ren <jamesren@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@4452 592f7852-d20e-0410-864c-8624ca9c26a4
lient/AfeClient.launch
lient/EmbeddedTkoClient.launch
lient/TestPlannerClient.launch
lient/TkoClient.launch
lient/src/autotest/TestPlannerClient.gwt.xml
lient/src/autotest/common/PaddedJsonRpcProxy.java
lient/src/autotest/planner/triage/FailureTableDisplay.java
eaaacf0fbe4e6c1d4f642f2c15c30c0ebe0a6864 28-Apr-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Fix iteration attribute custom field on TKO

Signed-off-by: James Ren <jamesren@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@4451 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/tko/IterationAttributeField.java
6275824bd8458253d3ba30beb0f224bc24529158 28-Apr-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Change Triage View's tables to ScrollTable, and implement bulk triage

Signed-off-by: James Ren <jamesren@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@4450 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/TestPlannerClient.gwt.xml
lient/src/autotest/common/Utils.java
lient/src/autotest/planner/triage/FailureTable.java
lient/src/autotest/planner/triage/FailureTableDisplay.java
lient/src/autotest/planner/triage/TriagePopup.java
lient/src/autotest/planner/triage/TriageViewDisplay.java
lient/src/autotest/planner/triage/TriageViewPresenter.java
lient/src/autotest/public/TestPlannerClient.html
lient/src/autotest/public/scrolltable.css
lanner/rpc_interface.py
lanner/rpc_interface_unittest.py
lanner/rpc_utils.py
lanner/rpc_utils_unittest.py
42cc7a8827faf7cad130de15d3f1300e7a3dc7ac 24-Apr-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Remove obsolete compile script

Signed-off-by: James Ren <jamesren@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@4445 592f7852-d20e-0410-864c-8624ca9c26a4
lient/TestPlannerClient-compile
0c1460908bfe438a9367beb187ce1b87cad486c1 23-Apr-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Have Test Planner execution engine catch and log exceptions, then retry

Signed-off-by: James Ren <jamesren@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@4443 592f7852-d20e-0410-864c-8624ca9c26a4
lanner/execution_engine.py
6b8e806faf6f605e6e11dc762e94484e2ca50726 21-Apr-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Add back gwt_dir

Signed-off-by: Scott Zawalski <scottz@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4440 592f7852-d20e-0410-864c-8624ca9c26a4
lient/gwt_dir
76fcf19ec42d5c7580d2e7891e4610e5fe725286 21-Apr-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Add ability to associate drone sets with jobs. This restricts a job to
running on a specified set of drones.

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4439 592f7852-d20e-0410-864c-8624ca9c26a4
fe/admin.py
fe/doctests/001_rpc_test.txt
fe/frontend_test_utils.py
fe/models.py
fe/models_test.py
fe/resources.py
fe/resources_test.py
fe/rpc_interface.py
fe/rpc_interface_unittest.py
lient/src/autotest/afe/AfeUtils.java
lient/src/autotest/afe/CreateJobView.java
lient/src/autotest/afe/JobDetailView.java
lient/src/autotest/afe/UserPreferencesView.java
lient/src/autotest/public/AfeClient.html
igrations/058_drone_management.py
cd7a81a58c7a1a16105dcd9dca5e9f1680eff65b 21-Apr-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Various changes to support further high-level automation efforts.

* added a RESTful interface for TKO. right now there's only a single, simple resource for accessing test attributes.
* extended the REST server library in a few ways, most notably to support
* querying on keyvals, with something like ?has_keyval=mykey=myvalue&...
* operators, delimited by a colon, like ?hostname:in=host1,host2,host3
* loading relationships over many items efficiently (see InstanceEntry.prepare_for_full_representation()). this is used to fill in keyvals when requesting a job listing, but it can (and should) be used in other places, such as listing labels for a host collection.
* loading a collection with inlined full representations, by passing full_representations=true
* added various features to the AFE RESTful interface as necessary.
* various fixes to the rest_client library, most notably
* changed HTTP client in rest_client.py to use DI rather than singleton, easing testability. the same should be done for _get_request_headers(), to be honest.
* better support for query params, including accepting a MultiValueDict and supporting URIs that already have query args
* basic support for redirects
* builtin support for requesting a full collection (get_full()), when clients explicitly expect the result not to be paged. i'm still considering alternative approaches to this -- it may make sense to have something like this be the default, and have clients set a default page size limit rather than passing it every time.
* minor change to mock.py to provide better debugging output.

Signed-off-by: Steve Howard <showard@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@4438 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
fe/resources.py
fe/resources_test.py
hared/query_lib.py
hared/resource_lib.py
hared/resource_test_utils.py
hared/rest_client.py
ko/resources.py
ko/resources_test.py
ko/rpc_interface_unittest.py
ko/urls.py
1e0a4cef1096904103eb29cbde8d65ad892579a4 21-Apr-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Fix frontend.afe.rpc_interface_unittest to not require Python >=2.5
(there is no Exception.message in Python <2.5).

Signed-off-by: Mihai Rusu <dizzy@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@4436 592f7852-d20e-0410-864c-8624ca9c26a4
fe/model_logic.py
ca2a900b17c70744fc6f70a51332d83372d5ec73 21-Apr-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Add GWT Incubator to site-packages

Signed-off-by: James Ren <jamesren@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@4433 592f7852-d20e-0410-864c-8624ca9c26a4
lient/.classpath
lient/AfeClient-shell
lient/AfeClient.launch
lient/EmbeddedTkoClient-shell
lient/EmbeddedTkoClient.launch
lient/TestPlannerClient-compile
lient/TestPlannerClient-shell
lient/TestPlannerClient.launch
lient/TkoClient-shell
lient/TkoClient.launch
e38a0a7d14c0a824ce41173c9db54d70b5c58d83 19-Apr-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Add an 'owner' param to the Test Planner execution engine and the REST
client

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4431 592f7852-d20e-0410-864c-8624ca9c26a4
fe/resources.py
fe/resources_test.py
lanner/execution_engine.py
lanner/execution_engine_control.srv
lanner/execution_engine_unittest.py
lanner/rpc_utils.py
2f4cc045ea7bede3c90a399e2f359ddd03bab1fe 19-Apr-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Make the execution engine's set_atomic_group subtask parallel

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4430 592f7852-d20e-0410-864c-8624ca9c26a4
lanner/set_atomic_group_control.srv
d8b6e17c8edb2845949e39c75337e4389a630c63 17-Apr-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Handle the case where keyvals may not be specified on job create. This
also fixes a bug where recurring jobs are created incorrectly.

Signed-off-by: James Ren <jamesren@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@4423 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
ab9e11bf4c7ac33d8beed96475750ffd26c4f160 17-Apr-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Remove reference to undefined variable

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4422 592f7852-d20e-0410-864c-8624ca9c26a4
lanner/rpc_utils.py
121eee6338595777668e669c61521ceeceeeddf7 13-Apr-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Removing references to the 'Dead' host status

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4409 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
758bd7c5b61a6e29853cfd97383ba1fce43f17f2 12-Apr-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Fix display of metahost jobs

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4406 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/JobStatusDataSource.java
2275ef1f0f92093cd9bb406acf5c3fbc0bd47b6b 12-Apr-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Implement creation and cloning of hostless jobs from AFE Create Job tab

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4405 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpc_interface.py
fe/rpc_utils.py
lient/src/autotest/afe/CreateJobView.java
lient/src/autotest/afe/HostSelector.java
lient/src/autotest/afe/JobDetailView.java
lient/src/autotest/public/AfeClient.html
e5dd765d4900a6a826356075906ed552c59d6427 12-Apr-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Add listeners to the TestPlanSelector, and have Triage View refresh on
entering a test plan.

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4403 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/common/ui/HasTabVisible.java
lient/src/autotest/common/ui/TabView.java
lient/src/autotest/planner/TestPlanSelector.java
lient/src/autotest/planner/triage/TriageViewPresenter.java
lient/src/autotest/planner/triage/TriageViewTab.java
9497a0972f9e18f623d1aa8ffe926d4e0ecbc097 12-Apr-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Add some spacing to the Triage View tables

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4402 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/planner/triage/TriageViewDisplay.java
30f06c77e2c74017d59257782e2c5cf67d2a5869 12-Apr-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Set execution engine parameters directly instead of using keyvals, to
support cloning of the execution engine job

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4401 592f7852-d20e-0410-864c-8624ca9c26a4
lanner/execution_engine_control.srv
lanner/rpc_utils.py
16e1bbb5b9a733ced284976aa84af3e1dd34386d 12-Apr-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Setting Test Planner execution engine max_runtime_hrs to a large number

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4400 592f7852-d20e-0410-864c-8624ca9c26a4
lanner/rpc_utils.py
fb5df1a221e3e091bf9d9891cd667f5b0e0e93f3 12-Apr-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Don't render a Test Planner triage failure table if it's empty

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4398 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/planner/triage/FailureTable.java
1602f3be8cfc9588ffd47ac79c5eb95cf3eaa27b 09-Apr-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Add get_motd() to Planner RPC functions

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4395 592f7852-d20e-0410-864c-8624ca9c26a4
lanner/rpc_interface.py
0cde1eb576672b830433dfa66a78d26bb2b5bbcc 09-Apr-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Increase Test Planner execution engine AFE timeout

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4393 592f7852-d20e-0410-864c-8624ca9c26a4
lanner/rpc_utils.py
93270d1519a4ab7f6be9d21d03cbce8f021dbcc2 09-Apr-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> If REST client gets an invalid response, regenerate the cached request
headers and try one more time

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4392 592f7852-d20e-0410-864c-8624ca9c26a4
hared/rest_client.py
ea3b3bed4739807f65ce69ee5a9f244acca6bf06 09-Apr-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Fix display of hostless jobs on AFE View Job tab

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4390 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/JobStatusDataSource.java
4be631f3f1d88231dbfbfd946ef396165511f58a 09-Apr-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Add Planner triage actions


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4389 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/planner/resources/PlannerImageBundle.java
lient/src/autotest/planner/resources/close.png
lient/src/autotest/planner/triage/FailureTable.java
lient/src/autotest/planner/triage/FailureTableDisplay.java
lient/src/autotest/planner/triage/TriagePopup.java
lient/src/autotest/planner/triage/TriagePopupDisplay.java
igrations/057_add_planner_triage_actions.py
lanner/failure_actions.py
lanner/models.py
lanner/rpc_interface.py
lanner/rpc_interface_unittest.py
lanner/rpc_utils.py
lanner/rpc_utils_unittest.py
dbeebf80126efd53a4847911628686f3c2767f0d 09-Apr-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Add implementation for the Planner global support. Also add unit tests
for the execution engine.

Caveat: the global support is still single-threaded with the execution
engine. I have an idea of how to make it multi-threaded, but I'd like to
push this simpler prototype out first.

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4388 592f7852-d20e-0410-864c-8624ca9c26a4
igrations/056_planner_global_support.py
lanner/execution_engine.py
lanner/execution_engine_unittest.py
lanner/models.py
lanner/rpc_interface.py
lanner/rpc_interface_unittest.py
lanner/rpc_utils.py
lanner/rpc_utils_unittest.py
lanner/support.py
b852bce914dc3ae05b26f3655bfb437ffed001be 07-Apr-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Adding the GWT framework for the Test Planner.

This adds a new application to the GWT frontend, but does not link to it
from the other frontends yet. I don't anticipate anyone to be using this
just yet. Once the project reaches the point where I can release a working
prototype, I will create user documentation for it and send an announcement.

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4378 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/common/DomUtils.java
lient/src/autotest/common/JsonRpcProxy.java
lient/src/autotest/common/spreadsheet/Spreadsheet.java
lient/src/autotest/common/spreadsheet/SpreadsheetSelectionManager.java
lient/src/autotest/common/table/FragmentedTable.java
lient/src/autotest/common/table/TableRenderer.java
lient/src/autotest/planner/AutoprocessedTab.java
lient/src/autotest/planner/AutoprocessedTabDisplay.java
lient/src/autotest/planner/HistoryTab.java
lient/src/autotest/planner/HistoryTabDisplay.java
lient/src/autotest/planner/MachineViewTab.java
lient/src/autotest/planner/MachineViewTabDisplay.java
lient/src/autotest/planner/OverviewTab.java
lient/src/autotest/planner/OverviewTabDisplay.java
lient/src/autotest/planner/TestPlanSelector.java
lient/src/autotest/planner/TestPlanSelectorDisplay.java
lient/src/autotest/planner/TestPlannerClient.java
lient/src/autotest/planner/TestViewTab.java
lient/src/autotest/planner/TestViewTabDisplay.java
lient/src/autotest/planner/triage/FailureTable.java
lient/src/autotest/planner/triage/FailureTableDisplay.java
lient/src/autotest/planner/triage/TriageViewDisplay.java
lient/src/autotest/planner/triage/TriageViewPresenter.java
lient/src/autotest/planner/triage/TriageViewTab.java
lient/src/autotest/public/TestPlannerClient.html
lient/src/autotest/tko/FragmentedTable.java
lient/src/autotest/tko/Spreadsheet.java
lient/src/autotest/tko/SpreadsheetDataProcessor.java
lient/src/autotest/tko/SpreadsheetSelectionManager.java
lient/src/autotest/tko/SpreadsheetView.java
lient/src/autotest/tko/TableRenderer.java
lient/src/autotest/tko/TkoUtils.java
lanner/models.py
lanner/rpc_interface.py
81acee0a7b941a54438f8c0d0d0e3b2d7e5d019f 05-Apr-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Make sure the 'invalidated' test label exists

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4373 592f7852-d20e-0410-864c-8624ca9c26a4
igrations/055_ensure_invalidated_test_label_exists.py
26f9a9fa41cfd4b866f8f1cce85912088d9a6bdf 02-Apr-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Changing IterationResult.value to FloatField in Django models, to match
the FLOAT type in the underlying database.

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4372 592f7852-d20e-0410-864c-8624ca9c26a4
ko/models.py
d93f21c7fe1c5d0731931a18503b2f32a417b157 01-Apr-2010 lmr <lmr@592f7852-d20e-0410-864c-8624ca9c26a4> Autotest web database: Adding NOP migrations

Adding migrations 053 and 054, which are no-ops. We want to do this because
we had to use up some extra migrations in our internal branch, and we are
now at version 54, which is identical to version 52 in SVN.

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4361 592f7852-d20e-0410-864c-8624ca9c26a4
igrations/053_nop.py
igrations/054_nop.py
79697287380810bc23b8a80ace94e6aa0fb82657 01-Apr-2010 lmr <lmr@592f7852-d20e-0410-864c-8624ca9c26a4> Web frontend database: Expand test and subdir fields from tko_tests v2

Tests with large tags, such as the ones that can appear on the kvm test
can have the fields 'test' and 'subdir' too large. This patch raises
the length of such fields on the autotest_web database to 300.

Changes from v1:

* Realized that tko/migrations/019_widen_test_name_field.py expands the
original value that test has from 30 to 60, so changed the downgrade
sql statement accordingly.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4360 592f7852-d20e-0410-864c-8624ca9c26a4
igrations/052_expand_test_subdir_fields.py
ko/models.py
708f1c00df025e899c0ee8080fa4e81f71b8fa4c 31-Mar-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Support job keyvals and iteration attributes in the query_results CLI

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4359 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/tko/CommonPanel.java
lient/src/autotest/tko/IterationAttributeField.java
lient/src/autotest/tko/IterationResultField.java
lient/src/autotest/tko/JobKeyvalField.java
lient/src/autotest/tko/ParameterizedField.java
lient/src/autotest/tko/TestAttributeField.java
lient/src/autotest/tko/TestLabelField.java
ko/models.py
ko/rpc_interface_unittest.py
f665cf254ebc901b2540642cffaee061416f5f64 31-Mar-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Remove useless "noauth" handling from the RPC interface.

Signed-off-by: Gregory Smith <gps@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4358 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/001_rpc_test.txt
ko/urls.py
rls_common.py
8d0d3d568102e6362b01456a9aef368a8071d3a1 25-Mar-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Add Planner RPC utils unit tests

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4346 592f7852-d20e-0410-864c-8624ca9c26a4
lanner/planner_test_utils.py
lanner/rpc_interface_unittest.py
lanner/rpc_utils_unittest.py
92afa568352ba2aa921c7fefce907c2119eaa0b9 23-Mar-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Rebase the AUTOTEST_WEB database to schema version 51. Also default the
migration manager to the AUTOTEST_WEB database, since the TKO database
has been removed.

Signed-off-by: James Ren <jamesren@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@4317 592f7852-d20e-0410-864c-8624ca9c26a4
igrations/001_initial_db.py
bc13c812b36124095ecfa4ee2744b4fdc2676400 11-Mar-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Missed 2 migrations...

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4303 592f7852-d20e-0410-864c-8624ca9c26a4
igrations/050_more_test_planner_additions.py
igrations/051_rename_planner_tests.py
163decb592e0d9013d0a80310152d86cd86e7acc 11-Mar-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Add missing frontend/planner/model_attributes.py

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4302 592f7852-d20e-0410-864c-8624ca9c26a4
lanner/model_attributes.py
3e9f609dcf3c3d75daec1af5551b0e4022f5e629 11-Mar-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Implementation of Test Planner execution engine. Is currently able to
schedule single-host tests and place the results in the proper planner
tables for analysis.

TODO: global support object, execution_engine.py unit tests

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4301 592f7852-d20e-0410-864c-8624ca9c26a4
fe/model_logic.py
fe/resources.py
lanner/execution_engine.py
lanner/execution_engine_control.srv
lanner/models.py
lanner/rpc_interface.py
lanner/rpc_interface_unittest.py
lanner/rpc_utils.py
lanner/views.py
b68405d62bbf9b0c4a640754d6aaa2892eca9948 11-Mar-2010 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Better error handling and message when the (site) authentication didn't work.

Signed-off-by: Jean-Marc Eurin <jmeurin@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4299 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpc_client_lib.py
248a43336f472666afdbe53d7399a5bfffb3c772 02-Mar-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Modify Test Planner execution engine to use new model_attributes

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4281 592f7852-d20e-0410-864c-8624ca9c26a4
lanner/execution_engine_control.srv
dd855244f44b65d0508345c6fef74846652c8c26 02-Mar-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Abstract out common models used in the frontend's models.py so that django is not required to interact with non Django portions of the code.

This includes the enums RebootBefore, RebootAfter and Test.Type


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4280 592f7852-d20e-0410-864c-8624ca9c26a4
fe/frontend_test_utils.py
fe/model_attributes.py
fe/models.py
fe/resources.py
fe/resources_test.py
fe/rpc_interface.py
fe/rpc_utils.py
e36562323bec14d14b7c583db94788a725514ff2 02-Mar-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Fix Django's QuerySet.delete() behavior for models that use "invalid"

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4276 592f7852-d20e-0410-864c-8624ca9c26a4
fe/model_logic.py
fe/models.py
fe/models_test.py
f0f9dae372fa1abb91a653e7a74ff16ed25abb89 26-Feb-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> * fix a bug in utils.run() when passing stdin as an empty string
* remove the check in AFE that a metahost job request has enough hosts under the label. with the new pluggable metahost handlers, it's possible for handlers to create hosts on the fly to satisfy metahost requests
* expand the size of the label name field in the AFE DB to support more powerful metahost handlers

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4273 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpc_utils.py
igrations/048_expand_label_name_field.py
128d418ec352d6783a9df32b83fe18c0d0d4a797 19-Feb-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Forgot to use "self" a few times

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4261 592f7852-d20e-0410-864c-8624ca9c26a4
lanner/models.py
c394022b7667899b845fa6a7f7712070fef33b8c 19-Feb-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Add framework for Test Planner execution engine, and the supporting RPC
interfaces

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4260 592f7852-d20e-0410-864c-8624ca9c26a4
fe/resources.py
igrations/049_test_planner_additions.py
lanner/execution_engine.py
lanner/execution_engine_control.srv
lanner/models.py
lanner/planner_test_utils.py
lanner/rpc_interface.py
lanner/rpc_interface_unittest.py
lanner/rpc_utils.py
lanner/rpc_utils_unittest.py
lanner/set_atomic_group_control.srv
dcfa6589ac6bd4f516bbd04358a2fb908e7cb151 19-Feb-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Update frontend/client_compliation_unittest.py to use new GWT compilation
tool.

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4257 592f7852-d20e-0410-864c-8624ca9c26a4
lient_compilation_unittest.py
cae88c6d6f2bd3c8b3ee46628e7230b3fcc444bc 19-Feb-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Have the description column in AFE admin interface for tests actually
say "Description"

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4251 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
35a70227e69e03d0492fe47608a6370dc339f287 16-Feb-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Fix styling of AFE admin interface.

The change_list.html for tests used to be handled as a customized
template. The template failed to extend the original change_list.html,
so it didn't keep up to date. Additionally, the "white-space:pre" style
was applied to all <td> elements instead of just the description.

This restricts that style change to the description only, and also
maintains most of the default Django admin interface.

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4247 592f7852-d20e-0410-864c-8624ca9c26a4
fe/admin.py
fe/models.py
emplates/admin/afe/test/change_list.html
3981f4450e25f75aa81844f7a45c5850d2444040 16-Feb-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> New, more consistent way of representing relationships in RESTful interface. A relationship between two entrys is represented as a relationship itself, contained in a collection. For example,

resources/labelings?host=myhost

retrieves all labelings for host myhost, each with a URI like

resources/labelings/myhost,mylabel

which represents that the label mylabel is applied to the host myhost. The user can POST to the former URI to relate two entries and DELETE the latter URI to break the relationship. This makes it much more consistent with the rest of the interface.

There are various other refactorings included in here that came up as I worked through this change to make things cleaner.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4246 592f7852-d20e-0410-864c-8624ca9c26a4
fe/resources.py
fe/resources_test.py
fe/urls.py
hared/query_lib.py
hared/resource_lib.py
32444c27d21894efa319826a91c4467f2042ffd8 12-Feb-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Fix debug paths in AFE URLs

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4237 592f7852-d20e-0410-864c-8624ca9c26a4
fe/urls.py
ac21e9983bd3c3095f17f22418abae458c03e60f 12-Feb-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Un-hardcode the TKO database name in AUTOTEST_WEB migration 45

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4236 592f7852-d20e-0410-864c-8624ca9c26a4
igrations/045_test_planner_framework.py
5a13d8d71bb1408341fd2f87284ae7e26ae613b0 12-Feb-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Add authorization headers to REST client requests

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4235 592f7852-d20e-0410-864c-8624ca9c26a4
hared/rest_client.py
3386534fd1e2d354cf983efb3a3ae9d9b4cd4540 12-Feb-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Change the "exclude only if needed labels" selector in AFE host tables to "exclude atomic groups". The "only if needed" labels feature is entirely deprecated, and there's demand for this new atomic groups filter.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4234 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/HostSelector.java
lient/src/autotest/afe/HostTableDecorator.java
1a2914a3501e0c719e2f77e1ce9ab4f9eb12067a 12-Feb-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Move the cli RPC authorization_headers code into rpc_client_lib.
Make autotest_lib.server.frontend use it for RPC authentication.

Signed-off-by: Gregory Smith <gps@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4231 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpc_client_lib.py
da9311af0530b175386a244dc072a38e308ada83 11-Feb-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Again, a mistake in merging. Line had been changed in file in new_tko/tko,
but was not persisted to frontend/tko.


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4230 592f7852-d20e-0410-864c-8624ca9c26a4
ko/graphing_utils.py
ko/preconfigs.py
b3dfe5fa841e185fcdab21d5dab85193db442076 04-Feb-2010 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> add missing __init__.py file



git-svn-id: http://test.kernel.org/svn/autotest/trunk@4215 592f7852-d20e-0410-864c-8624ca9c26a4
hared/__init__.py
6f80e7a07b277d54cbc88069122b12849fd4832e 04-Feb-2010 lmr <lmr@592f7852-d20e-0410-864c-8624ca9c26a4> Autotest: Porting all hash operations to use utils.hash

Replace all functions that evaluate md5 and sha1 hexdigests
with the function utils.hash().

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4204 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpc_utils.py
lanner/models.py
00b167c04566f968182b709ff8821503ca5b5055 03-Feb-2010 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Fix handling of "alt=json-html" parameter to rest client

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4198 592f7852-d20e-0410-864c-8624ca9c26a4
ettings.py
f46ad4c84892c3a88d61f76f4f07888be6798a61 03-Feb-2010 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> * make RESTful interface use absolute URIs throughout. there's little price to this and it makes the system more consistent, makes responses valid standalone data, simplifies the job of the clients, and could be useful in building services that operate across multiple autotest servers in the future.
* since generating abolute URIs requires the request object (we need to examine headers), there were some substantial refactorings to pass around the request object more widely.
* also fixing a little bug where the django exceptions module was being hidden.
* finally, refactored request code in rest_client a bit to improve testability

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4195 592f7852-d20e-0410-864c-8624ca9c26a4
fe/resources.py
fe/resources_test.py
hared/resource_lib.py
hared/rest_client.py
ed62e48ffbf9f4959c5352efa5d518c58b7accc5 03-Feb-2010 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Add a comment and a TODO about a hard-coded URL path

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4194 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/JobDetailView.java
08ff9adcaab55db7f4c35a875a93355d0226727a 28-Jan-2010 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Properly instantiate the JSON encoder and decoder.

Signed-off-by: John Admanski <jadmanski@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4175 592f7852-d20e-0410-864c-8624ca9c26a4
fe/json_rpc/proxy.py
656b3b8e1092a08bb60a059b09d1dbafadcbd037 28-Jan-2010 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Do simplejson imports in json_rpc lazily, so that the library is only needed
for actually making rpc calls, not just for importing the json_rpc module.

Also adjust the unittest suite runner to add the serviceHandler_unittest
to the long tests, since it requires a build externals run to actually work.

Signed-off-by: John Admanski <jadmanski@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4174 592f7852-d20e-0410-864c-8624ca9c26a4
fe/json_rpc/proxy.py
f828c77299496a78d8e7f5afe608f7e73851fbd0 25-Jan-2010 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> First version of new RESTful AFE interface. Includes a substantial library (under frontend/shared) and a definition of the interface for AFE (frontend/afe/resources.py).

If you want to see what this interface looks like, I encourage you to check out

http://your-autotest-server/afe/server/resources/?alt=json-html

>From there you can explore the entire interface through your browser (this is one of its great strengths).

For an introduction to the idea behind RESTful services, try http://bitworking.org/news/How_to_create_a_REST_Protocol.

This is still very much under development and there are plenty of TODOs, but it's working so I wanted to get it out there so we can start seeing how useful it turns out to be.

Signed-off-by: Steve Howard <showard@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@4165 592f7852-d20e-0410-864c-8624ca9c26a4
fe/model_logic.py
fe/resources.py
fe/resources_test.py
fe/urls.py
hared/common.py
hared/exceptions.py
hared/json_html_formatter.py
hared/query_lib.py
hared/resource_lib.py
hared/rest_client.py
tatic/jsonview.css
rls.py
85284636658da64563868be4a6106fc61b94d388 22-Jan-2010 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Fix Triage Failures link

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4158 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/JobDetailView.java
4d23375ff9b95e95e9f1d8443d20122f7270c1f4 20-Jan-2010 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Move logic for job reboot before/after defaults from RPC method to job model. Also adding a reference to the host protections enum in the Host model as a convenience for some other code I'm working on.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4157 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
fe/rpc_interface.py
fd8b89f0117366c9aeaad9b600a43238a84b4ab9 20-Jan-2010 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> don't set the current user to my_user in frontend_test_utils. let it default to the new autotest_system user.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4156 592f7852-d20e-0410-864c-8624ca9c26a4
fe/frontend_test_utils.py
fe/models_test.py
fe/rpc_interface_unittest.py
7e67b433965702c0ffd8205ac08f5e801d9f98a6 20-Jan-2010 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> New code for performing explicit joins with custom join conditions.
* added ExtendedManager.join_custom_field(), which uses the introspection magic from populate_relationships (now factored out) to infer the type of relationship between two models and construct the correct join. join_custom_field() presents a much simpler, more Django-y interface for doing this sort of thing -- compare with add_join() above it.
* changed TKO custom fields code to use join_custom_field()
* added some cases to AFE rpc_interface_unittest to ensure populate_relationships() usage didn't break
* simplified _CustomQuery and got rid of _CustomSqlQ. _CustomQuery can do the work itself and its cleaner this way.
* added add_where(), an alternative to extra(where=...) that fits more into Django's normal representation of WHERE clauses, and therefore supports & and | operators later

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4155 592f7852-d20e-0410-864c-8624ca9c26a4
fe/model_logic.py
fe/rpc_interface_unittest.py
ko/models.py
ko/rpc_interface_unittest.py
39637e34a99ac712366b15c80bd2110cc453c9c8 20-Jan-2010 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> In TKO, don't quote fields involving SQL functions. This was causing a bug when drilling down on function fields, introduced with recent changes to quote all fields when constructing the WHERE clause.

This approach is a bit of a hack, but doing a proper fix would be a lot of effort, and hopefully the whole mechanism of passing direct SQL conditions will be obsolete soon enough, making this whole thing moot.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4154 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/tko/HeaderField.java
a849ceba3b15091666ddec1630b1fb64635f45df 20-Jan-2010 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Fix error when accessing some pages of the admin interface

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4153 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpc_handler.py
fe/urls.py
lanner/views.py
rls_common.py
7909732c14de46239e07e59a07597b7029720520 20-Jan-2010 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> ensure TKO custom fields work with the "extra_info" feature in get_latest_tests. adding tests for get_status_counts and get_latest_tests while i'm at it -- they we're previously untestable, but i recently added a mechanism to implement "if" in sqlite, making them testable

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4152 592f7852-d20e-0410-864c-8624ca9c26a4
ko/rpc_interface.py
ko/rpc_interface_unittest.py
2edbd92187b87128ba9cab4d5272f266f6a0aeb1 20-Jan-2010 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Add site_urls back to TKO urls

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4150 592f7852-d20e-0410-864c-8624ca9c26a4
rls.py
12454c6398a2197f959bc1c2839b83981a9c35ca 15-Jan-2010 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Fix interactions with db_utils when simulating. Anyone attempting to
simulate a migration through AUTOTEST_WEB version 46 before this
change would actually drop the TKO database.

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4129 592f7852-d20e-0410-864c-8624ca9c26a4
igrations/046_merge_databases.py
c1a98d1e146080bd3e4f034cb13d740dfb1535f4 15-Jan-2010 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Support for job keyvals
* can be passed as an argument to create_job, stored in AFE DB
* scheduler reads them from the AFE DB and writes them to the job-level keyval file before the job starts
* parser reads them from the keyval file and writes them to the TKO DB in a new table

Since the field name "key" happens to be a MySQL keyword, I went ahead and made db.py support proper quoting of field names. Evetually it'd be really nice to deprecate db.py and use Django models exclusively, but that is a far-off dream.

Still lacking support in the AFE and TKO web clients and CLIs, at least the TKO part will be coming soon

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4123 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
fe/rpc_interface.py
fe/rpc_interface_unittest.py
igrations/047_job_keyvals.py
ko/models.py
82e1b48c44797fb0a240e1aa5b59b0cd6ce2b3ce 15-Jan-2010 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Correcting issues in AUTOTEST_WEB migration 45

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4122 592f7852-d20e-0410-864c-8624ca9c26a4
igrations/045_test_planner_framework.py
igrations/046_merge_databases.py
c510344e16543a6f810be1608e1a879c3d3bbcde 15-Jan-2010 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Slight change to SpecialTask scheduling code. This is to support my work on a new AFE interface.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4118 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
fe/rpc_interface.py
cfcdd80cb19eb28f593a79f9ba2e56496c3f9f3a 15-Jan-2010 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Fix creation of the autotest_system user

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4116 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
64a9595406f2884fb3ece241190b10aa054439a9 13-Jan-2010 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> When using Django models from a script, make the current user default to an actual database user named "autotest_system". This allows for simpler, more consistent code.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4114 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/001_rpc_test.txt
fe/models.py
fe/rpc_handler.py
fe/rpc_interface.py
hread_local.py
ko/rpc_interface.py
3adac3671a998be8421238d3a08a7ffd2c3cbe1c 13-Jan-2010 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Display the text required for filtering on custom fields in the TKO UI.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4113 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/common/table/DataTable.java
lient/src/autotest/public/afeclient.css
lient/src/autotest/tko/AttributeField.java
lient/src/autotest/tko/LabelField.java
lient/src/autotest/tko/ParameterizedField.java
lient/src/autotest/tko/ParameterizedFieldListDisplay.java
lient/src/autotest/tko/ParameterizedFieldListPresenter.java
250d84d0a9716ece8ec4eb50b3702a167cda038d 12-Jan-2010 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Merge the two databases and the two Django projects. Note that the TKO migrations are still present, as is the migration code for TKO in database/migrate.py. These pieces of code are preserved because they are required by this change to perform the merge. A separate change will be submitted in the future to remove those references.

WARNING: This change will move all tables in TKO into the AUTOTEST_WEB database. Custom TKO tables that Autotest does not know about will NOT be moved, and will be lost after this change. Any scripts that run directly against the TKO database will cease functioning until they are modified to run against AUTOTEST_WEB. Additionally, if you were importing any .py files from autotest_lib.new_tko.tko.*, they have been moved to autotest_lib.frontend.tko.*

Notes: You will need to manually create two files after applying this patch
touch <autotest_dir>/frontend/tko/__init__.py
touch <autotest_dir>/tko/migrations/__init__.py
Optionally, you can also remove the entire <autotest_dir>/new_tko/ directory, as it is no longer needed.

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4106 592f7852-d20e-0410-864c-8624ca9c26a4
igrations/046_merge_databases.py
lanner/models.py
ettings.py
ko/__init__.py
ko/common.py
ko/csv_encoder.py
ko/csv_encoder_unittest.py
ko/graphing_utils.py
ko/models.py
ko/preconfigs/metrics/kernel_compare
ko/preconfigs/metrics/perf
ko/preconfigs/qual/pre
ko/rpc_interface.py
ko/rpc_interface_unittest.py
ko/tko_rpc_utils.py
ko/urls.py
ko/views.py
rls.py
1c4d2394fbe949fef83661a25cfe9a32bf8569aa 12-Jan-2010 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Slightly more useful drilldown ordering for spreadsheet view. This avoids an "infinite loop" case where there's more than one result of the same test name in a single job tag.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4104 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/tko/SpreadsheetView.java
e9d4f7e6af6122ed69aa04c476186187612790f6 12-Jan-2010 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Made AFE doctest Y2K10-compliant

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4099 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/001_rpc_test.txt
669624aad9c77060f706fde7c8e717cb636f9be8 12-Jan-2010 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Fix rendering of empty cells in Job Details host table

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4097 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/common/table/DataTable.java
12b45582c04b2417036a6f11afc843ac5fddea50 11-Jan-2010 lmr <lmr@592f7852-d20e-0410-864c-8624ca9c26a4> Massive permission fix

Fix permissions for all the development tree

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4094 592f7852-d20e-0410-864c-8624ca9c26a4
lient/EmbeddedTkoClient-shell
igrations/009_add_timeout_to_jobs.py
lanner/models_test.py
2f081f66df76861923e5b931d68558aa4a259df5 11-Jan-2010 lmr <lmr@592f7852-d20e-0410-864c-8624ca9c26a4> Finish change meant for changeset:4088

Also get rid of the simplejson code in the autotest tree.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>

git-svn-id: http://test.kernel.org/svn/autotest/trunk@4091 592f7852-d20e-0410-864c-8624ca9c26a4
fe/simplejson/__init__.py
fe/simplejson/decoder.py
fe/simplejson/encoder.py
fe/simplejson/scanner.py
b012d2d9c7b4d530818f4747b30739f7561f7b54 11-Jan-2010 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Some people have raised questions about the GPLv2 vs v3 compatibility with
GWT etc. I'm not really convinced whether this is an issue or not, but
we decided easiest way out is to dual license this with apache.

Signed-off-by: Martin J. Bligh <mbligh@google.com>
Acked-by: showard@google.com



git-svn-id: http://test.kernel.org/svn/autotest/trunk@4090 592f7852-d20e-0410-864c-8624ca9c26a4
ICENSE
19841977ef4d476c48b7f62207f9c81183871f46 11-Jan-2010 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Remove the bundled version of simplejson on the autotest
tree, in order to be coherent with the general policy of
just requiring that the server admin install external
dependencies. For simplejson, it's not necessary to
maintain the module on our tree, since we are mostly
using a version very close to upstream 2.0.9.

Also, added simplejson to utils/build_externals.py, to
make installation straightforward in case one doesn't
want/can use the package shipped by the distro.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4088 592f7852-d20e-0410-864c-8624ca9c26a4
fe/json_rpc/proxy.py
fe/json_rpc/serviceHandler.py
fc3da5bbdcd63840cd224282812c050c5520db11 06-Jan-2010 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> More refactoring of the job state code to try and reduce duplication.
In the process this also refactors a bunch of job.run_test duplication,
specifically the handling of job tags (there's still lots more
duplicate code left).

The main state refactorings are:
* Storing client job steps in __steps and special-casing its removal
in autotest.py is gone. Instead the client job code stores it
directly in the client.steps state, and the autotest.py generically
removes the entire client.* namespace.
* Moving this out of the publicly-visible state and into a private
namespace allows us to drop has_state and discard_state, which
were really only needed so that we could make sure __steps was
not persisted across jobs.
* All of this client-job state managment was moved out of autotest.py
and into server_job.py. All the data this code was getting and
setting was in the job object, so it seemed like a better place
to put it. This also let us drop load_state and save_state, which
were really only there to make it easier to push and pull data
out of the server job object.
* Eliminate get_run_number and set_run_number in favour of a generic
use_sequence_number boolean property (that works on both client
and server automatically). The get_ and set_ code was only being
used to enable and disable an auto-incrementing sequence number to
be tagged to each test run, so we can just make the public
interface a boolean and handle the sequencing under the covers.
* Drop all the test_tag and test_tag_prefix code in favour of a
persistent automatic_test_tag property. Also drops the
show_kernel_in_test_tag flag in favour of just implementing this
in the control file generator using automatic_test_tag.
* Refactor all the test name tagging code into base_job, unifying
the tagging portion of the job.run_test code.

Signed-off-by: John Admanski <jadmanski@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4075 592f7852-d20e-0410-864c-8624ca9c26a4
fe/control_file.py
fe/doctests/001_rpc_test.txt
fe/doctests/003_misc_rpc_features.txt
1db39ae07d69fefa4fb431e5edc845d2ac85e01a 05-Jan-2010 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Fix up the 003_misc_rpc_features.txt doctest. For some reason svn rev
3916 didn't correctly apply the patch to it; the mailed patch included three
fixups, but only one was applied.

Signed-off-by: John Admanski <jadmanski@google.com>




git-svn-id: http://test.kernel.org/svn/autotest/trunk@4072 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/003_misc_rpc_features.txt
4608b005f15444d2ec4601b8274828ad52b5ea51 05-Jan-2010 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Add a new Archiving stage to the scheduler, which runs after Parsing. This stage is responsible for copying results to the results server in a drone setup, a task currently performed directly by the scheduler, and allows for site-specific archiving functionality, replacing the site_parse functionality. It does this by running autoserv with a special control file (scheduler/archive_results.control.srv), which loads and runs code from the new scheduler.archive_results module. The implementation was mostly straightfoward, as the archiving stage is fully analogous to the parser stage. I did make a couple of refactorings:
* factored out the parser throttling code into a common superclass that the ArchiveResultsTask could share
* added some generic flags to Autoserv to duplicate special-case functionality we'd added for the --collect-crashinfo option -- namely, specifying a different pidfile name and specifying that autoserv should allow (and even expect) an existing results directory. in the future, i think it'd be more elegant to make crashinfo collection run using a special control file (as archiving works), rather than a hard-coded command-line option.
* moved call to server_job.init_parser() out of the constructor, since this was an easy source of exceptions that wouldn't get logged.

Note I believe some of the functional test changes slipped into my previous change there, which is why that looks smaller than you'd expect.

Signed-off-by: Steve Howard <showard@google.com>

==== (deleted) //depot/google_vendor_src_branch/autotest/tko/site_parse.py ====


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4070 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpc_interface.py
7a8765140efbebb867f9b8ca7323a7d548db24ad 31-Dec-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Remove a redundant method declaration that I foolishly added in a recent change.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4068 592f7852-d20e-0410-864c-8624ca9c26a4
fe/model_logic.py
c2c56193bc6aab1a22e086eb73abbde7034ca3a8 30-Dec-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Add missing __init__.py.


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4065 592f7852-d20e-0410-864c-8624ca9c26a4
lanner/__init__.py
8b0ea2285c1327a686ff0b6ab245915e7fd20094 23-Dec-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Overhaul how we deal with related data in TKO -- test labels, test attributes, machine labels, and iteration results.

This has proven one of the trickiest areas of TKO. The first foray into this area was machine label headers, an early feature request implemented in a pretty ad-hoc manner in spreadsheet view which allowed them to be used as header fields. (Ironically, this was closest to the right approach on the server side, but I didn't appreciate it at the time. The original client-side implementation was a mess.) Next was filtering on test attributes and test labels, implemented with the include_labels, exclude_labels, include_attributes_where, and exclude_attributes_where options. This server-side implementation supported filtering but not viewing, grouping or sorting at all. Furthermore, even the filtering support was weak -- it only supporting ORing of inclusion requests and ANDing of exclusion requests. The client-side implementation was still pretty messy but was moving towards correctness. Finally, support was recently added for viewing iteration results in table view, but
grouping and filtering were excluded since they would've been very difficult to fit into the design. This was again a limited server-side approach, though the client-side implementation continued improving, albeit still using the trouble "generator items" in the mutliple list selector widget.

When I started working on support for test iterations and attributes in TKO table view, I finally hit upon the right server-side approach: specify the attributes that you're interested in, have the server perform a separate JOIN for each one, so that there's now a new column for each one, NULL if the attribute didn't exist and having the attribute's value if it did. Once it's created as a normal column, the user can do selection, grouping, sorting and filtering using the regular mechanisms. Everything just works. (For labels, it's slightly different, since whether or not a label is attached to a test is a boolean value. I opted to have the column's value be either NULL or the name of the label.)

Well, not quite perfectly. MySQL lets us define column aliases in a SELECT which are then usable in GROUP BY and ORDER BY. They aren't however, usable in the WHERE clause, because certain select expressions may not exist at the time the WHERE is applied. (Our expressions happen to be fine, but MySQL will have none of it.) There's absolutely no way I can see to define aliases for use in the WHERE clause. And unfortunately, our current interface allows users to provide a WHERE clause directly, so we can't perform translations or substitutions. As a result, filtering must be performed a little differently for these fields. You can't just say <field_name> = "<value>", like you can for most fields. For test attributes and iteration results, you say <field_name>.value = "<value>". For test labels and machine labels, you say <field_name>.id IS [NOT] NULL.

The first part of this CL is changing the server to use this new approach. get_test_views() now accepts test_label_fields, test_attribute_fields, machine_label_fields, and iteration_result_fields parameters, which allow the user to add extra fields based on these data types.

At the same time, I've changed how the TKO web clients deals with these data types in a way that mirrors the new way of handling these features on the server. There is now a global widget for adding custom fields based on any of the four data types. Once one is created, it can be used just like any other field in spreadsheet view, table view, and the global condition. This vastly simplifies most pieces of the code that previously dealt with these features, and it greatly expands the available space of features. Where we formerly had spreadsheet grouping/filtering on machine labels, table viewing of iteration results, and limited filtering on test labels and attributes, we now have viewing, grouping sorting, and filtering on all four.

High-level changes involved:

Server side
* added code to TestViewManager to handle the new options for creating fields, documented them, and documented that these options are supported and the rest are deprecated (we can probably delete them but we should check, they might be in use)
* added thorough unit tests for all of the above. on a side note, i discovered a neat feature of SQLite where you can add any function you've defined at a callable function from SQL statements. I used this to add some functions emulating MySQL-only functions. This could be used to good effect elsewhere, but this CL is big enough :)
* got rid of now-obsolete code for machine_label_headers option and iteration views

Client side:
* made HeaderFields immutable. Mutable HeaderFields turned out to be way too much of a nightmare. Users can specify values for ParameterizedFields at creation time, and if they want to modify them, they can delete and add.
* made all parts of the application (namely SpreadsheetView (both header selectors), TableView, and CommonPanel) use a single global HeaderFieldCollection
* changed ParameterizedFieldListPresenter to handle the new job of allowing creation and deletion of any kind of ParameterizedField. This new widget replaces the label/attribute filtering widget in the CommonPanel -- I got rid of all the code for that widget.
* removed the now-obsolete code for "generator items" in the MultiListSelectPresenter.
* finally made TableView use HeaderSelect. Since HeaderSelect plays a more significant role and it's role is more unified, it made sense to finally do this (TableView was previously duplicating logic from HeaderSelect, which was only used in SpreadsheetView). Since the HeaderSelect used in TableView is much simpler than the one used in SpreadsheetView, I extracted a new class SpreadsheetHeaderSelect, using composition rather than inheritance (it didn't really follow an is-a relationship).

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4049 592f7852-d20e-0410-864c-8624ca9c26a4
fe/model_logic.py
lient/src/autotest/common/SimpleChangeListener.java
lient/src/autotest/common/SimpleChangeListenerCollection.java
lient/src/autotest/common/Utils.java
lient/src/autotest/common/table/DataTable.java
lient/src/autotest/common/ui/MultiListSelectPresenter.java
lient/src/autotest/public/tkoclient.css
lient/src/autotest/tko/AttributeField.java
lient/src/autotest/tko/CommonPanel.java
lient/src/autotest/tko/ContentSelect.java
lient/src/autotest/tko/HeaderField.java
lient/src/autotest/tko/HeaderFieldCollection.java
lient/src/autotest/tko/HeaderSelect.java
lient/src/autotest/tko/HeaderSelectorView.java
lient/src/autotest/tko/IterationDataSource.java
lient/src/autotest/tko/IterationResultField.java
lient/src/autotest/tko/LabelField.java
lient/src/autotest/tko/MachineLabelField.java
lient/src/autotest/tko/ParameterizedField.java
lient/src/autotest/tko/ParameterizedFieldListDisplay.java
lient/src/autotest/tko/ParameterizedFieldListPresenter.java
lient/src/autotest/tko/SimpleHeaderField.java
lient/src/autotest/tko/SpreadsheetHeaderSelect.java
lient/src/autotest/tko/SpreadsheetHeaderSelectorView.java
lient/src/autotest/tko/SpreadsheetView.java
lient/src/autotest/tko/StringParameterizedField.java
lient/src/autotest/tko/TableView.java
lient/src/autotest/tko/TestAttributeField.java
lient/src/autotest/tko/TestLabelField.java
f713072bce3ed8fe082449baddf7b2ac2d74f2de 23-Dec-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Change URLconfs to collect results of calling patterns(), rather than collecting tuples and then passing them into patterns all at the end. Django documentation specifically states that it's OK to concatenate the result of patterns().
Also get rid of the from ... import * to conform to our coding style.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4042 592f7852-d20e-0410-864c-8624ca9c26a4
fe/urls.py
lanner/urls.py
rls.py
rls_common.py
78f5b016b5367cb51b1f031b31e3afea6ebd2d74 23-Dec-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Update to Django 1.1.1. I want to use a new feature for my RESTful interface prototyping (direct inclusion of URL patterns in URLconfs).

The one obstacle this presented was that Django 1.1.1 changes the DB connection object to accept DB config information in its constructor, rather than reading it from django.conf.settings on-demand. This was a problem because we change stuff in django.conf.settings on the fly to do our fancy test DB stuff -- basically, we initialize a SQLite DB once, copy it off, and then copy it between test cases, rather than clearing and reconstructing the initial DB. I did measurements and it turns out all that jazz wasn't really saving us much time at all, so I just got rid of it all. Django's testing stuff has improved and v1.1 even has some new tricks for using transactions to accomplish the above with a dramatic speedup, so we ought to look into using that in the future.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4041 592f7852-d20e-0410-864c-8624ca9c26a4
fe/frontend_test_utils.py
etup_test_environment.py
eab66ce582bfe05076ff096c3a044d8f0497bbca 23-Dec-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Rename the tables in the databases, by prefixing the app name. This is
in preparation for merging the two databases and the two Django
projects into one.

Note that this renames *all* standard Autotest DB tables in both the
autotest_web and tko databases. If you have scripts written directly
against these databases, *they will break*. If your scripts access
the RPC interfaces, they should continue to work.

Another patch will be along within the next few weeks to actually move
the TKO tables into the autotest_web database.

From: James Ren <jamesren@google.com>
Signed-off-by: Steve Howard <showard@google.com>

Rename the tables in the databases, by prefixing the app name. This is
in preparation for merging the two databases and the two Django projects
into one.

Note that this renames *all* standard Autotest DB tables in both the autotest_web and tko databases. If you have scripts written directly against these databases, *they will break*. If your scripts access the RPC interfaces, they should continue to work.

From: James Ren <jamesren@google.com>
Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4040 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
fe/rpc_utils.py
lient/src/autotest/public/EmbeddedTkoClientTest.html
lient/src/autotest/tko/DBColumnSelector.java
lient/src/autotest/tko/MachineQualHistogramFrontend.java
lient/src/autotest/tko/MetricsPlotFrontend.java
ake_superuser.py
igrations/044_rename_afe_tables.py
igrations/common.py
26b7ec787d8fc566294ca39f6ceb7dcf3395a6c2 21-Dec-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Note: This change is to support a project that I am working on. You should see no change in the behavior of your current Autotest installations.

-----

Implement the models and set up the RPC framework for the Test Planner

Signed-off-by: James Ren <jamesren@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@4039 592f7852-d20e-0410-864c-8624ca9c26a4
fe/frontend_test_utils.py
fe/model_logic.py
fe/rpc_utils.py
fe/urls.py
fe/views.py
igrations/045_test_planner_framework.py
lanner/common.py
lanner/models.py
lanner/models_test.py
lanner/rpc_interface.py
lanner/urls.py
lanner/views.py
ettings.py
rls.py
rls_common.py
iews_common.py
01339bf92db5b19f22f57cb4c0c3757ee0a11250 21-Dec-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Add new hostless param to the docstring -- missed this comment from gps in my previous change.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4037 592f7852-d20e-0410-864c-8624ca9c26a4
fe/frontend_test_utils.py
4e545a5ef263106c2d5316ad02c6c14b8d28e5b2 19-Dec-2009 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Adds a command line tool to reverify all Repair Failed hosts.
Also adds a convenience function to server.frontend to make this RPC call.
The reverify_hosts RPC now returns a list of hostnames sent to be
reverified so that the command line tool can print a list.

Signed-off-by: Gregory Smith <gps@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4028 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpc_interface.py
fe/rpc_interface_unittest.py
fbf73aecdd357094ae05e7d1e4ea99b1ecf93ee4 19-Dec-2009 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Refactor all of the persistent state managing code in the client job
class into the common_lib base job class. The major change is that all
the details of the job state implementation are pulled into a class
base_job.job_state. This class handles all the internal details of the
state managment that job.get_state and job.set_state did, and also
provides a mechanism for creating job properties which are automatically
backed by job state.

This also includes a major change to the semantics of state passed
between autoserv and any Autotest clients it ran. Previously the code
was a bit of a hack, with code in server/autotest.py that would
pre-generate a state file with some pre-set values, and code that would
pull back the state file after the client job and extract some values
from it, as well as some more code that would prepend a bunch of
state initialization calls at the start of the client control file.
Instead of all these hacks, autotest.py now just pushes all of the
state out to the client at the start, and pulls it all back at the end
(with a special exception for __steps that's difficult to eliminate at
this time). So any state set on the server is automatically available
on the client, and any state set on the client is automatically
available on the server.

There are a couple of outstanding issues:
* More of the set_state/get_state code needs to be refactored
to either use the automatic properties, or the job-internal state
API (so that all the internal state is not leaked into the "public"
state visible through get_state and set_state).
* The job_state class is perhaps somewhat overengineered. It's really
just a big, automatically persistect dict-of-dicts. But most of it
is actually code that was being used before, it was just scattered
all over server/autotest.py and client/bin/job.py so it was less
obvious how overgrownt he state managment is.

Risk: High
Visibility: Replaces a bunch of get_*+set_*+default_* methods with
persistent properties, refactors all of the state managment
into a single implementation consistently used in both the
server and client.

Signed-off-by: John Admanski <jadmanski@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4022 592f7852-d20e-0410-864c-8624ca9c26a4
fe/control_file.py
fe/doctests/003_misc_rpc_features.txt
a9545c0ab3d8f3e36efadaefdcf37393708666d9 18-Dec-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> backend support for hostless jobs
* support in rpc_interface.create_job() and models for creating a hostless job -- a job with one queue entry with no host, meta_host or atomic_group
* support in scheduler for recognizing and executing such a job. the bulk of the work was in extracting an AbstractQueueTask class from QueueTask, containing all the logic not pertaining to hosts. I then added a simple HostlessQueueTask class also inheriting from it.
Also got rid of HostQueueEntry.get_host() and added an extra log line when AgentTasks finish (used to be for QueueTasks only).

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4018 592f7852-d20e-0410-864c-8624ca9c26a4
fe/frontend_test_utils.py
fe/models.py
fe/rpc_interface.py
fe/rpc_interface_unittest.py
2ca64c940277d6ee38a084dc71fa8d3003aedddf 10-Dec-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> * add a couple simple test cases to the scheduler functional test for metahosts
* augment one of the logging lines in the scheduler

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4009 592f7852-d20e-0410-864c-8624ca9c26a4
fe/frontend_test_utils.py
d11956572cb7a5c8e9c588c9a6b4a0892de00384 08-Dec-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Make drone_manager track running processes counts using only the information passed in from the scheduler. Currently it also uses process counts derived from "ps", but that is an unreliable source of information. This improves accuracy and consistency and gives us full control over the process.

This involves a few primary changes:
* made the drone_manager track process counts with each PidfileId
* added method declare_process_count() for the scheduler to indicate the process count of a pidfile ID during recovery (in other cases, the DroneManager gets that info in execute_process())

Doing this involved some extensive refactorings. Because the scheduler now needs to declare process counts during recovery, and because the AgentTasks are the entities that know about process counts, it made sense to move the bulk of the recovery process to the AgentTasks. Changes for this include:
* converted a bunch of AgentTask instance variables to abstract methods, and added overriding implementations in subclasses as necessary
* added methods register_necessary_pidfiles() and recover() to AgentTasks, allowing them to perform recovery for themselves. got rid of the recover_run_monitor() argument to AgentTasks as a result.
* changed recovery code to delegate most of the work to the AgentTasks. The flow now looks like this: create all AgentTasks, call them to register pidfiles, call DroneManager to refresh pidfile contents, call AgentTasks to recover themselves, perform extra cleanup and error checking. This simplified the Dispatcher somewhat, in my opinion, though there's room for more simplification.

Other changes include:
* removed DroneManager.get_process_for(), which was unused, as well as related code (include the DroneManager._processes structure)
* moved logic from HostQueueEntry.handle_host_failure to SpecialAgentTask._fail_queue_entry. That was the only call site.
And some other bug fixes:
* eliminated some extra state from QueueTask
* fixed models.HostQueueEntry.execution_path(). It was returning the wrong value, but it was never used.
* eliminated some big chunks from monitor_db_unittest. These broke from the refactorings described above and I deemed it not worthwhile to fix them up for the new code. I checked and the total coverage was unaffected by deleting these chunks.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4007 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
fe/models_test.py
fe/rpc_interface_unittest.py
e299d4603ca85fb3fea2bfbac52c16c66475ac1f 23-Nov-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Do not allow empty global filter in Spreadsheet view or Table view.

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3970 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/tko/CommonPanel.java
lient/src/autotest/tko/ConditionTabView.java
lient/src/autotest/tko/SpreadsheetView.java
lient/src/autotest/tko/TableView.java
9bb960b90d5102cce1c8a15314900035c6c4e69a 19-Nov-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Support restricting access to drones by user. Administrators can put lines like

<hostname>_users: showard,scottz

in the global config, where <hostname> is a drone hostname. That drone will then be limited to use by those users (that is, by jobs launched by those users, and tasks launched due to those jobs). This required numerous changes:
* added a requested_by field to SpecialTask (with corresponding migration). For tasks with queue_entries, we can infer this from the job, but for those without, we need this information explicitly declared. Note this can be null if the task was created by the system, not in response to any user action. The only place this occurs now is in scheduler recovery (Dispatcher._recover_hosts_where()), but there may be an upcoming feature to periodically reverify hosts, which would be another (much more common) case.
* modified all SpecialTask creation sites to pass requested_by if necessary.
* modified AgentTask to keep a username attribute, and modified its run() method to pass that to PidfileRunMonitor.run(), which passes it along to DroneManager.execute_command().
* modified Agent to always keep self.task around, there's no reason to throw it away and now that we're looking at it from other classes, it's problematic if it disappears.
* modified Dispatcher throttling code to pass the username when requesting max runnable processes.
* added an allowed_users property to _AbstractDrone, and made DroneManager load it from the global config.
* made DroneManager's max_runnable_processes() and _choose_drone_for_execution() methods accept the username and obey user restrictions.
* added extensive tests for everything. the modiications required to monitor_db_unittest were annoying but not too bad. but parts of that file may need to be removed as they'll be obsoleted by monitor_db_functional_test and they'll become increasingly annoying to maintain.

couple other related changes:
* got rid of CleanupHostsMixin. it was only acutally needed by GatherLogsTasks (since we made the change to have GatherLogsTask always run), so I inlined it there and simplified code accordingly.
* changed a bunch of places in the scheduler that were constructing new instances of Django models for existing rows. they would do something like "models.Host(id=<id of existing host>)". that's correct for scheduler DBModels, but not for Django models. For Django models, you only instantiate new instances when you want to create a new row. for fetching existing rows you always use a manager -- Model.objects.get() or Model.objects.filter() etc. this was an existing bug but wasn't exposed until I made some of the changes involved in this feature.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3961 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
fe/models_test.py
fe/rpc_interface_unittest.py
igrations/043_add_special_task_requested_by.py
348594fb586abfedea0106b3188fa3a1f6932563 19-Nov-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> add a unique key over job_id and host_id on host_queue_entries. we don't want to depend on this from the frontend (since that'd result in half-created jobs), but it's a good safety net.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3960 592f7852-d20e-0410-864c-8624ca9c26a4
igrations/042_unique_index_on_hqe_job_and_host.py
09d80f925e7b7da5c1cced0ca1f9cc2aea3decd4 19-Nov-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Prevent duplicate hosts in job creation.
* have rpc_utils.create_new_job() explicitly check for duplicate hosts and throw a ValidationError, and add a test for this
* add a unique key over job_id and host_id on host_queue_entries. we don't want to depend on this from the frontend (since that'd result in half-created jobs), but it's a good safety net.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3959 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpc_interface_unittest.py
fe/rpc_utils.py
3f2110914421a50e509e619e6e876e0af096144c 19-Nov-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Fix the "swap" link in TKO spreadsheet view, at least partially. It was completely broken and now it works except when machine label headers are active. I did it in a way that moves towards a more correct design, by introducing a State class for the HeaderSelect presenter, which sort of acts as a model class. Nonetheless, I performed a minimal refactoring necessary to accomplish the goal cleanly. Possible next steps would be:
* introduce a State object for the ParameterizedFieldListPresenter, and make HeaderSelect.State include an instance of that state. This, if done properly, would fix the above-mentioned remaining bug with swapping.
* move functionality to the State object that belongs there. addHistoryArguments() and handleHistoryArguments are two examples. It could also use accessors/mutators for its fields.
* change the interface of HeaderSelect to get rid of all the state accessors/mutators. Rather, there should be one pair of getter/setter for the saved state. Clients should perform other actions on the State object itself.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3958 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/tko/HeaderSelect.java
lient/src/autotest/tko/SpreadsheetView.java
1651fda92951af1d75634d95edb6dced07fce0bd 19-Nov-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Improved job owner filtering making it easier to select "all users". Replaced the simple list box with a pair of radio buttons, one to select "all users" and the other to select from a list.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3957 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/AfeUtils.java
lient/src/autotest/afe/JobListView.java
lient/src/autotest/afe/JobOwnerFilter.java
lient/src/autotest/afe/RecurringView.java
lient/src/autotest/public/AfeClient.html
3696d5b67f34edbcdd4b2dd2596d96f5a9b0b9ac 13-Nov-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> File got left out of rev 3943.


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3947 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/models/Host.java
e60e44ece1445d97977a77cb79f0896989b869d7 13-Nov-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Special tasks show "Failed" as their status instead of "Completed" if
they failed

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3946 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
fe/models_test.py
igrations/041_add_special_task_success.py
4879914c122f4ed97eae3b08c5af1930fd75b39d 13-Nov-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Fix a bug where hosts in a cloned job weren't removed when deselecting the host from the "browse hosts" table. This again turned out to be a relatively large change for the size of the problem, because I chose to make some refactorings to attack some of the design problems the were behind the issue in the first place.

The core issue was that the ArrayDataSource was using a plain SortedSet<JSONObject> to hold the selected hosts. This is good because we want to keep them sorted, but it's bad because it uses the default object equality for JSONObjects, but we want to use a special equality based on ID or hostname. We normally accomplish that with JSONObjectSet, but that doesn't get us the sorting. JSONObjectSet is sort of a hack in the first place; the proper solution is to use custom objects that override equality.

As a first step towards that approach, I created a Host class under the models autotest.afe.models package. In order to minimize the scope of this change, I made it override JSONObject, so that we can move incrementally. I made it override equality and made HostDataSource convert the results to Host objects before returning them.

Unfortunately, I ran into a second problem. DataSource passes back results in a JSONArray. Since JSONArrays use native JS objects for storage, the objects you get out of them are always real JSONObjects, constructed on the fly -- not just declared type, but actual type. Even if you put Hosts in, they'll effectively be converted back to plain JSONObjects when you take them out. This is a reflection of a second design problem -- we shouldn't use JSONArrays to pass data around within the program. So I changed DataSource to convert results to List<JSONObject> and pass the data back that way. This required minor changes across many files, but nothing drastic anywhere.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3943 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/HostDataSource.java
lient/src/autotest/afe/HostDetailView.java
lient/src/autotest/afe/HostSelector.java
lient/src/autotest/afe/JobDetailView.java
lient/src/autotest/afe/JobStatusDataSource.java
lient/src/autotest/common/Utils.java
lient/src/autotest/common/table/ArrayDataSource.java
lient/src/autotest/common/table/DataSource.java
lient/src/autotest/common/table/DataTable.java
lient/src/autotest/common/table/DynamicTable.java
lient/src/autotest/common/table/DynamicTableSelectionManager.java
lient/src/autotest/common/table/RpcDataSource.java
lient/src/autotest/tko/IterationDataSource.java
lient/src/autotest/tko/SpreadsheetDataProcessor.java
lient/src/autotest/tko/TestDetailView.java
lient/src/autotest/tko/TestGroupDataSource.java
lient/src/autotest/tko/TestViewDataSource.java
b5611a81e109472e81e983da636d9580c3022b24 13-Nov-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> In HostDetailView, move code that fills in "type" on host queue entries to the data source itself, so that the objects always have the variable. Without this, the select all link was broken because it'd fetch items outside of the table but then expect this attribute to be present.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3942 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/HostDetailView.java
b9c6617bdc063b3b4aa760a0a45190ee069139fd 13-Nov-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Modify DynamicTableSelectionListener so that when select "none" is clicked, only deselect items matching the current filters. This turned out to be considerably trickier than i thought, because in order to do this properly and efficiently, it has to work like this: when "none" is clicked, make an RPC call for all the currently filtered items that fall within the currently selected set. We do this by making a new request to the table's active DataSource with the current filtering params plus a list of selected item IDs. There are two problems here:

1) Not all tables show DB objects -- in particular, the host detail view table when "show verifies/repairs/cleanups" is selected, and the TKO table view when grouping is active. For these tables, there's no way to filter on selected items by ID. Fortunately, these tables also happen to not be prone to the original issue, because they can't be filtered. So I added some code to only apply this new logic to tables that have active user-controlled filters. This is pretty ugly from a design point of view, but I had to take a practical approach -- it works, the code is clean, and I couldn't think of any workable alternatives.

2) The DataSource interface was poorly designed such that an RpcDataSource was stateful (and mutated every time a new request was made), so there was no way to make a new request on a DataSource without messing up its state and therefore potentially messing up the table using it. To get around this, I redesigned the interface to use a separate Query objects. Now, both the DataSource and the Query objects are immutable, and making a separate query for the "select none" operation is easy. (This actually also encouraged a change to DynamicTable to be much more efficient by avoiding many unnecessary get_num_*() calls.)

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3941 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/HostDetailView.java
lient/src/autotest/afe/HostSelector.java
lient/src/autotest/common/JSONArrayList.java
lient/src/autotest/common/table/ArrayDataSource.java
lient/src/autotest/common/table/DataSource.java
lient/src/autotest/common/table/DynamicTable.java
lient/src/autotest/common/table/DynamicTableSelectionManager.java
lient/src/autotest/common/table/Filter.java
lient/src/autotest/common/table/JSONObjectSet.java
lient/src/autotest/common/table/RpcDataSource.java
lient/src/autotest/common/table/SelectionManager.java
lient/src/autotest/tko/SpreadsheetDataProcessor.java
lient/src/autotest/tko/SpreadsheetView.java
lient/src/autotest/tko/TableView.java
lient/src/autotest/tko/TestGroupDataSource.java
lient/src/autotest/tko/TkoUtils.java
b584458d8d4c8df052dddb02fae033611a80c03d 10-Nov-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Removed uses of deprecated method ListBox.setMultipleSelect()

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3924 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/common/table/ListFilter.java
lient/src/autotest/common/table/MultipleListFilter.java
lient/src/autotest/tko/ExistingGraphsFrontend.java
d2ef40969bbfff6ed5803aec4dd39688f70c654c 10-Nov-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Removed use of deprecated method Event.cancelBubble()

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3923 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/tko/FilterStringViewer.java
232b7aecb0df153af9e6dca62bb6194acbfd709c 10-Nov-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Added a upload_kernel_config (default False) keyword parameter to
generate_control_file RPC that produces server side code to download and
transfer on the client the kernel config (if it's an URL) and point the
client to the new location. This allows to have kernel config URLs point
to servers that are reachable on the server side but not on the client
side.

Signed-off-by: Mihai Rusu <dizzy@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3920 592f7852-d20e-0410-864c-8624ca9c26a4
fe/control_file.py
fe/doctests/003_misc_rpc_features.txt
fe/rpc_interface.py
39c843ba954b8862f7da24a4533de2069eed48fc 10-Nov-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Do not overwrite the per kernel config_file value if it has been
provided by the RPC user.

Signed-off-by: Mihai Rusu <dizzy@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3919 592f7852-d20e-0410-864c-8624ca9c26a4
fe/control_file.py
fe/doctests/003_misc_rpc_features.txt
0d0f67d6abcf6599ad626042d3236020e093558f 06-Nov-2009 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Create a base_job class in the common library as a location to
refactor common code from client/bin/job.py and server/server_job.py
into.

Initially this just creates the base class and attempts to refactor
some of the __init__ code into it. It also tries to refactor the
code somewhat so that it's easier to test without mocking out large
amounts of external dependencies.

It also tries to standardize on what all the official publicly usable
attributes are, mostly on the basis of looking at what the existing
code actually does. In general my approach was:
* If used outside of the job class, assume it's public. If it makes
sense make it a requirement of the base class, but in some cases
(e.g. serverdir) it really only makes sense for one of the
concrete classes. In that case I define it as "OPTIONAL", which
means it can be None (but must still be defined).
* If only used within the job class, make it "private", which means
it gets a name starting with _. Code outside of the job modules
shouldn't be touching these. This includes the "constants" that
we define with all-caps in the class.
* I added unit tests which try to actually validate this, by
checking that all the correct properties are defined after
__init__. The test cases are still limited (all the test_init
cases) because of the pain of stubbing out __init__, but I will
be adding more in the future as I refactor to reduce the amount
of mocking/stubbing needed.

Overall, I don't know that this change by itself makes the code
simpler or easier to understand, but I don't think it makes things
any worse in that regard, while adding a lot more testing and
finally giving us a well-defined interface. I have more changes
planned, but as new improvements kept creeping into this change I
decided to mail it out for some review instead of letting it grow
even bigger. Some ideas of what I still intend to do in followup
patches:
* Refactor more of the directory management code into job_directory, to
get the rest of the mkdir/makedirs code into there. That way all
of the directory management is concentrated in one place.
* Refactor all the of get_state/set_state code into something that
automatically handles this with properties without a lot of
boilerplate getter and setter methods.
* Refactor the job.record related code that's duplicated into the
base class.

Risk: High
Visibility: Mostly code changes to standardize the job class interface.
A lot of the visible changes just involve privatizing attributes
that aren't used outside of job anyway, or adding in attributes
that were only defined on the client or server.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@3916 592f7852-d20e-0410-864c-8624ca9c26a4
fe/control_file.py
fe/doctests/001_rpc_test.txt
fe/doctests/003_misc_rpc_features.txt
eaa75e594adc241b99522e57354e9efc8ba30808 06-Nov-2009 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Fixed support for source kernels in generated control files (using the
generate_control_file() RPC) by adding a build() method call.
Implemented a dummy do nothing build() version in rpm_kernel (also
inherited by suse_rpm_kernel). Updated unittests.

Signed-off-by: Mihai Rusu <dizzy@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3909 592f7852-d20e-0410-864c-8624ca9c26a4
fe/control_file.py
fe/doctests/001_rpc_test.txt
fe/doctests/003_misc_rpc_features.txt
3c0ea96b0dbd2115fa2057c21635173eaf461f31 06-Nov-2009 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Added common_lib.hosts.Host.cleanup_kernels and some functions it's using.
Implemented a fast local version of those functions in LocalHost to not
go through the run() abstraction. Added call to cleanup_kernels() from
generated control file code so it will cleanup unused kernels (not
referenced by the bootloader) each time it's installing a new kernel.

Signed-off-by: Mihai Rusu <dizzy@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3907 592f7852-d20e-0410-864c-8624ca9c26a4
fe/control_file.py
fe/doctests/001_rpc_test.txt
fe/doctests/003_misc_rpc_features.txt
f541a0ae5e2fdb7e4f32422eb005ac2296bb8323 04-Nov-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Remove uses of deprecated class HistoryListener

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3897 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/common/CustomHistory.java
42d4d6221bed50117da283d7ad8917b5eaf414e3 28-Oct-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Expose locked_by and lock_time on AFE host detail page.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3889 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/HostDetailView.java
546c86118653c3044a84424de00ef8a069ac9a32 19-Oct-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Explicitly make the global config option rpc_num_old_logs an integer

Signed-off-by: Scott Zawalski <scottz@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3868 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpcserver_logging.py
d2b0c883cf9a8f98751d9bad684072cea85f2606 19-Oct-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Add almost everything necessary for UI support for including test attributes in table view.

This was almost straightforward. The one roadblock came with grouping support. I didn't bother including that for iteration results, but I did try to make it work here. Grouping is fine; unfortunately, grouping implies drilldown, and drilldown implies arbitrary filtering on test attributes, which requires a bit more sophisticated handling of which attribute fields we're requesting in each query (it's not just the ones we're viewing, but also any fields used in filtering). This isn't a terribly hard problem to solve, but doing it right will require some design changes, notably separating the fields in the view from the fields used in actual query, which will probably require making HeaderFields immutable to be done properly. I need to verify that this feature is high-enough priority to justify getting into all that, but I'd like to check in what I've got so far so I don't lose it.

To be clear, this doesn't actually add any user-visible changes yet.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3866 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/tko/HeaderField.java
lient/src/autotest/tko/IterationResultField.java
lient/src/autotest/tko/ParameterizedField.java
lient/src/autotest/tko/SimpleHeaderField.java
lient/src/autotest/tko/StringParameterizedField.java
lient/src/autotest/tko/TestAttributeField.java
97b7f5aa9e1c970d4db1ee25753db1325ff6b1db 14-Oct-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Create a compile script for our GWT clients. There are now 4 different bash
scripts to compile GWT clients. This has become unmaintainable and is a problem
when other scripts need to compile clients
* automatically enumerate clients available
* Quick compile all
* Compile individual clients
* Do not replace current client if compile fails
* Move current client to an old directory for a backup of a known working client

Signed-off-by: Scott Zawalski <scottz@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@3853 592f7852-d20e-0410-864c-8624ca9c26a4
lient/AfeClient-compile
lient/EmbeddedTkoClient-compile
lient/README.compile
lient/TkoClient-compile
lient/gwt_dir
d8bfa2880587a3e638ba512951a8b843e1eca533 14-Oct-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Add a control on the Create Job tab for the new profile_only option that jadmanski added to generate_control_file(). I opted to phrase it in the reverse -- "Run each test without profilers first", defaulting to true.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3850 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/CreateJobView.java
lient/src/autotest/public/AfeClient.html
91f8510c0fabcdbc3108034ce4d573a3f68fe567 12-Oct-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> NOTE: This depends on the (not yet committed) patches mailed out to add default_profile_only support to the job classes.

Add a new parameter to the control file generation RPCs to generate
control files that make use of the new set_default_profile_only
API.

Risk: Low
Visibility: Adds a new optional parameter to generate_control_file

Signed-off-by: John Admanski <jadmanski@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3829 592f7852-d20e-0410-864c-8624ca9c26a4
fe/control_file.py
fe/doctests/003_misc_rpc_features.txt
fe/rpc_interface.py
4cd4763a855d8eb7d25fd4963babc432eb4d25e6 12-Oct-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Remove deprecated uses of TableListener, and implement proper
oncontextmenu handling

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3828 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/HostDetailView.java
lient/src/autotest/afe/HostListView.java
lient/src/autotest/afe/HostSelector.java
lient/src/autotest/afe/JobDetailView.java
lient/src/autotest/afe/JobListView.java
lient/src/autotest/afe/RecurringView.java
lient/src/autotest/afe/TestSelector.java
lient/src/autotest/common/table/DataTable.java
lient/src/autotest/common/table/DynamicTable.java
lient/src/autotest/common/table/TableDecorator.java
lient/src/autotest/common/ui/RightClickTable.java
lient/src/autotest/tko/FragmentedTable.java
lient/src/autotest/tko/Spreadsheet.java
lient/src/autotest/tko/SpreadsheetView.java
lient/src/autotest/tko/TableView.java
7074b747cc3aa313ad4b70c036758f36dc4d84ae 12-Oct-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> i stupidly broke query limiting without a start limit in my previous change to model_logic. Should've checked the unit tests...

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3822 592f7852-d20e-0410-864c-8624ca9c26a4
fe/model_logic.py
8bfb5cb38c7daca2378cd5b278ca8b5b249123f7 07-Oct-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Add support for joining specific test attributes to the TKO get_test_view() method and friends (including grouping friends). This required a number of cleanups to the code in general. The two big changes were:
* making application of "presentation" arguments (sorting and paging) optional in query_objects() and separately applicables
* getting rid of the explicit passing of extra_select_fields in grouping code, and using Django query builtin extra selects instead (i.e. query.extra(select=...))

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3806 592f7852-d20e-0410-864c-8624ca9c26a4
fe/model_logic.py
d3771cc3c770639a816a6a3065aa5a1ef53a24a1 07-Oct-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> tiny change necessary to keep frontend/afe/rpc_interface_unittest happy

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3803 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
6c65d25764ad8a8e19fe9829048828101aec4acc 01-Oct-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> change logic for filtering queued, running and finished jobs in AFE to consider parsing jobs as running instead of queued.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3779 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpc_interface_unittest.py
fe/rpc_utils.py
afd97de523a8c8c4e6657fa2db6214fda68d8086 01-Oct-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> When a one-time host is added as a normal host while it's running a job, the status would get reset, messing up the scheduler. Fix that.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3778 592f7852-d20e-0410-864c-8624ca9c26a4
fe/model_logic.py
fe/models.py
fe/models_test.py
16ab5250da1e2e5fb1549774063a8099ce4fdbb8 29-Sep-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Bugfix to ensure generated items are added to the MultiListSelectPresenter when necessary. Since there's no nice way in general for the clients to know if they've already added a generated item, made MultiListSelectPresenter allow multiple additions of a generated item. This ugliness should go away with the refactoring to formally separate state from presenter (mentioned in a previous change). Also, this change fixes an immediate bug where "Triage failures" wouldn't work (fixed by the change to TableView), but I made a parallel change to HeaderSelect, bringing similar code bits more in line to eventually (soon?) be consolidated.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3771 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/common/ui/MultiListSelectPresenter.java
lient/src/autotest/tko/HeaderSelect.java
lient/src/autotest/tko/TableView.java
14444e7ca7923ead8fc52ae98cdb9dee416ab6c5 23-Sep-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Minor updates to new Create Job host selection UI
* make Browse Hosts the default view
* under Specify hostnames, add a checkbox "Allow hosts not in Autotest" to explicitly enable one-time hosts, deselected by default

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3762 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/CreateJobView.java
lient/src/autotest/afe/HostSelector.java
lient/src/autotest/afe/HostSelectorDisplay.java
cb6ae7dca0f7eff64e3c9e413f0c30e008a2a6ae 23-Sep-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Added ability to view iteration results in TKO Table View. The primary changes towards this end were:
* add IterationResultField, a ParameterizedField subclass for iteration results, and make TableView capable of generating and handling these fields
* add an IterationDataSource to make requests to get_iteration_views(), made TableView use this when iterations are selected
The feature is somewhat crude -- the name of the field is always something like "Iteration result 0" when it should really be something like "Iteration result: throughput" -- but I think it's a very functional first pass.

Unfortunately a lot of other changes were required, some quite ugly. This was due to two problems. First, TableView and HeaderSelect duplicate a lot of logic, but in slightly different ways. Unifying them will fix a lot of things. Second, there's a generally misguided use pattern field "names" and field "SQL names" in the code. That needs to be thought through and fixed.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3761 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/tko/HeaderField.java
lient/src/autotest/tko/HeaderFieldCollection.java
lient/src/autotest/tko/HeaderSelect.java
lient/src/autotest/tko/IterationDataSource.java
lient/src/autotest/tko/IterationResultField.java
lient/src/autotest/tko/MachineLabelField.java
lient/src/autotest/tko/ParameterizedField.java
lient/src/autotest/tko/ParameterizedFieldListPresenter.java
lient/src/autotest/tko/TableView.java
8064ff55bfb61b6f69120dd534677b410a371e0e 18-Sep-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Set invalid host_id references to a new __missing_host__ host before
adding the foreign key to host_queue_entries.

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3754 592f7852-d20e-0410-864c-8624ca9c26a4
igrations/040_add_foreign_keys.py
227a7a1d0484dcfa4c6d996a1c10e95437d059ef 18-Sep-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> OK, one more refactoring -- this time, make TableView use HeaderFields (something I've wanted to do for a long time, there has been a TODO at the top of TableView for ages) and consolidate a lot more logic. Part of this was in the introduction of a new HeaderFieldCollection class, the rest was simple consolidation due to common use of HeaderField.

TableView and HeaderSelect still duplicate a lot of logic, sadly, although it's not immediately obvious from the code at this point. Eventually I'll fully consolidate those, but at this point I think I'm ready to implement the iteration view feature that I've been working towards the whole time.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3753 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/common/ui/MultiListSelectPresenter.java
lient/src/autotest/tko/ConditionTestSet.java
lient/src/autotest/tko/HeaderField.java
lient/src/autotest/tko/HeaderFieldCollection.java
lient/src/autotest/tko/HeaderSelect.java
lient/src/autotest/tko/ParameterizedField.java
lient/src/autotest/tko/ParameterizedFieldListPresenter.java
lient/src/autotest/tko/SimpleHeaderField.java
lient/src/autotest/tko/SpreadsheetView.java
lient/src/autotest/tko/TableView.java
bd18ab7a821be9735d0805c81918a4534847c0fa 18-Sep-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Don't bother trying to validate uniqueness of autogenerated fields that have yet to be generated -- it doesn't make sense.
This should've been safe anyway - an autogenerated field would never be NULL, so the NULL value on a yet-to-be-added row should always be "unique". But there is apparently a Django/MySQLdb/MySQL bug where the following query can return 1:
SELECT COUNT(*) FROM WHERE .uid=67217(showard) gid=5000(eng) groups=7(lp),20(dialout),24(cdrom),25(floppy),29(audio),44(video),46(plugdev),60(games),499(google),5000(eng),5001(guest),5221(extmailproxy),5762(nonconf),6604(platformdocs),66815(kernel-testers),67686(autotest) IS NULL
This started happening when creating a job with a new one-time host. The query would only return 1 once, subsequent identical queries would return 0. That merits some further investigation, but this fixes the immediate problem.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3752 592f7852-d20e-0410-864c-8624ca9c26a4
fe/model_logic.py
7f2b0e15c5928ea2914d078e385ca717d078c6d5 17-Sep-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Removed deprecated uses of isChecked/setChecked.

(Depends on [PATCH] Removed deprecated ChangeListener uses.)

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3738 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/CreateJobView.java
lient/src/autotest/afe/HostDetailView.java
lient/src/autotest/afe/UserPreferencesView.java
lient/src/autotest/common/table/CheckboxFilter.java
lient/src/autotest/common/table/SelectionManager.java
lient/src/autotest/common/ui/RadioChooser.java
lient/src/autotest/tko/CommonPanel.java
lient/src/autotest/tko/ExistingGraphsFrontend.java
lient/src/autotest/tko/FilterSelector.java
lient/src/autotest/tko/MetricsPlotFrontend.java
lient/src/autotest/tko/SeriesSelector.java
lient/src/autotest/tko/SpreadsheetView.java
lient/src/autotest/tko/TableView.java
0d92da0fe19a095fc5678c4159e6a1756df65e48 17-Sep-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Removed deprecated ChangeListener uses.

(Depends on [PATCH] Removed the uses of some deprecated classes)

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3737 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/CreateJobView.java
lient/src/autotest/afe/JobListView.java
lient/src/autotest/afe/LabelFilter.java
lient/src/autotest/afe/TestSelector.java
lient/src/autotest/common/table/DynamicTable.java
lient/src/autotest/common/table/Filter.java
lient/src/autotest/common/table/ListFilter.java
lient/src/autotest/common/ui/Paginator.java
lient/src/autotest/tko/ContentSelect.java
lient/src/autotest/tko/FilterSelector.java
lient/src/autotest/tko/FilterStringViewer.java
lient/src/autotest/tko/GraphingView.java
lient/src/autotest/tko/MetricsPlotFrontend.java
lient/src/autotest/tko/PreconfigSelector.java
lient/src/autotest/tko/SavedQueriesControl.java
lient/src/autotest/tko/SeriesSelector.java
lient/src/autotest/tko/SpreadsheetView.java
5f0ca29fe6e34b4e5c256f6331e6767ab2b67c16 17-Sep-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Fix URL encoding of CSV export link.

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3736 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/tko/TkoUtils.java
79a7b0d387aac103fc1d125353eefa361030452a 17-Sep-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Removed the uses of the following deprecated classes:

DisclosureEvent
DisclosureHandler
FocusListener
KeyboardListener
TabListener
PopupListener
SuggestionEvent
SuggestionHandler
ScrollListener
WindowResizeListener

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3735 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/CreateJobView.java
lient/src/autotest/common/table/LinkSetFilter.java
lient/src/autotest/common/table/SearchFilter.java
lient/src/autotest/common/ui/ContextMenu.java
lient/src/autotest/common/ui/CustomTabPanel.java
lient/src/autotest/common/ui/DetailView.java
lient/src/autotest/common/ui/TableActionsPanel.java
lient/src/autotest/tko/ExistingGraphsFrontend.java
lient/src/autotest/tko/FilterStringViewer.java
lient/src/autotest/tko/Spreadsheet.java
lient/src/autotest/tko/SpreadsheetView.java
lient/src/autotest/tko/TestDetailView.java
8aa84fcb18489265208f291a7fc3c33799696eae 16-Sep-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> make invalid hosts viewable in the AFE view host tab

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3729 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpc_interface.py
fe/rpc_interface_unittest.py
fe/rpc_utils.py
lient/src/autotest/afe/HostDetailView.java
a29e963afc9de0ab086b5b2bea2e43ff68d81038 14-Sep-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Fix a ConcurrentModificationException bug in MultiListSelectPresenter than appears when drilling down after a machine label field was selected, and remove an obsolete comment.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3714 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/common/ui/MultiListSelectPresenter.java
2b50af17e5fa602821a8917bbb9fa1ddb1176ee3 14-Sep-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Refactoring to extract and generalize all logic used for machine label headers. All the UI and presentation logic was tied closely to HeaderSelect, but the forthcoming feature to view iterations in table view will require the exact same logic in a different context. So in this change I extracted a generic superclass for MachineLabelField, ParameterizedField, and extracted all the relevant UI logic from HeaderSelect to a separate presenter, ParameterizedFieldListPresenter.

This is hopefully the last refactoring necessary before actually implementing the iteration view feature. And ideally, the test attribute view feature will follow on easily from that.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3712 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/tko/HeaderField.java
lient/src/autotest/tko/HeaderSelect.java
lient/src/autotest/tko/HeaderSelectorView.java
lient/src/autotest/tko/MachineLabelField.java
lient/src/autotest/tko/ParameterizedField.java
lient/src/autotest/tko/ParameterizedFieldListDisplay.java
lient/src/autotest/tko/ParameterizedFieldListPresenter.java
902c96c119f8862724916f37f1f6f71db4d90483 11-Sep-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> whoops, need to change both places that check the rpc_logging
variable. instead, just eliminate the duplication.

Signed-off-by: Steve Howard <showard@google.com>

whoops, need to change both places that check the rpc_logging variable. instead, just eliminate the duplication.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3710 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpc_handler.py
fe/rpcserver_logging.py
a5e4d84c9616b0a987e904d3b1d1d3fd9ca1be86 11-Sep-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Removed deprecated ClickListener uses.

Mostly trivial changes. The only significant change was in
autotest.common.ui.SimpleHyperlink.

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3709 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/AbortSynchronousDialog.java
lient/src/autotest/afe/CreateJobView.java
lient/src/autotest/afe/HostDetailView.java
lient/src/autotest/afe/HostSelector.java
lient/src/autotest/afe/JobDetailView.java
lient/src/autotest/afe/RecurringView.java
lient/src/autotest/afe/UserPreferencesView.java
lient/src/autotest/common/table/CheckboxFilter.java
lient/src/autotest/common/table/SearchFilter.java
lient/src/autotest/common/table/TableClickWidget.java
lient/src/autotest/common/ui/CustomTabPanel.java
lient/src/autotest/common/ui/DetailView.java
lient/src/autotest/common/ui/Paginator.java
lient/src/autotest/common/ui/SimpleDialog.java
lient/src/autotest/common/ui/SimpleHyperlink.java
lient/src/autotest/common/ui/TableActionsPanel.java
lient/src/autotest/common/ui/TableSelectionPanel.java
lient/src/autotest/common/ui/ToggleLink.java
lient/src/autotest/tko/CommonPanel.java
lient/src/autotest/tko/ContentSelect.java
lient/src/autotest/tko/DynamicGraphingFrontend.java
lient/src/autotest/tko/ExistingGraphsFrontend.java
lient/src/autotest/tko/FilterSelector.java
lient/src/autotest/tko/FilterStringViewer.java
lient/src/autotest/tko/GraphingFrontend.java
lient/src/autotest/tko/MetricsPlot.java
lient/src/autotest/tko/MetricsPlotFrontend.java
lient/src/autotest/tko/SavedQueriesControl.java
lient/src/autotest/tko/SeriesSelector.java
lient/src/autotest/tko/SpreadsheetView.java
lient/src/autotest/tko/TableView.java
lient/src/autotest/tko/TestLabelManager.java
lient/src/autotest/tko/WidgetList.java
eaa408e59e932ce0540aa5d50bf966af5080f640 11-Sep-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Add atomic group job support to why_isnt_my_job_running to be able to
tackle the frequently asked questions regarding forever queued jobs.

Signed-off-by: Gregory Smith <gps@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3707 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
2df3c69dfb02a69d64f93aeb6cc7887ca032ff27 11-Sep-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> fix jsonp_fetcher, it was broken by changes to ServiceHandler in previous patch to implement RPC logging

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3706 592f7852-d20e-0410-864c-8624ca9c26a4
fe/json_rpc/serviceHandler.py
ce7c092dd0d866eb0c81eb795e9372267e04c7f9 11-Sep-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Added checking to RPC modify_host() to fail if locking/unlocking has
been requested on an already locked/unlocked host. Updated frontend
doctests.

Visibility: Since this RPC seems to be used only by the CLI (the web
interface uses modify_hosts() RPC) the change will be visible to the CLI
when you try to lock/unlock hosts that are already locked/unlocked.

Signed-off-by: Mihai Rusu <dizzy@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3705 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/001_rpc_test.txt
fe/rpc_interface.py
fe/rpc_utils.py
578619189b91e8ab1ab3bfbe4b2a43127dbb122c 09-Sep-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> fix disabling of rpc_logging from global_config. when reading the rpc_logging config var, need to pass type=bool or it'll parse as the string "False" which, being non-empty, evaluates to True

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3680 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpcserver_logging.py
5deef7fb0ae83271dbd43f4f381d4cdbe5fa6480 09-Sep-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> * added import_site_module() to other import_site_*() methods in utils
* made site_rpc_interface.py optional using import_site_module()
* made import_site_*() methods fail with an exception when a site file exists but isn't readable

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3679 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpc_handler.py
fe/views.py
02ed4bd47471fe77270e5507a7f9759955b3753c 09-Sep-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Reorganization of the host selection UI in the AFE create job view.
* refactor HostSelector into view/presenter (except for tables, which will need to be separately refactored)
* reorganize view into a tabbed format with a unified hostname input tab, a metahost tab and a browse hosts tab, with the selected hosts table alongside as before

Minor changes:
* extracted SimplifiedList interface from MultiListSelectPresenter to a top-level entity
* made ExtendedListBox implement the SimplifiedList interface and got rid of the SimplifiedListWrapper decorator class
* rewrote ArrayDataSource to use a TreeSet so it's actually efficient. this should speed up working with large host selections.
* edited JSONObjectComparator to be consistent with equals() for safety when using with TreeSet
* small change to how widths get set on TabPanels in AFE. rather than setting 100% width on all tab panels' contents using CSS, set it explicitly in CustomTabPanel. there's a quirk with browser layout where a 100% width div inside a table cell will have it's width set larger than the cell containing it. it happens on IE6, FF3 and Chrome so it's probably not something that's going to change. It does go away with the "border box" model, which can be enabled with nonstandard CSS attributes, but it's easy enough to work around here that that's not worth it.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3673 592f7852-d20e-0410-864c-8624ca9c26a4
fe/json_rpc/serviceHandler.py
fe/rpc_handler.py
fe/rpcserver_logging.py
lient/src/autotest/afe/CreateJobView.java
lient/src/autotest/afe/HostSelector.java
lient/src/autotest/afe/HostSelectorDisplay.java
lient/src/autotest/common/table/ArrayDataSource.java
lient/src/autotest/common/table/JSONObjectComparator.java
lient/src/autotest/common/ui/CustomTabPanel.java
lient/src/autotest/common/ui/DoubleListSelector.java
lient/src/autotest/common/ui/ExtendedListBox.java
lient/src/autotest/common/ui/MultiListSelectPresenter.java
lient/src/autotest/common/ui/SimplifiedList.java
lient/src/autotest/public/AfeClient.html
lient/src/autotest/public/common.css
lient/src/autotest/tko/HeaderSelectorView.java
1624542756819af6ac07685eb2a4c3cfb71d21b6 08-Sep-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Fix a name resolution error when trying to delete an attribute does not exist.

Signed-off-by: Jiqing Tang <jiqingtang@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3670 592f7852-d20e-0410-864c-8624ca9c26a4
fe/model_logic.py
e463a8a926964673c2517bfef5ac394206b106c6 08-Sep-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Add default drilldown behavior. Drilldown from new fields (like job queued day) was broken because they weren't added to the drilldown map; it's best to just have a default and not allow things to break like that.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3669 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/tko/SpreadsheetView.java
03ce7332ad69618c5955fcafc5b7670b6b868567 08-Sep-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> update GWT run scripts/launch configs for GWT 1.6+

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3668 592f7852-d20e-0410-864c-8624ca9c26a4
lient/AfeClient-compile
lient/AfeClient-shell
lient/AfeClient.launch
lient/EmbeddedTkoClient-compile
lient/EmbeddedTkoClient-shell
lient/EmbeddedTkoClient.launch
lient/TkoClient-compile
lient/TkoClient-shell
lient/TkoClient.launch
lient_compilation_unittest.py
d9e04c1a7950691cc348e70fa2470f8c414ae94f 08-Sep-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Refactor code related to double list selecting widgets used in both SpreadsheetView and TableView. This will facilitate code reuse and extension for the feature to include test iterations/attributes in table view. It will also make unit testing possible and provides cleaner organization of the code.

The major effort here is to refactor the DoubleListSelector and HeaderSelect classes according to the Passive View pattern. There are passive view classes for the double list itself (DoubleListSelector, which will be renamed in a future CL to DoubleListSelectDisplay) and for the spreadsheet header selection UI (HeaderSelectorView, which I think I'll also rename to HeaderSelectorDisplay). There is a new class, MultiListSelectPresenter, which incorporates the presentation logic from DoubleListSelector as well as some of the logic from HeaderSelect related to switch between a single and double list. The remaining code in HeaderSelect was modified to use a MultiListSelectPresenter as much as possible, retaining only the code specific to the spreadsheet header selection (and the management of machine label headers will likely be extracted and generalized too, since that sort of thing will be necessary for test attributes/iterations). Finally, small modifications to Spreadsheet
View and TableView were required to have them use the new interfaces.

This change also introduces the ToggleLink widget, consolidating logic that had been applied in multiple places previously.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3667 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/common/ui/DoubleListSelector.java
lient/src/autotest/common/ui/MultiListSelectPresenter.java
lient/src/autotest/common/ui/ToggleControl.java
lient/src/autotest/common/ui/ToggleLink.java
lient/src/autotest/tko/HeaderSelect.java
lient/src/autotest/tko/HeaderSelectorView.java
lient/src/autotest/tko/MachineLabelField.java
lient/src/autotest/tko/SpreadsheetView.java
lient/src/autotest/tko/TableView.java
8cbaf1e2984c7c5f7df877ac6a759ccef13aa63a 08-Sep-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Disallow modification of the Everyone ACL from the CLI. Also removed
Everyone from the Django admin interface.

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3666 592f7852-d20e-0410-864c-8624ca9c26a4
fe/admin.py
fe/models.py
c1c1caf3a47c8659218aeaf92c07598d591ea39b 08-Sep-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Added afe_job_id column to the jobs table in TKO. Changed link in AFE
View Job tab to use afe_job_id instead of job_tag.

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3665 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/JobDetailView.java
8cc058f50a46976e0a446aa3054f7f2349d6291a 08-Sep-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Make scheduler more stateless. Agents are now scheduled only by the
dispatcher, and agents no longer use in-memory state to remember
multiple tasks. All state is managed by the database.

Risk: high (large scheduler change)
Visibility: medium (scheduler restarts are now more stable)

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3664 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
fe/models_test.py
fe/rpc_interface.py
6aaab2e8cd8428456959b1aeeb1eebccb4908048 03-Sep-2009 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Fixed frontend and CLI for space delimited multiple kernel versions.

Signed-off-by: Mihai Rusu <dizzy@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3643 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/CreateJobView.java
f13f498e7d63595d0dffe2301d186c4845d6f411 31-Aug-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> diff --git a/frontend/client/src/autotest/public/afeclient.css b/frontend/client/src/autotest/public/afeclient.css
index dc4d377..4025568 100644


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3633 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/public/afeclient.css
lient/src/autotest/public/common.css
lient/src/autotest/public/standard.css
lient/src/autotest/public/tkoclient.css
8ac6ae54e4e6bee9f8721d97d38713e5dee30f63 31-Aug-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Added some foreign key constraints to AUTOTEST_WEB.

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3631 592f7852-d20e-0410-864c-8624ca9c26a4
igrations/040_add_foreign_keys.py
9e494cc00c53f9dd0fabec3ef675cb874a9130ad 31-Aug-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Change how we construct the basic page layout in AFE and TKO. The TabView class was sneaking around with RootPanels and DOM elements, effectively doing what HTMLPanel was intended to do in a naughty way. GWT 1.5 was naive enough to let that go on behind its back (although it wasn't safe), but GWT 1.6 won't have any of it (read: assertion errors). So change TabView to use an HTMLPanel properly, and change all the views to construct their layouts the new, nice way.

More info, in an old email from Joel Webber: http://markmail.org/message/xliklhgn5vvuibm4#query:%22A%20widget%20that%20has%20an%20existing%20parent%20widget%20may%20not%20be%20added%20to%20the%20detach%20list%22+page:1+mid:xliklhgn5vvuibm4+state:results

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3630 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/CreateJobView.java
lient/src/autotest/afe/HostDetailView.java
lient/src/autotest/afe/HostListView.java
lient/src/autotest/afe/HostSelector.java
lient/src/autotest/afe/JobDetailView.java
lient/src/autotest/afe/JobListView.java
lient/src/autotest/afe/RecurringView.java
lient/src/autotest/afe/UserPreferencesView.java
lient/src/autotest/common/Utils.java
lient/src/autotest/common/ui/DetailView.java
lient/src/autotest/common/ui/TabView.java
lient/src/autotest/tko/CommonPanel.java
lient/src/autotest/tko/GraphingView.java
lient/src/autotest/tko/SpreadsheetView.java
lient/src/autotest/tko/TableView.java
lient/src/autotest/tko/TestDetailView.java
3370e75d01859f756f96b7a68dce457f02d8ea8b 31-Aug-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Fixed re-adding of previously deleted host and labels from the Django
admin interface.

Risk: low
Visibility: medium (bug fix)

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3626 592f7852-d20e-0410-864c-8624ca9c26a4
fe/admin.py
9bb33ecab4a6e147e295e1df40ffaa155a5c7276 31-Aug-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> handy-dandy function to print all SQL queries executed during a unit test, useful for debugging failing tests (you can call it from tearDown)

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3624 592f7852-d20e-0410-864c-8624ca9c26a4
etup_test_environment.py
c47804079f4154e9c4084584386a8cbdebbde925 31-Aug-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Add RPCs for requesting iteration results from TKO and supporting model code.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3623 592f7852-d20e-0410-864c-8624ca9c26a4
fe/model_logic.py
6631273af8b88842cbd6202cc4615daf050cc957 27-Aug-2009 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Make a bunch of stuff executable



git-svn-id: http://test.kernel.org/svn/autotest/trunk@3621 592f7852-d20e-0410-864c-8624ca9c26a4
fe/json_rpc/serviceHandler_unittest.py
fe/models_test.py
fe/rpc_interface_unittest.py
lient_compilation_unittest.py
rontend_unittest.py
ake_superuser.py
c9b6d11c55bc266d684e10abd0eb151213d538c4 25-Aug-2009 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> fix logo

Signed-off-by: Martin J. Bligh <mbligh@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@3619 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/public/header.png
0d9b58fcab62475b297a67d42457b522df29b824 25-Aug-2009 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> New logo image.

Signed-off-by: Lucas Meneghel Rodrigues



git-svn-id: http://test.kernel.org/svn/autotest/trunk@3618 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/public/header.png
01d6329ddd2b854a2531d432d6083a59c85f9358 25-Aug-2009 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Update spinner gif.

Signed-off-by: Martin J. Bligh <mbligh@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@3616 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/common/ui/NotifyManager.java
lient/src/autotest/public/mbligh_spinner.gif
lient/src/autotest/public/spinner.gif
a3c58d20be69a292bdfafbea735c7503f4995b4c 25-Aug-2009 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Added support to specify cmdline for the booted kernels for both the CLI
and the frontend. For the CLI there is a new "--kernel-cmdline" option
that will be used for all the kernels configured to be booted with the
"-k" parameter. For the frontend there's a new textbox under the kernel
version one where cmdline can be provided.

Signed-off-by: Mihai Rusu <dizzy@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3599 592f7852-d20e-0410-864c-8624ca9c26a4
fe/control_file.py
fe/doctests/001_rpc_test.txt
fe/doctests/003_misc_rpc_features.txt
fe/rpc_interface.py
lient/src/autotest/afe/CreateJobView.java
lient/src/autotest/public/AfeClient.html
be0d8692a68c879c4d03a39789bf249b8178114e 21-Aug-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Remove the ability for host status to be modified via the frontend.
This interferes with the monitor_db scheduler as it owns the field.

Signed-off-by: Gregory Smith <gps@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3584 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/001_rpc_test.txt
fe/rpc_interface.py
fe/rpc_utils.py
87cc38ffa581c0b82e6e69ffd8538db15f9f393a 21-Aug-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Add the ability to exclude hosts in an atomic group from the get_hosts RPC.

Signed-off-by: Gregory Smith <gps@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3581 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpc_interface.py
fe/rpc_interface_unittest.py
fe/rpc_utils.py
a5a72c9d41d83f244533892d409c10b58398c1d5 21-Aug-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Change populate_relationships() for many-to-one relationships to pull only the base object ID, and not a whole instance of the base model, from each related model. Pulling the whole base model involves an unnecessary DB query each time, since the initial query for related models already includes all the base model IDs, and that's all we're interested in. I don't know if this was new with Django 1.0 or not, but it was massively slowing down populate_relationships() for large queries, commonly encountered with get_detailed_test_views.

My test query went from ~55s to ~2s with this change.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3579 592f7852-d20e-0410-864c-8624ca9c26a4
fe/model_logic.py
5ce8fa84161b4ee0bd27a50c93bdf1083b39441a 21-Aug-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Frontend unittest fix.

Signed-off-by: Gregory Smith <gps@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3578 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/003_misc_rpc_features.txt
65974e1a3971c9f60106102495fa205fbd298aea 21-Aug-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Remove invalidated records from the select lists in the admin interface.
Ex: Hosts edit page no longer shows invalidated labels in the "Labels"
selection boxes.

Risk: low
Visibility: medium (bug fix)

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3577 592f7852-d20e-0410-864c-8624ca9c26a4
fe/admin.py
1fe34c4612727e711818f1414072149f685e364e 21-Aug-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Add options to reinstall hosts from the Host List tab and the View Host
tab in AFE.

To use this feature, you must implement the machine_install() function of the
Host class in <autotest>/server/hosts/base_classes.py. Also, you must add the
test in <autotest>/server/tests/reinstall/control to the frontend with the name
"Re-install Machine".

Risk: low
Visibility: medium (UI change)

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3576 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/AfeClient.java
lient/src/autotest/afe/AfeUtils.java
lient/src/autotest/afe/HostDetailView.java
lient/src/autotest/afe/HostListView.java
lient/src/autotest/public/AfeClient.html
474d136e9a71f288f294167112dda909f9273434 21-Aug-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Sanity check: prevent the creation of a special task for a host that already
has an active special task. This should only ever happen due to a scheduler
bug. Dying to allow human cleanup is desirable today.

Signed-off-by: Gregory Smith <gps@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3571 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
fe/models_test.py
8177fbc25a28881eb25fe0d5acd906ecb33f6f72 21-Aug-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Add a "(triage failures)" link to the AFE Job Details tab.

Risk: low
Visibility: medium (UI change)

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3567 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/JobDetailView.java
lient/src/autotest/public/AfeClient.html
eb0fd4c76ee70e68d907e542423ae249489552c2 21-Aug-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Clicking any part of the row in the hosts table in AFE View Job tab
should take us to the AFE View Host tab. Previously, the right side of
the row would do nothing upon click if the "Status Log" and "Debug Log"
links were not present.

Risk: low
Visibility: low

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3566 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/JobDetailView.java
lient/src/autotest/common/table/DataTable.java
f2f84cd56026a86fe0c4ec7acae558c1ece26016 03-Aug-2009 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Do not blindly set Host entries to 'Ready' in the database when adding a one
time host. If the entry already exists and has a verify/repair/cleanup task
running on it, this will circumvent that and allow things to run before that
task is finished. Newly created one time hosts are status='Ready' by
default. Existing ones will reach 'Ready' or 'Repair-Failed' once
their current task finishes.

This means that a one time host that enters 'Repair-Failed' -after- a
new job was scheduled on it will stuck until the user Clones and
Resubmits the job. Annoying and a bug of its own, but much better than
triggering the scheduler's "one task per host" assertion or allowing
multiple things to run at once on a host.

Signed-off-by: Gregory Smith <gps@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3488 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
1ef218db76c473c28627377d8f50d6e6c6743289 03-Aug-2009 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> This is the result of a batch reindent.py across our tree.
As Martin pointed out, we ought to be more careful and
create a pre-svn commit script to avoid inserting trash
in the tree, meanwhile, this is a good start to cleanup
things

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3487 592f7852-d20e-0410-864c-8624ca9c26a4
fe/frontend_test_utils.py
fe/model_logic.py
fe/rpc_utils.py
rls.py
7eacbc2a36d4efc50b67a429bf7b2f0cc0155b9a 29-Jul-2009 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Fixed multiple kernel control file generation for server side tests for
the case where the original test control file gives a "tag" argument to
run_test() by changing the server_job default test_tag attribute to be
instead a tag prefix.

Signed-off-by: Mihai Rusu <dizzy@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3465 592f7852-d20e-0410-864c-8624ca9c26a4
fe/control_file.py
fe/doctests/003_misc_rpc_features.txt
7890e797d3549b3de18b79448817ffc80a79ee42 28-Jul-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Fix models.host_or_metahost_name to not die when called on a HostQueueEntry
with no host or meta_host because it is a queued atomic_group job.

Signed-off-by: Gregory Smith <gps@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3461 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/001_rpc_test.txt
fe/models.py
2a61b4ec31c4ced4e592aa20627e9997dfdd2c02 28-Jul-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Some fixes to history handling of label and attribute filters in CommonPanel. The separation between state and view wasn't being properly handled.

I think the whole state/view refactoring is moving slowly towards this model-view-presenter thingy I saw at Google I/O. When the need arises we should go ahead and make that a clean cut to avoid issues like this. Increased open-in-new-window support will probably be the feature to drive that change (it was the motivation behind the original separation).

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3460 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/tko/CommonPanel.java
9e8ac0dd59c7bc2e5200643e1f841d8643e49d95 28-Jul-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Remove the "View Status/Debug Logs" link from the AFE View Host tab, as
it is no longer required with the new Cleanup/Verify/Repair tracking.

Risk: low
Visibility: medium (UI change)

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3458 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/HostDetailView.java
lient/src/autotest/public/AfeClient.html
a5288b4bb2b09aafe914d0b7d5aab79a7e433eaf 28-Jul-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Upgrade from Django 0.96 to Django 1.0.2.

Risk: high (framework change)
Visibility: medium

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3457 592f7852-d20e-0410-864c-8624ca9c26a4
fe/admin.py
fe/doctests/001_rpc_test.txt
fe/doctests/003_misc_rpc_features.txt
fe/feeds/feed.py
fe/json_rpc/proxy.py
fe/json_rpc/serviceHandler.py
fe/management.py
fe/model_logic.py
fe/models.py
fe/models_test.py
fe/readonly_connection.py
fe/rpc_handler.py
fe/rpc_interface_unittest.py
fe/rpc_utils.py
fe/test.py
fe/urls.py
fe/views.py
pache_auth.py
lient/src/autotest/afe/HostDetailView.java
lient/src/autotest/afe/HostTableDecorator.java
anage.py
ettings.py
etup_test_environment.py
emplates/admin/afe/test/change_list.html
rls.py
752383196594a228f95384e11b8e5b669ba682ee 28-Jul-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Make paginators left-aligned instead of center-aligned in AFE and TKO. Center-alignment was annoying sometimes because tables could be really wide or could vary in width with each page.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3456 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/common/table/TableDecorator.java
907f2159b2605a4b330ee76dbafc99c8bda4bca0 28-Jul-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> AFE Create Job now runs tests in the order in which they were selected.

Risk: low
Visibility: low

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3455 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/TestSelector.java
1a5a408d993bd406b58395904a4bcfe2c993cce7 28-Jul-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> add get_special_tasks() RPC, useful for scripts looking at repairs on a machine etc

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3452 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpc_interface.py
fe/rpc_interface_unittest.py
97446887819594f1e2a329dcff289ee8e934b626 21-Jul-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Log changes to SpecialTask active/complete state.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3426 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
fe/models_test.py
f60b136cacbc62927100280b00c6fcde764d14d5 16-Jul-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Fix "Graph" button on graphing frontend. Was disabling and not
re-enabling after submitting a query with no global filter.

Risk: low
Visibility: low

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3421 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/tko/DynamicGraphingFrontend.java
aa3c687e0c87cd7323054fbb1064b448728da7e7 16-Jul-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Access special task logs through retrieve_logs.cgi so it'll redirect to drones if necessary.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3420 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/HostDetailView.java
8fb1fde951dcfbdaf8330ed3dd8ba4b2472f4db7 11-Jul-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Ensure IDs are unique in interleave_entries. This fixes a little bug
because gather_unique_dicts depends on unique IDs.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3410 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpc_utils.py
1ab1f14f90c3088fabdf4b23f69a68318b509904 11-Jul-2009 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> remove zero sized file

Signed-off-by: Martin J. Bligh <mbligh@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@3391 592f7852-d20e-0410-864c-8624ca9c26a4
fe/site_rpc_interface.py
c0ac3a79cceec87873257a8b6d41d19c9fb02ec6 08-Jul-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> add the option to include verify/repair/cleanups in the job table on the host detail page.
* added RPC get_host_queue_entries_and_special_tasks, which returns both HQEs and SpecialTasks for a host in a carefully interleaved manner.
* added appropriate frontend unit tests
* added support to HostDetailView to include these entries, but have them not be selectable. this required changes to SelectionManager. I also added a utility method to Utils for opening a new window, and changed all sites that do that to call the new method.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3385 592f7852-d20e-0410-864c-8624ca9c26a4
fe/model_logic.py
fe/models.py
fe/models_test.py
fe/rpc_interface.py
fe/rpc_interface_unittest.py
fe/rpc_utils.py
lient/src/autotest/afe/HostDetailView.java
lient/src/autotest/common/Utils.java
lient/src/autotest/common/table/DynamicTable.java
lient/src/autotest/common/table/SelectionManager.java
lient/src/autotest/common/table/SimpleFilter.java
lient/src/autotest/common/table/TableDecorator.java
lient/src/autotest/common/ui/TabView.java
lient/src/autotest/tko/EmbeddedTkoClient.java
lient/src/autotest/tko/ExistingGraphsFrontend.java
lient/src/autotest/tko/TkoUtils.java
0228f1407cce41f6567b7200a4fed7be6f4b5511 08-Jul-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Add (atomic group) to the displayed text of any atomic group label in
selectors within the GUI.

Signed-off-by: Gregory Smith <gps@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3383 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/AfeUtils.java
909c914c55da05961c46621da856fecc73168ccb 07-Jul-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Return the atomic group name as part of the RPC interface get_hosts() call.

Signed-off-by: Gregory Smith <gps@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3382 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/001_rpc_test.txt
fe/rpc_interface.py
fe/rpc_utils.py
ed2afea4ca6e23a82d20d1f2ee1067d0c25a8cc2 07-Jul-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> make SpecialTasks recoverable. this involves quite a few changes.
* run tasks in determined dirs instead of temp dirs. the dir paths look like hosts//-, for example, hosts/myhost/4-verify. the ID comes from the SpecialTask DB row. this allows us to find the pidfile when we go looking for it during recovery, and it makes it simple to find the logs for any given special task, much like for HostQueueEntries. added SpecialTask.execution_path() for this purpose, and added models_test to test it.
* added execution_path() to HostQueueEntry to match the interface of SpecialTask, allowing for more polymorphism, and changed most call sites to use it.
* since we're running in these dirs, copy the full results back in these dirs, instead of just copying a single log file.
* move process recovery code up into AgentTask, so that all AgentTasks can share the same generic process recovery code.
* change SpecialTask recovery code to do process recovery.
* change VerifyTask handling of multiple pending verify requests for a machine. instead of updating all the requests, just delete all other tasks. they're not specially tracked in any way so it's simplest to just delete them.
* made special tasks get marked is_active=False when they complete, to be consistent with HQEs

other changes:
* added null=True to SpecialTask.time_started definition
* made EmailManager.enqueue_notify_email always log the message, and removed explicit logging calls from call sites
* added feature to DroneManager.execute_command() to automatically substitute the working directory into the command. this avoids some duplicate information being passed around and simplifies the unit test.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3380 592f7852-d20e-0410-864c-8624ca9c26a4
fe/frontend_test_utils.py
fe/models.py
fe/models_test.py
igrations/039_remove_special_tasks_log_file.py
a2c80bd73f486fbc14d3c1dfc0dfa0487561aa54 06-Jul-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Allow "installing" of GWT under the site-packages directory where other
externals are installed by utils/build_externals.py (now that
build_externals knows how to fetch and install a copy of gwt there).
Defaults to the old /usr/local/lib/gwt behavior.

Signed-off-by: Gregory Smith <gps@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3376 592f7852-d20e-0410-864c-8624ca9c26a4
lient/gwt_dir
6157c63947d2d628d187a084acb0a48473af1c79 06-Jul-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Make the scheduler robust to finding a HostQueueEntry with more than one
atomic group label. Log a detailed error message and continue rather than
bailing out with a SchedulerError.

Signed-off-by: Gregory Smith <gps@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3373 592f7852-d20e-0410-864c-8624ca9c26a4
fe/frontend_test_utils.py
2fe3f1df42f5fd1dc6296219df289851dcf77025 06-Jul-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Enter all Verify/Cleanup/Repair tasks into the special_tasks table. Also
keep track of which Host Queue Entry (if any) each Verify/Cleanup/Repair
task belongs to. Additionally, implement recovery for jobs in
Verify/Cleanup/Repair (i.e., do not simply reverify the host and requeue
the job).

Risk: medium (scheduler changes)
Visibility: medium (functionality change)

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3372 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
fe/rpc_interface.py
igrations/038_add_hqe_id_and_foreign_keys_to_special_tasks.py
3ed34fd6fa69b0d1faba7b67ab32225b63b64781 06-Jul-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Expand the test name column in the AFE Create Job tests list to fit the
table, if the test names are all too short to reach the end of the table
column.

Risk: low
Visibility: medium (UI bug fix)

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3371 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/TestSelector.java
lient/src/autotest/common/table/DataTable.java
b4c1dfea53be3b10694cb1a10a13dc4c8719e39a 06-Jul-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Change control file font to fixed-width on the AFE Job Details view.

Risk: low
Visibility: medium (UI change)

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3370 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/JobDetailView.java
lient/src/autotest/public/common.css
a984ad1d988fe5035df315f58cbe89e0bab77071 02-Jul-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Remove duplicate "sync count" line from Django admin interface's edit
page for Tests.

Risk: low
Visibility: low

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3352 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
204e17e072fbd527a338e3416092bd2b2426963a 30-Jun-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> This code was still depending on the old deleted common_lib/debug.py

Signed-off-by: Gregory Smith <gps@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3333 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/001_rpc_test.txt
b12eb059d53b6bcdd68ec093b4196cc450854cda 30-Jun-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> fix unit tests for previous frontend/afe/control_file.py change.

Signed-off-by: Gregory Smith <gps@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3331 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/003_misc_rpc_features.txt
a5512cdc604c8b9091e9a75bed7da05811e9575a 30-Jun-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Fix missing import error in kernel control file install stanza. It was
about to raise an error anyways so as it is, it still bails out but the
error message isn't the desired nice one.

Signed-off-by: Gregory Smith <gps@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3330 592f7852-d20e-0410-864c-8624ca9c26a4
fe/control_file.py
e9450c934d860f154175abfee19389bf1a114305 30-Jun-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Change the default for the max_number_of_machines in atomicgroups to be infinite*. (* see definition of INFINITE_MACHINES).
The common use for rack tests is to want all machines in the group to be used.

Signed-off-by: Gregory Smith <gps@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3328 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
fe/rpc_interface.py
6cff4ad7aa317bab05ab9dc1249dbea652944750 30-Jun-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Fix FindBugs flags

Risk: low
Visibility: low

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3327 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/common/JsonRpcProxy.java
lient/src/autotest/common/table/JSONObjectComparator.java
c873032a61c06fb21e5caf9466b412321e2d5a49 30-Jun-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Infer the atomic group automatically by looking at the meta host when
creating a job.

This means that the -G atomicgroupname parameter on cli/atest is unnecessary
when giving an atomic group label name directly.

It also makes it possible to schedule onto atomic group labels via the GUI
rather than getting an error message.

Signed-off-by: Gregory Smith <gps@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3324 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/001_rpc_test.txt
fe/rpc_interface.py
fe/rpc_utils.py
cb881ae6105a841389d8239e12c23561cc762e29 30-Jun-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Currently, the 'view all logs' link is less usable as it is pointing to the /results/JOB/HOST/ directory while the
Test Details view refers to specific test.
Adding a new link - 'view test logs' - pointing to the specific test result dir (e.g. /results/JOB/HOST/TEST/).

Signed-off-by: Dror Russo <drusso@redhat.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3323 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/tko/TestDetailView.java
2d7ac830df51784968f0befec33f6417dc08007a 22-Jun-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> fix little-used RPC documentation handler in AFE/TKO

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3306 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpc_handler.py
f6853c0cbbbff801306dac5299f5ffd86fd11ade 22-Jun-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Fix collapsable control file in AFE Job Detail view. Element was
duplicating on refresh.

Risk: low
Visibility: medium (UI bug fix)

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3304 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/JobDetailView.java
98ead171ab61b324414dd0316478c5ce72bbd4b1 22-Jun-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> exempt one-time-hosts from ACL checking in the frontend as well, and update the unit test to accurately test this

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3303 592f7852-d20e-0410-864c-8624ca9c26a4
fe/frontend_test_utils.py
fe/models.py
130ba9e35163d5b5e1fc5767b3b0975d40bc0a6f 19-Jun-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> point drop-down test log viewers to newly named logs, and tweak the name of the autoserv logs to be a bit more accurate

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3300 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/tko/TestDetailView.java
2924b0ac9e0ca35e2cd45a23b60ecfc204360c44 19-Jun-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Ensure one-time-hosts aren't in the Everyone ACL, and make the scheduler ignore this.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3299 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
fe/rpc_interface_unittest.py
f1175bb2edbe3215f4bfa117c6b93d2e090d1d07 17-Jun-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> get rid of some now-obsolete code
* client.common_lib.debug
* abstract_ssh.LoggerFile

Changed {SSHHost,ParamikoHost}.run() default args for std{out,err}_tee to a constant instead of None. This preserves the existing behavior (teeing to the logging module by default), but allows callers to explicitly avoid this teeing if desired, by passing stdout_tee=None.

Signed-off-by: Steve Howard <showard@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@3293 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
415dc21b70d44616803cb5c0245248514b73adca 15-Jun-2009 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Allow a couple of machines failing to install a kernel in the server
test kernel install stanza when >4 machines are supplied. This allows
for large multi-machine tests to proceed even if some of the hosts die
while failing to install or boot the kernel.

This is not the ideal implementation. We should really have some way
to get the max(sync_count) required by a control file's step_test()
before calling the function. Ideas welcome.

Signed-off-by: Gregory Smith <gps@google.com>

(this patch breaks a unittest. the fix is the patch i am mailing immediately after this, sorry. 2 mails is easier than merging them into one patch before mailing for me)


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3284 592f7852-d20e-0410-864c-8624ca9c26a4
fe/control_file.py
4cfdce15cca1bc4988ee4531e49f3da4d18848ea 15-Jun-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> change client-side logging to split files by severity and move them to the debug dir. this is quite a bit more involved than the server-side change because of some buggy directory clearing code and various places that depended on the old configuration.

This also changes the TKO autoserv log viewers to point to the new .DEBUG and .ERROR autoserv logs.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3266 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/tko/TestDetailView.java
b7ad096c69ddd9a11a470954d051053d3106eac8 15-Jun-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Don't mess with the www directory when running GWT code verification.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3264 592f7852-d20e-0410-864c-8624ca9c26a4
lient_compilation_unittest.py
f77862cb4c75fb91ec06db14f9fe68aac5368e4b 15-Jun-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Moved the notification box to the top-center of the browser, and changed
the background color to yellow (a la Gmail). Also removed the DOCTYPE
strings from the HTML files, as they were causing problems with GWT
hosted mode.

Risk: low
Visibility: medium (UI change)

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3263 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/common/ui/NotifyManager.java
lient/src/autotest/public/AfeClient.html
lient/src/autotest/public/TkoClient.html
lient/src/autotest/public/afeclient.css
42572b5d1971839c55de92367c1443fda2dd8562 15-Jun-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Make control file hidden by default on the AFE Job Detail view.

Risk: low
Visibility: medium (UI change)

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3262 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/JobDetailView.java
lient/src/autotest/public/AfeClient.html
lient/src/autotest/public/common.css
d86debefb7364d0f4631bb6772594642a5da2782 10-Jun-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> fix a typo in generate_control_file that was messing up the check for no tests but not erroring due to a funny coincidence

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3242 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpc_interface.py
6d7b2ff05b2232b1b225a4cb3521d76c0152cad9 10-Jun-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Redesign the reverify hosts feature. Host status is no longer changed
from the frontend; scheduler deals with all status changes.

Risk: medium (scheduler behavior change)
Visibility: low

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3238 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
fe/rpc_interface.py
igrations/036_add_special_tasks.py
bc93f0f9e11345d39ac655612e8697a2e07ed70c 10-Jun-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Make launching jobs on atomicgroups more user friendly:
Allow atomic group names to be specified on create_job as meta_hosts without
demanding that they be supplied in the atomic_group_name parameter.

Signed-off-by: Gregory Smith <gps@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3237 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/001_rpc_test.txt
fe/rpc_interface.py
77182562edaaeeffcb98f48a7236a727136aa8ec 10-Jun-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Have the scheduler wait a configurable amount of time before starting
atomic group jobs as soon as minimum synch count hosts are available
in Pending state up until AtomicGroup.max_number_of_hosts are available.

Adds a DelayedCallTask class to monitor_db along with logic in the Job
class to use this to delay the job becoming ready to run for a little
while as well as making sure the job is run at the end of the delay
without needing to wait for another host to change state.

Signed-off-by: Gregory Smith <gps@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3236 592f7852-d20e-0410-864c-8624ca9c26a4
fe/frontend_test_utils.py
557419494536e8820706979298b6f6cde427e902 09-Jun-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Show host protection level in host detail view.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3229 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/HostDetailView.java
lient/src/autotest/public/AfeClient.html
194a59d6976a5dbd5b464fb2b14ceb089f91c050 09-Jun-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Modify TKO Spreadsheet View's "Export to CSV" function to handle
selection of cell content. Also fixed history token handling to handle
content value customization.

Additionally, fixed history token handling of the "latest only latest
test per cell" checkbox in the TKO Spreadsheet View.

Risk: low
Visibility: medium (functionality change)

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3228 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/tko/ContentSelect.java
lient/src/autotest/tko/SpreadsheetView.java
25aaf3f6c159bc00233d9180e4d724e53cbfc53c 09-Jun-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Minor change to get_status_counts() to ensure all jobs have status counts. This only makes a difference when there's a job with no HQEs, which is an invalid case anyway, but we might as well handle it as smoothly as possible.

Signed-off-by: Steve Howard <showard@google.com>

Autotest@test.kernel.org
http://test.kernel.org/cgi-bin/mailman/listinfo/autotest


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3225 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
defe53e2f33de8afebb1f1ad76fc4cdbf8a24d36 09-Jun-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> add DB constraints to all M2M pivot tables to enforce unique rows and enforce foreign key relationships. cleanup the DB beforehand to ensure these constraints will be applicable. this is by far the most complex migration to date, but it appears to work just fine.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3224 592f7852-d20e-0410-864c-8624ca9c26a4
igrations/037_db_constraints.py
51f73a242561febf5a47365528d787ab8894677d 08-Jun-2009 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Who knew how much damage a little bang could do :)

Signed-off-by: Travis Miller <raphtee@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3206 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/tko/CommonPanel.java
01a5167f13d9788c9f359ebba31358e329c98ebc 29-May-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Have the scheduler check for and sometimes cleanup various DB inconsistencies.
* in periodic cleanup, check for relationships to invalidated objects, and remove them (and send notification email)
* in 24hr cleanup, check for hosts with != 1 platform, and send notification email
Also changed AFE models to have labels remove associations with tests (as dependencies) when deleted (invalidated).

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3185 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
844960a5d0a96d6a03296c4267d6295e4d479919 29-May-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> make the readonly connection fallback to the regular Django connection when running in the scheduer. this is really important, because otherwise the readonly connection is not autocommit and bad, bad things could happen, though i'm not sure exactly what existing problems there might have been. we used to do this only for testing, but since we do it in another context here, i renamed the method to be more generic and appropriate.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3183 592f7852-d20e-0410-864c-8624ca9c26a4
fe/readonly_connection.py
etup_test_environment.py
8765816bbc4055458169db9c9bf9977322172acc 29-May-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Creating a job from the command line now inherits reboot before/after
settings specified in the AFE User Preferences.

Risk: low
Visibility: medium (minor functionality change)

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3182 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpc_interface.py
d9642c07837d093b13bc5c32132874f8ecce5d5d 29-May-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Allow hosts to be reverified from the AFE Host Details view.

Risk: low
Visibility: medium (UI change)

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3181 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/AfeUtils.java
lient/src/autotest/afe/HostDetailView.java
lient/src/autotest/afe/HostListView.java
lient/src/autotest/public/AfeClient.html
cafd16e33f658f63fcbe09b0f232bbeb354ae3c2 29-May-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Check for multiple platforms on a machine when modifying labels.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3179 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
fe/rpc_interface.py
fe/rpc_interface_unittest.py
b6d1662e18d756483d5fd81f4057cae4ef62152c 26-May-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> fix JobManager.get_status_counts, which was returning incorrect counts in some cases when jobs were aborted. the problem was that it's possible for a complete entry to have aborted set or not and have the same full status, which was violating an assumption of the method.

to test it, instead of adding stuff to the doctests (which would be messy in this particular case, since we need to reach in and mess with HQE stauses), i instead started a new rpc_interface_unittest, which seems to be the way of the future. since it shared a bunch of logic with the scheduler unit test (which also depends on setting up a fake AFE database), i extracted common logic into frontend/afe/frontend_test_utils.py. i also fixed up some of the logic extracted from monitor_db_unittest for reusing an initial DB between tests.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3177 592f7852-d20e-0410-864c-8624ca9c26a4
fe/common.py
fe/frontend_test_utils.py
fe/models.py
fe/rpc_interface_unittest.py
77401f351bd4ef6b6af99e46a9f905b161062574 26-May-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Allow all TestView fields to be displayed in cell contents in
Spreadsheet View.

Risk: low
Visibility: medium (UI change)

Signed-off-by: James Ren <jamesren@google.com>

http://test.kernel.org/cgi-bin/mailman/listinfo/autotest


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3175 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/common/Utils.java
lient/src/autotest/public/TkoClient.html
lient/src/autotest/tko/ContentSelect.java
lient/src/autotest/tko/SpreadsheetDataProcessor.java
lient/src/autotest/tko/SpreadsheetView.java
lient/src/autotest/tko/StatusSummary.java
lient/src/autotest/tko/TableView.java
e529d0fe5667f9796c7a6d607322fbfdf6a85cd2 26-May-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Add ability to lock/unlock a host from the AFE View Host tab.

Risk: low
Visibility: medium (UI change)

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3172 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/AfeUtils.java
lient/src/autotest/afe/HostDetailView.java
lient/src/autotest/afe/HostListView.java
lient/src/autotest/public/AfeClient.html
30680e4190269a00dd7aaa888f9379e3b0af5088 26-May-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Added DOCTYPE declarations to HTML files. Using HTML 5 because the GWT
history support iframe requires the id to start with an underscore.

Risk: low
Visibility: low

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3169 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/public/AfeClient.html
lient/src/autotest/public/TkoClient.html
c2ba43512fd68695f26e3729fe5d11b631f12e29 20-May-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Failing to distinguish between Types of Filters in CommonPanel.handleHistoryArguments().

Signed-off-by: Travis Miller <raphtee@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3151 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/tko/CommonPanel.java
276f9443bbfc12c781d86f51dab5d8988e2090f5 20-May-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Added ability to lock/unlock machines from the AFE Host List view

Risk: low
Visibility: medium (UI change)

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3150 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/001_rpc_test.txt
fe/rpc_interface.py
lient/src/autotest/afe/AfeUtils.java
lient/src/autotest/afe/HostListView.java
68693f79df6953d7bc0fe8de8e13e5fd501a2aed 20-May-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> populate_relationships() wasn't working for IterationAttributes and IterationResults in TKO, because they don't have primary key columns. the "right" fix would be to simply add primary key ID columns. unfortunately, that can take forever since it's a sort of intensive operation and iteration_result is a big table. so this change reworks how populate_relationships() works. it now handles many-to-one relationships separately, using a single query over the related table that doesn't depend on a primary key. also extended the unit test to cover this case, which required some annoying changes to account for this shortcoming of these tables.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3148 592f7852-d20e-0410-864c-8624ca9c26a4
fe/model_logic.py
69ec4b9133094caa9c1f80a0794ee2eb23efe4b8 16-May-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Change color of message of the day in AFE and TKO.

Signed-off-by: Travis Miller <raphtee@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@3146 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/public/afeclient.css
60a833bc9fd2c7fce2f84287b55c7d1829616771 16-May-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Fixed FindBugs warning: unchecked cast

risk: low
visibility: low

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3145 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/tko/WidgetList.java
a965cef89e47cf2ceb5742ce8948d4f4f5e759ba 16-May-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Added ability to clone jobs on all failed hosts.

Risk: low
Visbility: medium (UI change)

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3144 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpc_interface.py
fe/rpc_utils.py
lient/src/autotest/afe/JobDetailView.java
1ff7b2e88ae7a382f85ab76e786a471134e8a6a0 16-May-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Add ability to reverify a host from the Host List.

Risk: low
Visilibity: medium (UI change)

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3143 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
fe/rpc_interface.py
lient/src/autotest/afe/AfeUtils.java
lient/src/autotest/afe/HostListView.java
lient/src/autotest/afe/HostTable.java
10f416750fb1e942b842a5a844c7ec0512e40d6a 13-May-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> rpc_utils.extra_job_filters() should always return a dict

Signed-off-by: Steve Howard <showard@google.com>

Autotest@test.kernel.org
http://test.kernel.org/cgi-bin/mailman/listinfo/autotest


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3135 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpc_utils.py
12f3e3212795a539d95973f893ac570e669e3a22 13-May-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Add job maximum runtime, a new per-job timeout that counts time since the job actually started.
* added started_on field to host_queue_entries, so that we could actually compute this timeout
* added max_runtime_hrs to jobs, with default in global config, and added option to create_job() RPC
* added the usual controls to AFE and the CLI for the new job option
* added new max runtime timeout method to
* added migration to add new fields and set a safe default max runtime for existing jobs

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3132 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/001_rpc_test.txt
fe/models.py
fe/rpc_interface.py
lient/src/autotest/afe/CreateJobView.java
lient/src/autotest/afe/JobDetailView.java
lient/src/autotest/public/AfeClient.html
igrations/035_job_max_runtime.py
a1e74b3e9d68792fae0c926f89b6de1736b1fe21 12-May-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Add job option for whether or not to parse failed repair results as part of a job, with a default value in global_config. Since the number of options associated with a job is getting out of hand, I packaged them up into a dict in the RPC entry point and passed them around that way from then on.

Signed-off-by: Steve Howard <showard@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@3110 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/001_rpc_test.txt
fe/model_logic.py
fe/models.py
fe/rpc_interface.py
fe/rpc_utils.py
lient/src/autotest/afe/CreateJobView.java
lient/src/autotest/afe/JobDetailView.java
lient/src/autotest/public/AfeClient.html
igrations/034_add_parse_failed_repair_option.py
f8b19046c4496f570c776b65288382108a633ab4 12-May-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Add the ability for users to add test attributes. Non-user-created attributes (added by the parser) are still immutable.
* add boolean column user_created to TestAttribute to keep track of which are user created, so we can preserve immutability
* add id primary key field to test attributes. Django requires this and we've been squeezing by without it, but we can't anymore.
* declare some PK fields AutoFields in TKO models, as they should be. this didn't matter before but does now that we have a TKO unit test, since these models determine how the test DB gets created.
* add set_test_attribute RPC to set/delete attributes
* modify get_detailed_test_views() to use the new populate_relationships() method to gather attributes and labels much more efficiently
* add rpc_interface_unittest, a unit test for the TKO rpc interface. TKO was previously completely untested, so this is the first unit test of any kind for it. since the doctests on AFE turned out to be quite unpopular, I'm using the unittest framework this time. this required some changes to AFE testing code.
* various fixes to model_logic to account for assumptions we were previously making that aren't true in TKO (mostly about PK fields being named "id").

Note that the migration may be slow, as it's adding two columns to test_attributes, a potentially large table.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3109 592f7852-d20e-0410-864c-8624ca9c26a4
fe/model_logic.py
fe/models.py
fe/rpc_interface.py
rontend_unittest.py
etup_test_environment.py
0957a848a17b726836aaf9b5532bbfac691d983d 11-May-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Add host attributes to AFE. Nothing actually uses them; they're purely for users and scripts to use. Initial implementation is minimal - new migration, new model, RPCs to set and delete, and inclusion of attributes in get_hosts() results. No CLI or frontend support.

The change to get_hosts() is of primary interest here, since it involves a general problems that occurs many places in Autotest -- gathering relationships over many items. This has been solved in two ways previously:
* inefficiently in most places, by doing a query for each object (i.e. get_hosts()).
* efficiently in one place (JobManager.populate_dependencies()), by doing a single query to get a pivot table and then postprocessing to gather relationships for each object.

This time, I went ahead and implemented a general solution, a generalized version of JobManager.populate_dependencies(). I removed populate_dependencies() and made get_jobs() use the new general version instead, and I made get_hosts() use the new version for all its relationships, including the new attributes relationship. This should speed up get_hosts() considerably. In a future change I'll apply this solution to get_test_views() over in TKO land, since that would also greatly benefit in performance.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3102 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/001_rpc_test.txt
fe/model_logic.py
fe/models.py
fe/rpc_interface.py
fe/rpc_utils.py
igrations/033_add_host_attributes.py
4b63b74f6ef31649a26f8f6d8667ab297b73115b 08-May-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Add the ability to filter based on test labels.

Signed-off-by: Travis Miller <raphtee@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3101 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/tko/CommonPanel.java
lient/src/autotest/tko/WidgetList.java
4d07756ac199e75f9303eb32fdade43277bc8848 08-May-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Added ability to clone a job on specific hosts.

Risk: low
Visibility: medium (UI change)

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3100 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpc_interface.py
fe/rpc_utils.py
lient/src/autotest/afe/JobDetailView.java
fc5b779418b0f578a867386a2f09e12248c8c25b 08-May-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Fixed compiler warnings: unused imports and unread field.

Risk: low
Visibility: low

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3097 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/JobTable.java
lient/src/autotest/afe/RecurringTable.java
lient/src/autotest/afe/RecurringView.java
37059dfb50063124bce014d3217e3610499d917f 08-May-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> CSV support for TKO table view.
* make RpcDataSource record the last RPC call params, since that's much easier than reconstructing it as I was doing for SpreadsheetVIew
* make TableDecorator support an Export CSV link, and make TableView use it
* add onExportCsv to Table View, plus some minor refactorings there that happened along the way
* generalize CSV code from SpreadsheetView and extract to TkoUtils
* add new CsvEncoder classes for table view
* change logic to decide which CsvEncoder to use -- simply dispatching off of method name isn't good enough, since both spreadsheet and table views use get_status_counts
* add new unit tests for the three kinds of table view

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3096 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/common/table/RpcDataSource.java
lient/src/autotest/common/table/TableDecorator.java
lient/src/autotest/tko/SpreadsheetView.java
lient/src/autotest/tko/TableView.java
lient/src/autotest/tko/TestGroupDataSource.java
lient/src/autotest/tko/TkoUtils.java
8748ed29ef2a3a73c18cb387765d595db452256c 05-May-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Fixed FindBugs warnings: changed some inner classes to static inner classes.

Risk: low
Visibility: low

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3092 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/CreateJobView.java
lient/src/autotest/common/XhrJsonRpcProxy.java
lient/src/autotest/tko/HeaderSelect.java
f6348c999bc60ee048510c8e28f74f944a32d97c 05-May-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> When the user performs a whole-table action in TKO spreadsheet view, make sure we obey the "show only latest test per cell" checkbox. This isn't an issue for actions on particular cells, because we already use the test IDs since the cells are single-test cells. But whole table actions use a different code path.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3090 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/tko/ConditionTestSet.java
lient/src/autotest/tko/SpreadsheetView.java
1a6d3f3be1e3dca752e087645ea4b3db3906259c 05-May-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Improved embedded graphing sample -- use more generic, more interesting queries.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3088 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/public/EmbeddedTkoClientTest.html
3d6ae118f69717e68bc15b9aed7b6a6c7dd9bab0 02-May-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Export CSV support for spreadsheet view.
* new Django view, handle_csv(), with a new URLconf, /new_tko/server/csv/
* new module csv_encoder with logic to encode the results of an RPC call into CSV format
* logic in csv_encoder to convert results of get_status_counts() (or get_latest_tests()) into CSV, when called from spreadsheet view
* added optional "Export to CSV" link to TableActionsPanel, and made SpreadsheetView use it with a handler to call the /csv/ entry point with the current query info

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3086 592f7852-d20e-0410-864c-8624ca9c26a4
fe/json_rpc/serviceHandler.py
fe/rpc_handler.py
fe/rpc_utils.py
lient/src/autotest/common/JsonRpcProxy.java
lient/src/autotest/common/table/RpcDataSource.java
lient/src/autotest/common/ui/TableActionsPanel.java
lient/src/autotest/tko/SpreadsheetDataProcessor.java
lient/src/autotest/tko/SpreadsheetView.java
lient/src/autotest/tko/TestGroupDataSource.java
6d845a7ecbd6d25dfb456cc38472bc916c2db7b6 01-May-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> increase timeout for JSON-P queries to 60s

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3074 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/common/PaddedJsonRpcProxy.java
d262415ddedd11391e7625b248cb44ab0fc13380 29-Apr-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Fixed indenting multiline quoted content that uses triple quotes (" or
') in the control file generation RPC. Added unittest case to catch this
issue in the future.

Signed-off-by: Mihai Rusu <dizzy@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3065 592f7852-d20e-0410-864c-8624ca9c26a4
fe/control_file.py
fe/doctests/003_misc_rpc_features.txt
fe/doctests/test.control.3
29f7cd27e51add2648fb62ab2a0c588f9acb1ec4 29-Apr-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Here is a patch, which extends the autotest system with recurring job
executions. When you create a new recurring job, you can specify:
- start time (on server)
- loop count (0 means infinite): how many times it will executed
- loop period: how many time will wait between two execution

Added features:
- Create job: can create Template job.
- View job: added "Create recurring job"
- New tab "Recurring job"
- list of recurring jobs
- can click on it to view the executed job
- selection support
- Action -> remove
- creation panel (can be accessible thru "Create recurring job")
- submit/create
- reset
- data validity check

From: Zoltan Sogor <weth@inf.u-szeged.hu>
Signed-off-by: Steve Howard <showard@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@3064 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
fe/rpc_interface.py
fe/rpc_utils.py
lient/src/autotest/afe/AfeClient.java
lient/src/autotest/afe/AfeUtils.java
lient/src/autotest/afe/CreateJobView.java
lient/src/autotest/afe/JobDetailView.java
lient/src/autotest/afe/JobListView.java
lient/src/autotest/afe/JobTable.java
lient/src/autotest/afe/RecurringTable.java
lient/src/autotest/afe/RecurringView.java
lient/src/autotest/public/AfeClient.html
igrations/032_add_recurring_run.py
b4df82db72b3c2c3737b4f12c485fc3267866dbc 29-Apr-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> add @Override tag to overriding method in CommonClassFactory.java

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3062 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/common/CommonClassFactory.java
fbdab0b28c8084441225523a68db7f0ef63f5ca0 29-Apr-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Support multiple embedded graph widgets on the same page. This required two changes to avoid collisions with shared global resources:
* Make each plot generate a unique drilldown callback name for itself. These need to be unique because they all get stuck into as global methods. This also required passing the callback name to the server and having the server use it when generating callback code.
* Make the server generate a unique name for the image map, when generating HTML to return to the client. These image maps all sit in the document and thus must have unique names. It generates the map name from the passed-in drilldown name.

Since this change required passing around yet another piece of information to describe a plot in graphing_utils.py, I gathered all the parameters describing a plot into new parameter object classes, MetricsPlot and QualificationHistogram, rather than just extending the method signature of a bunch of methods. As usual, the natural next step is to move functionality into the classes, which would make graphing_utils much cleaner I believe. But I'm leaving that for later.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3061 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/public/EmbeddedTkoClientTest.html
lient/src/autotest/tko/EmbeddedTkoClient.java
lient/src/autotest/tko/MachineQualHistogram.java
lient/src/autotest/tko/MetricsPlot.java
lient/src/autotest/tko/Plot.java
c86113b66e4f06eeecf0f3268d6abcf2e4ab4c3e 28-Apr-2009 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Added server_job.set_test_tag() function (similar to the one in
client.bin.job) to specify a default tag to use for run_test() runs.

Changed the way we generate server control files from the frontend to be
very close to the client control files (using step_init(), stepX()
functions for each phase of the test). Changed generation of server
kernel control file to properly deal with the case of multiple kernels
given.

Risk: High

Signed-off-by: Mihai Rusu <dizzy@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3051 592f7852-d20e-0410-864c-8624ca9c26a4
fe/control_file.py
fe/doctests/003_misc_rpc_features.txt
56cdd56d3ea032fe412e7e7cf5c72c5f36733b96 27-Apr-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Fixed a unittest that previous change of mine broke.

Signed-off-by: Travis Miller <raphtee@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3042 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/001_rpc_test.txt
d04b4b6cdb0a22b0f7b2c1b150c6834f065d76c7 27-Apr-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Changed get_hosts() rpc call to return the acls to which the hosts
belong as well. This info is then used to display the ACLS to
which a host belongs in the HostDetailView tab in AFE.

Signed-off-by: Travis Miller <raphtee@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3041 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpc_interface.py
lient/src/autotest/afe/HostDataSource.java
lient/src/autotest/afe/HostDetailView.java
lient/src/autotest/public/AfeClient.html
b7a52fd2431275381b3ecab7d631a135fb613866 27-Apr-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> globalInitialize() starts a timer and every 10 minutes requests
the motd again and sets the string in the dom.

Signed-off-by: Travis Miller <raphtee@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3040 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpc_interface.py
lient/src/autotest/common/CommonClassFactory.java
9f4500a294eea35dce003cee41c558fcde3eb09f 27-Apr-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Last few changes necessary to make first embeddable graphs work
* add EmbeddedTkoClient GWT entry point + associated scripts, GWT XML, launch configuration, apache config
* added EmbeddedTkoClientTest.html, a simple demonstration/test of using embedded widgets
* modify TabView to no longer be a Composite, but instead have a getWidget element. this allows us to defer any DOM manipulation to initialize() and therefore avoid executing it at all in the embedded case. the introduction of code to TabView.initialize() (it was previous abstract) required adding a super.initalize() call to *all* subclasses, and there are a lot, hence the large number of files in this change.
* added Plot.getNativeProxy(), generating a native JS object that acts as a proxy to the GWT Plot object
* extend JsonRpcProxy to allow use of PaddedJsonRpcProxys
* remove debug prints from PaddedJsonRpcProxy
* fix a little bug where a return statement was missing from Plot.showDrilldown()

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3039 592f7852-d20e-0410-864c-8624ca9c26a4
lient/EmbeddedTkoClient-compile
lient/EmbeddedTkoClient-shell
lient/EmbeddedTkoClient.launch
lient/src/autotest/EmbeddedTkoClient.gwt.xml
lient/src/autotest/afe/CreateJobView.java
lient/src/autotest/afe/HostListView.java
lient/src/autotest/afe/JobListView.java
lient/src/autotest/afe/UserPreferencesView.java
lient/src/autotest/common/JsonRpcProxy.java
lient/src/autotest/common/PaddedJsonRpcProxy.java
lient/src/autotest/common/ui/CustomTabPanel.java
lient/src/autotest/common/ui/DetailView.java
lient/src/autotest/common/ui/TabView.java
lient/src/autotest/public/EmbeddedTkoClientTest.html
lient/src/autotest/tko/EmbeddedTkoClient.java
lient/src/autotest/tko/GraphingView.java
lient/src/autotest/tko/Plot.java
lient/src/autotest/tko/SpreadsheetView.java
lient/src/autotest/tko/TableView.java
lient_compilation_unittest.py
d3dc199703bfb8784a2f8f072d0514532c86c0a9 22-Apr-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Add support to the scheduler to run autoserv --collect_crashinfo after a job finishes or is aborted.
* added new state "Gathering" for when we're running collect_crashinfo and copying logs to the results repository
* added new GatherLogsTask to the scheduler to perform these two tasks, and made it get run either after a job finishes or after a job is aborted. this task shares a lot with FinalReparseTask, so extracted common code into a new PostJobTask.
* made changes to scheduler/drone code to support generic monitoring and recovery of processes via pidfiles, since we need to be able to recover the collect_crashinfo processes too. this will also made the scheduler recover parse processes instead of just killing them as it does now, which is nice.
* changed abort logic significantly. since we now need to put aborted jobs through the gathering and parsing stages, but then know to put them into "aborted" afterwards, we can't depend on the old path of abort -> aborting -> aborted statuses. instead, we need to add an "aborted" flag to the HQE DB table and use that. this actually makes things generally cleaner in my opinion -- for one, we can get rid of the "Abort" and "Aborting" statuses altogether. added a migration to add this flag, edited model and relevant logic appropriately, including changing how job statuses are reported for aborted entries.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3031 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/001_rpc_test.txt
fe/model_logic.py
fe/models.py
fe/rpc_interface.py
lient/src/autotest/afe/JobDetailView.java
igrations/031_add_hqe_aborted_flag.py
3181c7baa64106cb176f26dcc6b1f9ea87141382 20-Apr-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Prevent jobs from being created via the RPC interface that will be impossible
to schedule due to labels that conflict with the supplied atomic group (or
lack thereof) being specified as meta_hosts or dependencies.

Signed-off-by: Gregory Smith <gps@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3018 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/001_rpc_test.txt
fe/rpc_interface.py
47d9e7dae3d9edfc8bd153f627307b44801d31fb 20-Apr-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Prevent creating jobs via the frontend directly on hosts in an atomic group
if the atomic group was not requested for that job.

Signed-off-by: Gregory Smith <gps@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3017 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/001_rpc_test.txt
fe/rpc_interface.py
4d35724705fb82094fdd27b0396f70fe6a163e31 16-Apr-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Provide the ability to Re-validate results that had been
previously marked as invalid.

Signed-off-by: Travis Miller <raphtee@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@2996 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/tko/TestContextMenu.java
lient/src/autotest/tko/TestLabelManager.java
eaccf8fd578288cb20941dad6c621182db29e9ae 16-Apr-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> move escape_user_sql() to the manager class, where it better belongs, make it not static, since it doesn't need to be and we need to override it, and override it in TestViewManager to specify the table whenever test_idx is used (otherwise it can be an ambiguous reference when test_attributes is joined)

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@2995 592f7852-d20e-0410-864c-8624ca9c26a4
fe/model_logic.py
d1677ebabfa318980cc6fbe3cd08990d800dcbac 16-Apr-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Update management so that it adds permissions for 'atomicgroup'

Signed-off-by: Scott Zawalski <scottz@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@2992 592f7852-d20e-0410-864c-8624ca9c26a4
fe/management.py
edd589779be93badb9f9dc546bad538db5a0318d 16-Apr-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Allow display of the message of the day in tko as well as afe.
refactored as common code in CommonClassFactory.

Signed-off-by: Travis Miller <raphtee@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@2991 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/AfeClient.java
lient/src/autotest/common/CommonClassFactory.java
lient/src/autotest/public/TkoClient.html
3b2b93007ba7ca0a8ace90f2ac0c2263ccbcbdf8 15-Apr-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> * extract code for displaying graphs and managing graph drilldowns from *Frontend classes (which also manage the graph setup UI) into separate Plot classes (a Plot class and two subclasses, MetricsPlot and MachineQualHistogram). eventually, these plot classes will be usable as embedded widgets in other pages.
* made some changes to graphing_utils.py to support the new design, including encoding drilldown information with JSONEncoder
* extract GraphingDialog class to a generic, top-level SimpleDialog class, so that it'll be usable from embedded widgets without including all the frontend code
* fix a bug in FilterSelector.isEmpty() (it would always return false previously)
* fix a bug in TableView where no column would be sorted on in some cases

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@2990 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/common/ui/SimpleDialog.java
lient/src/autotest/tko/DynamicGraphingFrontend.java
lient/src/autotest/tko/ExistingGraphsFrontend.java
lient/src/autotest/tko/FilterSelector.java
lient/src/autotest/tko/GraphingFrontend.java
lient/src/autotest/tko/MachineQualHistogram.java
lient/src/autotest/tko/MachineQualHistogramFrontend.java
lient/src/autotest/tko/MetricsPlot.java
lient/src/autotest/tko/MetricsPlotFrontend.java
lient/src/autotest/tko/Plot.java
lient/src/autotest/tko/TableView.java
946a7af939c07a64a627234acb094688125c4683 15-Apr-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Ensure one-time-hosts get unlocked when created.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@2989 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
225bdc14c3ca58cf6de6c10456796a9ef65481af 13-Apr-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Allow the RPC interface to create jobs in an atomic group without
requiring any meta_hosts or hosts to be specified.

Signed-off-by: Gregory Smith <gps@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@2985 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/001_rpc_test.txt
fe/rpc_utils.py
3562a064089d02e86e40a5ccca2f753afc341a83 13-Apr-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Fix get_info_for_clone() when the HQE has no atomic_group.

Signed-off-by: Gregory Smith <gps@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@2984 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/001_rpc_test.txt
fe/rpc_interface.py
c92da83b2dd5b7992a58081c6005b3dfb3dc5f3a 07-Apr-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Add support for atomic groups to the frontend RPC interface.

Signed-off-by: Gregory Smith <gps@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@2967 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/001_rpc_test.txt
fe/models.py
fe/rpc_interface.py
fe/rpc_utils.py
a901ba9a627c31238bd1b674a532d2e860a1a348 07-Apr-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Add more infrastructure for common (between AFE & TKO) site-specific features, and various UI widget enhancements.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@2965 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/AfeClient.java
lient/src/autotest/common/CommonClassFactory.java
lient/src/autotest/common/SiteCommonClassFactory.java
lient/src/autotest/common/ui/ContextMenu.java
lient/src/autotest/common/ui/SimpleHyperlink.java
lient/src/autotest/public/common.css
lient/src/autotest/tko/TkoClient.java
264ce291285e40d734e6aa53daca43860d1f7d78 07-Apr-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Only show "Status log" and "Debug logs" links when a host queue entry's execution_subdir is nonempty. With the way things work now, execution_subdir should be nonempty iff results of some kind have been generated. This simplified the surrounding logic a bit.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@2964 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/JobDetailView.java
89e7a1205ac482a0f57d7d5e0e87d62c57afe6dd 07-Apr-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Fix log viewers sizing in FF3. Apparently elements with overflow: auto (or scroll or hidden) don't do percentage widths properly in FF3. So to workaround it, shrink the scrolling div to zero width, let the parent assume the correct size, then dynamically grab that width in pixels and set the scroller to that exact width. Also listen for and respond to window resize events, since we're setting explicit widths.

Also had to change styles of the outermost tab frame to ensure the logs would expand to fit the window, and while i was at it, started a new common.css file for such things.

This still doesn't work in Chrome. I'm convinced it's a Chrome bug, but I'll have to wait till I have more free time later next week to investigate. Sorry Chrome users.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@2962 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/AfeClient.gwt.xml
lient/src/autotest/TkoClient.gwt.xml
lient/src/autotest/public/afeclient.css
lient/src/autotest/public/common.css
lient/src/autotest/tko/TestDetailView.java
191452dea8e80dee8428f1b7b4c70cd09eb81a8f 01-Apr-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> * rewrite CreateJobView.cloneJob() to call selectObjects() once instead of selectObject() within a loop. The former is much more efficient, as it postpones UI updates until after all selections have been made. This makes job cloning much quicker for large jobs and should help avoid the slow script warning.
* simplify the code by using a JSONArrayList

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@2946 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/CreateJobView.java
ef6fe028fcc667366e8ac30fe63ba314a4b1d745 27-Mar-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> generic JSON-RPC implementation using JSONP (JSON with Padding). the central part of this change consists of:
* refactored JsonRpcProxy to extract all XmlHttpRequest-specific logic into a new XhrHttpRequest subclass, and made JsonRpcProxy abstract
* introduced new PaddedJsonRpcProxy subclass of XmlHttpRequest that uses JSONP instead of XHR
* added new handle_jsonp_rpc_request() method to rpc_handler.py, to handle JSONP requests on the server side

This enables the entire frontend (either AFE or TKO) to operate via JSONP instead of XHR. I didn't make them do that now, since there's no reason, but it will be critical when we go to make components embeddable in other pages (on other domains). Other changes here include:

* made TestDetailView use PaddedJsonRpcProxy (it previous had its own custom JSONP logic, which was all moved into PaddedJsonRpcProxy).
* made retrieve_logs.cgi and jsonp_fetcher.cgi support JSONP requests, so that log fetching requests could go through the shared JsonRpcProxy codepath. retrieve_logs.cgi still supports the old GET params interface for backwards compatibility (at least old TKO still uses it, and possible other scripts).

Signed-off-by: Steve Howard <showard@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2943 592f7852-d20e-0410-864c-8624ca9c26a4
fe/json_rpc/serviceHandler.py
fe/rpc_handler.py
fe/urls.py
fe/views.py
lient/src/autotest/afe/JobDetailView.java
lient/src/autotest/common/JsonRpcCallback.java
lient/src/autotest/common/JsonRpcProxy.java
lient/src/autotest/common/PaddedJsonRpcProxy.java
lient/src/autotest/common/Utils.java
lient/src/autotest/common/XhrJsonRpcProxy.java
lient/src/autotest/tko/TestDetailView.java
d2f13cd202b636406b298d7515d3c4e2d150ffc1 27-Mar-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Add missing atomic_groups.invalid migration.

Signed-off-by: Gregory Smith <gps@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2941 592f7852-d20e-0410-864c-8624ca9c26a4
igrations/029_add_atomic_group_invalid.py
b0a73038e9bd3e67bfcb9b8f18d39024be89bf22 27-Mar-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Fix model_logic so that smart_get will perform the lookup by -either-
id or the primary key (normally name) automagically.

Signed-off-by: Gregory Smith <gps@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2940 592f7852-d20e-0410-864c-8624ca9c26a4
fe/model_logic.py
470aa3fbc1df5207c371112f9161a61adbca2ab8 25-Mar-2009 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Add migration to add a proper default value to hosts invalid column. Since we rely on this default we should explicitly set it.

Signed-off-by: Scott Zawalski <scottz@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2930 592f7852-d20e-0410-864c-8624ca9c26a4
igrations/030_update_hosts_invalid.py
6448386cc4f36817cd5b7e82f5c79c6693f353bf 21-Mar-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Fix doctests that are "failing" due to being overly pedantic.

Signed-off-by: Gregory Smith <gps@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2922 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/001_rpc_test.txt
de9c3303091fc2ec7aaffa2f3ba7915ab328e4c8 21-Mar-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> columnNames must always be a mutable List. this was breaking querying
after doing a drilldown or loading a page from history.

Signed-off-by: Steve Howard <showard@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2921 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/tko/TableView.java
3af93f9e93f65dad8d8e5b808d67320c2da958d5 21-Mar-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> fix some unused code eclipse warnings that had been disabled but should've been enabled

Signed-off-by: Steve Howard <showard@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2920 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/CreateJobView.java
lient/src/autotest/common/ui/TabView.java
lient/src/autotest/tko/SeriesSelector.java
lient/src/autotest/tko/TkoUtils.java
6315df674f86f719e816ea5689768ef0944e9f51 21-Mar-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> -support command-click for macs, since control-click is overridden on macs to get context menu behavior. i forgot about that.
-fix a style issue i forgot to include in the original patch.

Signed-off-by: Steve Howard <showard@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2919 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/common/ui/TabView.java
lient/src/autotest/tko/CommonPanel.java
205fd60f9c9d2f64ec2773f295de1cf5cfd3bc77 21-Mar-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Fix the AtomicGroup name display in the admin interface.

Adds an invalid bool column and use the existing invalid model to avoid
problems when deleting from the django admin interface.

Signed-off-by: Gregory Smith <gps@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2918 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
fe/views.py
89f84dbadf071ba430244356b57af395c79486e4 12-Mar-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Add the concept of an Atomic Group to the scheduler and database.

Scheduling a job on an atomic group means that all of the Ready machines
(up to a maximum specified in the atomic group) in a single label associated
with that atomic group will be used to run the job.

The job synch_count becomes a minimum when scheduling on an atomic group.

Both HostQueueEntrys and Labels may have an AtomicGroup associated with
them:

* A HostQueueEntry with an AtomicGroup acts to schedule a job on
all Ready machines of a single Label associated with that AtomicGroup.

* A Label with an AtomicGroup means that any Hosts bearing that Label
may only be scheduled together as a group with other hosts of that Label
to satisify a Job's HostQueueEntry bearing the same AtomicGroup.
Such Hosts will never be scheduled as normal metahosts.

Future patches are coming that will add the ability to schedule jobs using
this feature to the RPC interface, CLI and GUI.

Signed-off-by: Gregory Smith <gps@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2878 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
igrations/028_add_atomic_groups.py
c674d3ea684f75f3e05f5834b598050eb1c8856d 12-Mar-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Support opening drilldown results in a new tab/window from spreadsheet and table views. When drilling down in spreadsheet and table views, either into a group or into test details for a single test, the user can open the resulting page in a new tab/window by control-clicking. I included what I believe is the proper code to support middle-clicking as well, but I could not see any browser support. I tried the GWT hosted mode browser and FF2 on linux, and FF3 and windows. I changed the cell selection combo to shift-click (I don't think anyone uses that feature anyway :-/).

The main technical challenge here was this: state is spread throughout a number of classes that make up the UI for spreadsheet and table views. Code for generating the history tokens is spread throughout those classes as well, using the state in each class. In order to open results in a new tab, the code needs to generate the history token for the new state while leaving the current UI unmodified. It would do this by calling functions to modify application state appropriately, then generate the new history token, and then revert to the previous application state before opening the new token in a new tab. However, since the UI widgets and state variables were all intertwined and spread throughout the code, this was impossible to do as the code was -- when we made calls to setup the new application state, the displayed UI would be modified.

To solve this, I modified a number of classes to implement updateStateFromView() and updateViewFromState() methods, making the separation between state and view more well-defined and explicitly controlled. I ensured that all state-modifying functions don't modify the view at all, and I made sure the history-generating functions only use state variables. Then I was able to make the code setup a new state, generate the new history token, and restore the old state without any visible changes in the UI.

I also modified CustomHistory and clients so that history tokens passing into and out of CustomHistory are dictionaries instead of strings. I find this a bit cleaner and (more importantly) it allows CustomHistory to safely check for token equality (checking strings is unsafe because ordering could change while content remains the same, thanks to the nondeterminism of iterating over a Map).

Signed-off-by: Steve Howard <showard@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2876 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/JobListView.java
lient/src/autotest/common/CustomHistory.java
lient/src/autotest/common/Utils.java
lient/src/autotest/common/ui/DetailView.java
lient/src/autotest/common/ui/TabView.java
lient/src/autotest/tko/CommonPanel.java
lient/src/autotest/tko/ConditionTabView.java
lient/src/autotest/tko/GraphingFrontend.java
lient/src/autotest/tko/GraphingView.java
lient/src/autotest/tko/HeaderSelect.java
lient/src/autotest/tko/SavedQueriesControl.java
lient/src/autotest/tko/SpreadsheetView.java
lient/src/autotest/tko/TableView.java
lient/src/autotest/tko/TestSelectionListener.java
lient/src/autotest/tko/TkoClient.java
c9c0f3cd3cd727f6b5450f110ba312cb350eeb8c 09-Mar-2009 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Added frontend/migrations/027_fix_innodb.py and
tko/migrations/021_fix_innodb.py to perform ALTER TABLE ...
ENGINE=InnoDB on tables missing it.

Signed-off-by: Mihai Rusu <dizzy@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2858 592f7852-d20e-0410-864c-8624ca9c26a4
igrations/027_fix_innodb.py
ca4bbc1c3d6c1c6465900965b68e724736a240bc 25-Feb-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> acl_group -> aclgroup, per gps' previous change. without this, syncdb won't add the permission for editing ACL groups and they'll disappear from the admin interface.

Signed-off-by: Steve Howard <showard@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2821 592f7852-d20e-0410-864c-8624ca9c26a4
fe/management.py
313ab769319f56b940a2784cc0cbdebd005c5799 25-Feb-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Make the "Triage failures" button obey the "show only latest test per cell" filter. This is a bit of a tricky issue because the table view doesn't have a concept of cells, so the option can't carry directly over. I've made it simply add a big SQL filter "AND test_idx IN (...)" where ... is a giant list of all the test IDs displayed in the spreadsheet. I don't think it's ideal but it should work well, at least for now.

Signed-off-by: Steve Howard <showard@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2818 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/tko/Spreadsheet.java
lient/src/autotest/tko/SpreadsheetView.java
9cd2711280fcf6506bf2d4682c471c5fd3744813 17-Feb-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Mention the profile_only=True flag for users to disable the first iteration.

Signed-off-by: Gregory Smith <gps@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2814 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/public/AfeClient.html
25cbdbd2da6242296abe6b1342521b29993993f2 17-Feb-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Get rid of the host_queue_entries.priority field in the autotest_web DB. It's just duplicating information from the jobs table.

Signed-off-by: Steve Howard <showard@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2813 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/001_rpc_test.txt
fe/models.py
igrations/026_remove_hqe_priority.py
aac13a7fbd2d380e98f088b89fb57cb4fa3cf853 17-Feb-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> When clicking "Triage failures", only include failed results.

Signed-off-by: Steve Howard <showard@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2812 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/tko/CommonPanel.java
lient/src/autotest/tko/SpreadsheetView.java
a4ea574950e15e9be4df38464046584bf8f380f7 17-Feb-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> convert_human_readable_values should access all objects, not ust valid ones. while I was changing smart_get, I got rid of the **kwargs behavior, because it was completely unused.

Signed-off-by: Steve Howard <showard@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2811 592f7852-d20e-0410-864c-8624ca9c26a4
fe/model_logic.py
451954477f807e31a3b82a85d914807f4cf66a80 17-Feb-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> When cloning a job with preserve_metahosts=False, keep all entries, even if the eventually assigned host when then deleted.

Signed-off-by: Steve Howard <showard@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2810 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpc_interface.py
ddb90994ef51aed28feea1cee810ce6312efae51 12-Feb-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Make smart_get only return valid objects. This should prevent adding deleted labels to hosts from the CLI, among other things. Also cleanup a bit of code in ModelWithInvalid.

Signed-off-by: Steve Howard <showard@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2782 592f7852-d20e-0410-864c-8624ca9c26a4
fe/model_logic.py
e5ae165c706c465f08a732a361ea0aa5d75075ea 12-Feb-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Widespread cleanup of graphing code, based on notes I took during the original review of the code that never got implemented.

Signed-off-by: Steve Howard <showard@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2781 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/JobListView.java
lient/src/autotest/common/Utils.java
lient/src/autotest/tko/DBColumnSelector.java
lient/src/autotest/tko/DynamicGraphingFrontend.java
lient/src/autotest/tko/ExistingGraphsFrontend.java
lient/src/autotest/tko/FilterSelector.java
lient/src/autotest/tko/FilterStringViewer.java
lient/src/autotest/tko/GraphingFrontend.java
lient/src/autotest/tko/GraphingView.java
lient/src/autotest/tko/MachineQualHistogramFrontend.java
lient/src/autotest/tko/MetricsPlotFrontend.java
lient/src/autotest/tko/PreconfigSelector.java
lient/src/autotest/tko/SeriesSelector.java
lient/src/autotest/tko/TkoUtils.java
529cdf3a9bd5d6c8f29d18a00ce3229880b9e4c2 07-Feb-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Fix two typos, one just a UI typo and one an actual bug due to an incorrect field name that was breaking the test selector.

Signed-off-by: Steve Howard <showard@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2767 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/TestSelector.java
d9ac445a60d6d11537f566503164344e09527917 07-Feb-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Remove the old acl_groups_users and acl_groups_hosts many2many pivot
table column name hack.

Renames acl_group_id -> aclgroup_id.

Adds a migration script and updates the one piece of code that actually
depended upon the old name.

This is needed for my upcoming change that ports autotest to Django
1.0.x but seems worth cleaning up as a change of its own.

Signed-off-by: Gregory Smith <gps@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2764 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/001_rpc_test.txt
fe/doctests/002_site_rpc_test.txt
fe/models.py
lient/src/autotest/afe/HostTableDecorator.java
igrations/025_aclgroup_id_column_rename.py
11e73f087d5d162d02a9e9c47adcdbe130773d54 04-Feb-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Up both stack limits to 512, since people have reported problems.

Signed-off-by: Steve Howard <showard@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2745 592f7852-d20e-0410-864c-8624ca9c26a4
lient/AfeClient-compile
lient/TkoClient-compile
35120fef9a1b56fd419ec9794ebb5985754aee98 03-Feb-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> -changes to TableView to more transparently treat "special" columns (group count, pass rate) more like normal columns.
-make TableView keep track of currently visible column names, instead of field names. This is better for tracking "special" columns because they don't necessarily have field names (e.g. pass rate uses the special widget column "field name").
-since TRIAGE_GROUP_COLUMNS now includes the count column, put it on the left of reason.
-for part of the change, it ended up being easier to stop using the whole "readonly item" thing in DoubleListSelector, so got rid of that and made TableView handle when "special" columns are removed through the DoubleListSelector.

Signed-off-by: Steve Howard <showard@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2736 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/common/ui/DoubleListSelector.java
lient/src/autotest/tko/TableView.java
93c80e69e7f3547d722d4ec3719c83895d52b1f7 03-Feb-2009 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Filename for rpc_test.txt is wrong

Signed-off-by: Martin Bligh <mbligh@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2733 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpc_interface.py
94ae6d6686befa237773954bda9e721dbd6595be 03-Feb-2009 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Add help text for the new reboot before/after rpc params.

Signed-off-by: Jeremy Orlow <jorlow@google.com>




git-svn-id: http://test.kernel.org/svn/autotest/trunk@2732 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpc_interface.py
b5b7b5dc37012db192e9e47cd8e6fe45faf48dd2 03-Feb-2009 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Improved error message to the user when entering a host as a one time
host that is already in the autotest database.

Signed-off-by: Gregory Smith <gps@google.com>




git-svn-id: http://test.kernel.org/svn/autotest/trunk@2731 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
d429b730d75161e62c3c392c51f9170a1180d73b 03-Feb-2009 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> The name field in the labels table was not declared UNIQUE in the initial
database setup (001_initial_db.py). This adds the UNIQUE flag.

No automatic fixups are applied if the constraint cannot be applied, if any
duplicate names exist the administrator will need to investigate and clean up
on their own.

Signed-off-by: Gregory Smith <gps@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2726 592f7852-d20e-0410-864c-8624ca9c26a4
igrations/024_make_label_name_unique.py
14374b1b5cc1a5878e8e9d7148fd3c33324718bf 31-Jan-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Fix the missing 'os' import.
Allow prepare_generate_control_file() to work with an empty list of tests.

Signed-off-by: Gregory Smith <gps@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2722 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpc_utils.py
ade14e2081e4b934f4b2232acfbf4c1b2f3bece4 26-Jan-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Bypass only_if_needed labels for specifically-requested hosts. This is a workaround for generating proper job dependencies in an easier/more automated manner when selected specific hosts with only_if_needed labels.

Signed-off-by: Steve Howard <showard@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2692 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpc_utils.py
77618197b903dbf95d26314478b742497e482359 26-Jan-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Wrap the TKO drop-down log viewers with scroll panels, so that when FF3 cuts them off, you can still scroll over to see their contents. Also limit the max height, since there's a scroller now.

This is far from perfect in FF3 (works beautifully in FF2) but I think it's an improvement, and I don't have time at the moment to mess around with FF's quirky layout mechanisms.

Signed-off-by: Steve Howard <showard@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2691 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/public/tkoclient.css
lient/src/autotest/tko/TestDetailView.java
d94f4db2fb1a88cb1de25e9a8a81cd59177ea0c8 26-Jan-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Add job name search feature to AFE job list tab.

Signed-off-by: Steve Howard <showard@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2690 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/HostTableDecorator.java
lient/src/autotest/afe/JobDetailView.java
lient/src/autotest/afe/JobListView.java
lient/src/autotest/common/table/SearchFilter.java
lient/src/autotest/public/AfeClient.html
120351e0c9a9b3e7c7a4aa51477a5ae2366214c4 24-Jan-2009 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Allow control file body text to be supplied as an alternative to a list
of test names for the generate_control_file AFE rpc.

This allows the command line client to let people specify a kernel along with
their own client-side control file.

It adds the kernel installation steps to the supplied control file before
scheduling it (using the same code as the GUI when a kernel is specified
there).

Signed-off-by: Gregory Smith <gps@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2684 592f7852-d20e-0410-864c-8624ca9c26a4
fe/control_file.py
fe/doctests/001_rpc_test.txt
fe/rpc_interface.py
748b23f40bacb4a98c2c5762ec76e05f032748f1 23-Jan-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Remove three unused imports added by someone's previous change. Grr.

Signed-off-by: Steve Howard <showard@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2678 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/AfeClient.java
65c267df570f3e827c8a700f3746c26fcab0da77 21-Jan-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Check synch_count in RPC method, before job gets created. Also remove a call to save() which was redundant (it gets called in add_object()). It was misleading and was probably what led to the erroneously placed check on synch_count.

Signed-off-by: Steve Howard <showard@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2661 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
fe/rpc_interface.py
8fbae65f091151643460b14a1fd0c933b7090d43 21-Jan-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> First pass at an implementation of a message of the day. Decided
that storing the motd in a file called motd.txt in the same dir
as rpc_interface.py was the simplest solution.

Signed-off-by: Travis Miller <raphtee@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2659 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpc_interface.py
fe/rpc_utils.py
lient/src/autotest/afe/AfeClient.java
lient/src/autotest/public/AfeClient.html
lient/src/autotest/public/afeclient.css
0c18519078035741b9f86c3db65b7cb98e9d4e09 16-Jan-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> When aborting host queue entries, completely ignore completed entries. They get ignored eventually anyway, but if we include them here, we can end up erroneously rejecting abort requests for synchronous jobs.

This happens because entries that fail verify get their execution_subdir set to their hostname. Then, when we see the abort, we find only one entry with that execution tag while the sync_count is >1, so we think the user is aborting part of a synchronous execution and throw an exception.

Signed-off-by: Steve Howard <showard@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2658 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpc_interface.py
67831ae7a70ab35b48f98fad7f14d285b69d9bd9 16-Jan-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Make sure the debug messages from models get printed while the scheduler is running. The log level got changed during the scalability work and this was disabling output of these important status change messages,

Signed-off-by: Steve Howard <showard@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2657 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
043abc1c38db968c11d7a5f8b5830e53ceb0fc8c 14-Jan-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Fix creation of "Basic Admin" group, at long last.

Signed-off-by: Steve Howard <showard@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2652 592f7852-d20e-0410-864c-8624ca9c26a4
fe/management.py
7a37db1b0455d4637395d101ffeb412f469dc182 09-Jan-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> since global config now considers an empty value as a nonexistent value, change settings.py files to handle empty passwords correctly

Signed-off-by: Steve Howard <showard@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2612 592f7852-d20e-0410-864c-8624ca9c26a4
ettings.py
9484c31ad1021c2e3b0ae796c31b220b5c057436 07-Jan-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Change drop-down log viewers in TKO to use JSONP instead of XmlHttpRequests so they can handle direction to other servers. If you're curious what the hell JSONP is, check out http://code.google.com/docreader/#p=google-web-toolkit-doc-1-5&s=google-web-toolkit-doc-1-5&t=Article_UsingGWTForJSONMashups.

Signed-off-by: Steve Howard <showard@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2607 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/common/Utils.java
lient/src/autotest/tko/TestDetailView.java
5a198b91a2d3ca37943d02628886aac19ee04283 11-Dec-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Add 'Verifying' to class Job so that View Job allows users to view machines
that are 'stuck' in verify or taking a long time in cleaning

Signed-off-by: Scott Zawalski <scottz@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2560 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
fe/rpc_interface.py
2e9415ff78190fcc6f1c6c9cb6ca5529316b7248 05-Dec-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Preparation for adding a widget to edit job dependencies in the create job view in AFE. I haven't quite finalized how to add the widget so it's not there yet, but this puts the piece in place, and also fixes a bug with job cloning and dependencies.

Signed-off-by: Steve Howard <showard@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2547 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpc_interface.py
lient/src/autotest/afe/CreateJobView.java
lient/src/autotest/common/ui/DoubleListSelector.java
8ae7d3a68b34bb2791f4206d3b41b1fcc50a0d00 03-Dec-2008 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Add profiler support to the server-side control file generation.

Risk: Low
Visibility: You can use the frontend control file generation for
profilers even with server-side jobs.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2539 592f7852-d20e-0410-864c-8624ca9c26a4
fe/control_file.py
lient/src/autotest/afe/CreateJobView.java
d876f459fff6cc4994cab329b1f80c99a86edcbd 03-Dec-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> gps pointed out that "== and != work in most cases but its better to use is
and is not as you'll never run into a case where someone's __eq__ or __ne__
method do the wrong thing."

Signed-off-by: Martin J. Bligh <mbligh@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2533 592f7852-d20e-0410-864c-8624ca9c26a4
fe/json_rpc/proxy.py
fe/json_rpc/serviceHandler.py
7a3ebe30d13f8dc7e8d3d0990b8b026be75771f3 01-Dec-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Continue to add/remove the valid hosts or users to the label or ACL in
case of failures.
Modified the corresponding unittests:
. Adding check_playback() everywhere.
. Changing to the new Exception messages.
. New test for partial failure.

Risk: medium
Visibility: medium

Signed-off-by: Jean-Marc Eurin <jmeurin@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2525 592f7852-d20e-0410-864c-8624ca9c26a4
fe/model_logic.py
c85c21bec23af3f805397f1f7e289b9a4b0bfd05 24-Nov-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> * allow scheduler email "from" address to be specified in global config
* allow global config to specify statuses which should trigger immediate emails (in addition to email upon job completion)
* make "Parsing" an active=complete=0 status, and modify Job.num_complete() appropriately
* restructuring of scheduler email notification code to only email at the right time and to include a more informative message


git-svn-id: http://test.kernel.org/svn/autotest/trunk@2506 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
25087accde32cf921991e6a20aea35986141cec4 24-Nov-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Return a decent error message when an invalid label is passed to create_job() as a metahost.



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2505 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpc_interface.py
87181c038a4cdf90f5ed933c8692cb6f0779e2be 19-Nov-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Eliminate duplicate exception messages for non-requested only_if_needed labels. Need to use distinct() on the query since we're doing joins.



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2451 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpc_utils.py
869806f6a6063d4c5d83eed04ee63419d9ad8226 19-Nov-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> When reading machine label values from history, ensure they get stored correctly and used.



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2449 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/tko/HeaderSelect.java
a58594a616ab5364c72aa824b7ccde7cdd7b10bf 18-Nov-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Don't reset active bit when setting status to "Abort". The scheduler needs that information to know if the queue entry was active before being aborted.



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2436 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
e39c3c0dbe16caed55f4538af78dbf057deb7fad 18-Nov-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Adds the ability for the client.bin.base_job class to keep track of run
numbers and include them in the output directory / test tag.

Adds the ability for the client.bin.base_job class to insert the
running kernel version (uname -r) into the test tag at run_test() time.

Enabled both of the above in the generated control file when multiple
kernels are given via the AFE.

Signed-off-by: Gregory Smith <gps@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2433 592f7852-d20e-0410-864c-8624ca9c26a4
fe/control_file.py
fe/doctests/001_rpc_test.txt
fe/doctests/003_misc_rpc_features.txt
15fe995a5e73b0ed8da3a9450a9a42a48bcdf475 15-Nov-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Fix server side control files generated by the autotest front end. kernel is no longer defined, kernel_list is.

Signed-off-by: Gregory Smith <gps@google.com>




git-svn-id: http://test.kernel.org/svn/autotest/trunk@2424 592f7852-d20e-0410-864c-8624ca9c26a4
fe/control_file.py
fe/doctests/003_misc_rpc_features.txt
e42e0cdf153b5dd47866566cbca3b08771dbad56 14-Nov-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Fix job detail tab to point to the right place for status and debug log links for single-host results.



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2420 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/JobDetailView.java
3f15eedfedab4b1b9eaff67713e411eada0c84df 14-Nov-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Fix bug in generation of error message for unauthorized abort that occurs when aborting an unassigned metahost entry.



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2419 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
1429e7f683cefd9849bef084e6fbb3d0577fa7b2 14-Nov-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Don't include a kernel version tag prefix when only one kernel was given.

Signed-off-by: Gregory Smith <gps@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2418 592f7852-d20e-0410-864c-8624ca9c26a4
fe/control_file.py
fe/doctests/001_rpc_test.txt
a772fcf207a6c9a1b8dedd129d5fb5474ae683db 14-Nov-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Fix the frontend unittests for the previous change.

Signed-off-by: Gregory Smith <gps@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2417 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/001_rpc_test.txt
0174619ea845a9675548e4e213aba7d3e1eb25ad 13-Nov-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Fix a typo in rpc_utils.py



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2414 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpc_utils.py
f7eac6f85897631eff27883360491e8ad2b36800 13-Nov-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Handle the case where there are no only_if_needed labels in host filtering code.



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2413 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpc_utils.py
74354f734a1c54c25cddd1c15bf7115d539c51f3 13-Nov-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Use quotes instead of underscores in the UI because underscores are ugly.



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2412 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/HostTableDecorator.java
6d2a6f937d417f45f64cefcae7a5f888ba1fcbcb 13-Nov-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> The kernel global needs to be set properly for many existing control files to
run.

Signed-off-by: Gregory Smith <gps@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2411 592f7852-d20e-0410-864c-8624ca9c26a4
fe/control_file.py
97db5ba6cdee21d706dff3d1a85c354c8bfae6f9 12-Nov-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Add a user preference for showing experimental tests, disabled by default, and add code to obey it in TestSelector.



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2406 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/001_rpc_test.txt
fe/models.py
lient/src/autotest/afe/CreateJobView.java
lient/src/autotest/afe/TestSelector.java
lient/src/autotest/afe/UserPreferencesView.java
igrations/023_add_show_experimental_pref.py
43a3d26e49662d06e145ed94a0c2dfb2b455126f 12-Nov-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Add a checkbox 'exclude "only if needed" labels' to host lists in AFE, checked by default in the create job view.
-add exclude_only_if_needed_labels argument to get_hosts() and get_num_hosts() RPCs
-move some code from tko.models.TestViewManager up into model_logic, since AFE now needs it to create a custom join for doing many-valued exclusion



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2405 592f7852-d20e-0410-864c-8624ca9c26a4
fe/model_logic.py
fe/rpc_interface.py
fe/rpc_utils.py
lient/src/autotest/afe/HostSelector.java
lient/src/autotest/afe/HostTableDecorator.java
lient/src/autotest/common/table/CheckboxFilter.java
dc8175153fc559d1e7ae605b15ceef66e25e7cde 12-Nov-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> allow users to abort other users' jobs, but only for machines that are both ACL-accessible and *not* in the "Everyone" ACL.



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2404 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
fe/rpc_interface.py
be3ec04ed2053527e19c1593eda0184f18799868 12-Nov-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Return an actually useful error when one of the bulk RPCs is called (i.e. label_add_hosts) and some of the objects (hosts in this case) do not exist.



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2403 592f7852-d20e-0410-864c-8624ca9c26a4
fe/model_logic.py
fe/rpc_interface.py
2bab8f45adedeacbf2d62d37b90255581adc3c7d 12-Nov-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Implement sync_count. The primary change here is replacing the job.synch_type field with a synch_count field. There is no longer just a distinction between synchronous and asynchronous jobs. Instead, every job as a synch_count, with synch_count = 1 corresponding to the old concept of synchronous jobs. This required:
-changes to the job creation RPC and corresponding client code in AFE and the CLI
-massive changes to the scheduler to schedule all jobs in groups based on synch_count (this unified the old synch and async code paths)
-changed results directory structure to accomodate synchronous groups, as documented at http://autotest.kernel.org/wiki/SchedulerSpecification, including widespread changes to monitor_db and a change in AFE
-changes to AFE abort code to handle synchronous groups instead of just synchronous jobs
-also got rid of the "synchronizing" field in the jobs table, since I was changing the table anyway and it seems very likely now that that field will never be used

other changes included:
-add some logging to afe/models.py to match what the scheduler code does, since the scheduler is starting to use the models more
-added checks for aborts of synchronous groups to abort_host_queue_entries RPC



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2402 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/001_rpc_test.txt
fe/model_logic.py
fe/models.py
fe/rpc_interface.py
fe/rpc_utils.py
lient/src/autotest/afe/AbortSynchronousDialog.java
lient/src/autotest/afe/AfeUtils.java
lient/src/autotest/afe/CreateJobView.java
lient/src/autotest/afe/JobDetailView.java
lient/src/autotest/afe/TestSelector.java
lient/src/autotest/public/AfeClient.html
lient/src/autotest/public/afeclient.css
igrations/022_implement_sync_count.py
hread_local.py
51eb7f2424fb18ee15ff8eef8372d2d6a4d5b1e5 11-Nov-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Disable test type selector when "edit control file" mode is entered.



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2400 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/TestSelector.java
12eafffaf869570a62025f4dc1f3d0908285998c 06-Nov-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Add support for running a test over a sequence of multiple kernels.
The kernel version is appended to the test name before the tag when
making directories for test output.

Signed-off-by: Gregory Smith <gps@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2382 592f7852-d20e-0410-864c-8624ca9c26a4
fe/control_file.py
fe/doctests/001_rpc_test.txt
45ae819b2d2a67d0882edafaf1a8f7b95c3fb9d2 05-Nov-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Add a formal cleanup phase to the scheduler flow.
-add a --cleanup or -C option to autoserv, which runs a new control segment, cleanup. this option and control segment obsolete the old -b option and reboot_segment control segment.
-change the RebootTask in the scheduler into a more generic CleanupTask, which calls autoserv --cleanup.
-change the host status "Rebooting" to "Cleaning"



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2377 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
5deb67762f941ba8c8a5810795d7a2d8a55575da 04-Nov-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Add two missing status descriptions to the AFE dictionary of status messages, for "Parsing" and "Stopped".



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2375 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpc_interface.py
fa8629c3a28b0ccebbd339218883e5e6cbb1ce16 04-Nov-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> -ensure Django connection is autocommit enabled, when used from monitor_db
-fix HostScheduler to not crash when there are no ready hosts
-change RebootTask to optionally take a queue entry and pass it to the RepairTask if reboot fails. This allows jobs to be failed if the pre-verify reboot fails, instead of being left hanging.
-add unit test for RebootTask
-add check for DB inconsistencies to cleanup step. Currently this just checks for HQEs with active=complete=1.
-when unexpected existing results files are found, email a warning



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2368 592f7852-d20e-0410-864c-8624ca9c26a4
etup_django_environment.py
97aed504f709270614ccbcef299e394333a76598 04-Nov-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Rewrite final reparse code in scheduler. the final reparse is now handled by a separate AgentTask, and there's a "Parsing" status for queue entries. This is a cleaner implementation that allows us to still implement parse throttling with ease and get proper recovery of reparses after a system crash fairly easily.



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2367 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
a3ab0d56117c0d55f768d5817284c2c2a0b0305d 03-Nov-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> -change AFE abort code to always set to "Abort" status and never skip straight to "Aborted". Doing so is prone to a race condition with the scheduler. The scheduler handles a non-active "Abort" entries perfectly already, setting them immediately to "Aborted" without trying to kill anything.
-change scheduler timeout code to use AFE models abort code instead of it's own SQL version. unfortunately we need need a fragment of SQL to do the timeout computation, which means no testability under SQLite. I manually tested it.
-also extracted the scheduler periodic "cleanup" code into its own method.



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2365 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/001_rpc_test.txt
fe/models.py
9886397ceb7c752db78a6acd9737992db891015b 29-Oct-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Add job start timeout for synchronous jobs. This timeout applies to synchronous jobs that are holding a public pool machine (i.e. in the Everyone ACL) as "Pending". This includes a new global config option, scheduler code to enforce the timeout and a unit test.

Note that the new scheduler code uses the Django models instead of making DB queries directly. This is a first example of how the scheduler can use the models to simplify DB interaction and reuse code from the frontend. I'd like to move in this direction from now on, although I certainly won't be making any sweeping changes to rewrite existing code.



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2358 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
38c7ba581b25be4ceb36105029a295ea05d784a4 27-Oct-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Ensure MachineLabelField never has a null labels list.



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2340 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/tko/MachineLabelField.java
291476fb2561cbd02d465d763e39686cf4c7a611 27-Oct-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Handle null values properly in updateObject(). When it's called from TestGroupDataSource.updateData(), source may be null.



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2339 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/common/Utils.java
0fb9335593fed571e94b9242e4afdaf298153dfe 27-Oct-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Fix two bugs with machine label fields.
-ensure they get completely removed from data structures when they're removed.
-make the swap link work correctly by making selectItems handle MachineLabelHeaders.



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2338 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/tko/HeaderSelect.java
71a3c720d13be42da2f35a298fce05bfa560725e 27-Oct-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Add help link to create job tab and make it stand out.



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2336 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/public/AfeClient.html
f248952e42ea33c34e41a49817e50f98c65c2716 24-Oct-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Add feature to make spreadsheet header fields from combinations of machine labels. The user can create as many different machine-label-based fields as she wishes. For each field, she can enter a list of labels to be included. The field will then group on each combination of those labels.

-added new HeaderField abstract class with two implementations - SimpleHeaderField for normal fields and MachineLabelField for the new machine label fields
-made HeaderSelect capable of creating MachineLabelFields. In single header mode, selecting "Machine labels..." creates one, and deselecting it destroys it. In multiple header mode, each time "Machine labels..." is selected a new machine labels field is created, and deselecting one destroys it.
-made HeaderSelect display text boxes for each MachineLabelField for the user to input the label list.
-created HeaderSelect.addQueryParameters, moved fixed value logic into it (from SpreadsheetView.java), and put logic for machine label header in it.
-made TestGroupDataSource accept raw query parameters, and updated SpreadsheetDataProcessor to pass it through.
-modified SpreadsheetView to use HeaderFields throughout. Eventually other code (such as TableView) should be made to use them.
-added capability for ConditionTestSet to accept raw condition pieces. Eventually it will only work this way and I'll get rid of the field setting logic, since that's been moved to SimpleHeaderField.
-added ExtendedListBox class containing a bunch of utilities for ListBoxes that I've wanted for a long time. Several other parts of the code (DoubleListSelector, some of the graphing stuff) should be changes to use these utilities eventually.
-added ChangeListener support to DoubleListSelector
-made rpc interface accept a new "machine_label_headers" parameters, and added logic to tko_rpc_utils.py to construct SQL for machine label headers
-modified TestView manager to support a join into host labels



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2331 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/HostSelector.java
lient/src/autotest/common/Utils.java
lient/src/autotest/common/ui/DoubleListSelector.java
lient/src/autotest/common/ui/ExtendedListBox.java
lient/src/autotest/tko/ConditionTestSet.java
lient/src/autotest/tko/HeaderField.java
lient/src/autotest/tko/HeaderSelect.java
lient/src/autotest/tko/MachineLabelField.java
lient/src/autotest/tko/SimpleHeaderField.java
lient/src/autotest/tko/SpreadsheetDataProcessor.java
lient/src/autotest/tko/SpreadsheetView.java
lient/src/autotest/tko/TableView.java
lient/src/autotest/tko/TestGroupDataSource.java
lient/src/autotest/tko/TkoUtils.java
0fc3830f17d644bab74bfe38556299f5e58bc0fa 23-Oct-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Add user preferences for reboot options, including simple user preferences tab which could later be expanded to include more options.


git-svn-id: http://test.kernel.org/svn/autotest/trunk@2330 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/001_rpc_test.txt
fe/models.py
fe/rpc_interface.py
lient/src/autotest/afe/AfeClient.java
lient/src/autotest/afe/AfeUtils.java
lient/src/autotest/afe/CreateJobView.java
lient/src/autotest/afe/HostTableDecorator.java
lient/src/autotest/afe/JobListView.java
lient/src/autotest/afe/UserPreferencesView.java
lient/src/autotest/common/StaticDataRepository.java
lient/src/autotest/public/AfeClient.html
lient/src/autotest/tko/SavedQueriesControl.java
igrations/021_add_user_reboot_prefs.py
1a9eaa51c52c10953e3d296a7b529126eab2862e 23-Oct-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Fix the test selector in AFE create job view.
-make TestSelector handle null DB values more gracefully
-include a previous change from number -> boolean that got left behind


git-svn-id: http://test.kernel.org/svn/autotest/trunk@2329 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/TestSelector.java
e78760cb696ef36fd1c58fd21a80282255a8ca95 23-Oct-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> TableDecorator needs to create a DynamicTableSelectionManager for "select all" to work correctly.


git-svn-id: http://test.kernel.org/svn/autotest/trunk@2328 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/common/table/SelectionManager.java
lient/src/autotest/common/table/TableDecorator.java
b7920902a2d05712bc61dabb57b9bc62a80f39a4 21-Oct-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Migration script for job reboot changes.



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2319 592f7852-d20e-0410-864c-8624ca9c26a4
igrations/020_add_host_dirty_and_job_reboots.py
21baa459ea14f96e06212f1f35fcddab9442b3fc 21-Oct-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Add options to control reboots before and after a job.

-add reboot_before and reboot_after fields to Job, along with enums for each
-add options to create_job RPC for reboot_before and reboot_after
-add options to job create CLI for these fields, and made job stat -v display them
-add widgets to job create page in AFE for these fields and made job detail view display them

-add dirty field to Hosts, defaulting to True, and set to True when a host is locked
-made scheduler set this field when a job runs and clear it when a host is rebooted

-updated scheduler's PidfileRunMonitor to read a new three-line .autoserv_execute format, where the third line contains the number of tests that failed
-made scheduler Job.run() include a RebootTask before the verify task according to the reboot_before option
-made QueueTask.epilog() launch a RebootTask for each host according to the reboot_after option

-updated autoserv to write out a third line to .autoserv_execute containing the number of failed tests.

Other changes:
-added support for displaying Job.run_verify in the CLI (job stat -v) and job detail page on AFE
-updated ModelExtensions to convert BooleanField values to actual booleans. The MySQL Django backend just leaves them as ints (as they are represented in the DB), and it's stupid and annoying (Yes, bool is a subclass of int, so it's often not a problem. But yes, it can be.).
-get rid of use of Job.synch_count since we don't actually support it. I think this was meant for inclusion in a previous change and got left out.
-made the scheduler use the new setup_django_environment stuff to import and use the django models. It doesn't *really* use the models yet -- it just uses the Job.Reboot{Before,After} enum objects -- but this shows we could easily start using the models, and that's definitely the direction I want to go long term.
-refactored PidfileRunMonitor generally and made it a bit more robust by having it email errors for corrupt pidfiles and continue gracefully, instead of just crashing the scheduler
-changed the way Agent.tick() works. now, it basically runs through as much work as it can in a single call. for example, if there's a RebootTask and a VerifyTask, and the RebootTask has just finished, in a single call it will finish up the RebootTask and start the VerifyTask. this used to take two cycles and that was problematic for cases like this one -- the RebootTask would like to set host.status=Ready, but then the host could get snatched up on the next scheduling round, before the VerifyTask got started. This was sort of solved previously by keeping the HostQueueEntry active, and we could apply that approach here by making a new status for HostQueueEntries like "Rebooting". But I prefer this approach as I think it's more efficient, more powerful and easier to work with.

Risk: extremely high
Visibility: new reboot options for jobs, skip verify now displayed in AFE + CLI

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@2308 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/001_rpc_test.txt
fe/model_logic.py
fe/models.py
fe/rpc_interface.py
lient/src/autotest/afe/AfeUtils.java
lient/src/autotest/afe/CreateJobView.java
lient/src/autotest/afe/JobDetailView.java
lient/src/autotest/afe/JobStatusDataSource.java
lient/src/autotest/common/ui/RadioChooser.java
lient/src/autotest/public/AfeClient.html
lient/src/autotest/public/afeclient.css
364fe862f88bcaaefaa40dd145a777bee840ec9b 17-Oct-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Refactor the basic environment setup code out of django_test_utils.py into setup_django_environment.py, and rename django_test_utils.py to setup_test_environment.py. Also changed the environment setup code to run at import time. This makes it easy for scripts, both test and non-test, to use Django models without running through manage.py. The idea is that scripts will import setup_django_environment before importing Django code (somewhat akin to common.py), and test code will subsequently import setup_test_environment.



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2297 592f7852-d20e-0410-864c-8624ca9c26a4
jango_test_utils.py
rontend_unittest.py
etup_django_environment.py
etup_test_environment.py
cfd66a35e8c939998ff354d9740bdf15c9bc3fda 15-Oct-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Make scheduler set host status to "Pending" when there's a pending queue entry against the host.


git-svn-id: http://test.kernel.org/svn/autotest/trunk@2292 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
9976ce9873867a397e448d358543a9dc1d33aa77 15-Oct-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> -make monitor_db implement "skip verify" properly, and add unit tests for it
-change order of a couple fields in AFE models to match DB order



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2291 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
9dbdcda5104991cbf344ea5cba1aa58e1af444f3 14-Oct-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Add feature to abort individual host queue entries the job detail and host detail pages. Performed a few other cleanups along the way.
-refactor TableActionsPanel, extract separate TableSelectionPanel
-make TableDecorator support a SelectionManager and a TableActionsPanel (or selection panel or other such control) just above the paginator. this makes it really easy to add selection capability, selection checkboxes, and an actions menu to any table.
-refactor TableDecorator in general since it kinda sucked
-change all existing code that does table selection to use the new TableDecorator support. this include job list view, the create job host selector (which now uses the common selection links instead of its old buttons), and tko.TableView.
-add selection support + a bulk abort action to the tables in job detail view and host detail view. this itself was easy given the above refactorings.


git-svn-id: http://test.kernel.org/svn/autotest/trunk@2283 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/001_rpc_test.txt
fe/models.py
fe/rpc_interface.py
lient/src/autotest/afe/AbortSynchronousDialog.java
lient/src/autotest/afe/AfeUtils.java
lient/src/autotest/afe/HostDetailView.java
lient/src/autotest/afe/HostSelector.java
lient/src/autotest/afe/JobDetailView.java
lient/src/autotest/afe/JobListView.java
lient/src/autotest/afe/JobStatusDataSource.java
lient/src/autotest/afe/JobTable.java
lient/src/autotest/common/table/JSONObjectSet.java
lient/src/autotest/common/table/SelectionManager.java
lient/src/autotest/common/table/TableDecorator.java
lient/src/autotest/common/ui/TableActionsPanel.java
lient/src/autotest/common/ui/TableSelectionPanel.java
lient/src/autotest/public/AfeClient.html
lient/src/autotest/public/TkoClient.html
lient/src/autotest/tko/SpreadsheetView.java
lient/src/autotest/tko/TableView.java
68c7aa02743042196b1f4ce3d41cbe03d1152ec0 09-Oct-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Don't rely on auto_now_add in Django, since it is apparently not
reliable.

Risk: Low
Visibility: No more supposedly auto-add date fields set to zero in
the database instead of now.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2258 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
f213eb27c9f5ce24cd60e1f7598624fcd71daa75 08-Oct-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Fix a bug with preserving ordering of group count/status count columns when restoring from history. The old code would always reset these columns to the end of the column list.



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2256 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/tko/TableView.java
2e490cc4d9aaaf7419db0c8bd7271171dcf747fa 08-Oct-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> -refactor frontend/afe/test.py to make it reusable by new_tko
-make frontend_unittest.py use code in django_test_utils instead of its own copy



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2255 592f7852-d20e-0410-864c-8624ca9c26a4
fe/test.py
rontend_unittest.py
ettings.py
b1e5187f9aa303c4fc914f07312286d302b46a0e 07-Oct-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Get the scheduler unittest to run against SQLite!

* get rid of monitor_db.DatabaseConn, and make monitor_db use the new DatabaseConnection
* modify some queries in monitor_db that weren't SQLite-compatible (SQLite doesn't support TRUE and FALSE literals)
* add frontend/django_test_utils.py, which contains utilities to
* setup a django environment (something manage.py normally does for you)
* replace the configured DB with a SQLite one, either in-memory or on disk
* run syncdb on the test DB
* backup and restore the test DB, handy because then we can syncdb once, save the fresh DB, and quickly restore it between unittests without having to run syncdb again (syncdb is terribly slow for whatever reason)
* modify monitor_db_unittest to use these methods to set up a temporary SQLite DB, run syncdb on it, and test against it
* replace much of the data modification code in monitor_db_unittest with use of the django models. The INSERTs were very problematic with SQLite because syncdb doesn't set database defaults, but using the models solves that (django inserts the defaults itself). using the models is much cleaner anyway as you can see. it was just difficult to do before, but now that we've got the infrastructure to setup the environment anyway, it's easy. this is a good model for how we can make the scheduler use the django models eventually.
* reorder fields of Label model to match actual DB ordering; this is necessary since monitor_db depends on field ordering
* add defaults to some fields in AFE models that should've had them
* make DatabaseConnection.get_test_database support SQLite in files, which gives us persistence that is necessary and handy in the scheduler unittest
* add a fix to _SqliteBackend for pysqlite2 crappiness

The following are extras that weren't strictly necessary to get things working:
* add a debug feature to DatabaseConnection to print all queries
* add an execute_script method to DatabaseConnection (it was duplicated in migrate and monitor_db_unittest)
* rename "arguments" to "parameters" in _GenericBackend.execute, to match the DB-API names
* get rid of some debug code that was left in monitor_db, and one unnecessary statement

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@2252 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
fe/rpc_interface.py
jango_test_utils.py
ac363b115caf922e362dc311638b1a53d09e4f1d 07-Oct-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> -fix frontend/generate-javadoc script for new (although now quite old) package organization
-increase java runtime stack size for TkoClient-compile, which was mysteriously running out for some users


git-svn-id: http://test.kernel.org/svn/autotest/trunk@2250 592f7852-d20e-0410-864c-8624ca9c26a4
lient/TkoClient-compile
lient/generate-javadoc
ff72fe3c9b8d95235c12328fe876459d0f7e9a95 07-Oct-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> add debug urlconf to frontend/urls.py to make development shell work smoother


git-svn-id: http://test.kernel.org/svn/autotest/trunk@2249 592f7852-d20e-0410-864c-8624ca9c26a4
rls.py
56e9377c6d618ba478bc2ca92679830ece8dff52 06-Oct-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Get the frontend unittest to run against SQLite. This required scattered changes:
* change frontend_unittest to tell django to use an in-memory SQLite database. ideally this would've been the only change necessary, but it wasn't.
* change long to ints in the doctests, as that's what pysqlite2 returns
* change several imports to use autotest_lib; that's the way everything should be, and the other way breaks things when trying to run stuff without using manage.py
* make readonly_connection better support testing by changing its connection attribute into a method, and adding the capability to disable the module completely during testing
* get rid of use of GROUP_CONCAT in models.py; SQLite doesn't support it (not in our old version anyhow), and it;s really not necessary anyway



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2244 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/001_rpc_test.txt
fe/doctests/003_misc_rpc_features.txt
fe/management.py
fe/model_logic.py
fe/models.py
fe/readonly_connection.py
fe/rpc_interface.py
rontend_unittest.py
7c199df48dbc7a8200c8a80a163be4613c405cb8 03-Oct-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Fix a couple of little bugs in TKO.
-make sure selected fields get properly referenced by their table to avoid ambiguity errors
-centralize functionality for escaping user SQL - I did this because I was messing around with the code, but ended up leaving it unmodified. But it's better centralized.
-change the way extra_select_fields are passed, to use a dictionary mapping alias to SQL instead of just passing the plain SQL with AS included. This is how Django does it, and it's more safe and correct (in this case, it helps us automatically quote field names properly).
-clean up SQL for completed test count a bit
-replace "escapeQuotes" on client with "escapeSqlValue", which actually does the right thing, and get rid of TODO to this effect. This was causing a bug when trying to drilldown on complex values (like long reasons)



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2235 592f7852-d20e-0410-864c-8624ca9c26a4
fe/model_logic.py
lient/src/autotest/tko/ConditionTestSet.java
0e73c855576b97891ec38b6512bf040f3a1e1e40 03-Oct-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Add a generic database wrapper, supporting different database backends, to be used by migrate, scheduler, parser (eventually), and maybe others. This will consolidate the multiple database wrappers we have throughout the code and allow us to swap in SQLite for MySQL for unit testing purposes.

-add database/ directory for database libraries. migrate.py will move here soon.
-add database_connection.py under server_common, a basic database wrapper supporting both MySQL and SQLite. PostgreSQL should be an easy future addition (any library supporting Python DB-API should be trivial to add). DatabaseConnection also supports graceful handling of dropped connections.
-add unittest for DatabaseConnection
-change migrate.py to use common DatabaseConnection. Scheduler will be changed to use it in a coming CL and in the future hopefully the TKO parser will be able to use it as well.
-change migrate_unittest.py to use SQLite.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@2234 592f7852-d20e-0410-864c-8624ca9c26a4
igrations/001_initial_db.py
53e20174e849edc9a8d1864251a0981cc4333d8b 02-Oct-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> The 'locked_by_id' field was incorrectly named on the 'migrate_down' method to 'locked_by'. Which caused an error when 'locked_by' field was being dropped from the hosts table. I changed 'locked_by' to 'locked_by_id' and tested on my client and this corrected the error.

Signed-off-by: Jeremy Dawson <jdawson@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2230 592f7852-d20e-0410-864c-8624ca9c26a4
igrations/015_add_locked_by_and_lock_time.py
9a1f2e1062efdef97e3d229dbdc941b61545e7b2 02-Oct-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Two bug fixes:
* When aborting host queue entries upon job deletion, need to pass the current user, since abort() now accepts an aborted_by param.
* Fixed bug in JobManager.populate_dependencies(), where it would construct an invalid SQL query because it assumed the input list wasn't empty.



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2221 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
8a6eb0cf5777dded2354408e8007d9223e813c92 01-Oct-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> add support for showing only the latest test run per cell in spreadsheet mode. this involved some extensive refactorings on both the client and server which then made the actual change quite simple.

Refactorings:
-refactored group querying code in TempManager to be much more general (it's no longer oriented around fetching COUNT(*)). it also now returns dicts instead of raw rows.
-refactored much of the logic from the group-related RPCs into GroupDataProcessor, which now handles almost all the work, again in a more general fashion
-made group RPCs always return a sample test index with each group; this allows the client to drill down to single-test groups without having to make an extra query to get the test index (it also makes the latest test feature easier to implement)
-refactor TestSet to support a new getTestIndex method for single test sets
-added new SingleTestSet class for single test sets, and made TestSet creation code use this instead of ConditionTestSet
-made ConditionTestSet always be for multiple tests
-changed drilldown code to use TestSet.getTestIndex() to avoid making an extra RPC call (in both spreadsheet and table views)
-got rid fo TkoUtils.getTestId, which is no longer needed since test IDs are passed down with the groups

New features:
-added get_latest_tests RPC to get the latest test per group, but still return information in the same format as get_status_counts
-added "show only latest test per cell" to spreadsheet view, made it control the RPC that gets called, and added history support for it

About the get_latest_tests RPC - it uses two rather simple SQL queries and some processing in Python. I tried six different ways of computing this information, some using a single SQL query to do everything and some doing everything in Python, and this approach was by far the fastest.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@2216 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/tko/CompositeTestSet.java
lient/src/autotest/tko/ConditionTestSet.java
lient/src/autotest/tko/SingleTestSet.java
lient/src/autotest/tko/Spreadsheet.java
lient/src/autotest/tko/SpreadsheetDataProcessor.java
lient/src/autotest/tko/SpreadsheetView.java
lient/src/autotest/tko/TableView.java
lient/src/autotest/tko/TestContextMenu.java
lient/src/autotest/tko/TestGroupDataSource.java
lient/src/autotest/tko/TestSet.java
lient/src/autotest/tko/TkoUtils.java
989f25dcbb6361218f0f84d1c8404761b4c39d96 01-Oct-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> two new major features:

(1) added test and job dependencies
-added M2M relationship between tests and labels and between jobs and labels, for tracking the labels on which a test/job depends
-modified test_importer to read the DEPENDENCIES field and create the right M2M relationships
-modified generate_control_file() RPC to compute and return the union of test dependencies. since generate_control_file now returns four pieces of information, i converted its return type from tuple to dict, and changed clients accordingly.
-modified job creation clients (GWT and CLI) to pass this dependency list to the create_job() RPC
-modified the create_job() RPC to check that hosts satisfy job dependencies, and to create M2M relationships
-modified the scheduler to check dependencies when scheduling jobs
-modified JobDetailView to show a job's dependencies

(2) added "only_if_needed" bit to labels; if true, a machine with this label can only be used if the label is requested (either by job dependencies or by the metahost label)
-added boolean field to Labels
-modified CLI label creation/viewing to support this new field
-made create_job() RPC and scheduler check for hosts with such a label that was not requested, and reject such hosts

also did some slight refactoring of other code in create_job() to simplify it while I was changing things there.

a couple notes:
-an only_if_needed label can be used if either the job depends on the label or it's a metahost for that label. we assume that if the user specifically requests the label in a metahost, then it's OK, even if the job doesn't depend on that label.
-one-time-hosts are assumed to satisfy job dependencies.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@2215 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/001_rpc_test.txt
fe/doctests/003_misc_rpc_features.txt
fe/models.py
fe/rpc_interface.py
fe/rpc_utils.py
lient/src/autotest/afe/CreateJobView.java
lient/src/autotest/afe/JobDetailView.java
lient/src/autotest/public/AfeClient.html
igrations/018_add_label_only_if_needed.py
751f55016802ec9995a3bad3b60d5d017e35dbde 30-Sep-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Forgot to add this file, as part of the test attributes filtering patch



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2205 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/tko/WidgetList.java
8c9b839c2f5073a755952a8a865a04db3b2d4547 30-Sep-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> add primitive support for fixed header values. fixed headers can be given as a comma- or whitespace-separated list of values. they will serve to both limit to results to the given values (something that could just as well be done with the WHERE clause, but I figured people would want this behavior) + force all given values to be included, even if it causes empty rows.
* add support to GroupDataProcessor + relevant RPCs in Django to support fixed header values, but *only* for single headers (i.e. no composite header support)
* add text box to GWT spreadsheet UI, accessible only in single-header mode. box accepts comma- or whitespace-separated list of values.
* add necessary code to process fixed headers + pass them to server, and to store in history URL



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2204 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/HostSelector.java
lient/src/autotest/common/Utils.java
lient/src/autotest/tko/HeaderSelect.java
lient/src/autotest/tko/SpreadsheetDataProcessor.java
lient/src/autotest/tko/SpreadsheetView.java
lient/src/autotest/tko/TestGroupDataSource.java
11bd063f741c3bcfc59cadfed4abc65686df10ef 30-Sep-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> add test attributes to test detail view, in an expandable box


git-svn-id: http://test.kernel.org/svn/autotest/trunk@2202 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/public/TkoClient.html
lient/src/autotest/public/tkoclient.css
lient/src/autotest/tko/TestDetailView.java
4c119049441b360505464f31cfd113b5dc90b241 29-Sep-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Add support to the frontend for logging a user and timestamp when a
job is aborted. Doesn't actually expose this information anywhere
(yet).

Risk: Medium
Visibility: A new table with extra log info about aborted host queue
entries will be written to when jobs are aborted.

Signed-off-by: John Admanski <jadmanski@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@2201 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
igrations/019_add_abort_log.py
e732ee7d450b11261c82df0950fde8e02f839b26 23-Sep-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> -added capability to have site-specific urlconfs in TKO Django server
-added ClassFactory and SiteClassFactory to autotest.tko package to allow site-specific modifications to TKO. made some changes to the AFE [Site]ClassFactory files (reducing visibility)
-added get_detailed_test_views RPC which includes information about labels and attributes associated with a test
-made changes to model_logic.py to make fewer assumptions about models. these were basically bugs in model_logic.py that weren't exposed until i started doing fancier queries on TKO models.
-made list_objects capable of accepting a field list
-changed JsonRpcProxy to contain base URLs instead of full RPC urls; this way, other components (graphing, jlog) can use them (and i made graphing use this URL, which fixes a bug in the embedded URL generation)
-added RealHyperlink widget to common.ui package, representing a plain old hyperlink to another page (as opposed to Hyperlink and SimpleHyperlink, which are intended to be handling by the GWT code). this eases dynamic updates of the link href and allows setting visibility, styles, and all the other good stuff that comes along with using Widgets.



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2191 592f7852-d20e-0410-864c-8624ca9c26a4
fe/model_logic.py
lient/src/autotest/afe/AfeClient.java
lient/src/autotest/afe/ClassFactory.java
lient/src/autotest/afe/SiteClassFactory.java
lient/src/autotest/common/JsonRpcProxy.java
lient/src/autotest/common/Utils.java
lient/src/autotest/common/ui/RealHyperlink.java
lient/src/autotest/public/TkoClient.html
lient/src/autotest/tko/ClassFactory.java
lient/src/autotest/tko/GraphingFrontend.java
lient/src/autotest/tko/SiteClassFactory.java
lient/src/autotest/tko/SpreadsheetView.java
lient/src/autotest/tko/TestDetailView.java
lient/src/autotest/tko/TkoClient.java
lient/src/autotest/tko/TkoUtils.java
6b5a72c12b84e0e8f15839c641bbc718fe52cbd7 22-Sep-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> The new code for building condition strings was potentially buggy when using multiple selection.


git-svn-id: http://test.kernel.org/svn/autotest/trunk@2181 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/tko/TkoUtils.java
50c0e71efd37c04c5f441c7e4dd095632b54c35f 22-Sep-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> -add --force option to migrations to disable user confirmation because this can make migrations unscriptable
-add default migration dirs to migrate.py because having to cd to the right place was getting really annoying!
-made monitor_db_unittest.py use migrations to initialize the test DB schema instead of copying the schema from the real DB, since that was creating problems for testing. this slows down the test considerably, but it's better than no testing at all (and we can improve it in the future).
-made global_config allow overriding options, which is useful for testing

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@2180 592f7852-d20e-0410-864c-8624ca9c26a4
igrations/001_initial_db.py
6e745dbb056debdcc78bd1172644d8081eb4bfbb 22-Sep-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Update wiki links to point to new trac thingy.



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2179 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/public/TkoClient.html
64aeecdec485192241e5377b3fa5ac7cf57a0c12 19-Sep-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> -add feature to filter on test attributes in TKO
-new server arguments "include_attributes_where" and "exclude_attributes_where" for filtering on test attributes
-refactor joining code in TKO models.py to support test attributes joining
-add new UI to CommonPanel.java to filter on test attributes. some of the UI code was written in a general way so that in the future it could be merged with some of the graphing UI code.
-modified TestSets and code that uses them to fix two bugs - first, TestSets didn't contain all the relevant filtering information (only the SQL clause), and second, the SQL clause would build up incorrectly during drilldown



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2177 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/common/Utils.java
lient/src/autotest/tko/CommonPanel.java
lient/src/autotest/tko/CompositeTestSet.java
lient/src/autotest/tko/ConditionTestSet.java
lient/src/autotest/tko/MachineQualHistogramFrontend.java
lient/src/autotest/tko/SpreadsheetView.java
lient/src/autotest/tko/TableView.java
lient/src/autotest/tko/TestContextMenu.java
lient/src/autotest/tko/TestLabelManager.java
lient/src/autotest/tko/TestSet.java
lient/src/autotest/tko/TkoClient.java
lient/src/autotest/tko/TkoUtils.java
542e840486b02b5025d26da16f98fed97898a601 19-Sep-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Added email_list field to front end. On job completion emails on this
list will be notified of the completion. Also the function send_email
in manage_db.py adds the ability to put a delimited email list in
global_settings.py rather than just a single email.

Signed-off-by: Bryce Boe <bboe@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@2173 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/001_rpc_test.txt
fe/models.py
fe/rpc_interface.py
lient/src/autotest/afe/CreateJobView.java
lient/src/autotest/afe/JobDetailView.java
lient/src/autotest/public/AfeClient.html
igrations/017_add_email_list.py
ce12f55f4530950f7fc6ddc73d1867ebb954d356 19-Sep-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Attached is a large patch for a powerful and flexible new graphing system for new TKO. This system subsumes all the previous kernel graphing scripts under the tko/ directory and is capable of much more. These wiki pages document usage of the new system and give an idea of what it's capable of:

http://autotest.kernel.org/wiki/MetricsPlot
http://autotest.kernel.org/wiki/MachineQualHistograms

Feel free to try it out and please let us know if you run into any trouble.

This system is the work of our summer intern James Ren. Thank you for all your fantastic work, James!

From: James Ren <jamesren@stanford.edu>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2171 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/public/TkoClient.html
lient/src/autotest/tko/DBColumnSelector.java
lient/src/autotest/tko/ExistingGraphsFrontend.java
lient/src/autotest/tko/FilterSelector.java
lient/src/autotest/tko/FilterStringViewer.java
lient/src/autotest/tko/GraphingFrontend.java
lient/src/autotest/tko/GraphingView.java
lient/src/autotest/tko/MachineQualHistogramFrontend.java
lient/src/autotest/tko/MetricsPlotFrontend.java
lient/src/autotest/tko/PreconfigSelector.java
lient/src/autotest/tko/SeriesSelector.java
lient/src/autotest/tko/SpreadsheetView.java
lient/src/autotest/tko/TableView.java
lient/src/autotest/tko/TkoClient.java
3de2a76fa5e272503f924b5cd9d100931bed1e7f 12-Sep-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Add a unit test to check that the GWT code compiles. I added it as a "long" test because it requires GWT to be set up (and it's kind of slow).



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2138 592f7852-d20e-0410-864c-8624ca9c26a4
lient_compilation_unittest.py
1a573ca82a41152550731dd82bb18d3df2e26ef0 11-Sep-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Allow a different host for readonly connections.

Signed-off-by: Jeremy Orlow <jorlow@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@2131 592f7852-d20e-0410-864c-8624ca9c26a4
fe/readonly_connection.py
ettings.py
f484b97027f3ce357b4f1ae6dbbc98eaebd692da 09-Sep-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Change the "Wiki" links to point to the new trac page.

Risk: low
Visibility: low

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@2117 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/public/AfeClient.html
lient/src/autotest/public/TkoClient.html
d50ffb4b0ef514fb969d53b82e23ab41d4d3812e 04-Sep-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> -add easy invalidation functionality
-add "show invalidated tests" option to common panel, disabled by default
-made client submit "exclude_labels" option to exclude invalid tests. this required somewhat widespread changes because it means the global condition is no longer just a SQL string but now a collection of parameters
-add "invalidate tests" option to spreadsheet/table context menu, and button to test detail view. it's really just a shortcut to add the "invalidated" label.
-added logic to the server to handle "exclude_labels" option. it was done in this generic way because in the future i plan to add a UI to exclude any label or labels.
-force test label names to be unique
-fix a bug in logic to determine all labels assigned to a set of tests
-got rid of auto-refresh when changing between spreadsheet and table after the condition had changed

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@2099 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/common/table/SimpleFilter.java
lient/src/autotest/public/TkoClient.html
lient/src/autotest/tko/CommonPanel.java
lient/src/autotest/tko/ConditionTabView.java
lient/src/autotest/tko/ConditionTestSet.java
lient/src/autotest/tko/SpreadsheetDataProcessor.java
lient/src/autotest/tko/SpreadsheetView.java
lient/src/autotest/tko/TableView.java
lient/src/autotest/tko/TestContextMenu.java
lient/src/autotest/tko/TestLabelManager.java
lient/src/autotest/tko/TkoUtils.java
945072fbd2a19409649de3afe2074c3f09fe8e0a 03-Sep-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Change the Clone Job button to show options to keep the specific
metahost assignments.

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@2090 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpc_interface.py
lient/src/autotest/afe/JobDetailView.java
a77e26761e00d50b5dedf1b63fed0c3706f0ddc7 03-Sep-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Changed some tabs to spaces in TkoClient.html.

Risk: low
Visibility: low

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@2088 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/public/TkoClient.html
5b3439bc1bdd2cce58bef49bd2575559b1835275 03-Sep-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Fixed copying the control file for a cloned job. A listener was
unexpectedly firing and causing a new (empty) control file to be
generated.

Also changed the order that AfeClient.java clones a job and shows it,
so that the transition appears smoother (does not show the old job, and
then quickly flip to the cloned one).

Risk: low
Visibility: medium (UI bug fix)

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@2087 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/AfeClient.java
lient/src/autotest/afe/CreateJobView.java
0dc911ab62ab6cca016ad5c28b0c35e86895f588 03-Sep-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Prevent adding of blank one-time host. Blank entry or entry preceded
with whitespace will no longer add an empty one-time host.

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@2086 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/HostSelector.java
3b8563acf345e8327fb3f0cb4b96869ce55f5080 03-Sep-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Change mouse selection events to include ctrl-click and meta-click (which is command-click on the mac and windows-click on windows, but the former is the important one).



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2085 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/tko/ConditionTabView.java
lient/src/autotest/tko/SpreadsheetView.java
lient/src/autotest/tko/TableView.java
699b07928d798ab02cefc5b497cebde588bee579 22-Aug-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Fix the import usage for the frontend doctest.
This part of the unittest fixes had not been patched at the time because of a dependency.

Signed-off-by: Jean-Marc Eurin <jmeurin@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2023 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/001_rpc_test.txt
0281350135d1216c722e1d6fb85044f505c2319b 20-Aug-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> -rewrite most of the label management code to be much more efficient
-ensure label modifications always act on the right set of tests, by ensuring the GWT client uses the right SQL condition



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2017 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/tko/SpreadsheetView.java
21085f22a3c616ff12bf80b997187a00e44f851b 19-Aug-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> -fix bug with drilldown on null headers
-make spreadsheet view ignore TEST_NA entries

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@2012 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/common/Utils.java
lient/src/autotest/tko/ConditionTestSet.java
lient/src/autotest/tko/SpreadsheetView.java
3c0f07d0127913043c2c704a6a374e20b62f6b30 19-Aug-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Several small TKO-related bugfixes/improvements:
-include show_incomplete option in afe "new results interface" link
-escape single quotes in history URLs
-reset table view to default when using "view in table" feature from spreadsheet



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2011 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/JobDetailView.java
lient/src/autotest/common/Utils.java
lient/src/autotest/tko/TableView.java
lient/src/autotest/tko/TkoClient.java
8d631c9d894ee1f438733530fb8b966f7af07df1 18-Aug-2008 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Convert all our uses of directoy SSHHost construction over to use
hosts.create_host instead. Also, fix up a bunch of test references
that use SSHHost when they really just need to run against the
abstract RemoteHost class, and fix some tests using the MockSSHHost
class to just use the mock framework instead.

Risk: Low
Visibility: Gets rid of almost all direct references to SSHHost.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2010 592f7852-d20e-0410-864c-8624ca9c26a4
fe/control_file.py
8579ea343f8d4c74b44d5b5cb2df3ef7552b2f6e 18-Aug-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Snazzy new interface for select tests in Create Job view. Tests are now presented in a scrollable table, with descriptions displayed in a resizable pane on the right.
-new TestSelector widget displays table + description
-move some basic input handling from DynamicTable into DataTable, so that plain DataTables can handle clicks
-make DataTable capable of refreshing the widgets in a table without refreshing the entire table - this is useful for updating selections
-pull out the one bit of SelectionManager that depends on DynamicTable (as opposed to DataTable). This is now in DynamicTableSelectionManager, and the orignal SelectionManager can work with a plain DataTable.
-make SelectionManager capable of acting as the TableWidgetFactory for producing row selection checkboxes
-fix a couple minor bugs in (Site)CreateJobView, where some controls (skip verify, custom tests) would not be disabled when they should be
-pull out ArrayDataSource.JSONObjectComparator into a top-level class, so it could be used elsewhere to sort JSONObjects



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2004 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/CreateJobView.java
lient/src/autotest/afe/JobListView.java
lient/src/autotest/afe/TestSelector.java
lient/src/autotest/common/table/ArrayDataSource.java
lient/src/autotest/common/table/DataTable.java
lient/src/autotest/common/table/DynamicTable.java
lient/src/autotest/common/table/DynamicTableSelectionManager.java
lient/src/autotest/common/table/JSONObjectComparator.java
lient/src/autotest/common/table/SelectionManager.java
lient/src/autotest/public/AfeClient.html
lient/src/autotest/public/afeclient.css
bad4f2df9af76d1fb9dafe832fd48b500b71e60e 15-Aug-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Fixed removing hosts from a cloned job. Hosts can now be removed correctly.
Also cleaned up some code in the RPC interface method for job cloning.

Risk: low
Visibility: medium (bug fix on Create Job tab)

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1997 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpc_interface.py
lient/src/autotest/afe/CreateJobView.java
109c915cd87db289bb100559793c72902832815c 15-Aug-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> -fix bug in spreadsheet view where columns would be misaligned in certain cases. We need to fill the window before matching column widths, because in Firefox filling the window can cause columns to change width.
-fix bug in TKO condition building that would cause a lot of duplication in the condition



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1996 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/tko/CommonPanel.java
lient/src/autotest/tko/ConditionTestSet.java
lient/src/autotest/tko/Spreadsheet.java
115fe720d4226d734fb3b7b4ecdc7547c5676ef2 15-Aug-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Some AFE/TKO page layout changes:
-shrink the logo and put all links on one line, to reduce wasted space at top of page
-make the set of links the same on both AFE and TKO
-reduce the links to normal-sized font
Also fix the admin interface link from new TKO.


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1994 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/public/AfeClient.html
lient/src/autotest/public/TkoClient.html
lient/src/autotest/public/afeclient.css
lient/src/autotest/public/header.png
5fd7085c104370a45682549299d31ca3e4db0eac 13-Aug-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Make these new scripts executable.


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1982 592f7852-d20e-0410-864c-8624ca9c26a4
lient/TkoClient-compile
lient/TkoClient-shell
cc0def989c9154f3ae0b5cfda16aa18c03490683 13-Aug-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Make popup stay fixed position, and cleanup related styles


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1980 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/common/ui/NotifyManager.java
lient/src/autotest/public/afeclient.css
eb3805f0a7b5c55b9c9d9d2f976966be0bfb23c0 13-Aug-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Added a hint to the benchmark control in the TKO graphing tab. Hint
instructs user to select a host before the allowed benchmarks will
appear.

Risk: low
Visibility: medium (UI change)

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1979 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/tko/ExistingGraphsFrontend.java
d5afc2f14dad583057f1eaffbc0366eb389f70d6 12-Aug-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Fixed the protection field in the Host class, so that a default of "No
protection" can be set correctly. Also fixed the RPC interface unit
tests.

Risk: low
Visibility: low

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1976 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/001_rpc_test.txt
fe/models.py
a3e21058a3d3fabc50dc0e0573a868a644169dc9 12-Aug-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Minor performance change. Changing an inner class to be static.

Risk: low
Visibility: low

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1974 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/tko/SpreadsheetDataProcessor.java
fb0762ed9e6e20c987d0613ffdf4859d14bd720a 08-Aug-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> add spinner image during AFE/TKO loading

Signed-off-by: Steve Howard <showard@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1967 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/common/ui/NotifyManager.java
lient/src/autotest/public/mbligh_spinner.gif
0c31bc5ef2ecdf8edf19468e1a373520110f5bc6 08-Aug-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Make query controls collapsible in spreadsheet and table views. This should help enable better use of screen real estate, especially in spreadsheet view.


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1965 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/common/ui/CustomTabPanel.java
lient/src/autotest/common/ui/ElementWidget.java
lient/src/autotest/common/ui/SimpleHyperlink.java
lient/src/autotest/common/ui/TabView.java
lient/src/autotest/public/TkoClient.html
lient/src/autotest/public/afeclient.css
lient/src/autotest/tko/CommonPanel.java
lient/src/autotest/tko/ConditionTabView.java
lient/src/autotest/tko/SpreadsheetView.java
lient/src/autotest/tko/TableView.java
lient/src/autotest/tko/TkoUtils.java
981049f069b497499c625fe293754a1ad334030a 08-Aug-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Changed whitespace appearance in the Admin Interface for tests to show
newlines as entered in the description.

Risk: low
Visibility: medium (UI change)

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1962 592f7852-d20e-0410-864c-8624ca9c26a4
emplates/admin/afe/test/change_list.html
76d9739461e614e04120383294974cc300fafa9d 08-Aug-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Fixing removing metahosts from the selected hosts in Create Job.
Removing metahosts should now work properly.

Risk: low
Visibility: medium (UI bug fix)

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1961 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/CreateJobView.java
lient/src/autotest/afe/HostSelector.java
lient/src/autotest/common/table/ArrayDataSource.java
a8411aff779fc965537d8470268107b5a87e0346 08-Aug-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Validate one-time hosts to avoid empty hostnames.


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1960 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
35444864c7b6f49865a7e17aa0052987b72e4728 08-Aug-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Initial checkin of new TKO interface.



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1959 592f7852-d20e-0410-864c-8624ca9c26a4
lient/TkoClient-compile
lient/TkoClient-shell
lient/TkoClient.launch
lient/src/autotest/TkoClient.gwt.xml
lient/src/autotest/afe/JobDetailView.java
lient/src/autotest/public/AfeClient.html
lient/src/autotest/public/TkoClient.html
lient/src/autotest/public/tkoclient.css
lient/src/autotest/tko/CommonPanel.java
lient/src/autotest/tko/CompositeTestSet.java
lient/src/autotest/tko/ConditionTabView.java
lient/src/autotest/tko/ConditionTestSet.java
lient/src/autotest/tko/ExistingGraphsFrontend.java
lient/src/autotest/tko/FragmentedTable.java
lient/src/autotest/tko/GraphingView.java
lient/src/autotest/tko/HeaderSelect.java
lient/src/autotest/tko/SavedQueriesControl.java
lient/src/autotest/tko/Spreadsheet.java
lient/src/autotest/tko/SpreadsheetDataProcessor.java
lient/src/autotest/tko/SpreadsheetSelectionManager.java
lient/src/autotest/tko/SpreadsheetView.java
lient/src/autotest/tko/StatusSummary.java
lient/src/autotest/tko/TableRenderer.java
lient/src/autotest/tko/TableView.java
lient/src/autotest/tko/TestContextMenu.java
lient/src/autotest/tko/TestDetailView.java
lient/src/autotest/tko/TestGroupDataSource.java
lient/src/autotest/tko/TestLabelManager.java
lient/src/autotest/tko/TestSelectionListener.java
lient/src/autotest/tko/TestSet.java
lient/src/autotest/tko/TestViewDataSource.java
lient/src/autotest/tko/TkoClient.java
lient/src/autotest/tko/TkoUtils.java
95128e53c4e807e98c2cd3588198a6a686e45e52 04-Aug-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Forgot a status in my mappings.

Signed-off-by: Travis Miller <raphtee@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1958 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpc_interface.py
e0b6362a94cd9ca1146fbdd93d04fa144d42122d 04-Aug-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Add a basic "echo" function to the afe rpc interface, to allow for a
lightweight connection test.

Risk: Low
Visibility: Adds an echo() function to the frontend rpc interface.

Signed-off-by: John Admanski <jadmanski@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1957 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/001_rpc_test.txt
fe/rpc_interface.py
8493a9fd99aae21ccfdd4cb5cb84bde43ca0c153 04-Aug-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Fixed the doctest to be consistent with our 4 spaces no tabs change, and fixed test.py to ignore files that end in ~.

Signed-off-by: Travis Miller <raphtee@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1956 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/001_rpc_test.txt
fe/doctests/003_misc_rpc_features.txt
fe/test.py
c5ddfd1f71caef9ec0c84c53ef7db42fcdc33e1c 04-Aug-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> I have attached the patch for the packaging system.
The documentation is available at : http://test.kernel.org/autotest/PackagingSystem

Signed-off-by: Ashwin Ganti <aganti@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1953 592f7852-d20e-0410-864c-8624ca9c26a4
fe/control_file.py
fe/rpc_utils.py
f0d4c570c1cf47af108bfc1236112a344ecc9c98 01-Aug-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Don't call preventDefault() on normal click events. This was breaking links in tables.


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1944 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/common/ui/RightClickTable.java
091d19411105375c39acae752b698e9feb658e2d 01-Aug-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> fix minor bug where detail views would refresh twice in a row


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1943 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/AfeClient.java
lient/src/autotest/afe/JobDetailView.java
lient/src/autotest/common/ui/DetailView.java
lient/src/autotest/common/ui/TabView.java
6a4e444d7b53d4171c0fbf50d442256e56959c55 01-Aug-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> -add support for "read only" items to DoubleListSelector, to support new TKO features


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1942 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/common/ui/DoubleListSelector.java
76ce410967787f0a834b1cb85fc4aec9261ac8bc 31-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> refactor RightClickTable to support some work on new TKO



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1940 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/common/ui/RightClickTable.java
5e2c70db8586bcd4c47fe6e5173357539c84cc57 31-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> couple of changes needed for new TKO:
-make NotifyManager support direct logging
-make DataSource support an onError method



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1938 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/HostDetailView.java
lient/src/autotest/common/table/DataSource.java
lient/src/autotest/common/table/DynamicTable.java
lient/src/autotest/common/table/RpcDataSource.java
lient/src/autotest/common/table/SelectionManager.java
lient/src/autotest/common/ui/NotifyManager.java
d9992fe7281bdcd9c99f85b1fc89c65131b49c45 31-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Cloning a job now also clones the one-time hosts correctly.

Risk: low
Visibility: medium (UI bug fix)

Signed-off-by: James Ren <jamesren@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1937 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpc_interface.py
lient/src/autotest/afe/CreateJobView.java
2f3facb29e220f88cf7b4c4d75f8e272967c21e7 31-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> One-time host control in Create Job now supports comma- and whitespace-
separated lists of hosts.

Risk: low
Visibility: medium (UI functionality change)

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1936 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/HostSelector.java
lient/src/autotest/public/AfeClient.html
6fc841f2c9cb4caf2cf7552950c9ea278b65aea2 31-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Fixed copying the "Skip verify" option when cloning a job.

Risk: low
Visibility: medium (UI bug fix)

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1935 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/CreateJobView.java
3d9899a76a88cd4dea4cb819b8b9eb2165ede4e0 31-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Provides a mechanism in the UI to choose to skip the verification stage.

Signed-off-by: Travis Miller <raphtee@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1934 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpc_interface.py
lient/src/autotest/afe/CreateJobView.java
lient/src/autotest/public/AfeClient.html
1ab512bb148ff7706cde7d74eee9de08c47118e0 31-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> One-time hosts now have their protection levels set to "Do not repair".

Risk: low
Visbility: medium (host creation behavior change)

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1932 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
1d2ff01b31f1918575a8ba3df30f7e79b97036d3 31-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Changed Select All on Job List tab to select all jobs across all pages.
Added a Select Visible link on Job List tab to select only visible
jobs.

Risk: low
Visibility: medium (UI change)

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1931 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/JobListView.java
lient/src/autotest/common/ui/TableActionsPanel.java
e65d2af7d45b147e899f9158f39aeeb204e8f5a9 31-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Creating a new job with a one-time host that has been previously used
now works as expected. The job will be successfully created.

Risk: low
Visibility: medium (UI bug fix)

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1930 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
c3afc99304e4d6ada6eaaaa0a1513e408ecca6eb 28-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> -move "select visible" logic into SelectionManager from HostSelector
-make JobListView "select all" link only select visible



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1918 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/HostSelector.java
lient/src/autotest/afe/JobListView.java
lient/src/autotest/common/table/SelectionManager.java
a62866be14da66b1d3f061837617eeea8a192aa7 28-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> -add test_view_2 with more sensible field names, for use by new TKO. this is intended to replace test_view, but I don't want to modify test_view until old TKO is phased out.
-ensure "loading" popup doesn't get closed too early, especially on initial load
-extend logic for RPC serialization for new TKO



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1916 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpc_utils.py
lient/src/autotest/common/ui/NotifyManager.java
8f5a6fbeb7b7ec58424f0cf0bdbcca0e084e3e34 28-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> use replace instead of replaceAll in common.Utils, since we don't actually need regexps

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1915 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/common/Utils.java
e90eb5e8e2e72aa1f4cd359a4d81d6cb323dd89e 25-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Fixed the path in TEMPLATE_DIRS. Now constructs the absolute path to
the templates directory.

Risk: low
Visbility: medium (404 and 500 pages should work now)

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1905 592f7852-d20e-0410-864c-8624ca9c26a4
ettings.py
04f2cd81b1776c12085d9314ec27c3a7f755a843 25-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Automatically add the current user to new ACL groups. Also add the user
to the group if all users are removed from the ACL group.

Risk: low
Visibility: medium (changing behavior)

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1904 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/001_rpc_test.txt
fe/models.py
fe/rpc_interface.py
517f205cb5dfbe5d52a4ab7a3811c93a97af4c46 25-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Remove requeue_job from the RPC interface. Neither the CLI nor the web
frontend uses it anymore. Also updated the frontend unit test.

Risk: low
Visibility: low

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1903 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/001_rpc_test.txt
fe/models.py
fe/rpc_interface.py
103690a0d2ebc312711f44882bfa82cba7d97839 25-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> make CustomHistory more robust (and simpler as it turns out)

Signed-off-by: Steve Howard <showard@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1902 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/common/CustomHistory.java
d7f7ddd04afef56d22ebefc9a13acace4b4f99d6 25-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Added a reset button to Create Job. Also reverted the behavior of
refresh on Create Job to original behavior (does not reset form
fields).

Risk: low
Visbility: medium (added UI element)

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1901 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/CreateJobView.java
lient/src/autotest/public/AfeClient.html
d823b3648911ef831e75c9d645bba5cfbfed4dc4 24-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Added handling in the frontend for the 'Starting' job status.

Risk: low
Visibility: medium (UI bug fix)

Signed-off-by: James Ren <jamesren@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1896 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
fe/rpc_interface.py
37c7fe6ad221d6d60807ce75220aee34e14020fd 24-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Added templates for 404 and 500 errors. 404 displays a generic "page
not found" error. 500 displays the exception type and value, along with
a traceback that can be revealed.

Risk: low
Visibility: medium (changing the look and feel of errors)

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1895 592f7852-d20e-0410-864c-8624ca9c26a4
fe/views.py
ettings.py
emplates/404.html
emplates/500.html
rls.py
e48cef5ec134a54dec94e5a201227680cb30b6b9 24-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Fixed the "Clone job" button's behavior with metahosts. Now clones only
the metahost, not the individual hosts assigned for it.

Risk: low
Visibility: medium (UI bug fix)

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1893 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpc_interface.py
237ba463f5d03aa26ae88df2c6c265fb1d8b447a 24-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> display timeout in the AFE job details tab

Signed-off-by: Steve Howard <showard@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1892 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/JobDetailView.java
lient/src/autotest/common/ui/DetailView.java
lient/src/autotest/public/AfeClient.html
353923be210e4c452ecd4e23ccd4e2d1fbebb490 24-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> more changes to history handling needed for new TKO. adds a utility for setting default values in maps, and avoids passing history arguments to TabViews if there is no actual information contained.

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1891 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/common/Utils.java
lient/src/autotest/common/ui/CustomTabPanel.java
a4c0c00c258e59e5d49552f82d4f25d9e535ed26 24-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Create Job view now resets when refresh is clicked, or user navigates to
it from a different tab.

Risk: low
Visibility: medium (UI behavior change)

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1890 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/AfeClient.java
lient/src/autotest/afe/CreateJobView.java
6eefdea631039525bd76ddda605ccd3a4cb97c5f 24-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Fixed bug in Create Job that did not allow user to deselect a one-time
host after it had been added to the host list.

Risk: low
Visbility: medium (bug fix)

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1889 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/HostSelector.java
585c2abb80648f80e7cb649b06dc9f6a8690a790 23-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> -fix query_count to support flexible filtering like everything else
-ensure DataTable always escapes HTML in contents, and use whitespace: pre so that newlines can still be included (i.e. in JobTable)




git-svn-id: http://test.kernel.org/svn/autotest/trunk@1882 592f7852-d20e-0410-864c-8624ca9c26a4
fe/model_logic.py
lient/src/autotest/afe/JobTable.java
lient/src/autotest/common/table/DataTable.java
lient/src/autotest/public/afeclient.css
31c570e61cdbe7edb5318525871162819d52fd96 23-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Fix adding new ACL groups. Adding a check that the original_object
attribute exists in AclGroup.Manipulator before accessing it.

Risk: low
Visibility: medium (UI bug fix)

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1881 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
b2f591d75a6739f5d4efdbba0806e88507864b8e 21-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> minor improvement to common.Utils needed for new TKO



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1876 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/common/Utils.java
7ac7b7aab4f7e8bf5685ba66197a6d007a267933 21-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> add some more flexibility to model_logic.py, needed for new TKO




git-svn-id: http://test.kernel.org/svn/autotest/trunk@1875 592f7852-d20e-0410-864c-8624ca9c26a4
fe/model_logic.py
ef37eccb66c415fdb6a8fe63244fe8545463719f 21-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> support real right-click events. GWT doesn't support this but will in the next 1.5 RC, but until then, we'll need this hackery to make it work. this is also definitely not cross-browser compatible, but it could be made so if necessary.




git-svn-id: http://test.kernel.org/svn/autotest/trunk@1874 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/common/table/DataTable.java
lient/src/autotest/common/ui/RightClickTable.java
5524adf7fc9d923ac3b6f7e7fa72158a5625460b 21-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> -unified code for table actions (select all/none and table-wide actions menu) between AFE and new TKO
-extracted non-TKO-specific code for ContextMenu into common.ui.ContextMenu
-moved TableActionsPanel to common.ui
-made JobListView use TableActionsPanel instead of its old private version of it
-made AfeClient module use standard.css stylesheet, since TableActionsPanel depends on it
-added checkbox to host tables to enable/disable filtering on ACL-accessible hosts, and got rid of now-unnecessary RpcHostTable class



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1873 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/AfeClient.gwt.xml
lient/src/autotest/afe/HostListView.java
lient/src/autotest/afe/HostSelector.java
lient/src/autotest/afe/HostTableDecorator.java
lient/src/autotest/afe/JobListView.java
lient/src/autotest/afe/RpcHostTable.java
lient/src/autotest/common/table/FieldFilter.java
lient/src/autotest/common/ui/ContextMenu.java
lient/src/autotest/common/ui/TableActionsPanel.java
lient/src/autotest/public/AfeClient.html
lient/src/autotest/public/standard.css
fb2a7fa621ddc91634dde6c56a47a1c8df2610ef 17-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Adding new columns "locked_by_id" and "lock_time" to the hosts table, to
indicate who locked a host and when.

Risk: low
Visibility: low

Signed-off-by: James Ren <jamesren@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1864 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/001_rpc_test.txt
fe/models.py
igrations/015_add_locked_by_and_lock_time.py
8ac29b478ab364b28573791e5f22c73830dda95a 17-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Add code to perform a translation on the status field to provide a more informative string for the UI.

Signed-off-by: Travis Miller <raphtee@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1863 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpc_interface.py
lient/src/autotest/afe/JobStatusDataSource.java
a79583c946ce95ba0b30f5ec2de58ff13f1006d0 17-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Fixed a bug that arrises when running the AfeClient for the first time on a fresh setup.

Signed-off-by: Travis Miller <raphtee@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1862 592f7852-d20e-0410-864c-8624ca9c26a4
pache_auth.py
9865997a354f669584c1cc3ef210efbdefb9a508 17-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> -fixed url redirection for /tko URLs in debug server, to ease development pain
-changed links in job details table a bit (added status log link and changed text)
-change rpc_utils.py to handle datetime.date objects
-various minor changes needed for new TKO



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1861 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpc_utils.py
fe/urls.py
lient/src/autotest/afe/JobDetailView.java
lient/src/autotest/common/Utils.java
lient/src/autotest/common/table/DataSource.java
lient/src/autotest/common/table/RpcDataSource.java
lient/src/autotest/common/ui/SimpleHyperlink.java
77f95db6250c71693f4775bb7b6f8876cb536000 17-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> -move log file URL logic to common.Utils
-make retrieve_logs.cgi use Location header instead of Refresh; this works with XmlHttpRequests
-small fix to JsonRpcProxy to show better error data in some cases



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1860 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/JobDetailView.java
lient/src/autotest/common/JsonRpcProxy.java
lient/src/autotest/common/Utils.java
b202ea9a61c3d6195e65b2f9e47ed51828209057 17-Jul-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Add removal migration script to ease migration from a incorrectly added field

Signed-off-by: Scott Zawalski <scottz@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1857 592f7852-d20e-0410-864c-8624ca9c26a4
igrations/016_remove_run_verify.py
35dbd8414c0e7022a6a4b54f7ef16b5ff51ae53b 16-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> introduced distinction between clickable and non-clickable widgets in DataTable; this allows read-only widgets that will still generate a row click event when clicked. this was necessary for new TKO development.


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1839 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/JobDetailView.java
lient/src/autotest/afe/JobTable.java
lient/src/autotest/common/table/DataTable.java
lient/src/autotest/common/table/DynamicTable.java
40376005c91d3a18e5ff7a12987534773476b2d1 16-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Raised coverage of serviceHandler.py from 0% to 81%!

Signed-off-by: Travis Miller <raphtee@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1838 592f7852-d20e-0410-864c-8624ca9c26a4
fe/json_rpc/common.py
fe/json_rpc/serviceHandler_unittest.py
909c7a661e6739c110690e70e2f4421ffc4e5433 15-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Initial release of test auto importer
Update models.py to reflect database changes
Add the following columns to autotests table:
* author
* dependencies
* experimental
* run_verify
* test_time
* test_category
* sync_count
Add run_verify to jobs table
Update scheduler to assert with run_verify

Risk: Medium
Visibility: High, people addings tests will now see more fields via the admin frontend

Signed-off-by: Scott Zawalski <scottz@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1837 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/001_rpc_test.txt
fe/doctests/test.control.2
fe/models.py
fe/rpc_interface.py
igrations/013_new_test_fields.py
5953f9588713ccbb96ac1b45aef0ffc90117a40f 15-Jul-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Add run_verify to host_queue_entries as a Small int

Risk: Low
Visibility: Required for scheduler to function properly due to previous change

Signed-off-by: Scott Zawalski <scottz@google.com>





git-svn-id: http://test.kernel.org/svn/autotest/trunk@1835 592f7852-d20e-0410-864c-8624ca9c26a4
igrations/014_run_verify.py
14213ec4f4773cec2b2cbd75b0d9516633556ae3 14-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Adding links from each host queue entry on the Job Details page to the
host's debug log.

Risk: low
Visbility: medium (adding links to a page)

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1831 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/JobDetailView.java
9f8a27b21824eb4a0f1c977ead46aea269b66e53 14-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> change DataSource (and subclasses) and DynamicTable to support multi-level sorting (sorting on multiple fields)


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1830 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/HostDetailView.java
lient/src/autotest/afe/HostSelector.java
lient/src/autotest/afe/JobTable.java
lient/src/autotest/common/table/ArrayDataSource.java
lient/src/autotest/common/table/DataSource.java
lient/src/autotest/common/table/DynamicTable.java
lient/src/autotest/common/table/RpcDataSource.java
lient/src/autotest/common/table/SelectionManager.java
4205d8965f7fdda6a9c24100be912f36f04a3fc9 14-Jul-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Fix the unittest to make it pass and avoid relying on a fragile parser:
- Use the __import__ and import coherently
- Use the standard unittest framework

Rewrite of the barrier unittest as using fork() completely messed up
the overall test suite and ran it 8 times.

Visibility: Medium/High - some import changes
Risk: Medium

Tests:
- Ran unittest_suite.py for all and individual directories
- Ran coverage_suite.py for all and individual directories
- Ran manage.py test from frontend directory.

Signed-off-by: Jean-Marc Eurin <jmeurin@google.com>




git-svn-id: http://test.kernel.org/svn/autotest/trunk@1827 592f7852-d20e-0410-864c-8624ca9c26a4
fe/test.py
rontend_unittest.py
3fa9db8b343151474035d7d199c318694d687c4f 11-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> improved selection support for AFE tables


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1813 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/common/table/SelectionManager.java
12fbfde376d7748085be73f9f45e46f7d3433590 11-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Fix some issues from findbugs.


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1812 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/JobListView.java
lient/src/autotest/afe/JobTable.java
lient/src/autotest/common/ui/DoubleListSelector.java
6f1593c98487664a6c330a638e6645dc39b4aca3 11-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> some changes to support saved queries for new TKO
-migration to add saved_queries table. this is different from the existing query_history table. this feature is incompatible with the old one and I didn't want to interfere with the old one.
-various modifications to history handling across the board to allow better support for saved queries (the url will show up as just "saved_query=123", without all the extra crap)
-refactoring of apache_auth.py to allow new TKO to use it without using all the Django auth crap


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1810 592f7852-d20e-0410-864c-8624ca9c26a4
pache_auth.py
lient/src/autotest/afe/AfeClient.java
lient/src/autotest/common/CustomHistory.java
lient/src/autotest/common/ui/CustomTabPanel.java
9d821ab7d97c677a63589e6d71ee3c9da46f7077 11-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Made use of the new Widget functionality in DataTable and added checkboxes to
JobTable. JobListView now makes use of the SelectionManager and its checkboxes
to allow mulitple persistent selection of entries. This required some small
change to the behvavior of SelectionManager -- namely that it no longer
subscribes to the table itsef. This required some change to HostSelector which
acts on HostTable (it now does the listening to the table and calls the
SelectionManager methods directly). Added hyperlinks for "Select All" "DeSelect
All" and "Abort Selected" to the JobListView. Added a new rpc call "abort_jobs"
which takes a list of ids of jobs to abort.

Signed-off-by: Travis Miller <raphtee@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1809 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpc_interface.py
lient/src/autotest/afe/HostSelector.java
lient/src/autotest/afe/JobListView.java
lient/src/autotest/afe/JobTable.java
lient/src/autotest/common/table/DataTable.java
lient/src/autotest/common/table/SelectionManager.java
lient/src/autotest/common/table/TableClickWidget.java
lient/src/autotest/public/AfeClient.html
ca67229c5d9d3d6d5a94164338cedb1db6fc49ca 11-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> rewrite history handling code to support generic storage of keyvals in the history token
-added support to Utils for encoding and decoding keyvals as GET args
-modify TabView and CustomTabPanel to use it for history token handling
-modify existing views that use history to use keyvals instead of the old ad-hoc format


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1807 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/JobListView.java
lient/src/autotest/common/Utils.java
lient/src/autotest/common/ui/CustomTabPanel.java
lient/src/autotest/common/ui/DetailView.java
lient/src/autotest/common/ui/TabView.java
1e935f132c1a1d2aad7bad28bfa9d030f5d9393d 11-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> -modify django-directives to support multiple Django servers
-modify JsonRpcProxy to support multiple proxies to different URLs
-add AFE RPC get_hqe_percentage_complete to get percentage of HQE's matching some filter that are complete
-ensure user-supplied WHERE conditions have %'s escaped


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1806 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/001_rpc_test.txt
fe/model_logic.py
fe/rpc_interface.py
lient/src/autotest/afe/AfeClient.java
lient/src/autotest/common/JsonRpcProxy.java
a334a8b203268d4a53dd39130add56874eabd012 11-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> some changes to CustomTabPanel to support a common area between tabs, and a new DoubleListSelector widget. both are used by the new TKO interface.


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1805 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/common/ui/CustomTabPanel.java
lient/src/autotest/common/ui/DoubleListSelector.java
be751d919fcbcb8ea0aad5e13e6e9485da1c707d 10-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Fixed the behavior of the Create Job page. After creating a job, the
tests checkboxes are now properly reset.

Risk: low
Visibility: medium (changing the behavior of a page)

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1800 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/CreateJobView.java
fd875cce6b75d140a62e3b3e257849ca25565ef9 10-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Added a link from the Host Detail page to the Verify/Repair logs for
that host.

Risk: low
Visibility: medium (adding a new link)

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1799 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/HostDetailView.java
lient/src/autotest/public/AfeClient.html
da8530638fec1c09e70627d71e1d3c5d5fe76b51 10-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Fixed the RPC interface unit test to work with the updated ACL group
handling.

Risk: low
Visibility: low

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1798 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/001_rpc_test.txt
3dd47c247147ca6920b222d43a8eb6ee54209c8f 10-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Added better ACL group support. More checks are now performed to ensure
that users cannot create jobs against hosts that are not visible, and
that users cannot abort other users' jobs.

Note: If applying this patch, you must also apply the "move logged-in user
tracking to separate thread_local module" patch.

Risk: medium (disallowing actions may have unintended side effects)
Visibility: medium (new errors will occur if disallowed actions are
attempted)

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1797 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
pache_auth.py
ake_superuser.py
igrations/012_reset_access_levels.py
affe09b5a82a1ec1a6c8f19571e07bb9f901abe0 10-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Narrowing the access modifiers on mutable static fields, to protect
against accidental or intentional modification of the fields' contents.

Note: Apply the "Some additions to autotest.common.Utils..." patch before
applying this one.

Risk: low
Visibility: low

Signed-off-by: James Ren <jamesren@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1796 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/HostDetailView.java
lient/src/autotest/afe/HostTable.java
lient/src/autotest/afe/JobDetailView.java
lient/src/autotest/afe/JobListView.java
lient/src/autotest/afe/JobTable.java
lient/src/autotest/common/Utils.java
lient/src/autotest/common/table/BooleanFilter.java
94b698cacab819b42104ce0db68aa63b68f4d1d1 08-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Added the ability to insert widgets into a column of DataTable.


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1778 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/common/table/DataTable.java
lient/src/autotest/common/table/DynamicTable.java
lient/src/autotest/common/table/TableClickWidget.java
a7c7a582e9774aa24a9f608fbb1dfcfb9e59bb0d 08-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Some additions to autotest.common.Utils made during new TKO development


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1777 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/common/Utils.java
lient/src/autotest/common/table/SelectionManager.java
ff9013885970838cb7b2e868ff7f8b762757b989 08-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> -move logged-in user tracking to separate thread_local module. models.py will soon depend on it, so it had to be moved to avoid circular dependencies.
-made apache_auth set the logged-in user, rather than the rpc handling view. this way the user will be set when the admin interface is accessed as well.



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1776 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/001_rpc_test.txt
fe/rpc_interface.py
fe/rpc_utils.py
fe/views.py
pache_auth.py
hread_local.py
09096d82f1f71cddf2165a9e169e4249cc0fd82a 08-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> -added readonly_connection.py, which opens a second DB connection using readonly credentials found in the global config
-added special QuerySet subclasses to model_logic that wrap DB access in a readonly connection
-made query_objects wrap the DB access in a readonly connection when given extra_args or extra_where (which can contain arbitrary SQL from the user)
-got rid of unnecessary call to query_objects


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1775 592f7852-d20e-0410-864c-8624ca9c26a4
fe/model_logic.py
fe/readonly_connection.py
fe/rpc_interface.py
ettings.py
7f69e4383fa7be06c28fe54d1a0c834fa1de313a 08-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> small change to StaticDataRepository needed for new TKO development


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1774 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/common/StaticDataRepository.java
df062569a6407ec084c4ee05b9390f8a0183d37b 03-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Adding protection levels to hosts. Allows the user to specify how much
the repair operation is allowed to do on the host (e.g., do not repair,
repair filesystem only, allow reimaging).

Risk: low
Visbility: medium (adding a new input field)


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1771 592f7852-d20e-0410-864c-8624ca9c26a4
fe/enum.py
fe/models.py
fe/rpc_interface.py
igrations/010_add_protection_to_hosts.py
58ea6ae3d686ead56eb0459eadafb6f32a39487a 03-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Small performance tweak. Replaced a keySet iterator followed by a
Map.get with an entrySet iterator.

Risk: low
Visibility: low


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1770 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/JobStatusDataSource.java
5df2b19e0c379d0915fae11401935725ad47d423 03-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Updating the RPC interface and scheduler unit tests to match up with
the recent changes to their respective subjects.

Risk: low
Visibility: low


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1769 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/001_rpc_test.txt
b8471e33ed22512001ec4cec8c33fdf01f32eb62 03-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Added a new input that allows used to specify a one-time host when
creating a job. The job will be run against that host once, and the
host will not appear in the "Available hosts" selector.

Risk: medium (deleting records from database)
Visibility: medium (adding an input field)


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1768 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
fe/rpc_interface.py
lient/src/autotest/afe/CreateJobView.java
lient/src/autotest/afe/HostSelector.java
lient/src/autotest/public/AfeClient.html
igrations/011_support_one_time_hosts.py
1c97b1bccd76a7ea8be5e25ca266da0143ef986d 03-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> migration file for job timeouts patch


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1767 592f7852-d20e-0410-864c-8624ca9c26a4
igrations/009_add_timeout_to_jobs.py
a8709c5d47fa6a2dfed0604c187afe86a72587a0 03-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Changing the Requeue Job button to redirect and populate the Create Job
interface so that the user can modify the cloned job.


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1766 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpc_interface.py
lient/src/autotest/afe/AfeClient.java
lient/src/autotest/afe/CreateJobView.java
lient/src/autotest/afe/HostSelector.java
lient/src/autotest/afe/JobDetailView.java
lient/src/autotest/public/AfeClient.html
3bb499f74c04acec1f802a531cdfcba8f5ac0164 03-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Adding a timeout field to the "Create Job" tab, modified the create_job
RPC to handle a "timeout" argument, and added a "timeout" column to the
AUTOTEST_WEB database. Sets how long a job should run until it is
automatically aborted.


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1765 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
fe/rpc_interface.py
lient/src/autotest/afe/CreateJobView.java
lient/src/autotest/public/AfeClient.html
6bc47015cce0ebc2fc255d3950bfeaf4851f36fd 03-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Refactoring to eliminate warnings and (slightly) improve performance.
Added @Override annotations, and added parameterizations for generic
collections. Made small performance tweaks.


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1761 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/AfeUtils.java
lient/src/autotest/afe/CreateJobView.java
lient/src/autotest/afe/HostDataSource.java
lient/src/autotest/afe/HostDetailView.java
lient/src/autotest/afe/HostListView.java
lient/src/autotest/afe/HostSelector.java
lient/src/autotest/afe/JobDetailView.java
lient/src/autotest/afe/JobListView.java
lient/src/autotest/afe/JobStatusDataSource.java
lient/src/autotest/afe/JobTable.java
lient/src/autotest/afe/LabelFilter.java
lient/src/autotest/common/CustomHistory.java
lient/src/autotest/common/JSONArrayList.java
lient/src/autotest/common/JsonRpcProxy.java
lient/src/autotest/common/StaticDataRepository.java
lient/src/autotest/common/UnmodifiableSublistView.java
lient/src/autotest/common/Utils.java
lient/src/autotest/common/table/ArrayDataSource.java
lient/src/autotest/common/table/BooleanFilter.java
lient/src/autotest/common/table/DataTable.java
lient/src/autotest/common/table/DynamicTable.java
lient/src/autotest/common/table/FieldFilter.java
lient/src/autotest/common/table/Filter.java
lient/src/autotest/common/table/LinkSetFilter.java
lient/src/autotest/common/table/ListFilter.java
lient/src/autotest/common/table/MultipleListFilter.java
lient/src/autotest/common/table/RpcDataSource.java
lient/src/autotest/common/table/SearchFilter.java
lient/src/autotest/common/table/SelectionManager.java
lient/src/autotest/common/table/SimpleFilter.java
lient/src/autotest/common/ui/DetailView.java
lient/src/autotest/common/ui/NotifyManager.java
lient/src/autotest/common/ui/Paginator.java
lient/src/autotest/common/ui/SimpleHyperlink.java
lient/src/autotest/common/ui/TabView.java
5ef36e9a24e3688893cf2a64824c29e55c391543 02-Jul-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> methods added to model_logic.py for the new TKO interface that i forgot to check in previously


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1759 592f7852-d20e-0410-864c-8624ca9c26a4
fe/model_logic.py
ba872901576ffb821ce28c00b5a8a8cc98ce44bd 28-Jun-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Added a check for the number of hosts available under a label, if a
test is filed against a metahost. Raises a ValidationError if the user
has requested more hosts than there are in a label.


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1749 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpc_interface.py
c9d462b1b48a8cd109308453dd5eb610dd7108cb 28-Jun-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Separating the Labels input field in the Create Job interface;
platforms and other labels are now separate. User can select zero or
one platform, and an unlimited number of other labels.


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1748 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/afe/AfeUtils.java
lient/src/autotest/afe/HostTableDecorator.java
lient/src/autotest/afe/LabelFilter.java
lient/src/autotest/common/table/TableDecorator.java
9b53349896f215b44ee0d39e9002da0a4c260554 28-Jun-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> some slight fixes to page refresh logic to make sure history works correctly (ran into issues during new TKO development)



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1747 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/common/table/LinkSetFilter.java
lient/src/autotest/common/ui/CustomTabPanel.java
lient/src/autotest/common/ui/TabView.java
08f981bddc5f1a199d789d177cbf583dee9f6c17 24-Jun-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> -every time an ACL group is edited, automatically
-add hosts to everyone if they aren't a member of any group
-remove hosts from everyone if they're a member of any other group
-simplify job block logic in models.py back to the way it used to be.
-get rid of some dead code in model_logic. we can resurrect it later if we need it later.



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1738 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/001_rpc_test.txt
fe/model_logic.py
fe/models.py
fe/rpc_interface.py
7beb2a245365806869bceaac835f567f9b24ee34 24-Jun-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> doctest included for frontend profilers change that didn't get added.



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1737 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/003_misc_rpc_features.txt
351d555bc3fc88439324ff75bc85ef217a961837 17-Jun-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Add a profiler table creation migration script

Signed-off-by: Scott Zawalski <scottz@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1714 592f7852-d20e-0410-864c-8624ca9c26a4
igrations/008_add_profiler_table.py
f5fdfab3cfb3078b2c068f05c79cb7d60fcde1c8 17-Jun-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> This changes the automatically generated control files to use spaces
instead of tabs.

Signed-off-by: Jeremy Orlow <jorlow@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1709 592f7852-d20e-0410-864c-8624ca9c26a4
fe/control_file.py
ec5546d44e555d597258aed66c630bd96ac4e997 16-Jun-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Fix a few erroneous references to ValidationError found by pylint.

Signed-off-by: Steve Howard <showard@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1707 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpc_interface.py
fe/rpc_utils.py
2b9a88bcd336233fa483490892338e29f0a5fa67 13-Jun-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> -allow users to select profilers in the create job view
-create profilers table + model
-create RPCs for managing profilers
-modify control file generation code to accept profilers
-modify create job view
-some refactoring to create job view to allow code sharing and do things more cleanly with some sweet generics
-get rid of rpc_utils.sorted. who knew there was a builtin?


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1703 592f7852-d20e-0410-864c-8624ca9c26a4
fe/control_file.py
fe/doctests/001_rpc_test.txt
fe/management.py
fe/models.py
fe/rpc_interface.py
fe/rpc_utils.py
lient/src/autotest/afe/CreateJobView.java
lient/src/autotest/public/AfeClient.html
2de8f901d7ec910faa3b02e4c6f512563a5d72fe 13-Jun-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> -change frontend_unittest to execute the unittests directly (as opposed to executing them through manage.py in a separate process). this should get us coverage data. the unit tests could still be better integrated into the whole suite, but this is an improvement.
-some fixes to doctests related to tab-space conversion


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1702 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/001_rpc_test.txt
fe/test.py
rontend_unittest.py
0afbb6369aa5aa9a75ea67dd9e95ec4b21c0c181 06-Jun-2008 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Convert all python code to use four-space indents instead of eight-space tabs.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1658 592f7852-d20e-0410-864c-8624ca9c26a4
fe/control_file.py
fe/enum.py
fe/feeds/feed.py
fe/json_rpc/proxy.py
fe/json_rpc/serviceHandler.py
fe/management.py
fe/model_logic.py
fe/models.py
fe/rpc_client_lib.py
fe/rpc_handler.py
fe/rpc_interface.py
fe/rpc_utils.py
fe/simplejson/__init__.py
fe/simplejson/decoder.py
fe/simplejson/encoder.py
fe/simplejson/scanner.py
fe/test.py
fe/urls.py
fe/views.py
pache_auth.py
rontend_unittest.py
anage.py
igrations/001_initial_db.py
igrations/002_cleanup_fields.py
igrations/003_test_synch_type.py
igrations/004_add_indexes.py
igrations/005_one_more_index.py
igrations/006_host_label_invalid.py
igrations/007_indexes_on_acl_tables.py
ettings.py
b0dfb9f75aadabf3c1461c524f98456bf42ff284 06-Jun-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> another lost part of the frontend refactoring. this is changes to rpc_utils to be more generic and more organized.


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1653 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpc_utils.py
2074cd82bf24a8169665f7ca93bab766e6946948 06-Jun-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Couple of reorg-related changes that didn't make it in.


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1645 592f7852-d20e-0410-864c-8624ca9c26a4
fe/urls.py
b0605d9c7bb7e60ee7441a5f59bd89039f3c811d 06-Jun-2008 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Fix up a few bits of whitespace silliness, unnecessary/missing newlines, etc.



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1642 592f7852-d20e-0410-864c-8624ca9c26a4
fe/model_logic.py
ee94ab1de622e7f491a3ee716b8f1e34bbfe5b8a 06-Jun-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Change new RPCs to be more consistent with rest of RPCs.


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1639 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpc_interface.py
bbabf50501415156d84e526b27ea4a8b2c2206e9 06-Jun-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Added 2 RPC interfaces to add multiple hosts to a label.
This will be needed by the new CLI.


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1625 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/001_rpc_test.txt
fe/rpc_interface.py
acdbe3535529a24555d874a8038993c62fd690c2 06-Jun-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> -moved StaticDataRepository to autotest.common so new TKO can use it
-moved most logic from AfeClient into CustomTabPanel so new TKO can share it
-converted JsonRpcProxy to use new RequestBuilder in place of deprecated HTTPRequest
-converted JSONArrayList to use generics
-few more leftover changes from GWT reorganization

This CL introduces dependency on GWT 1.5 so you'll need to upgrade before using this. It's a painless upgrade.


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1623 592f7852-d20e-0410-864c-8624ca9c26a4
fe/model_logic.py
fe/urls.py
lient/src/autotest/afe/AfeClient.java
lient/src/autotest/afe/AfeUtils.java
lient/src/autotest/afe/CreateJobView.java
lient/src/autotest/afe/HostTableDecorator.java
lient/src/autotest/afe/JobDetailView.java
lient/src/autotest/afe/JobListView.java
lient/src/autotest/afe/RpcHostTable.java
lient/src/autotest/afe/StaticDataRepository.java
lient/src/autotest/common/JSONArrayList.java
lient/src/autotest/common/JsonRpcProxy.java
lient/src/autotest/common/StaticDataRepository.java
lient/src/autotest/common/ui/CustomTabPanel.java
lient/src/autotest/public/AfeClient.html
e3f6868dac3b4c4714637d12b93d97823011a35c 05-Jun-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> GWT reorg part 3


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1614 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/autotest/AfeClient.gwt.xml
lient/src/autotest/afe/AfeClient.java
lient/src/autotest/afe/ClassFactory.java
lient/src/autotest/afe/CreateJobView.java
lient/src/autotest/afe/HostDataSource.java
lient/src/autotest/afe/HostDetailView.java
lient/src/autotest/afe/HostListView.java
lient/src/autotest/afe/HostSelector.java
lient/src/autotest/afe/HostTable.java
lient/src/autotest/afe/HostTableDecorator.java
lient/src/autotest/afe/JobDetailView.java
lient/src/autotest/afe/JobListView.java
lient/src/autotest/afe/JobStatusDataSource.java
lient/src/autotest/afe/JobTable.java
lient/src/autotest/afe/LabelFilter.java
lient/src/autotest/afe/RpcHostTable.java
lient/src/autotest/afe/SiteClassFactory.java
lient/src/autotest/afe/StaticDataRepository.java
lient/src/autotest/common/CustomHistory.java
lient/src/autotest/common/JSONArrayList.java
lient/src/autotest/common/JsonRpcCallback.java
lient/src/autotest/common/JsonRpcProxy.java
lient/src/autotest/common/SimpleCallback.java
lient/src/autotest/common/UnmodifiableSublistView.java
lient/src/autotest/common/Utils.java
lient/src/autotest/common/table/ArrayDataSource.java
lient/src/autotest/common/table/BooleanFilter.java
lient/src/autotest/common/table/DataSource.java
lient/src/autotest/common/table/DataTable.java
lient/src/autotest/common/table/DynamicTable.java
lient/src/autotest/common/table/FieldFilter.java
lient/src/autotest/common/table/Filter.java
lient/src/autotest/common/table/LinkSetFilter.java
lient/src/autotest/common/table/ListFilter.java
lient/src/autotest/common/table/MultipleListFilter.java
lient/src/autotest/common/table/RpcDataSource.java
lient/src/autotest/common/table/SearchFilter.java
lient/src/autotest/common/table/SelectionManager.java
lient/src/autotest/common/table/SimpleFilter.java
lient/src/autotest/common/table/TableDecorator.java
lient/src/autotest/common/ui/CustomTabPanel.java
lient/src/autotest/common/ui/DetailView.java
lient/src/autotest/common/ui/ElementWidget.java
lient/src/autotest/common/ui/NotifyManager.java
lient/src/autotest/common/ui/Paginator.java
lient/src/autotest/common/ui/SimpleHyperlink.java
lient/src/autotest/common/ui/TabView.java
lient/src/autotest/public/AfeClient.html
lient/src/autotest/public/ClientMain.html
a2216e7e327b464ac03932b4add2983e9052bc79 05-Jun-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> GWT reorg part 2


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1613 592f7852-d20e-0410-864c-8624ca9c26a4
lient/AfeClient-compile
lient/AfeClient-shell
lient/AfeClient.launch
lient/src/autotest/afe/AfeClient.java
lient/src/autotest/afe/AfeUtils.java
lient/src/autotest/afe/ClientMain.java
lient/src/autotest/afe/CustomHistory.java
lient/src/autotest/afe/CustomTabPanel.java
lient/src/autotest/afe/DetailView.java
lient/src/autotest/afe/ElementWidget.java
lient/src/autotest/afe/JSONArrayList.java
lient/src/autotest/afe/JsonRpcCallback.java
lient/src/autotest/afe/JsonRpcProxy.java
lient/src/autotest/afe/NotifyManager.java
lient/src/autotest/afe/Paginator.java
lient/src/autotest/afe/SimpleCallback.java
lient/src/autotest/afe/SimpleHyperlink.java
lient/src/autotest/afe/TabView.java
lient/src/autotest/afe/UnmodifiableSublistView.java
lient/src/autotest/afe/Utils.java
lient/src/autotest/common/CustomHistory.java
lient/src/autotest/common/JSONArrayList.java
lient/src/autotest/common/JsonRpcCallback.java
lient/src/autotest/common/JsonRpcProxy.java
lient/src/autotest/common/SimpleCallback.java
lient/src/autotest/common/UnmodifiableSublistView.java
lient/src/autotest/common/Utils.java
lient/src/autotest/common/ui/CustomTabPanel.java
lient/src/autotest/common/ui/DetailView.java
lient/src/autotest/common/ui/ElementWidget.java
lient/src/autotest/common/ui/NotifyManager.java
lient/src/autotest/common/ui/Paginator.java
lient/src/autotest/common/ui/SimpleHyperlink.java
lient/src/autotest/common/ui/TabView.java
ee4baa6495590c14c933f1fafa37d1843bbee996 05-Jun-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> GWT reorg step 1. This is just too difficult to do in one step. This temporarily breaks GWT but we'll just have to accept that. It'll be fixed soon.


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1612 592f7852-d20e-0410-864c-8624ca9c26a4
lient/.classpath
lient/.project
lient/AfeClient-compile
lient/AfeClient-shell
lient/AfeClient.launch
lient/ClientMain-compile
lient/ClientMain-shell
lient/ClientMain.launch
lient/src/afeclient/ClientMain.gwt.xml
lient/src/afeclient/client/ClassFactory.java
lient/src/afeclient/client/ClientMain.java
lient/src/afeclient/client/CreateJobView.java
lient/src/afeclient/client/CustomHistory.java
lient/src/afeclient/client/CustomTabPanel.java
lient/src/afeclient/client/DetailView.java
lient/src/afeclient/client/ElementWidget.java
lient/src/afeclient/client/HostDataSource.java
lient/src/afeclient/client/HostDetailView.java
lient/src/afeclient/client/HostListView.java
lient/src/afeclient/client/HostSelector.java
lient/src/afeclient/client/HostTable.java
lient/src/afeclient/client/HostTableDecorator.java
lient/src/afeclient/client/JSONArrayList.java
lient/src/afeclient/client/JobDetailView.java
lient/src/afeclient/client/JobListView.java
lient/src/afeclient/client/JobStatusDataSource.java
lient/src/afeclient/client/JobTable.java
lient/src/afeclient/client/JsonRpcCallback.java
lient/src/afeclient/client/JsonRpcProxy.java
lient/src/afeclient/client/LabelFilter.java
lient/src/afeclient/client/NotifyManager.java
lient/src/afeclient/client/Paginator.java
lient/src/afeclient/client/RpcHostTable.java
lient/src/afeclient/client/SimpleCallback.java
lient/src/afeclient/client/SimpleHyperlink.java
lient/src/afeclient/client/SiteClassFactory.java
lient/src/afeclient/client/StaticDataRepository.java
lient/src/afeclient/client/TabView.java
lient/src/afeclient/client/UnmodifiableSublistView.java
lient/src/afeclient/client/Utils.java
lient/src/afeclient/client/table/ArrayDataSource.java
lient/src/afeclient/client/table/BooleanFilter.java
lient/src/afeclient/client/table/DataSource.java
lient/src/afeclient/client/table/DataTable.java
lient/src/afeclient/client/table/DynamicTable.java
lient/src/afeclient/client/table/FieldFilter.java
lient/src/afeclient/client/table/Filter.java
lient/src/afeclient/client/table/LinkSetFilter.java
lient/src/afeclient/client/table/ListFilter.java
lient/src/afeclient/client/table/MultipleListFilter.java
lient/src/afeclient/client/table/RpcDataSource.java
lient/src/afeclient/client/table/SearchFilter.java
lient/src/afeclient/client/table/SelectionManager.java
lient/src/afeclient/client/table/SimpleFilter.java
lient/src/afeclient/client/table/TableDecorator.java
lient/src/afeclient/public/ClientMain.html
lient/src/afeclient/public/afeclient.css
lient/src/afeclient/public/arrow_down.png
lient/src/afeclient/public/arrow_up.png
lient/src/afeclient/public/header.png
lient/src/autotest/AfeClient.gwt.xml
lient/src/autotest/afe/ClassFactory.java
lient/src/autotest/afe/ClientMain.java
lient/src/autotest/afe/CreateJobView.java
lient/src/autotest/afe/CustomHistory.java
lient/src/autotest/afe/CustomTabPanel.java
lient/src/autotest/afe/DetailView.java
lient/src/autotest/afe/ElementWidget.java
lient/src/autotest/afe/HostDataSource.java
lient/src/autotest/afe/HostDetailView.java
lient/src/autotest/afe/HostListView.java
lient/src/autotest/afe/HostSelector.java
lient/src/autotest/afe/HostTable.java
lient/src/autotest/afe/HostTableDecorator.java
lient/src/autotest/afe/JSONArrayList.java
lient/src/autotest/afe/JobDetailView.java
lient/src/autotest/afe/JobListView.java
lient/src/autotest/afe/JobStatusDataSource.java
lient/src/autotest/afe/JobTable.java
lient/src/autotest/afe/JsonRpcCallback.java
lient/src/autotest/afe/JsonRpcProxy.java
lient/src/autotest/afe/LabelFilter.java
lient/src/autotest/afe/NotifyManager.java
lient/src/autotest/afe/Paginator.java
lient/src/autotest/afe/RpcHostTable.java
lient/src/autotest/afe/SimpleCallback.java
lient/src/autotest/afe/SimpleHyperlink.java
lient/src/autotest/afe/SiteClassFactory.java
lient/src/autotest/afe/StaticDataRepository.java
lient/src/autotest/afe/TabView.java
lient/src/autotest/afe/UnmodifiableSublistView.java
lient/src/autotest/afe/Utils.java
lient/src/autotest/common/table/ArrayDataSource.java
lient/src/autotest/common/table/BooleanFilter.java
lient/src/autotest/common/table/DataSource.java
lient/src/autotest/common/table/DataTable.java
lient/src/autotest/common/table/DynamicTable.java
lient/src/autotest/common/table/FieldFilter.java
lient/src/autotest/common/table/Filter.java
lient/src/autotest/common/table/LinkSetFilter.java
lient/src/autotest/common/table/ListFilter.java
lient/src/autotest/common/table/MultipleListFilter.java
lient/src/autotest/common/table/RpcDataSource.java
lient/src/autotest/common/table/SearchFilter.java
lient/src/autotest/common/table/SelectionManager.java
lient/src/autotest/common/table/SimpleFilter.java
lient/src/autotest/common/table/TableDecorator.java
lient/src/autotest/public/ClientMain.html
lient/src/autotest/public/afeclient.css
lient/src/autotest/public/arrow_down.png
lient/src/autotest/public/arrow_up.png
lient/src/autotest/public/header.png
20f47064be6855277e251cee7611d8336bcc9149 05-Jun-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> -check ACLs directly in the scheduler (bypassing ineligible_host_queues)

-rewrite scheduler queries to avoid all subqueries. they are just bad in mysql.

-rip out all that code related to using ineligible_host_queues to enforce ACLs. good riddance!

-update scheduler unit test to reflect this new policy (no ineligible_host_queue blocks for ACLs)

-minor bugfixes to scheduler unit test. this sucks, but i did go back and ensure the old scheduler passed the fixed up unit test suite as well.

-remove a blanket except: block from the scheduler. it wasn't necessary, it was inconsistent, and it was interfering with unit testing.



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1608 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/003_job_blocks_test.txt
fe/models.py
fe/rpc_interface.py
igrations/007_indexes_on_acl_tables.py
9ca52701a301ca2315e21d6ab876329decf53158 02-Jun-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> The autotest frontend should run every test in its own step so that, if one test reboots the machine, the rest keep running.


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1584 592f7852-d20e-0410-864c-8624ca9c26a4
fe/control_file.py
fe/doctests/001_rpc_test.txt
1c3ba7a68eae0cdc4b9806f96a1757fe004df7f8 02-Jun-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> frontend_unittest.py was not running successfully if run from any other directory other than its own.
This is now fixed.

Signed-off-by: Travis Miller <raphtee@google.com>




git-svn-id: http://test.kernel.org/svn/autotest/trunk@1579 592f7852-d20e-0410-864c-8624ca9c26a4
rontend_unittest.py
7c7852819d0611b4d0e8e69b3011b79e4016a770 29-May-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Restructure code in preparation for sharing with new TKO.

-Change structure of RPC handler. New rpc_handler.py is independent of AFE and will eventually be moved to a common dir, along with the JSON and JSON-RPC stuff. rpc_handler.py now has a class structure that allows multiple apps to instantiate their own RPC handlers with different interfaces.

-Pull out generic model extensions from models.py into model_logic.py. This too will be moved to a common dir.


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1566 592f7852-d20e-0410-864c-8624ca9c26a4
fe/model_logic.py
fe/models.py
fe/rpc_handler.py
fe/urls.py
fe/views.py
58ff09df13d02ffa71b66a0b60a324bc74c5908e 29-May-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Enforce doctest ordering. This is necessary since the different doctest files depend on each other.


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1565 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/001_rpc_test.txt
fe/doctests/002_site_rpc_test.txt
fe/doctests/003_job_blocks_test.txt
fe/doctests/job_blocks_test.txt
fe/doctests/rpc_test.txt
fe/test.py
ec82655bfbad989929ac69098b5b8aefaa57ddbf 29-May-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Use Vlad's new brief mode to embed the TKO table into a scrollable panel instead of using an iframe.


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1562 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/afeclient/client/JobDetailView.java
lient/src/afeclient/public/ClientMain.html
lient/src/afeclient/public/afeclient.css
5875c151c05a62346afd214ec35122018849ec99 28-May-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Add commits for the two methods in which we do manual queries that modify data. Turns out Django disables MySQL autocommit but has its own autocommit mechanism. This works fine until you start doing queries manually, in which case you need to commit manually.


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1560 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
3cbce7cce58bffe9b77ed5d06b9f118311aa5460 22-May-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Add one more SELECT DISTINCT that should've been there.


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1545 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
9a179061f5e5c7c412d5fd4768c1417f47c0dee9 20-May-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Change filter_[not_]in_subquery to use explicit joins instead of IN. Some versions of MySQL consider all subqueries dependent when used with IN, resulting in terrible performance. Explicit joins are properly optimized. This should be at least 50x faster.



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1532 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
fa29a2a27f1534c71587cde382709a4795a820a1 17-May-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Remove all places where we are hardcoding python2.4 on the
scripts

Risk: Medium/High

Summary: After going trough the autotest tree, removing all places where
we have statements like

#!/usr/bin/python2.4

As they're unnecessary for those who have python > 2.4. Still, we want
to make sure we're running on something > 2.4 for those scripts, so I
hooked the scripts to the common lib infrastructure when needed. I tried
to be as careful as possible to make sure all scripts worked after my
modifications, I could even spot an import problem on
server/standalone_profiler.py and fix it.

Also, while I was there, I fixed several inconsistencies on the
mirror/mirror script.

Signed-off-by: Lucas Meneghel Rodrigues <lucasmr@br.ibm.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1527 592f7852-d20e-0410-864c-8624ca9c26a4
rontend_unittest.py
21b4a1b7bffd11ee0472a7cc30a6b535f97f0fcf 15-May-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Need to call on_change() on the ACL object, not the manipulator.


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1520 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
e01a9ab0fe7e310c70f4df1d9159a4411ea4d468 13-May-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Add the ability to view all machines rebooting on the server.
I have been asked a few times about being able to view this and have also
run into some areas where this would have been helpful

Signed-off-by: Scott Zawalski <scottz@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1508 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
2a9378b200ace9a96afb1166105a57da6e862731 09-May-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Add docstrings I forgot to add previously.


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1500 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
b22dce9f01eab0caba5ce0bb43bc03c519ea1035 08-May-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> When a user clicks "edit control file", don't actually allow control file editing until we've finished retrieving the new control file. Otherwise, if getting the control file takes time, the user's edits may be overwritten.



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1497 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/afeclient/client/CreateJobView.java
5deff438c70df0c046b93b50088b361e22bfbcbb 08-May-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> When a non-JSON response is received, instead just raising a useless exception, include the entire response.



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1496 592f7852-d20e-0410-864c-8624ca9c26a4
fe/json_rpc/proxy.py
331a6c75547384e4bd8ff276c8098255493a0443 06-May-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> minor improvement to rpc_test doctest - this way if values don't
match, it shows the expected and actual values



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1493 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/rpc_test.txt
c2992ba854878443b7b7ca0f424edc3c06f1cfd8 06-May-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Speed up ineligible host queue computations, to speed up ACL
operations. This involved:
-extending filter_in_subquery() to be a bit more flexible
-added create_in_bulk() and delete_in_bulk() to create/delete many
rows with one SQL query. I don't think Django has a built-in way to
create in bulk, and it had become a major performance issue. The bulk
delete capabilities of Django were unnecessarily inefficient for the
present purposes, so I added that as well since it was pretty simple,
although it's a less significant performance increase.
-make job block recomputation upon ACL change a bit more focused (only
recompute for possibly affected jobs)

Also added a new doctest for block computation since it is
particularly error-prone, but it's kind of behind-the-scenes so I
didn't want it cluttering the main rpc_test.txt doctest.


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1492 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/job_blocks_test.txt
fe/models.py
fe/rpc_interface.py
38e9d78d2a8465973c2e4e07fcdc2b35d87ad27d 01-May-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Provide support for lazily retrieving job logs from an external storage source.

From: Svitlana Tumanova <stumanova@google.com>
Signed-off-by: Steve Howard <showard@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1478 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/afeclient/ClientMain.gwt.xml
lient/src/afeclient/client/JobDetailView.java
5da17d48573ce769a4bc94a228738fe4826af090 28-Apr-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> JSON service handler should return a traceback for this error path too


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1470 592f7852-d20e-0410-864c-8624ca9c26a4
fe/json_rpc/serviceHandler.py
b8d3424d02cfdc6c3eeda788e3527c5ec50ad8f9 25-Apr-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> -make get_host_queue_entries return full info on related objects (host + job)
-display said info in job detail + host detail pages



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1469 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/rpc_test.txt
fe/models.py
fe/rpc_interface.py
fe/rpc_utils.py
lient/src/afeclient/client/ClientMain.java
lient/src/afeclient/client/HostDataSource.java
lient/src/afeclient/client/HostDetailView.java
lient/src/afeclient/client/HostListView.java
lient/src/afeclient/client/HostTable.java
lient/src/afeclient/client/JobDetailView.java
lient/src/afeclient/client/JobStatusDataSource.java
lient/src/afeclient/client/RpcHostTable.java
lient/src/afeclient/client/Utils.java
lient/src/afeclient/client/table/DataTable.java
lient/src/afeclient/public/ClientMain.html
1c390f85167cfd1c14f9500bb20b05a9e677c0ea 25-Apr-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Add a host detail page showing the job queue for a host.



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1468 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/afeclient/client/ClientMain.java
lient/src/afeclient/client/DetailView.java
lient/src/afeclient/client/HostDetailView.java
lient/src/afeclient/client/JobDetailView.java
lient/src/afeclient/client/JobHostsTable.java
lient/src/afeclient/client/JobStatusDataSource.java
lient/src/afeclient/client/RpcHostTable.java
lient/src/afeclient/client/Utils.java
lient/src/afeclient/client/table/SimpleFilter.java
lient/src/afeclient/public/ClientMain.html
34dc5fa61c425389f9b931d3970c7556baf44dca 24-Apr-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> -instead of that stupid job_status RPC, just provide get_host_queue_entries directly.
-improve reporting of metahost entries in web frontend
-fix bugs in models code which didn't surface until I started querying host queue entries
-fix bug with job detail page refreshing



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1464 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/rpc_test.txt
fe/models.py
fe/rpc_interface.py
lient/src/afeclient/client/JobDetailView.java
lient/src/afeclient/client/JobHostsTable.java
lient/src/afeclient/client/JobStatusDataSource.java
5fdf7ceb5454af9cc57ffc5463be3614723360b3 24-Apr-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> fix host selection UI, which was apparently always broken :(


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1463 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/afeclient/client/HostSelector.java
5244cbb54ca3d426b35561c4853c356f3f51f5fd 24-Apr-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Never delete hosts or labels. Instead, mark them as invalid.

Signed-off-by: Steve Howard <showard@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1462 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/rpc_test.txt
fe/models.py
igrations/006_host_label_invalid.py
b91c3b91c2629d2408b26d99c3cdd2d8afe20b7d 23-Apr-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Add support for completed/failed jobs RSS feeds.
From: Ian Lawrence <ext-ian.lawrence@nokia.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1460 592f7852-d20e-0410-864c-8624ca9c26a4
fe/feeds/__init__.py
fe/feeds/feed.py
fe/templates/feeds/job_feed_description.html
fe/templates/feeds/job_feed_title.html
fe/urls.py
lient/src/afeclient/public/ClientMain.html
eb3be4d9b4c1bb292723ea1d52612aec5967ca0e 21-Apr-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> -include acl-inaccessible hosts in ineligible_host_queues blocks. This should make metahosts obey ACLs.
-recompute blocks for all non-complete jobs after any acl changes
-make the scheduler clear out blocks for completed jobs

This is messier than I would've liked because it required two things Django is not very good at - subqueries and notification of every time a many-to-many relationship changes. I used a custom manager and a custom manipulator to get around the two, respectively, in as clean a way I could find.



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1459 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
fe/rpc_interface.py
6437ff5bb28a0cc97582766826f866a39affae2a 17-Apr-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Use the new parser library directly inside of autoserv, instead of
repeatedly invoking the parser as a separate process. In order to
actually invoke this capability you need to pass in a -P option, with
the job name (tag).

In order to properly support the parsing of server-side synchronous
jobs you need to make use of job.parallel_simple, a new wrapper around
the standard parallel_simple that adds support for parsing the results
of all of the parallel jobs individually. Unfortunately, complex
server-side synch jobs that use parallel instead of parallel_simple
will be unable to take advantage of this.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1446 592f7852-d20e-0410-864c-8624ca9c26a4
fe/control_file.py
48d9020f137f9f161c0a2ecbe4df8fb713baee8b 15-Apr-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> -some fixes to history handling to fix bug where history wasn't updated on job requeue


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1430 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/afeclient/client/ClientMain.java
lient/src/afeclient/client/CustomHistory.java
lient/src/afeclient/client/JobDetailView.java
lient/src/afeclient/client/TabView.java
8e855a27a592142517ed4cc08fa1c59fb72bfeb5 15-Apr-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> -fix bug when headers is not passed to get_proxy
-print tracebacks from RPC proxy
-remove obsolete comment


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1429 592f7852-d20e-0410-864c-8624ca9c26a4
fe/json_rpc/proxy.py
fe/rpc_client_lib.py
3c43dc6bef3b2c551228027a43948208c1403024 15-Apr-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Need to use distinct() since we're sometimes querying over multiple tables. This was causing a bug in the host table.



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1428 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
7f080f62bfe6abf42cb20396f1fce11f0a7b0592 15-Apr-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Move some frontend client kernel code for job creation.


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1427 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/afeclient/client/CreateJobView.java
6f85a1f0e9f3dc342d376a9245075106eeedc011 10-Apr-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> To hide fields from the admin interface, use editable=False instead of the Admin fields specifier. This is the right way to do it and lets us hide a field (i.e. host status) without causing admin errors or weirdness.


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1423 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
f699a2c208726acfbe4e004c419a8dcc06fba04e 10-Apr-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Provide a unittest wrapper around Django's "manage.py test" so the
unittest_suite will include the Django unit tests.


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1422 592f7852-d20e-0410-864c-8624ca9c26a4
fe/test.py
rontend_unittest.py
b5bd2df9c8fd8368601e3508ffcfdc9d15814cdf 10-Apr-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Forgot to add files.


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1420 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/afeclient/client/CustomHistory.java
lient/src/afeclient/client/LabelFilter.java
lient/src/afeclient/client/table/MultipleListFilter.java
5c7e819b5f770656e4440989b153140823b1977b 08-Apr-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> -don't run pylint as part of the frontend unit test suite
-don't look for doctests in files that start with .



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1415 592f7852-d20e-0410-864c-8624ca9c26a4
fe/test.py
8e3aa5e027e28d8fdcb12083e0b30a80767a35cf 08-Apr-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> -support filtering on multiple labels
-fix problem mostly related to history handling that was causing unnecessary RPCs to be executed



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1414 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/rpc_test.txt
fe/models.py
fe/rpc_interface.py
fe/rpc_utils.py
lient/src/afeclient/client/ClientMain.java
lient/src/afeclient/client/HostSelector.java
lient/src/afeclient/client/HostTableDecorator.java
lient/src/afeclient/client/JobListView.java
lient/src/afeclient/client/TabView.java
lient/src/afeclient/client/Utils.java
lient/src/afeclient/client/table/BooleanFilter.java
lient/src/afeclient/client/table/DynamicTable.java
lient/src/afeclient/client/table/FieldFilter.java
lient/src/afeclient/client/table/LinkSetFilter.java
lient/src/afeclient/client/table/ListFilter.java
lient/src/afeclient/client/table/RpcDataSource.java
lient/src/afeclient/client/table/SearchFilter.java
lient/src/afeclient/public/afeclient.css
4f536598ef682ff6105f55533ba925c7d63b5872 08-Apr-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Need to use job filters in get_num_jobs.


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1413 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpc_interface.py
437c3a602ace473db615fae542f29fdd2b28adfb 07-Apr-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Since Django messes with paths and imports, using the "from common
import ..." trick doesn't work. So fix the one usage of this in the
frontend to do it the right way, using absolute imports.

Signed-off-by: Steve Howard <showard@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1402 592f7852-d20e-0410-864c-8624ca9c26a4
ettings.py
1b87bc53ddf0a18e85df7a862df9d559c8bec438 04-Apr-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Modify all the common.py to set up an autotest_lib.* namespace as well
as all the common.* module setup that it normally does.

This should make it easier to replace any funky sys.path code with
code that actually does the right thing, and imports modules using
absolute paths; as long as our entry points import common, code should
be able to count on imports like "import autotest_lib.tko.parse" or
"import autotest_lib.server.autotest" working, without having to
fiddle with sys.path themselves.

The client/bin/common.py is an exception; we can't assume anything
other than the client/ directory is present, so it only adds the
autotest_lib.client.* namespace. So code within the client
itself should only do absolute imports from the client directory
downward, not from the top-level dir downward.

We'll probably need to add __init__.py files to some directories to
allow us to import from them as packages, but that can be done on
an as-needed basis when we write import statements that require them.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1397 592f7852-d20e-0410-864c-8624ca9c26a4
ommon.py
1c8c2215e525de8813c375e796354f8ffb811a08 03-Apr-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Change all tables to be server-side. This required a lot of work, but I think I ended up cleaning up the frontend code a lot and providing room for more powerful features to be added in the future. It is a huge change though.


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1396 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/rpc_test.txt
fe/models.py
fe/rpc_interface.py
fe/rpc_utils.py
lient/src/afeclient/client/ClientMain.java
lient/src/afeclient/client/CreateJobView.java
lient/src/afeclient/client/CustomTabPanel.java
lient/src/afeclient/client/DataTable.java
lient/src/afeclient/client/DynamicTable.java
lient/src/afeclient/client/HostListView.java
lient/src/afeclient/client/HostSelector.java
lient/src/afeclient/client/HostTable.java
lient/src/afeclient/client/HostTableDecorator.java
lient/src/afeclient/client/JSONArrayList.java
lient/src/afeclient/client/JobDetailView.java
lient/src/afeclient/client/JobHostsTable.java
lient/src/afeclient/client/JobListView.java
lient/src/afeclient/client/JobStatusDataSource.java
lient/src/afeclient/client/JobTable.java
lient/src/afeclient/client/JsonRpcCallback.java
lient/src/afeclient/client/Paginator.java
lient/src/afeclient/client/RpcHostTable.java
lient/src/afeclient/client/SimpleCallback.java
lient/src/afeclient/client/TabView.java
lient/src/afeclient/client/UnmodifiableSublistView.java
lient/src/afeclient/client/Utils.java
lient/src/afeclient/client/table/ArrayDataSource.java
lient/src/afeclient/client/table/BooleanFilter.java
lient/src/afeclient/client/table/DataSource.java
lient/src/afeclient/client/table/DataTable.java
lient/src/afeclient/client/table/DynamicTable.java
lient/src/afeclient/client/table/FieldFilter.java
lient/src/afeclient/client/table/Filter.java
lient/src/afeclient/client/table/LinkSetFilter.java
lient/src/afeclient/client/table/ListFilter.java
lient/src/afeclient/client/table/RpcDataSource.java
lient/src/afeclient/client/table/SearchFilter.java
lient/src/afeclient/client/table/SelectionManager.java
lient/src/afeclient/client/table/TableDecorator.java
lient/src/afeclient/public/ClientMain.html
lient/src/afeclient/public/afeclient.css
4a3a08de5dc7d63a53dae25b504d167848dfdd84 03-Apr-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Forgot to add this file.


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1395 592f7852-d20e-0410-864c-8624ca9c26a4
ommon.py
915bbca1e09be679ec1eaa9c12bae184473b753a 01-Apr-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Add indexes to frontend DB.


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1386 592f7852-d20e-0410-864c-8624ca9c26a4
igrations/004_add_indexes.py
igrations/005_one_more_index.py
e8a79d58c48f7bdcab854f7d6ae23cb95892fa10 28-Mar-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Fix import in settings.py.


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1379 592f7852-d20e-0410-864c-8624ca9c26a4
ettings.py
fa7ef631f0f8f6c1e753b3e7bb11bcae7ed56ee9 26-Mar-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> The final piece of global_config is done. settings reads from global_config to fill in the needed info for django.



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1365 592f7852-d20e-0410-864c-8624ca9c26a4
ettings.py
05fb6997f37ad8f62142be3092fc10ddb36433fb 25-Mar-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> -On the server, when exceptions occur, pass back a traceback along with the other info
-On the client, added an expandable "error log" box at the bottom. When non-JSON responses are received, show the full response in the log. When exceptions are received, show the traceback in the log.



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1361 592f7852-d20e-0410-864c-8624ca9c26a4
fe/json_rpc/serviceHandler.py
lient/src/afeclient/client/ClientMain.java
lient/src/afeclient/client/JsonRpcCallback.java
lient/src/afeclient/client/JsonRpcProxy.java
lient/src/afeclient/client/NotifyManager.java
lient/src/afeclient/public/ClientMain.html
lient/src/afeclient/public/afeclient.css
31a244a2cc1c2947ce0f6672a9e71182a3737911 25-Mar-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Be sure to always trim hostnames.



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1360 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
c0ddd726e9dc42b17ff740951d394fc0af09194b 13-Mar-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Display and allow filtering on non-platform labels.


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1346 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/afeclient/client/ClientMain.java
lient/src/afeclient/client/DynamicTable.java
lient/src/afeclient/client/HostTable.java
lient/src/afeclient/client/Utils.java
1385b161fa9391242a2121027878e4943ce3c81f 13-Mar-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> On the frontend, only show hosts to which this user has ACL access.


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1343 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/rpc_test.txt
fe/models.py
fe/rpc_interface.py
fe/rpc_utils.py
lient/src/afeclient/client/HostTable.java
1d445e9d9bc140b685b10a9fdd53cb5e92c0883e 12-Mar-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Support kernel install for server-side control files


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1338 592f7852-d20e-0410-864c-8624ca9c26a4
fe/control_file.py
fe/rpc_interface.py
fe/rpc_utils.py
lient/src/afeclient/client/CreateJobView.java
aa383b7ef92e20aba5db6c336be2b9ffbe23d158 12-Mar-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Add downgrade support to migrate. This is really necessary for doing real migration development. Modified all existing migrations to support it.

Signed-off-by: Steve Howard <showard@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1336 592f7852-d20e-0410-864c-8624ca9c26a4
igrations/001_initial_db.py
igrations/002_cleanup_fields.py
1c4effd90f3ffa7053def1752515b8bc44b9265d 11-Mar-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Forgot to add this frontend migration file to the test synch type checkin.


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1323 592f7852-d20e-0410-864c-8624ca9c26a4
igrations/003_test_synch_type.py
8fd5824df1310058af6b8b503b800d575ddd3787 10-Mar-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> -associate a synch type with each test in the DB (defaults to asynchronous for all)
-make frontend default to synch type for selected tests, and disable selecting tests of different synch types
-allow synchronous client-side tests


git-svn-id: http://test.kernel.org/svn/autotest/trunk@1322 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/rpc_test.txt
fe/models.py
fe/rpc_interface.py
fe/rpc_utils.py
lient/src/afeclient/client/CreateJobView.java
45c64e425d3a785b5e15928e86c1eb57d4c2a2de 07-Mar-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> HostSelector is the widget used in (and only in) CreateJobView to show the HostTable.
We set the default filter for "Locked" status to "No" as we want to run tests on available machines.



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1313 592f7852-d20e-0410-864c-8624ca9c26a4
lient/src/afeclient/client/HostSelector.java
cca3b63aaa31c87f90323083cdc9aead8367e25b 06-Mar-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Remove comment on removed arg.



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1309 592f7852-d20e-0410-864c-8624ca9c26a4
fe/rpc_interface.py
99229e489bcda356c67582890124172307541fe9 06-Mar-2008 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Get rid of job.status and job.submitted_on usage in models.py. This field was deprecated but these (harmless) statements were left lying around.



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1308 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
3cab4a71760ef833f9549778f09197d1ffbe724c 06-Mar-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Requeue support for the frontend. Requeue creates and enqueues a copy of the job. Added support to Job model, new RPC method, and new button to job detail page.

Also fixing a little bug involving the is_dead() method.

Signed-off-by: Steve Howard <showard@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1305 592f7852-d20e-0410-864c-8624ca9c26a4
fe/doctests/rpc_test.txt
fe/models.py
fe/rpc_interface.py
lient/src/afeclient/client/JobDetailView.java
lient/src/afeclient/public/ClientMain.html
d5c95802c1045ca9fa84da18758bd9d44a7e04bf 05-Mar-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Implemented abort functionality in scheduler.

From: Svitlana Tumanova <stumanova@google.com>
Signed-off-by: Steve Howard <showard@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1293 592f7852-d20e-0410-864c-8624ca9c26a4
fe/models.py
lient/src/afeclient/client/JobDetailView.java
e8819cdf80ca0e0602d22551a50f970aa68e108d 15-Feb-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Add frontend and scheduler for Autotest

The attached tarball includes the new Autotest web frontend for creating
and monitoring jobs and the new scheduler for executing jobs. We've
been working hard to get these complete and stabilized, and although
they still have a long, long way to go in terms of features, we believe
they are now stable and powerful enough to be useful.

The frontend consists of two parts, the server and the client. The
server is implemented in Python using the Django web framework, and is
contained under frontend/ and frontend/afe. The client is implemented
using Google Web Toolkit and is contained under frontend/client. We
tried to use Dojo initially, as was generally agreed on, but it proved
to be too young and poorly documented of a framework, and developing in
it was just too inefficient.

The scheduler is contained entirely in the scheduler/monitor_db Python
file. It looks at the database used by the web frontend and spawns
autoserv processes to run jobs, monitoring them and recording status.
The scheduler was developed by Paul Turner, who will be sending out some
detailed documentation of it soon.

I've also included the DB migrations code for which I recently submitted
a patch to the mailing list. I've included this code because it is
necessary to create the frontend DB, but it will (hopefully) soon be
part of the SVN repository.

Lastly, there is an apache directory containing configuration files for
running the web server through Apache.

I've put instructions for installing a full Autotest server, including
the web frontend, the scheduler, and the existing "tko" results backend,
at http://test.kernel.org/autotest/AutotestServerInstall. I've also
created a brief guide to using the web frontend, with plenty of
screenshots, at http://test.kernel.org/autotest/WebFrontendHowTo.
Please let me know if you find any problems with either of these pages.

Take a look, try it out, send me comments, complaints, and bugs, and I
hope you find it useful!

Steve Howard, and the rest of the Google Autotest team

From: Steve Howard <showard@google.com>




git-svn-id: http://test.kernel.org/svn/autotest/trunk@1242 592f7852-d20e-0410-864c-8624ca9c26a4
_init__.py
fe/__init__.py
fe/control_file.py
fe/doctests/rpc_test.txt
fe/doctests/test.control
fe/enum.py
fe/fixtures/initial_data.json
fe/json_rpc/__init__.py
fe/json_rpc/proxy.py
fe/json_rpc/serviceHandler.py
fe/management.py
fe/models.py
fe/rpc_client_lib.py
fe/rpc_handler.py
fe/rpc_interface.py
fe/rpc_utils.py
fe/simplejson/__init__.py
fe/simplejson/decoder.py
fe/simplejson/encoder.py
fe/simplejson/scanner.py
fe/site_rpc_interface.py
fe/templates/admin/base.html
fe/test.py
fe/urls.py
fe/views.py
pache_auth.py
lient/.classpath
lient/.project
lient/ClientMain-compile
lient/ClientMain-shell
lient/ClientMain.launch
lient/generate-javadoc
lient/gwt_dir
lient/src/afeclient/ClientMain.gwt.xml
lient/src/afeclient/client/ClassFactory.java
lient/src/afeclient/client/ClientMain.java
lient/src/afeclient/client/CreateJobView.java
lient/src/afeclient/client/DataTable.java
lient/src/afeclient/client/DynamicTable.java
lient/src/afeclient/client/ElementWidget.java
lient/src/afeclient/client/HostListView.java
lient/src/afeclient/client/HostSelector.java
lient/src/afeclient/client/HostTable.java
lient/src/afeclient/client/JobDetailView.java
lient/src/afeclient/client/JobHostsTable.java
lient/src/afeclient/client/JobListView.java
lient/src/afeclient/client/JobTable.java
lient/src/afeclient/client/JsonRpcCallback.java
lient/src/afeclient/client/JsonRpcProxy.java
lient/src/afeclient/client/NotifyManager.java
lient/src/afeclient/client/Paginator.java
lient/src/afeclient/client/SimpleCallback.java
lient/src/afeclient/client/SimpleHyperlink.java
lient/src/afeclient/client/SiteClassFactory.java
lient/src/afeclient/client/StaticDataRepository.java
lient/src/afeclient/client/TabView.java
lient/src/afeclient/client/Utils.java
lient/src/afeclient/public/ClientMain.html
lient/src/afeclient/public/afeclient.css
lient/src/afeclient/public/arrow_down.png
lient/src/afeclient/public/arrow_up.png
lient/src/afeclient/public/header.png
anage.py
igrations/001_initial_db.py
igrations/002_cleanup_fields.py
ettings.py
rls.py