| 33e1289d9622cb2f8997eef6a8a7cef35db76233 |
|
26-May-2017 |
Richard Barnette <jrbarnette@chromium.org> |
[autotest] Delete the arc-bvt-cq suite.
The suite is now unused and redundant with bvt-arc.
BUG=chromium:647494
TEST=run the generate_controlfiles script
Change-Id: Ica9b4aaded48ec5722c6de87dcec347505e3d4b1
Reviewed-on: https://chromium-review.googlesource.com/516736
Commit-Ready: Richard Barnette <jrbarnette@chromium.org>
Tested-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Ilja H. Friedel <ihf@chromium.org>
/external/autotest/site_utils/gs_offloader_unittest.py
|
| 0f553bd773170c8ddb80d6061cdc0f133ba239b6 |
|
17-Jun-2017 |
Michael Tang <ntang@google.com> |
Wrap the pubsub into a cloud console client.
BUG=chromium:724523
TEST=unit tests.
Change-Id: I4338cce8c2c983b4cd50b71014d4f1ca5667cd78
Reviewed-on: https://chromium-review.googlesource.com/540137
Commit-Ready: Michael Tang <ntang@chromium.org>
Tested-by: Michael Tang <ntang@chromium.org>
Reviewed-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Michael Tang <ntang@chromium.org>
/external/autotest/site_utils/gs_offloader_unittest.py
|
| b41527d6f865688299dc5250f8d58baca9432777 |
|
23-Jun-2017 |
Allen Li <ayatane@chromium.org> |
[autotest] gs_offloader refactor
BUG=None
TEST=None
This commit is the combination of multiple commits. Their commit
messages are as follows:
commit 4f934510a2f28c5c838c514f8b7dcab6fa81fa66
Author: Allen Li <ayatane@chromium.org>
Date: Fri May 5 12:13:58 2017 -0700
[autotest] Move get_sanitized_name()
Doing some housekeeping
BUG=chromium:715435
TEST=None
Change-Id: I14a1a9886018cfe0eb6c3a1a77b45836d8476a21
commit c6f7fb299886b67c5eaeb133a59db3b80ef86d1c
Author: Allen Li <ayatane@chromium.org>
Date: Fri May 5 12:22:53 2017 -0700
[autotest] Add escape() tests
BUG=chromium:715435
TEST=None
Change-Id: Ibeee38113043e7d16605b1fe5efa8b31350b8629
commit ce42ca80d166330d15113169f6542b839ac417e8
Author: Allen Li <ayatane@chromium.org>
Date: Fri May 5 12:28:47 2017 -0700
[autotest] Clean up escape()
BUG=chromium:715435
TEST=None
Change-Id: I99edb6bf2bd6a9ffebf92893d1b834d815f797b8
commit 87353b0b7efb4ffc55946d40c34ae83410cf0314
Author: Allen Li <ayatane@chromium.org>
Date: Fri May 5 12:33:38 2017 -0700
[autotest] Extract _setup_logging()
BUG=chromium:715435
TEST=None
Change-Id: Iab967852f043238730d2c502dc030bdced156b60
commit 0ac380ec9e267efe317d43f1e303f426f6f1a3f4
Author: Allen Li <ayatane@chromium.org>
Date: Fri May 5 12:35:15 2017 -0700
[autotest] Extract _get_log_filename()
BUG=chromium:715435
TEST=None
Change-Id: I7cbdd4523a76a3d0d06b810b9b8d07208c951861
commit 387b946ca761076bd67934554b4f67af8dc02f3a
Author: Allen Li <ayatane@chromium.org>
Date: Fri May 5 12:50:14 2017 -0700
[autotest] Move enqueue_offload()
This is only used by gs_offloader, yet its a public method on a
general purpose class.
(Hypothesis: this is why the generic function paradigm is better than
methods. The set of functions that want to dispatch on the class is a
superset of the functions that want to be methods on the class.)
BUG=chromium:715435
TEST=None
Change-Id: I8bad5c2f8fea7b922769bc7a9e9121003b1aab60
commit f920fae06b621f6bc6ae429abbf508d191c62ac2
Author: Allen Li <ayatane@chromium.org>
Date: Fri May 5 12:58:32 2017 -0700
[autotest] Move logging constants next to logging setup
BUG=chromium:715435
TEST=None
Change-Id: I9904322ee39cfc07edabe0a1294ecab94201f40c
commit 5cf63fcf06e939d7f00dc4ecf6a0f3ecd71e0c32
Author: Allen Li <ayatane@chromium.org>
Date: Fri May 5 13:07:25 2017 -0700
[autotest] Extract _sanitize_symlinks()
BUG=chromium:715435
TEST=None
Change-Id: Ie5f523b0f4472d2a55f1472b6c68199c97fbb3a8
commit d108e7ba6aca44a13b8b0732f2bd7680a50c74f1
Author: Allen Li <ayatane@chromium.org>
Date: Fri May 5 13:13:13 2017 -0700
[autotest] Extract _sanitize_fifos()
BUG=chromium:715435
TEST=None
Change-Id: I7d3392fcb27cb629ede2547f865c8f26b432acc3
commit 1e73efdf7ada7c450f78882523b42f56c3c0b942
Author: Allen Li <ayatane@chromium.org>
Date: Fri May 5 14:18:00 2017 -0700
[autotest] Extract _escape_rename()
BUG=chromium:715435
TEST=None
Change-Id: I2f665ffd00552e2bfb62050f0242727cea34a286
commit 96d47cff7634c203b809a98483999dfbe38de5cc
Author: Allen Li <ayatane@chromium.org>
Date: Fri May 5 14:41:02 2017 -0700
[autotest] Inline get_directory_size_kibibytes_cmd_list()
This isnt actually mocked out or referenced anywhere. There are
better ways to mock this out anyway.
BUG=chromium:715435
TEST=None
Change-Id: If1fc3f72fbb32371b087143efaca6bb2406678c0
commit 104ef14998ef31a9eafe214463a96f66b75bdd6d
Author: Allen Li <ayatane@chromium.org>
Date: Fri May 5 14:42:23 2017 -0700
[autotest] Clean up sanitize_dir()
BUG=chromium:715435
TEST=None
Change-Id: Ib0cc1a5842616e1d333328e7f9e95cf973afee88
commit c34f213c482233807bf06929fce3534a93977804
Author: Allen Li <ayatane@chromium.org>
Date: Fri May 5 14:44:36 2017 -0700
[autotest] Move MAX_FILE_COUNT to where its used
BUG=chromium:715435
TEST=None
Change-Id: I728ffafeead8d4e4cfa97e023a6a817420edc2a2
commit 4b85648f8978e75c42e0c7ff007bc0e802dc6979
Author: Allen Li <ayatane@chromium.org>
Date: Fri May 5 14:52:43 2017 -0700
[autotest] Extract _make_into_tarball()
BUG=chromium:715435
TEST=None
Change-Id: I2869bbc2ee15ab97c0850b7421e3a946609901a6
commit d6be13aa211abae6c320fc5525097c62afc7258b
Author: Allen Li <ayatane@chromium.org>
Date: Fri May 5 15:00:03 2017 -0700
[autotest] Use tarfile for _make_into_tarball()
BUG=chromium:715435
TEST=None
Change-Id: Id0937d15957ce1227b2a29fbd4f388f11eb0179c
commit b746a4eb7e06405b4bbd4c35f4c68fbdc8ee44ae
Author: Allen Li <ayatane@chromium.org>
Date: Fri May 5 15:05:43 2017 -0700
[autotest] Extract _count_files()
BUG=chromium:715435
TEST=None
Change-Id: I454573879911e70bae0d0b50ad57bad5899daeac
commit fb0a007caaab379d2544f292b73f0a3efcc969bf
Author: Allen Li <ayatane@chromium.org>
Date: Fri May 5 15:08:00 2017 -0700
[autotest] Simplify _count_files()
BUG=chromium:715435
TEST=None
Change-Id: Ibdead3268a5051416bf6fff095e542a87d2d525a
commit e58932671a693dacc6b74e5bebc5b106aa95ddaa
Author: Allen Li <ayatane@chromium.org>
Date: Fri May 5 15:19:19 2017 -0700
[autotest] Move notification constants
BUG=chromium:715435
TEST=None
Change-Id: Iad2a2f7f869d66d230a0de62b4f75802f16709c4
commit 6fcf74934821a141977f025fe322722c4e43c689
Author: Allen Li <ayatane@chromium.org>
Date: Fri May 5 15:25:25 2017 -0700
[autotest] Make offload_dir a callable instead of a closure
I swear, the first thing that a programmer does after learning about
closures is to rewrite every object as a closure.
BUG=chromium:715435
TEST=None
Change-Id: Ifd3cf9b8821b4926dc496859620f16fa34e4960d
commit 7f58dfc1dff99b2679515598352da837236615e0
Author: Allen Li <ayatane@chromium.org>
Date: Fri May 5 15:34:30 2017 -0700
[autotest] Make delete_files() inherit from GSOffloader
BUG=chromium:715435
TEST=None
Change-Id: Idbe4b1bd1d73edb25857ad22f0017df0ecb74402
commit 3b2b10ff73900ad11ca3a5fc57703ffc52a66c4d
Author: Allen Li <ayatane@chromium.org>
Date: Fri May 5 18:40:06 2017 -0700
[autotest] Fix GSOffloader semantics
BUG=chromium:715435
TEST=None
Change-Id: I82adfdcc9349b3afb3b6ccd487c53e90afb91c59
commit 4884e2f3c38bd590cee5eee843a20650bf24b59b
Author: Allen Li <ayatane@chromium.org>
Date: Fri May 5 15:42:31 2017 -0700
[autotest] Use context manager for tempfiles
BUG=chromium:715435
TEST=None
Change-Id: I526708355247e54620723ae1eb6a1bd5ca7ef653
commit 2fd102484811a8980d6c9893969f3bf3188bc2e3
Author: Allen Li <ayatane@chromium.org>
Date: Fri May 5 15:53:40 2017 -0700
[autotest] Use chromite timeout library
BUG=chromium:715435
TEST=None
Change-Id: I85e6457d17d27580a4443f96021a8d280dec9d5d
commit 63229138951ee4e76c43dc2578b50aec21b87501
Author: Allen Li <ayatane@chromium.org>
Date: Fri May 5 16:25:42 2017 -0700
[autotest] Split GSOffload call body
This will make subsequent CL diffs clearer.
BUG=chromium:715435
TEST=None
Change-Id: I6eca38fc060639882d9d2869a0df774a597f171d
commit 89e5e4eb5cddd0b8565ceb568e87083304b40520
Author: Allen Li <ayatane@chromium.org>
Date: Fri May 5 16:27:32 2017 -0700
[autotest] Move TemporaryFile context out
Remove indentation levels.
BUG=chromium:715435
TEST=None
Change-Id: I602c16c241ada2bc452630523764168dd1dda8cb
commit 819c8b1f6c9ce236fb7a7e12992a8ca555b544d2
Author: Allen Li <ayatane@chromium.org>
Date: Fri May 5 16:29:37 2017 -0700
[autotest] Move error handling out
BUG=chromium:715435
TEST=None
Change-Id: I7e712a0c6d0087f7fa5b44ccc45dec72503f365b
commit 13e9132457b0e40f25fcc62df939dcb37c23fe23
Author: Allen Li <ayatane@chromium.org>
Date: Fri May 19 16:57:08 2017 -0700
[autotest] Extract _emit_offload_metrics()
BUG=chromium:715435
TEST=None
Change-Id: I2299344c2d63a3d9585c8f83daf341b7ba1c5d9d
commit 1b0b0708db7d8f370942be9b10f9456983d63ce4
Author: Allen Li <ayatane@chromium.org>
Date: Fri May 19 16:57:40 2017 -0700
[autotest] Remove useless variable
BUG=chromium:715435
TEST=None
Change-Id: I690f3fa6ae7a8c5a224586057292713ae763bb60
commit 92806326b6fe8173c98e0560247f2c64041705dd
Author: Allen Li <ayatane@chromium.org>
Date: Fri May 5 17:38:18 2017 -0700
[autotest] Move pruning logic out
This OSError is supposed to only apply to the pruning; do that. Also
disentangle this pile of spaghetti.
BUG=chromium:715435
TEST=None
Change-Id: If46a5962f4530044058a4cd5ba840a9195c142a7
commit c604443fb5299d26fad0bb00c2b4b0f3bc99b24d
Author: Allen Li <ayatane@chromium.org>
Date: Fri May 5 17:42:44 2017 -0700
[autotest] Decrease scope of error flag
BUG=chromium:715435
TEST=None
Change-Id: If6a86f04c54de0c44774bd842902d54433925273
commit a519607dc6bf93356d7b336e90e9d649c6da441b
Author: Allen Li <ayatane@chromium.org>
Date: Fri May 5 17:46:05 2017 -0700
[autotest] Extract pruning logic
BUG=chromium:715435
TEST=None
Change-Id: Ia454d9d441e3a4e404350219041eadcaf8bdb0ef
commit e0e79552fe4fb7982e7cdabcc3ae28c0c34e40f4
Author: Allen Li <ayatane@chromium.org>
Date: Fri May 5 17:47:39 2017 -0700
[autotest] Clean up pruning test logic
BUG=chromium:715435
TEST=None
Change-Id: Idcb936ab659223eed0476858a05ce0b7a6b4e61e
commit 53e076a46ea1a4c01f0d41d2616a84b8a1d02f60
Author: Allen Li <ayatane@chromium.org>
Date: Fri May 19 17:11:55 2017 -0700
[autotest] Remove error flag
Even C++ has exceptions.
BUG=chromium:715435
TEST=None
Change-Id: I0c2e61cb753f8130356388c307410bb8d37489a9
commit 546180941013147ff493139c33f983e67a03a789
Author: Allen Li <ayatane@chromium.org>
Date: Fri May 19 17:12:53 2017 -0700
[autotest] Remove redudant check
BUG=chromium:715435
TEST=None
Change-Id: Iecc17ec88a78af691a4b3f6a444d2c033bbc3ae5
commit ab2fbbb578eeeb73f5db71c0266c8a551bfef748
Author: Allen Li <ayatane@chromium.org>
Date: Mon May 15 14:48:09 2017 -0700
[autotest] Move uploaded test out
BUG=chromium:715435
TEST=None
Change-Id: Ic265aba95aec337132251c50b6d86a51b415b86b
commit 341f36d00d5b0b5973da3fc78f92db218d17b5ff
Author: Allen Li <ayatane@chromium.org>
Date: Mon May 15 14:49:23 2017 -0700
[autotest] Fix indentation
BUG=chromium:715435
TEST=None
Change-Id: I851a790d9470f99bfc53ca3f71efee8fbc5ab9d1
commit d63eb51d5e5d9553bf6b33dee9b6d7b49135e115
Author: Allen Li <ayatane@chromium.org>
Date: Mon May 15 14:58:11 2017 -0700
[autotest] Remove is_offloaded()
There are two concepts of is_offloaded being used in gs_offloader.
One of them is an existence check that isnt actually quite right.
Remove it.
BUG=chromium:715435
TEST=None
Change-Id: Ib9002b848e74b55e29d17131a3c4962ba32f6207
commit 40ff0da7e960e6861557f3064a3d4618369d8f9e
Author: Allen Li <ayatane@chromium.org>
Date: Mon May 15 15:00:43 2017 -0700
[autotest] Clarify _update_offload_results()
BUG=chromium:715435
TEST=None
Change-Id: I23986726ac1d02a60baf64ba4d137f132c1b9d4f
commit a004fd4c4ff7c9df390b80941bba3a8d38355132
Author: Allen Li <ayatane@chromium.org>
Date: Mon May 15 15:05:56 2017 -0700
[autotest] Dont re-add uploaded jobs
BUG=chromium:715435
TEST=None
Change-Id: I054552667d62110f1e3a16099aeacb99e7d31661
commit 85e22760e53e23be728bec9bb983f710e1c0cdcc
Author: Allen Li <ayatane@chromium.org>
Date: Mon May 15 15:15:33 2017 -0700
[autotest] Fix upload_testresult_files() name
BUG=chromium:715435
TEST=None
Change-Id: I2da7b5c0a25c6cd8f295aed795e614449d93f9ba
commit 20add289f72659648fa5904524eb6d2c9bbe06f8
Author: Allen Li <ayatane@chromium.org>
Date: Mon May 15 15:32:18 2017 -0700
[autotest] Extract _remove_log_directory_contents()
BUG=chromium:715435
TEST=None
Change-Id: I06f2b1f312a0836854bfd43aaa6145473b558c2f
commit a53a80a78c7d72999645d7cb20e9d3e1541bbabe
Author: Allen Li <ayatane@chromium.org>
Date: Mon May 15 15:33:40 2017 -0700
[autotest] Replace get_job_directory()
This isnt Java, we dont need a getter.
BUG=chromium:715435
TEST=None
Change-Id: If97efd0ff8c6698ee8ce386dd113e60c53d02627
commit 8cceb41b48ec68b68ee050c41cc8255ab7090086
Author: Allen Li <ayatane@chromium.org>
Date: Mon May 15 15:40:14 2017 -0700
[autotest] Extract _is_expired()
BUG=chromium:715435
TEST=None
Change-Id: I81924e5ca47489692d836866f83fd9506884f36f
commit f3bfca34e41b3a6643785184e7e6d3f236f90831
Author: Allen Li <ayatane@chromium.org>
Date: Mon May 15 15:42:50 2017 -0700
[autotest] Make get_cmd_list private
BUG=chromium:715435
TEST=None
Change-Id: Id39af7863844b5ed7d5ec3c212e0683b133921e8
commit db314fad7d61ee25af4ac3c3bf8a6ba88e0538d4
Author: Allen Li <ayatane@chromium.org>
Date: Mon May 15 15:48:23 2017 -0700
[autotest] Replace get_failure_time(), get_failure_count()
This isnt Java, we dont need a getter.
BUG=chromium:715435
TEST=None
Change-Id: I857de5947d230c0c7d9f01a11214831189268369
commit c6bd98ca98d92d669c9695d62edd40a67c17be4f
Author: Allen Li <ayatane@chromium.org>
Date: Fri Jun 16 15:26:26 2017 -0700
[autotest] Fix damned mox thing
I swear it works locally, but it fails on trybot. I think mock will
be more reliable.
BUG=None
TEST=None
Change-Id: I7bf11033aee567a5a081b4a2be64769eef3b7f15
commit bb3f60bb6470fd37ba645f562615c2142f4b1d64
Author: Allen Li <ayatane@chromium.org>
Date: Mon Jun 19 12:27:34 2017 -0700
[autotest] Extract tempdir logic into a separate class
This class will be reused for a non-mox test class, because mox is
giving much such a ridiculous time. Moving a few function around
costs hours fiddling with mox tests.
BUG=None
TEST=None
Change-Id: I58fd573a8dbb046b4ab2449dfa22044102088b2c
commit e3b27f31393b6488547103b4f605b49b897cf2be
Author: Allen Li <ayatane@chromium.org>
Date: Mon Jun 19 12:32:04 2017 -0700
[autotest] Add non-mox version of timeout tests
This is confirmed passing when run locally.
BUG=None
TEST=None
Change-Id: I2196d51928b0ca35d783727f2db109148f80fe77
Change-Id: I21d905707ff67a8c124a84458f7e0e313f5fa433
Reviewed-on: https://chromium-review.googlesource.com/538996
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Allen Li <ayatane@chromium.org>
/external/autotest/site_utils/gs_offloader_unittest.py
|
| 365049f691c80802030f62cf2fce345bb670e00d |
|
28-May-2017 |
Dan Shi <dshi@google.com> |
Revert "[autotest] move gs_offloader to use the chromite version of gsutil"
BUG=b:62157091
TEST=none
This reverts commit 82ad4501ac5d0fca275b4c370231f1c214da7922.
Change-Id: I597dac6b8b2523eee3cf54f7e352afddb5393644
Reviewed-on: https://chromium-review.googlesource.com/517398
Trybot-Ready: Dan Shi <dshi@google.com>
Reviewed-by: Dan Shi <dshi@google.com>
Commit-Queue: Dan Shi <dshi@google.com>
Tested-by: Dan Shi <dshi@google.com>
/external/autotest/site_utils/gs_offloader_unittest.py
|
| 9579b38072f8008b92623d7c30e3fc55c05c7561 |
|
06-May-2017 |
Allen Li <ayatane@chromium.org> |
[autotest] Abstract out upload marker file logic
BUG=chromium:715435
TEST=None
Change-Id: I7ab86b06dc86cb18d307ab0e726987ead199a152
Reviewed-on: https://chromium-review.googlesource.com/497722
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Allen Li <ayatane@chromium.org>
/external/autotest/site_utils/gs_offloader_unittest.py
|
| 02dd066dc11876736eaf1841e6893a805febd899 |
|
23-May-2017 |
Dan Shi <dshi@google.com> |
[autotest] Fix a bug in labellib and better handling exception in gs_offloader
BUG=chromium:506781
TEST=unittest
Change-Id: I66935289ad048ecc25d3f60c8138eef2b91ba31f
Reviewed-on: https://chromium-review.googlesource.com/512986
Commit-Queue: Dan Shi <dshi@google.com>
Tested-by: Dan Shi <dshi@google.com>
Trybot-Ready: Dan Shi <dshi@google.com>
Reviewed-by: Allen Li <ayatane@chromium.org>
/external/autotest/site_utils/gs_offloader_unittest.py
|
| 82ad4501ac5d0fca275b4c370231f1c214da7922 |
|
16-May-2017 |
Keith Haddow <haddowk@chromium.org> |
[autotest] move gs_offloader to use the chromite version of gsutil
Infra team moved devserver to use the chromite gsutil.
Move gs_offloader to be consistent
BUG=chromium:722976
TEST=run unit tests.
Change-Id: I58a94c29bbefb75d49fc053b7c0db8315d5cf22e
Reviewed-on: https://chromium-review.googlesource.com/506604
Commit-Ready: Keith Haddow <haddowk@chromium.org>
Tested-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Dan Shi <dshi@google.com>
/external/autotest/site_utils/gs_offloader_unittest.py
|
| 4ad0c3b47ea2fcf52a0cb3fe7f3dc6b5f853836a |
|
05-May-2017 |
Allen Li <ayatane@chromium.org> |
[autotest] Make unittests executable
BUG=chromium:715435
TEST=None
Change-Id: I7097061e31e271ce1d4bff144a60d9f819eb5bdf
Reviewed-on: https://chromium-review.googlesource.com/498050
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Aviv Keshet <akeshet@chromium.org>
/external/autotest/site_utils/gs_offloader_unittest.py
|
| 9358538d79b35ef0f586b7e9f2f0d6741f9c44c1 |
|
05-May-2017 |
Allen Li <ayatane@chromium.org> |
[autotest] Lint cleanup
BUG=chromium:715435
TEST=None
Change-Id: I8b360c57f2bba945d9f60354489aa74c67360255
Reviewed-on: https://chromium-review.googlesource.com/498049
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Aviv Keshet <akeshet@chromium.org>
/external/autotest/site_utils/gs_offloader_unittest.py
|
| 5ed7e63aeec90dc0c6b4229a66bf161410aa86a6 |
|
04-Feb-2017 |
Allen Li <ayatane@chromium.org> |
[autotest] Merge client/common_lib/site_utils
BUG=chromium:672727
TEST=None
Change-Id: I15859c0e05e99d244eec5b470a984ced0c874f19
Reviewed-on: https://chromium-review.googlesource.com/437727
Commit-Ready: Allen Li <ayatane@chromium.org>
Tested-by: Allen Li <ayatane@chromium.org>
Reviewed-by: Allen Li <ayatane@chromium.org>
/external/autotest/site_utils/gs_offloader_unittest.py
|
| ca7726dfcd4b9b70a75147bad1d1c2ec66b79816 |
|
15-Feb-2017 |
Laurence Goodby <lgoodby@google.com> |
gs_offloader: Don't upload named pipes or symlinks.
The gsutil cp command is known to hang on uploading named pipes.
This change removes named pipes and replaces them with a marker
file.
A similar transformation is done for symbolic links.
BUG=chromium:684122,chromium:692788
TEST=Running local autotest server with patch. Run gs_offloader_unittest.
Change-Id: I8d4c59f8864462cf71ece476f721e74fc70bc2b7
Reviewed-on: https://chromium-review.googlesource.com/442630
Commit-Ready: Laurence Goodby <lgoodby@chromium.org>
Tested-by: Laurence Goodby <lgoodby@chromium.org>
Reviewed-by: Don Garrett <dgarrett@chromium.org>
/external/autotest/site_utils/gs_offloader_unittest.py
|
| 73cf6cd0049a0eddd1cfdc3f73cbaf2d1f114577 |
|
01-Mar-2017 |
Ilja H. Friedel <ihf@chromium.org> |
gs_offloader: upload new cheets_CTS_N.
BUG=b:35605677
TEST=pylint, python gs_offloader_unittest.py
Change-Id: I0790119a06e5d0743458d56522c3d6cedaa5af72
Reviewed-on: https://chromium-review.googlesource.com/447974
Commit-Ready: Haixia Shi <hshi@chromium.org>
Tested-by: Ilja H. Friedel <ihf@chromium.org>
Reviewed-by: Haixia Shi <hshi@chromium.org>
Reviewed-by: Ilja H. Friedel <ihf@chromium.org>
Reviewed-by: Kazuhiro Inaba <kinaba@chromium.org>
/external/autotest/site_utils/gs_offloader_unittest.py
|
| ca4815909f753b9117e5e87045dda7f088ba8e6f |
|
31-Jan-2017 |
Prathmesh Prabhu <pprabhu@chromium.org> |
gs_offloader: Remove is_reportable from JobDirectory.
We no longer report failures via emails, so the is_reportable logic is
no longer needed.
BUG=chromium:686904
TEST=None
Change-Id: I2d5b0ee1aeb9a3d7a973b9fade818cf6b6a6d0f9
Reviewed-on: https://chromium-review.googlesource.com/435104
Commit-Ready: Prathmesh Prabhu <pprabhu@chromium.org>
Tested-by: Prathmesh Prabhu <pprabhu@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
Reviewed-by: Allen Li <ayatane@chromium.org>
/external/autotest/site_utils/gs_offloader_unittest.py
|
| 33e3e1ad214e6d681f64eb01835b18bb045ce463 |
|
31-Jan-2017 |
Prathmesh Prabhu <pprabhu@chromium.org> |
gs_offloader: Stop sending emails on offloading failures.
Better monarch based reporting is in place. A list of failed jobs is now
dropped on the server for debugging purposes.
This CL also removes the unittests around email reporting, since the
logic related to deciding when an email report should be sent is now
gone.
BUG=chromium:686904
TEST=None
Change-Id: I2db1463ac72d9c51e598345f02871aba65de744e
Reviewed-on: https://chromium-review.googlesource.com/435103
Commit-Ready: Prathmesh Prabhu <pprabhu@chromium.org>
Tested-by: Prathmesh Prabhu <pprabhu@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
/external/autotest/site_utils/gs_offloader_unittest.py
|
| 80dfb1e9366b74f6c5a928e33793012e5a8ebc5a |
|
31-Jan-2017 |
Prathmesh Prabhu <pprabhu@chromium.org> |
gs_offloader: Migrate unittests to new reporting method.
This CL replaces the unittest for email reporting from gs_offloader to
instead verify it's replacement -- the locally dropped file containing
information about failed offloader jobs.
BUG=chromium:686904
TEST=unittests.
Change-Id: Ia5b75a4bf6fffde82fe4f7353a7ed713bdf8bee7
Reviewed-on: https://chromium-review.googlesource.com/435102
Commit-Ready: Prathmesh Prabhu <pprabhu@chromium.org>
Tested-by: Prathmesh Prabhu <pprabhu@chromium.org>
Reviewed-by: Allen Li <ayatane@chromium.org>
/external/autotest/site_utils/gs_offloader_unittest.py
|
| 16f9e5c9cbeadbfb030d2b2cf90c0816712d43d9 |
|
31-Jan-2017 |
Prathmesh Prabhu <pprabhu@chromium.org> |
gs_offloader: Locally log the list of failed jobs.
We now maintain a log file on disk with the list of failed gs_offloader
jobs. Before this CL, a developer had to go scrub emails to find out
what jobs had failed from being offloaded. Now, they can look at the
file on disk on the server where gs_offloader runs.
This prepares the way to stop sending those annoying emails.
BUG=chromium:686904
TEST=None.
Change-Id: I691f04de07aafa01d606041ee0dc4809a60daf30
Reviewed-on: https://chromium-review.googlesource.com/435101
Commit-Ready: Prathmesh Prabhu <pprabhu@chromium.org>
Tested-by: Prathmesh Prabhu <pprabhu@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
/external/autotest/site_utils/gs_offloader_unittest.py
|
| beb9e01b8aebdd33b7c866fbdfef64bc77f139e5 |
|
31-Jan-2017 |
Prathmesh Prabhu <pprabhu@chromium.org> |
gs_offloader: Reorder imports, remove relative import.
BUG=chromium:686904
TEST=None
Change-Id: Ib88f08f3bcd88cd81de659d9a83be9ab34c3de92
Reviewed-on: https://chromium-review.googlesource.com/434970
Commit-Ready: Prathmesh Prabhu <pprabhu@chromium.org>
Tested-by: Prathmesh Prabhu <pprabhu@chromium.org>
Reviewed-by: Allen Li <ayatane@chromium.org>
/external/autotest/site_utils/gs_offloader_unittest.py
|
| bfa63148ef62dc36e2d4b3456a4b64ae69b21172 |
|
26-Jan-2017 |
Ilja H. Friedel <ihf@chromium.org> |
gs_offloader: adjust for new GTS paths.
With GTS 4.1 R1 the result files are located at
cheets_GTS.GtsAdminTestCases/results/android-gts/2017.01.25_23.57.54/test_result.xml
TEST=python gs_offloader_unittest.py
BUG=b:34176927
Change-Id: I395b628a81a5da69b0c767053d42effa7f12a452
Reviewed-on: https://chromium-review.googlesource.com/433378
Commit-Ready: Ilja H. Friedel <ihf@chromium.org>
Tested-by: Ilja H. Friedel <ihf@chromium.org>
Reviewed-by: Ilja H. Friedel <ihf@chromium.org>
/external/autotest/site_utils/gs_offloader_unittest.py
|
| b45c7237e725d044838840872a118536b549cee3 |
|
15-Nov-2016 |
Michael Tang <ntang@google.com> |
Added the <job>-moblab to the gcs_uri in pubsub notification.
BUG=chromium:664603
TEST=manual and unit test.
Change-Id: I88a5e7286b6fb1d6eb9e247e8aafa00def407ed5
Reviewed-on: https://chromium-review.googlesource.com/411516
Commit-Ready: Michael Tang <ntang@chromium.org>
Tested-by: Michael Tang <ntang@chromium.org>
Reviewed-by: Moises Osorio <moisesosorio@chromium.org>
Reviewed-by: Michael Tang <ntang@chromium.org>
/external/autotest/site_utils/gs_offloader_unittest.py
|
| 5ba5fb86d92a242691def5fc87ff35e51497b2ae |
|
09-Nov-2016 |
Keith Haddow <haddowk@chromium.org> |
Autotest: change gs_offloader to seperate upload age from delete age.
Currently gs_offloader has a single param to select how old a job
should be before it is uploaded and deleted.
Introduce two new params one to select how old the job is when it is
uploaded, and a second when it is deleted.
If these params are the same or the original param is set the
functionality of gs_offloader should be the same.
BUG=chromium:659202
TEST=local testing on moblab, ran/fixed available unit tests.
Change-Id: I91e63ba39ddbd8c0ea408c3f5111eead76eaa0f7
Reviewed-on: https://chromium-review.googlesource.com/410060
Commit-Ready: Keith Haddow <haddowk@chromium.org>
Tested-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Keith Haddow <haddowk@chromium.org>
Reviewed-by: Michael Tang <ntang@chromium.org>
/external/autotest/site_utils/gs_offloader_unittest.py
|
| 328073b9ee29148e5db89f8148a7571cd04f6fbb |
|
01-Nov-2016 |
Michael Tang <ntang@google.com> |
Adds extra fields to the test job offload notification.
BUG=chromium:660913
TEST=Unitest and Manual.
Change-Id: Id83f3a9fd1254e919c6486e55615e6d009ea06f5
Reviewed-on: https://chromium-review.googlesource.com/406447
Commit-Ready: Michael Tang <ntang@chromium.org>
Tested-by: Michael Tang <ntang@chromium.org>
Reviewed-by: Moises Osorio <moisesosorio@chromium.org>
Reviewed-by: Michael Tang <ntang@chromium.org>
/external/autotest/site_utils/gs_offloader_unittest.py
|
| 6384c1094ddf148de5d3ff16aed1bbab077b5f26 |
|
22-Oct-2016 |
Rohit Makasana <rohitbm@chromium.org> |
Whitelist test_that_wrapper in gs_offloader.
BUG=chromium:658458
TEST=None
Change-Id: Ic44c045c74ce43066fed5b448ecafb3dc0cff3d9
Reviewed-on: https://chromium-review.googlesource.com/401996
Commit-Ready: Rohit Makasana <rohitbm@chromium.org>
Tested-by: Rohit Makasana <rohitbm@chromium.org>
Reviewed-by: Dan Shi <dshi@google.com>
/external/autotest/site_utils/gs_offloader_unittest.py
|
| 8db632fba7f5cbbc4b6693a678582b13039f01c4 |
|
19-Aug-2016 |
Ningning Xia <nxia@chromium.org> |
Add filter for suite='arc-cts' in gs_offloader.upload_testresult_files
Only upload 'arc-cts' suite results while uploading CTS results to CTS
budkets.
BUG=chromium:637498
TEST=unit test
Change-Id: I8115f303b4caff2505c4f6d69fcac906c5b6953f
Reviewed-on: https://chromium-review.googlesource.com/373044
Commit-Ready: Ningning Xia <nxia@chromium.org>
Tested-by: Ningning Xia <nxia@chromium.org>
Reviewed-by: Ningning Xia <nxia@chromium.org>
/external/autotest/site_utils/gs_offloader_unittest.py
|
| 0c27d9befe9523e70b2d69ee353caa7b43270dbd |
|
04-Aug-2016 |
Ningning Xia <nxia@chromium.org> |
Add unit test cases for gs_offloader_unittest.
Add a test result folder with a different timestamp in
gs_offloader_unittest.test_upload_testresult_files.
Fix test_upload_files by moving try/except to the upper layer in
upload_testresult_files; otherwise, failure unittests will be swallowed
by the try/except block.
BUG=chromium:619209
TEST=unit test
Change-Id: I22985e167cdb1ab01d97602f52f2e48238509a2f
Reviewed-on: https://chromium-review.googlesource.com/366430
Commit-Ready: Dan Shi <dshi@google.com>
Tested-by: Ilja H. Friedel <ihf@chromium.org>
Tested-by: Ningning Xia <nxia@chromium.org>
Reviewed-by: Dan Shi <dshi@google.com>
/external/autotest/site_utils/gs_offloader_unittest.py
|
| 21922c807733cab1a86096500304aa8ae68bd897 |
|
29-Jul-2016 |
Ningning Xia <nxia@chromium.org> |
Only offload CTS/GTS results for release builds.
Only offload results for release builds, not including pfq, paladin or
trybots builds.
BUG=chromium:632490
TEST=unit_test;run_gs_offloader
Change-Id: I93107c773c242f0193447148a37f47491e5af651
Reviewed-on: https://chromium-review.googlesource.com/364720
Commit-Ready: Ningning Xia <nxia@chromium.org>
Tested-by: Ningning Xia <nxia@chromium.org>
Reviewed-by: Jason Kusuma <jkusuma@chromium.org>
Reviewed-by: Dan Shi <dshi@google.com>
/external/autotest/site_utils/gs_offloader_unittest.py
|
| 2d88eec8f87da75d3ae5046ee34573562add3da8 |
|
26-Jul-2016 |
Ningning Xia <nxia@chromium.org> |
Upload GTS results to GS buckets.
Upload GTS results to chromeos-cts-results and chromeos-cts-apfe.
BUG=chromium:630762
TEST=unit test;manually run gs_offloader
Change-Id: I33b761d5a682525fb376c64e2a8e44f1493396f5
Reviewed-on: https://chromium-review.googlesource.com/363510
Commit-Ready: Ningning Xia <nxia@chromium.org>
Tested-by: Ningning Xia <nxia@chromium.org>
Reviewed-by: Ningning Xia <nxia@chromium.org>
/external/autotest/site_utils/gs_offloader_unittest.py
|
| 97d188c2d78728c69a2d71f08de210fe683d3d1e |
|
25-Jun-2016 |
Michael Tang <ntang@google.com> |
Publish a cloud notification when a job directory is uploaded.
BUG=chromium:623239
TEST=Unittest and manual test.
Change-Id: I2dc645d3dd632b8f11618bdf5ef66ffde8254308
Reviewed-on: https://chromium-review.googlesource.com/358552
Commit-Ready: Michael Tang <ntang@chromium.org>
Tested-by: Michael Tang <ntang@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Reviewed-by: Michael Tang <ntang@chromium.org>
/external/autotest/site_utils/gs_offloader_unittest.py
|
| 2d981eee42e4f9fb4f6d726b97aa8122322543be |
|
07-Jul-2016 |
Ningning Xia <nxia@chromium.org> |
Upload CTS test results to seperate GS buckets.
Parse keyval to get build info and parent_job_id.
Add build info, parent_job_id and job_id to timestamp.zip gs path.
Add job_id to result.xml.gz gs path.
BUG=chromium:619209
TEST=manually run gs_offloader
Change-Id: I937afeaa8ff5f976724bd3e528bc16166aab4026
Reviewed-on: https://chromium-review.googlesource.com/358616
Commit-Ready: Ningning Xia <nxia@chromium.org>
Tested-by: Ningning Xia <nxia@chromium.org>
Reviewed-by: Dan Shi <dshi@google.com>
/external/autotest/site_utils/gs_offloader_unittest.py
|
| 4211124209fdb4469462b6e1b39433cc52b76d02 |
|
15-Jun-2016 |
Ningning Xia <nxia@chromium.org> |
Upload testResult files to cts_result_bucket
Upload testResult.xml.gz and timestamp.zip files to chromeos-cts-results.
BUG=chromium:619209
TEST=unit test; locally run gs_offloader
Change-Id: Id728eef286b8de09a52de69973cae0f7396842e0
Reviewed-on: https://chromium-review.googlesource.com/352815
Commit-Ready: Ningning Xia <nxia@chromium.org>
Tested-by: Ningning Xia <nxia@chromium.org>
Reviewed-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Dan Shi <dshi@google.com>
/external/autotest/site_utils/gs_offloader_unittest.py
|
| 0df2eb4e1e8638a1a165e8f2dc3b9230f85acf06 |
|
14-May-2016 |
Michael Tang <ntang@google.com> |
Add multiprocessing setting for gs_offloader.
If the command line flag "multiprocessing" is set, it turns on the -m option
for gsutil when gs_offloarder runs. If the flag is not set, the global config
setting "gs_offloader_multiprocessing" under the CROS section is used, which
is default to False.
BUG=chromium:590929
TEST=unit test and test on device.
Change-Id: I6580866b7691e1be29977f271dfae8d67e2e06cc
Reviewed-on: https://chromium-review.googlesource.com/344832
Commit-Ready: Michael Tang <ntang@chromium.org>
Tested-by: Michael Tang <ntang@chromium.org>
Reviewed-by: Michael Tang <ntang@chromium.org>
Reviewed-by: Nicolas Boichat <drinkcat@chromium.org>
/external/autotest/site_utils/gs_offloader_unittest.py
|
| 1b4c7c396d5043b24d6bf65c069a85371bb00e5c |
|
05-Oct-2015 |
Dan Shi <dshi@chromium.org> |
[autotest] Allow gs_offloader to limit the number of files in a result folder
If the number of files is too big, try to compress each subfolder.
BUG=chromium:537756
TEST=unittest
Change-Id: I833c2664c27bc566d08a0354ea4f2695dec2ec36
Reviewed-on: https://chromium-review.googlesource.com/304090
Commit-Ready: Dan Shi <dshi@chromium.org>
Tested-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
/external/autotest/site_utils/gs_offloader_unittest.py
|
| 686ae8c339d31988e07bde0ab5d7b3731c0570f1 |
|
09-Sep-2015 |
Kevin Cheng <kevcheng@google.com> |
[autotest] Add in the helper url to gs_offloader eMail.
go/cros-triage-gsoffloader contains more helpful info
with regards to triaging issues with gs_offloader.
BUG=chromium:529494
TEST=Ran gs_offloader unittests
Change-Id: I66890d130b1e16c986bdc20de711de021e89c69d
Reviewed-on: https://chromium-review.googlesource.com/298066
Commit-Ready: Kevin Cheng <kevcheng@google.com>
Tested-by: Kevin Cheng <kevcheng@google.com>
Reviewed-by: Richard Barnette <jrbarnette@chromium.org>
/external/autotest/site_utils/gs_offloader_unittest.py
|
| e93c85778aa8b22aa88482c43092133d59434147 |
|
11-Aug-2015 |
MK Ryu <mkryu@google.com> |
[autotest] Add -m option to gs_offloader
gs_offloader used -m option by default when running gsutil.
Make -m an option of gs_offloader, when it is enabled, then gs_offloader
uses -m option for gsutil. Otherwise, run gsutil without -m.
BUG=chromium:513467
TEST=Run gs_offloader in local machine and see gsutil is run
with/without -m option.
site_utils/gs_offloader_unittest.py
DEPLOY=gs_offloader,gs_offloader_s
Change-Id: If4db154f5c5e9649c032be6236eececec363e7c0
Reviewed-on: https://chromium-review.googlesource.com/292811
Trybot-Ready: Mungyung Ryu <mkryu@google.com>
Tested-by: Mungyung Ryu <mkryu@google.com>
Reviewed-by: Mungyung Ryu <mkryu@google.com>
Commit-Queue: Mungyung Ryu <mkryu@google.com>
/external/autotest/site_utils/gs_offloader_unittest.py
|
| affb922d30a3f5e528103c0314d5f0586b7a90dc |
|
16-Apr-2015 |
Dan Shi <dshi@chromium.org> |
[autotest] Sanitize folder and file names in results dir before uploading
Replace all invalid characters in folder and file names with ASCII value.
BUG=chromium:465612
TEST=unittest
Change-Id: I8111d5bd8bd3eaf2b1bc25e122fb8f46be891431
Reviewed-on: https://chromium-review.googlesource.com/265941
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
Trybot-Ready: Dan Shi <dshi@chromium.org>
Tested-by: Dan Shi <dshi@chromium.org>
Reviewed-by: David James <davidjames@chromium.org>
/external/autotest/site_utils/gs_offloader_unittest.py
|
| 1e10e92049f4b2724ea43186083e995e418f9802 |
|
17-Apr-2015 |
Simran Basi <sbasi@google.com> |
[autotest] Add --offload_failures_only flag for Suites
Adds a new run_suite flag: --offload_failures_only and the required
piping to set it as a job keyval for each subjob of a suite.
Updates the tko_parser to determine if a test was successful or not
and should it be offloaded or not.
Introduces the concept of gs_offloader_instructions json file for
each test. This allows us to improve the intelligence of
gs_offloader by allowing tests to supply information to gs_offloader.
BUG=chromium:471330
TEST=local moblab setup. Ensured that only failed jobs were offloaded.
DEPLOY=gs_offloader,apache
Change-Id: I2310d710f1dce39a70766f6408f6a3c94f493e0d
Reviewed-on: https://chromium-review.googlesource.com/266090
Reviewed-by: Simran Basi <sbasi@chromium.org>
Tested-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Simran Basi <sbasi@chromium.org>
/external/autotest/site_utils/gs_offloader_unittest.py
|
| dd0227d5503b9c0ea63837eec26f87c4e4958e4a |
|
11-Apr-2015 |
J. Richard Barnette <jrbarnette@chromium.org> |
[autotest] Convert gs_offloader away from the AFE.run() method.
In gs_offloader, we make various RPC calls via AFE.run(). However,
the AFE class has perfectly good methods to wrap calls to run(), so
use those instead.
BUG=None
TEST=Unit tests
Change-Id: Ic6f7fa7eb7855618aaff45daf993a55a93b3c96a
Reviewed-on: https://chromium-review.googlesource.com/265257
Commit-Queue: Richard Barnette <jrbarnette@chromium.org>
Tested-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Dan Shi <dshi@chromium.org>
/external/autotest/site_utils/gs_offloader_unittest.py
|
| cf4d2032ea4bf5af680383f36308d581876bbbb0 |
|
12-Mar-2015 |
Dan Shi <dshi@chromium.org> |
[autotest] Update autoserv to support server-side packaging.
Code change is made so autoserv will honer the --require-ssp arg. When the
argument is specified, autoserv will:
1. Try to locate server-side package based on --image, host attribute
job_repo_url and host's cros-version label.
2. If no server-side package can be staged, the job will be run without ssp.
3. If package can be staged, autoserv will start a container to run the test.
a) parent autoserv process's log will be stored in [results]/wrapper folder.
b) A container will be created from base container, with proper setup.
c) The same autoserv command line (with update on result path and control
etc.), will be executed inside the container.
d) Parent autoserv process will wait for above command to finish and destroy
the container afterwards. If the test job was aborted, container should
also be destroyed.
BUG=chromium:453624
TEST=local setup and run test command:
/usr/local/autotest/server/autoserv -p -r /usr/local/autotest/results/16-debug_\
user/$result_dir -m 172.27.215.232 -u udebug_user -l dummy_PassServer -s -P \
16-debug_user/$result_dir -n /usr/local/autotest/results/drone_tmp/attach.2 \
--verify_job_repo_url --require-ssp
create job from afe:
http://dshi.mtv.corp.google.com/afe/#tab_id=view_job&object_id=19
Change-Id: Ida36374dd500cd1fd0b67a86ab8c0198ade4cc36
Reviewed-on: https://chromium-review.googlesource.com/259834
Reviewed-by: Dan Shi <dshi@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
Trybot-Ready: Dan Shi <dshi@chromium.org>
Tested-by: Dan Shi <dshi@chromium.org>
/external/autotest/site_utils/gs_offloader_unittest.py
|
| f3e305f2c1342d8dd271256c2594ae54072c514b |
|
03-Oct-2014 |
Simran Basi <sbasi@chromium.org> |
[gs_offloader] Add gs_offloading_enabled to global_config.
Added a new config variable: gs_offloading_enabled.
This variable sets the default setting for the --delete_only flag.
For MobLab this will allow partners to enable/disable offloading
from the global_config.
BUG=chromium:420232
TEST=unit tests & locally ran with gs_offloading_enabled set
True/False.
Change-Id: I02077cd21da8a789039961200d4e42b44c6d6750
Reviewed-on: https://chromium-review.googlesource.com/221468
Tested-by: Simran Basi <sbasi@chromium.org>
Reviewed-by: Richard Barnette <jrbarnette@chromium.org>
Commit-Queue: Simran Basi <sbasi@chromium.org>
/external/autotest/site_utils/gs_offloader_unittest.py
|
| df4751eaab3405bc4f564d033681d093958f0c10 |
|
10-Oct-2014 |
Simran Basi <sbasi@chromium.org> |
[gs_offloader] Fix test_days_old_option unittest.
The test_days_old_option unit test was misnamed.
BUG=None.
TEST=ran unittests.
Change-Id: I635fdfe61617d594ad8cc2f08f9889341fddeecb
Reviewed-on: https://chromium-review.googlesource.com/222889
Tested-by: Simran Basi <sbasi@chromium.org>
Reviewed-by: Richard Barnette <jrbarnette@chromium.org>
Commit-Queue: Simran Basi <sbasi@chromium.org>
/external/autotest/site_utils/gs_offloader_unittest.py
|
| 24f22c2fda0fac684286b168f642c7326a20fac7 |
|
26-Sep-2014 |
Jakob Juelich <jakobjuelich@chromium.org> |
[autotest] Use rsync instead of cp in gs_offloader.
gsutil cp uses a lot of connections and has a big cpu overhead. Using
rsync instead allows
a) incremental uploads: If an upload failed, it doesn't need to be
re-started from scratch
b) faster uploads, because many files are sent over the same
connection.
BUG=None
DEPLOY=gs_offloader
TEST=Ran suites and tested with local results dir and testing bucket.
Change-Id: I6e920dfccba5b4d2b0b1a19031f1922a0758a3cc
Reviewed-on: https://chromium-review.googlesource.com/220147
Tested-by: Jakob Jülich <jakobjuelich@chromium.org>
Reviewed-by: Richard Barnette <jrbarnette@chromium.org>
Commit-Queue: Jakob Jülich <jakobjuelich@chromium.org>
/external/autotest/site_utils/gs_offloader_unittest.py
|
| dd129979e8227c960f4c7a3d2a1665007f8a424e |
|
11-Sep-2014 |
Simran Basi <sbasi@chromium.org> |
gs_offloader: Add Moblab support to gs_offloader.
Updates the gs_offloader to offload to a subdirectory of the
image_storage_server for Moblab devices. This subdirectory is
determined by the MAC address of the Moblab device and a UUID
stored on the stateful partition.
BUG=chromium:413383
TEST=Manually ran on Moblab, unittests. Ran locally to ensure
non-moblab usecase still works as expected.
Change-Id: Id4dc2a8b03eb6b174cb157e4d72f183dfaa39733
Reviewed-on: https://chromium-review.googlesource.com/217822
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Simran Basi <sbasi@chromium.org>
Tested-by: Simran Basi <sbasi@chromium.org>
/external/autotest/site_utils/gs_offloader_unittest.py
|
| dfea368e5c830b1d7950ced5ee7b191e3b141ca3 |
|
11-Aug-2014 |
Dan Shi <dshi@chromium.org> |
[autotest] Add RPC get_host_history to get host history.
The RPC takes in arguments like board, pool or a list of hosts, and return the
history of specified hosts. The history is a dictionary of each host and its
history of status. For example:
{'172.22.33.51': [{'status': 'Resetting'
'start_time': '2014-08-07 10:02:16',
'end_time': '2014-08-07 10:03:16',
'log_url': 'http://autotest/reset-546546/debug',
'task_id': 19441991},
{'status': 'Running'
'start_time': '2014-08-07 10:03:18',
'end_time': '2014-08-07 10:13:00',
'log_url': 'http://autotest/reset-546546/debug',
'job_id': 14995562}
]
}
Add a client util module time_utils to handle common task to do time string
and epoch time conversion.
BUG=chromium:394451
TEST=local setup
from autotest_lib.client.common_lib import global_config
from autotest_lib.server.cros.dynamic_suite import frontend_wrappers
instance_server = global_config.global_config.get_config_value(
'SERVER', 'hostname', type=str)
afe = frontend_wrappers.RetryingAFE(
server=instance_server, timeout_min=60, delay_sec=0)
result = afe.run('get_host_history', board='lumpy', pool='bvt',
start_time='2014-08-05 14:00:00',
end_time='2014-08-06 14:00:00')
import pprint
pprint.pprint(result)
rebuild local afe, test run_suite and create job from afe.
Change-Id: I66ecc5bb69a4a1fdb33437520288ed08e562ed00
Reviewed-on: https://chromium-review.googlesource.com/211772
Tested-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Prashanth B <beeps@chromium.org>
Commit-Queue: Dan Shi <dshi@chromium.org>
/external/autotest/site_utils/gs_offloader_unittest.py
|
| fb98e46b33c3f683173b980e027523dfb91f2759 |
|
18-Aug-2014 |
Simran Basi <sbasi@chromium.org> |
[autotest]: Update gs_offloader to work on finished times.
Currently gs_offloader offloads jobs based off of when they are
created. This causes issues for long running jobs that run longer
than the offload wait time, causing them to be offloaded or
deleted as soon as they finish.
This change utilizes the finished times of special tasks a job's
host queue entries rather than the started and created times.
If all the host queue entries have null finished_on times, it then
uses the job's created_on time.
BUG=chromium:404275
TEST=unittests, manually edited my local database so that the
started time was less than now + days_old while the finish time was
not and observed it offloaded when it was supposed to.
DEPLOY=gs_offloader
Change-Id: I00186e9854d4e6fe37c28dac65e4c235618a248c
Reviewed-on: https://chromium-review.googlesource.com/212884
Reviewed-by: Fang Deng <fdeng@chromium.org>
Reviewed-by: Richard Barnette <jrbarnette@chromium.org>
Commit-Queue: Simran Basi <sbasi@chromium.org>
Tested-by: Simran Basi <sbasi@chromium.org>
/external/autotest/site_utils/gs_offloader_unittest.py
|
| 22dd7487fba60c457747b478ab0822df3a6e8c64 |
|
23-Jun-2014 |
J. Richard Barnette <jrbarnette@chromium.org> |
Cut back log verbosity in gs_offloader.
Previously, gs_offloader made a point to log the disposition of
every job directory on every offload cycle. That turned out to be
too noisy. This reduces the logging to a handful of summary lines
once per cycle.
BUG=chromium:376069
TEST=run unit tests
Change-Id: I2abe127d8568225ad5b8f1082b15a122297fdb40
Reviewed-on: https://chromium-review.googlesource.com/205237
Commit-Queue: Richard Barnette <jrbarnette@chromium.org>
Tested-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
/external/autotest/site_utils/gs_offloader_unittest.py
|
| 2e443efce8dc34f6db6ab566a4c8a552fd8fff2d |
|
20-May-2014 |
J. Richard Barnette <jrbarnette@chromium.org> |
[autotest] Add unit test coverage for gs_offloader.offload_dir().
This adds unit test coverage, and fixes edge case bugs found by
the new tests.
BUG=None
TEST=run the new unit tests
DEPLOY=gs_offloader
Change-Id: If92b11d0872e79cc53f0ebc448cf9817e6fd450d
Reviewed-on: https://chromium-review.googlesource.com/200640
Tested-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Alex Miller <milleral@chromium.org>
Commit-Queue: Richard Barnette <jrbarnette@chromium.org>
/external/autotest/site_utils/gs_offloader_unittest.py
|
| 9f4be0d66119211b3ece24bb985f1412d1f61322 |
|
20-May-2014 |
J. Richard Barnette <jrbarnette@chromium.org> |
[autotest] Add unit test support for gs_offloader.get_cmd_list().
BUG=None
TEST=run the new unit test
DEPLOY=gs_offloader
Change-Id: I5b03da80088282feb128699858c7b01203f97095
Reviewed-on: https://chromium-review.googlesource.com/200616
Tested-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Alex Miller <milleral@chromium.org>
Commit-Queue: Richard Barnette <jrbarnette@chromium.org>
/external/autotest/site_utils/gs_offloader_unittest.py
|
| 2c72dddb1d05ed43b23ebe847601d7f160a9753a |
|
20-May-2014 |
J. Richard Barnette <jrbarnette@chromium.org> |
[autotest] Refactor special job construction in gs_offloader.
This moves handling for the distinction between source and destination
job directory names to be handled by common code in _JobDirectory.
This also updates the unit tests to explicitly cover the constructors
of both concrete subclasses of _JobDirectory.
BUG=None
TEST=run the unit tests.
DEPLOY=gs_offloader
Change-Id: I35d7e2f78eda3b1cc2227c160856683112026109
Reviewed-on: https://chromium-review.googlesource.com/200629
Tested-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Alex Miller <milleral@chromium.org>
Commit-Queue: Richard Barnette <jrbarnette@chromium.org>
/external/autotest/site_utils/gs_offloader_unittest.py
|
| ef4b47dedcb759b46fd1e415172a688f1448df54 |
|
19-May-2014 |
J. Richard Barnette <jrbarnette@chromium.org> |
[autotest] Add unit test coverage for Offloader constructor.
This adds coverage for gs_offloader.Offloader constructor in
preference to testing gs_offloader.parse_options() directly.
This has the advantage of killing two birds with one stone;
the disadvantage is that bugs in either option parsing or the
constructor can't be distinguished without debugging.
BUG=None
TEST=Run the new unit tests
DEPLOY=gs_offloader
Change-Id: I29a40aa608623958076a79d5571890a6ce08d205
Reviewed-on: https://chromium-review.googlesource.com/200630
Tested-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Alex Miller <milleral@chromium.org>
Commit-Queue: Richard Barnette <jrbarnette@chromium.org>
/external/autotest/site_utils/gs_offloader_unittest.py
|
| 3e3ed6a9627a156a2240d607565e7c6f78151f28 |
|
19-May-2014 |
J. Richard Barnette <jrbarnette@chromium.org> |
[autotest] Fix YASB in gs_offloader.
The constructor said `self._id`. The method said `self.id`. They
had to call the whole thing off.
This calls the calling off off, and adds a unit test to keep the
happy couple together.
BUG=chromium:374585
TEST=run the new unit test, with and without the bug fix
DEPLOY=gs_offloader
Change-Id: Ib1613bf8863722902948884c3ad61ae45125fab7
Reviewed-on: https://chromium-review.googlesource.com/200622
Tested-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Richard Barnette <jrbarnette@chromium.org>
/external/autotest/site_utils/gs_offloader_unittest.py
|
| 0880032b48fbcf11d50f33526f6817a78c2f17bf |
|
16-May-2014 |
J. Richard Barnette <jrbarnette@chromium.org> |
[autotest] Fix gs_offloader and the unit test that didn't catch its bug.
The previous gs_offloader change had some (glaring) unit test
coverage holes, including one that missed that gs_offloader would
fail at startup every time.
BUG=chromium:232058
TEST=run the new unit test with and without the fix
DEPLOY=gs_offloader
Change-Id: Id960a667e0a753fea6a2d4f34cc2fac2e4824a8d
Reviewed-on: https://chromium-review.googlesource.com/200612
Tested-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Richard Barnette <jrbarnette@chromium.org>
/external/autotest/site_utils/gs_offloader_unittest.py
|
| ea785366f46ec4ed7a75d382e152a77aa51069d7 |
|
18-Mar-2014 |
J. Richard Barnette <jrbarnette@chromium.org> |
[autotest] Limit gs_offloader notifications to once per hour.
This change reworks gs_offloader e-mail notifications so that if
multiple directories fail in a single loop through current results,
all the failures will be summarized in a single e-mail message.
Moreover, once notification of a problem has been sent out, no new
notifications will be sent out for at least one hour.
Additionally, this restricts the definition of "offload failure" so
that a directory that fails to offload only 1 time will never
trigger notification at all.
Finally, this change includes new unit tests covering the new
functionality, as well as some pre-existing functionality.
BUG=chromium:232058
TEST=run the shiny, new unit tests
DEPLOY=gs_offloader
Change-Id: I86573cb22a14a0e33cfb83a7bb31174c9abcebd8
Reviewed-on: https://chromium-review.googlesource.com/200611
Tested-by: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Simran Basi <sbasi@chromium.org>
Commit-Queue: Richard Barnette <jrbarnette@chromium.org>
/external/autotest/site_utils/gs_offloader_unittest.py
|
| 5a5e98d02343996bdf32eba6a8774b728527de17 |
|
19-May-2014 |
Prashanth B <beeps@chromium.org> |
Revert "[autotest] Limit gs_offloader notifications to once per hour."
This reverts commit 3b2ed3c0a0172158229105c52c99770b3764aba1.
TEST=Visual
BUG=None
Change-Id: Ieced724e40735939da1b68ea786af6d5edfe89d0
Reviewed-on: https://chromium-review.googlesource.com/200410
Reviewed-by: Prashanth B <beeps@chromium.org>
Commit-Queue: Prashanth B <beeps@chromium.org>
Tested-by: Prashanth B <beeps@chromium.org>
/external/autotest/site_utils/gs_offloader_unittest.py
|
| f6dc32b5ea833dd2fbdbae7c3a856ff2a83a1448 |
|
16-May-2014 |
J. Richard Barnette <jrbarnette@chromium.org> |
Fix gs_offloader and the unit test that didn't catch its bug.
The previous gs_offloader change had some (glaring) unit test
coverage holes, including one that missed that gs_offloader would
fail at startup every time.
BUG=chromium:232058
TEST=run the new unit test with and without the fix
Change-Id: Ic04933069f3fa079b3e9c5f4850ab43526739e70
Reviewed-on: https://chromium-review.googlesource.com/200300
Tested-by: Richard Barnette <jrbarnette@chromium.org>
Commit-Queue: Richard Barnette <jrbarnette@chromium.org>
Reviewed-by: Alex Miller <milleral@chromium.org>
/external/autotest/site_utils/gs_offloader_unittest.py
|
| 3b2ed3c0a0172158229105c52c99770b3764aba1 |
|
18-Mar-2014 |
J. Richard Barnette <jrbarnette@chromium.org> |
[autotest] Limit gs_offloader notifications to once per hour.
This change reworks gs_offloader e-mail notifications so that if
multiple directories fail in a single loop through current results,
all the failures will be summarized in a single e-mail message.
Moreover, once notification of a problem has been sent out, no new
notifications will be sent out for at least one hour.
Additionally, this restricts the definition of "offload failure" so
that a directory that fails to offload only 1 time will never
trigger notification at all.
Finally, this change includes new unit tests covering the new
functionality, as well as some pre-existing functionality.
BUG=chromium:232058
TEST=run the shiny, new unit tests
DEPLOY=gs_offloader
Change-Id: I27454de08ab75d822db10874a659627adc395aa8
Reviewed-on: https://chromium-review.googlesource.com/190440
Reviewed-by: Richard Barnette <jrbarnette@chromium.org>
Commit-Queue: Richard Barnette <jrbarnette@chromium.org>
Tested-by: Richard Barnette <jrbarnette@chromium.org>
/external/autotest/site_utils/gs_offloader_unittest.py
|