| 4b133961b76c2cb8bc58f0ea2cded9e3438ffb6f |
|
02-Nov-2016 |
Manoj Gupta <manojgupta@google.com> |
Download autotest files with crosperf and add autotest path to test_that
when running non-telemetry tests
BUG: chromium:647429
TEST:crosperf unit tests, sample experiment file that runs WebGl_Aquarium
Change-Id: I067e350fee36596ce269b954773c39a3868632aa
Reviewed-on: https://chrome-internal-review.googlesource.com/302296
Commit-Ready: Manoj Gupta <manojgupta@google.com>
Tested-by: Manoj Gupta <manojgupta@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/settings_factory.py
|
| c39917fe61858e884d06656122cf88a8c66fd825 |
|
01-Nov-2016 |
Manoj Gupta <manojgupta@google.com> |
Update formatting of files before committing changes for autotest. No functional change.
BUG: chromium:647429
TEST: crosperf unit tests
Change-Id: I827f9ec4d00a0d8c1ae8a9930d7832289e694dc8
Reviewed-on: https://chrome-internal-review.googlesource.com/301457
Commit-Ready: Manoj Gupta <manojgupta@google.com>
Tested-by: Manoj Gupta <manojgupta@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
/external/toolchain-utils/crosperf/settings_factory.py
|
| 9099a788cd7124024559c064e425ed9caef6e0ac |
|
23-Jul-2016 |
Caroline Tice <cmtice@google.com> |
[crosperf] Clean up exceptions.
Found all instances of 'raise Exception' and changed them to raise
something slightly more specific. Changed a few 'except Exception'
statements appropriately as well.
BUG=chromium:566255
TEST=None
Change-Id: If2666bd55838342bc71431e5e6ab2157c2c9e125
Reviewed-on: https://chrome-internal-review.googlesource.com/270731
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
/external/toolchain-utils/crosperf/settings_factory.py
|
| 7edbfa2e0ef52487ec1286886dcbc7d401528733 |
|
28-Jun-2016 |
George Burgess IV <gbiv@google.com> |
[crosperf] Fix setting descriptions; NFC.
This also removes an unused import, and adds a few docstrings/etc. to
make `cros lint` happy.
BUG=None
TEST=./run_tests.sh
Change-Id: I0279a0db1d1cb1a1bc655a06bfc6b33d87495610
Reviewed-on: https://chrome-internal-review.googlesource.com/267277
Commit-Ready: George Burgess <gbiv@google.com>
Tested-by: George Burgess <gbiv@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/settings_factory.py
|
| f2a3ef46f75d2196a93d3ed27f4d1fcf22b54fbe |
|
15-Dec-2015 |
Luis Lozano <llozano@chromium.org> |
Run pyformat on all the toolchain-utils files.
This gets rid of a lot of lint issues.
Ran by doing this:
for f in *.py; do echo -n "$f " ; if [ -x $f ]; then pyformat -i
--remove_trailing_comma --yapf --force_quote_type=double $f ; else
pyformat -i --remove_shebang --remove_trailing_comma --yapf
--force_quote_type=double $f ; fi ; done
BUG=chromium:567921
TEST=Ran simple crosperf run.
Change-Id: I59778835fdaa5f706d2e1765924389f9e97433d1
Reviewed-on: https://chrome-internal-review.googlesource.com/242031
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
/external/toolchain-utils/crosperf/settings_factory.py
|
| 585192fb87a05d0b2b24cc01d9a05b55c6b5849c |
|
19-Nov-2015 |
Ting-Yuan Huang <laszio@google.com> |
Make 'run_local=True' default in crosperf.
BUG=None
TEST=tested with run_local={,True,False}
Change-Id: Id23c02b4111fafe8c8c392d39b15521045a78550
Reviewed-on: https://chrome-internal-review.googlesource.com/238861
Commit-Ready: Ting-Yuan Huang <laszio@google.com>
Tested-by: Ting-Yuan Huang <laszio@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
/external/toolchain-utils/crosperf/settings_factory.py
|
| ddde50532281f7f796dd7dc44b562b29d25ab381 |
|
23-Sep-2015 |
Caroline Tice <cmtice@google.com> |
Add compiler info to experiment data.
Check the build label to see which compiler was used. Store the
compiler in the label, and add it to the json filename
and output.
Also update nightly buildbot script to add 'compiler' field to
labels.
BUG=None
TEST=Tested in my directory.
Change-Id: I124c4910cce1145708bb2a52e3a6d6dff93777c3
Reviewed-on: https://chrome-internal-review.googlesource.com/231811
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/settings_factory.py
|
| ef4ca8a8cfe05ec09447896db42fb220d07834f8 |
|
25-Aug-2015 |
Caroline Tice <cmtice@google.com> |
Add json report archiving.
This CL adds a new output, in json format, for archiving test
results. It also adds a new flag, --json_report, to crosperf to
tell it to generate the json output file in the usual results
directory. The json output is always in addition to the other
reports that Crosperf generates. This CL also make some minor
changes to the tool that maintains the telemetry default results
file, so it can be used by the json report generator.
BUG=None
TEST=Tested Crosperf with & without new flag; it did what it should.
Change-Id: Id0e476716ceee208d33e6f9427ea95235fbf11d4
Reviewed-on: https://chrome-internal-review.googlesource.com/228325
Reviewed-by: Han Shen <shenhan@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/settings_factory.py
|
| bc2d3d13d2e1928217140a76acdf9f9917b1fa30 |
|
02-Jul-2015 |
Ting-Yuan Huang <laszio@google.com> |
Add 'run_local' to run the test harness locally.
The option 'run_local: True|False' will be passed into autotest.
For example, this is how test_that will be invoked:
test_that --arges="run_local=True ..." ...
BUG=None
TEST=Tested with the folllowing combinations.
benchmark: sunspider, smoothness.top_25_smooth
perf_args: record -e cycles,instructions / (none)
Change-Id: I7315027a7c9433d17a9f3fe54d7e8c3f480ea4f4
Reviewed-on: https://chrome-internal-review.googlesource.com/217370
Reviewed-by: Caroline Tice <cmtice@google.com>
Tested-by: Ting-Yuan Huang <laszio@google.com>
Commit-Queue: Ting-Yuan Huang <laszio@google.com>
/external/toolchain-utils/crosperf/settings_factory.py
|
| 73bf2e9cc58d8429dbb7636e996062e7605a7222 |
|
20-Jun-2015 |
Han Shen <shenhan@google.com> |
Remove "import lock_machine".
We do not have this file any longer, remove it. Rahul and I are having
problem running crosperf because of this.
TEST=run crosperf successfully
BUG=None
Change-Id: I396d30e6bb2362db99f9b16095ef862a1ab417f5
Reviewed-on: https://chrome-internal-review.googlesource.com/219770
Reviewed-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Han Shen <shenhan@google.com>
Tested-by: Han Shen <shenhan@google.com>
/external/toolchain-utils/crosperf/settings_factory.py
|
| 517dc98884e398fa863ce22b6bc1ef8323853416 |
|
12-Jun-2015 |
cmtice <cmtice@google.com> |
Add flag to crosperf to use file lock mechanism.
This CL adds a flag to crosperf that specifies using the old file
locking mechanism, rather than the new AFE server locking mechanism.
This is useful when there are issues with accessing the AFE server.
BUG=None
TEST=Tested crosperf with this, both using the AFE locking mechanism
and using the file locking mechanism.
Change-Id: If6aa3e97a77d8ccb53f0e8f0405cc7f45e983b75
Reviewed-on: https://chrome-internal-review.googlesource.com/219096
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/settings_factory.py
|
| e5bc63bbed4e001b080c4ce0b18c5c78900d4786 |
|
28-May-2015 |
cmtice <cmtice@google.com> |
Implement new global locking scheme for machines.
This CL implements a new machine locking mechanism using
Autotest Front End servers. When locking/unlocking a lab machine,
it uses the ChromeOS HW lab server; when locking/unlocking a local
machine, it uses a local AFE server on chrotomation2.
BUG=None
TEST=Tested the script manually to lock/unlock machines and query status.
Also tested with image_chromeos and with crosperf scripts.
Change-Id: I2793bc1f7dc056e725694e81ded656d9f49d227b
Reviewed-on: https://chrome-internal-review.googlesource.com/217560
Reviewed-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: David Sharp <dhsharp@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/settings_factory.py
|
| df76222bf1cbdbc42bb41934fb960bac8694eca4 |
|
19-May-2015 |
Luis Lozano <llozano@chromium.org> |
Added benchmark retry support.
We often get spurious failures from the benchmarks runs. This CL adds
support for a "retries" field that will automatically retry the benchmark
run by the amount specified.
BUG=None
TEST=Test by hand.
Change-Id: Ib881c40d766e67a7daf855ce9137d3e3d631f12e
Reviewed-on: https://chrome-internal-review.googlesource.com/217525
Reviewed-by: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Luis Lozano <llozano@chromium.org>
/external/toolchain-utils/crosperf/settings_factory.py
|
| 5c09fc2966ac49263ce7154c2905f2a86aeda297 |
|
22-Apr-2015 |
cmtice <cmtice@google.com> |
Fix current problem with machine locks.
Currently if crosperf can't access the default locks directory, it
assumes that it is safe to proceed without locks and it uses machines
without checking locks. This script does two things: It causes
crosperf to fail if it can't access a locks directory; and it allows
the user to specify an alternate locks directory. As a by-product it
also adds a boolean prompting function to utils/misc.py
BUG=None
TEST=Tested with default directory accessible & inaccessible; tested using
local locks directory.
Change-Id: I6f24f772986813bf089f5f4e40335a15e8ee4837
Reviewed-on: https://chrome-internal-review.googlesource.com/214129
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/settings_factory.py
|
| 1a224369afcbfd0276f4c7bdc625dec7f7b30d01 |
|
17-Oct-2014 |
cmtice <cmtice@google.com> |
Fix issue where crosperf can't find local cros_scratch.
Crosperf was assuming the user always has an NFS home directory,
and was looking for the cros_scratch directory there. This makes
crosperf use the user's home directory, whether it's NFS or local.
Also change --share_users, which also assumed NFS home directories,
to --share_cache, which requires specification of the cache path(s)
to use.
BUG=None
TEST=Tested this with my new local homedir.
Change-Id: I28d15467dfbd25b8b237cefa2979d3fd9464a7a7
Reviewed-on: https://chrome-internal-review.googlesource.com/179978
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/settings_factory.py
|
| 2a370f7af34f6b126e80ef55891b9bff6a2858bb |
|
30-May-2014 |
cmtice <cmtice@google.com> |
Add --cache_only global setting to crosperf.
The CL adds a new setting, --cache_only, which causes crosperf to
ONLY report test results that it finds in the cache (it will NOT
rerun any tests, even if no cache results are found). This is partly
for implementing a weekly summary report, where we want to compare the
nightly results from the past week, but not actually rerun anything.
BUG=None
TEST=I tested this option while generating weekly test reports, and it
worked as it is supposed to.
Change-Id: Icb1f5343f76ce999bebd0810bb86ab64a7577313
Reviewed-on: https://chrome-internal-review.googlesource.com/164890
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/settings_factory.py
|
| 42709763f709b0e08d4ed041e4374cb9dd5d6570 |
|
16-May-2014 |
cmtice <cmtice@google.com> |
Add unit test for experiment_runner.py
BUG=None
TEST=Ran unittest.
Change-Id: I13b2822f57f4201c94215c603a60f12755230f71
Reviewed-on: https://chrome-internal-review.googlesource.com/163833
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/settings_factory.py
|
| 98a53692fb946a8eac46e3e82257f540d1350c18 |
|
16-Apr-2014 |
cmtice <cmtice@google.com> |
Clean up Crosperf options and help.
Crosperf's help was completely out of date and incorrect. Fixed
that. Also, I cleaned up the options, removed some that were no
longer appropriate or not ever used, and made some options global-only.
Also removed all references to pyauto. We no longer use or run pyauto
tests.
Option changes: Completely removed outlier_range (it was not actually doing
anything), key_results_only (applied only to pyauto tests), use_test_that
(not needed now that we're not using pyauto) and md5sum (never used).
Made the following options global-only: rm_chroot_tmp, perf_args, logging_level,
show_all_results, board. Removed 'build' from global settings and made
it an image-only setting (it made no sense as a global setting).
I also adjusted the unittests to work with these changes.
BUG=None
TEST=I tested all the options in all their various positions; I tested with and
wtihout caching; and I tested local and official images. I also ran the unittests. Everything works.
Change-Id: I7653663f65c0a3363a1904c0220f1066e72c7f9d
Reviewed-on: https://chrome-internal-review.googlesource.com/160758
Reviewed-by: Bhaskar Janakiraman <bjanakiraman@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/settings_factory.py
|
| 0537956ef5981f4069fcaaefd3d25298f9d6ebd8 |
|
07-Apr-2014 |
cmtice <cmtice@google.com> |
Always write verbose logs to files.
The option that reduces the amount of screen output (--logging_level
= 'quiet' or 'average') was also reducing what got written to the log
files. This fixes that. With this CL, even though the amount of screen
output is reduced, what gets written to the log files is always the same
(verbose). This CL also makes "average" the default logging level.
BUG=None
TEST=I tested all three logging levels, saving the results in different
log directories and compared the log files afterwards, to make sure
the log files contained the same amount of logging, regardless of the
screen logging level.
Change-Id: I6982499d67c080afa835e01ab4fa2282dd992052
Reviewed-on: https://chrome-internal-review.googlesource.com/159654
Reviewed-by: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/settings_factory.py
|
| 139092449a545d4d87c06af3a6d8f3d71e42e927 |
|
11-Mar-2014 |
cmtice <cmtice@google.com> |
Add --logging_level option to crosperf.
Add --logging_level={verbose, average, quiet} option to crosperf,
optionally reducing the amount of logging output. "verbose" gives
the same output Crosperf always gave; "quiet" gives minimal output, and
"average" is in the middle. The default for now is "verbose". Doing this
also involved adding logging levels to the command_executer, and to
image_chromeos (again, with the default being "verbose").
BUG=None
TEST=Tested all three levels of logging, with and without having to
reimage the DUT. It all worked properly.
Change-Id: Icf8c9a6831fe25202adcb624c6c7e1d1a7ac25a5
Reviewed-on: https://chrome-internal-review.googlesource.com/156883
Reviewed-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/settings_factory.py
|
| 0cc4e7790afbd514675801a1ffb90517c147270f |
|
31-Jan-2014 |
cmtice <cmtice@google.com> |
Add 'build' settings option and xbuddy image format.
This patch updates Crosperf to add the 'build' tag in the
experiment file, and to allow that field to contain xbuddy
syntax for using trybot and/or official builds in the test runs.
It also adds a bit more checking to make sure we have everything
necessary for running 'cros flash' before attempting to use it.
BUG=None
TEST=I have run this using an experiment file that compares a local
image, a trybot image and an official image against each other. It
all worked.
Change-Id: Ia896799061508fb5718a3201b1599d8622de0b3f
Reviewed-on: https://chrome-internal-review.googlesource.com/154097
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/settings_factory.py
|
| 384d5f8f3eeb04d5063607c691b3732f0489934f |
|
19-Dec-2013 |
cmtice <cmtice@google.com> |
Change to use test_that by default instead of run_remote_tests.sh.
BUG=None
TEST=Tested this with BootPerServer in my directory. Checked the
logs to make sure the correct script was being used (test_that by
default now).
Change-Id: I39b2901c2ef3c8070c76b15b929df5da6b168c9c
Reviewed-on: https://chrome-internal-review.googlesource.com/150695
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/settings_factory.py
|
| 9847df92a2b5f76ccddc4bf10288819712a8ca47 |
|
07-Dec-2013 |
cmtice <cmtice@google.com> |
Use default results for Telemetry tests; add option to use all results.
When running telemetry_Crosperf, use the json file of report defaults to
determine which results to actually include in the report. This CL
also adds a flag to Crospref, --show_all_results=True, that allows the
user to override the defaults and generate a report showing all the values.
BUG=None
TEST=Tested this both with and without the --show_all_results flag. It
works as expected for me.
Change-Id: I91364572ebdf3d54bb8b408fbd5a2580923f6829
Reviewed-on: https://chrome-internal-review.googlesource.com/149457
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/settings_factory.py
|
| b47bff4d3336c5fe5593a95963c0f3dc20a02f68 |
|
20-Aug-2013 |
Caroline Tice <cmtice@chromium.org> |
Make test_that available for running tests.
Add a new flag, --use_test_that=True/False, to crosperf, to allow users to
tell crosperf to use the test_that script rather than the run_remote_tests.sh
script for running their tests.
BUG=None
TEST=Tested with and without the flag, to verify that it does the right thing.
Change-Id: I1a040d23adbf364b4a47320d9fd759a2d46e6689
Reviewed-on: https://gerrit-int.chromium.org/43109
Reviewed-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/settings_factory.py
|
| 04dc5dc8547dbfbe524cf35ac39537346ad749bb |
|
24-Apr-2013 |
Yunlian Jiang <yunlian@google.com> |
Adding support of telemetry to crosperf
BUG=None
TEST=run crosperf with pyauto/telemetry test with/without cache.
all pass.
Change-Id: If07ac020a9107a79d5780a58fd6dcc924d07f07f
Reviewed-on: https://gerrit-int.chromium.org/36594
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-on: https://gerrit-int.chromium.org/39241
Reviewed-by: Yunlian Jiang <yunlian@google.com>
/external/toolchain-utils/crosperf/settings_factory.py
|
| 00cc30ea5fb3a05530a3b1ec699ca7f9696ffbd0 |
|
28-Mar-2013 |
Yunlian Jiang <yunlian@google.com> |
Add option to specify output dir of crosperf
BUG=None
TEST=run crosperf with this option, the results are in the right dir
Change-Id: Id94251c72297f18f275564a1fd1ef0972227ddfc
Reviewed-on: https://gerrit-int.chromium.org/34621
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
/external/toolchain-utils/crosperf/settings_factory.py
|
| f81680c018729fd4499e1e200d04b48c4b90127c |
|
15-Mar-2013 |
Luis Lozano <llozano@chromium.org> |
Cleaned up directory after copy of tools from perforce directory
Got rid of stale copies of some tools like "crosperf" and moved all
files under v14 directory (that came from perforce) into the top
directory.
BUG=None
TEST=None
Change-Id: I408d17a36ceb00e74db71403d2351fd466a14f8e
Reviewed-on: https://gerrit-int.chromium.org/33887
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Luis Lozano <llozano@chromium.org>
/external/toolchain-utils/crosperf/settings_factory.py
|
| 4467f004e7f0854963bec90daff1879fbd9d2fec |
|
20-Dec-2012 |
Ahmad Sharif <asharif@chromium.org> |
Synced repos to: 64740
/external/toolchain-utils/crosperf/settings_factory.py
|
| f395c26437cbdabc2960447fba89b226f4409e82 |
|
10-Oct-2012 |
Ahmad Sharif <asharif@chromium.org> |
Synced repos to: 63271
/external/toolchain-utils/crosperf/settings_factory.py
|
| 5ae8a5c99e73be7f388f74551d5219cb676f8e41 |
|
18-May-2012 |
Ahmad Sharif <asharif@chromium.org> |
Synced repos to: 60588
/external/toolchain-utils/crosperf/settings_factory.py
|
| ee988c87971c07b7e6767adb63d151ea37e363f2 |
|
15-May-2012 |
Ahmad Sharif <asharif@chromium.org> |
Synced repos to: 60457
/external/toolchain-utils/crosperf/settings_factory.py
|
| 822c55d2adc5f6fecb6bb4bc6faa94e022b8012a |
|
09-Feb-2012 |
Ahmad Sharif <asharif@chromium.org> |
Synced repos to: 58313
/external/toolchain-utils/crosperf/settings_factory.py
|
| 0dcbc4b1714260820fd4b8d6536fbb05e139cc0f |
|
03-Feb-2012 |
Ahmad Sharif <asharif@chromium.org> |
Synced repos to: 58208
/external/toolchain-utils/crosperf/settings_factory.py
|