| 8316e3e6a41936eb14a609bb2b04987f13aedb36 |
|
10-Jan-2017 |
Ting-Yuan Huang <laszio@chromium.org> |
crosperf: disable cheets benchmarks in crosbolt
They have nothing to do with cros toolchain.
TEST=none
BUG=none
Change-Id: Ie06ccd1cddf7233c55b52fd77e1329311fcff9d9
Reviewed-on: https://chromium-review.googlesource.com/426379
Trybot-Ready: Ting-Yuan Huang <laszio@chromium.org>
Reviewed-by: Yunlian Jiang <yunlian@chromium.org>
Commit-Queue: Ting-Yuan Huang <laszio@chromium.org>
Tested-by: Ting-Yuan Huang <laszio@chromium.org>
/external/toolchain-utils/crosperf/experiment_factory.py
|
| 7a6bb17630ea86f12c1045c2c47339b02e79bdc2 |
|
09-Dec-2016 |
Ting-Yuan Huang <laszio@google.com> |
crosperf: Add all_graphics_perf and all_crosbolt_perf
BUG=none
TEST=Run them on samus.
Change-Id: I2d66351d80ce87f84924fe3bf55abf903fc54829
Reviewed-on: https://chrome-internal-review.googlesource.com/311196
Commit-Ready: Ting-Yuan Huang <laszio@google.com>
Tested-by: Ting-Yuan Huang <laszio@google.com>
Reviewed-by: Ting-Yuan Huang <laszio@google.com>
/external/toolchain-utils/crosperf/experiment_factory.py
|
| 4b133961b76c2cb8bc58f0ea2cded9e3438ffb6f |
|
02-Nov-2016 |
Manoj Gupta <manojgupta@google.com> |
Download autotest files with crosperf and add autotest path to test_that
when running non-telemetry tests
BUG: chromium:647429
TEST:crosperf unit tests, sample experiment file that runs WebGl_Aquarium
Change-Id: I067e350fee36596ce269b954773c39a3868632aa
Reviewed-on: https://chrome-internal-review.googlesource.com/302296
Commit-Ready: Manoj Gupta <manojgupta@google.com>
Tested-by: Manoj Gupta <manojgupta@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/experiment_factory.py
|
| e82513b0aec27bf5d3ca51789edc48dde5ee439b |
|
27-Oct-2016 |
Caroline Tice <cmtice@google.com> |
[toolchain-utils] Update page_cycler in nightly tests.
Currently page_cycler_v2.typical_25 tries to run too many times,
causing some of the nightly builders to take nearly 24 hours to
complete (they take about 6 hours without page_cycler). Also,
page_cycler_v2.typical_25 cannot run locally on some boards (such
as daisy).
This CL removes page_cycler_v2.typical_25 from the all_toolchain_perf
set, and adds it separately to the nightly tests, with only 2
iterations, no retries, and forcing it to not run locallys on the DUT.
BUG=chromium:660087
TEST=Tested the new page cycler settings in a crosperf run.
Change-Id: Ia68abce4e8ee29a671e58db13f269e531d31736e
Reviewed-on: https://chrome-internal-review.googlesource.com/300855
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
/external/toolchain-utils/crosperf/experiment_factory.py
|
| 33b6586b464e590d85781f63bd6bca9518deeaf4 |
|
26-Sep-2016 |
Caroline Tice <cmtice@google.com> |
[crosperf] Add new results fields for other page_cycler_v2 tests.
This Cl adds the default result fields for the other page_cycler_v2
tests (already done for page_cycler_v2.typical_25). It also temporarily
removes some tests from the all_pagecyclers set, as they appear to
have been removed from telemetry. Will remove these permanently when
this is confirmed.
BUG=chromium:650307
TEST=Ran crosperf with all_pagecyclers.
Change-Id: I9e6c3ae140f66a11d792350b74c8897df74d944c
Reviewed-on: https://chrome-internal-review.googlesource.com/289865
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
/external/toolchain-utils/crosperf/experiment_factory.py
|
| 479a91d1a637ec4e355699c1726806f01ac09cdd |
|
29-Aug-2016 |
George Burgess IV <gbiv@google.com> |
crosperf: Fix a failing test.
An earlier commit added extra spacing between remotes in
crosperf/default_remotes. This resulted in
experiment_factory_unittests.py reporting that there were 5 remotes in
some cases (2 of which were ''), since we were splitting on one space.
It seems sensible to split on any amount of whitespace, instead.
BUG=None
TEST=./run_tests.sh passes.
Change-Id: I95bd85ba92e5e1f5b30b0273b1b4a6c8853b1995
Reviewed-on: https://chrome-internal-review.googlesource.com/281538
Commit-Ready: George Burgess <gbiv@google.com>
Tested-by: George Burgess <gbiv@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/experiment_factory.py
|
| babb65232cbb7de2365d117f3593cdea8a263f05 |
|
27-Jul-2016 |
Caroline Tice <cmtice@google.com> |
[crosperf] Change 'page_cycler.' to 'page_cycler_v2.' everywhere.
Telemetry is switching to using page_cyler_v2 and will stop supporting
old page_cycler soon. This CL switches everything in toolchain-utils
to reference page_cycler_v2.
BUG=None
TEST=Ran unittests in crosperf with no issues.
Change-Id: I0e3ce29bfaf483507a9ba528c6c7c5a8fa00f00e
Reviewed-on: https://chrome-internal-review.googlesource.com/271598
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/experiment_factory.py
|
| 969432132589753e9ef630c38c83d8f9ae2eccde |
|
18-Feb-2016 |
Caroline Tice <cmtice@google.com> |
[crosperf] Remove media.tough_video_cases from nightly tests.
media.tough_video_cases does not have any good result field to
use and it has too high a stddev/mean to be a reliable test.
This CL removes it from our nightly test set.
BUG=None
TEST=None
Change-Id: Id80a9dd11de36a1fed01d7735f32e98908c7e1b0
Reviewed-on: https://chrome-internal-review.googlesource.com/248615
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Luis Lozano <llozano@chromium.org>
/external/toolchain-utils/crosperf/experiment_factory.py
|
| 88272d479f2761cc1906fea564c73033f77a6270 |
|
13-Jan-2016 |
Caroline Tice <cmtice@google.com> |
Fix cros lint errors.
Also move deprecated scripts to the 'deprecated' directory.
BUG=chromiumos:570464
TEST=tested scripts to make sure they still work.
Change-Id: I3442a86d898104591233a0849ea0bafb52ecf1f7
Reviewed-on: https://chrome-internal-review.googlesource.com/244221
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
/external/toolchain-utils/crosperf/experiment_factory.py
|
| f2a3ef46f75d2196a93d3ed27f4d1fcf22b54fbe |
|
15-Dec-2015 |
Luis Lozano <llozano@chromium.org> |
Run pyformat on all the toolchain-utils files.
This gets rid of a lot of lint issues.
Ran by doing this:
for f in *.py; do echo -n "$f " ; if [ -x $f ]; then pyformat -i
--remove_trailing_comma --yapf --force_quote_type=double $f ; else
pyformat -i --remove_shebang --remove_trailing_comma --yapf
--force_quote_type=double $f ; fi ; done
BUG=chromium:567921
TEST=Ran simple crosperf run.
Change-Id: I59778835fdaa5f706d2e1765924389f9e97433d1
Reviewed-on: https://chrome-internal-review.googlesource.com/242031
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
/external/toolchain-utils/crosperf/experiment_factory.py
|
| 6a7dfb33f661a0e0193716a3fcc46a6984486401 |
|
15-Dec-2015 |
Caroline Tice <cmtice@google.com> |
Fix issues caused by 'None' being default for List fields.
BUG=None
TEST=Tested in my directory.
Change-Id: Ib355808db1923b51455f0a2c1d83bc9139030c5d
Reviewed-on: https://chrome-internal-review.googlesource.com/242215
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/experiment_factory.py
|
| e627fd61c2edba668eb2af8221892286b13f05a3 |
|
11-Dec-2015 |
Caroline Tice <cmtice@google.com> |
Crosperf: Finish fixing unittest problems.
This patch finishes fixing the unittest problems. They now all
work.
BUG=chromium:538397
TEST=./run_tests.sh now completes with no errors.
Change-Id: Ie501437d199bfafc8a84cd2a95862550992c2cb8
Reviewed-on: https://chrome-internal-review.googlesource.com/241925
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/experiment_factory.py
|
| 6e8726de19436e28c4d22c3f3dc3bbae87dd4c12 |
|
09-Dec-2015 |
Caroline Tice <cmtice@google.com> |
Fix various Mock classes & calls so dry runs work.
The --dry_run option has been failing because parameter changes in
real classes have not always been reflected in Mock classes. This CL
fixes that.
BUG=chromium:545211
TEST=tested both real and --dry_run runs of Crosperf.
Change-Id: I0d0aa144e9fb3c40e1a95314691eacffb007cbe4
Reviewed-on: https://chrome-internal-review.googlesource.com/241525
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
/external/toolchain-utils/crosperf/experiment_factory.py
|
| 41ea8ed6b09b566f1db551ca4ffb3e27c8e3079f |
|
09-Dec-2015 |
Yunlian Jiang <yunlian@google.com> |
crosperf: fix lint for expeirment_factory.py
BUG=chromium:567921
TEST=the file passes lint checking.
Change-Id: I6d2187d11a51f4e87b5bbd8b905e285bb4ed2475
Reviewed-on: https://chrome-internal-review.googlesource.com/241465
Commit-Ready: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/experiment_factory.py
|
| dd4176150697727cdc89574698fd53f556c1304d |
|
08-Dec-2015 |
Luis Lozano <llozano@chromium.org> |
Print friendlier error message when no benchmark is given.
Catch no-benchmarks and no-labels errors earlier to avoid
weird error messages later.
BUG=chromium:432616
TEST=tested with different experiment files.
Change-Id: I8a05dbac4a47a0f005f0b3c175770a0d88cb4a62
Reviewed-on: https://chrome-internal-review.googlesource.com/241096
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
/external/toolchain-utils/crosperf/experiment_factory.py
|
| 1489d6440dc37585f3bb9325f3d12fbaa5a37b2b |
|
08-Dec-2015 |
Luis Lozano <llozano@chromium.org> |
Re-enable graphics_WebGLAquarium from buildbots testing.
With some recent changes to the test_that invocation this should work
now.
Still there is an issue with benchmarks that require compilation
for the particular board. That remains to be fixed.
BUG=chromium:551710
TEST=None. This is a revert.
Change-Id: I547a4853302f11dc58ddc493ee5368b1e2d33c0e
Reviewed-on: https://chrome-internal-review.googlesource.com/241035
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/experiment_factory.py
|
| 3a7ada298dfaa66575ef72b9ed3ee21c9b784653 |
|
07-Dec-2015 |
Yunlian Jiang <yunlian@google.com> |
crosperf: fix a typo.
BUG=None
TEST=the typo is fixed.
Change-Id: I11249c57e9483a8d1c1a2ca72853b6870949364d
Reviewed-on: https://chrome-internal-review.googlesource.com/240878
Commit-Ready: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Han Shen <shenhan@google.com>
/external/toolchain-utils/crosperf/experiment_factory.py
|
| 96da9060697ad889a82ceb7b2b0fbce8f6572c72 |
|
04-Dec-2015 |
Caroline Tice <cmtice@google.com> |
Fix parameters passed to non-telemetry autotest benchmarks.
The parameters being passed to the non-telemetry Benchmarks were
not quite right. This CL fixes that.
BUG=chromium:567333
TEST=Tested in my chroot with bvt tests.
Change-Id: Ied834e0904eec7801118d4996045d96c1fb4d329
Reviewed-on: https://chrome-internal-review.googlesource.com/240576
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/experiment_factory.py
|
| 7f20acb9c9036015a665a14f27633c639e9b9d91 |
|
05-Nov-2015 |
Luis Lozano <llozano@chromium.org> |
Temporarily remove graphics_WebGLAquarium from buildbot testing.
For the images that we build in trybots, this test cannot be
executed because we are not downloading the necessary artifacts
for autotest. See chromium:551710.
This change temporarily removes that test from the nightly testing that
we do based on trybot images. These changes should be removed
once this bug is fixed.
BUG=chromium:551710
TEST=None
Change-Id: I15ec9386d52809492345d9cd46a6f0ad39330dbc
Reviewed-on: https://chrome-internal-review.googlesource.com/237838
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/experiment_factory.py
|
| 1efd02bef790dea5f60419a7e333e020f65a46ac |
|
12-Oct-2015 |
Yunlian Jiang <yunlian@google.com> |
crosperf: remove dhtml from pagecyler test.
Telemetry removed dhtml, so crosperf should remove it accordingly.
BUG=none
TEST=the benchmark is removed.
Change-Id: I0bcab62e9e726d87a31febcc80121086a87337fa
Reviewed-on: https://chrome-internal-review.googlesource.com/234195
Commit-Ready: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
/external/toolchain-utils/crosperf/experiment_factory.py
|
| ddde50532281f7f796dd7dc44b562b29d25ab381 |
|
23-Sep-2015 |
Caroline Tice <cmtice@google.com> |
Add compiler info to experiment data.
Check the build label to see which compiler was used. Store the
compiler in the label, and add it to the json filename
and output.
Also update nightly buildbot script to add 'compiler' field to
labels.
BUG=None
TEST=Tested in my directory.
Change-Id: I124c4910cce1145708bb2a52e3a6d6dff93777c3
Reviewed-on: https://chrome-internal-review.googlesource.com/231811
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/experiment_factory.py
|
| ba64928c5dcbacbc70b4358881a89ad96227164d |
|
06-Aug-2015 |
Han Shen <shenhan@google.com> |
Crosperf schedv2 (1) - new option and integrating new scheduler.
This Cl introduces a new option '--schedv2' which uses the new
scheduler to allocate jobs (benchmark_runs) to duts.
With this option, schedv2 takes control of storing/allocating jobs and
reimage machines using the new algorithm. This CL leaves actual
reimaging and running jobs non-op (a random time sleep is used for
each such op, which would be replace in later CLs.)
You may try this CL like this and see how schedv2 works -
crosperf --locks_dir=/usr/local/google/home/shenhan/tmp --use_file_locks=True --logging_level=verbose --schedv2 some.exp
Change-Id: If5bb7751b466c39e54c93fe8f0b4e363be6d9165
Reviewed-on: https://chrome-internal-review.googlesource.com/225515
Commit-Queue: Han Shen <shenhan@google.com>
Tested-by: Han Shen <shenhan@google.com>
Reviewed-by: Han Shen <shenhan@google.com>
/external/toolchain-utils/crosperf/experiment_factory.py
|
| bc2d3d13d2e1928217140a76acdf9f9917b1fa30 |
|
02-Jul-2015 |
Ting-Yuan Huang <laszio@google.com> |
Add 'run_local' to run the test harness locally.
The option 'run_local: True|False' will be passed into autotest.
For example, this is how test_that will be invoked:
test_that --arges="run_local=True ..." ...
BUG=None
TEST=Tested with the folllowing combinations.
benchmark: sunspider, smoothness.top_25_smooth
perf_args: record -e cycles,instructions / (none)
Change-Id: I7315027a7c9433d17a9f3fe54d7e8c3f480ea4f4
Reviewed-on: https://chrome-internal-review.googlesource.com/217370
Reviewed-by: Caroline Tice <cmtice@google.com>
Tested-by: Ting-Yuan Huang <laszio@google.com>
Commit-Queue: Ting-Yuan Huang <laszio@google.com>
/external/toolchain-utils/crosperf/experiment_factory.py
|
| 0c84ea75c90b494df6b1ea6923a48611be81b96c |
|
25-Jun-2015 |
cmtice <cmtice@google.com> |
Update all_toolchain_perf to use new recommended set.
Update the all_toolchain_perf alias to use the new recommended
set of tests. Also update the default test values to include
default values for the newly recommended tests. Also update the
nightly tests to use the newly recommended set.
BUG=None
TEST=Tested new alias with crosperf.
Change-Id: I603f866723646a2d5b6d86fbe1de3c4617e09046
Reviewed-on: https://chrome-internal-review.googlesource.com/220385
Reviewed-by: Han Shen <shenhan@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/experiment_factory.py
|
| 517dc98884e398fa863ce22b6bc1ef8323853416 |
|
12-Jun-2015 |
cmtice <cmtice@google.com> |
Add flag to crosperf to use file lock mechanism.
This CL adds a flag to crosperf that specifies using the old file
locking mechanism, rather than the new AFE server locking mechanism.
This is useful when there are issues with accessing the AFE server.
BUG=None
TEST=Tested crosperf with this, both using the AFE locking mechanism
and using the file locking mechanism.
Change-Id: If6aa3e97a77d8ccb53f0e8f0405cc7f45e983b75
Reviewed-on: https://chrome-internal-review.googlesource.com/219096
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/experiment_factory.py
|
| df76222bf1cbdbc42bb41934fb960bac8694eca4 |
|
19-May-2015 |
Luis Lozano <llozano@chromium.org> |
Added benchmark retry support.
We often get spurious failures from the benchmarks runs. This CL adds
support for a "retries" field that will automatically retry the benchmark
run by the amount specified.
BUG=None
TEST=Test by hand.
Change-Id: Ib881c40d766e67a7daf855ce9137d3e3d631f12e
Reviewed-on: https://chrome-internal-review.googlesource.com/217525
Reviewed-by: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Luis Lozano <llozano@chromium.org>
/external/toolchain-utils/crosperf/experiment_factory.py
|
| 5c09fc2966ac49263ce7154c2905f2a86aeda297 |
|
22-Apr-2015 |
cmtice <cmtice@google.com> |
Fix current problem with machine locks.
Currently if crosperf can't access the default locks directory, it
assumes that it is safe to proceed without locks and it uses machines
without checking locks. This script does two things: It causes
crosperf to fail if it can't access a locks directory; and it allows
the user to specify an alternate locks directory. As a by-product it
also adds a boolean prompting function to utils/misc.py
BUG=None
TEST=Tested with default directory accessible & inaccessible; tested using
local locks directory.
Change-Id: I6f24f772986813bf089f5f4e40335a15e8ee4837
Reviewed-on: https://chrome-internal-review.googlesource.com/214129
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/experiment_factory.py
|
| 0e2d9a0bafc6bff6a53ca4ddf779715994f28ea8 |
|
26-Mar-2015 |
Luis Lozano <llozano@chromium.org> |
Remove deprecated Telemetry benchmarks.
A bunch of benchmarks listed in our suites have been removed from
Telemetry. Stop trying to use them.
BUG=None
TEST=Verified by hand
Change-Id: Ic624c1867f6129650f9a02f7325d94d7abb970ee
Reviewed-on: https://chrome-internal-review.googlesource.com/210560
Commit-Queue: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/experiment_factory.py
|
| a1fbd8af3ea4efc3a9bda1717212a9bacf30950a |
|
05-Feb-2015 |
cmtice <cmtice@google.com> |
Make crosperf fail if 'build' field is empty.
Change-Id: Ia929969e0e82cc16f2ebd7c020b1af7e0b5c84ff
Reviewed-on: https://chrome-internal-review.googlesource.com/196107
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/experiment_factory.py
|
| 1a224369afcbfd0276f4c7bdc625dec7f7b30d01 |
|
17-Oct-2014 |
cmtice <cmtice@google.com> |
Fix issue where crosperf can't find local cros_scratch.
Crosperf was assuming the user always has an NFS home directory,
and was looking for the cros_scratch directory there. This makes
crosperf use the user's home directory, whether it's NFS or local.
Also change --share_users, which also assumed NFS home directories,
to --share_cache, which requires specification of the cache path(s)
to use.
BUG=None
TEST=Tested this with my new local homedir.
Change-Id: I28d15467dfbd25b8b237cefa2979d3fd9464a7a7
Reviewed-on: https://chrome-internal-review.googlesource.com/179978
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/experiment_factory.py
|
| 2a370f7af34f6b126e80ef55891b9bff6a2858bb |
|
30-May-2014 |
cmtice <cmtice@google.com> |
Add --cache_only global setting to crosperf.
The CL adds a new setting, --cache_only, which causes crosperf to
ONLY report test results that it finds in the cache (it will NOT
rerun any tests, even if no cache results are found). This is partly
for implementing a weekly summary report, where we want to compare the
nightly results from the past week, but not actually rerun anything.
BUG=None
TEST=I tested this option while generating weekly test reports, and it
worked as it is supposed to.
Change-Id: Icb1f5343f76ce999bebd0810bb86ab64a7577313
Reviewed-on: https://chrome-internal-review.googlesource.com/164890
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/experiment_factory.py
|
| 2250df9b2fb7ffe523b4f90e06b0d7602b83876f |
|
09-May-2014 |
cmtice <cmtice@google.com> |
Expand experiment_factory_unittest.py to be more complete.
This CL adds more testing to experiment_factory_unittest.py, and it
fixes one small bug in experiment_factory.py.
BUG=None
TEST=Ran unittest.
Change-Id: I9fd13c05d793bb71accf3e19d361dff79f7110e9
Reviewed-on: https://chrome-internal-review.googlesource.com/163149
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/experiment_factory.py
|
| 98a53692fb946a8eac46e3e82257f540d1350c18 |
|
16-Apr-2014 |
cmtice <cmtice@google.com> |
Clean up Crosperf options and help.
Crosperf's help was completely out of date and incorrect. Fixed
that. Also, I cleaned up the options, removed some that were no
longer appropriate or not ever used, and made some options global-only.
Also removed all references to pyauto. We no longer use or run pyauto
tests.
Option changes: Completely removed outlier_range (it was not actually doing
anything), key_results_only (applied only to pyauto tests), use_test_that
(not needed now that we're not using pyauto) and md5sum (never used).
Made the following options global-only: rm_chroot_tmp, perf_args, logging_level,
show_all_results, board. Removed 'build' from global settings and made
it an image-only setting (it made no sense as a global setting).
I also adjusted the unittests to work with these changes.
BUG=None
TEST=I tested all the options in all their various positions; I tested with and
wtihout caching; and I tested local and official images. I also ran the unittests. Everything works.
Change-Id: I7653663f65c0a3363a1904c0220f1066e72c7f9d
Reviewed-on: https://chrome-internal-review.googlesource.com/160758
Reviewed-by: Bhaskar Janakiraman <bjanakiraman@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/experiment_factory.py
|
| 43f1a45c8dddfc4ff8c9dfcd87070811abf936dd |
|
04-Apr-2014 |
cmtice <cmtice@google.com> |
Download official & trybot images; treat as local images for tests.
Currently we cannot get md5sums for official & trybot images, which
causes multiple problems: Crosperf re-images the DUT every time it
runs a test for an image, even if the image is already on the DUT;
if crosperf is running an experiment with multiple official or trybot
images, it can get confused about which results belong to which image;
caching does not work properly without md5sums.
To fix all of these problems, this CL changes Crosperf to download the
official or trybot images into the /tmp directory in the chroot, and from
there it can treat them as local images (getting md5sums etc). In order
to download them, it first has to translate the xbuddy syntax (which can
contain aliases such as 'lumpy/dev-latest') into the actual image name
(e.g. lumpy-release/R36-5727.0.0). This translation is done by a new
script, translate_xbuddy.py, which must be run from the toolchain-utils
directory inside the chroot.
BUG=356279,356474,356476
TEST=Tested with official and trybot images, using both correct
designations and xbuddy aliases; tested with the image missing from
the chroot and with it already in /tmp; tested with and without caching.
Also tested with the experiment file included in the 3 bugs.
Change-Id: I917b5520604b7d4851db2c8123165e81b866da2b
Reviewed-on: https://chrome-internal-review.googlesource.com/159465
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/experiment_factory.py
|
| 2cce1bd6a77fe537fc36d6e1b804bc9f19a67651 |
|
27-Mar-2014 |
cmtice <cmtice@google.com> |
Add page_cycler.bloat to Toolchain team recommended test set.
BUG=None
TEST=Ran test with this change.
Change-Id: I89aab58c8890a7c3f36707d2ffe7422fbfa17547
Reviewed-on: https://chrome-internal-review.googlesource.com/158585
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/experiment_factory.py
|
| 6de7f8fe91a1a36d0e7c578ac8d170bf483fcfbe |
|
14-Mar-2014 |
cmtice <cmtice@google.com> |
Various small fixes for nightly infrastructure tests.
- Change the number of iterations for tests from 1 to 3.
- Increase the timeout for image_chromeos.py, so 'cros flash', which
takes a bit longer than cros_image_to_target.sh did, doesn't get killed
just as its finishing.
- Comment out smoothness.top_25 for now, since it's broken and/or
hanging.
BUG=None
TEST=Successfully ran nightly infrastructure tests with these changes.
Change-Id: I77b663b1910fe12ef927afdf15d88a7c1e82d491
Reviewed-on: https://chrome-internal-review.googlesource.com/157419
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/experiment_factory.py
|
| 139092449a545d4d87c06af3a6d8f3d71e42e927 |
|
11-Mar-2014 |
cmtice <cmtice@google.com> |
Add --logging_level option to crosperf.
Add --logging_level={verbose, average, quiet} option to crosperf,
optionally reducing the amount of logging output. "verbose" gives
the same output Crosperf always gave; "quiet" gives minimal output, and
"average" is in the middle. The default for now is "verbose". Doing this
also involved adding logging levels to the command_executer, and to
image_chromeos (again, with the default being "verbose").
BUG=None
TEST=Tested all three levels of logging, with and without having to
reimage the DUT. It all worked properly.
Change-Id: Icf8c9a6831fe25202adcb624c6c7e1d1a7ac25a5
Reviewed-on: https://chrome-internal-review.googlesource.com/156883
Reviewed-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/experiment_factory.py
|
| 0cc4e7790afbd514675801a1ffb90517c147270f |
|
31-Jan-2014 |
cmtice <cmtice@google.com> |
Add 'build' settings option and xbuddy image format.
This patch updates Crosperf to add the 'build' tag in the
experiment file, and to allow that field to contain xbuddy
syntax for using trybot and/or official builds in the test runs.
It also adds a bit more checking to make sure we have everything
necessary for running 'cros flash' before attempting to use it.
BUG=None
TEST=I have run this using an experiment file that compares a local
image, a trybot image and an official image against each other. It
all worked.
Change-Id: Ia896799061508fb5718a3201b1599d8622de0b3f
Reviewed-on: https://chrome-internal-review.googlesource.com/154097
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/experiment_factory.py
|
| 126e0c31bf6b973c500f86296391b954ccbae218 |
|
10-Jan-2014 |
cmtice <cmtice@google.com> |
Add test group containing recommended Telemetry performance tests
for toolchain group.
Change-Id: I178975045ed78c93800d4d6c1f6c86fcb88e3ecd
Reviewed-on: https://chrome-internal-review.googlesource.com/151653
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/experiment_factory.py
|
| 9847df92a2b5f76ccddc4bf10288819712a8ca47 |
|
07-Dec-2013 |
cmtice <cmtice@google.com> |
Use default results for Telemetry tests; add option to use all results.
When running telemetry_Crosperf, use the json file of report defaults to
determine which results to actually include in the report. This CL
also adds a flag to Crospref, --show_all_results=True, that allows the
user to override the defaults and generate a report showing all the values.
BUG=None
TEST=Tested this both with and without the --show_all_results flag. It
works as expected for me.
Change-Id: I91364572ebdf3d54bb8b408fbd5a2580923f6829
Reviewed-on: https://chrome-internal-review.googlesource.com/149457
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/experiment_factory.py
|
| 0440388677b2445f4f8273b6c2b4d82601b1aca3 |
|
04-Nov-2013 |
cmtice <cmtice@google.com> |
Make changes to run Telemetry tests in nightly infrastructure tests.
BUG=None
TEST=Tested this running the nightly infrastructure cron jobs.
Change-Id: I6cccee222b074e9410b45df6750c097430207746
Reviewed-on: https://chrome-internal-review.googlesource.com/147515
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/experiment_factory.py
|
| 9c956f58876917af6cf5c6961241ddeaf1baf826 |
|
23-Oct-2013 |
Luis Lozano <llozano@chromium.org> |
Fixed name of telemetry benchmark page_cycler.basic_insert
BUG=None
TEST=Tested by hand using the new name
Change-Id: I7246f9f5d2a5e17133613bc4f9038fe1b42332e9
Reviewed-on: https://chrome-internal-review.googlesource.com/146909
Reviewed-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
/external/toolchain-utils/crosperf/experiment_factory.py
|
| f8b3a5ada4ae7c60d459fc843b1809acd7726613 |
|
25-Sep-2013 |
Caroline Tice <cmtice@chromium.org> |
Allow "." in benchmark names; Allow multiple test sets.
This fixes a few minor problems with crosperf:
1). When using run_remote_tests.sh, it adds the "--use_emerged" flag,
which is a recent requirement.
2). It updates the regular expression parsing to not mess up escaped
characters, and to allow "." in benchmark names (common in Telemetry).
3). It allows there to be multiple "sets" of benchmarks that can be
specified together, e.g. perfv2 or pagecyclers (or more in the future).
BUG=None
TEST=I've run all the various telemetry_Crosperf variations (individual
perf_v2 tests; individual telemetry page cycler tests; 'all_perf v2', and
'all_pagecyclers'.
Change-Id: Idd45caa6487780b92b8eb321b37b873978f96dca
Reviewed-on: https://chrome-internal-review.googlesource.com/145305
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/experiment_factory.py
|
| 92774196e083e04b954d02f8d463998d549269eb |
|
11-Sep-2013 |
Caroline Tice <cmtice@chromium.org> |
Update crosperf to run telemetry test via autotest.
This updates crosperf to recognize the suite 'telemetry_Crosperf',
which allows users to run telemetry performance tests via test_that
and the telemetry_Crosperf autotest. This won't be usable in general until
CL 168491 (the telemetry_Crosperf CL) gets committed.
BUG=None
TEST=Invoked telemetry tests using this change.
Note: Profilers don't quite work properly with telemetry_Crosperf yet,
but everything else does.
Change-Id: Id14edc89db79bc244496e8f6a5b614300a00be54
Reviewed-on: https://chrome-internal-review.googlesource.com/144440
Reviewed-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/experiment_factory.py
|
| b47bff4d3336c5fe5593a95963c0f3dc20a02f68 |
|
20-Aug-2013 |
Caroline Tice <cmtice@chromium.org> |
Make test_that available for running tests.
Add a new flag, --use_test_that=True/False, to crosperf, to allow users to
tell crosperf to use the test_that script rather than the run_remote_tests.sh
script for running their tests.
BUG=None
TEST=Tested with and without the flag, to verify that it does the right thing.
Change-Id: I1a040d23adbf364b4a47320d9fd759a2d46e6689
Reviewed-on: https://gerrit-int.chromium.org/43109
Reviewed-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/experiment_factory.py
|
| 2dd2e9e80ad235793e1591d21a16d23b2b5243d7 |
|
01-Jul-2013 |
Yunlian Jiang <yunlian@google.com> |
crosperf: filter out "/' in remote
TEST=nightly test passes.
BUG=None
Change-Id: I525e488c9f4c47cfe09bd8c15d06997b9e75f140
Reviewed-on: https://gerrit-int.chromium.org/40463
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Yunlian Jiang <yunlian@google.com>
/external/toolchain-utils/crosperf/experiment_factory.py
|
| 04dc5dc8547dbfbe524cf35ac39537346ad749bb |
|
24-Apr-2013 |
Yunlian Jiang <yunlian@google.com> |
Adding support of telemetry to crosperf
BUG=None
TEST=run crosperf with pyauto/telemetry test with/without cache.
all pass.
Change-Id: If07ac020a9107a79d5780a58fd6dcc924d07f07f
Reviewed-on: https://gerrit-int.chromium.org/36594
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-on: https://gerrit-int.chromium.org/39241
Reviewed-by: Yunlian Jiang <yunlian@google.com>
/external/toolchain-utils/crosperf/experiment_factory.py
|
| 00cc30ea5fb3a05530a3b1ec699ca7f9696ffbd0 |
|
28-Mar-2013 |
Yunlian Jiang <yunlian@google.com> |
Add option to specify output dir of crosperf
BUG=None
TEST=run crosperf with this option, the results are in the right dir
Change-Id: Id94251c72297f18f275564a1fd1ef0972227ddfc
Reviewed-on: https://gerrit-int.chromium.org/34621
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
/external/toolchain-utils/crosperf/experiment_factory.py
|
| f81680c018729fd4499e1e200d04b48c4b90127c |
|
15-Mar-2013 |
Luis Lozano <llozano@chromium.org> |
Cleaned up directory after copy of tools from perforce directory
Got rid of stale copies of some tools like "crosperf" and moved all
files under v14 directory (that came from perforce) into the top
directory.
BUG=None
TEST=None
Change-Id: I408d17a36ceb00e74db71403d2351fd466a14f8e
Reviewed-on: https://gerrit-int.chromium.org/33887
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Luis Lozano <llozano@chromium.org>
/external/toolchain-utils/crosperf/experiment_factory.py
|
| 4467f004e7f0854963bec90daff1879fbd9d2fec |
|
20-Dec-2012 |
Ahmad Sharif <asharif@chromium.org> |
Synced repos to: 64740
/external/toolchain-utils/crosperf/experiment_factory.py
|
| f395c26437cbdabc2960447fba89b226f4409e82 |
|
10-Oct-2012 |
Ahmad Sharif <asharif@chromium.org> |
Synced repos to: 63271
/external/toolchain-utils/crosperf/experiment_factory.py
|
| 0dcbc4b1714260820fd4b8d6536fbb05e139cc0f |
|
03-Feb-2012 |
Ahmad Sharif <asharif@chromium.org> |
Synced repos to: 58208
/external/toolchain-utils/crosperf/experiment_factory.py
|