| f6ef4395fe1896ba68c80e52cb24763b0fcfe7f8 |
|
07-Apr-2017 |
Caroline Tice <cmtice@google.com> |
[toolchain-utils] Fix remaining lint errors in toolchain-utils.
In addition to fixing the lint errors, this also fixes the Python
formatting issues (ran tc_pyformat on nearly all the files).
BUG=chromium:570450
TEST=Ran all crosperf & bisect tool unit tests.
Ran afe_lock_machine.py (check machine status)
Ran full crosperf test (octane, speedometer, BootPerf) on alex.
Change-Id: Ic86f9192801ac67769f3de30f1c5f0d203ce0831
Reviewed-on: https://chromium-review.googlesource.com/471886
Commit-Ready: Caroline Tice <cmtice@chromium.org>
Tested-by: Caroline Tice <cmtice@chromium.org>
Reviewed-by: Manoj Gupta <manojgupta@chromium.org>
/external/toolchain-utils/crosperf/benchmark.py
|
| 9088f2b8997baf37098bce7a464348296e0dbe47 |
|
10-Mar-2017 |
Ting-Yuan Huang <laszio@chromium.org> |
crosperf: make default iterations of benchmarks >= 2
So as to get standard deviations, which are needed in ttest for p-value.
TEST=none
BUG=none
Change-Id: If59fb46b62c0cb58610507962f8ca13ccc0b7d01
Reviewed-on: https://chromium-review.googlesource.com/452796
Commit-Ready: Ting-Yuan Huang <laszio@chromium.org>
Tested-by: Ting-Yuan Huang <laszio@chromium.org>
Reviewed-by: Manoj Gupta <manojgupta@chromium.org>
/external/toolchain-utils/crosperf/benchmark.py
|
| 8332364c0237ca6c4976c5206346ab9a596c8e98 |
|
17-Feb-2017 |
Ting-Yuan Huang <laszio@google.com> |
crosperf: set recommended iterations for benchmarks
This CL associates estimated standard deviations to each benchmark,
according to experiments. The recommended iterations can be specified by
setting iterations = 0 in the experiment files. Setting it to numbers
greater than 0 will override the default iterations and behaves exactly
the same as before.
With this change, benchmarks in all_toolchain_perf get no more than 2%
margin of error within 90% of time. See crbug.com/673558 for how the
standard deviations are estimated.
BUG=chromium:673558
TEST=all_toolchain_perf + page_cycler_v2.typical_25 finishes in 3.5
hours for an image on chell.
Change-Id: Ie2ed07878c1237ad31a8568ae3fd3fb96cd11f3b
Reviewed-on: https://chromium-review.googlesource.com/424915
Commit-Ready: Ting-Yuan Huang <laszio@chromium.org>
Tested-by: Ting-Yuan Huang <laszio@chromium.org>
Reviewed-by: Caroline Tice <cmtice@chromium.org>
/external/toolchain-utils/crosperf/benchmark.py
|
| 9099a788cd7124024559c064e425ed9caef6e0ac |
|
23-Jul-2016 |
Caroline Tice <cmtice@google.com> |
[crosperf] Clean up exceptions.
Found all instances of 'raise Exception' and changed them to raise
something slightly more specific. Changed a few 'except Exception'
statements appropriately as well.
BUG=chromium:566255
TEST=None
Change-Id: If2666bd55838342bc71431e5e6ab2157c2c9e125
Reviewed-on: https://chrome-internal-review.googlesource.com/270731
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
/external/toolchain-utils/crosperf/benchmark.py
|
| 0d39863d8578fb1c6e7077b2c7cc5ca79edcf4dd |
|
02-Feb-2016 |
Han Shen <shenhan@google.com> |
Fix lint for toolchain-utils.
Tested: run_tests.sh && cros lint on all the modified files.
Change-Id: Ife1e9620535f70849c58e932637b7b0f150b13ac
Reviewed-on: https://chrome-internal-review.googlesource.com/246650
Commit-Ready: Han Shen <shenhan@google.com>
Tested-by: Han Shen <shenhan@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/benchmark.py
|
| f2a3ef46f75d2196a93d3ed27f4d1fcf22b54fbe |
|
15-Dec-2015 |
Luis Lozano <llozano@chromium.org> |
Run pyformat on all the toolchain-utils files.
This gets rid of a lot of lint issues.
Ran by doing this:
for f in *.py; do echo -n "$f " ; if [ -x $f ]; then pyformat -i
--remove_trailing_comma --yapf --force_quote_type=double $f ; else
pyformat -i --remove_shebang --remove_trailing_comma --yapf
--force_quote_type=double $f ; fi ; done
BUG=chromium:567921
TEST=Ran simple crosperf run.
Change-Id: I59778835fdaa5f706d2e1765924389f9e97433d1
Reviewed-on: https://chrome-internal-review.googlesource.com/242031
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
/external/toolchain-utils/crosperf/benchmark.py
|
| bc2d3d13d2e1928217140a76acdf9f9917b1fa30 |
|
02-Jul-2015 |
Ting-Yuan Huang <laszio@google.com> |
Add 'run_local' to run the test harness locally.
The option 'run_local: True|False' will be passed into autotest.
For example, this is how test_that will be invoked:
test_that --arges="run_local=True ..." ...
BUG=None
TEST=Tested with the folllowing combinations.
benchmark: sunspider, smoothness.top_25_smooth
perf_args: record -e cycles,instructions / (none)
Change-Id: I7315027a7c9433d17a9f3fe54d7e8c3f480ea4f4
Reviewed-on: https://chrome-internal-review.googlesource.com/217370
Reviewed-by: Caroline Tice <cmtice@google.com>
Tested-by: Ting-Yuan Huang <laszio@google.com>
Commit-Queue: Ting-Yuan Huang <laszio@google.com>
/external/toolchain-utils/crosperf/benchmark.py
|
| df76222bf1cbdbc42bb41934fb960bac8694eca4 |
|
19-May-2015 |
Luis Lozano <llozano@chromium.org> |
Added benchmark retry support.
We often get spurious failures from the benchmarks runs. This CL adds
support for a "retries" field that will automatically retry the benchmark
run by the amount specified.
BUG=None
TEST=Test by hand.
Change-Id: Ib881c40d766e67a7daf855ce9137d3e3d631f12e
Reviewed-on: https://chrome-internal-review.googlesource.com/217525
Reviewed-by: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Luis Lozano <llozano@chromium.org>
/external/toolchain-utils/crosperf/benchmark.py
|
| 226e3e08106dffd3086a7eee7007950a1d8ddfaf |
|
28-Apr-2014 |
cmtice <cmtice@google.com> |
Fix small bugs with non-autotest Telemetry; other cleanups.
This CL fixes various small problems found during recent testing:
- Force show_all_results to be true for 'pure' telemetry case (the
results filtering needs the autotest results to work properly)
- Fix bug that was inserting extra 'src' into Chrome src path sometimes
- Since non-autotest Telemetry and non-Telemetry-Crosperf tests can't
handle 'perf' make it a Fatal error to call them with perf args.
- Add missing field to example experiment file.
- Add status output messages to downlaod_images.py for non-verbose logging.
BUG=None
TEST=Tested all these fixes on multiple test runs.
Change-Id: If8209356c695dc8b21f8627399a1cbd4a858df23
Reviewed-on: https://chrome-internal-review.googlesource.com/161759
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/benchmark.py
|
| 98a53692fb946a8eac46e3e82257f540d1350c18 |
|
16-Apr-2014 |
cmtice <cmtice@google.com> |
Clean up Crosperf options and help.
Crosperf's help was completely out of date and incorrect. Fixed
that. Also, I cleaned up the options, removed some that were no
longer appropriate or not ever used, and made some options global-only.
Also removed all references to pyauto. We no longer use or run pyauto
tests.
Option changes: Completely removed outlier_range (it was not actually doing
anything), key_results_only (applied only to pyauto tests), use_test_that
(not needed now that we're not using pyauto) and md5sum (never used).
Made the following options global-only: rm_chroot_tmp, perf_args, logging_level,
show_all_results, board. Removed 'build' from global settings and made
it an image-only setting (it made no sense as a global setting).
I also adjusted the unittests to work with these changes.
BUG=None
TEST=I tested all the options in all their various positions; I tested with and
wtihout caching; and I tested local and official images. I also ran the unittests. Everything works.
Change-Id: I7653663f65c0a3363a1904c0220f1066e72c7f9d
Reviewed-on: https://chrome-internal-review.googlesource.com/160758
Reviewed-by: Bhaskar Janakiraman <bjanakiraman@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/benchmark.py
|
| 384d5f8f3eeb04d5063607c691b3732f0489934f |
|
19-Dec-2013 |
cmtice <cmtice@google.com> |
Change to use test_that by default instead of run_remote_tests.sh.
BUG=None
TEST=Tested this with BootPerServer in my directory. Checked the
logs to make sure the correct script was being used (test_that by
default now).
Change-Id: I39b2901c2ef3c8070c76b15b929df5da6b168c9c
Reviewed-on: https://chrome-internal-review.googlesource.com/150695
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/benchmark.py
|
| 9847df92a2b5f76ccddc4bf10288819712a8ca47 |
|
07-Dec-2013 |
cmtice <cmtice@google.com> |
Use default results for Telemetry tests; add option to use all results.
When running telemetry_Crosperf, use the json file of report defaults to
determine which results to actually include in the report. This CL
also adds a flag to Crospref, --show_all_results=True, that allows the
user to override the defaults and generate a report showing all the values.
BUG=None
TEST=Tested this both with and without the --show_all_results flag. It
works as expected for me.
Change-Id: I91364572ebdf3d54bb8b408fbd5a2580923f6829
Reviewed-on: https://chrome-internal-review.googlesource.com/149457
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/benchmark.py
|
| b47bff4d3336c5fe5593a95963c0f3dc20a02f68 |
|
20-Aug-2013 |
Caroline Tice <cmtice@chromium.org> |
Make test_that available for running tests.
Add a new flag, --use_test_that=True/False, to crosperf, to allow users to
tell crosperf to use the test_that script rather than the run_remote_tests.sh
script for running their tests.
BUG=None
TEST=Tested with and without the flag, to verify that it does the right thing.
Change-Id: I1a040d23adbf364b4a47320d9fd759a2d46e6689
Reviewed-on: https://gerrit-int.chromium.org/43109
Reviewed-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/benchmark.py
|
| 04dc5dc8547dbfbe524cf35ac39537346ad749bb |
|
24-Apr-2013 |
Yunlian Jiang <yunlian@google.com> |
Adding support of telemetry to crosperf
BUG=None
TEST=run crosperf with pyauto/telemetry test with/without cache.
all pass.
Change-Id: If07ac020a9107a79d5780a58fd6dcc924d07f07f
Reviewed-on: https://gerrit-int.chromium.org/36594
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-on: https://gerrit-int.chromium.org/39241
Reviewed-by: Yunlian Jiang <yunlian@google.com>
/external/toolchain-utils/crosperf/benchmark.py
|
| f81680c018729fd4499e1e200d04b48c4b90127c |
|
15-Mar-2013 |
Luis Lozano <llozano@chromium.org> |
Cleaned up directory after copy of tools from perforce directory
Got rid of stale copies of some tools like "crosperf" and moved all
files under v14 directory (that came from perforce) into the top
directory.
BUG=None
TEST=None
Change-Id: I408d17a36ceb00e74db71403d2351fd466a14f8e
Reviewed-on: https://gerrit-int.chromium.org/33887
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Luis Lozano <llozano@chromium.org>
/external/toolchain-utils/crosperf/benchmark.py
|
| 4467f004e7f0854963bec90daff1879fbd9d2fec |
|
20-Dec-2012 |
Ahmad Sharif <asharif@chromium.org> |
Synced repos to: 64740
/external/toolchain-utils/crosperf/benchmark.py
|
| f395c26437cbdabc2960447fba89b226f4409e82 |
|
10-Oct-2012 |
Ahmad Sharif <asharif@chromium.org> |
Synced repos to: 63271
/external/toolchain-utils/crosperf/benchmark.py
|
| 0dcbc4b1714260820fd4b8d6536fbb05e139cc0f |
|
03-Feb-2012 |
Ahmad Sharif <asharif@chromium.org> |
Synced repos to: 58208
/external/toolchain-utils/crosperf/benchmark.py
|