| f6ef4395fe1896ba68c80e52cb24763b0fcfe7f8 |
|
07-Apr-2017 |
Caroline Tice <cmtice@google.com> |
[toolchain-utils] Fix remaining lint errors in toolchain-utils.
In addition to fixing the lint errors, this also fixes the Python
formatting issues (ran tc_pyformat on nearly all the files).
BUG=chromium:570450
TEST=Ran all crosperf & bisect tool unit tests.
Ran afe_lock_machine.py (check machine status)
Ran full crosperf test (octane, speedometer, BootPerf) on alex.
Change-Id: Ic86f9192801ac67769f3de30f1c5f0d203ce0831
Reviewed-on: https://chromium-review.googlesource.com/471886
Commit-Ready: Caroline Tice <cmtice@chromium.org>
Tested-by: Caroline Tice <cmtice@chromium.org>
Reviewed-by: Manoj Gupta <manojgupta@chromium.org>
/external/toolchain-utils/crosperf/results_report.py
|
| 6c438e04d12f5dd94dd060c9ab41a61766dc313e |
|
27-Oct-2016 |
Caroline Tice <cmtice@google.com> |
[crosperf] Fix buildbot_utils for rotating tester waterfall.
The rotating testers trybot waterfall location has moved. This CL
updates the script to look for their build logs in the correct
builder location. It also makes sure we don't crash/die because
their build status is non-zero (ie. a HWTest failed). This also fixes
a tiny bug in the results reporting.
BUG=chromium:660085
TEST=Tested in the role account.
Change-Id: I396ec0ad588e54e88bc44dcc0e72eda45d077173
Reviewed-on: https://chrome-internal-review.googlesource.com/300598
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
/external/toolchain-utils/crosperf/results_report.py
|
| 2368b41ea869a6904ae8320ad69f1b8b9a9a3714 |
|
25-Aug-2016 |
George Burgess IV <gbiv@google.com> |
crosperf: Make results_report more general.
The goal of this patch is to allow us to use results_report to generate
non-ChromeOS-specific results reports. It's meant to be a nop for
regular crosperf runs (except that, in HTML report generation,
whitespace in the HTML itself will look a bit different.)
Moreover, results_report used to shuffle Experiments around, each of
which, unsurprisingly, contained *tons* of data about an experiment.
So, part of this patch was reducing results_report's reliance on
Experiments, in favor of a new type, BenchmarkResult.
Some parts of results_report still rely on Experiments, but only to
provide *extra* data. The minimum amount of data needed to make a
results report is contained in a BenchmarkResult, and there's a
convenient API we can use to make a BenchmarkResult out of an
Experiment.
This patch also does a massive refactor of results_report, because lots
of the code was mildly more icky than it is with this patch applied.
The refactor-for-prettiness and refactor-for-BenchmarkResults kind of
go hand-in-hand, so it's really hard to split them out. The only part
that's not so difficult to split out is the refactor to
results_report_templates, but the value of splitting that out is
questionable (to me).
Speaking of which, all HTML magicks that were in results_report is now
in results_report_templates, and now use *actual* Templates. It -- and
HTMLReportGenerator -- are hopefully more readable as a result.
Next, this makes JSONRsultsReport's GetReport() return a string
containing JSON data, rather than calling a callback to write
that string to a file.
Finally, this includes a change to perf_table. Namely, its removal. It
was otherwise unused, and I couldn't get it to even work, so I cleaned
it up, made a perf report parser (because I couldn't get the cros_utils
one to work, either...), and made sure it functions. If we're able to
produce perf reports that cros_utils can parse, I'm happy to back my
new parser out; I was primarily using it so I could be sure I didn't
break HTML report generation.
BUG=chromium:641098
TEST=./run_tests.sh passes
Change-Id: I437de9eb39e00c9dd5c223ecd27feaaab544a6fd
Reviewed-on: https://chrome-internal-review.googlesource.com/282217
Commit-Ready: George Burgess <gbiv@google.com>
Tested-by: George Burgess <gbiv@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/results_report.py
|
| afb8cc77e82c35faedfe541d097fc01fd1d7ca3d |
|
25-Aug-2016 |
George Burgess IV <gbiv@google.com> |
crosperf: refactor results_organizer; NFC.
As part of playing with results_report (so I could make it less
ChromeOS-dependent), I started refactoring results_organizer, which is
only used by results_report.
This was the result. The goal was to make it easier to understand what,
exactly, results_organizer is trying to do.
BUG=chromium:641098
TEST=./test_all.sh passes.
Change-Id: I8e53175fbc1ded6f23631550223905b3321376ca
Reviewed-on: https://chrome-internal-review.googlesource.com/280815
Commit-Ready: George Burgess <gbiv@google.com>
Tested-by: George Burgess <gbiv@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/results_report.py
|
| eb9fce674ff90a6de04827bfe1ef6e07a99c8f61 |
|
19-Aug-2016 |
Caroline Tice <cmtice@google.com> |
[crosperf] Fix reading from cache.
The read-from-cache code needed to be updated to deal with the new
telemetry results format, as did the code for generating json
reports (for archiving in the database).
BUG=chromium:639106
TEST=Tested with crosperf reading from cache & generating json results.
Change-Id: I2c97e5b393583048cc74559d065f498c0f0675a1
Reviewed-on: https://chrome-internal-review.googlesource.com/278298
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
/external/toolchain-utils/crosperf/results_report.py
|
| 63f13489b1a02bbbc75b46fec6ae7a817442df94 |
|
30-Jun-2016 |
George Burgess IV <gbiv@google.com> |
[crosperf] Add more tests for results_report.
With this patch, all of the classes in results_report have tests. The
tests aren't overly extensive, but we can always improve on them later.
This also includes a few small cleanups of parts of machine_manager, and
results_report.
BUG=None
TEST=./run_tests.sh
Change-Id: I9613579d4f8c1079b452e1a0a2edf47b189ac1cd
Reviewed-on: https://chrome-internal-review.googlesource.com/268057
Commit-Ready: George Burgess <gbiv@google.com>
Tested-by: George Burgess <gbiv@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/results_report.py
|
| 2e9f8a097c095ca93052b368ffab4c850d4d3d0f |
|
29-Jun-2016 |
George Burgess IV <gbiv@google.com> |
[crosperf] Add results_report tests, minor refactors.
This adds unit tests for results_report.JSONResultsReport and
results_report.ParseChromeosImage. The test for the former aren't as
extensive as they could be, but I hope to add more in the future.
This also simplifies ParseChromeosImage, and does a few refactors to
JSONResultsReport.
This also fixes all of `cros lint`'s complaints about results_report.
BUG=None
TEST=./run_tests.sh
Change-Id: Ice4267a5ff2e10df6690595c6b80db1ee1977d50
Reviewed-on: https://chrome-internal-review.googlesource.com/267665
Commit-Ready: George Burgess <gbiv@google.com>
Tested-by: George Burgess <gbiv@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/results_report.py
|
| 74d85cf83860538fcf57c413cf8ae416baa0ef51 |
|
15-Apr-2016 |
Caroline Tice <cmtice@google.com> |
Fix error with json reports with multi-valued results.
BUG=None
TEST=Tested in in role account.
Change-Id: I502dda87f9a3647e8bea61a54f587813eb18b755
Reviewed-on: https://chrome-internal-review.googlesource.com/255739
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
/external/toolchain-utils/crosperf/results_report.py
|
| 285a66e9ee9d59c3e3051512db060ee485c440f0 |
|
30-Mar-2016 |
Caroline Tice <cmtice@google.com> |
[crosperf] Don't fail completely on missing result field.
After running to completion, while generating the json report
Crosperf sometimes tries to look for a result field that should be
there but isn't. Previously this would cause a runtime exception
and Crosperf would fail without generating any reports at all. This
CL fixes that (now a report still gets generated, without the one
missing field).
BUG=None
TEST=Tested this in nightly tests.
Change-Id: I35043e8c90fad6dbb5bfd5dede1905a2238bd3b7
Reviewed-on: https://chrome-internal-review.googlesource.com/253035
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Han Shen <shenhan@google.com>
/external/toolchain-utils/crosperf/results_report.py
|
| f2a3ef46f75d2196a93d3ed27f4d1fcf22b54fbe |
|
15-Dec-2015 |
Luis Lozano <llozano@chromium.org> |
Run pyformat on all the toolchain-utils files.
This gets rid of a lot of lint issues.
Ran by doing this:
for f in *.py; do echo -n "$f " ; if [ -x $f ]; then pyformat -i
--remove_trailing_comma --yapf --force_quote_type=double $f ; else
pyformat -i --remove_shebang --remove_trailing_comma --yapf
--force_quote_type=double $f ; fi ; done
BUG=chromium:567921
TEST=Ran simple crosperf run.
Change-Id: I59778835fdaa5f706d2e1765924389f9e97433d1
Reviewed-on: https://chrome-internal-review.googlesource.com/242031
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
/external/toolchain-utils/crosperf/results_report.py
|
| 75e1ccc6513c4529e47007105b7c523755f8e0c0 |
|
13-Dec-2015 |
Caroline Tice <cmtice@google.com> |
Fix two minor bugs introduced in the Fix-It that broke nightly tests.
BUG=None
TEST=Tested with role account running simple crosperf test.
Change-Id: Ib9d69ce8815b674ad57784d90c7e3cc7a3fba6ed
Reviewed-on: https://chrome-internal-review.googlesource.com/241967
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
/external/toolchain-utils/crosperf/results_report.py
|
| 0009415847e04be4670c025e1b475adc7de23cc8 |
|
10-Dec-2015 |
Yunlian Jiang <yunlian@google.com> |
crosperf: clean lint warning for results_report.py
BUG=chromium:567921
TEST=it is lint warning free now.
Change-Id: I623851a3420033f9a43e92c77423479c5cd766d9
Reviewed-on: https://chrome-internal-review.googlesource.com/241528
Commit-Ready: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
/external/toolchain-utils/crosperf/results_report.py
|
| 0d1a9f32c928e21a72547f3d334d631c5861f027 |
|
09-Dec-2015 |
Yunlian Jiang <yunlian@google.com> |
crosperf: use cros_utils instead of utils.
We got some lint warnings about some module is not in utils.
So we change the name.
This CL creates a symbolic link cros_utils from utils. Once we are sure
that we clean all the code, we can rename the utils to cros_utils
directly.
BUG=chromium:567921 chromium:568195
TEST=the lint warning is gone.
Change-Id: I776e45c7bdd75d1f065ea5797f3bcff457203ca2
Reviewed-on: https://chrome-internal-review.googlesource.com/241457
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/results_report.py
|
| 89d263c7cf9773129cbe8e8858ad21ea539a2ba0 |
|
04-Nov-2015 |
Luis Lozano <llozano@chromium.org> |
Fixed bad key problem while generating json report.
In some weird cases, telemetry benchmarks incorrectly
return empty performance results. This was causing an invalid
key error while generating the json report.
BUG=chromium:551105
TEST=Forced generation of empty results and tested we dont
have the dictionary missing key issue.
Change-Id: I32803c573f1d268e010313292b99c0017a5dff92
Reviewed-on: https://chrome-internal-review.googlesource.com/237516
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/results_report.py
|
| ddde50532281f7f796dd7dc44b562b29d25ab381 |
|
23-Sep-2015 |
Caroline Tice <cmtice@google.com> |
Add compiler info to experiment data.
Check the build label to see which compiler was used. Store the
compiler in the label, and add it to the json filename
and output.
Also update nightly buildbot script to add 'compiler' field to
labels.
BUG=None
TEST=Tested in my directory.
Change-Id: I124c4910cce1145708bb2a52e3a6d6dff93777c3
Reviewed-on: https://chrome-internal-review.googlesource.com/231811
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/results_report.py
|
| 31fedb0322eea850ed4ee45cb008c5b7ca5498b0 |
|
15-Sep-2015 |
Caroline Tice <cmtice@google.com> |
Get Chrome version and store in cache & json files.
This updates crosperf to query the DUT for the version of Chrome
running on it, and to store the Chrome version both in the json
reports and in the human-readable cache keys file.
BUG=None
TEST=None
Change-Id: Ie8215528b8119f95306cad694e6898cc4810730e
Reviewed-on: https://chrome-internal-review.googlesource.com/230537
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Han Shen <shenhan@google.com>
/external/toolchain-utils/crosperf/results_report.py
|
| 9b852cfd9a602c5f80c8e621c696b796ce5177fd |
|
09-Sep-2015 |
Caroline Tice <cmtice@google.com> |
Store machine description info in cache and json reports.
This CL updates the cache mechanism to store the machine spec
info (the checksum string) in the cache, as well as storing the
machine information in the json reports (to go into the dashboard
archive). It also updates the json reports to always include the
full set of result fields for a test, and it verifies that the
old cache directory exists before trying to remove it.
BUG=None
TEST=Tested in my directory, with --cache_only=True, --rerun=True and
--json_report=True
Change-Id: Ib12bae4ca10141de0b972e3a7aeee83c7b59049b
Reviewed-on: https://chrome-internal-review.googlesource.com/229765
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Han Shen <shenhan@google.com>
/external/toolchain-utils/crosperf/results_report.py
|
| f63421f72bc7606ce8bfc04ce9acf25e3bf8c4c0 |
|
09-Sep-2015 |
Caroline Tice <cmtice@google.com> |
Fix minor bug with nightly tests.
Check that iter_results is not empty before trying to check a
value in it.
BUG=None
TEST=None
Change-Id: I60946e4351c137c2ba79fc978d1f6ec6635cc40e
Reviewed-on: https://chrome-internal-review.googlesource.com/229755
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Rahul Chaudhry <rahulchaudhry@google.com>
/external/toolchain-utils/crosperf/results_report.py
|
| 225fc17884a659647ed53ffd305b5e87533c18eb |
|
02-Sep-2015 |
Caroline Tice <cmtice@google.com> |
Make ParseChromeosImage globally callable.
Move the _ParseChromeosImage out of the JSONResultsReport class, and
make it a global function, so it can be easily called from other
places.
BUG=None
TEST=Tested with and without generating json reports, with the function
moved, and everything still works.
Change-Id: I71c0435e0da9e687f69b4cebd5e3f36764d495e1
Reviewed-on: https://chrome-internal-review.googlesource.com/228840
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/results_report.py
|
| c90a8069fa018bec72a459f444b3a0073e540e4c |
|
26-Aug-2015 |
Caroline Tice <cmtice@google.com> |
Fix minor bugs with json report output.
This CL fixes two minor bug: make sure an item is a list before
trying to get it's first element; don't try to convert a word string
to a float.
BUG=None
TEST=Tested on the cases where it was a problem and it works now.
Change-Id: Ia26cf96593997aade3ac78feefc4dfc22f0dcbf8
Reviewed-on: https://chrome-internal-review.googlesource.com/228152
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
/external/toolchain-utils/crosperf/results_report.py
|
| ef4ca8a8cfe05ec09447896db42fb220d07834f8 |
|
25-Aug-2015 |
Caroline Tice <cmtice@google.com> |
Add json report archiving.
This CL adds a new output, in json format, for archiving test
results. It also adds a new flag, --json_report, to crosperf to
tell it to generate the json output file in the usual results
directory. The json output is always in addition to the other
reports that Crosperf generates. This CL also make some minor
changes to the tool that maintains the telemetry default results
file, so it can be used by the json report generator.
BUG=None
TEST=Tested Crosperf with & without new flag; it did what it should.
Change-Id: Id0e476716ceee208d33e6f9427ea95235fbf11d4
Reviewed-on: https://chrome-internal-review.googlesource.com/228325
Reviewed-by: Han Shen <shenhan@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/results_report.py
|
| 74b3d3234dc857ebee3af81faa3f8f420ac6ecce |
|
15-Jul-2014 |
cmtice <cmtice@google.com> |
Fix issue where Telemetry tests appear to pass, but really failed
Occasionally Telemetry tests will return a "pass" even though the test
did not run (e.g. if the test was disabled). Crosperf was looking for
results for every passing test and dying when it failed to find a
result for these false passes. This CL looks for the false pass and
changes it fail, as well as writing out a warning message.
A bug was filed for this in Buganizer:
https://b.corp.google.com/issue?id=16190266, but since this was not
filed as a Chrome/ChromeOS bug, I will leave the bug mark below as
None (to avoid confusing the bug droid).
BUG=None
TEST=Tested this against a Telemetry run with both false and real
passes, and it worked properly.
Change-Id: Ie010c0cd5b0775d0e20cfd6967975eb4e768c55c
Reviewed-on: https://chrome-internal-review.googlesource.com/169115
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/results_report.py
|
| 71e004237b85bdcab97c09b7f58305e5f5eff7f8 |
|
18-Apr-2014 |
cmtice <cmtice@google.com> |
Various minor crosperf cleanups.
- Add results summary information for peacekeeper.html to json file.
- Don't add section for 'perf table' to html report if there is no perf report.
- Remove the color box for the gmean speedup (there's no good way to label it
and it's confusing unless you know what it is).
BUG=None
TEST=Ran Crosperf with these changes. It all seems to work correctly.
Change-Id: I2bf5bdfc2667ae665767374d10672d2d19696267
Reviewed-on: https://chrome-internal-review.googlesource.com/160896
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/results_report.py
|
| df3d7d197f33c19e51ca688bdcc2d750e8ffb484 |
|
03-Apr-2014 |
cmtice <cmtice@google.com> |
Remove 'Completed' column; Remove 'retval' row from summary table.
The "Completed" column causes confusion because tests that fail are
still considered to have completed. This CL removes the 'Completed'
column, and instead posts the number of iterations that passed or failed,
for each test image, in the table header, beside the image name.
It also removes the 'retval' row from the summary table, since the retval
will either be '0' if a test iteration passes or '1' if it fails, and seeing
the average of all the retvals is useless, unless they all pass or fail anyway.
The retval row is still there in the full table, which shows all the results
for each iteration.
BUG=357340
TEST=I ran crosperf with this change in all configurations I could think of and
it seems to work.
Change-Id: I0e64b9ed016b0a4604814bac49442848cd27f5d3
Reviewed-on: https://chrome-internal-review.googlesource.com/159113
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
/external/toolchain-utils/crosperf/results_report.py
|
| 04dc5dc8547dbfbe524cf35ac39537346ad749bb |
|
24-Apr-2013 |
Yunlian Jiang <yunlian@google.com> |
Adding support of telemetry to crosperf
BUG=None
TEST=run crosperf with pyauto/telemetry test with/without cache.
all pass.
Change-Id: If07ac020a9107a79d5780a58fd6dcc924d07f07f
Reviewed-on: https://gerrit-int.chromium.org/36594
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-on: https://gerrit-int.chromium.org/39241
Reviewed-by: Yunlian Jiang <yunlian@google.com>
/external/toolchain-utils/crosperf/results_report.py
|
| f81680c018729fd4499e1e200d04b48c4b90127c |
|
15-Mar-2013 |
Luis Lozano <llozano@chromium.org> |
Cleaned up directory after copy of tools from perforce directory
Got rid of stale copies of some tools like "crosperf" and moved all
files under v14 directory (that came from perforce) into the top
directory.
BUG=None
TEST=None
Change-Id: I408d17a36ceb00e74db71403d2351fd466a14f8e
Reviewed-on: https://gerrit-int.chromium.org/33887
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Luis Lozano <llozano@chromium.org>
/external/toolchain-utils/crosperf/results_report.py
|
| 4467f004e7f0854963bec90daff1879fbd9d2fec |
|
20-Dec-2012 |
Ahmad Sharif <asharif@chromium.org> |
Synced repos to: 64740
/external/toolchain-utils/crosperf/results_report.py
|
| f395c26437cbdabc2960447fba89b226f4409e82 |
|
10-Oct-2012 |
Ahmad Sharif <asharif@chromium.org> |
Synced repos to: 63271
/external/toolchain-utils/crosperf/results_report.py
|
| ee988c87971c07b7e6767adb63d151ea37e363f2 |
|
15-May-2012 |
Ahmad Sharif <asharif@chromium.org> |
Synced repos to: 60457
/external/toolchain-utils/crosperf/results_report.py
|
| 0dcbc4b1714260820fd4b8d6536fbb05e139cc0f |
|
03-Feb-2012 |
Ahmad Sharif <asharif@chromium.org> |
Synced repos to: 58208
/external/toolchain-utils/crosperf/results_report.py
|