• Home
  • History
  • Annotate
  • only in /external/toolchain-utils/
History log of /external/toolchain-utils/
Revision Date Author Comments (<<< Hide modified files) (Show modified files >>>)
3690e025de8daaed03c4acb02d2b054e5c4c0dd5 23-Feb-2017 Chih-Hung Hsieh <chh@google.com> Use registered users in toolchain-utils/OWNERS. am: ca171c41df am: 7091edfaa0
am: 297e39c82b

Change-Id: I36f903fafc6aa3e8c37a8ad7d236a124f8732d01
297e39c82bbafae9d120812045210d3ed0dbb4cf 23-Feb-2017 Chih-Hung Hsieh <chh@google.com> Use registered users in toolchain-utils/OWNERS. am: ca171c41df
am: 7091edfaa0

Change-Id: I39661895ca7502e55186a63866133f57e9466901
7091edfaa0ec531905b7d914e9307cd74caf02ea 23-Feb-2017 Chih-Hung Hsieh <chh@google.com> Use registered users in toolchain-utils/OWNERS.
am: ca171c41df

Change-Id: I168c7e727657c6b5de8cf44bf23748c09bd8175c
ca171c41dfa8f27e8281057e04fe24553921ed86 23-Feb-2017 Chih-Hung Hsieh <chh@google.com> Use registered users in toolchain-utils/OWNERS.

AOSP project owners should be registered in Gerrit Code Review.

Bug: 33166666
Test: default build targets
Change-Id: I1196d33f47283f2f6ba8d7373190dcf9a2d80934
WNERS
9826fbe38e2bc35fc6c680cc216dcb993b54d2ff 08-Feb-2017 Stephen Hines <srhines@google.com> Merge remote-tracking branch 'aosp/mirror-chromium-master' into initial_import am: 870a8df6fc am: 9c6fa5f9e5 am: c5804ce784
am: 271389dd31

Change-Id: I22e657bc86117da9a6a45329a67a21096736da03
271389dd3199539c4474c351942f4d4fa975b81b 08-Feb-2017 Stephen Hines <srhines@google.com> Merge remote-tracking branch 'aosp/mirror-chromium-master' into initial_import am: 870a8df6fc am: 9c6fa5f9e5
am: c5804ce784

Change-Id: I5ff109272784db60dfef5145242a68779f7f0ccb
c5804ce784c39d6cf4f69139ab3197d989181cf9 08-Feb-2017 Stephen Hines <srhines@google.com> Merge remote-tracking branch 'aosp/mirror-chromium-master' into initial_import am: 870a8df6fc
am: 9c6fa5f9e5

Change-Id: Ie306363c46445f59e341b69121c4b0e113ad79b2
9c6fa5f9e514c743e62be0ee401c12e10c94d7f1 08-Feb-2017 Stephen Hines <srhines@google.com> Merge remote-tracking branch 'aosp/mirror-chromium-master' into initial_import
am: 870a8df6fc

Change-Id: I98e43c690589937e0db0981fc88383583c7b36d3
870a8df6fcb12de32fa2dd83b6ed0f7b24dbab1e 07-Feb-2017 Stephen Hines <srhines@google.com> Merge remote-tracking branch 'aosp/mirror-chromium-master' into initial_import

Initial import of Chromium's toolchain-utils project.

Bug: http://b/31321592
Test: None necessary, as this is just a helper repository.
Change-Id: I61d2caaf1195da18cfaa7795706e8fc7fecff1d4
058aae85dcfb12049ef90137915ec7e981288569 20-Sep-2016 Evelina Dumitrescu <evelinad@google.com> user_activity: Added tools for the benchmark profile collection.

The scripts run the Telemetry benchmarks, collects their perf profiles, use
local_cwp to do the profile symbolization and collect the hot
functions and callchains with pprof.

BUG=None
TEST=None

Change-Id: I2170e53d95924f1a3ba134bd62b1eead9e7a3077
Reviewed-on: https://chrome-internal-review.googlesource.com/288517
Reviewed-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Evelina Dumitrescu <evelinad@google.com>
Commit-Queue: Evelina Dumitrescu <evelinad@google.com>
Tested-by: Evelina Dumitrescu <evelinad@google.com>
Reviewed-on: https://chromium-review.googlesource.com/435923
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
ser_activity_benchmarks/collect_pprof_data.sh
ser_activity_benchmarks/collect_telemetry_profiles.sh
ser_activity_benchmarks/symbolize_profiles.sh
ser_activity_benchmarks/telemetry_benchmarks_R52_8350.68
215fd6f8720cd3f6c691e256e59fcdef4a8ac17b 04-Oct-2016 Evelina Dumitrescu <evelinad@google.com> user_acivity: Changed the paths and names for some Chrome OS groups.

BUG=None
TEST=None

Change-Id: I1124c62a67394f5f43e003338bad83855539e76a
Reviewed-on: https://chrome-internal-review.googlesource.com/292637
Tested-by: Evelina Dumitrescu <evelinad@google.com>
Reviewed-by: Evelina Dumitrescu <evelinad@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/435922
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
ser_activity_benchmarks/cwp_hot_functions_groups.txt
86c1c0b9354bc7ae4ca8bb3557459b1b9540e771 14-Oct-2016 Evelina Dumitrescu <evelinad@google.com> user_activity: Fixed a regression in ComputeScoreForBenchmarkSet.

The score fraction for a group was computed wrong.
The fraction should be computed as the score of the group over the
score of that group in the ideal case (each CWP function matching that group
should have the 1.0 value).

BUG=None
TEST=None

Change-Id: I294b42f0c8112ab9bd6f5d983876b6d779aad5e4
Reviewed-on: https://chrome-internal-review.googlesource.com/297675
Tested-by: Evelina Dumitrescu <evelinad@google.com>
Reviewed-by: Evelina Dumitrescu <evelinad@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/435921
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
ser_activity_benchmarks/benchmark_metrics.py
63dce8585d339dffcda92ab05c79fbb4409796cc 20-Oct-2016 Evelina Dumitrescu <evelinad@google.com> user_acivity: Changed some replacing pair strings.

Changed ('/build/gnawty', '/build/BOARD') to ('gnawty', 'BOARD') and
('/build/amd64-generic', '/build/BOARD') to ('amd64-generic', 'BOARD').

In the CWP data we have multiple paths containing the name of the board.
For example, we have /build/gnawty, but we also have
/var/cache/chromeos-chrome/chrome-src-internal/src/out_gnawty.
I made the subtitution generic.

BUG=None
TEST=None

Change-Id: Ie716a0f966b78a3c4ff85746cfa01b8b210d8b59
Reviewed-on: https://chrome-internal-review.googlesource.com/298336
Tested-by: Evelina Dumitrescu <evelinad@google.com>
Reviewed-by: George Burgess <gbiv@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Evelina Dumitrescu <evelinad@google.com>
Reviewed-on: https://chromium-review.googlesource.com/435920
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
ser_activity_benchmarks/utils.py
731ad0791050f4070b3d30fd859f6fd43892247c 21-Oct-2016 Evelina Dumitrescu <evelinad@google.com> user_activity: Moved some functions to the utils module.

Moved ComputeCWPCummulativeInclusiveStatistics and ComputeCWPChildFunctionsFractions
from the process_hot_functions module to the utils module.

BUG=None
TEST=None

Change-Id: Ib5968ae722cc8a775744418f24512e3d727d3936
Reviewed-on: https://chrome-internal-review.googlesource.com/298618
Commit-Queue: Evelina Dumitrescu <evelinad@google.com>
Tested-by: Evelina Dumitrescu <evelinad@google.com>
Reviewed-by: Evelina Dumitrescu <evelinad@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/435919
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
ser_activity_benchmarks/process_hot_functions.py
ser_activity_benchmarks/utils.py
45a52fb3586d4fbaf6ffc5f3595ae417b5d148be 21-Oct-2016 Evelina Dumitrescu <evelinad@google.com> user_activity: Fix regression in the benchmark_metrics_expriment.

The child fractions were not computed correctly after I updated the
extracted fields from the data collection scripts.

BUG=None
TEST=None

Change-Id: I8e672e0c245696e3351f1be9a3b391f46accea9f
Reviewed-on: https://chrome-internal-review.googlesource.com/298619
Commit-Queue: Evelina Dumitrescu <evelinad@google.com>
Tested-by: Evelina Dumitrescu <evelinad@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Evelina Dumitrescu <evelinad@google.com>
Reviewed-by: George Burgess <gbiv@google.com>
Reviewed-on: https://chromium-review.googlesource.com/435918
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
ser_activity_benchmarks/benchmark_metrics_experiment.py
ser_activity_benchmarks/benchmark_metrics_experiment_unittest.py
45478f9b3c3d0068eed177df6524c0e7f00a736c 08-Oct-2016 Evelina Dumitrescu <evelinad@google.com> user_activity: Added some parameters to the Dremel SQL scripts.

Used --min_completion_ration=1.0 to to ensure that Dremel queries all
available data and --sql_dialect=GoogleSQL to set up the dialect.
Added session_count > 2000 to extract only the busy profiles.
Added Chrome version as an argument in the command line.

BUG=None
TEST=None

Change-Id: I990499325f8e81ec0ea583077134f1416b56797a
Reviewed-on: https://chrome-internal-review.googlesource.com/294918
Tested-by: Evelina Dumitrescu <evelinad@google.com>
Commit-Queue: Evelina Dumitrescu <evelinad@google.com>
Reviewed-by: Evelina Dumitrescu <evelinad@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/435917
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
ser_activity_benchmarks/collect_experiment_data.sh
ser_activity_benchmarks/collect_experiment_data_odd_even_session.sh
0228f92f87bc785c3f8184ce79e2d016c95941fc 24-Sep-2016 Evelina Dumitrescu <evelinad@google.com> user_activity: Computed the metrics for the common functions.

Integrated the script with the utils module.

BUG=None
TEST=None

Change-Id: I4c8d0bdbda6df5996eae873655cfaf355cf559a2
Reviewed-on: https://chrome-internal-review.googlesource.com/290455
Reviewed-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Evelina Dumitrescu <evelinad@google.com>
Tested-by: Evelina Dumitrescu <evelinad@google.com>
Reviewed-on: https://chromium-review.googlesource.com/435916
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
ser_activity_benchmarks/process_hot_functions.py
c7faa09f456ca5c651ac373ad897aa4be6ad2717 29-Sep-2016 Evelina Dumitrescu <evelinad@google.com> user_activity: Created utility module for grouping parsing functions.

The modules contains the functions that do the parsing for the pprof top
and tree, CWP data and Chrome OS groups files.

BUG=None
TEST=utils_unittest passes

Change-Id: Ida94a2d456250afe4630a146349937eb0bab0767
Reviewed-on: https://chrome-internal-review.googlesource.com/291201
Reviewed-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Evelina Dumitrescu <evelinad@google.com>
Tested-by: Evelina Dumitrescu <evelinad@google.com>
Reviewed-on: https://chromium-review.googlesource.com/435915
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
ser_activity_benchmarks/testdata/input/cwp_function_groups.txt
ser_activity_benchmarks/testdata/input/inclusive_count_reference.csv
ser_activity_benchmarks/testdata/input/inclusive_count_test.csv
ser_activity_benchmarks/testdata/input/pairwise_inclusive_count_reference.csv
ser_activity_benchmarks/testdata/input/pairwise_inclusive_count_test.csv
ser_activity_benchmarks/testdata/input/pprof/file1.pprof
ser_activity_benchmarks/testdata/input/pprof/file2.pprof
ser_activity_benchmarks/testdata/input/pprof/file3.pprof
ser_activity_benchmarks/testdata/input/pprof_top/file1.pprof
ser_activity_benchmarks/testdata/input/pprof_top/file2.pprof
ser_activity_benchmarks/testdata/input/pprof_top/file3.pprof
ser_activity_benchmarks/testdata/input/pprof_top_csv/file1.csv
ser_activity_benchmarks/testdata/input/pprof_tree/file1.pprof
ser_activity_benchmarks/testdata/input/pprof_tree_csv/file1.csv
ser_activity_benchmarks/utils.py
ser_activity_benchmarks/utils_unittest.py
edc6eb87218afbde3c3df6e0faa99958f709c825 28-Sep-2016 Evelina Dumitrescu <evelinad@google.com> user_activity: Added script to select the optimal benchmark set.

It generates all possible combinations of benchmark sets of a given size
and it computes for every set a metric. Three different metrics have been
used: function count, distance variation and score.

TEST=None
BUG=None

Change-Id: I4e4af9f46a3056cfbcdc9d89b2c1333650b41ccc
Reviewed-on: https://chrome-internal-review.googlesource.com/292435
Reviewed-by: George Burgess <gbiv@google.com>
Reviewed-by: Ting-Yuan Huang <laszio@google.com>
Reviewed-by: Evelina Dumitrescu <evelinad@google.com>
Tested-by: Evelina Dumitrescu <evelinad@google.com>
Reviewed-on: https://chromium-review.googlesource.com/435914
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Luis Lozano <llozano@chromium.org>
ser_activity_benchmarks/benchmark_metrics.py
ser_activity_benchmarks/select_optimal_benchmark_set.py
e93f47d88b6af945db2953f5663a591b9994a92c 29-Sep-2016 Evelina Dumitrescu <evelinad@google.com> user_activity: Updated the extracted fields from the data collection scripts.

Added the inclusive count fields for the inclusive count statistics.
Renamed inclusive_count_fraction to inclusive_count for the pairwise
inclusive statistics.

BUG=None
TEST=None

Change-Id: Iac7d9a55443448ab32765039803a9e3e069c49ac
Reviewed-on: https://chrome-internal-review.googlesource.com/291203
Reviewed-by: George Burgess <gbiv@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Evelina Dumitrescu <evelinad@google.com>
Commit-Queue: Evelina Dumitrescu <evelinad@google.com>
Tested-by: Evelina Dumitrescu <evelinad@google.com>
Reviewed-on: https://chromium-review.googlesource.com/435913
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
ser_activity_benchmarks/collect_experiment_data.sh
ser_activity_benchmarks/collect_experiment_data_odd_even_session.sh
bf687a5ce7196c72ecf7d69b730658f4f2f2672a 29-Sep-2016 Evelina Dumitrescu <evelinad@google.com> user_activity: Integrated benchmark_metrics_experiment with utils.

Removed duplicate code from the process_hot_functions and
benchmark_metrics_experiment.

BUG=None
TEST=benchmark_metrics_experiment passes

Change-Id: I11ed11b4f1cbb573e1d10a2795570b4820602031
Reviewed-on: https://chrome-internal-review.googlesource.com/291202
Reviewed-by: George Burgess <gbiv@google.com>
Reviewed-by: Ting-Yuan Huang <laszio@google.com>
Reviewed-by: Evelina Dumitrescu <evelinad@google.com>
Tested-by: Evelina Dumitrescu <evelinad@google.com>
Reviewed-on: https://chromium-review.googlesource.com/435912
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Luis Lozano <llozano@chromium.org>
ser_activity_benchmarks/benchmark_metrics_experiment.py
ser_activity_benchmarks/benchmark_metrics_experiment_unittest.py
523b2ae25b5b98512babb5051b6f8f4dd92ef7cf 13-Sep-2016 Evelina Dumitrescu <evelinad@google.com> user_activity: Added implementation for the metrics experiment.

The validation of the benchmark metrics is done on a pair of CWP
function sets.

BUG=None
TEST=benchmark_metrics_validation_unittest passes

Change-Id: I15cb5976ba4d9a41ff97376b0ac3320e9a798fd5
Reviewed-on: https://chrome-internal-review.googlesource.com/286160
Reviewed-by: Luis Lozano <llozano@chromium.org>
Tested-by: Evelina Dumitrescu <evelinad@google.com>
Reviewed-on: https://chromium-review.googlesource.com/435911
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
ser_activity_benchmarks/benchmark_metrics_experiment.py
77abf01f2cff8bb3a5564f554d9218c085b83d65 13-Sep-2016 Evelina Dumitrescu <evelinad@google.com> user_activity: Added implementation for the metrics.

For every function, we compute a distance based on the sum of
inclusive value fraction differences of pairs of common child functions
from a sample, and reference function sets.

For every function, we assign a score based on the distance, inclusive
value fraction from the sample, and reference function sets.

For every Chrome OS component, we compute a set of metrics consisting
in the number of functions, the average, and cumulative distance and
the total score of the functions matching the group.

BUG=None
TEST=benchmark_metrics_unittest passes

Change-Id: I7810ec86bc4fa488b49ee18de3b3bda2ef95ba09
Reviewed-on: https://chrome-internal-review.googlesource.com/286161
Reviewed-by: Luis Lozano <llozano@chromium.org>
Tested-by: Evelina Dumitrescu <evelinad@google.com>
Reviewed-on: https://chromium-review.googlesource.com/435910
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
ser_activity_benchmarks/benchmark_metrics.py
daddec971693271ea429fce9026adc33dc71bcb6 16-Sep-2016 Evelina Dumitrescu <evelinad@google.com> user_activity: Added scripts for collecting the experiment data sets.

BUG=None
TEST=None

Change-Id: I7b00ce4359cb378e45bb673e023c12137d03e9b8
Reviewed-on: https://chrome-internal-review.googlesource.com/287959
Tested-by: Evelina Dumitrescu <evelinad@google.com>
Reviewed-by: George Burgess <gbiv@google.com>
Reviewed-by: Ting-Yuan Huang <laszio@google.com>
Reviewed-by: Evelina Dumitrescu <evelinad@google.com>
Reviewed-on: https://chromium-review.googlesource.com/435909
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Luis Lozano <llozano@chromium.org>
ser_activity_benchmarks/collect_experiment_data.sh
ser_activity_benchmarks/collect_experiment_data_odd_even_session.sh
9c4762d55252fe48671766435e01d5f21bf4719f 17-Sep-2016 Evelina Dumitrescu <evelinad@google.com> user_activity: Added unit tests for benchmark metrics.

BUG=None
TEST=None

Change-Id: I792f03a4cb76acf18691eeb1455a16a2d65537cf
Reviewed-on: https://chrome-internal-review.googlesource.com/287960
Tested-by: Evelina Dumitrescu <evelinad@google.com>
Reviewed-by: George Burgess <gbiv@google.com>
Reviewed-by: Ting-Yuan Huang <laszio@google.com>
Reviewed-by: Evelina Dumitrescu <evelinad@google.com>
Reviewed-on: https://chromium-review.googlesource.com/435908
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Luis Lozano <llozano@chromium.org>
ser_activity_benchmarks/benchmark_metrics_unittest.py
e9895d0f43b3078b847b8d495a7c1e8ae309d382 17-Sep-2016 Evelina Dumitrescu <evelinad@google.com> user_activity: Added unit tests for benchmark metrics experiment.

BUG=None
TEST=None

Change-Id: I8876f7ad4d4f05dad7e8329984f972a94df58edf
Reviewed-on: https://chrome-internal-review.googlesource.com/287961
Tested-by: Evelina Dumitrescu <evelinad@google.com>
Reviewed-by: George Burgess <gbiv@google.com>
Reviewed-by: Ting-Yuan Huang <laszio@google.com>
Reviewed-by: Evelina Dumitrescu <evelinad@google.com>
Reviewed-on: https://chromium-review.googlesource.com/435907
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Luis Lozano <llozano@chromium.org>
ser_activity_benchmarks/benchmark_metrics_experiment_unittest.py
ser_activity_benchmarks/testdata/input/inclusive_count_reference.csv
ser_activity_benchmarks/testdata/input/inclusive_count_test.csv
ser_activity_benchmarks/testdata/input/pairwise_inclusive_count_reference.csv
ser_activity_benchmarks/testdata/input/pairwise_inclusive_count_test.csv
fc3b78160c20dc5e0b50f0edb80901009c84aaa4 30-Aug-2016 Evelina Dumitrescu <evelinad@google.com> user activity: Unit tests for group extra CWP functions.

BUG=None
TEST=None

Change-Id: I6454b4f55a3c47387b1b2cc35342358937026cfb
Reviewed-on: https://chrome-internal-review.googlesource.com/282200
Commit-Queue: Evelina Dumitrescu <evelinad@google.com>
Tested-by: Evelina Dumitrescu <evelinad@google.com>
Reviewed-by: George Burgess <gbiv@google.com>
Reviewed-by: Ting-Yuan Huang <laszio@google.com>
Reviewed-on: https://chromium-review.googlesource.com/435906
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Luis Lozano <llozano@chromium.org>
ser_activity_benchmarks/process_hot_functions_unittest.py
eedb0d0aa34000af448d92d59f634df4d203f378 27-Aug-2016 Evelina Dumitrescu <evelinad@google.com> user activity: Normalize the pathname from pprof and CWP data.

I noticed that lots of files have .. symbols in the pathname.

BUG=None
TEST=None

Change-Id: I7dbdc84b36b9496f2d8fa21ec4fafe0f1417f256
Reviewed-on: https://chrome-internal-review.googlesource.com/282197
Tested-by: Evelina Dumitrescu <evelinad@google.com>
Commit-Queue: Evelina Dumitrescu <evelinad@google.com>
Reviewed-by: George Burgess <gbiv@google.com>
Reviewed-by: Ting-Yuan Huang <laszio@google.com>
Reviewed-on: https://chromium-review.googlesource.com/435905
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Luis Lozano <llozano@chromium.org>
ser_activity_benchmarks/process_hot_functions.py
ser_activity_benchmarks/testdata/input/parse_cwp_statistics.csv
ser_activity_benchmarks/testdata/input/pprof/file2.pprof
a78d63f8571cce07e39fc3ad50d8a49979413b9f 01-Sep-2016 Evelina Dumitrescu <evelinad@google.com> user activity: Added group extra CWP functions functionality.

Write the data of the functions that belong to a group in a CSV file,
sorted by their inclusive count value, in descendant order.

Create a CSV file containing the name of the groups, their common path,
the total inclusive count value of all the functions declared in files
that share the common path, sorted in descendant order by the inclusive
count value.

BUG=None
TEST=process_hot_functions_unittest passes

Change-Id: Ie0fe406ec0fefed02a5547dffa02e650dddd783a
Reviewed-on: https://chrome-internal-review.googlesource.com/282198
Reviewed-by: George Burgess <gbiv@google.com>
Reviewed-by: Ting-Yuan Huang <laszio@google.com>
Tested-by: Evelina Dumitrescu <evelinad@google.com>
Commit-Queue: Evelina Dumitrescu <evelinad@google.com>
Reviewed-on: https://chromium-review.googlesource.com/435904
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Luis Lozano <llozano@chromium.org>
ser_activity_benchmarks/process_hot_functions.py
10a5bbd90509148db04d3a84ba58ce297583fd35 27-Aug-2016 Evelina Dumitrescu <evelinad@google.com> user activity: Dremel query to extract the top CWP hot functions.

BUG=None
TEST=None

Change-Id: Iefa803394329f24e850e53e7db692bc004dafb51
Reviewed-on: https://chrome-internal-review.googlesource.com/282195
Tested-by: Evelina Dumitrescu <evelinad@google.com>
Reviewed-by: George Burgess <gbiv@google.com>
Reviewed-by: Ting-Yuan Huang <laszio@google.com>
Commit-Queue: Evelina Dumitrescu <evelinad@google.com>
Reviewed-on: https://chromium-review.googlesource.com/435457
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Luis Lozano <llozano@chromium.org>
ser_activity_benchmarks/select_hot_functions.sql
d8b488f9b1038b02ce98f2784134a7c31c772e87 24-Aug-2016 Evelina Dumitrescu <evelinad@google.com> user_activity_benchmarks: Added unit tests for the process_hot_functions
module.

BUG=None
TEST=None

Change-Id: I41fb0cd89e108f96cc061bf3ac72403a7108b30b
Reviewed-on: https://chrome-internal-review.googlesource.com/279955
Commit-Queue: Evelina Dumitrescu <evelinad@google.com>
Tested-by: Evelina Dumitrescu <evelinad@google.com>
Reviewed-by: George Burgess <gbiv@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/435456
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
ser_activity_benchmarks/process_hot_functions_unittest.py
ser_activity_benchmarks/testdata/expected/pprof_common/file1.pprof
ser_activity_benchmarks/testdata/expected/pprof_common/file2.pprof
ser_activity_benchmarks/testdata/expected/pprof_common/file3.pprof
ser_activity_benchmarks/testdata/input/cwp_functions_file.csv
ser_activity_benchmarks/testdata/input/parse_cwp_statistics.csv
ser_activity_benchmarks/testdata/input/pprof/file1.pprof
ser_activity_benchmarks/testdata/input/pprof/file2.pprof
ser_activity_benchmarks/testdata/input/pprof/file3.pprof
ser_activity_benchmarks/testdata/results/pprof_common/file1.pprof
ser_activity_benchmarks/testdata/results/pprof_common/file2.pprof
ser_activity_benchmarks/testdata/results/pprof_common/file3.pprof
9b33372c45fb525383a4e1e5cbb06904e5fb4c60 27-Aug-2016 Evelina Dumitrescu <evelinad@google.com> user_activity_benchmarks: Added file with function groups that
represent ChromeOS components.

A function belongs to a group that is defined by a given file path
if it is declared in a file that shares that path.

BUG=None
TEST=None

Change-Id: Ic5b9fd8871d0fc53f10590b63d9277ec2fce6dd8
Reviewed-on: https://chrome-internal-review.googlesource.com/282196
Tested-by: Evelina Dumitrescu <evelinad@google.com>
Reviewed-by: George Burgess <gbiv@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/435455
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
ser_activity_benchmarks/cwp_hot_functions_groups.txt
33d11c87bad5dc998d48bb2feb81b7e2918528e2 20-Aug-2016 Evelina Dumitrescu <evelinad@google.com> user_activity_benchmarks: Added tool for processing pprof and CWP data.

BUG=None
TEST=process_hot_functions_unitttest passes

Change-Id: I9956b34e9d1ca902a23be86dd8e21208462f0117
Reviewed-on: https://chrome-internal-review.googlesource.com/278697
Reviewed-by: Luis Lozano <llozano@chromium.org>
Tested-by: Evelina Dumitrescu <evelinad@google.com>
Reviewed-on: https://chromium-review.googlesource.com/435454
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
ser_activity_benchmarks/process_hot_functions.py
3594db8be24f785b80781bfa43bacb4186cf8b30 31-Jan-2017 Manoj Gupta <manojgupta@google.com> Fix WebGLAquarium fails from trybot images.

WebGLAquarium fails since it can't find autotest packages.
But we don't want to download more files to chrotomation2.
So reuse autotest files from vanilla image for trybot images.

BUG=chromium:687262
TEST=autotest_path added to generated experiment files.

Change-Id: I99b139451c2d39da928554747f714a6e536c7b40
Reviewed-on: https://chromium-review.googlesource.com/435438
Commit-Ready: Manoj Gupta <manojgupta@chromium.org>
Tested-by: Manoj Gupta <manojgupta@chromium.org>
Reviewed-by: Ting-Yuan Huang <laszio@chromium.org>
uildbot_test_toolchains.py
9e32aa6f93c7f703dd0352b66da1a0126c023338 27-Jan-2017 Yunlian Jiang <yunlian@google.com> add caroline and kevin boards to rotation builder.

Bug=None
Test=The boards are added.

Change-Id: Ie8619c84c0e4d8377153d8c14e3b3f5f7a082723
Reviewed-on: https://chromium-review.googlesource.com/434118
Commit-Ready: Yunlian Jiang <yunlian@chromium.org>
Tested-by: Yunlian Jiang <yunlian@chromium.org>
Reviewed-by: Caroline Tice <cmtice@chromium.org>
uildbot_test_llvm.py
4762fbb38cdb6a86aeb66b04b710f29a097bd5b0 23-Jan-2017 Manoj Gupta <manojgupta@google.com> Cleanup stale files from chrotomation2.

Remove images and autotest_files directories left in
chromeos chroot *-pfq directories.
Slight refactoring to avoid code duplication for release
and pfq directories.
Minor mandatory formatting updates.

BUG=chromium:656755
TEST=Old files show up for removal in dry run

Change-Id: I91230c0696148847dc408a3047f468c663b8ab8d
Reviewed-on: https://chromium-review.googlesource.com/431268
Commit-Ready: Manoj Gupta <manojgupta@chromium.org>
Tested-by: Manoj Gupta <manojgupta@chromium.org>
Reviewed-by: Caroline Tice <cmtice@chromium.org>
uto_delete_nightly_test_data.py
4e25342bd03fc8f5df7e4e4fe33649de82526730 14-Jan-2017 Manoj Gupta <manojgupta@google.com> Fix frequency pinning on crosperf.

Crosperf was pinning frequency to maximum based on cpu0
information. This does not work as intended for arm
Big-Little cores. Instead do it for each cpu based on its
own information.

BUG=chromium:680742
TEST=Tested on elm, kevin, falco, samus, peach_pit and veyron_jaq.

Change-Id: I77fc76d8f1e94563fe12b3555bf08511fdbfdc32
Reviewed-on: https://chromium-review.googlesource.com/428186
Commit-Ready: Manoj Gupta <manojgupta@chromium.org>
Tested-by: Manoj Gupta <manojgupta@chromium.org>
Reviewed-by: Luis Lozano <llozano@chromium.org>
rosperf/suite_runner.py
rosperf/suite_runner_unittest.py
5775e7e3e85529f18220d5d6bc47df6c2fc79a6a 23-Jan-2017 Yunlian Jiang <yunlian@google.com> remove perf binary from toolchain-utils.

This reverts this CL
https://chrome-internal-review.googlesource.com/#/c/157756/,
because the bug in perf is gone.

BUG=chromium:665988
TEST=run a crosperf with perf, the report report is correct.
No 20-50% of the report attributed to 'chrome [unknown]'.
run_test.sh passes.

Change-Id: I4b2b18e9653d95bde5caf076114bf53a84c2a4fd
Reviewed-on: https://chromium-review.googlesource.com/431234
Commit-Ready: Yunlian Jiang <yunlian@chromium.org>
Tested-by: Yunlian Jiang <yunlian@chromium.org>
Reviewed-by: Caroline Tice <cmtice@chromium.org>
rosperf/results_cache.py
rosperf/results_cache_unittest.py
erf.static
3fb928efb42405e386da41adf4bd674e86e23022 09-Jan-2017 Manoj Gupta <manojgupta@google.com> Fix test_that run for non-telemtry tests.

Do not add the autotest_dir argument to test_that
for non-telemetry tests.
Minor change: Remove a leftover print from download_images
unit test.

BUG: chromium:679001
TEST:crosperf unit tests
Change-Id: I0ff5365902af40097b61ae57d7c0d8d43ea308a8
Reviewed-on: https://chromium-review.googlesource.com/426073
Reviewed-by: Caroline Tice <cmtice@chromium.org>
Commit-Queue: Manoj Gupta <manojgupta@chromium.org>
Tested-by: Manoj Gupta <manojgupta@chromium.org>
Trybot-Ready: Manoj Gupta <manojgupta@chromium.org>
rosperf/download_images_unittest.py
rosperf/suite_runner.py
rosperf/suite_runner_unittest.py
70204447c3c103508f1b8e58d59a9b8f165e1c21 16-Dec-2016 Luis A. Lozano <llozano@google.com> Fix some lint warning in suite_runner*.

BUG=chromium:632113
TEST=run_tests

Change-Id: I9fe070420153f5ae175db86961c95b1686ea7c8f
Reviewed-on: https://chromium-review.googlesource.com/421107
Reviewed-by: Caroline Tice <cmtice@chromium.org>
Commit-Queue: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
rosperf/suite_runner.py
rosperf/suite_runner_unittest.py
8316e3e6a41936eb14a609bb2b04987f13aedb36 10-Jan-2017 Ting-Yuan Huang <laszio@chromium.org> crosperf: disable cheets benchmarks in crosbolt

They have nothing to do with cros toolchain.

TEST=none
BUG=none

Change-Id: Ie06ccd1cddf7233c55b52fd77e1329311fcff9d9
Reviewed-on: https://chromium-review.googlesource.com/426379
Trybot-Ready: Ting-Yuan Huang <laszio@chromium.org>
Reviewed-by: Yunlian Jiang <yunlian@chromium.org>
Commit-Queue: Ting-Yuan Huang <laszio@chromium.org>
Tested-by: Ting-Yuan Huang <laszio@chromium.org>
rosperf/experiment_factory.py
fa9abaf1258205f90cc566069ad94f12cff6ecf1 28-Dec-2016 Manoj Gupta <manojgupta@google.com> Reduce cros_sdk calls in crosperf.

Improves crosperf performance by reducing calls to ChrootRunCommand,
Instead use RunCommand.

BUG: chromium:564889
TEST:crosperf unit tests
Change-Id: I0705273d1126b81167d7a789514771122a71b8d3
Reviewed-on: https://chromium-review.googlesource.com/424088
Reviewed-by: Caroline Tice <cmtice@chromium.org>
Commit-Queue: Manoj Gupta <manojgupta@chromium.org>
Tested-by: Manoj Gupta <manojgupta@chromium.org>
rosperf/download_images.py
rosperf/download_images_unittest.py
9949a46570763c933053cc28693c72145d70664a 04-Jan-2017 Caroline Tice <cmtice@google.com> Add llvm_next_builder to list of known builders.

This will fix the problem with rotating builders; they were
updated to use the llvm_next builder, but this script was not updated
to use llvm_next.

BUG=chromium:674804
TEST=Tested in role account on chrotomation2, launching rotating
builder.

Change-Id: Ifd4ba57cf403feae0b0b84c8b21accb34987ecbc
Reviewed-on: https://chromium-review.googlesource.com/424893
Reviewed-by: Yunlian Jiang <yunlian@chromium.org>
Commit-Queue: Caroline Tice <cmtice@chromium.org>
Tested-by: Caroline Tice <cmtice@chromium.org>
ros_utils/buildbot_utils.py
53930064c8b68258872af1646c7fe18112ca494a 03-Jan-2017 Yunlian Jiang <yunlian@google.com> buildbot_utils.py: do not get build_id for async build.

The build_id is not useful for async builds, so there is no need to
try to get it.

BUG=chromium:674804
TEST=None

Change-Id: If5781955834ca35958b1efead2dd9628e7a905de
Reviewed-on: https://chromium-review.googlesource.com/424794
Reviewed-by: Ting-Yuan Huang <laszio@chromium.org>
Commit-Queue: Yunlian Jiang <yunlian@chromium.org>
Tested-by: Yunlian Jiang <yunlian@chromium.org>
ros_utils/buildbot_utils.py
c411035e2f13ed71d2dc0238f5cba31443d03d84 28-Dec-2016 Manoj Gupta <manojgupta@google.com> Update ChromeOS Toolchain buildbot for llvm-next jobs.

Update toolchain buildbot script to report data for
llvm-next builds.

BUG=chromium:677317
TEST=None

Change-Id: I45b25a735ee8ba956d38c1e45d0db2d8acd06e70
Reviewed-on: https://chromium-review.googlesource.com/424317
Commit-Queue: Manoj Gupta <manojgupta@chromium.org>
Tested-by: Manoj Gupta <manojgupta@chromium.org>
Reviewed-by: Yunlian Jiang <yunlian@chromium.org>
uildbot_test_toolchains.py
6b8bacbba4df2ac7b98074978f30658ce81bba72 15-Dec-2016 Yunlian Jiang <yunlian@google.com> toolchain-utils: Enable to print waterfall summary by date.

BUG=chromium:674727
TEST=the '--date' option is added.

Change-Id: Ia28faa8fd41145fbb27b79065518eb68da654f90
Reviewed-on: https://chrome-internal-review.googlesource.com/311924
Commit-Ready: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
enerate-waterfall-reports.py
e58198707a6524ad3f731c61c9d5ef140a342451 15-Dec-2016 Ting-Yuan Huang <laszio@google.com> buildbot_test_toolchains: Use LATEST-master as control group

Rather than finding the official image with exact the same version
as the trybot image, let's just use the latest one since trybot
always get lastest or latest + 1.

This also solves the problem that the official image specified may
not exist.

BUG=chromium:674652
TEST=Tested ToolchainComparator._GetVanillaImageName locally.

Change-Id: I44fbceb99d8f6cf3187366c075c2a9522869b5a5
Reviewed-on: https://chrome-internal-review.googlesource.com/312516
Commit-Ready: Ting-Yuan Huang <laszio@google.com>
Tested-by: Ting-Yuan Huang <laszio@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
uildbot_test_toolchains.py
ros_utils/buildbot_utils.py
9c0b33bc875650e8f27709e66e43b42e09b6f6fe 15-Dec-2016 Manoj Gupta <manojgupta@google.com> Remove local references to cmtice files.

Previous change to script left references to test directories
in personal location.

BUG=None
TEST=No error when fetching files

Change-Id: I33347a0fc2a09ccb66ae2e8a4b229f5adaf29aaf
Reviewed-on: https://chrome-internal-review.googlesource.com/311869
Commit-Ready: Manoj Gupta <manojgupta@google.com>
Tested-by: Manoj Gupta <manojgupta@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
enerate-waterfall-reports.py
faa3c555dd0b636cdd19a42399b152fc77b8cbe3 13-Dec-2016 Caroline Tice <cmtice@google.com> Add argument options to waterfall summary report.

Now there are options to generate just the main waterfall report, or
just the rotating builders report, with or without the failures
section; or just the failures report section. There's also the option
to NOT update the data files (makes it easier to re-run reports). Also,
replaces the llvm rotating builder with the llvm-next rotating builder.

BUG=chromium:673833
TEST=Tested in my directory.

Change-Id: If784758112c36c3d18c4b292f9229b53b047848c
Reviewed-on: https://chrome-internal-review.googlesource.com/311775
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
enerate-waterfall-reports.py
63824523d7f5b3afa6a8e939db3503cf867e7c40 14-Dec-2016 Manoj Gupta <manojgupta@google.com> Check for prodacess before running waterfall summary report.

Check if prodaccess has not expired. If it is expired,
error out and ask to run prodacess.

BUG=chromium:674218
TEST=Locally tested

Change-Id: I20129c69c17956ed6edda57949ea49546c9b1d85
Reviewed-on: https://chrome-internal-review.googlesource.com/312040
Commit-Ready: Manoj Gupta <manojgupta@google.com>
Tested-by: Manoj Gupta <manojgupta@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
enerate-waterfall-reports.py
e84ea3d3e9fbe92d9854a85d67ab01fb488781c7 12-Dec-2016 Yunlian Jiang <yunlian@google.com> toolchain-utils: add build link to daily trybot logs.

This adds build link to daily trybot logs. It also waits 2 hours
before polling the building status to trybot.

BUG=chromium:673419
TEST=The build link is added in the log.

Change-Id: I8fc0ea413ee0f40b398d6eb80c3308c5a1e7d793
Reviewed-on: https://chrome-internal-review.googlesource.com/311042
Commit-Ready: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
uildbot_test_toolchains.py
ros_utils/buildbot_utils.py
7a6bb17630ea86f12c1045c2c47339b02e79bdc2 09-Dec-2016 Ting-Yuan Huang <laszio@google.com> crosperf: Add all_graphics_perf and all_crosbolt_perf

BUG=none
TEST=Run them on samus.

Change-Id: I2d66351d80ce87f84924fe3bf55abf903fc54829
Reviewed-on: https://chrome-internal-review.googlesource.com/311196
Commit-Ready: Ting-Yuan Huang <laszio@google.com>
Tested-by: Ting-Yuan Huang <laszio@google.com>
Reviewed-by: Ting-Yuan Huang <laszio@google.com>
rosperf/experiment_factory.py
c0194f9187813b63b31ad9e54649ee259c2231e5 08-Dec-2016 Rahul Chaudhry <rahulchaudhry@chromium.org> default_remotes: update device addresses after the recent move.

context: http://g/c-compiler-chrome/duTz8l1GR8w

BUG=None
TEST=Pinged the new addresses successfully.

Change-Id: Ie80855629d238a55b21fe78f2fdbfe4e585e577f
Reviewed-on: https://chrome-internal-review.googlesource.com/310183
Commit-Ready: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
Reviewed-by: Ting-Yuan Huang <laszio@google.com>
rosperf/default_remotes
34915f5308b7e81b6927a58a3c8fcabc6b20b546 07-Dec-2016 Caroline Tice <cmtice@google.com> [toolchain-utils] Add missing import statement.

BUG=chromium:672207
TEST=Tested in my directory.

Change-Id: I90166be8cd2c30d2643904af96b320162b59d3e0
Reviewed-on: https://chrome-internal-review.googlesource.com/310104
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
erify_compiler.py
a8fd4f7693fbb616e73bb3d1b0c121d201de8156 06-Dec-2016 Manoj Gupta <manojgupta@google.com> Cleanup stale files on chrotomation2.

Remove cros-update* and autotest_files directories left in chromeos
chroot. Improve auto_delete script dry run to include chroot tmp
cleanup. Minor mandatory formatting updates.

BUG=chromium:656755
TEST=Old files show up for removal in dry run

Change-Id: I9d190f3220fc68a8f2cb437c08ae9506960f4983
Reviewed-on: https://chrome-internal-review.googlesource.com/310057
Commit-Ready: Manoj Gupta <manojgupta@google.com>
Tested-by: Manoj Gupta <manojgupta@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
uto_delete_nightly_test_data.py
213f3c07b696bce9a54b2039c855a9f593cdb511 06-Dec-2016 Rahul Chaudhry <rahulchaudhry@chromium.org> generate-waterfall-reports: use full path to sendgmr.

Use the sendgmr binary from x20. The user running the script
may not have (and does not need to have) sendgmr installed
on the local workstation.

BUG=None
TEST='Email sent by the script arrived in Inbox'

Change-Id: Ie642dea6317bdd3d5517ef4a383c8bae7b496f15
Reviewed-on: https://chrome-internal-review.googlesource.com/310215
Commit-Ready: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
enerate-waterfall-reports.py
baba90fd78c18585d22430dc95c748f96ad0c772 06-Dec-2016 kuantung <kuantung@google.com> Initial empty repository
e54c75c2f1b65054ce7562132102f194cf1c7d48 02-Dec-2016 Caroline Tice <cmtice@google.com> [toolchain-utils] Stop copying files into weekly-reports dir.

Now that we no longer run/use the weekly reports, we should stop
copying the data into the weekly reports directories.

BUG=chromium:656755
TEST=None

Change-Id: I93e371e2c04cd607f47b31ef2eb86be21b5883a1
Reviewed-on: https://chrome-internal-review.googlesource.com/308836
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
uildbot_test_toolchains.py
est_toolchains.py
c7ee80e420941341fedaf8e620776f5b4a9df9df 30-Nov-2016 Caroline Tice <cmtice@google.com> [binary search tool] Allow non-flashing images to be fails.

Currently if an image is so bad that it can't be flashed, it gets
marked as 'skipped', which is different from bad/failed. This can
cause the binary search to never find the bad image. We need an
option to treat a non-flashing image as a failure. That's what
this CL does.

BUG=None
TEST=Tested by Manoj: without this change, binary search failed to
find any bad packages, but with this change it worked.

Change-Id: Ib223338a1820314e85352827857fb8f3ba1ed73a
Reviewed-on: https://chrome-internal-review.googlesource.com/307824
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Manoj Gupta <manojgupta@google.com>
inary_search_tool/android/test_setup.sh
inary_search_tool/binary_search_state.py
inary_search_tool/common/test_setup.sh
e02e9f819648419cc1d6e59d02ddf50d16f0ca22 01-Dec-2016 Caroline Tice <cmtice@google.com> [toolchain-utils] Update waterfall report (email, archive, pruning).

The CL udpates the waterfall summary report script to: 1). Send an
email containing the reports to whoever ran the script; 2). Copy the
generated reports to
/google/data/rw/users/mo/mobiletc-prebuild/waterfall-reports
so everyone on the team can see all the reports; 3). Prunes failures
that are older than 7 days from the failure dictionary.

BUG=None
TEST=Tested in my own directory.

Change-Id: I624f2f6f7575bd1095969282e8cdcec4558b9191
Reviewed-on: https://chrome-internal-review.googlesource.com/308300
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
enerate-waterfall-reports.py
a8517eb004edaffb972a429915dac2b30bb22dff 30-Nov-2016 Caroline Tice <cmtice@google.com> [toolchain-utils] Space ARM rotating builds more evenly.

This CL reorders the rotating builders to space the ARM
builds out more evenly, and also adds more data about
each board being built.

BUG=None
TEST=None

Change-Id: Icf158a28b4b41689ec6031e8be37c537d5836dd8
Reviewed-on: https://chrome-internal-review.googlesource.com/308295
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
uildbot_test_llvm.py
f0ad65cb9c6310c51a6970c50826ec0b8a7df646 29-Nov-2016 Caroline Tice <cmtice@google.com> [toolchain-utils] Add test suite colors to waterfall report.

This CL updates the daily waterfall summary report with the color of
each test suite (green = all passed, red = something failed,
orange = all passed, with some retries).

BUG=None
TEST=Tested generating the reports in my own tree.

Change-Id: I4747ec505a8991e638667ba9744e2572a85c0c4e
Reviewed-on: https://chrome-internal-review.googlesource.com/307858
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
enerate-waterfall-reports.py
48462067d260ce85bb77565cf62b471ab71d389f 19-Nov-2016 Caroline Tice <cmtice@google.com> [toolchain-utils] Add new script for generating waterfall reports.

There are still things that should be done to improve this, but the
basic script is working now.

BUG=chromium:649175
TEST=Tested extensively in my account.

Change-Id: I4b0e833d635047293b74b7971a1b5d3f2676b440
Reviewed-on: https://chrome-internal-review.googlesource.com/305905
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
enerate-waterfall-reports.py
9eabe97c2dfaf0c472e3bf43849817c668b0b70a 22-Nov-2016 Manoj Gupta <manojgupta@google.com> Fix crosperf generating duplicate key entries.

Fixes crosperf when reading data from cache.
Fix string find check for json file.

BUG: chromium:658875
TEST:crosperf unit tests
Change-Id: I34529102c29ed88efd848beb0363066f3b6e50f0
Reviewed-on: https://chrome-internal-review.googlesource.com/305910
Commit-Ready: Manoj Gupta <manojgupta@google.com>
Tested-by: Manoj Gupta <manojgupta@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/results_cache.py
c145de0147859ddc9f7b82c451c2dda36a0433d2 22-Nov-2016 Manoj Gupta <manojgupta@google.com> Fix crosperf generating duplicate key entries.

Fixes crosperf when reading data from cache.
Find and assign result data file if it exists in cache.
Fix string find check for json file.

Some formatting changes required by tc_python and lint.

BUG: chromium:658875
TEST:crosperf unit tests
Change-Id: Ib13e083464062c16c8a2f1921446441c9973e41a
Reviewed-on: https://chrome-internal-review.googlesource.com/305866
Commit-Ready: Manoj Gupta <manojgupta@google.com>
Tested-by: Manoj Gupta <manojgupta@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/results_cache.py
ac9bd776a41cc591eb47c9d588f236a9aeb7b218 15-Nov-2016 Manoj Gupta <manojgupta@google.com> Do no exit or raise exception if autotest files are not found.

If autotest files are missing from server, just emit an warning
and continue.

BUG=chromium:665483
TEST=crosperf unit tests

Change-Id: Ieaec9b53cbd270bc50f2c5b412b153ddc568104d
Reviewed-on: https://chrome-internal-review.googlesource.com/304808
Commit-Ready: Manoj Gupta <manojgupta@google.com>
Tested-by: Manoj Gupta <manojgupta@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
rosperf/download_images.py
b187da39bb5da01bcb729c0bc2b2921c581fa7e3 11-Nov-2016 Caroline Tice <cmtice@google.com> [toolchain-utils] Use new system for accessing build logs.

The old system for accessing build logs is being deprecated and
turned off. This CL switches over to using the new recommended
system for accessing build log records.

BUG=chromium:664537
TEST=Tested in my account & role account.

Change-Id: Id964c43ce09df9c52a6337ea57fb469423c3cf51
Reviewed-on: https://chrome-internal-review.googlesource.com/304258
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
ros_utils/buildbot_utils.py
ros_utils/chromeos-toolchain-credentials.json
fd4e283258e169a10a749db4f2ee141366c7e379 09-Nov-2016 Manoj Gupta <manojgupta@google.com> Crosperf: Remove tar file after uncompressing image

Remove downloaded chromium image.tar.xz file after uncompression.
Change 2 step untar and uncompress to 1. Catch exceptions during
uncompression to avoid having a corrupted bin file if
uncompression is interrupted.

BUG: chromium:663426
TEST: crosperf unit tests pass
Change-Id: I82f674a26bf99cb3491e7db770664c73c004a93b
Reviewed-on: https://chrome-internal-review.googlesource.com/303875
Commit-Ready: Manoj Gupta <manojgupta@google.com>
Tested-by: Manoj Gupta <manojgupta@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/download_images.py
rosperf/download_images_unittest.py
c00e1d61dd82a9f9eab25e45b888b6f6966904b2 09-Nov-2016 Manoj Gupta <manojgupta@google.com> Fix crosperf crashes when running non-telemetry test on R53 or older images.

Do not call decrease wait time for non-telemetry tests.

BUG: chromium:663435
TEST:crosperf unit tests
Change-Id: Ia13eb8f258ab8fc09fdfbdb4b93a6745e06b44b1
Reviewed-on: https://chrome-internal-review.googlesource.com/303736
Commit-Ready: Manoj Gupta <manojgupta@google.com>
Tested-by: Manoj Gupta <manojgupta@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/suite_runner.py
097419758febae084477a01fd92d678218b578e5 02-Nov-2016 Caroline Tice <cmtice@google.com> [toolchain-utils] Update rotating testers to launch asynchronously.

Currently the rotating testers launch one trybot job, then wait for
it to finish (to get the trybot image name) before launching the
second job. Now that all the testing happens directly in the
buildbot HWTest stages, this waiting in not needed. This CL removes
the synchronous waiting from the rotating testers. (Allows the two
jobs to run in parallel).

BUG=None
TEST=None

Change-Id: I6c96c684dc69b205b847b584b4bbc9e10cf5313f
Reviewed-on: https://chrome-internal-review.googlesource.com/302137
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
uildbot_test_llvm.py
ros_utils/buildbot_utils.py
4b133961b76c2cb8bc58f0ea2cded9e3438ffb6f 02-Nov-2016 Manoj Gupta <manojgupta@google.com> Download autotest files with crosperf and add autotest path to test_that
when running non-telemetry tests

BUG: chromium:647429
TEST:crosperf unit tests, sample experiment file that runs WebGl_Aquarium

Change-Id: I067e350fee36596ce269b954773c39a3868632aa
Reviewed-on: https://chrome-internal-review.googlesource.com/302296
Commit-Ready: Manoj Gupta <manojgupta@google.com>
Tested-by: Manoj Gupta <manojgupta@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/benchmark_run_unittest.py
rosperf/download_images.py
rosperf/download_images_unittest.py
rosperf/experiment_factory.py
rosperf/experiment_factory_unittest.py
rosperf/experiment_file.py
rosperf/label.py
rosperf/machine_manager_unittest.py
rosperf/mock_instance.py
rosperf/results_cache_unittest.py
rosperf/settings.py
rosperf/settings_factory.py
rosperf/settings_factory_unittest.py
rosperf/settings_unittest.py
rosperf/suite_runner.py
rosperf/suite_runner_unittest.py
c39917fe61858e884d06656122cf88a8c66fd825 01-Nov-2016 Manoj Gupta <manojgupta@google.com> Update formatting of files before committing changes for autotest. No functional change.

BUG: chromium:647429
TEST: crosperf unit tests
Change-Id: I827f9ec4d00a0d8c1ae8a9930d7832289e694dc8
Reviewed-on: https://chrome-internal-review.googlesource.com/301457
Commit-Ready: Manoj Gupta <manojgupta@google.com>
Tested-by: Manoj Gupta <manojgupta@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
rosperf/benchmark_run_unittest.py
rosperf/download_images.py
rosperf/download_images_unittest.py
rosperf/experiment_file.py
rosperf/machine_manager_unittest.py
rosperf/mock_instance.py
rosperf/results_cache_unittest.py
rosperf/settings_factory.py
rosperf/suite_runner.py
rosperf/suite_runner_unittest.py
d768dc1c6dbd2cdc8f2503574e88ad88316e24f2 28-Oct-2016 Caroline Tice <cmtice@google.com> [crosperf] Remove stumpy machines.

We are decommissioning our stumpy lab machines and returning them
to the ChromeStop. This CL removes the stumpy machines from the
default_remotes file.

BUG=chromium:660536
TEST=None

Change-Id: I5ecf1536fa399ff70a89271394c8d4e6e09e64e1
Reviewed-on: https://chrome-internal-review.googlesource.com/301258
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
rosperf/default_remotes
rosperf/experiment_factory_unittest.py
6c438e04d12f5dd94dd060c9ab41a61766dc313e 27-Oct-2016 Caroline Tice <cmtice@google.com> [crosperf] Fix buildbot_utils for rotating tester waterfall.

The rotating testers trybot waterfall location has moved. This CL
updates the script to look for their build logs in the correct
builder location. It also makes sure we don't crash/die because
their build status is non-zero (ie. a HWTest failed). This also fixes
a tiny bug in the results reporting.

BUG=chromium:660085
TEST=Tested in the role account.

Change-Id: I396ec0ad588e54e88bc44dcc0e72eda45d077173
Reviewed-on: https://chrome-internal-review.googlesource.com/300598
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
ros_utils/buildbot_utils.py
rosperf/results_report.py
e82513b0aec27bf5d3ca51789edc48dde5ee439b 27-Oct-2016 Caroline Tice <cmtice@google.com> [toolchain-utils] Update page_cycler in nightly tests.

Currently page_cycler_v2.typical_25 tries to run too many times,
causing some of the nightly builders to take nearly 24 hours to
complete (they take about 6 hours without page_cycler). Also,
page_cycler_v2.typical_25 cannot run locally on some boards (such
as daisy).

This CL removes page_cycler_v2.typical_25 from the all_toolchain_perf
set, and adds it separately to the nightly tests, with only 2
iterations, no retries, and forcing it to not run locallys on the DUT.

BUG=chromium:660087
TEST=Tested the new page cycler settings in a crosperf run.

Change-Id: Ia68abce4e8ee29a671e58db13f269e531d31736e
Reviewed-on: https://chrome-internal-review.googlesource.com/300855
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
uildbot_test_toolchains.py
rosperf/experiment_factory.py
aee96b71ad4ffde231ca5bf9c0509d15b0e9b753 24-Oct-2016 Manoj Gupta <manojgupta@google.com> Handle exceptions in crosperf.

Change error message in buildbot.
Catch exceptions in crosperf exceution to do a graceful exit.

BUG=chromium:622875
TEST=ran run_tests

Change-Id: I03093ab27edcc85e318ad43097ab8cd316792dcb
Reviewed-on: https://chrome-internal-review.googlesource.com/299275
Commit-Ready: Manoj Gupta <manojgupta@google.com>
Tested-by: Manoj Gupta <manojgupta@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
uildbot_test_toolchains.py
rosperf/crosperf.py
est_toolchains.py
de600776d8342db1ae2579efd4ca07adbc310d6c 19-Oct-2016 Caroline Tice <cmtice@google.com> [toolchain-utils] Fix rotating builders to work with no patches.

Currently if no patch is passed to cbuildbot, it hangs; similarly if
no patch is given to the rotating builder script, it dies. This CL
fixes both of those. It also temporarily stops the rotating builders
from sending emails until we can figure out how to fix their reports.

BUG=chromium:657060
TEST=Tested changes in role account;seemed to work.

Change-Id: I4ba70b562f9cb44d396361cd6461230d7080c6e9
Reviewed-on: https://chrome-internal-review.googlesource.com/297584
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Ting-Yuan Huang <laszio@google.com>
uildbot_test_llvm.py
ros_utils/buildbot_utils.py
bf3d6f017149ee218f731d21b9bf076637733348 04-Oct-2016 Caroline Tice <cmtice@google.com> [toolchain-utils] New script to verify which compiler was used.

This script takes either a single file or a directory tree and
verifies that all the compilation units in the file(s) were built
with the specified compiler.

BUG=chromium:652835
TEST=Tested on subtree within a build tree.

Change-Id: Ib365b9c4590b539f5f515df24f6214846183d9e7
Reviewed-on: https://chrome-internal-review.googlesource.com/293115
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Ting-Yuan Huang <laszio@google.com>
erify_compiler.py
36f73cbff7a9d74745a6c9278e01bbe3977bd1fe 12-Oct-2016 Ting-Yuan Huang <laszio@google.com> crosperf: update lab machine names.

Two elm are added.

TEST=none
BUG=chromium:655227

Change-Id: I2475aeb0e7aa4964b93babfb6fb626a2de7f5dcb
Reviewed-on: https://chrome-internal-review.googlesource.com/296135
Commit-Ready: Ting-Yuan Huang <laszio@google.com>
Tested-by: Ting-Yuan Huang <laszio@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/default_remotes
1ba6d5764234e1df0ff56a256a2e2cae94f28353 10-Oct-2016 Caroline Tice <cmtice@google.com> [toolchain-utils] Add --hwtest flag to rotating tester builds.

The rotating testers need to add --hwtest to the trybot builds to
get the hwtests to actually run. This CL adds that flag.

BUG=None
TEST=Tested in role account.

Change-Id: I6ef93f529b6616dd8d1c357743593ed369d1022a
Reviewed-on: https://chrome-internal-review.googlesource.com/295075
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Ting-Yuan Huang <laszio@google.com>
uildbot_test_llvm.py
4bd70465c2e5c60802460aec4fdc698e1cfab8e3 06-Oct-2016 Caroline Tice <cmtice@google.com> [toolchain-utils] Update rotating testers to use new configs.

This CL updates the rotating testers to use the new chromes_configs
for building trybots & running HW tests.

BUG=chromium:653296
TEST=Ran scripts in my directory; it worked.

Change-Id: Ia60af8b921a966c86b335ef51db5372bf740ed95
Reviewed-on: https://chrome-internal-review.googlesource.com/293775
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
uildbot_test_llvm.py
ros_utils/buildbot_utils.py
59b328726cdf961f8c839fed049f6df31d614d0b 06-Oct-2016 Caroline Tice <cmtice@google.com> [crosperf] Fix xbuddy to work with new -toolchain builders.

The xbuddy translating code was improperly prepending to board
name to the waterfall builder trybot images (the ones with the
generic architecture names). This CL fixes that.

BUG=chromium:653210
TEST=Tested crosperf with 'build: x86-llvm-toolchain/R55-8865.0.0-rc1';
failed without this CL, worked with it.

Change-Id: I706c9d882f9c99f8af11eb1782ea6edd1eedbfe5
Reviewed-on: https://chrome-internal-review.googlesource.com/293815
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
rosperf/settings.py
33da0c4d4b1186d1795dabb037de2e721b95a816 05-Oct-2016 Yunlian Jiang <yunlian@google.com> crosperf: fix the error unexpected keyword argument 'command'

BUG=none
TEST=the error is gone.

Change-Id: I8dad69d2b45ed821be0c7be77e07b33cf0663eb9
Reviewed-on: https://chrome-internal-review.googlesource.com/293515
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
rosperf/suite_runner.py
9198db666212c457d231f2db6dd534d25f9bbc67 01-Oct-2016 Yunlian Jiang <yunlian@google.com> crosperf: Decrease 10 seconds delay for pagecyclers.

New telemetry adds 10 seconds delay after rendering a page.
We do not need that so this CL changes the 10 secnods delay
to 2 seconds.

BUG=None
TEST=the 10 seconds delay is changed to 2 seconds..

Change-Id: I5c835acdf4068fba1aceb0d2a5a45e7113a558d3
Reviewed-on: https://chrome-internal-review.googlesource.com/291818
Reviewed-by: Ting-Yuan Huang <laszio@google.com>
Commit-Queue: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
rosperf/suite_runner.py
0a6266710aa7133cb8ef105575b8fb9e74f7896d 27-Sep-2016 Caroline Tice <cmtice@google.com> [bisection tool] Don't make failure to flash bad image fatal error.

Currently during initial setup, if the known 'bad' image fails to
flash onto the device, the user is given the option to 'skip'.
This causes the entire bisection to fail/stop. This CL changes
the skip to allow bisection to continue (it is not unreasonable for
a bad image to fail to flash).

BUG=chromium:641539
TEST=None

Change-Id: Iddc5312ecf593192d563766d677e8d401824ae7d
Reviewed-on: https://chrome-internal-review.googlesource.com/290617
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Ting-Yuan Huang <laszio@google.com>
inary_search_tool/binary_search_state.py
33b6586b464e590d85781f63bd6bca9518deeaf4 26-Sep-2016 Caroline Tice <cmtice@google.com> [crosperf] Add new results fields for other page_cycler_v2 tests.

This Cl adds the default result fields for the other page_cycler_v2
tests (already done for page_cycler_v2.typical_25). It also temporarily
removes some tests from the all_pagecyclers set, as they appear to
have been removed from telemetry. Will remove these permanently when
this is confirmed.

BUG=chromium:650307
TEST=Ran crosperf with all_pagecyclers.

Change-Id: I9e6c3ae140f66a11d792350b74c8897df74d944c
Reviewed-on: https://chrome-internal-review.googlesource.com/289865
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
rosperf/default-telemetry-results.json
rosperf/experiment_factory.py
4bf85d17e38c3f36f64c761a0743591d296f41a5 23-Sep-2016 George Burgess IV <gbiv@google.com> [crosperf] Make generate_report report actual failures.

We would never increment `num_success`, so we'd always return failure.
This passed tests, since there were no tests that checked for success.

Also, testRunActionsRunsAllActionsRegardlessOfExceptions wasn't
really doing what it was supposed to; exceptions wouldn't get raised
(since the mocked WriteFile turns into a nop), so all of the WriteFile
actions were "passing". Because of the aforementioned bug, this test
seemed to work, as well.

BUG=None
TEST=generate_report_unittest.py passes; cros lint is happy.

Change-Id: If4d71af3415d97be8d5ac3bec796fe4589d50aaa
Reviewed-on: https://chrome-internal-review.googlesource.com/289802
Commit-Ready: George Burgess <gbiv@google.com>
Tested-by: George Burgess <gbiv@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/generate_report.py
rosperf/generate_report_unittest.py
0347c840ed3cbf878947d086d14d2d0ffa27e92c 15-Sep-2016 Caroline Tice <cmtice@google.com> [toolchain-utils] Add licensing information.

BUG=b:31321592
TEST=None

Change-Id: I9d6b407f389c7cb42007cb7ded59b8261fb70b49
Reviewed-on: https://chrome-internal-review.googlesource.com/287475
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Luis Lozano <llozano@chromium.org>
ICENSE
EADME.chromium
be2c71afb92e76414957f36a8d4ecfdf0cf750e6 23-Sep-2016 Yunlian Jiang <yunlian@google.com> rotation_tester: use toolchain-tests instead of paygen_au_canary

BUG=None
TEST=it uses toolchain-tests now.

Change-Id: I5b0e350e0fc0b365be5e65587b62e022f342338e
Reviewed-on: https://chrome-internal-review.googlesource.com/289837
Commit-Ready: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
uildbot_test_llvm.py
23e403298416d3036eab179a94d210441e69cc6e 22-Sep-2016 Caroline Tice <cmtice@google.com> Revert "[crosperf] Remove --autotest_dir arg from non-telemetry tests."

This reverts commit 70462462d54070a2440fccb506dcea2ea076cc05.

Upon further discussion I realize this is the wrong fix for this problem.

Change-Id: I7e9c94e763378e4834858acd826985c55bfcea4d
Reviewed-on: https://chrome-internal-review.googlesource.com/289636
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
rosperf/suite_runner.py
4598b132f8815ad899e9506dbdfec2e177242b48 22-Sep-2016 Yunlian Jiang <yunlian@google.com> cros-utils: fix an out of bound error.

The input rgb2short can reach to 255. When the value exceeds 250,
it causes out of bound error. This CL fixes that.

BUG=chromium:649374
TEST=The out of bound error is fixed.

Change-Id: Iee31337e82907e1803a84b5837542733bb224e8f
Reviewed-on: https://chrome-internal-review.googlesource.com/289536
Commit-Ready: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
ros_utils/misc.py
91c9393de517beb79e86272048d9c7d75a94065f 22-Sep-2016 Caroline Tice <cmtice@google.com> [crosperf] Update default values for page_cycler test.

page_cycler_v2.typical_25 has different field names than
page_cycler v1. This updates the default field names used by
crosperf to generate reports, to use the new field names for
page_cycer.

BUG=None
TEST=Tested in my source tree.

Change-Id: Id91fa7ad7d2ebc915ccad054bb2408a22629272a
Reviewed-on: https://chrome-internal-review.googlesource.com/289235
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
rosperf/default-telemetry-results.json
0c023884a22be7135a17ba950833fb957b741990 15-Sep-2016 Yunlian Jiang <yunlian@google.com> toolchain-utils: remove colortrans.py

This removes colortrans.py from cros_utils. We add a single
function inside misc.py instead.

BUG=chromium:647308
TEST=crosperf still works, the color text is shown on terminal.

Change-Id: I1871bd2c57608f59ded6a8cf720c52cb2c07c3ff
Reviewed-on: https://chrome-internal-review.googlesource.com/287538
Commit-Ready: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
ros_utils/colortrans.py
ros_utils/misc.py
ros_utils/tabulator.py
70462462d54070a2440fccb506dcea2ea076cc05 16-Sep-2016 Caroline Tice <cmtice@google.com> [crosperf] Remove --autotest_dir arg from non-telemetry tests.

Some autotests that are not telemetry tests will fail if we pass
the --autotest_dir arg. This CL removes that arg from non-telemetry
tests.

BUG=chromium:647429
TEST=ran graphics_WebGLAquarium ok.

Change-Id: I00c79a6dc8cc4642b5b9dd342a92df91d0cb9c2c
Reviewed-on: https://chrome-internal-review.googlesource.com/287597
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
rosperf/suite_runner.py
6b16138b3b3f1e5f91fecf54589bf08e49500d31 16-Sep-2016 Caroline Tice <cmtice@google.com> [crosperf] Don't block on local server if not using local machines.

Occasionally the local AFE server goes down; this currently causes
the afe_lock_machine.py script to fail, whether the user is trying
to lock a local machine or not. This CL fixes that, so if the user
specified a HW Lab machine, we don't try to look at the local
server at all. Also cleans up some formatting.

BUG=chromium:647427
TEST=Tested getting status of lab machine when local server was
down.

Change-Id: Iff7799b2deea142bcbd85f0e3373cf0462590b72
Reviewed-on: https://chrome-internal-review.googlesource.com/287456
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
fe_lock_machine.py
7248b3f00937ced8385259923aa038fc00d413cb 14-Sep-2016 Caroline Tice <cmtice@google.com> [bisection] Fix various minor bugs in Android bisection.

Recent testing has found a few minor bugs in the Android
bisection tool. This CL fixes those bugs.

BUG=chromium:646948
TEST=Ran full Android bisection.

Change-Id: I5b228105ec719db89f0ab34b29fde11dbb1e5074
Reviewed-on: https://chrome-internal-review.googlesource.com/287195
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
inary_search_tool/android/setup.sh
inary_search_tool/android/test_setup.sh
inary_search_tool/bisect_driver.py
6395d39689b42e519aa37c6ac9ca0fedd765b391 14-Sep-2016 Caroline Tice <cmtice@google.com> [crosperf] Don't run kernel test in llvm rotating testsers.

Update the nightly rotating testers to not run the kernel tests
for the LLVM images, since the kernel is not built with LLVM

BUG=None
TEST=None

Change-Id: Ib2efe2211d6c9e86b80db2470c428d70e30e5516
Reviewed-on: https://chrome-internal-review.googlesource.com/286800
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
uildbot_test_llvm.py
ed69676d435b7b6983271ed8fab200627a0b966e 13-Sep-2016 George Burgess IV <gbiv@google.com> crosperf+cros_utils: Swap to #!/usr/bin/env.

`cros lint` was recently updated to complain about
using #!/usr/bin/python{,2,3} directly. Instead, it prefers
/usr/bin/env python{2,3} now.

BUG=None
TEST=./run_tests.sh passes; linter seems much happier with crosperf.

Change-Id: I9b88a7af1f8e03b7c870781c3d8dc1b872a3dfc3
Reviewed-on: https://chrome-internal-review.googlesource.com/286341
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Luis Lozano <llozano@chromium.org>
ros_utils/buildbot_json.py
ros_utils/command_executer_unittest.py
ros_utils/email_sender.py
ros_utils/perf_diff.py
rosperf/benchmark_run_unittest.py
rosperf/benchmark_unittest.py
rosperf/config_unittest.py
rosperf/crosperf.py
rosperf/crosperf_test.py
rosperf/crosperf_unittest.py
rosperf/download_images_buildid_test.py
rosperf/download_images_unittest.py
rosperf/experiment_factory_unittest.py
rosperf/experiment_file_unittest.py
rosperf/experiment_runner_unittest.py
rosperf/flag_test_unittest.py
rosperf/generate_report_unittest.py
rosperf/machine_image_manager_unittest.py
rosperf/machine_manager_unittest.py
rosperf/results_cache_unittest.py
rosperf/results_organizer_unittest.py
rosperf/results_report_unittest.py
rosperf/schedv2_unittest.py
rosperf/settings_factory_unittest.py
rosperf/settings_unittest.py
rosperf/suite_runner_unittest.py
c9df56c133c35bb5154a6c947d0986c35097c6e1 08-Sep-2016 George Burgess IV <gbiv@chromium.org> crosperf: Add generate_report.py.

This script allows us to generate HTML, JSON, text, and email reports
for JSON inputs, rather than needing to run things through crosperf.

BUG=chromium:641098
TEST=cros lint on new files passes, ./run_tests.sh passes.

Change-Id: I213149c0feca8b97f72fb447030c670e2f631a87
Reviewed-on: https://chrome-internal-review.googlesource.com/285238
Commit-Ready: George Burgess <gbiv@google.com>
Tested-by: George Burgess <gbiv@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/generate_report.py
rosperf/generate_report_unittest.py
43fe499320c0d77c7cd2b3189896a0d9791821c5 08-Sep-2016 George Burgess IV <gbiv@chromium.org> Fix test failures in results_report refactors.

...I have absolutely no clue how I missed these failures (since I had to
run results_report_unittest to run the new perf parser test), but
apparently I did. The failure happened because we had a _chart_js arg
name, which didn't match up with chart_js (in the caller).

This also fixes a few minor linter complaints that I introduced in
patch-set 2 of said review. All of the other files I touched are clean
(except for a complaint about use of _logger in one of them, but that
was there before I made my changes).

BUG=None
TEST=./run_tests.sh *actually* passes this time. I promise. :)

Change-Id: I98e43b29ff9cef5fc9cf33143adc26b98a861f3e
Reviewed-on: https://chrome-internal-review.googlesource.com/285436
Commit-Ready: George Burgess <gbiv@google.com>
Tested-by: George Burgess <gbiv@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/results_report_unittest.py
a12e97482b0c8b21805a468109e1b5fa7d424304 08-Sep-2016 Caroline Tice <cmtice@google.com> [Crosperf] Fix typo & update boards.

This CL fixes a typo in the rotating tester tests, and updates the
board list. Also reformats script (using tc_pyformat).

BUG=None
TEST=None

Change-Id: I0a21bd8fd9d20d5d7dcd49e3484201c1860d352f
Reviewed-on: https://chrome-internal-review.googlesource.com/285456
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
uildbot_test_llvm.py
2368b41ea869a6904ae8320ad69f1b8b9a9a3714 25-Aug-2016 George Burgess IV <gbiv@google.com> crosperf: Make results_report more general.

The goal of this patch is to allow us to use results_report to generate
non-ChromeOS-specific results reports. It's meant to be a nop for
regular crosperf runs (except that, in HTML report generation,
whitespace in the HTML itself will look a bit different.)

Moreover, results_report used to shuffle Experiments around, each of
which, unsurprisingly, contained *tons* of data about an experiment.
So, part of this patch was reducing results_report's reliance on
Experiments, in favor of a new type, BenchmarkResult.

Some parts of results_report still rely on Experiments, but only to
provide *extra* data. The minimum amount of data needed to make a
results report is contained in a BenchmarkResult, and there's a
convenient API we can use to make a BenchmarkResult out of an
Experiment.

This patch also does a massive refactor of results_report, because lots
of the code was mildly more icky than it is with this patch applied.
The refactor-for-prettiness and refactor-for-BenchmarkResults kind of
go hand-in-hand, so it's really hard to split them out. The only part
that's not so difficult to split out is the refactor to
results_report_templates, but the value of splitting that out is
questionable (to me).

Speaking of which, all HTML magicks that were in results_report is now
in results_report_templates, and now use *actual* Templates. It -- and
HTMLReportGenerator -- are hopefully more readable as a result.

Next, this makes JSONRsultsReport's GetReport() return a string
containing JSON data, rather than calling a callback to write
that string to a file.

Finally, this includes a change to perf_table. Namely, its removal. It
was otherwise unused, and I couldn't get it to even work, so I cleaned
it up, made a perf report parser (because I couldn't get the cros_utils
one to work, either...), and made sure it functions. If we're able to
produce perf reports that cros_utils can parse, I'm happy to back my
new parser out; I was primarily using it so I could be sure I didn't
break HTML report generation.

BUG=chromium:641098
TEST=./run_tests.sh passes

Change-Id: I437de9eb39e00c9dd5c223ecd27feaaab544a6fd
Reviewed-on: https://chrome-internal-review.googlesource.com/282217
Commit-Ready: George Burgess <gbiv@google.com>
Tested-by: George Burgess <gbiv@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/experiment_runner.py
rosperf/experiment_runner_unittest.py
rosperf/perf_files/perf.data.report.0
rosperf/perf_table.py
rosperf/results_report.py
rosperf/results_report_templates.py
rosperf/results_report_unittest.py
a3e8787abdde02848ff987b67fa9ac0b22e24b7b 03-Sep-2016 Yunlian Jiang <yunlian@chromium.org> cros_utils: increast timeout for trybot.

This increase time out for trybot jobs to 8 hours.

BUG=None
TEST=None

Change-Id: Ieed04fa7082e4d71ee1ed9cee62e6eb33e6763a9
Reviewed-on: https://chrome-internal-review.googlesource.com/283936
Commit-Ready: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Ting-Yuan Huang <laszio@google.com>
ros_utils/buildbot_utils.py
f12b7da4343ceda0fe52c950a44bc6a25867c742 31-Aug-2016 Yunlian Jiang <yunlian@chromium.org> crosperf: fix appears to have succeeded but returned no results error

This fixes the error appears to have succeeded but returned no results
for a set of benchmarks.

BUG=chromium:642525
TEST=the results appear for these benchmarks

Change-Id: Icad1e1da5881f664cdd96b4cfc3e9ac590db087d
Reviewed-on: https://chrome-internal-review.googlesource.com/282696
Commit-Ready: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/default-telemetry-results.json
38e6afbc9be12a3a3337f39fa30ddba337fb250c 29-Aug-2016 Luis Lozano <llozano@chromium.org> Update owner file to include everyone in the team.

BUG=None
TEST=None

Change-Id: I1caa63c014921a09358cb115d56a3333fa858be9
Reviewed-on: https://chrome-internal-review.googlesource.com/281539
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Caroline Tice <cmtice@google.com>
WNERS
479a91d1a637ec4e355699c1726806f01ac09cdd 29-Aug-2016 George Burgess IV <gbiv@google.com> crosperf: Fix a failing test.

An earlier commit added extra spacing between remotes in
crosperf/default_remotes. This resulted in
experiment_factory_unittests.py reporting that there were 5 remotes in
some cases (2 of which were ''), since we were splitting on one space.

It seems sensible to split on any amount of whitespace, instead.

BUG=None
TEST=./run_tests.sh passes.

Change-Id: I95bd85ba92e5e1f5b30b0273b1b4a6c8853b1995
Reviewed-on: https://chrome-internal-review.googlesource.com/281538
Commit-Ready: George Burgess <gbiv@google.com>
Tested-by: George Burgess <gbiv@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/experiment_factory.py
afb8cc77e82c35faedfe541d097fc01fd1d7ca3d 25-Aug-2016 George Burgess IV <gbiv@google.com> crosperf: refactor results_organizer; NFC.

As part of playing with results_report (so I could make it less
ChromeOS-dependent), I started refactoring results_organizer, which is
only used by results_report.

This was the result. The goal was to make it easier to understand what,
exactly, results_organizer is trying to do.

BUG=chromium:641098
TEST=./test_all.sh passes.

Change-Id: I8e53175fbc1ded6f23631550223905b3321376ca
Reviewed-on: https://chrome-internal-review.googlesource.com/280815
Commit-Ready: George Burgess <gbiv@google.com>
Tested-by: George Burgess <gbiv@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/results_organizer.py
rosperf/results_organizer_unittest.py
rosperf/results_report.py
2f4a226d58efdf4eabe560a64e711c16599db08d 26-Aug-2016 Rahul Chaudhry <rahulchaudhry@chromium.org> crosperf: update lab machine names.

Updated default_remotes to reflect the latest locations of all toolchain machines from
https://sites.google.com/a/google.com/englabs/resources/infrastructure/chromeos/chromeos-devices/special-duts/toolchain

BUG=chromium:639389
TEST=None

Change-Id: I1da60a961560551a6a63f92f38798f7153197089
Reviewed-on: https://chrome-internal-review.googlesource.com/281375
Commit-Ready: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/default_remotes
1ced193baabbf6d6aa1aa3913bcff714e5ae376d 23-Aug-2016 Yunlian Jiang <yunlian@chromium.org> crosperf: fix telemetry results parsing.

New telemetry results provide multiple values for a single key.
This CL uses the average value to make crosperf work.

BUG=chromium:639896
TEST=run crosperf with 2 iterations of speedometer.

Change-Id: I32e7a00a57dbbe995915af0c32918f1d680ae484
Reviewed-on: https://chrome-internal-review.googlesource.com/279057
Commit-Ready: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
ros_utils/tabulator.py
rosperf/results_cache.py
eb9fce674ff90a6de04827bfe1ef6e07a99c8f61 19-Aug-2016 Caroline Tice <cmtice@google.com> [crosperf] Fix reading from cache.

The read-from-cache code needed to be updated to deal with the new
telemetry results format, as did the code for generating json
reports (for archiving in the database).

BUG=chromium:639106
TEST=Tested with crosperf reading from cache & generating json results.

Change-Id: I2c97e5b393583048cc74559d065f498c0f0675a1
Reviewed-on: https://chrome-internal-review.googlesource.com/278298
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
rosperf/results_cache.py
rosperf/results_report.py
2adc55674e2444d371d5e351e500acaced490eea 19-Aug-2016 Caroline Tice <cmtice@google.com> [crosperf] Update lab machine names/locations.

This CL updates the lab machine names & locations now that they
have been moved.

BUG=chromium:639389
TEST=pinged all the machines

Change-Id: Id30c8635ea3b33e789294f731f28ee2a88b5f17a
Reviewed-on: https://chrome-internal-review.googlesource.com/278119
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
rosperf/default_remotes
3b0ea76081a892ad757b65db753e739b2362ecd2 11-Aug-2016 Cassidy Burden <cburden@google.com> binary search tool: Update ChromeOS Object bisector

Update ChromeOS Object bisector to work with new compiler wrapper. This
new compiler wrapper means we can now do incremental switches and triage
builds that skip compiler calls.

TEST=Run full bisection
CQ-DEPEND=CL:368231

Change-Id: If34877519ad6dfe3189a5e934e174fd13dae5c89
Reviewed-on: https://chrome-internal-review.googlesource.com/274236
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
inary_search_tool/bisect.py
inary_search_tool/sysroot_wrapper/switch_to_bad.sh
inary_search_tool/sysroot_wrapper/switch_to_good.sh
882e707d7e4f63c8e64fa554e77303c4800258c7 16-Aug-2016 Caroline Tice <cmtice@google.com> [crosperf] Fix telemetry results parsing.

Many telemetry results fields now contain multiple values. This CL
fixes crosperf to accomodate that.

BUG=None
TEST=Ran crosperf with change and it worked.

Change-Id: Iccb8926223c8447b72d51f413756f5dbc09f89de
Reviewed-on: https://chrome-internal-review.googlesource.com/276817
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
ros_utils/tabulator.py
rosperf/default-telemetry-results.json
rosperf/results_cache.py
babb65232cbb7de2365d117f3593cdea8a263f05 27-Jul-2016 Caroline Tice <cmtice@google.com> [crosperf] Change 'page_cycler.' to 'page_cycler_v2.' everywhere.

Telemetry is switching to using page_cyler_v2 and will stop supporting
old page_cycler soon. This CL switches everything in toolchain-utils
to reference page_cycler_v2.

BUG=None
TEST=Ran unittests in crosperf with no issues.

Change-Id: I0e3ce29bfaf483507a9ba528c6c7c5a8fa00f00e
Reviewed-on: https://chrome-internal-review.googlesource.com/271598
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/default-telemetry-results.json
rosperf/experiment_factory.py
rosperf/experiment_files/README
rosperf/experiment_files/bloat_perf.exp
rosperf/experiment_files/morejs_perf.exp
rosperf/experiment_files/page_cycler_perf.exp
rosperf/experiment_files/telemetry-crosperf-with-profiler.exp
rosperf/experiment_files/telemetry_perf_perf
rosperf/help.py
rosperf/machine_manager_unittest.py
4cc3dd8a3b2ab04cf861d7c71967de0298746cb9 12-Aug-2016 Cassidy Burden <cburden@google.com> binary search tool: Run tc_pyformat on all python scripts

REFACTOR WARNING! Run tc_pyformat on all python scripts.

TEST=Run unit tests, Android bisection

Change-Id: I9624e7be2670bdb29bcdb59404bf9e8c2f7d81c5
Reviewed-on: https://chrome-internal-review.googlesource.com/274615
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
inary_search_tool/binary_search_perforce.py
inary_search_tool/binary_search_state.py
inary_search_tool/bisect.py
inary_search_tool/common.py
inary_search_tool/cros_pkg/create_cleanup_script.py
inary_search_tool/test/binary_search_tool_tester.py
inary_search_tool/test/gen_init_list.py
inary_search_tool/test/gen_obj.py
b217c8a28be52f90572d26647d38dcfc9e08ce0b 09-Aug-2016 Cassidy Burden <cburden@google.com> binary search tool: Add sample NDK bisection

Add sample NDK bisection that can be given to those inside or outside of
Google. This sample includes a compressed source tree of the Teapot
sample NDK app, and a script showing the full setup and process for
running the bisection tool. It also includes a couple patches that show
the necessary changes to the build system, and the "compiler error" that
we will be bisecting.

TEST=Run DO_BISECTION.sh

Change-Id: I11279fe33e833b0dfe2fb147fde6f46b347d62be
Reviewed-on: https://chrome-internal-review.googlesource.com/274145
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
inary_search_tool/ndk/DO_BISECTION.sh
inary_search_tool/ndk/PATCH1
inary_search_tool/ndk/PATCH2
inary_search_tool/ndk/README
inary_search_tool/ndk/Teapot.tar.gz
inary_search_tool/ndk/boot_test.sh
inary_search_tool/ndk/get_initial_items.sh
inary_search_tool/ndk/switch_to_good.sh
inary_search_tool/ndk/test_setup.sh
a19ea38e4482af3bc98365be773305b5167eddfd 10-Aug-2016 Cassidy Burden <cburden@google.com> binary search tool: Make 1st resumed iteration non-incremental

If resuming a bisection, we want to make sure all current good items are
switched to good and all current bad items are switched to bad. This is
so the bisection tool will resume in a known, stable state. Incremental
builds are essentially diffs from previous builds, and the base
environment may have changed since the last execution.

TEST=Add test code, run unit tests

Change-Id: I9ac855efbcfb48bc86f245f33ee29d0c38aa7af9
Reviewed-on: https://chrome-internal-review.googlesource.com/274175
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
inary_search_tool/binary_search_state.py
inary_search_tool/test/binary_search_tool_tester.py
b9f97db723647b5e4507eac2e0ae53869da57d4a 11-Aug-2016 Caroline Tice <cmtice@google.com> [crosperf] Update parsing of telemetry json file.

They've updated the format of the output in the json file. This
updates crosperf to work with either the old or the new format.

BUG=None
TEST=./run_tests and ran crosperf successfully.

Change-Id: I0b22f4c740943c55c0c521ab715e0b70fbbdfc69
Reviewed-on: https://chrome-internal-review.googlesource.com/274195
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
rosperf/results_cache.py
e56ceb4e7387a7f7ccb6d8d3b03e3e9692453cbd 09-Aug-2016 George Burgess IV <gbiv@google.com> Random nit fixes in crosperf; NFC.

This also removes results_sorter.py, which seems to be unused and
untested.

TEST=./run_tests.sh passes
BUG=None

Change-Id: I926affb23c78a8a0aedea4ab279da590f8cf93b5
Reviewed-on: https://chrome-internal-review.googlesource.com/273995
Commit-Ready: George Burgess <gbiv@google.com>
Tested-by: George Burgess <gbiv@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/benchmark_run.py
rosperf/experiment.py
rosperf/experiment_status.py
rosperf/machine_manager.py
rosperf/results_cache.py
rosperf/results_sorter.py
rosperf/schedv2.py
458dda2951fbffd8def8bc04b9568458fbe22e52 08-Aug-2016 Cassidy Burden <cburden@google.com> binary search tool: Add maintenance document

Change-Id: I2c7e10f4f0622569490f97fd9406e1deb5940041
Reviewed-on: https://chrome-internal-review.googlesource.com/273451
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
inary_search_tool/MAINTENANCE
6e57349f7dbbde843bd3d14cb09b09ed75446aa1 10-Aug-2016 Cassidy Burden <cburden@google.com> binary search tool: Fix edge cases in compiler wrapper

Fix compiler wrapper edge cases that exist with the NDK and
ChromeOS build systems. For ChromeOS we support the -MMD flag when
caching dependency files. For NDK we support calls to compiler that
give no output file (these calls are meant to check compiler
capabilities).

TEST=NDK bisection and ChromeOS population

Change-Id: Ic23bc7876e9844eacb106d49082e0509ea1ee31b
Reviewed-on: https://chrome-internal-review.googlesource.com/273985
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
inary_search_tool/bisect_driver.py
inary_search_tool/compiler_wrapper.py
a5e3929d7a155606bc541777fc84e4989389e13c 08-Aug-2016 Cassidy Burden <cburden@google.com> binary search tool: Rename install to test_setup

Refactor warning! Rename install script to be test_setup script.

TEST=Run unit tests

Change-Id: Iab7e01136fdcf2dedbfc94ece6ddb4b4fa3c0f68
Reviewed-on: https://chrome-internal-review.googlesource.com/273484
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
inary_search_tool/README.bisect
inary_search_tool/android/README.android
inary_search_tool/android/boot_test.sh
inary_search_tool/android/install.sh
inary_search_tool/android/interactive_test.sh
inary_search_tool/android/test_setup.sh
inary_search_tool/binary_search_state.py
inary_search_tool/bisect.py
inary_search_tool/common.py
inary_search_tool/common/boot_test.sh
inary_search_tool/common/hash_test.sh
inary_search_tool/common/install.sh
inary_search_tool/common/interactive_test.sh
inary_search_tool/common/test_setup.sh
inary_search_tool/cros_pkg/README.cros_pkg_triage
inary_search_tool/cros_pkg/install.sh
inary_search_tool/sysroot_wrapper/install.sh
inary_search_tool/test/binary_search_tool_tester.py
inary_search_tool/test/common.py
inary_search_tool/test/install.py
inary_search_tool/test/install_bad.py
inary_search_tool/test/is_good.py
inary_search_tool/test/is_good_noinc_prune.py
inary_search_tool/test/test_setup.py
inary_search_tool/test/test_setup_bad.py
d0b69bedbb9eb087ab231adbbe3af1ba26a945a3 08-Aug-2016 George Burgess IV <gbiv@google.com> Tiny refactor of a function; NFC.

TEST=./run_tests.sh passes
BUG=None

Change-Id: I15220e3ac0f3b6b8a932e2d8dcfaf105be7a9794
Reviewed-on: https://chrome-internal-review.googlesource.com/273450
Commit-Ready: George Burgess <gbiv@google.com>
Tested-by: George Burgess <gbiv@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/config.py
a8f9ceefac8a7088f3bb1154c8add6d24d5e62da 02-Aug-2016 Cassidy Burden <cburden@google.com> binary search tool: Add option to always run compiler

Similar to sysroot_wrapper, add option that will always call compiler
during triage stage. This option is safer, but much slower.

TEST=Test with NDK app with missing side effects

Change-Id: I242d7267c52e50989bcc5968f132956ba1cd3105
Reviewed-on: https://chrome-internal-review.googlesource.com/273457
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
inary_search_tool/bisect_driver.py
inary_search_tool/compiler_wrapper.py
f1fe83c3276136a7b7dcd7e499e67b4a16e4a5f1 05-Aug-2016 Cassidy Burden <cburden@google.com> binary search tool: Clean up output

Make output of binary search tool less confusing, provide PROGRESS
output for each verification run, make output for found items cleaner.

TEST=Run unit tests

Change-Id: I65628fb7147419ad5381726217e649a3e16a1cab
Reviewed-on: https://chrome-internal-review.googlesource.com/273446
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
inary_search_tool/binary_search_state.py
3892aa1c459f07a5355a8aa94c7ab3c5f6e5cf39 05-Aug-2016 Cassidy Burden <cburden@google.com> binary search tool: Show how to reconstruct bisection env

When bisect.py finishes, give users instructions on how to reconstruct
bisection environment. This essentially gives the same setup.sh call
that bisect.py uses.

Change-Id: I060a4f3397e7d83d83321d25e751d6c47951436b
Reviewed-on: https://chrome-internal-review.googlesource.com/273407
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
inary_search_tool/bisect.py
e1a28bd82a8718b67a370433dd41932c567f6899 03-Aug-2016 Caroline Tice <cmtice@google.com> [Crosperf] Fix lint errors.

This fixes the crosper lint errors that were assigned to me.

BUG=chromium:632110
TEST=ran run_tests.sh; did full crosperf run.

Change-Id: I68d136c7399eb4befa2e36e7f513612ca5e53769
Reviewed-on: https://chrome-internal-review.googlesource.com/272755
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
rosperf/experiment.py
rosperf/machine_image_manager_unittest.py
6c921aec5a85f8de8d6e45f7bd8fdae08717348c 03-Aug-2016 Cassidy Burden <cburden@google.com> binary search tool: Properly cache all side effects

Properly cache all side effects in compiler wrapper. Extract some common
caching logic to their own functions (cache_file/restore_file).

TEST=Full Android bisection, and full NDK bisection with dwo files

Change-Id: I4c3be2cb5dae1922c145ba6be85ae65d1ed5cda2
Reviewed-on: https://chrome-internal-review.googlesource.com/273115
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
inary_search_tool/bisect_driver.py
a0955f22087ecdab7a7f51909cc424725b3643dd 04-Aug-2016 Cassidy Burden <cburden@google.com> binary search tool: Support @file compiler arguments

Gradle passes arguments to gcc/clang by using the @file argument.
Implement expansion of this argument so that bisect_driver can analyze
all arguments being sent to the compiler.

TEST=Tested with NDK sample app

Change-Id: I1e1dae001a062c118b6d5c36e367612c9e4116b7
Reviewed-on: https://chrome-internal-review.googlesource.com/272935
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
inary_search_tool/compiler_wrapper.py
b1d0c4e00ed787befa9591231ac5d04d2cf6795d 03-Aug-2016 Cassidy Burden <cburden@google.com> binary search tool: Change verify_level to boolean

Previously, the user could run multiple levels of verification. This
would essentially repeat verification multiple times. It's unsure why a
user would want to set verify_level to any value other than 0 or 1. Thus
verify_level is more appropriate as a boolean.

TEST=Run unit tests

Change-Id: Id8832c54e2733881bf23f48af2a3b4c294117d1c
Reviewed-on: https://chrome-internal-review.googlesource.com/272766
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
inary_search_tool/README.bisect
inary_search_tool/binary_search_state.py
inary_search_tool/common.py
inary_search_tool/test/binary_search_tool_tester.py
2a141a72ae3f5d420f892cbc7a81fc40db076022 02-Aug-2016 Cassidy Burden <cburden@google.com> binary search tool: Use full obj path with compiler wrapper

In order to support the NDK, use absolute paths for object files.

TEST=Run full Android bisection

Change-Id: I588230fa5cd521cd7c7d48b87e8bab3e3b09b1aa
Reviewed-on: https://chrome-internal-review.googlesource.com/272756
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
inary_search_tool/android/switch_to_bad.sh
inary_search_tool/android/switch_to_good.sh
inary_search_tool/bisect_driver.py
2e307b303ac37c3e971086f23da2df25fef2c1b4 02-Aug-2016 Ting-Yuan Huang <laszio@google.com> crosperf: fix lint warnings

BUG=chromium:632098
TEST=cros lint [files]

Change-Id: Ieac300bf7cb3b560fd8caaaacc3714696ca5567f
Reviewed-on: https://chrome-internal-review.googlesource.com/272616
Commit-Ready: Ting-Yuan Huang <laszio@google.com>
Tested-by: Ting-Yuan Huang <laszio@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/download_images.py
rosperf/machine_image_manager.py
rosperf/results_organizer_unittest.py
711a224060336a4e1dcade76233312f08dec811b 28-Jul-2016 Cassidy Burden <cburden@google.com> binary search tool: Add android bisector documentation

Change-Id: I8f9c954accdac769d2a696f31fb1d90d50264eaa
Reviewed-on: https://chrome-internal-review.googlesource.com/272011
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
inary_search_tool/README.bisect
inary_search_tool/android/README.android
471aae79081b9a9097e5f57c8d6792f72853b03f 29-Jul-2016 Cassidy Burden <cburden@google.com> binary search tool: Add android bisector to bisect.py

Add android bisector implementation to bisect.py. Additionally add
simple cleanup.sh script.

TEST=Tested with full android bisection

Change-Id: Ib6c6ea85614dbe3d54865d9695863ee625588a2d
Reviewed-on: https://chrome-internal-review.googlesource.com/272166
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
inary_search_tool/android/cleanup.sh
inary_search_tool/bisect.py
a742f71e420a07c474a8fb9307426ddb0574dcf3 01-Aug-2016 Cassidy Burden <cburden@google.com> binary search tool: Change skip exit status to 125

Because exit status 2 means syntax error, it is confusing to also use
exit status 2 to mean "skip image". Change uses of status 2 to status
125.

Change-Id: I3620e3eb6e52523dc51804076284abf1e023d14f
Reviewed-on: https://chrome-internal-review.googlesource.com/272241
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
inary_search_tool/android/boot_test.sh
inary_search_tool/android/interactive_test.sh
inary_search_tool/binary_search_perforce.py
inary_search_tool/binary_search_state.py
inary_search_tool/common/boot_test.sh
inary_search_tool/common/interactive_test.sh
inary_search_tool/sysroot_wrapper/glibc_test_script.sh
inary_search_tool/sysroot_wrapper/test_script.sh
660e682dae1e34076cb758694be67edfc7bc6ed4 26-Jul-2016 Cassidy Burden <cburden@google.com> binary search tool: Fix off by one bug, add stress tests

Revert change that introduced bug when bad item is very last item in
list. Add tests that stress limits of binary search tool and try to look
for these off-by-one errors. Specifically one test checks for the bad
item being in every index of the binary search. Another tests checks for
if every single item is bad.

TEST=Add two new stress tests, run all unit tests

Change-Id: I2d5a0bda035b2c2b4994b0378aa416da19db651d
Reviewed-on: https://chrome-internal-review.googlesource.com/271916
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
inary_search_tool/binary_search_state.py
inary_search_tool/test/binary_search_tool_tester.py
inary_search_tool/test/gen_obj.py
25fcdc012532b8208cec9cf854de1d6df3cf0793 29-Jul-2016 Cassidy Burden <cburden@google.com> binary search tool: export TRIAGE stage for object bisector

Change-Id: I3c026f070f702cad1ab061a3ec20f19668775e8e
Reviewed-on: https://chrome-internal-review.googlesource.com/272009
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
inary_search_tool/sysroot_wrapper/setup.sh
12d0f2398963f4b9eaded60786422a455f199e45 30-Jul-2016 Yunlian Jiang <yunlian@chromium.org> crosperf: fix lint warnings.

This fixes the lint warnings from these four files.

BUG=chromium:632108
TEST=the lint warnings are gone.

Change-Id: Ia923a1e9b8b5ec5d48a02caa1f8ed611857f0ec5
Reviewed-on: https://chrome-internal-review.googlesource.com/272021
Commit-Ready: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/experiment_file.py
rosperf/experiment_file_unittest.py
rosperf/flag_test_unittest.py
rosperf/test_flag.py
40ca64bca93984da431ff83739467553bcf7882e 29-Jul-2016 Rahul Chaudhry <rahulchaudhry@chromium.org> toolchain-utils: remove uses of deprecated-module 'optparse'.

BUG=chromium:632109
TEST='./run_tests.py passes'

Change-Id: I30393ff8f835f7a9a414b8b1785fa600cb6f50f0
Reviewed-on: https://chrome-internal-review.googlesource.com/272174
Commit-Ready: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/download_images_buildid_test.py
34da9b80e2858b6ed6f76c9a23be6ce7df580ace 29-Jul-2016 Rahul Chaudhry <rahulchaudhry@chromium.org> toolchain-utils: fix *-docstring-* lint errors.

BUG=chromium:632109
TEST='./run_tests.py passes'

Change-Id: I008ad5784bf58fdf58c92f7246862a8e141b4cb0
Reviewed-on: https://chrome-internal-review.googlesource.com/272173
Commit-Ready: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/download_images_buildid_test.py
rosperf/image_checksummer.py
a4effef8ac1f221419ced5833c01a6d9f6f26d5f 29-Jul-2016 Rahul Chaudhry <rahulchaudhry@chromium.org> toolchain-utils: fix unused-import lint errors.

BUG=chromium:632109
TEST='./run_tests.py passes'

Change-Id: I7c2c9fc3881f3c59ac2a495c836863dcd693402b
Reviewed-on: https://chrome-internal-review.googlesource.com/272172
Commit-Ready: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/mock_instance.py
685ed888ddb849232644b20a2d9cb72be403aac3 29-Jul-2016 Rahul Chaudhry <rahulchaudhry@chromium.org> toolchain-utils: fix missing-print-function lint warnings.

BUG=chromium:632109
TEST='./run_tests.py passes'

Change-Id: Ie6881fb543cc145c9c09dd83f6b8a70c2f3083bd
Reviewed-on: https://chrome-internal-review.googlesource.com/272171
Commit-Ready: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/download_images_buildid_test.py
rosperf/image_checksummer.py
rosperf/mock_instance.py
90c7d37553ae22cfce56bc7423fc9735f271fc6f 29-Jul-2016 Rahul Chaudhry <rahulchaudhry@chromium.org> toolchain-utils: fix bad-shebang lint error.

BUG=chromium:632109
TEST='./run_tests.py passes'

Change-Id: I893f820e6e6fac2f08db364bb4ce238421a48791
Reviewed-on: https://chrome-internal-review.googlesource.com/272170
Commit-Ready: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/download_images_buildid_test.py
0c6b64a92171fc48773c6f83fcff4ee8e07b013b 29-Jul-2016 Rahul Chaudhry <rahulchaudhry@chromium.org> toolchain-utils: make file formatting conform to tc_pyformat.

BUG=chromium:632109
TEST=None

Change-Id: Ie3da2fa8707478b60e51a237adac7da11e180f9f
Reviewed-on: https://chrome-internal-review.googlesource.com/272169
Commit-Ready: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/download_images_buildid_test.py
rosperf/image_checksummer.py
rosperf/mock_instance.py
b45d1fa0609e196e48077314598ce9a17b1e6173 29-Jul-2016 Rahul Chaudhry <rahulchaudhry@chromium.org> toolchain-utils: fix results_cache_unittest.py

The test was failing with "RuntimeError: Could not run find command!".
It broke after https://chrome-internal-review.googlesource.com/#/c/270698
which added GetResultsFile(). This CL adds the Fake version for testing.

BUG=None
TEST='./results_cache_unittest.py passes'

Change-Id: I2bb92a394c8756bfb30941ce12e87396b22efe02
Reviewed-on: https://chrome-internal-review.googlesource.com/272168
Commit-Ready: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/results_cache_unittest.py
c19e66c24eb03c593f613b0f6668661ad8d6ab73 29-Jul-2016 Rahul Chaudhry <rahulchaudhry@chromium.org> toolchain-utils: fix benchmark_run_unittest.py

The 'show_all' argument for CreateFromRun was removed in
https://chrome-internal-review.googlesource.com/#/c/255303

BUG=None
TEST='./benchmark_run_unittest.py passes'

Change-Id: I984393d4db5bf98f0f273c25e366bb31a3e9b845
Reviewed-on: https://chrome-internal-review.googlesource.com/272167
Commit-Ready: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/benchmark_run_unittest.py
f014763a3e24c3f98969f5907b96bec4fec6316f 23-Jul-2016 Cassidy Burden <cburden@google.com> binary search tool: Add file locking to compiler wrapper

Because make can run with arbitrary number of jobs, there's potential
race conditions with the compiler wrapper's file operations. Replace all
open calls with function that locks files using fnctl module.

Additionally fix small formatting errors

CQ-DEPEND=CL:*270783

Change-Id: I4b94bfe3c0a5d7203f78e87130b7c4d71bc8e1c0
Reviewed-on: https://chrome-internal-review.googlesource.com/271747
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
inary_search_tool/bisect_driver.py
28683f7dacd94801e9a0d4b28e5aa8a700fc3d6f 25-Jul-2016 Cassidy Burden <cburden@google.com> binary search tool: Add compiler wrapper for Android bisection

Add compiler wrapper similar to ChromeOS object bisector's compiler
wrapper. bisect_driver.py is a modified version of bisect.py from the
ChromeOS sysroot_wrapper. compiler_wrapper.py is a simple skeleton script
used to invoke bisect_driver, and it is meant to be replaced by a more
robust compiler wrapper.

TEST=Populated android objects and triaged
CQ-DEPEND=CL:*270730

Change-Id: I4dfc676f6b659c2da8dbd01fd858c477ffbdeb9f
Reviewed-on: https://chrome-internal-review.googlesource.com/270783
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
inary_search_tool/bisect_driver.py
inary_search_tool/compiler_wrapper.py
f67b68da6d8ed5e8e84409a1c78be4d457e792b4 28-Jul-2016 Cassidy Burden <cburden@google.com> binary search tool: Add android test scripts

Add android test scripts: boot test and interactive test. Boot test
waits for device to boot up and reach homescreen. Interactive test waits
for device to boot and asks user if the image is good or bad.

TEST=Run full bisection with boot test, test code paths with interactive
test

Change-Id: I04490da381964ca157f72f70cca49213e0e827c0
Reviewed-on: https://chrome-internal-review.googlesource.com/271975
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Han Shen <shenhan@google.com>
inary_search_tool/android/boot_test.sh
inary_search_tool/android/interactive_test.sh
c583dac5abda3ecb5f05e35f77251fa3cef0c13c 28-Jul-2016 Cassidy Burden <cburden@google.com> binary search tool: Add android get_initial_items script

This script will simply output the list of all files in the good object
cache.

TEST=Used with test android bisection

Change-Id: I5caa7b3d1372d0b5ee046f2ceb814c88122d5ec6
Reviewed-on: https://chrome-internal-review.googlesource.com/271947
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Han Shen <shenhan@google.com>
inary_search_tool/android/get_initial_items.sh
c3fd4279b4220e53d767b3407c757a31f08ad659 27-Jul-2016 Cassidy Burden <cburden@google.com> binary search tool: Add android switch scripts

Add switch scripts for android bisector. Each script hardlinks files
from GOOD/BAD cache into working tree. ln/touch are invoked via xargs
to allow easier input parsing and parallel execution.

TEST=Tested various code paths and tried with prototype bisector

Change-Id: I9f52358737e8d18486287eb9e4d30bf3e1ad13ea
Reviewed-on: https://chrome-internal-review.googlesource.com/271725
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Han Shen <shenhan@google.com>
inary_search_tool/android/switch_to_bad.sh
inary_search_tool/android/switch_to_good.sh
d0c2d0dcecaa61a4a4d5e8d2612a8fbb457bad34 27-Jul-2016 Cassidy Burden <cburden@google.com> binary search tool: Add android install script

Add install script for android bisector. This script will build android
and attempt to flash the newly built image. If flashing fails, the
install script will ask the user for help.

TEST=Tested each code path

Change-Id: I89845edd8a2bafedc9e6cabffe7bf92f2f48bc0d
Reviewed-on: https://chrome-internal-review.googlesource.com/271482
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Han Shen <shenhan@google.com>
inary_search_tool/android/install.sh
2711ac8d32ff2d03a16eeef65952e3c1a1e60fd0 27-Jul-2016 Cassidy Burden <cburden@google.com> binary search tool: Alert user if --file_args required

Check for OS Error E2BIG when running switch script. If this happens
that means that --file_args is required.

TEST=Test with list of android objects, properly fails

Change-Id: If795ea5f45956ed0e5dd5eb6a9852f02d9f5978f
Reviewed-on: https://chrome-internal-review.googlesource.com/271688
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Han Shen <shenhan@google.com>
inary_search_tool/binary_search_state.py
1bb0c4dc6cb39b36db6e3c3f901b3fcc5525e950 27-Jul-2016 Cassidy Burden <cburden@google.com> Remove "--remove_trailing_comma" option from tc_pyformat

pyformat recently removed the "--remove_trailing_comma" option. Remove
this from tc_pyformat to prevent tool from crashing.

Change-Id: I899ae005230ef3de6f72fd69c963697bda5c53e0
Reviewed-on: https://chrome-internal-review.googlesource.com/271686
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
in/tc_pyformat
77efb771911728cb6911f4f1b1741a31341ed2da 27-Jul-2016 Cassidy Burden <cburden@google.com> binary search tool: Fix linting error

TEST=Run unit tests

Change-Id: I235148ca88e3f86f8bb00467a2b01ebfa99b0a88
Reviewed-on: https://chrome-internal-review.googlesource.com/271528
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
inary_search_tool/binary_search_perforce.py
9099a788cd7124024559c064e425ed9caef6e0ac 23-Jul-2016 Caroline Tice <cmtice@google.com> [crosperf] Clean up exceptions.

Found all instances of 'raise Exception' and changed them to raise
something slightly more specific. Changed a few 'except Exception'
statements appropriately as well.

BUG=chromium:566255
TEST=None

Change-Id: If2666bd55838342bc71431e5e6ab2157c2c9e125
Reviewed-on: https://chrome-internal-review.googlesource.com/270731
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
inary_search_tool/binary_search_state.py
ros_utils/file_utils.py
ros_utils/manifest_versions.py
ros_utils/misc.py
rosperf/benchmark.py
rosperf/benchmark_run.py
rosperf/benchmark_run_unittest.py
rosperf/experiment_file.py
rosperf/field.py
rosperf/image_checksummer.py
rosperf/label.py
rosperf/machine_manager.py
rosperf/machine_manager_unittest.py
rosperf/results_cache.py
rosperf/settings.py
rosperf/settings_factory.py
ejagnu/gdb_dejagnu.py
ejagnu/run_dejagnu.py
do_scripts/profile_cycler.py
do_scripts/summarize_hot_blocks.py
do_scripts/vanilla_vs_fdo.py
mage_chromeos.py
emote_gcc_build.py
epo_to_repo.py
est_gcc_dejagnu.py
est_gdb_dejagnu.py
19e119b46c14f57988ba3504369e815d3aab0122 26-Jul-2016 Cassidy Burden <cburden@google.com> binary search tool: Add setup script for android bisector

Add setup script for android bisector. This setup script will:
1. Generate all necessary variables for the switch/test/etc. scripts
and put them in a common.sh file.
2. Ensure the GOOD/BAD object caches exist, and have the same objects
lists
3. Make sure that the android build environment is properly configured
(with build/envsetup.sh and lunch)

Change-Id: Id2343b5d4bbc51b06fab261863ccbe7c507c33b4
Reviewed-on: https://chrome-internal-review.googlesource.com/270976
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
inary_search_tool/android/setup.sh
8f7e4bc040879b7a2eb22033f2c83d99cedbe6f4 22-Jul-2016 Cassidy Burden <cburden@google.com> binary search tool: Fix edge case with noincremental and prune

Fix edge case where not all items were sent to switch_to_good script
when noincremental and prune were both set. Pruning progressively narrows
down the total number of items being searched across because some items
can be assumed good. This fix ensures that these "known good" items are
also sent to the switch script if noincremental is set.

TEST=Add unit test for this case, run other tests

Change-Id: I1145f6eea224f06ceb2352ec77e17e849d489606
Reviewed-on: https://chrome-internal-review.googlesource.com/270718
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
inary_search_tool/binary_search_state.py
inary_search_tool/test/binary_search_tool_tester.py
inary_search_tool/test/is_good_noinc_prune.py
inary_search_tool/test/switch_to_bad_noinc_prune.py
inary_search_tool/test/switch_to_good_noinc_prune.py
5a9c4419a7625926712263b345f21ad6d204f130 22-Jul-2016 Cassidy Burden <cburden@google.com> binary search tool: Provide temp files that hold GOOD/BAD sets

Provide temp files to scripts ($BISECT_GOOD_SET, $BISECT_BAD_SET) that
provide a full listing of the items in the good set and items in the bad
set. This can be used for debugging purposes or so scripts can analyze
the state of the current binary search iteration.

TEST=Add unit test that utilizes these files

Change-Id: I32bc393644f1cb998c2d818a9026486ef83cc4b8
Reviewed-on: https://chrome-internal-review.googlesource.com/270730
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Han Shen <shenhan@google.com>
inary_search_tool/binary_search_state.py
inary_search_tool/test/binary_search_tool_tester.py
inary_search_tool/test/switch_to_bad_set_file.py
inary_search_tool/test/switch_to_good_set_file.py
0ded515f8f340188634bd8d14e3184b98e4f06de 25-Jul-2016 Caroline Tice <cmtice@google.com> [crosperf] Change flags to cbuildbot.

The command for cbuildbot it buildbot_utils had hardcoded the
'--notests' options; this was right for the normal nightly jobs, but
not for the rotating tests. This CL makes that option a variable, and
also allows for users to be able to add other custom flags to their
cbuildbot invocations.

BUG=chromium:631149
TEST=none

Change-Id: Ifd2abd42808d99a28c3f7860f67a2eb2c9cd0118
Reviewed-on: https://chrome-internal-review.googlesource.com/271005
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
uildbot_test_llvm.py
uildbot_test_toolchains.py
ros_utils/buildbot_utils.py
180fb3868ae7a625cd82427ad29bc6224504c757 23-Jul-2016 Caroline Tice <cmtice@google.com> [crosperf] Get telemetry results from file instead of stdout.

This CL fixes crosperf to read the telemetry json results file for
getting its results, rather than parsing stdout.

BUG=chromium:620464
TEST=Successfully got results for octane, kraken & speedometer on daisy.

Change-Id: I5bf8cf143688d83ca38c6453006360a45e945544
Reviewed-on: https://chrome-internal-review.googlesource.com/270698
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
rosperf/results_cache.py
a8af9a7a2462b00e72deff99327bdb452a715277 20-Jul-2016 Caroline Tice <cmtice@google.com> [toolchain-utils] Finish switching utils/ to cros_utils/.

This CL finishes switching the subdirectory from 'utils' to
'cros_utils'. It changes all the remaining import statements to
use 'cros_utils'; it removes the 'cros_utils' symlink, and it
renames the 'utils' subdirectory to 'cros_utils'.

BUG=chromium:568195
TEST=ran crosperf & binary search tool unittests.

Change-Id: I7427f8bfb2ddac3a4b6108e46782039059684382
Reviewed-on: https://chrome-internal-review.googlesource.com/270396
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
fe_lock_machine.py
inary_search_tool/binary_search_state.py
inary_search_tool/bisect.py
inary_search_tool/test/binary_search_tool_tester.py
inary_search_tool/test/gen_init_list.py
uildbot_test_llvm.py
uildbot_test_toolchains.py
rb/autotest_run.py
rb/crb_driver.py
rb/machine_manager_singleton.py
ros_utils
ros_utils/__init__.py
ros_utils/buildbot_json.py
ros_utils/buildbot_utils.py
ros_utils/colortrans.py
ros_utils/command_executer.py
ros_utils/command_executer_unittest.py
ros_utils/constants.py
ros_utils/email_sender.py
ros_utils/file_utils.py
ros_utils/html_tools.py
ros_utils/locks.py
ros_utils/logger.py
ros_utils/machines.py
ros_utils/manifest_versions.py
ros_utils/misc.py
ros_utils/misc_test.py
ros_utils/no_pseudo_terminal_test.py
ros_utils/perf_diff.py
ros_utils/pstat.py
ros_utils/stats.py
ros_utils/tabulator.py
ros_utils/tabulator_test.py
ros_utils/timeline.py
ros_utils/timeline_test.py
rosperf/crosperf.py
rosperf/download_images.py
wp/interpreter/symbolizer.py
ejagnu/gdb_dejagnu.py
ejagnu/run_dejagnu.py
do_scripts/divide_and_merge_profiles.py
do_scripts/divide_and_merge_profiles_test.py
do_scripts/profile_cycler.py
do_scripts/summarize_hot_blocks.py
do_scripts/vanilla_vs_fdo.py
em_tests/mem_groups.py
em_tests/total_mem_actual.py
em_tests/total_mem_sampled.py
emote_gcc_build.py
epo_to_repo.py
tils/__init__.py
tils/buildbot_json.py
tils/buildbot_utils.py
tils/colortrans.py
tils/command_executer.py
tils/command_executer_unittest.py
tils/constants.py
tils/email_sender.py
tils/file_utils.py
tils/html_tools.py
tils/locks.py
tils/logger.py
tils/machines.py
tils/manifest_versions.py
tils/misc.py
tils/misc_test.py
tils/no_pseudo_terminal_test.py
tils/perf_diff.py
tils/pstat.py
tils/stats.py
tils/tabulator.py
tils/tabulator_test.py
tils/timeline.py
tils/timeline_test.py
19b6f5fc11dcf97144e9723c8f78534cce27423a 22-Jul-2016 Cassidy Burden <cburden@google.com> binary search tool: Print out elapsed time every iteration

Print out time since execution has begun every progress update. Also
give final execution time after tool completes.

Change-Id: Ibba29feac42c8b637b1d3595a381ad3b4e9e974d
Reviewed-on: https://chrome-internal-review.googlesource.com/270695
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
inary_search_tool/binary_search_state.py
0f07f97cc8ff4ab42b05e32e3a008b2f618cc51f 19-Jul-2016 Cassidy Burden <cburden@google.com> binary search tool: Add new common hash_test.sh

Add new test script for all bisecting tools. This test simply runs a
checksum on a given file. If the checksum fails the image is "bad",
otherwise the image is "good". This test replaces the less generic
system test for cros_pkg (cros_pkg/testing_test.sh).

TEST=Run system test for package bisector with hash_test.sh

Change-Id: Ie1358101386ec2a04a077c8d81e73706f28112c0
Reviewed-on: https://chrome-internal-review.googlesource.com/270095
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
inary_search_tool/README.bisect
inary_search_tool/common/hash_test.sh
inary_search_tool/cros_pkg/testing_test.sh
4362e0a3a114215498593a589c5ee8faca9dbfe2 18-Jul-2016 Cassidy Burden <cburden@google.com> toolchain-utils: Fix use before defined error, cleanup linting errors

If an exception was raised during the POpen call, calling the except_handler
would crash the program and hide the original exception. Now initialize
process object to None before starting try block.

TEST=RunCommandGeneric no longer crashes if POpen fails

Change-Id: I310678fc85e94d066b1da304549e97fee7b99073
Reviewed-on: https://chrome-internal-review.googlesource.com/270045
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
tils/command_executer.py
d00ad41cf8d79f0760fa546dcdbb20a05c715177 03-Jul-2016 Caroline Tice <cmtice@google.com> [crosperf] Separate output for different compilers.

Currently the rotating nightly tester is writing all the test output
to the same directory, based on board name, so the GCC & LLVM output
files for a particular are overwriting each other. This CL puts
the GCC & LLVM output into different directories so that won't happen.

BUG=None
TEST=None

Change-Id: Ifc564a51bd0f44c9a701805975cbb64325074aab
Reviewed-on: https://chrome-internal-review.googlesource.com/268335
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Ting-Yuan Huang <laszio@google.com>
uildbot_test_llvm.py
6bee8a50519fe42aa3c9cf727c1832510c3f0157 07-Jul-2016 Cassidy Burden <cburden@google.com> binary search tool: Update found_items when prune is not set

Previously when prune was not set found_items would not be updated with
the single bad item found. This commit fixed this edge case and adds a
unit test for it.

TEST=Run unit tests

Change-Id: Iac56c53fd4de97fa12b14fc9ea1e05d021cc6495
Reviewed-on: https://chrome-internal-review.googlesource.com/268725
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Han Shen <shenhan@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
inary_search_tool/binary_search_state.py
inary_search_tool/test/binary_search_tool_tester.py
f56836d4c54b9e7cd09c7d7c681ebac360aa640a 07-Jul-2016 Cassidy Burden <cburden@google.com> binary search tool: Add more unit tests

Add tests for failed verification and iterations limit.

Change-Id: I22c26ed1c5f779ee5f81256c2315a6c523e1dfa8
Reviewed-on: https://chrome-internal-review.googlesource.com/268243
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Han Shen <shenhan@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
inary_search_tool/test/binary_search_tool_tester.py
d68bc3fbf3103ab6655ba5af1fa30b555d8402c2 07-Jul-2016 Cassidy Burden <cburden@google.com> Add docstrings to binary search state methods

Change-Id: Iba4bf1e8df155a38c4c12fa488bee7df3ffd0291
Reviewed-on: https://chrome-internal-review.googlesource.com/268538
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Han Shen <shenhan@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
inary_search_tool/binary_search_state.py
39c0c2109ad3b7e89a2a51e0c567b94103415d69 02-Jul-2016 Cassidy Burden <cburden@google.com> binary search tool: Fix typos and small output errors

Change-Id: I0859c43b3bfe8917ebf1e5337f2e39cbb2a55fa0
Reviewed-on: https://chrome-internal-review.googlesource.com/268247
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
inary_search_tool/README.bisect
inary_search_tool/binary_search_state.py
inary_search_tool/common.py
c4671bdd601ff945b2cccbfe1aee9f931dc908c7 01-Jul-2016 Cassidy Burden <cburden@google.com> binary search tool: (Refactor) Extract common scripts into common folder

Refactor warning! Move install.sh, boot_test.sh, and interactive_test.sh
into common folder so they can be shared by both package and object
bisectors. To allow this sharing, sysroot_wrapper now has a setup script
similar to cros_pkg. All scripts now source common/common.sh.

TEST=Run unit tests, run system tests, run couple iterations of install
script for both bisectors.

Change-Id: I9e164b4e6b842ff321c2400201e6ac0984f99088
Reviewed-on: https://chrome-internal-review.googlesource.com/268027
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
inary_search_tool/bisect.py
inary_search_tool/common/boot_test.sh
inary_search_tool/common/install.sh
inary_search_tool/common/interactive_test.sh
inary_search_tool/cros_pkg/create_cleanup_script.py
inary_search_tool/cros_pkg/get_initial_items.sh
inary_search_tool/cros_pkg/install.sh
inary_search_tool/cros_pkg/setup.sh
inary_search_tool/cros_pkg/switch_to_bad.sh
inary_search_tool/cros_pkg/switch_to_good.sh
inary_search_tool/cros_pkg/testing_test.sh
inary_search_tool/sysroot_wrapper/README
inary_search_tool/sysroot_wrapper/cleanup.sh
inary_search_tool/sysroot_wrapper/common.sh
inary_search_tool/sysroot_wrapper/get_initial_items.sh
inary_search_tool/sysroot_wrapper/glibc_test_script.sh
inary_search_tool/sysroot_wrapper/install.sh
inary_search_tool/sysroot_wrapper/setup.sh
inary_search_tool/sysroot_wrapper/switch_to_bad.sh
inary_search_tool/sysroot_wrapper/switch_to_good.sh
inary_search_tool/sysroot_wrapper/test_script.sh
63f13489b1a02bbbc75b46fec6ae7a817442df94 30-Jun-2016 George Burgess IV <gbiv@google.com> [crosperf] Add more tests for results_report.

With this patch, all of the classes in results_report have tests. The
tests aren't overly extensive, but we can always improve on them later.

This also includes a few small cleanups of parts of machine_manager, and
results_report.

BUG=None
TEST=./run_tests.sh

Change-Id: I9613579d4f8c1079b452e1a0a2edf47b189ac1cd
Reviewed-on: https://chrome-internal-review.googlesource.com/268057
Commit-Ready: George Burgess <gbiv@google.com>
Tested-by: George Burgess <gbiv@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/machine_manager.py
rosperf/results_report.py
rosperf/results_report_unittest.py
d5113fb3bb67fc27f6fdc04e757e7a4b65e2a5f4 29-Jun-2016 Cassidy Burden <cburden@google.com> binary search tool: Add object bisecting to bisect.py

Add object file bisecting to bisect.py. This is the frontend to the
sysroot_wrapper bisection scripts. This bisection mode takes the
following arguments: board, remote IP, package to bisect, and optionally
the directory for the good/bad build trees.

Adds default install/test scripts to sysroot_wrapper (similar to
default scripts in cros_pkg) to support this bisection mode.

TEST=Run unit tests and system test with cryptohome package
CQ-DEPEND=CL:*267995

Change-Id: I1e9166b753f78ca7bdcf6ddbd056af62bc03923b
Reviewed-on: https://chrome-internal-review.googlesource.com/268035
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
inary_search_tool/README.bisect
inary_search_tool/bisect.py
inary_search_tool/common/boot_test.sh
inary_search_tool/common/interactive_test.sh
inary_search_tool/sysroot_wrapper/install.sh
caf9d96c94a914866c4cb9dfe401dfcee02c6047 28-Jun-2016 Cassidy Burden <cburden@google.com> binary search tool: Add pretty printing of arguments to bisect.py

Add pretty printing of arguments at beginning of bisect.py. Additionally
extract argument overriding logic into helper function in base class.

TEST=Try various argument overrides and verify output

Change-Id: I15b33ee944356e9db880a3518bcea48e022f4442
Reviewed-on: https://chrome-internal-review.googlesource.com/267995
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
inary_search_tool/bisect.py
a9d7077bfbd626ecd316a5944c4180f861e22356 29-Jun-2016 Cassidy Burden <cburden@google.com> Add README for bisect.py

Change-Id: I9bc78a3a69609354fe88ad5123ebb748e211f1e6
Reviewed-on: https://chrome-internal-review.googlesource.com/267412
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
inary_search_tool/README.bisect
bfe9c5300fca09e8790ed25b41028cba8cb0a78f 29-Jun-2016 Cassidy Burden <cburden@google.com> Add argument overriding to bisect.py

Add the ability to override the default arguments given to the package
and object bisecting tools. This allows users to specify their own
test/install/etc. script but keep the others the same.

Example 1 (do boot test instead of interactive test):
./bisect.py package daisy 172.17.211.182 --test_script=cros_pkg/boot_test.sh

Example 2 (do package bisector system test instead of interactive test):
./bisect.py package daisy 172.17.211.182 \
--test_script=cros_pkg/testing_test.sh --install_script=""

TEST=Run unit tests and ran above examples among others (such as
overriding verbose, prune, etc. options)
CQ-DEPEND=CL:*266996

Change-Id: I5ee24826d4d7e22e7328ce98c6b8cda9917df533
Reviewed-on: https://chrome-internal-review.googlesource.com/266305
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
inary_search_tool/bisect.py
inary_search_tool/test/binary_search_tool_tester.py
2e9f8a097c095ca93052b368ffab4c850d4d3d0f 29-Jun-2016 George Burgess IV <gbiv@google.com> [crosperf] Add results_report tests, minor refactors.

This adds unit tests for results_report.JSONResultsReport and
results_report.ParseChromeosImage. The test for the former aren't as
extensive as they could be, but I hope to add more in the future.

This also simplifies ParseChromeosImage, and does a few refactors to
JSONResultsReport.

This also fixes all of `cros lint`'s complaints about results_report.

BUG=None
TEST=./run_tests.sh

Change-Id: Ice4267a5ff2e10df6690595c6b80db1ee1977d50
Reviewed-on: https://chrome-internal-review.googlesource.com/267665
Commit-Ready: George Burgess <gbiv@google.com>
Tested-by: George Burgess <gbiv@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/results_report.py
rosperf/results_report_unittest.py
7edbfa2e0ef52487ec1286886dcbc7d401528733 28-Jun-2016 George Burgess IV <gbiv@google.com> [crosperf] Fix setting descriptions; NFC.

This also removes an unused import, and adds a few docstrings/etc. to
make `cros lint` happy.

BUG=None
TEST=./run_tests.sh

Change-Id: I0279a0db1d1cb1a1bc655a06bfc6b33d87495610
Reviewed-on: https://chrome-internal-review.googlesource.com/267277
Commit-Ready: George Burgess <gbiv@google.com>
Tested-by: George Burgess <gbiv@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/settings_factory.py
2e37b144ecd1476964e2dc72ac6b5b2a81a32018 24-Jun-2016 Cassidy Burden <cburden@google.com> Add common module for binary search tool

common module holds common logic between bisect and
binary_search_state for adding utils to PYTHONPATH
and for building the argument parser.

TEST=Run unit tests, run cros_pkg system test

Change-Id: I4085b28de011132c53b8fddf06e8745042ef8ea2
Reviewed-on: https://chrome-internal-review.googlesource.com/266996
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
inary_search_tool/binary_search_state.py
inary_search_tool/bisect.py
inary_search_tool/common.py
314ea5683c0b4c7658f6c3f659cb5c53218a613c 25-Jun-2016 Caroline Tice <cmtice@google.com> [toolchain-utils] Update rotating tester to allow GCC tests, too.

BUG=None
TEST=Tested on chrotomation2 with role account.

Change-Id: I232df188f05a13cdc59a53173b7c46f9b1b1b282
Reviewed-on: https://chrome-internal-review.googlesource.com/266966
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
uildbot_test_llvm.py
83558f81695925a40d5066c24404903b29976686 23-Jun-2016 Cassidy Burden <cburden@google.com> binary search tool: Enforce execution from root directory

Both cros_pkg and sysroot_wrapper rely on the current working directory
being toolchain-utils/binary_search_tool. bisect.py will now enforce
this by cd'ing to this directory before execution.

TEST=Run unit tests and cros_pkg
CQ-DEPEND=CL:*266365

Change-Id: Ie30272ba31cd80a1b2c715e0f9a42a4e26889f16
Reviewed-on: https://chrome-internal-review.googlesource.com/266329
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
inary_search_tool/bisect.py
545b947888df1d07f4ad530e1c5eec930fc283c2 23-Jun-2016 Cassidy Burden <cburden@google.com> binary search tool: Move root path of cros_pkg execution, rename scripts

Update cros_pkg scripts and bisect.py so that the package bisector needs
to be run from binary_search_tool/ instead of
binary_search_tool/cros_pkg. This fits with how sysroot_wrapper
implements its scripts. Also rename all cros_pkg scripts to remove
cros_pkg prefix and remove undo_eclean.py.

TEST=Run unit tests and run interactive/testing cros_pkg test

Change-Id: I2781319934b704b91346745ae2d4d916fee35d02
Reviewed-on: https://chrome-internal-review.googlesource.com/266365
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
inary_search_tool/bisect.py
inary_search_tool/common/boot_test.sh
inary_search_tool/common/interactive_test.sh
inary_search_tool/cros_pkg/README.cros_pkg_triage
inary_search_tool/cros_pkg/create_cleanup_script.py
inary_search_tool/cros_pkg/cros_pkg_boot_test.sh
inary_search_tool/cros_pkg/cros_pkg_create_cleanup_script.py
inary_search_tool/cros_pkg/cros_pkg_get_initial_items.sh
inary_search_tool/cros_pkg/cros_pkg_install.sh
inary_search_tool/cros_pkg/cros_pkg_interactive_test.sh
inary_search_tool/cros_pkg/cros_pkg_setup.sh
inary_search_tool/cros_pkg/cros_pkg_switch_to_bad.sh
inary_search_tool/cros_pkg/cros_pkg_switch_to_good.sh
inary_search_tool/cros_pkg/cros_pkg_testing_test.sh
inary_search_tool/cros_pkg/cros_pkg_undo_eclean.py
inary_search_tool/cros_pkg/get_initial_items.sh
inary_search_tool/cros_pkg/install.sh
inary_search_tool/cros_pkg/setup.sh
inary_search_tool/cros_pkg/switch_to_bad.sh
inary_search_tool/cros_pkg/switch_to_good.sh
inary_search_tool/cros_pkg/testing_test.sh
58f24cae7e6dfed8196d0b96713afaa42cd1fdde 21-Jun-2016 Cassidy Burden <cburden@google.com> Add initial unified bisection script

Add new script for unifying package and object bisection script
interfaces. Currently only package bisection is implemented.

TEST=Add unit test, test cros_pkg with testing script and interactive
script

Change-Id: I7491e3fb73eae863e24de3869092cfbf9df56c8d
Reviewed-on: https://chrome-internal-review.googlesource.com/266096
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
inary_search_tool/bisect.py
inary_search_tool/test/binary_search_tool_tester.py
61ace4c75764cbab29bc0bcddf40d6c2d71996a6 22-Jun-2016 Cassidy Burden <cburden@google.com> binary search tool: Make BST tester check output more robust

Updated binary_search_tool_tester's output checking to be more robust,
and fully check the state of the reported good/bad sets.

Change-Id: Id7ab8c006133d67b75ed902a437ca817e62cd219
Reviewed-on: https://chrome-internal-review.googlesource.com/266029
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
inary_search_tool/test/binary_search_tool_tester.py
88c8aac3277c6fd841cb774f6815a14e37f3182f 22-Jun-2016 Cassidy Burden <cburden@google.com> Add tests to and update sysroot_wrapper bisector

Add new testing script to sysroot_wrapper bisector and make all shell
scripts for sysroot_wrapper bisector executable.

TEST=Run new test on chromeos-chrome

Change-Id: I312e50108e36341517ed1f8ea6b8d415e4184c2e
Reviewed-on: https://chrome-internal-review.googlesource.com/265975
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
inary_search_tool/sysroot_wrapper/common.sh
inary_search_tool/sysroot_wrapper/get_initial_items.sh
inary_search_tool/sysroot_wrapper/glibc_test_script.sh
inary_search_tool/sysroot_wrapper/switch_to_bad.sh
inary_search_tool/sysroot_wrapper/switch_to_good.sh
inary_search_tool/sysroot_wrapper/test_script.sh
inary_search_tool/sysroot_wrapper/testing_test.py
54e72b32bcadbc70b81cfa1c62ee65948b3fb78c 21-Jun-2016 Yunlian Jiang <yunlian@google.com> Continue test if the test for one board failed.

We run llvm validation test for several boards in a single script.
During the test, if something goes wrong and it calls logger.LogFatal(),
the whole test terminates because the LogFatal calls sys.exit() and
it will terminate the whole script. This CL fixes that by catching the
exception raised by sys.exit().

BUG=None
TEST=the SystemExit exception is handled.

Change-Id: I502361d4c55bf58c4c8106067768c17e94ce8631
Reviewed-on: https://chrome-internal-review.googlesource.com/265905
Commit-Ready: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
uildbot_test_llvm.py
068c5fb61394771043ed39b11cb996cfd9874c83 20-Jun-2016 Cassidy Burden <cburden@google.com> binary search tool: Add Run method and move Main logic there

Add Run method so that the binary search tool can be easily invoked from
other Python scripts (see update to binary_search_tool_tester.py). Now
Main only does very basic arg_parse logic and leaves rest up to Run.

Test=Unit tests run, cros_pkg tester passes

Change-Id: I90400f9223c7aad0a2701944ba8282fb763f1cf7
Reviewed-on: https://chrome-internal-review.googlesource.com/265628
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Han Shen <shenhan@google.com>
inary_search_tool/binary_search_state.py
inary_search_tool/test/binary_search_tool_tester.py
702bf8793c082c144b57e445694ccc548a90ac21 17-Jun-2016 Cassidy Burden <cburden@google.com> Binary search tool: Refactor bs to binary_search

TEST=Run unit tests

Change-Id: Ia7f682c42602e9355d1e2db67e258e2864c78126
Reviewed-on: https://chrome-internal-review.googlesource.com/265276
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Han Shen <shenhan@google.com>
inary_search_tool/binary_search_state.py
47de616141aac91d6d3e15e80591d1aca8dffc57 18-Jun-2016 Cassidy Burden <cburden@google.com> binary search tool: Cleanup state files after SaveState unit test

Running test_bad_save_state would previously leave a save state with no
symlink pointing to it. This would cause multiple save states to gather
and clutter up the directory. We now simply cleanup the generated save
state in this test case.

TEST=Run unit test and see if any state files remain

Change-Id: If594c7266b2ff24a260e746a4d06bddea2c0bf78
Reviewed-on: https://chrome-internal-review.googlesource.com/265286
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
inary_search_tool/test/binary_search_tool_tester.py
d6d5b5e0a1f3912501f791cbb0b544ae1f114949 17-Jun-2016 Cassidy Burden <cburden@google.com> Remove unnecessary prune iterations from binary search tool

Previously the binary search tool would do multiple prune iterations to
check if it was complete (using num_bad_items_history). This could lead
to dozens of unneccesary prune iterations. Now we just do one extra prune
iteration before completing by checking the found_items set.

TEST=Run unit tests and test cros_pkg bisector

Change-Id: Id5d6e45c15dcf4b4c0a1c1e6c3feb97d5c0d6d62
Reviewed-on: https://chrome-internal-review.googlesource.com/265275
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
inary_search_tool/binary_search_state.py
cc9b45677f0838f99c1961142fef4559de931604 14-Jun-2016 Caroline Tice <cmtice@google.com> [crosperf] Add phony image testing, for easier package development.

Using a 'real' bad image for bisecting tool development makes it
very slow because it often requires manual intervention to push
images onto device and to verify if image is good or bad.

This CL creates a phony good/bad test that just looks at the md5 hash
value of Chrome package, allowing testing to be done with two good images,
which greatly simplifies bisecting tool development.

BUG=None
TEST=Tested these scripts individually in my chroot.

Change-Id: I4342f8f152162e12f49b916f5977776aa79e1b03
Reviewed-on: https://chrome-internal-review.googlesource.com/264605
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
inary_search_tool/cros_pkg/cros_pkg_testing_test.sh
b51148ca9f8b8a9a1322342836ac5906f2564f85 17-Jun-2016 Cassidy Burden <cburden@google.com> Fix binary search tool SaveState to correctly save bisecting logic

Previously SaveState wasn't saving the full program state. Now resuming
can continue at the latest possible iteration.

TEST=Unit tests run, tested with package bisector

Change-Id: Id8542a61d16b4c3e18d696d0b601fbd0dd98d048
Reviewed-on: https://chrome-internal-review.googlesource.com/265096
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Han Shen <shenhan@google.com>
inary_search_tool/binary_search_state.py
dcafebee1897d69f20282dc52082e1e5602cdbf0 16-Jun-2016 Cassidy Burden <cburden@google.com> Fix cros_pkg switch scripts so last input is properly read

When using --file_args the last line of input won't have a newline. This
commit fixes the switch scripts so that read doesn't finish before
reading the last line.

Change-Id: Ia180ca2c26a40d6d0dafdb53d9129f3079e79a1b
Reviewed-on: https://chrome-internal-review.googlesource.com/265085
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Han Shen <shenhan@google.com>
inary_search_tool/cros_pkg/cros_pkg_switch_to_bad.sh
inary_search_tool/cros_pkg/cros_pkg_switch_to_good.sh
76c2be6cad46342f69ca5dd3d69f4f4d31f85de9 13-Jun-2016 Cassidy Burden <cburden@google.com> Add resuming to binary search tool

Move around logic so that the binary search tool can be properly resumed
and add the --resume option.

TEST=Added unit test for LoadState

Change-Id: I7059fab9ac6c37ecfd79e4cf2ef976bdebfef8fb
Reviewed-on: https://chrome-internal-review.googlesource.com/264778
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Han Shen <shenhan@google.com>
inary_search_tool/.gitignore
inary_search_tool/binary_search_state.py
inary_search_tool/test/binary_search_tool_tester.py
c57133760fc479a1c3553172934e25fca5750851 15-Jun-2016 Yunlian Jiang <yunlian@google.com> Add script for llvm validation test.

This create a script for llvm validation test.

BUG=none
TEST=it lanches trybot, runs test and sends out email.

Change-Id: I85c5c8d1d9cdd9229a766b2cd5c307bdd35fc970
Reviewed-on: https://chrome-internal-review.googlesource.com/264996
Commit-Ready: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
uildbot_test_llvm.py
242c6cbf1a3c7d1d09cde13281d8bb36d7ef6001 15-Jun-2016 Cassidy Burden <cburden@google.com> Fix binary search tool's temp file creation

Fix the binary search tool so that temp files are created properly. This
means that temp files made for the switch scripts should be automatically
deleted once the switch script completes.

TEST=New unit test for temp file deletion

Change-Id: I5d36bfa07e274cd79965f182d85d6a7ea3006c85
Reviewed-on: https://chrome-internal-review.googlesource.com/264735
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
inary_search_tool/binary_search_state.py
inary_search_tool/test/binary_search_tool_tester.py
inary_search_tool/test/switch_tmp.py
9c665a01b55ee4e672b29ff352989a640eac084f 15-Jun-2016 Cassidy Burden <cburden@google.com> Fix binary search tool install unit tests

Change from using module wide variable common.installed to using a file
called "./installed". Previously common.installed wouldn't actually be
set properly because there would be two separate instances of Python
running.

TEST=More robust install unit test made

Change-Id: Iee87a52e0f54fc63837b25918764e804747525db
Reviewed-on: https://chrome-internal-review.googlesource.com/264745
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
inary_search_tool/test/binary_search_tool_tester.py
inary_search_tool/test/install.py
inary_search_tool/test/is_good.py
51ba54f3375c17e2a33c1854b635b144802457a4 15-Jun-2016 Cassidy Burden <cburden@google.com> Fix binary search tool tester to process output correctly

Fix the unit tests so that they run tail on the correct logs. Also fix
edge case when only one bad item is found.

Change-Id: I31f34755440af51a8a2310d9393aa87dc852c31d
Reviewed-on: https://chrome-internal-review.googlesource.com/264695
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
inary_search_tool/binary_search_state.py
inary_search_tool/test/binary_search_tool_tester.py
4ddf846fc0f8f3ea261b608e473cbd781aad1109 13-Jun-2016 Cassidy Burden <cburden@google.com> Update BinarySearchState's SaveState to be atomic

SaveState will now save the current state of the binary search tool to a
temp file then atomically switch the symlink to the new file.

TEST=Added new unit test for SaveState

Change-Id: Idf97c8ef346700a6e653206083ee7ab75832dbd5
Reviewed-on: https://chrome-internal-review.googlesource.com/264419
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
inary_search_tool/binary_search_state.py
inary_search_tool/test/binary_search_tool_tester.py
649273e844d7e86a781cb9f7a870b504bc0febbe 10-Jun-2016 Cassidy Burden <cburden@google.com> Add constant progress output to binary search tool

Add output for each iteration of the binary search tool to inform user
of current status: number of remaining iterations, found bad items, etc.

TEST=Ran binary_search_tool_tester and cros_pkg

Change-Id: I835893330b532b7f0de9b4fe072ca72c6d7f3b6c
Reviewed-on: https://chrome-internal-review.googlesource.com/264258
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
inary_search_tool/binary_search_state.py
bdd09b15b0ac1e60b3495e3698cb411926b21f9c 10-Jun-2016 Cassidy Burden <cburden@google.com> [crosperf] Update binary search scripts to use same logger.

TEST=Run with package bisector, logs no longer overwritten
BUG=619081

Change-Id: I0f9b76a2b08c98b53a7e7a98b52ba4a1f02a0276
Reviewed-on: https://chrome-internal-review.googlesource.com/263946
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
inary_search_tool/binary_search_perforce.py
inary_search_tool/binary_search_state.py
333fe383dcd0c35981a5473bc913d07b52c310a1 13-Jun-2016 Cassidy Burden <cburden@google.com> Update bisection tool to use RunCommandWExceptionCleanup

Update all calls to RunCommand to RunCommandWExceptionCleanup so that
processes spawned by the bisection tool are properly cleaned up.

TEST=Ctrl-c properly kill cros flash
BUG=chromium:618461

Change-Id: I53871a911b723007d550876241129708b7ae2a92
Reviewed-on: https://chrome-internal-review.googlesource.com/264278
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
inary_search_tool/binary_search_state.py
acb7a1933e5e0834592d59fd00547a3926c0e880 10-Jun-2016 Cassidy Burden <cburden@google.com> Add verbose option to binary search tool

Ignore most output from binary search tool and provide option to see full
output with the -V or --verbose option. Full output will still logged even
without the --verbose option.

TEST=Ran binary_search_tool_tester and cros_pkg

Change-Id: I02d958f79813083b48d989c9881536ad025bb1dc
Reviewed-on: https://chrome-internal-review.googlesource.com/264075
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Han Shen <shenhan@google.com>
inary_search_tool/binary_search_perforce.py
inary_search_tool/binary_search_state.py
a02937d6530b19b5635119a3f40169a604b0ca26 09-Jun-2016 Cassidy Burden <cburden@google.com> toolchain-utils: add RunCommandWExceptionCleanup to command_executer

RunCommandWExceptionCleanup will kill the spawned child process if a exception
is thrown. This mainly handles the case of a KeyboardInterrupt being raised
but the child process will continue to run. Added except_handler to
RunCommandGeneric and RunCommand2 to support this functionality.

TEST=Run cros flash usb:// and Ctrl-c properly kills cros flash
BUG=chromium:618461

Change-Id: I8195afcc310ab36b2631bb1ecc43a6fb8befec46
Reviewed-on: https://chrome-internal-review.googlesource.com/263665
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
tils/command_executer.py
f78d29852eafba827b6c95783049a28d308e86da 09-Jun-2016 Cassidy Burden <cburden@google.com> Add default install script to cros_pkg

Provide install script to package bisecting tool. This script handles
failed flash by prompting user to flash via usb or resetting the
Chromebook.

Test=A few iterations of bisecting tool run. Successfully install after
initial failed flash.

Change-Id: I161c697a87e87cb66ce5a2efc152505d8a2a5b8e
Reviewed-on: https://chrome-internal-review.googlesource.com/263915
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
inary_search_tool/cros_pkg/README.cros_pkg_triage
inary_search_tool/cros_pkg/cros_pkg_boot_test.sh
inary_search_tool/cros_pkg/cros_pkg_install.sh
inary_search_tool/cros_pkg/cros_pkg_interactive_test.sh
7f3c25952c446585241f1e37a61ca61535b1197c 08-Jun-2016 Ting-Yuan Huang <laszio@google.com> [crosperf] Fix default result for telemetry tests.

The tests fixed are:
smoothness.tough_webgl_cases
page_cycler.typical_25

The names of the default summary fields for these tests have
changed, which makes crosperf fail since it can't find the
right result fields. This CL fixes that problem.

BUG=chromium:618224
TEST=Tested locally

Change-Id: I1f3a8eed476732f967d5a628bea0f9fe358ed3e9
Reviewed-on: https://chrome-internal-review.googlesource.com/263236
Commit-Ready: Ting-Yuan Huang <laszio@google.com>
Tested-by: Ting-Yuan Huang <laszio@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/default-telemetry-results.json
bccc576843ee27fefda5094afc88489f4cbff3ff 07-Jun-2016 Cassidy Burden <cburden@google.com> Add install_script argument to binary search tool.

Add optional install_script argument that will run before test_script
use this install_script to perform building/flashing/other misc. setup.
If your build or flash succeeds then return 0, otherwise return 1 and
this build will be "skipped" (equivalent to the test script returning
a 2 exit code).

Test=Update unit tests to test for successful and failed install

Change-Id: Ic492f08230796f50b7bb93aebe078960419a3c99
Reviewed-on: https://chrome-internal-review.googlesource.com/262988
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
inary_search_tool/binary_search_state.py
inary_search_tool/test/binary_search_tool_tester.py
inary_search_tool/test/common.py
inary_search_tool/test/install.py
inary_search_tool/test/install_bad.py
inary_search_tool/test/is_good.py
42c64464b643df152c0f761fff4e7f6e257c27d5 03-Jun-2016 Cassidy Burden <cburden@google.com> Fix typo with package bisector switch_to_good script

Fix error where cros_pkg_switch_to_good.sh was using the bad build files
if passed a file argument.

Test=Switch to good verification passed with cros_pkg bisector

Change-Id: Ie2d5e1e741ceb97ba05d6f827161d12c103d9e48
Reviewed-on: https://chrome-internal-review.googlesource.com/262466
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
inary_search_tool/cros_pkg/cros_pkg_switch_to_good.sh
4a1e2c7b112f9a8b4afeff23686af5354c903738 01-Jun-2016 Cassidy Burden <cburden@google.com> Fix linting errors with binary search tool.

Fix basic style errors in BST tests and remove use of optparse across
scripts.

Change-Id: I4f484fb22ca6bb6849d79144bf7bdb7a38e49468
Reviewed-on: https://chrome-internal-review.googlesource.com/261739
Commit-Ready: Cassidy Burden <cburden@google.com>
Tested-by: Cassidy Burden <cburden@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Han Shen <shenhan@google.com>
inary_search_tool/binary_search_perforce.py
inary_search_tool/binary_search_state.py
inary_search_tool/test/gen_obj.py
inary_search_tool/test/is_good.py
inary_search_tool/test/switch_to_good.py
c4d58d04b0f92b52c645f0b8a0fc1d5e5b983e5f 25-May-2016 Caroline Tice <cmtice@google.com> [crosperf] Update bisecing triage tool to not update build_image.

The package bisecting tool used to actually edit the build_image
script, to prevent it from calling 'eclean', which messed up the
bisecting tool. Now there's a flag '-noeclean' that can be passed
to build_image instead. This CL updates the tool to use the flag,
which is much safer, rather than modifying the build_image script.
It also cleans up a few other minor issues with the tool

BUG=chromium:614829
TEST=Ran the tool with these changes.

Change-Id: If92148306c041b49d3fa45253ea5451bc840d862
Reviewed-on: https://chrome-internal-review.googlesource.com/260726
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
inary_search_tool/cros_pkg/README.cros_pkg_triage
inary_search_tool/cros_pkg/cros_pkg_boot_test.sh
inary_search_tool/cros_pkg/cros_pkg_create_cleanup_script.py
inary_search_tool/cros_pkg/cros_pkg_interactive_test.sh
inary_search_tool/cros_pkg/cros_pkg_setup.sh
c5c905559df6890ff5c59b88b7a6856346874faf 25-May-2016 Caroline Tice <cmtice@google.com> [Crosperf] Update machine names.

Update default_remotes to reflect the new names/locations of our
lab lumpy machines.

BUG=None
TEST=ping'd & ssh'd into the new names/locations.

Change-Id: Icc75361359ede9cf0aeb398563dd9939320a3531
Reviewed-on: https://chrome-internal-review.googlesource.com/260566
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
rosperf/default_remotes
1f3bdd80ac5d9489dbc1bad969f8757a545d1e77 16-May-2016 Caroline Tice <cmtice@google.com> [crosperf] Update machine locations.

This updates the default_remotes file so the machines reflect the
new names/locations for the HW lab (the machines got moved).

BUG=chromium:612170
TEST=Pinged the new machines; ssh'd into them to verify them; looked
at them on the AFE web site.

Change-Id: Ib27612555e6d8441bec4670d76fd4cf7b2b7b5a7
Reviewed-on: https://chrome-internal-review.googlesource.com/258934
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
rosperf/default_remotes
12e22fd759418bea20cce8cf996e3d6327c43931 13-May-2016 David Sharp <dhsharp@google.com> crosperf: Add script to measure overhead of some perf commands

BUG=none
TEST=Ran the script

Change-Id: I07d998df4e43e632308c90ea336c228c34b7a2a7
Reviewed-on: https://chrome-internal-review.googlesource.com/258705
Commit-Ready: David Sharp <dhsharp@google.com>
Tested-by: David Sharp <dhsharp@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
rosperf/experiment_files/telemetry_perf_perf
d1c31b2853e1b7bf47935cffd41ffd4f790d8d9f 18-Apr-2016 Han Shen <shenhan@google.com> Update build_tool.py to work for aosp-toolchain repos.

Fixed the build_tool.py so that it works for aosp binutils / gcc repos.

TEST=tested manually with a local binutils source tree and a local
binutils branch.
BUG=None

Change-Id: I4756abcf234651e18284be38e50d4b534aeff942
Reviewed-on: https://chrome-internal-review.googlesource.com/255905
Commit-Ready: Han Shen <shenhan@google.com>
Tested-by: Han Shen <shenhan@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
uild_tool.py
eprecated/repo_to_repo.py
epo_to_repo.py
74d85cf83860538fcf57c413cf8ae416baa0ef51 15-Apr-2016 Caroline Tice <cmtice@google.com> Fix error with json reports with multi-valued results.

BUG=None
TEST=Tested in in role account.

Change-Id: I502dda87f9a3647e8bea61a54f587813eb18b755
Reviewed-on: https://chrome-internal-review.googlesource.com/255739
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
rosperf/results_report.py
eddb06396dc17fcec418c3aadc234f737b63c876 14-Apr-2016 Caroline Tice <cmtice@google.com> [crosperf] Fix more lint errors.

BUG=chromium:567921
Test=Tested in my directory.

Change-Id: Ibe128dd030887e0372380fb8f95a5e37ee63feeb
Reviewed-on: https://chrome-internal-review.googlesource.com/255303
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
uildbot_test_toolchains.py
rosperf/benchmark_run.py
rosperf/benchmark_unittest.py
rosperf/config_unittest.py
rosperf/crosperf_test.py
rosperf/crosperf_unittest.py
rosperf/perf_table.py
rosperf/results_cache.py
rosperf/results_cache_unittest.py
rosperf/schedv2.py
rosperf/schedv2_unittest.py
rosperf/settings_factory_unittest.py
un_tests.py
eabfd367c30c20ecdf0adb1f0281fc02dca10102 13-Apr-2016 Caroline Tice <cmtice@google.com> Fix Crosperf to deal with a list of values being returned.

BUG=None
TEST=Tested in my tree.

Change-Id: I85e5729f857e7771e4a59a3879081e6504a1fb44
Reviewed-on: https://chrome-internal-review.googlesource.com/255298
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Han Shen <shenhan@google.com>
rosperf/results_cache.py
1a70e60b5e28403c184d6a73c97a01ba3402ada4 31-Mar-2016 Caroline Tice <cmtice@google.com> Update default remotes for new lumpy location.

The HW lab has moved one of the lumpy machines. This CL updates
default_remotes to reflect the new lumpy machine name.

BUG=None
TEST=looked at new machine on AFE.

Change-Id: Ia05a3928d6b4ed251662a3d0a51d4248d632634d
Reviewed-on: https://chrome-internal-review.googlesource.com/253100
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
rosperf/default_remotes
878f9d1108230185d5eb9962a11bfaccf534b1f9 30-Mar-2016 Rahul Chaudhry <rahulchaudhry@chromium.org> toolchain-utils: scripts and patches for running the full Go testsuite.

BUG=None
TEST=None

Change-Id: I37a4cd0c98c3aefe940472990ed3fec5691e6df7
Reviewed-on: https://chrome-internal-review.googlesource.com/253047
Commit-Ready: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
o/go_local
o/patch/go0.patch
o/patch/go1.patch
o/patch/go2.patch
o/patch/go3.patch
o/patch/go4.patch
o/patch/go5.patch
o/patch/go6.patch
o/patch/go7.patch
o/test_go
72725f0149c91eda72e6734717f5c857f4659d65 30-Mar-2016 Caroline Tice <cmtice@google.com> [crosperf] Update to use new telemetry json output, if present.

Autotest and telemetry and have changed their output and now put
out a json file instead of the old keyvals file. This CL updates
Crosperf to check for both kinds of output file and do the right
thing with either.

BUG=chromium:597099
TEST=Tested in nightly tests.

Change-Id: Ic7dc9d63d87917f50ff7f817bea7c1512594eb07
Reviewed-on: https://chrome-internal-review.googlesource.com/253019
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
rosperf/results_cache.py
285a66e9ee9d59c3e3051512db060ee485c440f0 30-Mar-2016 Caroline Tice <cmtice@google.com> [crosperf] Don't fail completely on missing result field.

After running to completion, while generating the json report
Crosperf sometimes tries to look for a result field that should be
there but isn't. Previously this would cause a runtime exception
and Crosperf would fail without generating any reports at all. This
CL fixes that (now a report still gets generated, without the one
missing field).

BUG=None
TEST=Tested this in nightly tests.

Change-Id: I35043e8c90fad6dbb5bfd5dede1905a2238bd3b7
Reviewed-on: https://chrome-internal-review.googlesource.com/253035
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Han Shen <shenhan@google.com>
rosperf/results_report.py
f637a050cc27b206824d7b1262d2aa9de0668bef 29-Mar-2016 Rahul Chaudhry <rahulchaudhry@chromium.org> toolchain-utils: scripts for building and testing the Go toolchain.

BUG=None
TEST=None

Change-Id: I07bd5ca5c2e005e3b476654ee3ff5fefc205e95a
Reviewed-on: https://chrome-internal-review.googlesource.com/252756
Commit-Ready: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
o/android/adb_shamu
o/android/adb_volantis
o/android/build_go
o/android/go_shamu
o/android/go_shamu_exec
o/android/go_volantis
o/android/go_volantis_exec
o/android/target_cp
o/android/target_sh
o/android/target_tmpdir
o/chromeos/build_go
o/chromeos/go_daisy
o/chromeos/go_daisy_exec
o/chromeos/go_panther
o/chromeos/go_panther_exec
o/chromeos/go_x86-zgb
o/chromeos/go_x86-zgb_exec
o/chromeos/target_cp
o/chromeos/target_sh
o/chromeos/target_tmpdir
o/go_target
o/go_target_exec
o/push_goroot
25b0005755697f571b816d286497c3fddfa06fd2 25-Mar-2016 Caroline Tice <cmtice@google.com> Update default_remotes to reflect new machine locations.

The stumpy & parrot chromebooks in the HW lab have been moved
again; this CL reflects their latest locations.

BUG=chromium:597689
TEST=Verified the new machines via the AFE website.

Change-Id: Iecb6186adcf242e53727857f81b1012f39c8798c
Reviewed-on: https://chrome-internal-review.googlesource.com/252406
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
rosperf/default_remotes
02586eb89d002d0e1474f02094119ba186aef423 24-Mar-2016 Caroline Tice <cmtice@google.com> Update default_remotes to reflect new machine locations.

The stumpy & parrot chromebooks in the HW lab have been moved;
this CL reflects their new locations.

BUG=chromium:597689
TEST=Verified the new machines via the AFE website.

Change-Id: Ia000621a117d923cb7d18442cf4ae9bbf66bbda5
Reviewed-on: https://chrome-internal-review.googlesource.com/252186
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
rosperf/default_remotes
70673dd6a98c2fd0860fac450b96cdf4fc541e71 09-Mar-2016 Luis Lozano <llozano@chromium.org> Add new parrots/stumpys to default_remote

See b/27522806

BUG=None
TEST=None

Change-Id: I81ed220c5c37b447bab78295f435baac359ff3cc
Reviewed-on: https://chrome-internal-review.googlesource.com/250685
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Luis Lozano <llozano@chromium.org>
rosperf/default_remotes
25d32b4add9a0d0ac4ec682062e45baea97bff04 03-Mar-2016 Caroline Tice <cmtice@google.com> [crosperf] Fix missing chrome_version problem (v2).

When generating the json reports, occasionally the chrome_version
comes up blank (this can happen when reading some results from the
cache, for example). A blank chrome_version messes up our dremel
database scripts. This CL updates the read-cache scenario to find
the chrome_version in the cache keys (human-readable) file.

BUG=chromium:591494
TEST=Tested with experiment that was generating blank Chrome
versions and this fixed the problem.

Change-Id: I29985581aa00bc30fb65720fb9d197f1475445e4
Reviewed-on: https://chrome-internal-review.googlesource.com/249960
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
rosperf/benchmark_run.py
rosperf/results_cache.py
eff2fc1c86365c53ae228df4653524ceb6168d92 24-Feb-2016 Rahul Chaudhry <rahulchaudhry@chromium.org> toolchain-utils: fix remaining cros lint errors.

BUG=chromium:570458
TEST='crosperf/run_tests.sh' passes.

Change-Id: I5566349cc97233e1ddb6d98121cd6f44e14bf0a9
Reviewed-on: https://chrome-internal-review.googlesource.com/249482
Commit-Ready: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
rosperf/crosperf.py
rosperf/suite_runner.py
tils/buildbot_json.py
tils/misc.py
72aaf19572d48184380153bea46d6017a01c9109 22-Feb-2016 Caroline Tice <cmtice@google.com> [crosperf] Fix default result for smoothness.tough_webgl_cases.

The name of the default summary field for this test has changed,
which makes crosperf fail since it can't find the right result
field. This CL fixes that problem.

BUG=None
TEST=Tested with nightly testd on chrotomation2.

Change-Id: I049c8abd2dd30b64aec8be65feb56b31115a4921
Reviewed-on: https://chrome-internal-review.googlesource.com/249130
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
rosperf/default-telemetry-results.json
c75fd0585d55b9a4b9d7999a41ce81c43dca0957 20-Feb-2016 Luis Lozano <llozano@chromium.org> Improve search for non-AFDO images for performace comparison.

To compare O2 vs AFDO, we try to use images from the chrome-pfq.
We now search the PFQ image created by the previous version of
chromeos since that is the one that validated the chrome for the
current version. If we dont find a previous, we try 2 versions
before.

BUG=None
TEST=White testing.ran simplified version of buildbot_test_toolchains.

Change-Id: I800255508117c7aa7f91468243e4713432e5561a
Reviewed-on: https://chrome-internal-review.googlesource.com/249245
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Luis Lozano <llozano@chromium.org>
uildbot_test_toolchains.py
99c79302832b804422ceb23cd6ab4f8113ebbed7 19-Feb-2016 Caroline Tice <cmtice@google.com> [crosperf] Fix recent change that is searching for wrong path.

The recent fixes added a check for the downloaded image that uses
the wrong filename (checks for the untar'd & uncompressed image).
This CL fixes this new bug.

BUG=None
TEST=Tested crosperf with this fix and downloading images now works
again.

Change-Id: Iebdb0cfb65885dbaf4a184bae89049f31739e302
Reviewed-on: https://chrome-internal-review.googlesource.com/248987
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
rosperf/download_images.py
c7ed2bca6f5dbf0cc6ee7cd1e7d30ea3119adbe8 19-Feb-2016 Luis Lozano <llozano@chromium.org> refactor and add some checks to image downloading code.

The nightly testing is failing in misterious ways while
downloading images for testing. I am adding a check for the
image after "gsutil cp" and, in the process, refactored
the code a little.

BUG=None
TEST=run_tests

Change-Id: I3e84ee9f2ea3b9b66cc96bf723bd1bcb94f26af7
Reviewed-on: https://chrome-internal-review.googlesource.com/248966
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Han Shen <shenhan@google.com>
rosperf/download_images.py
rosperf/download_images_unittest.py
rosperf/settings.py
rosperf/settings_unittest.py
969432132589753e9ef630c38c83d8f9ae2eccde 18-Feb-2016 Caroline Tice <cmtice@google.com> [crosperf] Remove media.tough_video_cases from nightly tests.

media.tough_video_cases does not have any good result field to
use and it has too high a stddev/mean to be a reliable test.
This CL removes it from our nightly test set.

BUG=None
TEST=None

Change-Id: Id80a9dd11de36a1fed01d7735f32e98908c7e1b0
Reviewed-on: https://chrome-internal-review.googlesource.com/248615
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Luis Lozano <llozano@chromium.org>
rosperf/experiment_factory.py
0435c01425e166f7f8b190d6f9021e7473ea8bc3 11-Feb-2016 Rahul Chaudhry <rahulchaudhry@chromium.org> toolchain-utils: fix bad-option-value lint warning.

The pylint directive is resulting in a pylint warning:
"Bad option value 'g-import-not-at-top' (bad-option-value)"

BUG=chromium:570458
TEST='crosperf/run_tests.sh' passes.

Change-Id: Ic016fb2a985836b9437c5cb090450d48cc93d56c
Reviewed-on: https://chrome-internal-review.googlesource.com/248318
Commit-Ready: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
Reviewed-by: Han Shen <shenhan@google.com>
tils/tabulator.py
6b4c5890248803137ab5a8ee3e60cb9dd6c734f0 11-Feb-2016 Rahul Chaudhry <rahulchaudhry@chromium.org> toolchain-utils: fix too-many-format-args lint warning.

BUG=chromium:570458
TEST='crosperf/run_tests.sh' passes.

Change-Id: I938307d3c937dac04a8b65489379ac6c141d6bd5
Reviewed-on: https://chrome-internal-review.googlesource.com/248317
Commit-Ready: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
tils/locks.py
33b20e20272c6ce7f3cb70d10ccaafad8a0ce47c 11-Feb-2016 Rahul Chaudhry <rahulchaudhry@chromium.org> toolchain-utils: fix arguments-differ lint warning.

BUG=chromium:570458
TEST='crosperf/run_tests.sh' passes.

Change-Id: I762f4e505740425445443cdd14dc7eaa3085f5c1
Reviewed-on: https://chrome-internal-review.googlesource.com/248316
Commit-Ready: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
tils/file_utils.py
6f29e8d8b6690987c4311236a77c95e53f9c4683 11-Feb-2016 Rahul Chaudhry <rahulchaudhry@chromium.org> toolchain-utils: disable eval-used lint warning.

It looks ok in this one case!!

BUG=chromium:570458
TEST='crosperf/run_tests.sh' passes.

Change-Id: I96be23784f1f6efc97a7d20e002ad9add8043f86
Reviewed-on: https://chrome-internal-review.googlesource.com/248315
Commit-Ready: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
tils/buildbot_json.py
42a9f4b89aaaec9a5f8e13415afcfa7b78868703 11-Feb-2016 Rahul Chaudhry <rahulchaudhry@chromium.org> toolchain-utils: disable global-statement lint warning.

It looks ok at these two places!!

BUG=chromium:570458
TEST='crosperf/run_tests.sh' passes.

Change-Id: I5e46fed0b4c9a6e3f80ffca360ed9be9a140f901
Reviewed-on: https://chrome-internal-review.googlesource.com/248234
Commit-Ready: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
tils/command_executer.py
tils/logger.py
8997f2f67506d42e005fc62383026f2da1dca31d 11-Feb-2016 Rahul Chaudhry <rahulchaudhry@chromium.org> toolchain-utils: fix bare-except lint warning.

"using a bare except: is almost never a good idea"
- https://docs.python.org/2/howto/doanddont.html#except

BUG=chromium:570458
TEST='crosperf/run_tests.sh' passes.

Change-Id: I2222fbc0fdc116961388c67b51ee259b49d0be88
Reviewed-on: https://chrome-internal-review.googlesource.com/248233
Commit-Ready: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/machine_manager_unittest.py
906dc58573d86c64aedc290864d0bc84302265bd 11-Feb-2016 Rahul Chaudhry <rahulchaudhry@chromium.org> toolchain-utils: fix unused-import lint warning.

BUG=chromium:570458
TEST='crosperf/run_tests.sh' passes.

Change-Id: I879b3d85264f1f1cabeaf1ee9703d373132fc78f
Reviewed-on: https://chrome-internal-review.googlesource.com/248232
Commit-Ready: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/machine_manager_unittest.py
8b06f631ad7b4b3c75438f58a8292eae7e444566 11-Feb-2016 Rahul Chaudhry <rahulchaudhry@chromium.org> toolchain-utils: disable arguments-differ lint warnings.

These are ok for overriding the setUp method of
unittest.Testcase when using the mock package.

BUG=chromium:570458
TEST='crosperf/run_tests.sh' passes.

Change-Id: Id501fe845d4b2cdf9029fc57b20a8e1b26ab7c27
Reviewed-on: https://chrome-internal-review.googlesource.com/248231
Commit-Ready: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/experiment_runner_unittest.py
rosperf/machine_manager_unittest.py
a091484e7d44ad185ee0936674a2baa6362c4e13 11-Feb-2016 Rahul Chaudhry <rahulchaudhry@chromium.org> toolchain-utils: fix no-name-in-module and import-error lint errors.

Many names in the email package were renamed in Python 2.5:
https://docs.python.org/release/2.7/library/email.html#package-history

BUG=chromium:570458
TEST='crosperf/run_tests.sh' passes.

Change-Id: I34acaa1d9c55d75cb0ea5e2eeca0968fb9288990
Reviewed-on: https://chrome-internal-review.googlesource.com/248230
Commit-Ready: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
Reviewed-by: Han Shen <shenhan@google.com>
tils/email_sender.py
d80e9a8970ed046d4a29ca7603d827f76360d577 10-Feb-2016 Caroline Tice <cmtice@google.com> [crosperf] Fix default result value for media.tough_video_cases.

The telemetry test media.tough_video_cases does not always
generate the result value that the defaults file was looking
for. This causes crosperf to fail entirely on this test sometimes.
Also, the old default value is no longer the "value of interest"
for that test. This CL removes the no-longer-valid default values.

BUG=chromium:585136
TEST=None

Change-Id: I484408f54dcac29efbec9879147d9490a3b17eba
Reviewed-on: https://chrome-internal-review.googlesource.com/248045
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
rosperf/default-telemetry-results.json
0d39863d8578fb1c6e7077b2c7cc5ca79edcf4dd 02-Feb-2016 Han Shen <shenhan@google.com> Fix lint for toolchain-utils.

Tested: run_tests.sh && cros lint on all the modified files.

Change-Id: Ife1e9620535f70849c58e932637b7b0f150b13ac
Reviewed-on: https://chrome-internal-review.googlesource.com/246650
Commit-Ready: Han Shen <shenhan@google.com>
Tested-by: Han Shen <shenhan@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
ros_login.py
rosperf/benchmark.py
rosperf/benchmark_run.py
rosperf/benchmark_run_unittest.py
rosperf/machine_manager.py
rosperf/suite_runner.py
c55a14c3dd2fe62775bc35a09bd3467389d9bdd8 30-Jan-2016 Rahul Chaudhry <rahulchaudhry@chromium.org> toolchain-utils: fix attribute-defined-outside-init lint errors.

BUG=chromium:570458
TEST='crosperf/run_tests.sh' passes.

Change-Id: If8c79fa74f30c40a67a9f0dee855a38d3e3a9c46
Reviewed-on: https://chrome-internal-review.googlesource.com/246491
Commit-Ready: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/experiment_runner_unittest.py
rosperf/machine_manager_unittest.py
tils/no_pseudo_terminal_test.py
tils/perf_diff.py
tils/tabulator.py
4f07bbf3bc13b73d42c94608a9707d6c6f890015 30-Jan-2016 Rahul Chaudhry <rahulchaudhry@chromium.org> toolchain-utils: remove uses of deprecated-module 'optparse'.

BUG=chromium:570458
TEST='crosperf/run_tests.sh' passes.

Change-Id: I0e239a91abad72920c0d5e6e70b43683ecccb868
Reviewed-on: https://chrome-internal-review.googlesource.com/246640
Commit-Ready: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
rosperf/crosperf.py
rosperf/crosperf_unittest.py
tils/perf_diff.py
37ef035cebdb8c0eb70f063d179b1a2f8e86145d 29-Jan-2016 Rahul Chaudhry <rahulchaudhry@chromium.org> toolchain-utils: fix lint warnings related to table_type argument.

This CL fixes docstring-missing-args and no-value-for-parameter
lint warnings related to the table_type argument.

BUG=chromium:570458
TEST='crosperf/run_tests.sh' passes.

Change-Id: I2d3b38b53efb78d39adb78d2c51015bc117b9c31
Reviewed-on: https://chrome-internal-review.googlesource.com/246639
Commit-Ready: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
tils/tabulator.py
30840d776f503121d551d25d44a7dc2de8ca62b3 29-Jan-2016 Rahul Chaudhry <rahulchaudhry@chromium.org> toolchain-utils: fix docstring-* lint warnings.

This CL fixes docstring-section-newline, docstring-arg-spacing,
and docstring-cuddled-quotes lint warnings.

BUG=chromium:570458
TEST='crosperf/run_tests.sh' passes.

Change-Id: I20c94d72199920ef1f78395dcc3f2ac051fc33cd
Reviewed-on: https://chrome-internal-review.googlesource.com/246638
Commit-Ready: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
tils/buildbot_json.py
tils/machines.py
tils/misc.py
tils/tabulator.py
021381f22366eca609b35a7d8d68916f03dc8536 29-Jan-2016 Rahul Chaudhry <rahulchaudhry@chromium.org> toolchain-utils: apply pyformat to the source file.

BUG=chromium:570458
TEST='crosperf/run_tests.sh' passes.

Change-Id: Id2e9b2d406ed5690da8d613369501f1d93c135f9
Reviewed-on: https://chrome-internal-review.googlesource.com/246637
Commit-Ready: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
rosperf/machine_manager_unittest.py
f96c6580aa907d08955a4e1e8130b92942e86a06 28-Jan-2016 David Sharp <dhsharp@google.com> crosperf: Canonicalize chromeos_root in Settings.GetXbuddyPath

This allows setting chromeos_root to values like "~/chromiumos", and the
tilde will get correctly expanded.

TEST=ran crosperf with chromeos_root=~/chromiumos, and it worked.
BUG=none

Change-Id: I0860721c2880a366aaa644f59a70c36ef93559d2
Reviewed-on: https://chrome-internal-review.googlesource.com/246159
Commit-Ready: David Sharp <dhsharp@google.com>
Tested-by: David Sharp <dhsharp@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/settings.py
3e8c2196b562fade5a3282288a0392fcb0087be4 28-Jan-2016 David Sharp <dhsharp@google.com> misc: Fix escaping of some regular expressions

The characters in the last pattern of GetFilenameFromString were
particularly mangled, and probably do not implement what was originally
intended. This change has it delete any of r"\" (backslash), "$", "=",
'"' (double quote), "?", or "^".

Also replace assertTrue(a == b) with assertEqual(a, b) in tests.

TEST=unit test
BUG=none

Change-Id: I9c12ce585e5a93be90a8e4054754115c47fdb7b2
Reviewed-on: https://chrome-internal-review.googlesource.com/246158
Commit-Ready: David Sharp <dhsharp@google.com>
Tested-by: David Sharp <dhsharp@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
tils/misc.py
tils/misc_test.py
9f9e6ac52051b8df2c11cdf51300fdc9ebce6d99 28-Jan-2016 David Sharp <dhsharp@google.com> crosperf: machine_manager_unittest should pass without access to locks dir

TEST=fixed unit test
BUG=none

Change-Id: I9c9817148b5eaf7ea8c0690051d47d39c7caf817
Reviewed-on: https://chrome-internal-review.googlesource.com/246157
Commit-Ready: David Sharp <dhsharp@google.com>
Tested-by: David Sharp <dhsharp@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/machine_manager_unittest.py
a20e03025ccce5670136f4aaf54f7bb5a0909b66 26-Jan-2016 David Sharp <dhsharp@google.com> image_chromeos: Rewrite cros flash retry and xbuddy detection

Rewrite the retry loop so that the first try is also in the
loop. (Reducing duplicated code.)

Use str.startswith to search for "xbuddy://". (Say what you
mean.)

Simplify assembly of cros_flash_args to reduce duplication.

TEST=running crosperf
BUG=none

Change-Id: I6158ce0ce0ad6a5e76492177a6d4700be3912f27
Reviewed-on: https://chrome-internal-review.googlesource.com/245902
Commit-Ready: David Sharp <dhsharp@google.com>
Tested-by: David Sharp <dhsharp@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
mage_chromeos.py
2358779f86a6aa86fb495d3181e5e11f06fda067 27-Jan-2016 Caroline Tice <cmtice@google.com> crosperf: Names of lumpy lab machines changed.

The ChromeOS HW lab has moved and renamed our lumpy machines.
This updates the default_remotes file to match the new names.

BUG=None
TEST=None

Change-Id: I0998b35c05f5cc7a7e5eb5de0b2821bf094cfa98
Reviewed-on: https://chrome-internal-review.googlesource.com/245923
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Rahul Chaudhry <rahulchaudhry@google.com>
rosperf/default_remotes
4d4565e25b98743e36b731d2c88916a5d440d16b 27-Jan-2016 Rahul Chaudhry <rahulchaudhry@chromium.org> toolchain-utils: strip argv[0] before calling setup_chromeos.Main()

This is a followup to https://chrome-internal-review.googlesource.com/#/c/245868

test_toolchains.py directly calls setup_chromeos.Main(). setup_chromeos.py was
changed recently to use argparse instead of optparse. Remove the name of the
script from the arguments to Main() to keep argparse happy.

BUG=None
TEST='nightly_wrapper.sh lumpy ...' proceeds with no error during argument parsing.

Change-Id: I30e8e6f1372676126b0af08125467096ea866266
Reviewed-on: https://chrome-internal-review.googlesource.com/246266
Commit-Ready: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
est_toolchains.py
a93683485612c985c661eb4ebbb02a7ae1c4a623 23-Jan-2016 David Sharp <dhsharp@google.com> crosperf: Make things work without access to file locks dir

Before this change, running with --dry_run would result in an
error accessing the lock file directory if the user does not
have access to the directory.

Also disable using file locks in machine_manager_unittest

TEST=run crosperf with --dry_run; unit tests
BUG=none

Change-Id: I8283bc38fd388dfa90fb772e76370a674e86b86f
Reviewed-on: https://chrome-internal-review.googlesource.com/245899
Commit-Ready: David Sharp <dhsharp@google.com>
Tested-by: David Sharp <dhsharp@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/benchmark_run_unittest.py
rosperf/experiment.py
rosperf/machine_manager.py
rosperf/machine_manager_unittest.py
rosperf/results_cache_unittest.py
748254e9a5dd7227806ca618d1f9a32ab49f4056 26-Jan-2016 Rahul Chaudhry <rahulchaudhry@chromium.org> toolchain-utils: strip argv[0] before calling Main().

https://chrome-internal-review.googlesource.com/#/c/244221 fixed some
cros lint errors, and in the process, modified test_toolchains.py to
use argparse instead of optparse.

The flags parsing semantics of these modules seem to differ when they
see an argument which is not a flag. As a result, "nightly_wrapper.sh"
nightly cron jobs are failing with error: "unrecognized arguments:
/usr/local/google/crostc/toolchain-utils/test_toolchains.py"

This CL strips the argv[0] before calling Main(), since the name of
the script is not really an argument and doesn't need to be parsed.

BUG=None
TEST='nightly_wrapper.sh lumpy ...' proceeds with no error during argument parsing.

Change-Id: Idf13ab457170bb051d4d0693a887795d0c16918d
Reviewed-on: https://chrome-internal-review.googlesource.com/245868
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Luis Lozano <llozano@chromium.org>
est_toolchains.py
49f3f09f8b5f243949968d88ce7fd9efe1393ad9 16-Jan-2016 Rahul Chaudhry <rahulchaudhry@chromium.org> toolchain-utils: disable protected-access lint warnings in tests.

crosperf/experiment_runner_unittest.py and crosperf/machine_manager_unittest.py
had a combined total of 53 protected-access lint warnings.

BUG=chromium:570458
TEST='crosperf/run_tests.sh' passes.

Change-Id: Icc31a63f3383068620314fd440460ab82df463a5
Reviewed-on: https://chrome-internal-review.googlesource.com/244868
Commit-Ready: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
rosperf/experiment_runner_unittest.py
rosperf/machine_manager_unittest.py
tils/tabulator_test.py
e7c6fbf9cfd5f45dda21f2c4742bf5a62589d2ed 16-Jan-2016 Rahul Chaudhry <rahulchaudhry@chromium.org> toolchain-utils: fix class-missing-docstring lint warnings.

BUG=chromium:570458
TEST='crosperf/run_tests.sh' passes.

Change-Id: I0640cab2c8d15380badbfbc0db39e3063cc85471
Reviewed-on: https://chrome-internal-review.googlesource.com/244867
Commit-Ready: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
inary_search_tool/test/binary_search_tool_tester.py
rosperf/crosperf.py
rosperf/experiment_runner_unittest.py
rosperf/machine_manager_unittest.py
rosperf/suite_runner.py
tils/buildbot_json.py
tils/command_executer_unittest.py
tils/misc_test.py
tils/no_pseudo_terminal_test.py
tils/perf_diff.py
tils/tabulator.py
tils/tabulator_test.py
tils/timeline.py
tils/timeline_test.py
e57523abd8f2a5aa0b93f1248d6d969d0865781b 15-Jan-2016 Rahul Chaudhry <rahulchaudhry@chromium.org> toolchain-utils: fix module-missing-docstring lint warnings.

BUG=chromium:570458
TEST='crosperf/run_tests.sh' passes.

Change-Id: Ifda134bf5f26db04551b915c189814b86e750df3
Reviewed-on: https://chrome-internal-review.googlesource.com/244866
Commit-Ready: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
inary_search_tool/cros_pkg/cros_pkg_create_cleanup_script.py
inary_search_tool/cros_pkg/cros_pkg_undo_eclean.py
inary_search_tool/test/common.py
rosperf/experiment_runner_unittest.py
rosperf/suite_runner.py
tils/file_utils.py
tils/html_tools.py
tils/logger.py
88272d479f2761cc1906fea564c73033f77a6270 13-Jan-2016 Caroline Tice <cmtice@google.com> Fix cros lint errors.

Also move deprecated scripts to the 'deprecated' directory.

BUG=chromiumos:570464
TEST=tested scripts to make sure they still work.

Change-Id: I3442a86d898104591233a0849ea0bafb52ecf1f7
Reviewed-on: https://chrome-internal-review.googlesource.com/244221
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
uto_delete_nightly_test_data.py
uild_benchmarks.py
uild_chrome_browser.py
uild_chromeos.py
uild_tc.py
uild_tool.py
hromiumos_image_diff.py
ommand_executer_timeout_test.py
ompare_benchmarks.py
rosperf/experiment_factory.py
rosperf/experiment_factory_unittest.py
eprecated/build_benchmarks.py
eprecated/compare_benchmarks.py
eprecated/repo_to_repo.py
eprecated/repo_to_repo_files/binutils-master.json.rtr
eprecated/repo_to_repo_files/binutils-mobile_toolchain_v16.json.rtr
eprecated/repo_to_repo_files/crosperf.json.rtr
eprecated/repo_to_repo_files/gcc-branches_google_4_7.json.rtr
eprecated/repo_to_repo_files/gcc-branches_google_main.json.rtr
eprecated/repo_to_repo_files/gcc-master.json.rtr
eprecated/repo_to_repo_files/gdb-master.json.rtr
eprecated/repo_to_repo_files/toolchain-utils.json.rtr
eprecated/report_generator.py
eprecated/run_benchmarks.py
eprecated/sheriff_rotation.py
eprecated/summarize_results.py
ile_lock_machine.py
et_common_image_version.py
eat_map.py
mage_chromeos.py
ock_machine_test.py
roduce_output.py
emote_gcc_build.py
emote_kill_test.py
emote_test.py
epo_to_repo.py
epo_to_repo_files/binutils-master.rtr
epo_to_repo_files/binutils-mobile_toolchain_v16.rtr
epo_to_repo_files/crosperf.rtr
epo_to_repo_files/gcc-branches_google_4_7.rtr
epo_to_repo_files/gcc-branches_google_main.rtr
epo_to_repo_files/gcc-master.rtr
epo_to_repo_files/gdb-master.rtr
epo_to_repo_files/toolchain-utils.rtr
eport_generator.py
un_benchmarks.py
etup_chromeos.py
heriff_rotation.py
ummarize_results.py
c_enter_chroot.py
est_gcc_dejagnu.py
est_gdb_dejagnu.py
est_toolchains.py
pdate_telemetry_defaults.py
eekly_report.py
439f2b77c86987362f53bd4f6e39896aa6d77f66 08-Jan-2016 Luis Lozano <llozano@chromium.org> Check that non AFDO images exist before trying to use them.

Check the non-AFDO images exist before we try to get performance
numbers for them.

BUG=None
TEST=white box testing.

Change-Id: I182c9da1dd29c377b98e3012ba7b5e186cdf2e4f
Reviewed-on: https://chrome-internal-review.googlesource.com/243956
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
uildbot_test_toolchains.py
tils/buildbot_utils.py
56f13765cdc7bb3f380ccbbc02b9af95af2e0592 06-Jan-2016 Yunlian Jiang <yunlian@google.com> toolchain-utils: check vanilla image before calling crosperf.

In some cases, the vanilla image builds finishes later than the
image from trybot, this could cause the nightly test job fail.
This CL makes sure the vanilla image is ready before calling
crosperf.

BUG=None
TEST=None

Change-Id: Id2f28717c5a8c6c6db0d6c2e240c687199ea6a7c
Reviewed-on: https://chrome-internal-review.googlesource.com/243618
Commit-Ready: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
uildbot_test_toolchains.py
tils/buildbot_utils.py
5dd0459f8a7207ddcee26baa4917c2a6e20fba67 30-Dec-2015 Luis Lozano <llozano@chromium.org> Modify pre-push hook to run pyformat.

Run pyformat before git push. Give error if files need formatting.
Also, made silly cleanup to pstat.py to be able to test the pre-push
hook.

BUG=None
TEST=white box testing

Change-Id: I79f12fa0acca4048b551f7dab25b0de8d5f05d75
Reviewed-on: https://chrome-internal-review.googlesource.com/243113
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Luis Lozano <llozano@chromium.org>
oolchain_utils_githooks/pre-push
oolchain_utils_githooks/pre-push.real
tils/pstat.py
6736a8e1044141d657aeac5916b32123da664e97 04-Jan-2016 Yunlian Jiang <yunlian@google.com> crosperf: fix a error.

_GetCacheDirForWrite was replaced by GetCacheDirForWrite
in result_cache.py, so this needs updated too.

BUG=none
TEST=it passes.

Change-Id: Ie3d63fbf2db2d3631644955cfa4bfda730099bdd
Reviewed-on: https://chrome-internal-review.googlesource.com/243468
Commit-Ready: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Han Shen <shenhan@google.com>
rosperf/experiment_runner.py
647950a869dbc19237843382b38b2e2867a3c73b 31-Dec-2015 Rahul Chaudhry <rahulchaudhry@chromium.org> toolchain-utils: fix docstring-second-line-blank lint warnings.

BUG=chromium:570458
TEST='crosperf/run_tests.sh' passes.

Change-Id: I5b8e7fafbfb73abedb03a1d01267b4216493773b
Reviewed-on: https://chrome-internal-review.googlesource.com/243456
Commit-Ready: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
inary_search_tool/cros_pkg/cros_pkg_create_cleanup_script.py
tils/buildbot_json.py
tils/buildbot_utils.py
3ebd06129a2d8cb9ff2301fffca77a9980989342 31-Dec-2015 Rahul Chaudhry <rahulchaudhry@chromium.org> toolchain-utils: fix docstring-leading-whitespace lint warnings.

BUG=chromium:570458
TEST='crosperf/run_tests.sh' passes.

Change-Id: I5f8cb5a89faa8d7f17d7b20a2198317a6f89b6d1
Reviewed-on: https://chrome-internal-review.googlesource.com/243455
Commit-Ready: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/suite_runner.py
b2d3dc7fbc760d1ea72d726e2316649c52b95493 31-Dec-2015 Rahul Chaudhry <rahulchaudhry@chromium.org> toolchain-utils: fix line-too-long lint warnings.

BUG=chromium:570458
TEST='crosperf/run_tests.sh' passes.

Change-Id: Ic4bd934dfb0b83e879a6f0c8a1ec121ac0a7de4b
Reviewed-on: https://chrome-internal-review.googlesource.com/243414
Commit-Ready: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
tils/buildbot_utils.py
d45e8a13e081b0c28618ddb94cb12e6c897859ff 31-Dec-2015 Rahul Chaudhry <rahulchaudhry@chromium.org> toolchain-utils: fix unused-argument lint warnings.

BUG=chromium:570458
TEST='crosperf/run_tests.sh' passes.

Change-Id: Iae6add81a1fb1c929a09808b5870d2a1762ef291
Reviewed-on: https://chrome-internal-review.googlesource.com/243413
Commit-Ready: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/experiment_runner_unittest.py
rosperf/machine_manager_unittest.py
tils/file_utils.py
tils/logger.py
cbc5a2617f8d63e3319e159eab2c407952df1df9 31-Dec-2015 Rahul Chaudhry <rahulchaudhry@chromium.org> toolchain-utils: fix missing-print-function lint warnings.

BUG=chromium:570458
TEST='crosperf/run_tests.sh' passes.

Change-Id: I2e0cf0d903b0ca7662132528ee6860ea8eaee83b
Reviewed-on: https://chrome-internal-review.googlesource.com/243412
Commit-Ready: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
inary_search_tool/cros_pkg/cros_pkg_create_cleanup_script.py
inary_search_tool/cros_pkg/cros_pkg_undo_eclean.py
inary_search_tool/test/binary_search_tool_tester.py
inary_search_tool/test/gen_init_list.py
inary_search_tool/test/switch_to_bad.py
rosperf/crosperf.py
rosperf/experiment_runner_unittest.py
rosperf/machine_manager_unittest.py
rosperf/suite_runner.py
tils/buildbot_json.py
tils/buildbot_utils.py
tils/command_executer.py
tils/command_executer_unittest.py
tils/email_sender.py
tils/file_utils.py
tils/locks.py
tils/logger.py
tils/machines.py
tils/manifest_versions.py
tils/misc.py
tils/misc_test.py
tils/no_pseudo_terminal_test.py
tils/perf_diff.py
tils/tabulator.py
tils/tabulator_test.py
tils/timeline.py
tils/timeline_test.py
278fc7437c8a31554903161eadfac2550507a47f 31-Dec-2015 Rahul Chaudhry <rahulchaudhry@chromium.org> toolchain-utils: apply pyformat to the source files.

BUG=chromium:570458
TEST='crosperf/run_tests.sh' passes.

Change-Id: I0cb94c77c26941fafb97cca2b9e9bb12cf4c210e
Reviewed-on: https://chrome-internal-review.googlesource.com/243411
Commit-Ready: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
inary_search_tool/__init__.py
inary_search_tool/test/__init__.py
rosperf/experiment_runner_unittest.py
rosperf/suite_runner.py
tils/buildbot_utils.py
tils/file_utils.py
tils/locks.py
tils/machines.py
tils/manifest_versions.py
tils/misc.py
tils/tabulator.py
1bd8cd1bb4d7697ce8591c90125dab962159ec4c 30-Dec-2015 Luis Lozano <llozano@chromium.org> Add a script to invoke pyformat.

Use this script to invoke pyformat according to our rules.

BUG=None
TEST=white box testing

Change-Id: Ia9563a60c8d47850dc52bfed6aa6ed88e30ffa86
Reviewed-on: https://chrome-internal-review.googlesource.com/243112
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Rahul Chaudhry <rahulchaudhry@google.com>
in/tc_pyformat
655dfa3bd83f6abbc7fe7cc0fa14ad38f379dcb6 30-Dec-2015 Yunlian Jiang <yunlian@google.com> crosperf: fix lint warnings.

BUG=chromium:570454
TEST=unittest passes, lint warning is gone.

Change-Id: I37f71a1195454657f205845d346b5ad76be878ec
Reviewed-on: https://chrome-internal-review.googlesource.com/243141
Commit-Ready: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
rosperf/results_cache.py
rosperf/results_cache_unittest.py
4ef135202ab304979e8cd53e8ac915a60aa456d3 29-Dec-2015 Yunlian Jiang <yunlian@chromium.org> crosperf: fix lint warnings.

BUG=chromium:570454
TEST=the lint warning is gone, unittest still passes.

Change-Id: Icdb9c0ba6ecdd7a04bec8eafc96f84813e59b583
Reviewed-on: https://chrome-internal-review.googlesource.com/243139
Commit-Ready: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
rosperf/results_cache_unittest.py
aefb76c242bd0df909e6d35fafd3d1682f0e0629 29-Dec-2015 Yunlian Jiang <yunlian@chromium.org> crosperf: fix lint warnings.

BUG=chromium:570454
TEST=fixed some lint warnings.

Change-Id: I645331f5d997cf7b470792239762fc0b78b21261
Reviewed-on: https://chrome-internal-review.googlesource.com/243239
Commit-Ready: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
rosperf/results_cache_unittest.py
26d821d63ace64a4ea72ff1b7ad262b232f2e607 29-Dec-2015 Yunlian Jiang <yunlian@chromium.org> crosperf: fix lint warnings.

BUG=chromium:570454
TEST=unittest passes, lint warning is gone.

Change-Id: I4801d560990ad2ed20e30d5cefd5f1ec2c6c5a3d
Reviewed-on: https://chrome-internal-review.googlesource.com/243408
Commit-Ready: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
rosperf/download_images.py
rosperf/download_images_buildid_test.py
rosperf/download_images_unittest.py
054a1230c49dc49d09c015ed83f9aeb3a96b87da 29-Dec-2015 Luis Lozano <llozano@chromium.org> Avoid lint of couple of files we did not author.

colortrans.py, pstat.py and stats.py were not written by us and we are
not maintaining them. They don't comply to our coding conventions. So,
it does not make sense to lint them.

BUG=chromium:570456
chromium:567921
TEST=None (trivial)

Change-Id: I20469396a55f6d5da0d73f5c7d33204fe6728c33
Reviewed-on: https://chrome-internal-review.googlesource.com/243258
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Luis Lozano <llozano@chromium.org>
tils/colortrans.py
tils/pstat.py
tils/stats.py
6853c967e4ca04f609eb52daeb0ceca873f730dc 23-Dec-2015 Rahul Chaudhry <rahulchaudhry@chromium.org> toolchain-utils: remove superfluous parentheses.

This CL fixes all superfluous-parens lint warnings.

BUG=chromium:570458
TEST='crosperf/run_tests.sh' passes.

Change-Id: I96cabd8e348787fbb86da221dd3d6f04ce210041
Reviewed-on: https://chrome-internal-review.googlesource.com/243097
Commit-Ready: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
tils/buildbot_json.py
tils/logger.py
9c59882cbda911cee0ecb908258ea3ee3cd2b6bc 23-Dec-2015 Rahul Chaudhry <rahulchaudhry@chromium.org> toolchain-utils: remove unused variables.

This CL fixes all unused-variable lint warnings.

BUG=chromium:570458
TEST='crosperf/run_tests.sh' passes.

Change-Id: I4f38ca9a428738125b09dc38b4ad8778dab77886
Reviewed-on: https://chrome-internal-review.googlesource.com/243119
Commit-Ready: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
tils/buildbot_utils.py
tils/file_utils.py
tils/logger.py
tils/tabulator.py
6405ba7d1c0396ccab8b41276654b428215e6076 23-Dec-2015 Rahul Chaudhry <rahulchaudhry@chromium.org> toolchain-utils: fix wrong continued indentation.

This CL fixes all bad-continuation lint warnings.

BUG=chromium:570458
TEST='crosperf/run_tests.sh' passes.

Change-Id: I8bb9cb6eb60f169a63c2e3e0f717924463da2a89
Reviewed-on: https://chrome-internal-review.googlesource.com/243257
Commit-Ready: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
tils/logger.py
d7444b7a6d96440d0229d898401a59f141576381 23-Dec-2015 Rahul Chaudhry <rahulchaudhry@chromium.org> toolchain-utils: fix shebang related lint warnings.

This CL fixes all missing-shebang and bad-shebang lint warnings.

BUG=chromium:570458
TEST='crosperf/run_tests.sh' passes.

Change-Id: I41bd3516496b246e4e4a8ccc9f97fb25007738a7
Reviewed-on: https://chrome-internal-review.googlesource.com/243136
Commit-Ready: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
inary_search_tool/__init__.py
inary_search_tool/cros_pkg/cros_pkg_create_cleanup_script.py
inary_search_tool/cros_pkg/cros_pkg_undo_eclean.py
inary_search_tool/test/__init__.py
inary_search_tool/test/binary_search_tool_tester.py
inary_search_tool/test/common.py
inary_search_tool/test/gen_init_list.py
inary_search_tool/test/switch_to_bad.py
rosperf/crosperf.py
rosperf/experiment_runner_unittest.py
rosperf/machine_manager_unittest.py
tils/buildbot_json.py
tils/command_executer_unittest.py
tils/email_sender.py
tils/perf_diff.py
9959b1eb9166ed81840c7e08ff179f00a5448cf2 23-Dec-2015 Rahul Chaudhry <rahulchaudhry@chromium.org> toolchain-utils: remove unused imports.

This CL fixes all unused-import lint warnings.

BUG=chromium:570458
TEST='crosperf/run_tests.sh' passes.

Change-Id: I145cebe61117388e729cf1b67001cb727fee1b33
Reviewed-on: https://chrome-internal-review.googlesource.com/243108
Commit-Ready: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
inary_search_tool/cros_pkg/cros_pkg_create_cleanup_script.py
rosperf/experiment_runner_unittest.py
rosperf/machine_manager_unittest.py
tils/file_utils.py
tils/locks.py
tils/misc.py
tils/misc_test.py
783954f594bf8ae0145c1e79cc5792b98df2e524 22-Dec-2015 Luis Lozano <llozano@chromium.org> Run experiments on non-AFDO image in our nightly testing.

We need to compare performance of AFDO vs non-AFDO built images.

BUG=None
TEST=in progress

Change-Id: I27f4cca5d3852521ed160aa7ee7f7576c1781e0a
Reviewed-on: https://chrome-internal-review.googlesource.com/243135
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Han Shen <shenhan@google.com>
uildbot_test_toolchains.py
4c416455faa23de6382eb2d6d39cfd56d583c37e 22-Dec-2015 Yunlian Jiang <yunlian@chromium.org> crosperf: fix lint warnings.

BUG=chromium:570454
TEST=unittest passes, lint warning is gone.

Change-Id: I230e9de9c1912f05774444d15310ad70e07e264b
Reviewed-on: https://chrome-internal-review.googlesource.com/243215
Commit-Ready: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
rosperf/suite_runner_unittest.py
tils/logger.py
b5cdd417bf92e40c9e77caf2e84f733bf1e51437 18-Dec-2015 Yunlian Jiang <yunlian@google.com> crosperf: clean lint warnings.

BUG=chromium:567921
TEST=it is lint clean now.

Change-Id: I4197ff8b16bc70edfd8570e5a619c6486a7c6d2e
Reviewed-on: https://chrome-internal-review.googlesource.com/242711
Commit-Ready: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Han Shen <shenhan@google.com>
rosperf/suite_runner_unittest.py
fbda04944190550706dfc8edfbd34c72d194e4fd 22-Dec-2015 Yunlian Jiang <yunlian@chromium.org> crosperf: fix lint warnings.

BUG=none
TEST=unittest passes.

Change-Id: I1fb0c8c3fbf31fb4a26b7e02c34207f16541e0b5
Reviewed-on: https://chrome-internal-review.googlesource.com/243086
Commit-Ready: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
rosperf/results_cache_unittest.py
944674b1e3c67d53b802337ae4bc7d11c269f652 21-Dec-2015 Yunlian Jiang <yunlian@chromium.org> crosperf: fix lint errors.

BUG=chromium:570454
TEST=it is lint clean.

Change-Id: I6d13788e7f5f75c2c2478e9d9b1343e21b56e53f
Reviewed-on: https://chrome-internal-review.googlesource.com/242947
Commit-Queue: Yunlian Jiang <yunlian@google.com>
Trybot-Ready: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Han Shen <shenhan@google.com>
rosperf/download_images_unittest.py
76cdacea0fef665b8540e78b79fc5dca83d43882 17-Dec-2015 Caroline Tice <cmtice@google.com> Create pre-push file, to run unittests before uploading CL.

This CL creates a pre-push file, to optionally run unittests
when people do 'git push'. The file will need to be copied
into .git/hooks when people set up their toolchain-utils (similar
to the way they need to get the commit-msg file).

BUG=chromium:567918
TEST=Tested in .git/hooks in my toolchain-utils tree.

Change-Id: I706e399554c89fca645b995332d17cb081021088
Reviewed-on: https://chrome-internal-review.googlesource.com/242523
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
oolchain_utils_githooks/pre-push
c3fdcc2da9c4c56f007199affb5cfa2ae6860ee8 17-Dec-2015 Caroline Tice <cmtice@google.com> Add final status line to run_tests.sh.

Now that run_tests.sh doesn't stop at the first failing test, we
need a final status line to tell us how many unittest files it ran
and how many passed/failed. This CL does that.

Last line of output now looks like this:

ALL TESTS PASSED (19 ran) or 3 TESTS FAILED (out of 19)

BUG=chromium:570742
TEST=Tested in my directory.

Change-Id: I75e5da8e64a9a527b1086b9a314dda1e2fb01678
Reviewed-on: https://chrome-internal-review.googlesource.com/242578
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Rahul Chaudhry <rahulchaudhry@google.com>
rosperf/run_tests.sh
f16faa3c13aad409ebdfea30c2ae6a1a03c57ffb 17-Dec-2015 Yunlian Jiang <yunlian@google.com> crosperf: fix lint warnings.

BUG=chromium:570454
TEST=crosperf_unittest.py is lint clean.

Change-Id: Ie33b95ec0cda61749b021e863ee65a47b491bb48
Reviewed-on: https://chrome-internal-review.googlesource.com/242520
Commit-Ready: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/crosperf_unittest.py
8d818fb82f1c2ba4ee0277e85dba919fa1d19586 17-Dec-2015 Caroline Tice <cmtice@google.com> Fix unittests that recent changes broke.

BUG=chromium:570522
TEST=Tested in my directory.

Change-Id: I197f5f5be3399babc838f4345173625e4d5bd91c
Reviewed-on: https://chrome-internal-review.googlesource.com/242565
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
rosperf/experiment_runner_unittest.py
rosperf/machine_manager_unittest.py
rosperf/results_cache_unittest.py
633ab684d64f68b03e8aa49f4bf9ef4193840275 16-Dec-2015 Yunlian Jiang <yunlian@google.com> toolchain-utils: clean lint warning.

BUG=chromium:567921
TEST=binary_search_state.py is lint clean.

Change-Id: Ie5f3b056e64167ac715b1cb03cc796aa4f3f33be
Reviewed-on: https://chrome-internal-review.googlesource.com/242515
Commit-Ready: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
inary_search_tool/binary_search_state.py
b860c4b6d6bda8bda32bcbf27adf3d8118c81ed7 16-Dec-2015 Yunlian Jiang <yunlian@google.com> toolchani-utils: clean lint warning for binary_search_perforce.py

BUG=chromium:567921
TEST=binary_search_perforce.py is lint clean.

Change-Id: Ib77e93f719df0de740e8954c67de40fd754dd245
Reviewed-on: https://chrome-internal-review.googlesource.com/242505
Commit-Ready: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Han Shen <shenhan@google.com>
inary_search_tool/binary_search_perforce.py
d97422aef5709f0a3b16b4efebc397c891940c95 16-Dec-2015 Yunlian Jiang <yunlian@google.com> crosperf: clean lint warnings.

BUG=chromium:567921
TEST=these three files are lint warning free.

Change-Id: I23f9120283fc82958abcc16f47cd29d5800bd181
Reviewed-on: https://chrome-internal-review.googlesource.com/242485
Commit-Ready: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
fe_lock_machine.py
uto_delete_nightly_test_data.py
rosperf/benchmark_run.py
f2a3ef46f75d2196a93d3ed27f4d1fcf22b54fbe 15-Dec-2015 Luis Lozano <llozano@chromium.org> Run pyformat on all the toolchain-utils files.

This gets rid of a lot of lint issues.
Ran by doing this:

for f in *.py; do echo -n "$f " ; if [ -x $f ]; then pyformat -i
--remove_trailing_comma --yapf --force_quote_type=double $f ; else
pyformat -i --remove_shebang --remove_trailing_comma --yapf
--force_quote_type=double $f ; fi ; done

BUG=chromium:567921
TEST=Ran simple crosperf run.

Change-Id: I59778835fdaa5f706d2e1765924389f9e97433d1
Reviewed-on: https://chrome-internal-review.googlesource.com/242031
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
fe_lock_machine.py
uto_delete_nightly_test_data.py
utomation/__init__.py
utomation/all_tests.py
utomation/clients/__init__.py
utomation/clients/android.py
utomation/clients/chromeos.py
utomation/clients/crosstool.py
utomation/clients/dejagnu_compiler.py
utomation/clients/helper/android.py
utomation/clients/helper/chromeos.py
utomation/clients/helper/crosstool.py
utomation/clients/helper/jobs.py
utomation/clients/helper/perforce.py
utomation/clients/nightly.py
utomation/clients/output_test.py
utomation/clients/pwd_test.py
utomation/clients/report/dejagnu/__init__.py
utomation/clients/report/dejagnu/main.py
utomation/clients/report/dejagnu/manifest.py
utomation/clients/report/dejagnu/report.py
utomation/clients/report/dejagnu/summary.py
utomation/clients/report/validate_failures.py
utomation/common/__init__.py
utomation/common/command.py
utomation/common/command_executer.py
utomation/common/command_executer_test.py
utomation/common/events.py
utomation/common/job.py
utomation/common/job_group.py
utomation/common/logger.py
utomation/common/machine.py
utomation/common/machine_test.py
utomation/common/state_machine.py
utomation/server/__init__.py
utomation/server/job_executer.py
utomation/server/job_group_manager.py
utomation/server/job_manager.py
utomation/server/machine_manager.py
utomation/server/machine_manager_test.py
utomation/server/monitor/__init__.py
utomation/server/monitor/dashboard.py
utomation/server/monitor/manage.py
utomation/server/monitor/settings.py
utomation/server/monitor/urls.py
utomation/server/server.py
utomation/server/server_test.py
estflags/example_algorithms.py
estflags/flags.py
estflags/flags_test.py
estflags/flags_util.py
estflags/generation.py
estflags/generation_test.py
estflags/genetic_algorithm.py
estflags/hill_climb_best_neighbor.py
estflags/iterative_elimination.py
estflags/mock_task.py
estflags/pipeline_process.py
estflags/pipeline_process_test.py
estflags/pipeline_worker.py
estflags/pipeline_worker_test.py
estflags/steering.py
estflags/steering_test.py
estflags/task.py
estflags/task_test.py
estflags/testing_batch.py
inary_search_tool/__init__.py
inary_search_tool/binary_search_perforce.py
inary_search_tool/binary_search_state.py
inary_search_tool/cros_pkg/cros_pkg_create_cleanup_script.py
inary_search_tool/cros_pkg/cros_pkg_undo_eclean.py
inary_search_tool/test/__init__.py
inary_search_tool/test/binary_search_tool_tester.py
inary_search_tool/test/common.py
inary_search_tool/test/gen_init_list.py
inary_search_tool/test/gen_obj.py
inary_search_tool/test/is_good.py
uild_benchmarks.py
uild_chrome_browser.py
uild_chromeos.py
uild_tc.py
uild_tool.py
uildbot_test_toolchains.py
ommand_executer_timeout_test.py
ompare_benchmarks.py
rb/autotest_gatherer.py
rb/autotest_run.py
rb/crb_driver.py
rb/machine_manager_singleton.py
rb/table_formatter.py
ros_login.py
rosperf/benchmark.py
rosperf/benchmark_run.py
rosperf/benchmark_run_unittest.py
rosperf/benchmark_unittest.py
rosperf/column_chart.py
rosperf/compare_machines.py
rosperf/config.py
rosperf/config_unittest.py
rosperf/crosperf.py
rosperf/crosperf_test.py
rosperf/crosperf_unittest.py
rosperf/download_images.py
rosperf/download_images_buildid_test.py
rosperf/download_images_unittest.py
rosperf/experiment.py
rosperf/experiment_factory.py
rosperf/experiment_factory_unittest.py
rosperf/experiment_file.py
rosperf/experiment_file_unittest.py
rosperf/experiment_runner.py
rosperf/experiment_runner_unittest.py
rosperf/experiment_status.py
rosperf/field.py
rosperf/flag_test_unittest.py
rosperf/help.py
rosperf/image_checksummer.py
rosperf/label.py
rosperf/machine_image_manager.py
rosperf/machine_image_manager_unittest.py
rosperf/machine_manager.py
rosperf/machine_manager_unittest.py
rosperf/mock_instance.py
rosperf/perf_table.py
rosperf/results_cache.py
rosperf/results_cache_unittest.py
rosperf/results_organizer.py
rosperf/results_organizer_unittest.py
rosperf/results_report.py
rosperf/results_sorter.py
rosperf/schedv2.py
rosperf/schedv2_unittest.py
rosperf/settings.py
rosperf/settings_factory.py
rosperf/settings_factory_unittest.py
rosperf/settings_unittest.py
rosperf/suite_runner.py
rosperf/suite_runner_unittest.py
rosperf/test_flag.py
rosperf/translate_xbuddy.py
wp/bartlett/server.py
wp/bartlett/test/server_tester.py
wp/interpreter/app_engine_pull.py
wp/interpreter/symbolizer.py
wp/performance/experiment_gen.py
ejagnu/__init__.py
ejagnu/gdb_dejagnu.py
ejagnu/run_dejagnu.py
do_scripts/divide_and_merge_profiles.py
do_scripts/divide_and_merge_profiles_test.py
do_scripts/profile_cycler.py
do_scripts/summarize_hot_blocks.py
do_scripts/vanilla_vs_fdo.py
ile_lock_machine.py
et_common_image_version.py
eat_map.py
mage_chromeos.py
ock_machine_test.py
em_tests/clean_data.py
em_tests/mem_groups.py
em_tests/total_mem_actual.py
em_tests/total_mem_sampled.py
em_tests/utils.py
roduce_output.py
emote_gcc_build.py
emote_kill_test.py
emote_test.py
epo_to_repo.py
eport_generator.py
un_benchmarks.py
un_tests.py
etup_chromeos.py
heriff_rotation.py
ummarize_results.py
c_enter_chroot.py
est_gcc_dejagnu.py
est_gdb_dejagnu.py
est_toolchains.py
pdate_telemetry_defaults.py
tils/__init__.py
tils/buildbot_json.py
tils/buildbot_utils.py
tils/colortrans.py
tils/command_executer.py
tils/command_executer_unittest.py
tils/constants.py
tils/email_sender.py
tils/file_utils.py
tils/html_tools.py
tils/locks.py
tils/logger.py
tils/machines.py
tils/manifest_versions.py
tils/misc.py
tils/misc_test.py
tils/no_pseudo_terminal_test.py
tils/perf_diff.py
tils/pstat.py
tils/stats.py
tils/tabulator.py
tils/tabulator_test.py
tils/timeline.py
tils/timeline_test.py
eekly_report.py
2a66f70fef907c1cb15229cb58e5129cb620ac98 15-Dec-2015 Caroline Tice <cmtice@google.com> Fix arguments to RunCommand from RunCommands.

Under the recent RunCommand changes, all the arguments to RunCommand
except the command itself must be keyword args. The initial change
missed fixing this in the RunCommands function, which resulted in
an assertion error in the overnight test. This CL fixes that issue.

BUG=None
TEST=Tested running setup_chromeos with the role account (where
previous failur occurred).

Change-Id: I73834309afc5cdbddd8953de5287bf298fb80b7e
Reviewed-on: https://chrome-internal-review.googlesource.com/241980
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
tils/command_executer.py
65ce172366a93dd3f5aa6630f4c97581bba9cf59 15-Dec-2015 Yunlian Jiang <yunlian@google.com> crosperf: improve summary of results show on the terminal

When using verbose mode, crosperf prints out elapsed time since last
event for each test run. This is not much useful except for the running
jobs. So this CL uses compact output even the --logging-level=debug is
set.

BUG=chroium:564890
TEST=it uses compact mode even for logging-level=verbose.

Change-Id: Iec8dccc0a1a5bc610907e86515a2f800b247cad1
Reviewed-on: https://chrome-internal-review.googlesource.com/242069
Commit-Ready: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/experiment_status.py
6a7dfb33f661a0e0193716a3fcc46a6984486401 15-Dec-2015 Caroline Tice <cmtice@google.com> Fix issues caused by 'None' being default for List fields.

BUG=None
TEST=Tested in my directory.

Change-Id: Ib355808db1923b51455f0a2c1d83bc9139030c5d
Reviewed-on: https://chrome-internal-review.googlesource.com/242215
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/experiment_factory.py
rosperf/experiment_runner.py
036c9233742004aa773a374df381b1cf137484f5 10-Dec-2015 Luis Lozano <llozano@chromium.org> crosperf: RunCommand should return one type of object.

Cleaned up the interfaces for the RunCommand routines.
These were returning different types (int or tuple) depending on
the value of the return_ouput parameter.
Returning different unrelated types from a routine is bad practice.
Linter complains about this with several warnings like this:

"Attempting to unpack a non-sequence defined at line XY of
utils.command_executer"

BUG=chromium:566256
TEST=ran crosperf with a example experiment file
Ran run_tests.

Change-Id: Ibb83ab9322c87558077fc4937ef5c0686bbe5417
Reviewed-on: https://chrome-internal-review.googlesource.com/241459
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Han Shen <shenhan@google.com>
uto_delete_nightly_test_data.py
inary_search_tool/binary_search_state.py
inary_search_tool/test/binary_search_tool_tester.py
inary_search_tool/test/gen_init_list.py
uild_chrome_browser.py
uild_tool.py
hromiumos_image_diff.py
rb/autotest_run.py
rb/machine_manager_singleton.py
rosperf/download_images.py
rosperf/machine_manager.py
rosperf/machine_manager_unittest.py
rosperf/results_cache.py
rosperf/results_cache_unittest.py
rosperf/schedv2.py
rosperf/suite_runner.py
rosperf/suite_runner_unittest.py
ejagnu/gdb_dejagnu.py
ejagnu/run_dejagnu.py
do_scripts/vanilla_vs_fdo.py
eat_map.py
mage_chromeos.py
emote_gcc_build.py
epo_to_repo.py
c_enter_chroot.py
tils/buildbot_utils.py
tils/command_executer.py
tils/file_utils.py
tils/misc.py
e627fd61c2edba668eb2af8221892286b13f05a3 11-Dec-2015 Caroline Tice <cmtice@google.com> Crosperf: Finish fixing unittest problems.

This patch finishes fixing the unittest problems. They now all
work.

BUG=chromium:538397
TEST=./run_tests.sh now completes with no errors.

Change-Id: Ie501437d199bfafc8a84cd2a95862550992c2cb8
Reviewed-on: https://chrome-internal-review.googlesource.com/241925
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/benchmark_run_unittest.py
rosperf/download_images.py
rosperf/download_images_unittest.py
rosperf/experiment.py
rosperf/experiment_factory.py
rosperf/image_checksummer.py
rosperf/machine_image_manager.py
rosperf/machine_image_manager_unittest.py
rosperf/machine_manager.py
rosperf/machine_manager_unittest.py
rosperf/results_cache.py
rosperf/results_cache_unittest.py
rosperf/run_tests.sh
rosperf/schedv2_unittest.py
rosperf/settings_factory_unittest.py
75e1ccc6513c4529e47007105b7c523755f8e0c0 13-Dec-2015 Caroline Tice <cmtice@google.com> Fix two minor bugs introduced in the Fix-It that broke nightly tests.

BUG=None
TEST=Tested with role account running simple crosperf test.

Change-Id: Ib9d69ce8815b674ad57784d90c7e3cc7a3fba6ed
Reviewed-on: https://chrome-internal-review.googlesource.com/241967
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
rosperf/field.py
rosperf/results_report.py
14cf596e1e8c4e8677dd075dc5a0e7e480b9e3e9 12-Dec-2015 Yunlian Jiang <yunlian@google.com> buildbot_test_toolchains.py: clean lint warning.

This cleans up lint warnings and changes the condition
to decide whether it is a clang job.

BUG=chromium:567921
TEST=it is lint warning free.

Change-Id: I0742b57c3f6bf6d5839e58d28907ded45d1ee40a
Reviewed-on: https://chrome-internal-review.googlesource.com/241985
Commit-Ready: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
uildbot_test_toolchains.py
7174b7ab6ba9a023eab2202c5c4664be3ec6ba3d 11-Dec-2015 Luis Lozano <llozano@chromium.org> crosperf: remove call to deleted _ParseCPUInfo

This do-nothing routine was deleted in one of the lint cleanups
but the call to it was not removed.

BUG=None
TEST=None (trivial)

Change-Id: I0600d2f98454d8ed22c890f814f93f09c6f97e91
Reviewed-on: https://chrome-internal-review.googlesource.com/241444
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
rosperf/machine_manager.py
7057cf67ba1dbdd4387f53e5fe47b43c955b1a53 10-Dec-2015 Caroline Tice <cmtice@google.com> Crosperf: Fix many broken unittests.

Apparently people have been making changes and not running the
unittests; many of the unittests were suffering from bit-rot. This
CL fixes most of the broken unittests (there are still 3 large ones
being worked on -- that will be in another CL).

BUG=chromium:567883
TEST=Verified that these unittests work now.

Change-Id: I0252a276a035894e70e04c61339cdba148c9bbfd
Reviewed-on: https://chrome-internal-review.googlesource.com/241513
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Luis Lozano <llozano@chromium.org>
rosperf/benchmark_run.py
rosperf/benchmark_run_unittest.py
rosperf/crosperf_unittest.py
rosperf/experiment.py
rosperf/experiment_factory_unittest.py
rosperf/experiment_runner.py
rosperf/experiment_runner_unittest.py
rosperf/label.py
rosperf/machine_manager.py
rosperf/mock_instance.py
rosperf/schedv2_unittest.py
rosperf/suite_runner.py
rosperf/suite_runner_unittest.py
939274265fbac8c7f3d47c4e36c680d58cd578e9 10-Dec-2015 Yunlian Jiang <yunlian@google.com> crosperf: fix lint warning.

BUG=chromium:567921
TEST=settings_unittest.py is clean.

Change-Id: Id9ac86b7d07ece37f9e578550bdcde5084b573b6
Reviewed-on: https://chrome-internal-review.googlesource.com/241492
Commit-Ready: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/settings_unittest.py
25cc6609029758944641f58c4ba8189bbbf717ff 08-Dec-2015 Caroline Tice <cmtice@google.com> Improve error message when crosperf can't find image to download.

Currently if crosperf tries to download an image from
gs://chromeos-image-archive, it fails with a very cryptic message
that does NOT indicate what really went wrong. This CL makes
crosperf look for the image before attempting to download it, and
fixes the error messages to make sure they reflect the actual
problem. It also has some minor formatting fixes.

BUG=chromium:533526
TEST=Tested with both existing and non-existing image names.

Change-Id: Ic2f5e4f25f7d63c130895dd4cb078164a370f081
Reviewed-on: https://chrome-internal-review.googlesource.com/241057
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/download_images.py
93950177ce54e64d29b3d4793f0a37cc4b0b4cde 10-Dec-2015 Yunlian Jiang <yunlian@google.com> crosperf: clean lint warnings.

BUG=chromium:567921
TEST=machine_manager.py is clean.

Change-Id: Ia3355ee78d15419ca531f358812f7882a64a6cbe
Reviewed-on: https://chrome-internal-review.googlesource.com/241491
Commit-Ready: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/benchmark_run.py
rosperf/machine_manager.py
rosperf/machine_manager_unittest.py
a19dc5530d982f07cf829ac954c420654587f5cd 10-Dec-2015 Yunlian Jiang <yunlian@google.com> crosperf: fix lint warning.

BUG=chromium:567921
TEST=label.py is clean.

Change-Id: I23820750a7eb970187bbf11a82b7e41c6c9aaf7d
Reviewed-on: https://chrome-internal-review.googlesource.com/241490
Commit-Ready: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/label.py
a5d9505420d73e6096631c99d4b168a163ca5ad9 10-Dec-2015 Yunlian Jiang <yunlian@google.com> crosperf: clean lint warnings.

BUG=chromium:567921
TEST=these three files are lint clean.

Change-Id: I99c2f46263d12b8d84c933f52393e4207e9cd0dd
Reviewed-on: https://chrome-internal-review.googlesource.com/241458
Commit-Ready: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/results_sorter.py
rosperf/test_flag.py
rosperf/translate_xbuddy.py
661dae3250c449e8fc4184d085557532e4511023 10-Dec-2015 Yunlian Jiang <yunlian@google.com> crosperf: clean lint warning.

BUG=chromium:567921
TEST=experiment_status.py is lint clean.

Change-Id: I58553bfe9990b6b8e93e3aeee3daf7e634a96ac8
Reviewed-on: https://chrome-internal-review.googlesource.com/241570
Commit-Ready: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/experiment_status.py
0009415847e04be4670c025e1b475adc7de23cc8 10-Dec-2015 Yunlian Jiang <yunlian@google.com> crosperf: clean lint warning for results_report.py

BUG=chromium:567921
TEST=it is lint warning free now.

Change-Id: I623851a3420033f9a43e92c77423479c5cd766d9
Reviewed-on: https://chrome-internal-review.googlesource.com/241528
Commit-Ready: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
rosperf/results_report.py
pdate_telemetry_defaults.py
742ed2c8cd29f8f2d1ff28f3e3aea48307c71839 10-Dec-2015 Yunlian Jiang <yunlian@google.com> crosperf: clean lint warnings.

BUG=chromium:567921
TEST=these files are lint clean.

Change-Id: I889c80bd4fa2b8f67f3e386c86b68e373574911a
Reviewed-on: https://chrome-internal-review.googlesource.com/241489
Commit-Ready: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/experiment.py
rosperf/field.py
rosperf/help.py
rosperf/settings.py
1974c70a4e0b161a18a843229fc1af8344e11600 10-Dec-2015 Yunlian Jiang <yunlian@google.com> crosperf: clean results_organizer.py

BUG=chromium:567921
TEST=it is lint clean now.

Change-Id: I0cf9445f5db822462c53d75a138e984070319714
Reviewed-on: https://chrome-internal-review.googlesource.com/241429
Commit-Ready: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/results_organizer.py
6e8726de19436e28c4d22c3f3dc3bbae87dd4c12 09-Dec-2015 Caroline Tice <cmtice@google.com> Fix various Mock classes & calls so dry runs work.

The --dry_run option has been failing because parameter changes in
real classes have not always been reflected in Mock classes. This CL
fixes that.

BUG=chromium:545211
TEST=tested both real and --dry_run runs of Crosperf.

Change-Id: I0d0aa144e9fb3c40e1a95314691eacffb007cbe4
Reviewed-on: https://chrome-internal-review.googlesource.com/241525
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
rosperf/crosperf.py
rosperf/experiment_factory.py
rosperf/experiment_runner.py
rosperf/label.py
e55d35ee104a41e5f7ae96f4e326f33f6beaf412 09-Dec-2015 Yunlian Jiang <yunlian@google.com> crosperf: fix two file for lint warning.

BUG=chromium:567921
TEST=the warning is gone.

Change-Id: Id7c0b04728342992ddc3909fa2c94e98b9e70fba
Reviewed-on: https://chrome-internal-review.googlesource.com/241485
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/column_chart.py
rosperf/compare_machines.py
0d1a9f32c928e21a72547f3d334d631c5861f027 09-Dec-2015 Yunlian Jiang <yunlian@google.com> crosperf: use cros_utils instead of utils.

We got some lint warnings about some module is not in utils.
So we change the name.
This CL creates a symbolic link cros_utils from utils. Once we are sure
that we clean all the code, we can rename the utils to cros_utils
directly.

BUG=chromium:567921 chromium:568195
TEST=the lint warning is gone.

Change-Id: I776e45c7bdd75d1f065ea5797f3bcff457203ca2
Reviewed-on: https://chrome-internal-review.googlesource.com/241457
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
ros_utils
rosperf/benchmark_run.py
rosperf/config.py
rosperf/download_images_unittest.py
rosperf/experiment.py
rosperf/experiment_factory_unittest.py
rosperf/experiment_runner.py
rosperf/machine_manager.py
rosperf/results_cache.py
rosperf/results_cache_unittest.py
rosperf/results_report.py
rosperf/schedv2.py
rosperf/settings.py
rosperf/settings_factory_unittest.py
rosperf/settings_unittest.py
rosperf/suite_runner_unittest.py
51d7a9b5663f6d3067baf06e2fc266265937f61f 09-Dec-2015 Caroline Tice <cmtice@google.com> Fix error where sometimes crosperf tries to lock all machines.

Crosperf was calling afe_lock_machines even when none of the
user-requested machines were up and running; this caused
afe_lock_machines to attempt to lock all the machines it knew
about. This CL fixes that problem.

BUG=chromium:545674
TEST=Tested in my directory; works properly.

Change-Id: Ie632a1b8de04b7fa686886e981b5b9275f44f0dd
Reviewed-on: https://chrome-internal-review.googlesource.com/241435
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
rosperf/experiment.py
41ea8ed6b09b566f1db551ca4ffb3e27c8e3079f 09-Dec-2015 Yunlian Jiang <yunlian@google.com> crosperf: fix lint for expeirment_factory.py

BUG=chromium:567921
TEST=the file passes lint checking.

Change-Id: I6d2187d11a51f4e87b5bbd8b905e285bb4ed2475
Reviewed-on: https://chrome-internal-review.googlesource.com/241465
Commit-Ready: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/experiment_factory.py
a4486459137d57aaa39aa6ae0299fa85eb04e5f1 08-Dec-2015 Caroline Tice <cmtice@google.com> Fix lint errors in afe_lock_machine.py

BUG=None
TEST=Ran crosperf with newly formatted file.

Change-Id: I998b75761281a1ce9b0af8f606b47be8b8ca6aee
Reviewed-on: https://chrome-internal-review.googlesource.com/241175
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
fe_lock_machine.py
059b81adf209cc4a754accc50e93deaff8027a82 08-Dec-2015 Yunlian Jiang <yunlian@google.com> crosperf: fix three trival unitttest failures.

BUG=chromium:567883
TEST=these three test passes.

Change-Id: Ia8692cdee660802450d528d40c9a6b457a1cba31
Reviewed-on: https://chrome-internal-review.googlesource.com/241176
Commit-Ready: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/benchmark_unittest.py
rosperf/results_organizer_unittest.py
rosperf/settings_factory_unittest.py
dd4176150697727cdc89574698fd53f556c1304d 08-Dec-2015 Luis Lozano <llozano@chromium.org> Print friendlier error message when no benchmark is given.

Catch no-benchmarks and no-labels errors earlier to avoid
weird error messages later.

BUG=chromium:432616
TEST=tested with different experiment files.

Change-Id: I8a05dbac4a47a0f005f0b3c175770a0d88cb4a62
Reviewed-on: https://chrome-internal-review.googlesource.com/241096
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
rosperf/experiment.py
rosperf/experiment_factory.py
62422c48de4ab503020b73be0a400873ab96d21d 08-Dec-2015 Yunlian Jiang <yunlian@google.com> crosperf: use compact status output

Previous, it shows the status as
FAILD: benchmark(1), benchmark(2), benchmark(3)
After the change, it shows
FAILD: benchmark(1,2,3)

BUG=chromium:564890
TEST=it shows the status in the right way.

Change-Id: I714a66d75dcabb7a3146acd67190a85064b82b32
Reviewed-on: https://chrome-internal-review.googlesource.com/241045
Commit-Ready: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/experiment_status.py
1489d6440dc37585f3bb9325f3d12fbaa5a37b2b 08-Dec-2015 Luis Lozano <llozano@chromium.org> Re-enable graphics_WebGLAquarium from buildbots testing.

With some recent changes to the test_that invocation this should work
now.
Still there is an issue with benchmarks that require compilation
for the particular board. That remains to be fixed.

BUG=chromium:551710
TEST=None. This is a revert.

Change-Id: I547a4853302f11dc58ddc493ee5368b1e2d33c0e
Reviewed-on: https://chrome-internal-review.googlesource.com/241035
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Caroline Tice <cmtice@google.com>
uildbot_test_toolchains.py
rosperf/experiment_factory.py
3f43271ed61442fce6833ae118fe228a7ef76116 07-Dec-2015 Caroline Tice <cmtice@google.com> Display '.cros' at end of lab machine names; add it when missing.

Before this, this script was confusing because it would display
lab machine names without the '.cros' at the end, but it would require
the '.cros' at the end of lab machine names (from users) to recognize
them. This CL fixes that: Now when displaying machine status, it
makes sure the .cros is at the end of the lab machine names; and it
makes sure the various functions work for lab machines whether or not
the user specified the '.cros' at the end of the machine name (it adds
it automatically, if it's missing).

BUG=chromium:525198
TEST=Tested all the commands on lab machines with and without '.cros'.

Change-Id: I8fd36d116c3c542fad936be73b946c88a9228551
Reviewed-on: https://chrome-internal-review.googlesource.com/240880
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
fe_lock_machine.py
98eacdfb58f8b43554febe26de948d3fe36dfac5 08-Dec-2015 Han Shen <shenhan@google.com> Fix crbug/528063 - misleading machine checksum calculating message.

This message "Checking machine characteristics for
chromeos2-row5-rack5-host12.cros" is a bogus, it is caused by
mis-indention.

TEST=Manually run
BUG=chromium:528063

Change-Id: I49eb8294759a77cac4664bc24dd02d898375aabd
Reviewed-on: https://chrome-internal-review.googlesource.com/240947
Commit-Ready: Han Shen <shenhan@google.com>
Tested-by: Han Shen <shenhan@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
rosperf/machine_manager.py
208a5339679c74a9430cb39a598dcb4674644886 07-Dec-2015 Yunlian Jiang <yunlian@google.com> crosperf: show the elapsed time for the running job.

BUG=chromium:564890
TEST=it shows the current running time for the running job.

Change-Id: I1203abcf4455d289b34f02cf542bd5dae62b8d0a
Reviewed-on: https://chrome-internal-review.googlesource.com/240881
Commit-Ready: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
rosperf/experiment_status.py
3a7ada298dfaa66575ef72b9ed3ee21c9b784653 07-Dec-2015 Yunlian Jiang <yunlian@google.com> crosperf: fix a typo.

BUG=None
TEST=the typo is fixed.

Change-Id: I11249c57e9483a8d1c1a2ca72853b6870949364d
Reviewed-on: https://chrome-internal-review.googlesource.com/240878
Commit-Ready: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Han Shen <shenhan@google.com>
rosperf/experiment_factory.py
738e6dee10f6aa08941fbc8131bfe87ca6cb1685 07-Dec-2015 Han Shen <shenhan@google.com> Fix crbug/529948 - schedv2 missing functionality.

(Optinally) force same image across different board when compute machine
checksum fails.

BUG=chromium:529948
TEST=manually tested in console

Change-Id: Ia3a86c9a6796cf547842c473c0f8b7b1ccaf6907
Reviewed-on: https://chrome-internal-review.googlesource.com/240916
Commit-Ready: Han Shen <shenhan@google.com>
Tested-by: Han Shen <shenhan@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
rosperf/experiment.py
96da9060697ad889a82ceb7b2b0fbce8f6572c72 04-Dec-2015 Caroline Tice <cmtice@google.com> Fix parameters passed to non-telemetry autotest benchmarks.

The parameters being passed to the non-telemetry Benchmarks were
not quite right. This CL fixes that.

BUG=chromium:567333
TEST=Tested in my chroot with bvt tests.

Change-Id: Ied834e0904eec7801118d4996045d96c1fb4d329
Reviewed-on: https://chrome-internal-review.googlesource.com/240576
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/experiment_factory.py
d0b46129b0a60c84258a81d3dcce935cff6ba9c3 08-Dec-2015 Luis Lozano <llozano@chromium.org> Sort CPU frequencies before picking the highest one.

We need to sort the available frequencies before selecting
the highest one. The frequency list order changes from
board to board

BUG=chromium:564887
TEST=Verified by hand and on Jaq machine

Change-Id: Ief9ca216489fcf84b4c34688309b89be07d52c1c
Reviewed-on: https://chrome-internal-review.googlesource.com/240946
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
rosperf/suite_runner.py
c152cdb3fa9aca8e1c9c9699913ca17d50f1e42b 07-Dec-2015 Yunlian Jiang <yunlian@google.com> crosperf: use specific exceptions.

BUG=chromium:566255
TEST=crosperf unittest passes.

Change-Id: Ia618687c5cbc487dda205ff5e227a5483b26db24
Reviewed-on: https://chrome-internal-review.googlesource.com/240875
Commit-Ready: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Han Shen <shenhan@google.com>
rosperf/machine_manager.py
8df449353158325b7ce5c026c86098790cb06081 02-Dec-2015 Caroline Tice <cmtice@google.com> Make non-telemetry autotests run faster.

This CL adds some flags to the non-telemetry runs of test_that
which are already used with the telemetry tests, and which make
the autotest runs faster.

BUG=None
TEST=Tested in my directory with bvt tests.

Change-Id: I50e4bbefa9f7a66fcfff3b478386c23b17ac0ea3
Reviewed-on: https://chrome-internal-review.googlesource.com/240191
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
rosperf/suite_runner.py
585192fb87a05d0b2b24cc01d9a05b55c6b5849c 19-Nov-2015 Ting-Yuan Huang <laszio@google.com> Make 'run_local=True' default in crosperf.

BUG=None
TEST=tested with run_local={,True,False}

Change-Id: Id23c02b4111fafe8c8c392d39b15521045a78550
Reviewed-on: https://chrome-internal-review.googlesource.com/238861
Commit-Ready: Ting-Yuan Huang <laszio@google.com>
Tested-by: Ting-Yuan Huang <laszio@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
rosperf/settings_factory.py
b02b811163e5ca1be4506d53448ee71ba450508c 11-Nov-2015 Caroline Tice <cmtice@google.com> Add a new shell script to figure out which compiler built image.

This adds a new script that checks to see which compiler (GCC or
clang) was used to build all the images in a ChromeOS image. It
outputs a count of how many packages it found that were built with
each compiler.

BUG=None
TEST=Tested in my chroot with both a locally-built GCC image and a
trybot-built clang image.

Change-Id: I26b5faea8616b5427c5bbe27a58ea3c9d4e36043
Reviewed-on: https://chrome-internal-review.googlesource.com/238685
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Han Shen <shenhan@google.com>
ompiler-test.sh
7f20acb9c9036015a665a14f27633c639e9b9d91 05-Nov-2015 Luis Lozano <llozano@chromium.org> Temporarily remove graphics_WebGLAquarium from buildbot testing.

For the images that we build in trybots, this test cannot be
executed because we are not downloading the necessary artifacts
for autotest. See chromium:551710.
This change temporarily removes that test from the nightly testing that
we do based on trybot images. These changes should be removed
once this bug is fixed.

BUG=chromium:551710
TEST=None

Change-Id: I15ec9386d52809492345d9cd46a6f0ad39330dbc
Reviewed-on: https://chrome-internal-review.googlesource.com/237838
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Caroline Tice <cmtice@google.com>
uildbot_test_toolchains.py
rosperf/experiment_factory.py
80eab98c8820e503118ba0962cf9f0f74a51e28d 04-Nov-2015 Caroline Tice <cmtice@google.com> Make 'label' names more meaningful in nightly tests.

The nightly crosperf tests currently use the label 'test_image' for
the images that we build or that the trybot built. This is not
a very informative name. This CL tries to make the names a least a
little bit more useful.

BUG=None
TEST=Tested by hand in role account directory tree.

Change-Id: Ib88f711ec96ec5cb71efebae79a76e6e8ebc784a
Reviewed-on: https://chrome-internal-review.googlesource.com/237796
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
uildbot_test_toolchains.py
est_toolchains.py
1af9294993f129f2286c961a19a04babddd2572d 05-Nov-2015 Luis Lozano <llozano@chromium.org> Make cleaning of nightly data less verbose.

The nightly process for clean up of old data is too verbose
and makes the summary report difficult to follow.

BUG=None
TEST=None

Change-Id: I3b1e18c0d913e47d4281617970a7e9a2ea0d9eb1
Reviewed-on: https://chrome-internal-review.googlesource.com/237839
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Han Shen <shenhan@google.com>
uto_delete_nightly_test_data.py
89d263c7cf9773129cbe8e8858ad21ea539a2ba0 04-Nov-2015 Luis Lozano <llozano@chromium.org> Fixed bad key problem while generating json report.

In some weird cases, telemetry benchmarks incorrectly
return empty performance results. This was causing an invalid
key error while generating the json report.

BUG=chromium:551105
TEST=Forced generation of empty results and tested we dont
have the dictionary missing key issue.

Change-Id: I32803c573f1d268e010313292b99c0017a5dff92
Reviewed-on: https://chrome-internal-review.googlesource.com/237516
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/results_organizer.py
rosperf/results_report.py
8447c1eae629d107acb3b6802ceb5a9dc1a6cfdb 29-Oct-2015 Yunlian Jiang <yunlian@google.com> buildbot_utils: make it accept multiple patches.

BUG=None
TEST=it accept multiple patches

Change-Id: I29154f79df4200ab5f03134e0a84624c75ad9b13
Reviewed-on: https://chrome-internal-review.googlesource.com/236752
Commit-Ready: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
tils/buildbot_utils.py
f935a64cba64a2d9de2db3a1e691b06b1aad7203 28-Oct-2015 Luis Lozano <llozano@chromium.org> This repo does not need to be tested in CQ.

This repo is not part of any of the CQ testing/building.
Mark it so that changes to this are submitted in the Pre-CQ.

Change-Id: Iebbc4a036bd6a492a8554e7d682c7ed609accaca
Reviewed-on: https://chrome-internal-review.googlesource.com/236658
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Caroline Tice <cmtice@google.com>
OMMIT-QUEUE.ini
0861749c29801159e6673695627a2fecc3ba8ed3 23-Oct-2015 Yunlian Jiang <yunlian@google.com> add heat map generate to toolchain-utils.

This add the script to generate heat maps for chrome.

BUG=chromium:546104
TEST=it can generate heat maps when profile and chroot is given.

Change-Id: Ic60349ee1024664beb0d7450db0d7282439f79ac
Reviewed-on: https://chrome-internal-review.googlesource.com/235857
Commit-Ready: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
eat_map.py
erf-to-inst-page.sh
db77ec326606edf53bffbd5f346f622445e15e04 20-Oct-2015 Luis Lozano <llozano@chromium.org> Fixed most of the warnings in experiment_runner.py.

Started by fixing a typo in one of my previous commits.
Fixed several warnings in the process.

BUG=None
TEST=simple run of crosperf.

Change-Id: I6c470cba52f47f52e63b94bc0171c2ef336b74de
Reviewed-on: https://chrome-internal-review.googlesource.com/235376
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
rosperf/experiment_runner.py
6a00af9872c9d25a685ff5dc8efdaaeef37fc269 14-Oct-2015 Caroline Tice <cmtice@google.com> Move LOCKS_DIR out of NFS file directory.

This changes the default LOCKS_DIR, for file locking, from the
role account's NFS directory to /google/data. It also updates the
file masks to meet the requirements of the new directory.

BUG=18379210
TEST=Tested from my home account.

Change-Id: I925c9adf589b2c46fc03207ef87ea69fb14ff395
Reviewed-on: https://chrome-internal-review.googlesource.com/234200
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
ile_lock_machine.py
515533979584c2d763bed2a0f79255c5fa4da5e7 14-Oct-2015 Caroline Tice <cmtice@google.com> Make sure benchmark_runs check the cache for non-schedv2 paths.

The recent changes for schedv2 caused the cache to NOT be read/checked
if we were not using schedv2. This fixes that problem.

BUG=None
TEST=Tested in my directory with and without --noschedv2 flag.

Change-Id: I638b22554f72cf1ea0fdc4e91a326218febbf250
Reviewed-on: https://chrome-internal-review.googlesource.com/234251
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Han Shen <shenhan@google.com>
rosperf/benchmark_run.py
1efd02bef790dea5f60419a7e333e020f65a46ac 12-Oct-2015 Yunlian Jiang <yunlian@google.com> crosperf: remove dhtml from pagecyler test.

Telemetry removed dhtml, so crosperf should remove it accordingly.

BUG=none
TEST=the benchmark is removed.

Change-Id: I0bcab62e9e726d87a31febcc80121086a87337fa
Reviewed-on: https://chrome-internal-review.googlesource.com/234195
Commit-Ready: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
rosperf/experiment_factory.py
bd447043d50a8bc5f8cb29aee8c1933fb3e874bd 01-Oct-2015 Luis Lozano <llozano@chromium.org> Fixed several warnings reported by pylint.

Fixed most of warnings in command_executer.py.
There are a couple that are harder to fix and will require changes
in many files.
Was doing this to get an idea of how difficult would it be to
fix all the warnings in toolchain-utils directory.

BUG=None
TEST=Ran some benchmarks afterwards.

Change-Id: I11eec9e86317431b8ed7779d17e2dfae1be9f594
Reviewed-on: https://chrome-internal-review.googlesource.com/232829
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: David Sharp <dhsharp@google.com>
rosperf/machine_manager.py
rosperf/suite_runner.py
tils/command_executer.py
45b53c5fee4c044cebd74dbbbd6119fbd51554e7 30-Sep-2015 Luis Lozano <llozano@chromium.org> Fixed some issues with signal handling and cleanup.

Fixed several issues related to signal handling and cleanup:
- Got rid of the extra process wrapper around crosperf. We now do an
"exec" to get rid of this extra process. This facilitated signal
handling.
- Found a better fix for the problem we had with ignoring
ctrl-c. Instead of creating a pipe for child process stdin, we now
create the child with setsid which has the side effect of creating the
child without a terminal associated to it (see man setsid).
- By creating the child with setsid, we also allow for easier killing of
the child and its descendants since it allows us to use killpg (see man
killpg).
- Handle SIGTERM by "converting" it into an exit call. This will make
sure the atexit routines are called and, furthermore, we can terminate
child process when you catch the "SystemExit" exception.
- With these changes, crosperf will send a SIGTERM to any cros_sdk,
test_that processes that it has started. Before, we would have left over
test_that processes that could interfere with a new test_that
invocation.

BUG=chromium:492826
BUG=chromium:470658
TEST=verified by hand signals are working. Ran toolchain suite.

Change-Id: Ibd09ff428a7b402ea2296ecf9b9a34aa5756c93f
Reviewed-on: https://chrome-internal-review.googlesource.com/232616
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
rosperf/crosperf
rosperf/crosperf.py
rosperf/experiment_runner.py
rosperf/suite_runner.py
tils/command_executer.py
60a68a5680235ac1a101fb35561901a58298e8ef 29-Sep-2015 Luis Lozano <llozano@chromium.org> Fixed ignoring of ctrl-c.

Found out that if I send the signal directly to the crosperf process
by "kill -SIGING <pid>", crosperf handles it correctly.
However, the ctrl-c from the keyboard did not work. I started tracing it
and found out that crosperf shares the stdin with the children process.
If I avoid this sharing of stdin everything works fine. I assume one of
the commands executed is modifying some settings in the terminal and
changes the keyboard mapping for SIGINT.

To fix it, when invoking popen, request the creating of a pipe to
be used as the stdin for the new process. Once popen returns close this
pipe to avoid having the child process blocked on this pipe.

BUG=chromium:492826
TEST=Ran a few tests in crosperf. Verified the ctrl-c now works.

Change-Id: I767054a6f9e43a5137a98e074515b41db9a222ab
Reviewed-on: https://chrome-internal-review.googlesource.com/232011
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Luis Lozano <llozano@chromium.org>
tils/command_executer.py
ddde50532281f7f796dd7dc44b562b29d25ab381 23-Sep-2015 Caroline Tice <cmtice@google.com> Add compiler info to experiment data.

Check the build label to see which compiler was used. Store the
compiler in the label, and add it to the json filename
and output.

Also update nightly buildbot script to add 'compiler' field to
labels.

BUG=None
TEST=Tested in my directory.

Change-Id: I124c4910cce1145708bb2a52e3a6d6dff93777c3
Reviewed-on: https://chrome-internal-review.googlesource.com/231811
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
uildbot_test_toolchains.py
rosperf/experiment.py
rosperf/experiment_factory.py
rosperf/label.py
rosperf/results_report.py
rosperf/settings_factory.py
c4995a4466121d971d64d09110a4769ecc9fdacd 24-Sep-2015 Caroline Tice <cmtice@google.com> Add arm64 as a valid architecture.

BUG=None
TEST=Tested in my directory (with build_tc.py --board=arm64-generic)

Change-Id: Ib4d0e61a77c279c2c47c5d8e6ac6b71af4adb25e
Reviewed-on: https://chrome-internal-review.googlesource.com/232166
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Han Shen <shenhan@google.com>
tils/misc.py
e066297f07a8d1e1ad3416b4b034b2943f47c648 19-Sep-2015 Han Shen <shenhan@google.com> Fix cache reading problem.

The previous implementation images a machine to match the
br(benchmarkrun)'s label before ReadCache is performed, that image step
is a huge waste.

Now re-implemented by starting a few threads at the very beginning to go
through all brs, and put cache-hit brs into _cached_br_list. Then we
start the DutWorker thread to go through _cached_br_list, which does not
require dut to bear any particular image. After this step, we proceed as usual.

Unittest cases also added.

Tested by manullay launch a crosperf and examining no re-image is done.

Change-Id: Ib611937f9fa28ae9bedce8a1a01ed303f14f838a
Reviewed-on: https://chrome-internal-review.googlesource.com/231585
Commit-Ready: Han Shen <shenhan@google.com>
Tested-by: Han Shen <shenhan@google.com>
Reviewed-by: Han Shen <shenhan@google.com>
rosperf/benchmark_run.py
rosperf/experiment.py
rosperf/machine_manager.py
rosperf/results_cache.py
rosperf/schedv2.py
rosperf/schedv2_unittest.py
3c43b87571a5f46b73c62ed92f11b5868e9f38c5 18-Sep-2015 Han Shen <shenhan@google.com> Fixed one runtime error and enabled more output for schedv2.

One bug fixed - runtime error in nightly - command_executor is not
imported, although this does not break crosperf functionality.

Enabled more output - I enabled thread status reporting even under
non-verbose, the output is concise and it is helpful in case of
debugging.

TEST=manually launched crosperf.

Change-Id: I775069d92b84e8f069d537df884ee07057abd3bf
Reviewed-on: https://chrome-internal-review.googlesource.com/231387
Reviewed-by: Han Shen <shenhan@google.com>
Tested-by: Han Shen <shenhan@google.com>
rosperf/experiment_status.py
rosperf/schedv2.py
624bae7781fe3fc8ecbf2e314b1b637ea808d74f 18-Sep-2015 Han Shen <shenhan@google.com> Added unittest for schedv2.

In this CL, I only implemented 2 unittest for problems I fixed in CL -
https://chrome-internal-review.googlesource.com/#/c/231038/ I will add
more later.

Tested by running unittest manually.

Change-Id: I69ce9a79f9974c8945a82b5a231a047cbca53753
Reviewed-on: https://chrome-internal-review.googlesource.com/231335
Reviewed-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Han Shen <shenhan@google.com>
Tested-by: Han Shen <shenhan@google.com>
rosperf/machine_manager.py
rosperf/schedv2_unittest.py
f9b503542669e34f83e5f6bfd3c4e19a0c43e015 17-Sep-2015 Han Shen <shenhan@google.com> Filter out unreachable machines from experiment.remote as well as
label.remote.

We had a bug for a experiment file with label L1 and L2, remote A & B,
and B is unreachable (dead machine). This ends up with
MachineManager._all_machines being [A], but L1.remote and L2.remote
being both [A, B], because B is locked even it is not reachable. This
caused an internal error because only 1 dut worker is create for 'A', no
dut worker created for 'B'.

Fixed by filter unreachable machines from both experiment.remote as well
as label.remote.

Tested by running a experiment file manually.

Change-Id: Ia3f563fe7e6d1521ec8d506b79cdf185af76df06
Reviewed-on: https://chrome-internal-review.googlesource.com/231038
Commit-Queue: Han Shen <shenhan@google.com>
Tested-by: Han Shen <shenhan@google.com>
Reviewed-by: Han Shen <shenhan@google.com>
rosperf/experiment.py
rosperf/experiment_runner.py
7a939a38850f3afff46a30b53e731ccafa961f54 16-Sep-2015 Han Shen <shenhan@google.com> Refactor 'schedv2' into a seperate file.

This CL also fixed a few minor errors in running unittest.

This is to ease future development on schedv2.

TEST=manually run crosperf with a test file.

Change-Id: Iea9221929506d68b794b40f1a2319ceb5f3c8f5d
Reviewed-on: https://chrome-internal-review.googlesource.com/230829
Commit-Ready: Han Shen <shenhan@google.com>
Tested-by: Han Shen <shenhan@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/experiment_runner.py
rosperf/label.py
rosperf/machine_manager.py
rosperf/machine_manager_unittest.py
rosperf/schedv2.py
e205d9066d4391035bd3d7f9652f45647a721486 15-Sep-2015 Han Shen <shenhan@google.com> Fix 1 syntax error in my previous CL.

TEST=run crosperf successfully.

Change-Id: I95b390afe1bde28765879c21ce04a0c2c2b62f28
Reviewed-on: https://chrome-internal-review.googlesource.com/230905
Reviewed-by: Caroline Tice <cmtice@google.com>
Tested-by: Han Shen <shenhan@google.com>
Commit-Queue: Han Shen <shenhan@google.com>
rosperf/crosperf.py
31fedb0322eea850ed4ee45cb008c5b7ca5498b0 15-Sep-2015 Caroline Tice <cmtice@google.com> Get Chrome version and store in cache & json files.

This updates crosperf to query the DUT for the version of Chrome
running on it, and to store the Chrome version both in the json
reports and in the human-readable cache keys file.

BUG=None
TEST=None

Change-Id: Ie8215528b8119f95306cad694e6898cc4810730e
Reviewed-on: https://chrome-internal-review.googlesource.com/230537
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Han Shen <shenhan@google.com>
rosperf/benchmark_run.py
rosperf/label.py
rosperf/machine_manager.py
rosperf/results_cache.py
rosperf/results_report.py
43494297a1745c084d8a2f3beb52ff1c70a2007c 14-Sep-2015 Han Shen <shenhan@google.com> Make new scheduler the default.

This CL also add a new option '--noschedv2' to use the original
scheduler in case of bugs in the new scheduler.

Change-Id: I11ddefe7e7c1162019e9ad72ed3cef0b962b4d06
Reviewed-on: https://chrome-internal-review.googlesource.com/230484
Commit-Ready: Han Shen <shenhan@google.com>
Tested-by: Han Shen <shenhan@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
uildbot_test_toolchains.py
rosperf/crosperf.py
est_toolchains.py
9b852cfd9a602c5f80c8e621c696b796ce5177fd 09-Sep-2015 Caroline Tice <cmtice@google.com> Store machine description info in cache and json reports.

This CL updates the cache mechanism to store the machine spec
info (the checksum string) in the cache, as well as storing the
machine information in the json reports (to go into the dashboard
archive). It also updates the json reports to always include the
full set of result fields for a test, and it verifies that the
old cache directory exists before trying to remove it.

BUG=None
TEST=Tested in my directory, with --cache_only=True, --rerun=True and
--json_report=True

Change-Id: Ib12bae4ca10141de0b972e3a7aeee83c7b59049b
Reviewed-on: https://chrome-internal-review.googlesource.com/229765
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Han Shen <shenhan@google.com>
rosperf/benchmark_run.py
rosperf/experiment_runner.py
rosperf/results_cache.py
rosperf/results_organizer.py
rosperf/results_report.py
f63421f72bc7606ce8bfc04ce9acf25e3bf8c4c0 09-Sep-2015 Caroline Tice <cmtice@google.com> Fix minor bug with nightly tests.

Check that iter_results is not empty before trying to check a
value in it.

BUG=None
TEST=None

Change-Id: I60946e4351c137c2ba79fc978d1f6ec6635cc40e
Reviewed-on: https://chrome-internal-review.googlesource.com/229755
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Rahul Chaudhry <rahulchaudhry@google.com>
rosperf/results_report.py
c87cb38b44bdaba67d1415386fcc7ff6a97e3365 08-Sep-2015 Caroline Tice <cmtice@google.com> Fix bug: Add missing argument to function call.

cache.Init() takes an extra argument now. This function call was
overlooked when the function was updated. This CL adds the appropriate
missing argument.

BUG=None
TEST=Tested in my directory.

Change-Id: I6b87843a296fad539477576536267e481e57cc88
Reviewed-on: https://chrome-internal-review.googlesource.com/229391
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Han Shen <shenhan@google.com>
rosperf/experiment_runner.py
f97c6a214a4e6a0fcffdcba7609f3f07c8e40282 04-Sep-2015 Caroline Tice <cmtice@google.com> Remove references to NonMatchineMachines exception.

NonMatchingMachines was removed in a CL yesterday, but a few
references to it were missed. This removes those references.

BUG=None
TEST=None

Change-Id: Iadc75d26bce6193d30c23c22b5fc1a529103b607
Reviewed-on: https://chrome-internal-review.googlesource.com/229501
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
rosperf/benchmark_run.py
rosperf/machine_manager_unittest.py
ebbc3daf71bbf5e09425653e7d4cd7e811db0a87 03-Sep-2015 Caroline Tice <cmtice@google.com> Start generating json archive files for nightly tests.

This turns on the option for the nightly tests to generate json
files, which get stored in a pending archives directory and can
later be picked up, converted to ColumnIO and stored on a colossus
server.

It also fixes a bug where the nightly test was looking for a .boto
file in /home/mobiletc-prebuild instead of
/usr/local/google/home/mobiletc-prebuild.

BUG=None
TEST=None

Change-Id: Id413097686d8e72499d141ca7b8d2e131a993cb9
Reviewed-on: https://chrome-internal-review.googlesource.com/228860
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
uildbot_test_toolchains.py
est_toolchains.py
5ea9f006b5de0d882d5b51da243806b7cac69938 02-Sep-2015 Caroline Tice <cmtice@google.com> Various small cache cleanups.

Add the machine used for running the test to benchmark run; use the
actual data from that machine for generating the cache entry for the
run. Also make sure, when calculating the common checksums for looking
machines up in the cache, that all the machines being used actually have
the same checksum. Also add a cache_keys.txt file to the cache directory,
to allow people to see what was used for generating the cache
entry/checksums.

BUG=None
TEST=Tested in my directory.

Change-Id: Ia4cf8316e920a24becf5d12fd4c8f9a3e30a180f
Reviewed-on: https://chrome-internal-review.googlesource.com/228854
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
rosperf/benchmark_run.py
rosperf/experiment_runner.py
rosperf/machine_manager.py
rosperf/machine_manager_unittest.py
rosperf/results_cache.py
225fc17884a659647ed53ffd305b5e87533c18eb 02-Sep-2015 Caroline Tice <cmtice@google.com> Make ParseChromeosImage globally callable.

Move the _ParseChromeosImage out of the JSONResultsReport class, and
make it a global function, so it can be easily called from other
places.

BUG=None
TEST=Tested with and without generating json reports, with the function
moved, and everything still works.

Change-Id: I71c0435e0da9e687f69b4cebd5e3f36764d495e1
Reviewed-on: https://chrome-internal-review.googlesource.com/228840
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/results_report.py
98351672bfc0c9082081a22f27282e0bfc471b99 03-Sep-2015 Han Shen <shenhan@google.com> Adding 2 optimizations to label allocation.

They are -

1) - before choosing a label for a dut, make sure there are at least
some pending benchmark left undone.

2) - if machine A is working on label A and machine B is working on
label B, and now machine C finishes its work and wanna to help
either A or B, then C will choose A if number of pending br on A
is more than that of label B.

BUG=None
TEST=did a full run locally.

Change-Id: I322bd6fa6d10d15425c452c1437b4fa7f396361f
Reviewed-on: https://chrome-internal-review.googlesource.com/229365
Commit-Ready: Han Shen <shenhan@google.com>
Tested-by: Han Shen <shenhan@google.com>
Reviewed-by: Han Shen <shenhan@google.com>
rosperf/experiment_runner.py
rosperf/machine_image_manager.py
68c1d611c1ab7d92199f12456b097cd19e18ac1c 02-Sep-2015 Caroline Tice <cmtice@google.com> Remove cache dirs before starting if --rerun=True.

Have Crosperf look for and remove any/all cache entries that match
the current experiment, if the flag --rerun=True was passed, before
starting the run. This is to prevent cases where the experiment dies
halfway through, leaving half the cache entries 'old' and half the cache
entries 'new' (i.e. leaving cache in an inconsistent state).

BUG=None
TEST=Tested in my directory; verified cache is deleted before run starts
but only when --rerun=True is passed.

Change-Id: I1b03792cba95b30b0270e0f26839618daa70c232
Reviewed-on: https://chrome-internal-review.googlesource.com/229355
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
rosperf/experiment_runner.py
2f56356f9b66da22a87058ab83b75b5690fe6fd8 28-Aug-2015 Yunlian Jiang <yunlian@google.com> buildbot_test_toolchains: differenciate llvm build

This changes the email title and the name of experiment file
when using the patch for LLVM build.

BUG=none
TEST=it did not die at first when running a test job.

Change-Id: Iaeafbd68eaa1e30d4e5ef5edc51bfa41f5f2e56e
Reviewed-on: https://chrome-internal-review.googlesource.com/228657
Commit-Ready: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
uildbot_test_toolchains.py
364131220999a99b408c5ef848da17844028c604 28-Aug-2015 Han Shen <shenhan@google.com> Add a new option '--schedv2' so later invocation to crosperf uses the new scheduler.

This option is added to buildbot_test_toolchains.py & test_toolchains.py.

TEST=None

Change-Id: I8f31d0dd7b4b1db6af35759dd476f0f55e224366
Reviewed-on: https://chrome-internal-review.googlesource.com/228645
Commit-Ready: Han Shen <shenhan@google.com>
Tested-by: Han Shen <shenhan@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
uildbot_test_toolchains.py
est_toolchains.py
d6ff63ad5f1c492de12c39f93fda86b17bfd9134 28-Aug-2015 Luis Lozano <llozano@chromium.org> Fixed unbound variable problem.

Variable "to_be_deletes" may not be initialized in cases
where os.write before its initialization fails.
Fixed by moving out initialization out of the try block.

Also fixed a few pylint warnings.

BUG=chromium:526186
TEST=None

Change-Id: I0aaada439d9145c0f37fae877b82a03ddb4fb476
Reviewed-on: https://chrome-internal-review.googlesource.com/228503
Commit-Ready: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Caroline Tice <cmtice@google.com>
tils/email_sender.py
245e0635a5ae426c5345a06baf807940d1784125 28-Aug-2015 Han Shen <shenhan@google.com> Fix the problem that machine checksum are not checked before starting tests.

TEST=run an experiment that has machines with different checksum

Change-Id: I0da8244d168bdb3a0939a877c2ac5aa4c64a02f5
Reviewed-on: https://chrome-internal-review.googlesource.com/228371
Commit-Ready: Han Shen <shenhan@google.com>
Tested-by: Han Shen <shenhan@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/experiment_runner.py
c90a8069fa018bec72a459f444b3a0073e540e4c 26-Aug-2015 Caroline Tice <cmtice@google.com> Fix minor bugs with json report output.

This CL fixes two minor bug: make sure an item is a list before
trying to get it's first element; don't try to convert a word string
to a float.

BUG=None
TEST=Tested on the cases where it was a problem and it works now.

Change-Id: Ia26cf96593997aade3ac78feefc4dfc22f0dcbf8
Reviewed-on: https://chrome-internal-review.googlesource.com/228152
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
rosperf/results_report.py
3c6e467ab8987af58fe9fda12511dd7a35b210eb 25-Aug-2015 Yunlian Jiang <yunlian@google.com> buildbot_test_toolchains.py: small refactoring.

This is some refactoring of this script. It only copy to
weeking report if the board is lumpy, which is the only
board that we report weekly result.

BUG=none
TEST=tested manually on mobiletc-prebuild.
It does not die at begining.

Change-Id: Ib270e96ff2328030f316f9388cb43db97378da62
Reviewed-on: https://chrome-internal-review.googlesource.com/227832
Commit-Ready: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
uildbot_test_toolchains.py
ef4ca8a8cfe05ec09447896db42fb220d07834f8 25-Aug-2015 Caroline Tice <cmtice@google.com> Add json report archiving.

This CL adds a new output, in json format, for archiving test
results. It also adds a new flag, --json_report, to crosperf to
tell it to generate the json output file in the usual results
directory. The json output is always in addition to the other
reports that Crosperf generates. This CL also make some minor
changes to the tool that maintains the telemetry default results
file, so it can be used by the json report generator.

BUG=None
TEST=Tested Crosperf with & without new flag; it did what it should.

Change-Id: Id0e476716ceee208d33e6f9427ea95235fbf11d4
Reviewed-on: https://chrome-internal-review.googlesource.com/228325
Reviewed-by: Han Shen <shenhan@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/crosperf.py
rosperf/experiment_runner.py
rosperf/results_report.py
rosperf/settings_factory.py
pdate-telemetry-defaults.py
pdate_telemetry_defaults.py
bb04def1e86a0949a1b1898e705e56c84c400dd9 24-Aug-2015 Yunlian Jiang <yunlian@google.com> buildbot_test_toolchains: fix a typo

BUG=none
TEST=it runs with a patch parameter.

Change-Id: I08437d0ec47ef27a3b88930358f3c149fd955cdc
Reviewed-on: https://chrome-internal-review.googlesource.com/227861
Reviewed-by: Han Shen <shenhan@google.com>
Commit-Queue: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
uildbot_test_toolchains.py
76259e625a3cecdbdea9c595945754d94015030a 21-Aug-2015 Yunlian Jiang <yunlian@google.com> buildbot_test_toolchains: fix several errors

This fixes some errors introduced by previous patch.

BUG=none
TEST=the script is running fine on chrotomation2

Change-Id: Id6b49941aef5f5c8902dae9dfeefe2acc51ef310
Reviewed-on: https://chrome-internal-review.googlesource.com/227840
Reviewed-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
uildbot_test_toolchains.py
e52838c4ceb4c59ec7328fed94f78d24603a67f0 20-Aug-2015 Yunlian Jiang <yunlian@google.com> buildbot_test_toolchains: make patch number a parameter.

We want to use the buildbot_test_toolchains.py to measure the
performance gap between gcc and clang. So we need a different
patch to create the image built with clang.

BUG=none
TEST=buildbot_test_toolchains.py now accepts --patch

Change-Id: I45dfd79630cea217bc6134802270c6e13b4121c1
Reviewed-on: https://chrome-internal-review.googlesource.com/227806
Reviewed-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
uildbot_test_toolchains.py
ba64928c5dcbacbc70b4358881a89ad96227164d 06-Aug-2015 Han Shen <shenhan@google.com> Crosperf schedv2 (1) - new option and integrating new scheduler.

This Cl introduces a new option '--schedv2' which uses the new
scheduler to allocate jobs (benchmark_runs) to duts.

With this option, schedv2 takes control of storing/allocating jobs and
reimage machines using the new algorithm. This CL leaves actual
reimaging and running jobs non-op (a random time sleep is used for
each such op, which would be replace in later CLs.)

You may try this CL like this and see how schedv2 works -
crosperf --locks_dir=/usr/local/google/home/shenhan/tmp --use_file_locks=True --logging_level=verbose --schedv2 some.exp

Change-Id: If5bb7751b466c39e54c93fe8f0b4e363be6d9165
Reviewed-on: https://chrome-internal-review.googlesource.com/225515
Commit-Queue: Han Shen <shenhan@google.com>
Tested-by: Han Shen <shenhan@google.com>
Reviewed-by: Han Shen <shenhan@google.com>
rosperf/benchmark_run.py
rosperf/crosperf.py
rosperf/experiment.py
rosperf/experiment_factory.py
rosperf/experiment_runner.py
rosperf/experiment_status.py
rosperf/label.py
rosperf/machine_image_manager.py
rosperf/machine_image_manager_unittest.py
rosperf/machine_manager.py
mage_chromeos.py
0aa17b4641bcc205c52e6db0c529dd3126003ec5 20-Aug-2015 Caroline Tice <cmtice@google.com> Finish removing temporary workaround.

Chrome in ChromeOS chroot for nightly buildbot tests now has a
good version of telemetry, so we can remove the temporary
workaround for the buildbots.

BUG=None
TEST=Ran telemetry by hand inside chroot & crosperf outside chroot.

Change-Id: I38db7d9a2f427df3cc29418bc866d236e213247b
Reviewed-on: https://chrome-internal-review.googlesource.com/227608
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
uildbot_test_toolchains.py
877af2d12d8c3664588f1bd7aa8963c341e24881 14-Aug-2015 Caroline Tice <cmtice@google.com> Remove temporary workaround.

A good version of Chrome (we think) has made it into ChromeOS now,
so this will remove the temporary workaround for the lumpy tests.
If the lumpy tests work with this removed, then we will remove it
for the buildbot tests.

BUG=None
TEST=None

Change-Id: I84526b06185176a3225ba52b3e3990d0d30ef170
Reviewed-on: https://chrome-internal-review.googlesource.com/227158
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
est_toolchains.py
f456b2035aec47012d7d111b0c5e21a3319823c0 12-Aug-2015 Caroline Tice <cmtice@google.com> Temporarily use good external Chrome source tree.

This is to work around a problem with the current version of Chrome
inside ChromeOS (telemetry is broken in that version of Chrome).
This CL makes the nightly tests use an external Chrome source tree
that has been verified to work properly. This will be un-done, once
ChromeOS gets a good version of Chrome.

BUG=None
TEST=Tested crosperf on chrotomation2 using this.

Change-Id: I41caef9bb1b1973c83745e3962a5c36c489d7514
Reviewed-on: https://chrome-internal-review.googlesource.com/226695
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
uildbot_test_toolchains.py
est_toolchains.py
2e5bdd80885d7fb675d4ff7bf8d78de45f89babc 07-Aug-2015 Caroline Tice <cmtice@google.com> Deal properly with non-unique buildbot build ids.

In the new buildbot waterfall arrangement, occasionally a buildbot
build id will not be unique. This was causing two different problems
for us, for a buildbot id that is not unique: 1. If our job finished
successfully and generated a build image artifact, we end up finding
multiple artifacts, and we were using the first one founs (which was the
earliest). 2. If the job failed to generate a build artifact, we would
still successfully find a build artifact from an earlier build with the
same id. This CL fixes both problems by 1. Checking the build status
and only looking for a build artifact if the job looks like it succeeded;
2. Sorting the build artifacts we find, to get and use the most recent.

BUG=None
TEST=Tested on the build logs with all the various cases that failed
improperly today.

Change-Id: I627aa48ae13d3c115acd6cb3c411e382f0a8f1c6
Reviewed-on: https://chrome-internal-review.googlesource.com/226096
Commit-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Han Shen <shenhan@google.com>
tils/buildbot_utils.py
dac5f567d00069a739d9d08935f9e28f97ab45e7 05-Aug-2015 Rahul Chaudhry <rahulchaudhry@chromium.org> buildbot_utils: for release builds, get logs from the 'release' builder.

The individual ${BOARD}-release builders are now grouped together
into one 'release' builder.

BUG=None
TEST=None

Change-Id: Ie0a2ea1298c63f71385f4f9331e611d8ac22efca
Reviewed-on: https://chrome-internal-review.googlesource.com/225253
Reviewed-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
tils/buildbot_utils.py
c7a9ac55b97ca3c2aafa3f3029a707fb1f6be508 05-Aug-2015 Rahul Chaudhry <rahulchaudhry@chromium.org> buildbot_utils: fix waiting logic in GetTrybotImage.

The problematic case is when buildbot_json.py execution fails and
the call to GetBuildInfo returns nothing (line 209). done is set
to False (line 216), and the full "if done:" block is skipped.
Unfortunately, the "if not done:" block starting at line 242 is
inside the "if done:" block (Yay Python!). This simply jumps back
to the beginning of the while loop (line 207) without sleeping for
SLEEP_TIME and without updating pending_time. Result: an infinite
loop with no delay spawning a new process on each iteration.

BUG=None
TEST=SLEEP_TIME and TIME_OUT are respected in the loop.

Change-Id: I7cc741b7a183233507968277345a079aecce9cd2
Reviewed-on: https://chrome-internal-review.googlesource.com/225252
Reviewed-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
tils/buildbot_utils.py
f3eb80354a8d7dd386c2880c282a767abad6a18a 27-Jul-2015 cmtice <cmtice@google.com> Make duplicate locking/unlocking warning instead of error.

Change the afe_lock_manager, so that an attempt to lock an already
locked machine or unlock an already unlocked machine is a warning
rather than an exception; also return a list of machines which we
were successfully able to lock/unlock.

Also update Crosperf: Previously if multiple machines were requested
and we were not able to lock all of them, the Crosperf job failed.
Now if we are able to successfully lock any of the requested machines,
Crosperf will try to run the job.

BUG=None
TEST=Tested in my workspace with various locked/unlocked
configurations for crosperf job with two machines.

Change-Id: I65a0a9cce4b8be8d0c043913dc2dd7f9f07cda0f
Reviewed-on: https://chrome-internal-review.googlesource.com/224180
Reviewed-by: Han Shen <shenhan@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
fe_lock_machine.py
rosperf/experiment.py
rosperf/experiment_runner.py
rosperf/machine_manager.py
e2b5a89e332aada734c265adc6c96898cffe37d7 29-Jul-2015 cmtice <cmtice@google.com> Add default results value for smoothness.tough_webgl_cases

BUG=None
TEST=Tested in my directory.

Change-Id: I55a31edd8d6f5bdbc1aaeb4cfa375f29ec93475a
Reviewed-on: https://chrome-internal-review.googlesource.com/224765
Reviewed-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/default-telemetry-results.json
146058f65658cdf073cba7d5d12b082b2eb854f2 24-Jul-2015 cmtice <cmtice@google.com> Remove temp file that breaks telemetry.

Occasionally after running Telemetry, it will leave behind a
file in /tmp of your chroot (/tmp/telemetry@<machine_name>), which
will cause any later attempt to run telemetry on that machine to
fail, until the file is removed.

This CL checks for such a file, and if found deletes it, before
attempting to run the telemetry tests.

BUG=chromium:513875
TEST=Tested in my chroot with & without such a file.

Change-Id: I3e0a3000a1614f46f9312abddf77fd48b25334da
Reviewed-on: https://chrome-internal-review.googlesource.com/223812
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/suite_runner.py
25c94f1849f617166781903637099d962673c60a 24-Jul-2015 cmtice <cmtice@google.com> Add lock_reason for local AFE server locking requests.

The local AFE server has been updated to use the newer code base,
and now it requires a locking reason (previously only the global
AFE server required one). Now the lock reason will be given
for both servers.

BUG=None
TEST=Tested in my work area with a local DUT.

Change-Id: I74ea1941b4c04d9647cc3bde6b1f1cce80c225b5
Reviewed-on: https://chrome-internal-review.googlesource.com/223810
Reviewed-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
fe_lock_machine.py
95cd449e2f4ebef9c6277c525b7d18df40c1cc9a 22-Jul-2015 Luis Lozano <llozano@chromium.org> Changed set of benchmarks to track in the weekly report.

BUG=None
TEST=None

Change-Id: I4affac33516c1037e3ca7360a4c7c6cb1a1818a9
Reviewed-on: https://chrome-internal-review.googlesource.com/223366
Reviewed-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
eekly_report.py
bc2d3d13d2e1928217140a76acdf9f9917b1fa30 02-Jul-2015 Ting-Yuan Huang <laszio@google.com> Add 'run_local' to run the test harness locally.

The option 'run_local: True|False' will be passed into autotest.
For example, this is how test_that will be invoked:
test_that --arges="run_local=True ..." ...

BUG=None
TEST=Tested with the folllowing combinations.
benchmark: sunspider, smoothness.top_25_smooth
perf_args: record -e cycles,instructions / (none)

Change-Id: I7315027a7c9433d17a9f3fe54d7e8c3f480ea4f4
Reviewed-on: https://chrome-internal-review.googlesource.com/217370
Reviewed-by: Caroline Tice <cmtice@google.com>
Tested-by: Ting-Yuan Huang <laszio@google.com>
Commit-Queue: Ting-Yuan Huang <laszio@google.com>
rosperf/benchmark.py
rosperf/benchmark_run.py
rosperf/experiment_factory.py
rosperf/results_cache.py
rosperf/results_cache_unittest.py
rosperf/settings_factory.py
rosperf/suite_runner.py
27dc583d33d461daad462f82ab7974b07cb31117 01-Jul-2015 cmtice <cmtice@google.com> Make json defaults file human-readable.

BUG=None
TEST=None

Change-Id: I75f94b3c29e7eebe001bfaca798172244fb60e55
Reviewed-on: https://chrome-internal-review.googlesource.com/221016
Reviewed-by: Han Shen <shenhan@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
rosperf/default-telemetry-results.json
pdate-telemetry-defaults.py
d1f03b83a267caa1e92d8d04c483c5ec0a8c5355 01-Jul-2015 cmtice <cmtice@google.com> Add 'retries' to crosperf for nightly non-buildbot tests.

BUG=None
TEST=None

Change-Id: Id991ed0d5e23d363940d65aeb49f4e4fbbd2051e
Reviewed-on: https://chrome-internal-review.googlesource.com/221000
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
est_toolchains.py
0c84ea75c90b494df6b1ea6923a48611be81b96c 25-Jun-2015 cmtice <cmtice@google.com> Update all_toolchain_perf to use new recommended set.

Update the all_toolchain_perf alias to use the new recommended
set of tests. Also update the default test values to include
default values for the newly recommended tests. Also update the
nightly tests to use the newly recommended set.

BUG=None
TEST=Tested new alias with crosperf.

Change-Id: I603f866723646a2d5b6d86fbe1de3c4617e09046
Reviewed-on: https://chrome-internal-review.googlesource.com/220385
Reviewed-by: Han Shen <shenhan@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
uildbot_test_toolchains.py
rosperf/default-telemetry-results.json
rosperf/experiment_factory.py
est_toolchains.py
80fd9bfaaf6146ae373b0669062832639c28662a 25-Jun-2015 cmtice <cmtice@google.com> Fix typo. Committing as obvious.

BUG=None
TEST=None

Change-Id: I5414a3cd4686dae5f34b3c3ae3951bee52f07a9f
Reviewed-on: https://chrome-internal-review.googlesource.com/220295
Reviewed-by: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
eekly_report.py
717bd4a85585bae37671e28e35757ea3ad4f3b5b 25-Jun-2015 cmtice <cmtice@google.com> Only search buildbot logs for appropriate builders.

Update the buildbot scripts to only search for build records on
the builders we launched the jobs on. Previously the scripts
were searching the build logs of all builders.

BUG=None
TEST=Tested with and without this on chrotomation2. Without this
it took 16 minutes; with it it took 1 minute.

Change-Id: Id36e6fc5688299f3af765b439de66a029b1f896e
Reviewed-on: https://chrome-internal-review.googlesource.com/220137
Reviewed-by: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Mungyung Ryu <mkryu@google.com>
tils/buildbot_utils.py
3717ea8063b5bf0eab573845c7ad5ba5689782de 25-Jun-2015 cmtice <cmtice@google.com> Fix email on weekly (seven-day) reports.

Forgot to fix these emails when fixing the other nightly test
emails. This should cause the weekly reports to be emailed
to team again.

BUG=None
TEST=None

Change-Id: I86ef96eb053f37465094fb60b337487cccb38393
Reviewed-on: https://chrome-internal-review.googlesource.com/220186
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
eekly_report.py
770a1603068b96d5bb2debdbad0d51443bb6ab6f 23-Jun-2015 cmtice <cmtice@google.com> Make crosperf unit tests executable.

BUG=None
TEST=None

Change-Id: I77e305e4e8711c86116838fd77fcc93fac2c7dc1
Reviewed-on: https://chrome-internal-review.googlesource.com/219922
Reviewed-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/benchmark_unittest.py
rosperf/crosperf_unittest.py
rosperf/download_images_buildid_test.py
rosperf/download_images_unittest.py
rosperf/settings_factory_unittest.py
rosperf/settings_unittest.py
rosperf/suite_runner_unittest.py
e30d342f44700a2052601e87b3c4aafaef0b7be7 24-Jun-2015 Rahul Chaudhry <rahulchaudhry@chromium.org> toolchain-utils: specify full path in script to be run in chroot.

Remove the assumption that the current directory is ~/trunk/src/scripts/
when the chroot is entered with cros_sdk.

BUG=chromium:429765
TEST=Ran image_chromeos.py with the changes.

Change-Id: I4aa9f233da5b3eef727009d044acdd1a27ea4873
Reviewed-on: https://chrome-internal-review.googlesource.com/219940
Reviewed-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
mage_chromeos.py
9db7ae58ddb0c2d5650c9464427c40a3070ec03a 23-Jun-2015 cmtice <cmtice@google.com> Remove temporary workaround for nightly tests locking.

Due to issues with mobiletc-prebuild not being able to access
the afe locking mechanisms, the nightly tests were temporarily
using the old file locking mechanism. The issues with the role
account using the afe locks has been resolved, so this CL reverts
the nightly tests to using the afe machine locking mechanism.

BUG=None
TEST=Ran crosperf tests on chrotomation2 from role account using
afe locking mechanism.

Change-Id: I243c28d80718c2e98913ad49d50bf5b46ff82758
Reviewed-on: https://chrome-internal-review.googlesource.com/219905
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
uildbot_test_toolchains.py
est_toolchains.py
aea5341cb080d5c181e452796534f184472e730a 19-Jun-2015 cmtice <cmtice@google.com> Add debugging/triaging message to buildbot job.

Check the buildbot launch results to make sure the buildbot
launched successfully before continuing. Otherwise check pending
time and either sleep some or issue fatal message (if pending time
times out).

BUG=None
TEST=Tested by hand in my directory.

Change-Id: I27bd6bf4fb88f0d9db584272e74f9a2ad31f9381
Reviewed-on: https://chrome-internal-review.googlesource.com/219727
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
tils/buildbot_utils.py
73bf2e9cc58d8429dbb7636e996062e7605a7222 20-Jun-2015 Han Shen <shenhan@google.com> Remove "import lock_machine".

We do not have this file any longer, remove it. Rahul and I are having
problem running crosperf because of this.

TEST=run crosperf successfully
BUG=None

Change-Id: I396d30e6bb2362db99f9b16095ef862a1ab417f5
Reviewed-on: https://chrome-internal-review.googlesource.com/219770
Reviewed-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Han Shen <shenhan@google.com>
Tested-by: Han Shen <shenhan@google.com>
rosperf/settings_factory.py
78f2595399c2c8616cb1aba781ca33779813f7d9 19-Jun-2015 cmtice <cmtice@google.com> Fix typo in file. Committing as obvious.

Change-Id: I68fc37b0eaf3ff5bec53abe7e11b8227644685e9
Reviewed-on: https://chrome-internal-review.googlesource.com/219726
Reviewed-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
est_toolchains.py
e1efeb8c08e51aac3ca80859a015758d682e86e6 17-Jun-2015 Luis Lozano <llozano@chromium.org> Add automatic retries to nightly testing.

Used new "retries" crosperf functionality in our nightly testing.

BUG=None
TEST=None (already tested in crosperf side)

Change-Id: I42b9d04ef690593485df825b2a3aa41b9e2f6cc3
Reviewed-on: https://chrome-internal-review.googlesource.com/219310
Reviewed-by: Han Shen <shenhan@google.com>
Commit-Queue: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
uildbot_test_toolchains.py
517dc98884e398fa863ce22b6bc1ef8323853416 12-Jun-2015 cmtice <cmtice@google.com> Add flag to crosperf to use file lock mechanism.

This CL adds a flag to crosperf that specifies using the old file
locking mechanism, rather than the new AFE server locking mechanism.
This is useful when there are issues with accessing the AFE server.

BUG=None
TEST=Tested crosperf with this, both using the AFE locking mechanism
and using the file locking mechanism.

Change-Id: If6aa3e97a77d8ccb53f0e8f0405cc7f45e983b75
Reviewed-on: https://chrome-internal-review.googlesource.com/219096
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/experiment.py
rosperf/experiment_factory.py
rosperf/experiment_runner.py
rosperf/machine_manager.py
rosperf/settings_factory.py
d1172b4df650c89b6fef7719db8225c31a4eba0e 13-Jun-2015 cmtice <cmtice@google.com> Minor fixes to make AFE server mechanism work for role account.

BUG=None
TEST=Tested in role account on chrotomation2.mtv.

Change-Id: I97661c14427989b084c5394a040e1055c47d90a9
Reviewed-on: https://chrome-internal-review.googlesource.com/219042
Reviewed-by: Han Shen <shenhan@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
fe_lock_machine.py
aa700b0a78b10b709b990bcb535bc5588ebb2bb4 12-Jun-2015 cmtice <cmtice@google.com> Update nightly tests to use old file locking mechanism for now.

There are some issue with the role account using the AFE server
locking mechanism, so we will temporarily revert the nightly tests
to use the old file locking mechanism. This CL depends on the CL
that re-enables the file locking mechanism (with a new flag).

BUG=None
TEST=None
CQ-DEPEND=CL:219096

Change-Id: If525e4fcf292d3c57aa9c69a1b796914bd6d434c
Reviewed-on: https://chrome-internal-review.googlesource.com/219088
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
uildbot_test_toolchains.py
est_toolchains.py
441c94998a0b1b547bd9b4e594acb05d9e5ed6eb 11-Jun-2015 Han Shen <shenhan@google.com> Correctly calculate the machines_file path.

Rahul and I were having problems running crosperf outside crosperf
directory, the problematic we got was here -

./afe_lock_machine.py:181: machines_file = os.path.join(os.getcwd(), 'crosperf', 'default_remotes')

This assumes that crosperf is invoked right under toolchain-utils,
which is, as likely as not, false.
fe_lock_machine.py
e5bc63bbed4e001b080c4ce0b18c5c78900d4786 28-May-2015 cmtice <cmtice@google.com> Implement new global locking scheme for machines.

This CL implements a new machine locking mechanism using
Autotest Front End servers. When locking/unlocking a lab machine,
it uses the ChromeOS HW lab server; when locking/unlocking a local
machine, it uses a local AFE server on chrotomation2.

BUG=None
TEST=Tested the script manually to lock/unlock machines and query status.
Also tested with image_chromeos and with crosperf scripts.

Change-Id: I2793bc1f7dc056e725694e81ded656d9f49d227b
Reviewed-on: https://chrome-internal-review.googlesource.com/217560
Reviewed-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: David Sharp <dhsharp@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
fe_lock_machine.py
rosperf/experiment.py
rosperf/experiment_runner.py
rosperf/machine_manager.py
rosperf/settings_factory.py
ile_lock_machine.py
mage_chromeos.py
ock_machine.py
tils/locks.py
tils/machines.py
tils/misc.py
0495c7175f1160877f121872241f44386705f667 29-May-2015 cmtice <cmtice@google.com> Fix typo.

BUG=None
TEST=None

Change-Id: If450b820bc218fbd02190982b0854c4a421e55c2
Reviewed-on: https://chrome-internal-review.googlesource.com/217620
Reviewed-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
est_toolchains.py
df76222bf1cbdbc42bb41934fb960bac8694eca4 19-May-2015 Luis Lozano <llozano@chromium.org> Added benchmark retry support.

We often get spurious failures from the benchmarks runs. This CL adds
support for a "retries" field that will automatically retry the benchmark
run by the amount specified.

BUG=None
TEST=Test by hand.

Change-Id: Ib881c40d766e67a7daf855ce9137d3e3d631f12e
Reviewed-on: https://chrome-internal-review.googlesource.com/217525
Reviewed-by: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Luis Lozano <llozano@chromium.org>
rosperf/benchmark.py
rosperf/experiment_factory.py
rosperf/settings_factory.py
rosperf/suite_runner.py
7cdb11a25795a7215d59aa49c7b8619780951afa 28-May-2015 cmtice <cmtice@google.com> Add missing import statement.

BUG=None
TEST=Testing by hand in role account.

Change-Id: If7657ae15bea5faee891e5ae9959082196d77e32
Reviewed-on: https://chrome-internal-review.googlesource.com/217513
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
est_toolchains.py
b45880993bbac2be9766defb99ccce32eab2a897 27-May-2015 cmtice <cmtice@google.com> Fix typo from previous commit.

Remove leftover colon from end of line that's no longer a
conditional check.

BUG=None
TEST=None

Change-Id: I3cbcc5f18280adb3044386df205de7309c7a7855
Reviewed-on: https://chrome-internal-review.googlesource.com/217440
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
est_toolchains.py
7f3190b3fb89522ea85b87200241962634369e9a 22-May-2015 cmtice <cmtice@google.com> Update nightly tests to use mail-sheriff program for reports.

This updates the nightly buildbot and local tester programs to
tell crosperf to put its results in a special directory, and to
not send email. After crosperf finishes, it looks for the email
information in the special directory and sends out then email using
the mail-sheriff program.

BUG=None
TEST=Currently running buildbot test. Will not commit the change
until the test passes. Since the test takes several hours to run
I wanted to put this out for review now.

Change-Id: Idd3d137e4a6e144b677690fc6775580dfac56af1
Reviewed-on: https://chrome-internal-review.googlesource.com/217286
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
uildbot_test_toolchains.py
est_toolchains.py
7658aecb942b63472453be82a24bc2036121383d 22-May-2015 cmtice <cmtice@google.com> Add email message body to results directory.

Save the email message body in the results directory. Also update
the email message to say where the results directory is.

BUG=None
TEST=Tested in my chroot.

Change-Id: I1da651549db87edc6b781525d443d21f97041d21
Reviewed-on: https://chrome-internal-review.googlesource.com/217265
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/experiment_runner.py
rosperf/experiment_runner_unittest.py
d96e457132abf36c05936121e59acbab819ecd8e 20-May-2015 cmtice <cmtice@google.com> Remove run_remote_tests references.

run_remote_tests.sh is about to be removed from ChromeOS. This
CL removes all references to run_remote_tests from toolchain-utils.
It also updates unittests to make sure they work with recent
changes.

BUG=crbug:498778
TEST=Verifed results_cache_unittest (and all the other unittests)
works with changes.

Change-Id: Ic9413bf77a1679e1267b81b182fa7f64da2629b5
Reviewed-on: https://chrome-internal-review.googlesource.com/216884
Reviewed-by: David Sharp <dhsharp@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rb/autotest_run.py
rosperf/benchmark_run_unittest.py
rosperf/experiment.py
rosperf/machine_manager.py
rosperf/machine_manager_unittest.py
rosperf/results_cache_unittest.py
rosperf/test_cache/compare_output/results.txt
emote_kill_test.py
emote_test.py
un_tests.py
tils/file_utils.py
54db5386573e81f36192d5cf0c843158ebafb356 21-May-2015 Luis Lozano <llozano@chromium.org> Fix confusing error message when cherrypy is not installed on DUT.

Also fixed a few warnings.

BUG=None
TEST=Installed a couple of images on machines with and without cherrypy.

Change-Id: I40d3fa1882e3bceb0e70a8d040a892e894cf50db
Reviewed-on: https://chrome-internal-review.googlesource.com/217045
Reviewed-by: Caroline Tice <cmtice@google.com>
Tested-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Luis Lozano <llozano@chromium.org>
mage_chromeos.py
tils/command_executer.py
17219394d2431154d99ab9ab7d09754569fac022 14-May-2015 cmtice <cmtice@google.com> Update email sender to use sendgmr, if it's available.

BUG=None
TEST=Tested with crosperf tests on my system.

Change-Id: I4db4cd70d74e5f13b1fe8bb022e5403f6902d1ee
Reviewed-on: https://chrome-internal-review.googlesource.com/216312
Reviewed-by: David Sharp <dhsharp@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
tils/email_sender.py
3931b8b72c9caf5b4874b693e33765f086a73135 28-Apr-2015 Han Shen <shenhan@google.com> Use "http://chromegw/i/tryserver.chromiumos/" for retrieving builder info.

This "/i/" url is the internal version of
"http://chromegw/p/tryserver.chromiumos/". And "/i/" links offer
waterfall controls (force build, stop build, etc.) and work for both
internal and public waterfalls, whereas "/p/" links only work for
public waterfalls.

BUG=None
TEST=buildbot_json.py works well with this url.

Change-Id: I55bebdb53020447156df2f7477dd3d9a32478660
Reviewed-on: https://chrome-internal-review.googlesource.com/214840
Reviewed-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Han Shen <shenhan@google.com>
Tested-by: Han Shen <shenhan@google.com>
tils/buildbot_utils.py
1d65210d275136943f092b7a8c51effeaf092078 22-Apr-2015 cmtice <cmtice@google.com> Fix various unittest failures.

Over the past few months some of the unittests have not kept up
with code changes and have stopped working. This CL fixes that.

BUG=None
TEST=Ran all the unittests by hand.

Change-Id: I4fdd09846ae92ce2d45ec478a39ff3ece30b4579
Reviewed-on: https://chrome-internal-review.googlesource.com/214134
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/crosperf_unittest.py
rosperf/experiment_factory_unittest.py
rosperf/experiment_file_unittest.py
rosperf/machine_manager_unittest.py
rosperf/settings_factory_unittest.py
rosperf/suite_runner_unittest.py
5c09fc2966ac49263ce7154c2905f2a86aeda297 22-Apr-2015 cmtice <cmtice@google.com> Fix current problem with machine locks.

Currently if crosperf can't access the default locks directory, it
assumes that it is safe to proceed without locks and it uses machines
without checking locks. This script does two things: It causes
crosperf to fail if it can't access a locks directory; and it allows
the user to specify an alternate locks directory. As a by-product it
also adds a boolean prompting function to utils/misc.py

BUG=None
TEST=Tested with default directory accessible & inaccessible; tested using
local locks directory.

Change-Id: I6f24f772986813bf089f5f4e40335a15e8ee4837
Reviewed-on: https://chrome-internal-review.googlesource.com/214129
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/experiment.py
rosperf/experiment_factory.py
rosperf/machine_manager.py
rosperf/settings_factory.py
tils/misc.py
d7d5084ffa40651b5adbaea4f4659a51e1045f06 24-Apr-2015 Luis Lozano <llozano@chromium.org> Fix typo in previous commit.

BUG=None
TEST=None

Change-Id: I08b51a5e716df35c768e64de95ae386937f14b12
Reviewed-on: https://chrome-internal-review.googlesource.com/214487
Commit-Queue: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Caroline Tice <cmtice@google.com>
tils/buildbot_utils.py
8a68b2d90fdafdd45c636cf48375c944107d5062 23-Apr-2015 Luis Lozano <llozano@chromium.org> Make sure latest toolchains are build in non vanilla testing.

Our non-vanilla testing builds with latest compiler but it
was not using --latest_toolchain argument for cbuildbot.
Also, did some minor cleanup.

BUG=None
TEST=None (will let it test in the nightly testing)

Change-Id: I7316fcd8b9548f7038b110bdf6cd4f60714f41d2
Reviewed-on: https://chrome-internal-review.googlesource.com/214363
Commit-Queue: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Caroline Tice <cmtice@google.com>
uildbot_test_toolchains.py
tils/buildbot_utils.py
eekly_report.py
a63ffc0d2f8d95af20860eab293b72270efa4ac0 21-Apr-2015 cmtice <cmtice@google.com> Wait 10 minutes after builbot launch before looking for data.

This is to fix a problem where sometimes there is a slight delay
in the buildbot launching, so we end up looking for the buildbot
data before it's there, which causes the script to fail.

BUG=None
TEST=None

Change-Id: Icf0206ce4012bef437a76f3ce4192461f15fd476
Reviewed-on: https://chrome-internal-review.googlesource.com/214009
Reviewed-by: Rahul Chaudhry <rahulchaudhry@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
tils/buildbot_utils.py
9b552385f43dbe21c55735a18389200e4a96fcd0 26-Mar-2015 Han Shen <shenhan@google.com> Add a new flavor of command executer.

Why -
The usefulness of this new cmd executor is that it allows programmer
to examine the execution output/err stream while the binary is being
executed.

What we can do with our current weapons in the command_executor
repertoire is to kick-off the execution, capture all the command
output/err in memory, wait for its finish and examine the output in
memory.

A concrete example -
Let me give a concrete example for this - while I was reading one of
the nightly scripts, I came across this -

commands = ("{0}/utils/buildbot_json.py builds "
"http://chromegw/p/tryserver.chromiumos/"
.format(file_dir))
_, buildinfo, _ = ce.RunCommand(commands, return_output=True,
print_to_console=False)

This kicks-off a long-run command line, captures all of its output in a string, and then parse to pick out some information.

There is some issues with this -
a) - memory consumption - we capture all output in a python string
b) - no way to control the execution, what if what we are interested
in is at the very beginning or in the middle of the command output,
and once we get that we are done, and we don't want it to run any
longer.
c) - we only have hinder sight about the execution, what if the
execution spit out errors like 'unable to connect, wait for 30
minutes till timeout', which we may know of and could act earlier
(for example, kill it prematurely and report back to user).

A better approach
So here is how the proposed cmd executor works to address the above limitations -
... ...
def my_line_consumer(line, output):
if output == 'output':
# parsing this line
if output == 'err':
# ignore this line
return true; // keep execution, false to kill the current execution.
commands = ("....")
rv = ce.RunCommand(commands, my_line_consumer) # no output string is returned.
... ...
The only difference is that we pass a 'line_consumer' to the cmd
executor, this consumer will be fed with output/stderr (in unit of
lines), and depending on line content, it decides whether to kill the
execution prematurely.

Current status
I have this implemented (not many lines of code, < 100) in my
bisecting scripts, and I plan to port it to toolchain_utils/utils.

Change-Id: I2318dda796b3dcfa6ebe604091b41f9d68525d95
Reviewed-on: https://chrome-internal-review.googlesource.com/208619
Reviewed-by: Han Shen <shenhan@google.com>
Tested-by: Han Shen <shenhan@google.com>
Commit-Queue: Han Shen <shenhan@google.com>
tils/command_executer.py
7141db2b3a3d0166ee202ff88c6d34fe05f419a3 09-Apr-2015 Luis Lozano <llozano@chromium.org> Allow weekly report to use a remote different from what is in the cache.

For the weekly report we dont do any reruns (cache_only). So the machine
used in remotes should not matter. Add "same_specs: False" to avoid
checking the specs of the machine in the remote field.

BUG=None
TEST=Reran report by hand with new field setting

Change-Id: I37e46bfc99d0a4e421953dacb1a36c41a769c9c8
Reviewed-on: https://chrome-internal-review.googlesource.com/212345
Commit-Queue: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Han Shen <shenhan@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
eekly_report.py
18f04033e45aec19ebbfe9017e02d2c74e979dc4 07-Apr-2015 Luis Lozano <llozano@chromium.org> Put back supposedly dead parrot in our list.

This machine was supposed to be dead chromeos2-row3-rack2-host10.cros.
But it is alive and kicking now. Putting it back in the list.

BUG=None
TEST=None

Change-Id: Iaa722b41ab694c98f2b28f3023e4b0970c729b84
Reviewed-on: https://chrome-internal-review.googlesource.com/212009
Commit-Queue: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/default_remotes
09b027f5ee8c44f0b965392a4020bef144c48d75 30-Mar-2015 Luis Lozano <llozano@chromium.org> Change default to build ChromeOS with AFDO.

Change the default USE flags to use AFDO since that is how the
released images are built. To build WITHOUT AFDO do:

build_chromeos.py --env='USE=-afdo_use' --board ....

BUG=None
TEST=Tested building with and without AFDO. Verified right USE
flags are generated.

Change-Id: I6ae2313b76b08fc0e0a22aae90f4ca0396d523e4
Reviewed-on: https://chrome-internal-review.googlesource.com/210834
Reviewed-by: Caroline Tice <cmtice@google.com>
Tested-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Luis Lozano <llozano@chromium.org>
uild_chromeos.py
de0a90d676297a216dde2e47562b3a9f57621c41 15-Dec-2014 Luis Lozano <llozano@chromium.org> Scripts to use with sysroot_wrapper bisecting.

Scripts to be used by the binary searching tool to interact with
the bisecting functionality in the sysroot_wrapper.

BUG=None
TEST=Tested on 3 different occasions while triaging problems
in flash, kernel and glibc.

Change-Id: Icf55f60b0cb160ae453c9384ca77da70710c3f48
Reviewed-on: https://chrome-internal-review.googlesource.com/188665
Reviewed-by: Han Shen <shenhan@google.com>
Commit-Queue: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
inary_search_tool/sysroot_wrapper/README
inary_search_tool/sysroot_wrapper/common.sh
inary_search_tool/sysroot_wrapper/get_initial_items.sh
inary_search_tool/sysroot_wrapper/glibc_test_script.sh
inary_search_tool/sysroot_wrapper/switch_to_bad.sh
inary_search_tool/sysroot_wrapper/switch_to_good.sh
inary_search_tool/sysroot_wrapper/test_script.sh
0e2d9a0bafc6bff6a53ca4ddf779715994f28ea8 26-Mar-2015 Luis Lozano <llozano@chromium.org> Remove deprecated Telemetry benchmarks.

A bunch of benchmarks listed in our suites have been removed from
Telemetry. Stop trying to use them.

BUG=None
TEST=Verified by hand

Change-Id: Ic624c1867f6129650f9a02f7325d94d7abb970ee
Reviewed-on: https://chrome-internal-review.googlesource.com/210560
Commit-Queue: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/experiment_factory.py
96d936c087ef7a6b628f305b45f57c7a9ddd7994 25-Mar-2015 Han Shen <shenhan@google.com> Remove checking 'cros flash' locally.

The 'cros flash' location is changed recently, and since cros flash
should be present everywhere, we just remove the local checking
part. However, we keep the part that checks remote's cherrypi module.

Change-Id: I3fa06e46eb3cb1d56c03f87ed5f9b632170fabcc
Reviewed-on: https://chrome-internal-review.googlesource.com/210270
Reviewed-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Han Shen <shenhan@google.com>
Tested-by: Han Shen <shenhan@google.com>
mage_chromeos.py
004bccbd890f184ea873ad6ab8260808e1e79988 21-Mar-2015 Luis Lozano <llozano@chromium.org> Use original autotest location for telemetry runs.

For telemetry benchmarks, there is not need for a board specific
version of autotest. There are no compiled tests. So it is easier to
use the "source" location of autotest. This avoid having to build
autotest for the specific board which can be time consuming.
This also fixes the problem where after doing a build, sometimes
the telemetry_Crosperf test cannot be found since there is no need
for the copy.

This does nothing regarding autotest compiled tests. These still need
to be under /build/<board>. If using downloaded image, we could also
download the auotest bits but we will probably need a new "mode" or
flag for crosperf to indicate we want to use the autotest bits that
come from the image and not the one in your chroot. So, more work is
required.

BUG=None
TEST=Tested by hand with multiple telemetry benchmarks.

Change-Id: Id669782a9612fad39326b4d820cbc8e78a8a3d0c
Reviewed-on: https://chrome-internal-review.googlesource.com/209980
Reviewed-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
rosperf/suite_runner.py
f9bb4a966e59736fe6c9eea198faf3613b5110f8 24-Mar-2015 Han Shen <shenhan@google.com> Remove invocation to 'cros_image_to_taget.py', which was deprecated on Feb 4.

Nightly failed because image_chromeos.py used
'cros_image_to_target.py' (this part still needs to examination ) to
image the machines.

Change-Id: I918e4a99a301ab6cc120fccd0b01c454597b8336
Reviewed-on: https://chrome-internal-review.googlesource.com/210135
Reviewed-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Han Shen <shenhan@google.com>
Tested-by: Han Shen <shenhan@google.com>
mage_chromeos.py
5d7e21d72b17274a81a64a8040e1a30697e1223b 21-Mar-2015 Luis Lozano <llozano@chromium.org> Run telemetry tests with test_that fast mode.

test_that has a "fast" mode that avoids copying large syslog
data that is not used in the case of telemetry runs unless
we are using the autotest profiling functionality.

BUG=None
TEST=Tested with several telemetry benchmarks.

Change-Id: Ib90004099be9cd92b20c9fb0a5f1042068224e6d
Reviewed-on: https://chrome-internal-review.googlesource.com/209816
Reviewed-by: Caroline Tice <cmtice@google.com>
Tested-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Luis Lozano <llozano@chromium.org>
rosperf/suite_runner.py
07eb87f6874d694d75887c0e889931a2cb4f36c4 11-Mar-2015 cmtice <cmtice@google.com> Fix typo that was causing some files to not be found/deleted.

BUG=None
TEST=Tested command by hand.

Change-Id: Ie61fcd802fe91ca02f361fe3665001409e4b185f
Reviewed-on: https://chrome-internal-review.googlesource.com/207315
Reviewed-by: Rahul Chaudhry <rahulchaudhry@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
uto_delete_nightly_test_data.py
a8446712a67f5a5d422ef1cbc40f72f4f9c4ae52 26-Feb-2015 Yunlian Jiang <yunlian@google.com> crosperf: fix the problem on machine assignment.

BUG=chromium:461628
TEST=it always reuses the image on the machine for test shown in bug.

Change-Id: I8106eaa31395db3ee95507b2a9cd7ed0439bc890
Reviewed-on: https://chrome-internal-review.googlesource.com/202667
Reviewed-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
rosperf/machine_manager.py
fe054f1eac0d0d739b74dee6bd45805af86568a2 19-Feb-2015 Han Shen <shenhan@google.com> Only tar chromiumos_test_image.bin file, instead of all files under images directory.

Change-Id: I27ca96a5e347de6d2e5ae280cfd37a7669056679
Reviewed-on: https://chrome-internal-review.googlesource.com/199736
Reviewed-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Han Shen <shenhan@google.com>
Tested-by: Han Shen <shenhan@google.com>
uildbot_test_toolchains.py
est_toolchains.py
fdd8a5ba9cb296696e8d3c814e2ac7561f7bff4a 18-Feb-2015 Han Shen <shenhan@google.com> Added a function that cleans temporaries/images under crostc/chromeos.

Change-Id: Ifa92fc519afadcf10297773a2d42e27e1c9c0f9d
Reviewed-on: https://chrome-internal-review.googlesource.com/199617
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Han Shen <shenhan@google.com>
Tested-by: Han Shen <shenhan@google.com>
uto_delete_nightly_test_data.py
374effc397a7a427b04aed64c07e234c78273133 13-Feb-2015 Rahul Chaudhry <rahulchaudhry@chromium.org> toolchain-utils: remove dead parrot

chromeos2-row3-rack2-host10.cros is dead.
http://b/19147456 says "unable to recover".
Do we need a new parrot machine?

BUG=None
TEST=None

Change-Id: Iabd3116dc558632c53ba3cc7d577de32b892d0f5
Reviewed-on: https://chrome-internal-review.googlesource.com/198821
Reviewed-by: Han Shen <shenhan@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
Commit-Queue: Rahul Chaudhry <rahulchaudhry@google.com>
rosperf/default_remotes
776d528ff8fa7e66d6abf96a735050c70f950a2b 13-Feb-2015 Rahul Chaudhry <rahulchaudhry@chromium.org> Mark buildbot_test_toolchains.py, test_toolchains.py, and weekly_report.py executable.

BUG=None
TEST=None

Change-Id: Ia49aef079414d7072a57dc14d9b0d99fa3a75d40
Reviewed-on: https://chrome-internal-review.googlesource.com/198670
Reviewed-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
uildbot_test_toolchains.py
est_toolchains.py
eekly_report.py
ce5ffa4249faa2f50f08c4168b61bb922381450f 13-Feb-2015 cmtice <cmtice@google.com> Add date to job descriptions; wait for pending builds.

Previously if a trybot job got put into a pending queue
rather than starting immediately, we didn't notice and
would end up using the previous day's trybot image. This
CL fixes that.

BUG=None
TEST=Ran successfully from role account.

Change-Id: I3dd2cf1edd639e562fa78769c86c99e4c51f3a0b
Reviewed-on: https://chrome-internal-review.googlesource.com/198610
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
uildbot_test_toolchains.py
tils/buildbot_utils.py
86cd5ab71d86a46853af6c25e5c98f8f15da6d5d 13-Feb-2015 Rahul Chaudhry <rahulchaudhry@chromium.org> New location for sheriffs.txt file.

BUG=None
TEST=None

Change-Id: I28c768ac45ca480d09cc1a6fa1fb141eb37d3ac5
Reviewed-on: https://chrome-internal-review.googlesource.com/198645
Reviewed-by: Han Shen <shenhan@google.com>
Commit-Queue: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
heriff_rotation.py
1047b071bef1c2839d3a2858b1eb5b91f606eb84 10-Feb-2015 cmtice <cmtice@google.com> Search Google Storage for trybot image artifacts.

Sometimes trybot jobs fail in the testing phase, but the build
finished and a build artifact exists. To deal with this, instead
of checking the trybot job status, we just search for the build
artifact in google storage. This also removes the
'--chrome_rev=tot' flag from the buildbot command.

Change-Id: I38c07270d774258953d9b1f1e694aba9730ac92b
Reviewed-on: https://chrome-internal-review.googlesource.com/197366
Reviewed-by: Luis Lozano <llozano@chromium.org>
Tested-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
tils/buildbot_utils.py
d6a79af67cea3715984c5acab383e52742ae12d8 31-Jan-2015 cmtice <cmtice@google.com> Add all the files and scripts for using the binary search tool to
do binary search triage on ChromeOS packages.

BUG=None
TEST=Tested the scripts on a lumpy build.

Change-ID: I131b4f24d53a32ccec70fe320119ffccb09e73ca
Reviewed-on: https://chrome-internal-review.googlesource.com/194849
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
inary_search_tool/cros_pkg/README.cros_pkg_triage
inary_search_tool/cros_pkg/cros_pkg_boot_test.sh
inary_search_tool/cros_pkg/cros_pkg_create_cleanup_script.py
inary_search_tool/cros_pkg/cros_pkg_get_initial_items.sh
inary_search_tool/cros_pkg/cros_pkg_interactive_test.sh
inary_search_tool/cros_pkg/cros_pkg_setup.sh
inary_search_tool/cros_pkg/cros_pkg_switch_to_bad.sh
inary_search_tool/cros_pkg/cros_pkg_switch_to_good.sh
inary_search_tool/cros_pkg/cros_pkg_undo_eclean.py
513d8520810810a9a0d27c1439a687602ea5a8f6 06-Feb-2015 cmtice <cmtice@google.com> Fix syntax error: Replace 'else if' with 'elif'.

Change-Id: I3be3893c69925dd9e5e38be0b885ddf18dc2edd6
Reviewed-on: https://chrome-internal-review.googlesource.com/196445
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
tils/command_executer.py
a1fbd8af3ea4efc3a9bda1717212a9bacf30950a 05-Feb-2015 cmtice <cmtice@google.com> Make crosperf fail if 'build' field is empty.

Change-Id: Ia929969e0e82cc16f2ebd7c020b1af7e0b5c84ff
Reviewed-on: https://chrome-internal-review.googlesource.com/196107
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/experiment_factory.py
37828578795f39daadcebe05cacc522d629b5c9c 05-Feb-2015 cmtice <cmtice@google.com> Update command_executer.py to allow a silent execution mode,
where it does not attempt to write any logs.

Update auto_delete_nightly_test_data.py to use the
command_executer in silent mode (so it doesn't run out of space
writing logs on the disk it is trying to clean up).

Change-Id: Id1c2c5f171a2d3854d3b53516ead31050da647b4
Reviewed-on: https://chrome-internal-review.googlesource.com/195877
Reviewed-by: Han Shen <shenhan@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
uto_delete_nightly_test_data.py
tils/command_executer.py
d54f980a7349e06059df5c830643e34aac79b86e 05-Feb-2015 cmtice <cmtice@google.com> Make fatal errors if trybot job fails or trybot image is empty.

Update GetTrybotImage to check the final trybot job status and
return an error if the job did not have a success status. Also
update the nightly buildbot test jobs to fail fatally if the
trybot or vanilla image names are empty.

BUG=None
TEST=Tested this in role account.

Change-Id: I17c4ca5fa77ea488c8bf3ae4e5c225b747c90c87
Reviewed-on: https://chrome-internal-review.googlesource.com/196095
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
uildbot_test_toolchains.py
tils/buildbot_utils.py
2da6494fed2e849e360ea2ae5d9e880afdb1cb68 27-Jan-2015 Luis Lozano <llozano@chromium.org> Removed ZGB machines. They dont longer exist in the lab.

BUG=None
TEST=None

Change-Id: Id6bdf17551552cfe9e04a89d8b76547959cfac3f
Reviewed-on: https://chrome-internal-review.googlesource.com/193701
Commit-Queue: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Caroline Tice <cmtice@google.com>
rosperf/default_remotes
46093e58cfe1bf29cc76e1ab87540239ddd7cb8b 09-Dec-2014 cmtice <cmtice@google.com> New script for using buildbots for nightly tests.

This script could eventually replace the current test_toolchains.py
script. Instead of downloading and building ChromeOS locally, it
launches a buildbot to build ChromeOS with the latest mobile
toolchain compiler. It then downloads the trybot image (and the
corresponding vanilla image), generates the crosperf experiment file,
and launches crosperf.

I have been running this as a second cron job on chrotomation2,
running daisy tests, and it has been working just fine.

BUG=None
TEST=Ran in cron jobs on chrotomation2.

Change-Id: I3b17d0373e2fae359fc8f4e0188dcec2943c7b71
Reviewed-on: https://chrome-internal-review.googlesource.com/187735
Reviewed-by: Caroline Tice <cmtice@google.com>
Trybot-Ready: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
uildbot_test_toolchains.py
tils/buildbot_utils.py
e12e5b2688f00fdecd11290a738022becf4272dd 16-Dec-2014 cmtice <cmtice@google.com> New script to tell if machines are 'identical' for crosperf.

This script calls the functions in machine_manager.py and
verifies that two or more machines are 'identical' for the purposes
of running crosperf. I have several times needed to do this type
of thing, so I decided to extract the functionality into its own
script.

BUG=None
TEST=Tested in my directory.

Change-Id: I2a410eefe7e1b46f6c5cad477640ea7974a614c7
Reviewed-on: https://chrome-internal-review.googlesource.com/188197
Reviewed-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/compare_machines.py
a68062289c9b2592dffd79ec76b2b0dd121db12a 09-Dec-2014 cmtice <cmtice@google.com> Add 3 new boards (9 new machines) to default_remotes.

BUG=None
TEST=Ping'd machines and ssh'd into them

Change-Id: I4308742350b8e453e5ea881f76d90829bdac7e0d
Reviewed-on: https://chrome-internal-review.googlesource.com/187709
Reviewed-by: Han Shen <shenhan@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/default_remotes
71f94b8a1c852aed792abfdb4f276dc904047fdd 13-Dec-2013 Han Shen <shenhan@google.com> Move binary search tool to toolchain-utils with a comprehensive test suite.

We now can test the binary search tool with -
./binary_search_tool_tester.py
This will generate a set of object files with some bad ones, and call
the binary search tool to find them out.

TEST=gpylint and passed my test suite
BUG=None

Change-Id: I56fbda7f75fe3bc239e456161c48b7611c3a315d
Reviewed-on: https://chrome-internal-review.googlesource.com/150255
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Han Shen <shenhan@google.com>
Tested-by: Han Shen <shenhan@google.com>
inary_search_tool/.gitignore
inary_search_tool/__init__.py
inary_search_tool/binary_search_perforce.py
inary_search_tool/binary_search_state.py
inary_search_tool/test/__init__.py
inary_search_tool/test/binary_search_tool_tester.py
inary_search_tool/test/common.py
inary_search_tool/test/gen_init_list.py
inary_search_tool/test/gen_obj.py
inary_search_tool/test/is_good.py
inary_search_tool/test/switch_to_bad.py
inary_search_tool/test/switch_to_good.py
8d77427d365e315c3694eebc520e6c454ed3f1d3 05-Nov-2014 Rahul Chaudhry <rahulchaudhry@chromium.org> Run a dummy command in ChrootRunCommand to create a chroot if needed.

In ChrootRunCommand, if return_output is set, run a dummy command first to
make sure the chroot exists before running the actual command whose output
is desired.

BUG=chromium:429823
TEST=Ran build_tc.py with freshly setup chromeos sources without a chroot.

Change-Id: Icdd954671e172066f8fb3402d04174295b1aff61
Reviewed-on: https://chrome-internal-review.googlesource.com/182696
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
tils/command_executer.py
215c12ff93f89edcf42fa744e9767ff606d4df0f 05-Nov-2014 Rahul Chaudhry <rahulchaudhry@chromium.org> Copy the gcc libs to the appropriate destination directory for the board.

Use get_board_arch defined in toolchain_utils to get the right destination
directory for gcc libs.

BUG=chromium:429775
TEST=Ran build_tc.py for panther board.

Change-Id: I1eff4ac788dd61338037838f7d3f688163a6c2bc
Reviewed-on: https://chrome-internal-review.googlesource.com/182626
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
uild_tc.py
tils/misc.py
8ac4ec04a55eff35b33977dbb5a0964fd62cacd8 01-Nov-2014 Rahul Chaudhry <rahulchaudhry@chromium.org> Specify full paths for scripts run inside the chroot.

Remove the assumption that the current directory is ~/trunk/src/scripts/
when the chroot is entered with cros_sdk.

BUG=chromium:429765
TEST=Ran build_chromeos.py, build_tc.py with the changes.

Change-Id: I9e54fda6b5f8ec8fe18b18cf67824c923a7e82e2
Reviewed-on: https://chrome-internal-review.googlesource.com/182270
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
uild_tc.py
tils/command_executer.py
tils/misc.py
c1635a82211eecd977fcf923a09017e8c9af12cd 31-Oct-2014 Rahul Chaudhry <rahulchaudhry@chromium.org> Remove the default=1 for --jobs option to setup_chromeos.py.

If the --jobs argument is not given, simply run repo sync
without -j and use repo's default (8).

BUG=None
TEST=Ran setup_chromeos.py with/without the flag. Runs repo sync as expected.

Change-Id: I89220496aa065db77b6e32cb97f7e50f6f9e73bc
Reviewed-on: https://chrome-internal-review.googlesource.com/182269
Reviewed-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
etup_chromeos.py
61cfa96e19c0d6b002fcf643c250a341963fa281 31-Oct-2014 Rahul Chaudhry <rahulchaudhry@chromium.org> Delete the .git5_specs crud from toolchain_utils.

git5_specs seems to have been accidentally included during initial import.

BUG=None
TEST=None

Change-Id: I221c39718fed4f0590c7eedffa8fcbdfc673bfb4
Reviewed-on: https://chrome-internal-review.googlesource.com/182268
Reviewed-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
git5_specs/H7c3295650ff892250c9c8f5086e5f120
cec0bd8d97149be185e0224fddfa0a8fd8d8dfde 23-Oct-2014 Rahul Chaudhry <rahulchaudhry@chromium.org> Replace "svn ls" with "git ls-remote".

Check for the chrome repository access in git instead of svn.

BUG=None
TEST=Ran setup_chromeos.py after the change and it worked.

Change-Id: I1fed7cb0caa97ce55f0963894c011b783bb5f9e5
Reviewed-on: https://chrome-internal-review.googlesource.com/180926
Reviewed-by: Rahul Chaudhry <rahulchaudhry@google.com>
Commit-Queue: Rahul Chaudhry <rahulchaudhry@google.com>
Tested-by: Rahul Chaudhry <rahulchaudhry@google.com>
etup_chromeos.py
1a224369afcbfd0276f4c7bdc625dec7f7b30d01 17-Oct-2014 cmtice <cmtice@google.com> Fix issue where crosperf can't find local cros_scratch.

Crosperf was assuming the user always has an NFS home directory,
and was looking for the cros_scratch directory there. This makes
crosperf use the user's home directory, whether it's NFS or local.

Also change --share_users, which also assumed NFS home directories,
to --share_cache, which requires specification of the cache path(s)
to use.

BUG=None
TEST=Tested this with my new local homedir.

Change-Id: I28d15467dfbd25b8b237cefa2979d3fd9464a7a7
Reviewed-on: https://chrome-internal-review.googlesource.com/179978
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/benchmark_run.py
rosperf/benchmark_run_unittest.py
rosperf/experiment.py
rosperf/experiment_factory.py
rosperf/results_cache.py
rosperf/results_cache_unittest.py
rosperf/settings_factory.py
rosperf/settings_factory_unittest.py
bf70b0431858abf04d9b894c6d0f060e85589dc5 18-Oct-2014 cmtice <cmtice@google.com> Fix assumption that 'graph' results field is non-null.

Crosperf dies handling the results of the graphics test,
because the 'graph' field in the results is null and crosperf
was expecting it to be non-null. This fixes that problem.

BUG=chromium:424623
TEST=Ran graphics test with this fix and it no longer crashes.

Change-Id:I6764e7c38db5d1913109948d5f4798bdf5d6fb7c
Reviewed-on: https://chrome-internal-review.googlesource.com/180212
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/results_cache.py
23130bd2ecffbf206338d00b4cd66933854a3857 15-Jul-2014 cmtice <cmtice@google.com> Add AFDO to nightly builds.

This adds an option, '--use_afdo' to build_chromeos.py, along with
a list of boards for which it is valid. The flag will cause
build_chromeos.py to add 'USE=afdo_use' to calls to build_packages,
if the board is appropriate. This CL also updates test_toolchains.py
to pass --use_afdo to build_chromeos, if appropriate.

BUG=None
TEST=Tried it in my own directory and it worked.

Change-Id: I0a2056f86783dbd30a2fd650729c888fd1df1f67
Reviewed-on: https://chrome-internal-review.googlesource.com/169174
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
est_toolchains.py
d207b21f57a4dd63ad47da05e0ff4c57b50503e0 23-Jul-2014 Luis Lozano <llozano@chromium.org> Changed name of one of our lab machines.

Changed name of one of our daisy machines according to
http://b.corp.google.com/issue?id=14896349.

Change-Id: I7e25b91dfb4421d021397ad43aabb9bdc4152494
Reviewed-on: https://chrome-internal-review.googlesource.com/170230
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
rosperf/default_remotes
50ef334ec9d72f3bfa06be51a493a1ff8f4c3162 10-Jul-2014 Soumya Basu <sbasu@google.com> Add framework for parsing logs obtained from heap measurements in ChromeOS machines.

BUG=None
TEST=Manual testing on chrome logs.

Change-Id: I48f0062180fd6ee723ccbff047b5fea625e4e3e1
Reviewed-on: https://chrome-internal-review.googlesource.com/168652
Reviewed-by: Simon Que <sque@google.com>
Tested-by: Soumya Basu <sbasu@google.com>
Commit-Queue: Soumya Basu <sbasu@google.com>
em_tests/README
em_tests/clean_data.py
em_tests/mem_groups.py
em_tests/total_mem_actual.py
em_tests/total_mem_sampled.py
em_tests/utils.py
74b3d3234dc857ebee3af81faa3f8f420ac6ecce 15-Jul-2014 cmtice <cmtice@google.com> Fix issue where Telemetry tests appear to pass, but really failed

Occasionally Telemetry tests will return a "pass" even though the test
did not run (e.g. if the test was disabled). Crosperf was looking for
results for every passing test and dying when it failed to find a
result for these false passes. This CL looks for the false pass and
changes it fail, as well as writing out a warning message.

A bug was filed for this in Buganizer:
https://b.corp.google.com/issue?id=16190266, but since this was not
filed as a Chrome/ChromeOS bug, I will leave the bug mark below as
None (to avoid confusing the bug droid).

BUG=None
TEST=Tested this against a Telemetry run with both false and real
passes, and it worked properly.

Change-Id: Ie010c0cd5b0775d0e20cfd6967975eb4e768c55c
Reviewed-on: https://chrome-internal-review.googlesource.com/169115
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/results_report.py
c490e073c811f60d6fdfec5c193e7f042302b5e8 14-Jun-2014 cmtice <cmtice@google.com> Update and expand regression tests for results_cache.py

BUG=None
TEST=Ran regression tests successfully. Ran 'normal' crosperf tests
successfully as well.

Change-Id: Ib3387cb63f1050cc0d561cc676d91be2334e0c79
Reviewed-on: https://chrome-internal-review.googlesource.com/166133
Reviewed-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/results_cache.py
rosperf/results_cache_unittest.py
rosperf/test_cache/compare_output/autotest.tbz2
rosperf/test_cache/compare_output/machine.txt
rosperf/test_cache/compare_output/results.txt
rosperf/test_cache/test_input/autotest.tbz2
rosperf/test_cache/test_input/machine.txt
rosperf/test_cache/test_input/results.txt
rosperf/test_cache/test_puretelemetry_input/machine.txt
rosperf/test_cache/test_puretelemetry_input/results.txt
rosperf/unittest_keyval_file.txt
4536ef6c9da0f8c0c544552798c0bcf516519f73 08-Jul-2014 cmtice <cmtice@google.com> Minor fixes to weekly report generation.

Update test_toolchains.py to clean out old (stale) data before
copying current image tar files to reports directory. Update
weekly_report.py to generate report for previous 6 days, depending on
when report is run, rather than being hard-coded to start the report on
a particular day.

BUG=Non
TEST=Ran test_toolchains.py as part of nightly test; it worked properly.
Ran weekly_report.py and it did the right thing.

Change-Id: Id173ccaa3146c76e19636501984bab11a41f8568
Reviewed-on: https://chrome-internal-review.googlesource.com/168465
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
est_toolchains.py
eekly_report.py
6367e17c2a01a67fe015f4a94a1cbb30d7958dfc 18-Jun-2014 cmtice <cmtice@google.com> Adding/updating a few more unittests for Crosperf.

BUG=None
Test=Ran all regression tests. Ran 'normal' crosperf.

Change-Id: If1dcafee39f10c20c935908c466fbae8fdb23a35
Reviewed-on: https://chrome-internal-review.googlesource.com/166485
Reviewed-by: Han Shen <shenhan@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/config_unittest.py
rosperf/crosperf_unittest.py
rosperf/experiment_file_unittest.py
rosperf/flag_test_unittest.py
rosperf/settings_factory_unittest.py
rosperf/settings_unittest.py
4f0309db0ba1f1d927d8c477afe253959180f6a7 15-Jun-2014 cmtice <cmtice@google.com> Add regression test for suite_runner.py

BUG=None
TEST=Ran regression test. Ran normal crosperf.

Change-Id: I35c5a83d691e0a2a9272422539afdc740e7828f6
Reviewed-on: https://chrome-internal-review.googlesource.com/166169
Reviewed-by: Han Shen <shenhan@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/suite_runner.py
rosperf/suite_runner_unittest.py
56fb7166feb18a20662672d7f8589a1336a8c5a3 18-Jun-2014 cmtice <cmtice@google.com> Update nightly tests (and weekly report stuff) to use official builds.

This CL changes the nightly tests to download and use the official
image for our 'vanilla' image, rather than building the vanilla image
ourselves. It also updates the code that creates & copies the tar
files of the images, for the weekly reports, to look for and copy the
official images.

BUG=None
TEST=Tested the changes by hand. Also let them run for a couple of
nights for our nightly tests.

Change-Id: I8c3215aab7550dc9f148ed79095295f702141017
Reviewed-on: https://chrome-internal-review.googlesource.com/166477
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
est_toolchains.py
tils/misc.py
03d309800564e52785cfa88a6756e35e6081cfaa 12-Jun-2014 Han Shen <shenhan@chromium.org> Adds a second build of host packages.

Adds a second build of host packages and do some testing, which
mimics the ways chromiumos-sdk cbuildbot does.

So actually we perform 2 builds of all host packages -
1. build new toolchain using old toolchain and build all host packages
using the newly built toolchain
2. build the new toolchain again but using new toolchain built in step 1,
and build all host packages using the newly built toolchain

TEST=locally
BUG=None

Change-Id: Ic35092893b10860ca1112a1abb3356a279d36869
Reviewed-on: https://chrome-internal-review.googlesource.com/163325
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Han Shen <shenhan@google.com>
Tested-by: Han Shen <shenhan@google.com>
uild_tool.py
4859f5f8feb4817a1b7ae4672b5bec7bafb79e8b 30-May-2014 cmtice <cmtice@google.com> Add script for generating weekly comparison report.

BUG=None
TEST=Ran this script multiple times from the role account, with
the nightly tests.

Change-Id: Ifbf723a77388908f6fe9c1b1816a20aa5c424f26
Reviewed-on: https://chrome-internal-review.googlesource.com/164978
Reviewed-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
eekly_report.py
94bc470daff948021e754e6a63061da221faaf77 30-May-2014 cmtice <cmtice@google.com> Update test_toolchains.py to save tar files of the images used.

This is for generating a weekly report that compares the nightly
test results. All of the nightly test results are cached, but in
order to find the correct cache results, you need the image from
which the results were created. Due to disk space, some of the
chroots will be deleted before the weekly report gets generated.
Therefore, this modification tars up the test & vanilla image
directories and stores them separately, where they can be accessed
for generating the weekly report.

BUG=None
TEST=Used this successfully last night for nightly test runs.

Change-Id: I01b457a402c3649d7904a478b37bad60b3395300
Reviewed-on: https://chrome-internal-review.googlesource.com/164879
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
est_toolchains.py
1505b6ac10710044361cd65c7890014cb43e9930 04-Jun-2014 cmtice <cmtice@google.com> Update and expand machine_manager_unittest.

Expand machine_manager_unittest.py to test all functions in
machine_manager.py. Modify MachineManager class to optionally take a
command_executer and logger as arguments; also to make command executer
and logger class member attributes (rather than external objects), to
simplify testing.

BUG=None
TEST=Ran unittest to make sure it works; ran all the other unittests; ran
several 'real' crosperf runs to verify that the changes didn't break
crosperf.

Change-Id: I29798457cb1f7572cb339722b572b9d95b21b989
Reviewed-on: https://chrome-internal-review.googlesource.com/165256
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/label.py
rosperf/machine_manager.py
rosperf/machine_manager_unittest.py
2a370f7af34f6b126e80ef55891b9bff6a2858bb 30-May-2014 cmtice <cmtice@google.com> Add --cache_only global setting to crosperf.

The CL adds a new setting, --cache_only, which causes crosperf to
ONLY report test results that it finds in the cache (it will NOT
rerun any tests, even if no cache results are found). This is partly
for implementing a weekly summary report, where we want to compare the
nightly results from the past week, but not actually rerun anything.

BUG=None
TEST=I tested this option while generating weekly test reports, and it
worked as it is supposed to.

Change-Id: Icb1f5343f76ce999bebd0810bb86ab64a7577313
Reviewed-on: https://chrome-internal-review.googlesource.com/164890
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/benchmark_run.py
rosperf/benchmark_run_unittest.py
rosperf/experiment_factory.py
rosperf/label.py
rosperf/machine_manager_unittest.py
rosperf/mock_instance.py
rosperf/results_cache_unittest.py
rosperf/settings_factory.py
4d24b3a0e8a8fae5ac9cf00c3889288c0571b1fc 30-May-2014 cmtice <cmtice@google.com> Fix problems found when running benchmark_run_unittest.

When the changes to benchmark_run_unittest got committed,
somehow the changes to logger.py did not get committed (adding
Mock functions to logger). Also, MockMachineManger.AcquireMachine
needs an extra parameter, to match change to
MachineManager.AcquireMachine. This CL fixes both of those problems.

BUG=None
TEST=Ran unittests successfully after these changes.

Change-Id: I7ae35241ee62ddb484728ffc5c26ea14bc1f7c7f
Reviewed-on: https://chrome-internal-review.googlesource.com/164889
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
rosperf/machine_manager.py
tils/logger.py
4eb645356394bb17913c7224767e1f47acab4fb4 16-May-2014 Don Garrett <dgarrett@google.com> Rename chromite.buildbot -> chromite.cbuildbot.

BUG=None
TEST=None
CQ-DEPEND=CL:199664

Change-Id: I6b2b119960cf4368a5cbc594523b4d44407e91f9
Reviewed-on: https://chrome-internal-review.googlesource.com/163786
Reviewed-by: Don Garrett <dgarrett@google.com>
Commit-Queue: Don Garrett <dgarrett@google.com>
Tested-by: Don Garrett <dgarrett@google.com>
utomation/clients/helper/chromeos.py
emote_gcc_build.py
ba0d0de24f34f008c150b9b5fab2f6dfa4c5217b 21-May-2014 cmtice <cmtice@google.com> Fix bug, parameter name masked module name.

New 'logger' parameter name in __init__ function masked module
name. This CL renames the parameter to something else.

BUG=None
TEST=Ran crosperf with the fix and it worked.

Change-Id: I19b5e452d50ce9937662ec3ee36ebb79747985f5
Reviewed-on: https://chrome-internal-review.googlesource.com/164135
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/experiment_runner.py
798a8fa986db930786b2a6777db0f3b06db995f6 12-May-2014 cmtice <cmtice@google.com> Better handling of "not identical machines" failure.

Nightly tests, especially on the x86-generic boxes, fail frequently
because the DUTs have different board images on them, so Crosperf
decides the machines are not identical and refuses to run the tests.
With this CL, if the machine_manager finds that the machines fail the
"identical" test, it will try to push the same image onto all the machines,
and then check them again to see if they are the same. It only tries this
once; if they fail the check the second time around, it is still a fatal
failure. This should eliminate many of the unnecessary failures in our
nightly tests.

This CL also fixes a small bug in the auto-delete script (this fix has
been running for a while on mobiletc-prebuild, but never got committed).

BUG=None
TEST=Ran several iterations where I forced the first "identical" check to
fail. The changes worked.

Change-Id: Ied2a55e5d3e2789e58a503aef03269888954b579
Reviewed-on: https://chrome-internal-review.googlesource.com/163334
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
uto_delete_nightly_test_data.py
rosperf/benchmark_run.py
rosperf/machine_manager.py
df8053336feea245a6bb43fe16dbc16a7b3d71cd 20-May-2014 cmtice <cmtice@google.com> Add --skip_chroot_upgrade flag to build_packages.

This CL adds the '--skip_chroot_upgrade' flag to all calls to
build_packages that clobber CFLAGS, LDFLAGS, etc. This is to
prevent the call to build_packages from deciding to try to rebuild
the compiler itself (which is not the right place to do that).

BUG=None
TEST=Tested adding flag by hand to call to build_packages that used to
fail; it passed with the flag.

Change-Id: Ia0152f76f8a25cf0488a4a72c4add7e4b3182feb
Reviewed-on: https://chrome-internal-review.googlesource.com/164050
Reviewed-by: Han Shen <shenhan@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
uild_chromeos.py
42709763f709b0e08d4ed041e4374cb9dd5d6570 16-May-2014 cmtice <cmtice@google.com> Add unit test for experiment_runner.py

BUG=None
TEST=Ran unittest.

Change-Id: I13b2822f57f4201c94215c603a60f12755230f71
Reviewed-on: https://chrome-internal-review.googlesource.com/163833
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/experiment_runner.py
rosperf/experiment_runner_unittest.py
rosperf/settings_factory.py
87ae9a0635ad5cb7606e2703ed60b3013920e727 08-May-2014 cmtice <cmtice@google.com> Update & complete benchmark_run_unittest.py

Add tests for all the methods in BenchmarkRun.

BUG=Non
TEST=Ran unittest.

Change-Id: I9046f010ce0e48ba8d897585fec2f95732df9cd3
Reviewed-on: https://chrome-internal-review.googlesource.com/162988
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/benchmark_run.py
rosperf/benchmark_run_unittest.py
2250df9b2fb7ffe523b4f90e06b0d7602b83876f 09-May-2014 cmtice <cmtice@google.com> Expand experiment_factory_unittest.py to be more complete.

This CL adds more testing to experiment_factory_unittest.py, and it
fixes one small bug in experiment_factory.py.

BUG=None
TEST=Ran unittest.

Change-Id: I9fd13c05d793bb71accf3e19d361dff79f7110e9
Reviewed-on: https://chrome-internal-review.googlesource.com/163149
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/experiment_factory.py
rosperf/experiment_factory_unittest.py
8dfdabd29b15f3bc5ebc4692d119e51826c6ef17 08-May-2014 Han Shen <shenhan@chromium.org> Correctly test failures while building boards.

TEST=None
BUG=None

Change-Id: Iaae09d30214d6f8bb8b069d5fc4009a20cb0e6d8
Reviewed-on: https://chrome-internal-review.googlesource.com/162912
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Han Shen <shenhan@google.com>
Tested-by: Han Shen <shenhan@google.com>
uild_tool.py
2fca8ce27632ebce63627b5a51e9749e68bcf8b9 07-May-2014 cmtice <cmtice@google.com> Add 3 unittests, for test_flag.py, config.py and benchmark.py

BUG=None
TEST=Ran unittests.

Change-Id: I2c421f8322e58173932656da9ae7a5af1bf3febd
Reviewed-on: https://chrome-internal-review.googlesource.com/162886
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/benchmark_unittest.py
rosperf/config_unittest.py
rosperf/flag_test_unittest.py
6ffbb92f0913f990b6a5c7ef706aee9bfb4faa9e 07-May-2014 cmtice <cmtice@google.com> Add unittesting for download_images.py (and translate_xbuddy.py).

This re-structures the code in download_images.py, for easier unit
testing, and adds unit tests for download_images. The
download_images_buildid_test.py tests translate_xbuddy.py as well.

BUG=None
TEST=Ran crosperf tests for official & trybot images with new
download_images.py file; Ran both unittest files.

Change-Id: Ib6ea17d5bd8ed8c87c403a7b1eef49237e5414b8
Reviewed-on: https://chrome-internal-review.googlesource.com/162866
Reviewed-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/download_images.py
rosperf/download_images_buildid_test.py
rosperf/download_images_unittest.py
fe3001c80006e3b86ee891d72a03b1f6ab04f5ed 29-Apr-2014 Han Shen <shenhan@chromium.org> Add support to build mutiple boards for binutils/gcc.

Now you can specify multiple boards in one command line like this
./build_tool.py --board='x86-alex,lumpy'

TEST=None
BUG=None

Change-Id: Idac899bb225ce51eab922f5ab0b08b36ce7b0e20
Reviewed-on: https://chrome-internal-review.googlesource.com/161923
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Han Shen <shenhan@google.com>
Tested-by: Han Shen <shenhan@google.com>
uild_tool.py
57f71b3df9db76e45412da01511899b36db2e3fc 29-Apr-2014 cmtice <cmtice@google.com> Fix bug to now allow mounting chrome_src from inside chroot.

Crosperf was checking to see if chrome_src was inside the chroot, and
if so was not mounting it. This patch fixes that problem.

BUG=None
TEST=Tested with my new chroot (which doesn't have chrome source in
'normal' place).

Change-Id: Ide3273ca73aecffa28fa8d19a53f300eb957c29e
Reviewed-on: https://chrome-internal-review.googlesource.com/162009
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/suite_runner.py
226e3e08106dffd3086a7eee7007950a1d8ddfaf 28-Apr-2014 cmtice <cmtice@google.com> Fix small bugs with non-autotest Telemetry; other cleanups.

This CL fixes various small problems found during recent testing:
- Force show_all_results to be true for 'pure' telemetry case (the
results filtering needs the autotest results to work properly)
- Fix bug that was inserting extra 'src' into Chrome src path sometimes
- Since non-autotest Telemetry and non-Telemetry-Crosperf tests can't
handle 'perf' make it a Fatal error to call them with perf args.
- Add missing field to example experiment file.
- Add status output messages to downlaod_images.py for non-verbose logging.

BUG=None
TEST=Tested all these fixes on multiple test runs.

Change-Id: If8209356c695dc8b21f8627399a1cbd4a858df23
Reviewed-on: https://chrome-internal-review.googlesource.com/161759
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/benchmark.py
rosperf/download_images.py
rosperf/experiment_files/telemetry-crosperf-with-external-chrome-src.exp
rosperf/label.py
rosperf/suite_runner.py
4375f760cb471a09b410d4034028eb8e45b1ba24 25-Apr-2014 Han Shen <shenhan@chromium.org> Abort deleting the directory if deletion of any of its chromeos subtree fails.

TEST=None
BUG=None

Change-Id: I4dfd2f67d8bc7f6f329b58800c148a923e78eace
Reviewed-on: https://chrome-internal-review.googlesource.com/161678
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Han Shen <shenhan@google.com>
Tested-by: Han Shen <shenhan@google.com>
uto_delete_nightly_test_data.py
666d175610adbd9e69bb19be92522e7bc3ffcc63 24-Apr-2014 cmtice <cmtice@google.com> Tell users where results files are stored.

This updates the output messages to actually tell users where the
results files are stored, especially the perf.data and perf.report files.

BUG=None
TEST=Ran Crosperf with these changes.

Change-Id: I4030d8fbf47c0a313e0ab349da9c4d201f946d11
Reviewed-on: https://chrome-internal-review.googlesource.com/161486
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/experiment_runner.py
rosperf/results_cache.py
a58a824b41a0121439794cd8a2392364e4d6975c 23-Apr-2014 Han Shen <shenhan@chromium.org> Add necessary output to auto_delete_nightly_test_data script and return with meaning exit code.

The script did not have any output in case of a success run, which
means we do not get emails about this cron job. Also the exit code was
always 0. Fixed by adding print statements and return correct exit
code.

TEST=None
BUG=None

Change-Id: If06a4d57a94387494f9c1b30fea765f66e6e0f56
Reviewed-on: https://chrome-internal-review.googlesource.com/161302
Reviewed-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Han Shen <shenhan@google.com>
Tested-by: Han Shen <shenhan@google.com>
uto_delete_nightly_test_data.py
f4220a30480a945c93d6fdf6fb5853b3ba36c6d1 23-Apr-2014 cmtice <cmtice@google.com> Update and standardize example experiment files.

This cleans up the example experiment files, makes sure they are
all up to date, and that they use consisten wording and examples
throughout. It also renames some of them.

BUG=None
TEST=None

Change-Id: I1bbf9e364a0dfae7cfb880011f2c140e1a347e98
Reviewed-on: https://chrome-internal-review.googlesource.com/161386
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/experiment_files/README
rosperf/experiment_files/aes_perf
rosperf/experiment_files/aes_perf.exp
rosperf/experiment_files/bloat_perf
rosperf/experiment_files/bloat_perf.exp
rosperf/experiment_files/morejs_perf
rosperf/experiment_files/morejs_perf.exp
rosperf/experiment_files/non-telemetry-tests.exp
rosperf/experiment_files/official-image.exp
rosperf/experiment_files/page_cycler
rosperf/experiment_files/page_cycler.exp
rosperf/experiment_files/page_cycler_perf
rosperf/experiment_files/page_cycler_perf.exp
rosperf/experiment_files/telemetry-crosperf-suites.exp
rosperf/experiment_files/telemetry-crosperf-with-external-chrome-src.exp
rosperf/experiment_files/telemetry-crosperf-with-profiler.exp
rosperf/experiment_files/telemetry-crosperf.exp
rosperf/experiment_files/telemetry-pure.exp
rosperf/experiment_files/telemetry-without-autotest.exp
rosperf/experiment_files/test_that.exp
rosperf/experiment_files/toolchain
rosperf/experiment_files/trybot-image.exp
dd75bad34c994c9f3638a83798aedab363e469f1 21-Apr-2014 Luis Lozano <llozano@chromium.org> Use CrosRunCommand to find if cherrypy is available. Fixed bug in CrosRunCommand

Call CrosRunCommand instead of calling ssh directly to find if cherrypy
is available.
Also fix bug in CrosRunCommand. The retval returned was always 0.

BUG=None
TEST=Reproduced with small test cases.

Change-Id: I277416b1809eb7287a75bd224b4154f442132a52
Reviewed-on: https://chrome-internal-review.googlesource.com/161123
Reviewed-by: Han Shen <shenhan@google.com>
Tested-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Caroline Tice <cmtice@google.com>
mage_chromeos.py
tils/command_executer.py
1e462d9f8650dbe3d904beff225748e42ebbf08b 19-Apr-2014 Luis Lozano <llozano@chromium.org> Fixed command to figure out if cherrypi is available on the DUT

Figured out why we sometimes we were not using cros flash for
imaging the machines. It turns out the ssh command used was incorrect
and (did not specify an empty file for the known hosts) so it will
sometimes will fail when connecting to the DUT.
There was another problem because the code assumed a return value of non-zero
meant cherrypi was not available. We know also check for error code 255
which means the ssh itself failed (not the remote command).

BUG=None
TEST=Imaged a machine

Change-Id: I7c5cef859b73bc9f2a777825914a1c1bb162704b
Reviewed-on: https://chrome-internal-review.googlesource.com/161009
Reviewed-by: Han Shen <shenhan@google.com>
Commit-Queue: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
mage_chromeos.py
98a53692fb946a8eac46e3e82257f540d1350c18 16-Apr-2014 cmtice <cmtice@google.com> Clean up Crosperf options and help.

Crosperf's help was completely out of date and incorrect. Fixed
that. Also, I cleaned up the options, removed some that were no
longer appropriate or not ever used, and made some options global-only.
Also removed all references to pyauto. We no longer use or run pyauto
tests.

Option changes: Completely removed outlier_range (it was not actually doing
anything), key_results_only (applied only to pyauto tests), use_test_that
(not needed now that we're not using pyauto) and md5sum (never used).
Made the following options global-only: rm_chroot_tmp, perf_args, logging_level,
show_all_results, board. Removed 'build' from global settings and made
it an image-only setting (it made no sense as a global setting).

I also adjusted the unittests to work with these changes.

BUG=None
TEST=I tested all the options in all their various positions; I tested with and
wtihout caching; and I tested local and official images. I also ran the unittests. Everything works.

Change-Id: I7653663f65c0a3363a1904c0220f1066e72c7f9d
Reviewed-on: https://chrome-internal-review.googlesource.com/160758
Reviewed-by: Bhaskar Janakiraman <bjanakiraman@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/benchmark.py
rosperf/benchmark_run_unittest.py
rosperf/crosperf.py
rosperf/experiment_factory.py
rosperf/experiment_file_unittest.py
rosperf/help.py
rosperf/image_checksummer.py
rosperf/label.py
rosperf/mock_instance.py
rosperf/results_cache.py
rosperf/results_cache_unittest.py
rosperf/results_organizer.py
rosperf/settings_factory.py
rosperf/suite_runner.py
rosperf/translate_xbuddy.py
71e004237b85bdcab97c09b7f58305e5f5eff7f8 18-Apr-2014 cmtice <cmtice@google.com> Various minor crosperf cleanups.

- Add results summary information for peacekeeper.html to json file.
- Don't add section for 'perf table' to html report if there is no perf report.
- Remove the color box for the gmean speedup (there's no good way to label it
and it's confusing unless you know what it is).

BUG=None
TEST=Ran Crosperf with these changes. It all seems to work correctly.

Change-Id: I2bf5bdfc2667ae665767374d10672d2d19696267
Reviewed-on: https://chrome-internal-review.googlesource.com/160896
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/default-telemetry-results.json
rosperf/perf_table.py
rosperf/results_report.py
f0b68e6708a801748908982a3b7bca906faa8a36 17-Apr-2014 Han Shen <shenhan@chromium.org> Rename bootstrap_compiler.py to build_tool.py.

TEST=None
BUG=None

Change-Id: I33d914575c17f4c2b6535ecdbdc474759e9f8f56
Reviewed-on: https://chrome-internal-review.googlesource.com/160841
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Han Shen <shenhan@google.com>
Tested-by: Han Shen <shenhan@google.com>
ootstrap_compiler.py
uild_tool.py
778cab4cfeb975fb49c5ebed79ec86001a0e3282 17-Apr-2014 Luis Lozano <llozano@chromium.org> Do not generate silly ssh warning while checking for cherrypi

BUG=None
TEST=Tested in nightly testing

Change-Id: Ia46a0cbb34890a0cb1222f16157c7c98179ec033
Reviewed-on: https://chrome-internal-review.googlesource.com/160769
Reviewed-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
mage_chromeos.py
819f8625e3a7b09beda3288410b6c99f81fc55b1 09-Apr-2014 Han Shen <shenhan@chromium.org> Add support for building gcc/binutils for a specific board for bootstrap_compiler.py.

TEST=None
BUG=None

Change-Id: I11edfd930b3c4d1985731a6912dec64e941ba03f
Reviewed-on: https://chrome-internal-review.googlesource.com/150959
Reviewed-by: Han Shen <shenhan@google.com>
Commit-Queue: Han Shen <shenhan@google.com>
Tested-by: Han Shen <shenhan@google.com>
ootstrap_compiler.py
tils/misc.py
43f1a45c8dddfc4ff8c9dfcd87070811abf936dd 04-Apr-2014 cmtice <cmtice@google.com> Download official & trybot images; treat as local images for tests.

Currently we cannot get md5sums for official & trybot images, which
causes multiple problems: Crosperf re-images the DUT every time it
runs a test for an image, even if the image is already on the DUT;
if crosperf is running an experiment with multiple official or trybot
images, it can get confused about which results belong to which image;
caching does not work properly without md5sums.

To fix all of these problems, this CL changes Crosperf to download the
official or trybot images into the /tmp directory in the chroot, and from
there it can treat them as local images (getting md5sums etc). In order
to download them, it first has to translate the xbuddy syntax (which can
contain aliases such as 'lumpy/dev-latest') into the actual image name
(e.g. lumpy-release/R36-5727.0.0). This translation is done by a new
script, translate_xbuddy.py, which must be run from the toolchain-utils
directory inside the chroot.

BUG=356279,356474,356476
TEST=Tested with official and trybot images, using both correct
designations and xbuddy aliases; tested with the image missing from
the chroot and with it already in /tmp; tested with and without caching.
Also tested with the experiment file included in the 3 bugs.

Change-Id: I917b5520604b7d4851db2c8123165e81b866da2b
Reviewed-on: https://chrome-internal-review.googlesource.com/159465
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/download_images.py
rosperf/experiment_factory.py
rosperf/experiment_file.py
rosperf/settings.py
rosperf/translate_xbuddy.py
mage_chromeos.py
tils/tabulator.py
c454cee542ca459ef9bd87c9f72e81c822caf1e5 09-Apr-2014 cmtice <cmtice@google.com> Update unittests to all pass.

Fix the parameters to the various unittests so they match recent
changes and the unittests all pass again.

BUG=None
TEST=I ran all the unittests with the changes.

Change-Id: I083b5127a2ade8f1dbaf2bb173d82183871cb7c7
Reviewed-on: https://chrome-internal-review.googlesource.com/159915
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/benchmark_run.py
rosperf/benchmark_run_unittest.py
rosperf/machine_manager.py
rosperf/machine_manager_unittest.py
rosperf/results_cache_unittest.py
rosperf/results_organizer_unittest.py
tils/command_executer_unittest.py
0537956ef5981f4069fcaaefd3d25298f9d6ebd8 07-Apr-2014 cmtice <cmtice@google.com> Always write verbose logs to files.

The option that reduces the amount of screen output (--logging_level
= 'quiet' or 'average') was also reducing what got written to the log
files. This fixes that. With this CL, even though the amount of screen
output is reduced, what gets written to the log files is always the same
(verbose). This CL also makes "average" the default logging level.

BUG=None
TEST=I tested all three logging levels, saving the results in different
log directories and compared the log files afterwards, to make sure
the log files contained the same amount of logging, regardless of the
screen logging level.

Change-Id: I6982499d67c080afa835e01ab4fa2282dd992052
Reviewed-on: https://chrome-internal-review.googlesource.com/159654
Reviewed-by: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
rosperf/settings_factory.py
tils/command_executer.py
tils/logger.py
df3d7d197f33c19e51ca688bdcc2d750e8ffb484 03-Apr-2014 cmtice <cmtice@google.com> Remove 'Completed' column; Remove 'retval' row from summary table.

The "Completed" column causes confusion because tests that fail are
still considered to have completed. This CL removes the 'Completed'
column, and instead posts the number of iterations that passed or failed,
for each test image, in the table header, beside the image name.
It also removes the 'retval' row from the summary table, since the retval
will either be '0' if a test iteration passes or '1' if it fails, and seeing
the average of all the retvals is useless, unless they all pass or fail anyway.
The retval row is still there in the full table, which shows all the results
for each iteration.

BUG=357340
TEST=I ran crosperf with this change in all configurations I could think of and
it seems to work.

Change-Id: I0e64b9ed016b0a4604814bac49442848cd27f5d3
Reviewed-on: https://chrome-internal-review.googlesource.com/159113
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/results_report.py
tils/tabulator.py
44a44befd1f500b9a227ebfd849702efce83ef6a 02-Apr-2014 cmtice <cmtice@google.com> Check for 'significant' results at report generation instead of caching.

The current implementation of the option that masks unimportant results
discards the uninteresting results before caching the test results. After
further thought, that was not the right design choice. Among other things it
can cause confusing results, such as seen in issue 357346. We should always
cache all the results, and do the result filtering during report generation.
This CL makes that change.

BUG=357346,357343
TEST=Ran crosperf tests with and without cache hits, and with and without entreies in the json file. It all seems to work as expected.

Change-Id: I778e5614c73bf751ebaa2d4606af636275247c60
Reviewed-on: https://chrome-internal-review.googlesource.com/159108
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/results_cache.py
rosperf/results_organizer.py
tils/tabulator.py
841f96b7f89dbe6f3e5bb013d211a6f799e4d3ab 04-Apr-2014 Luis Lozano <llozano@chromium.org> Add OWNERS file at the top level directory

BUG=None
TEST=None

Change-Id: I23a6270cb1c7176d0e12eb8d7f999814c6beaa10
Reviewed-on: https://chrome-internal-review.googlesource.com/159319
Reviewed-by: Han Shen <shenhan@google.com>
Commit-Queue: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Bhaskar Janakiraman <bjanakiraman@google.com>
WNERS
2cce1bd6a77fe537fc36d6e1b804bc9f19a67651 27-Mar-2014 cmtice <cmtice@google.com> Add page_cycler.bloat to Toolchain team recommended test set.

BUG=None
TEST=Ran test with this change.

Change-Id: I89aab58c8890a7c3f36707d2ffe7422fbfa17547
Reviewed-on: https://chrome-internal-review.googlesource.com/158585
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
rosperf/experiment_factory.py
91b5599aa5c8875a65b8fa1a1d6f2ba382e1451c 25-Mar-2014 cmtice <cmtice@google.com> Update Sheriff email to point to new web page.

Update the weekly sheriff rotation email to point the sheriff to the
new team Sheriff Duty Details page.

BUG=None
TEST=None

Change-Id: I0278186dbf1f004869e4956e546117cd6efa85c7
Reviewed-on: https://chrome-internal-review.googlesource.com/158315
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
heriff_rotation.py
42bb3ad0094669f0b9eae95e8da5fa278175f0a5 25-Mar-2014 cmtice <cmtice@google.com> Fix syntax error with using some official builds.

This CL fixes a bug with the syntax for specifying some official
builds with xbuddy.

BUG=None
TEST=Tested this version with the older specifications as well as the
new version that caused problems.

Change-Id: Ifa189f554ff2aa45b125716d51da1d9755fb9ddc
Reviewed-on: https://chrome-internal-review.googlesource.com/158255
Reviewed-by: Han Shen <shenhan@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/settings.py
24846b19591ca4fb1d03d0c80a2e86effc55c0f6 20-Mar-2014 cmtice <cmtice@google.com> Update to use "working" version of perf for now.

This is a temporary fix, until we can figure out why the 'normal' version
of 'perf' is not properly symbolizing ChromeOS perf data. This CL
commits a version of the 'perf' binary (based on perf 3.13) that does
properly symbolize ChromeOS perf data files. It also updates crosperf to
copy this working perf binary into the chroot (in /tmp) and to use it
when generating the perf report. Work to track down and fix the real
problem is ongoing, but in the meantime this allows Crosperf users to
get meaningful perf reports for ChromeOS.

BUG=None
TEST=Tested Crosperf using this patch and generating perf reports.

Change-Id: Ia8dd157b88d37a897eb9becf8fa444de41e88873
Reviewed-on: https://chrome-internal-review.googlesource.com/157756
Reviewed-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/results_cache.py
erf.static
77892948fa796a65ada0985acb4a945802a78d13 18-Mar-2014 cmtice <cmtice@google.com> Fix error that wrote logs to 'verbose' subdirectory.

The recent commit that introduced logging levels accidentally left in
a line that caused log files to be written to a subdirectory named
'verbose' rather than to 'logs'. This fixes that problem.

BUG=None
TEST=Tested this will all 3 levels of logging; it works properly.

Change-Id: I0587f242108a1e3762f4161e88f4e6ac119a367b
Reviewed-on: https://chrome-internal-review.googlesource.com/157420
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Han Shen <shenhan@google.com>
rosperf/experiment.py
6de7f8fe91a1a36d0e7c578ac8d170bf483fcfbe 14-Mar-2014 cmtice <cmtice@google.com> Various small fixes for nightly infrastructure tests.

- Change the number of iterations for tests from 1 to 3.
- Increase the timeout for image_chromeos.py, so 'cros flash', which
takes a bit longer than cros_image_to_target.sh did, doesn't get killed
just as its finishing.
- Comment out smoothness.top_25 for now, since it's broken and/or
hanging.

BUG=None
TEST=Successfully ran nightly infrastructure tests with these changes.

Change-Id: I77b663b1910fe12ef927afdf15d88a7c1e82d491
Reviewed-on: https://chrome-internal-review.googlesource.com/157419
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/experiment_factory.py
mage_chromeos.py
est_toolchains.py
139092449a545d4d87c06af3a6d8f3d71e42e927 11-Mar-2014 cmtice <cmtice@google.com> Add --logging_level option to crosperf.

Add --logging_level={verbose, average, quiet} option to crosperf,
optionally reducing the amount of logging output. "verbose" gives
the same output Crosperf always gave; "quiet" gives minimal output, and
"average" is in the middle. The default for now is "verbose". Doing this
also involved adding logging levels to the command_executer, and to
image_chromeos (again, with the default being "verbose").

BUG=None
TEST=Tested all three levels of logging, with and without having to
reimage the DUT. It all worked properly.

Change-Id: Icf8c9a6831fe25202adcb624c6c7e1d1a7ac25a5
Reviewed-on: https://chrome-internal-review.googlesource.com/156883
Reviewed-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/benchmark_run.py
rosperf/experiment.py
rosperf/experiment_factory.py
rosperf/experiment_runner.py
rosperf/experiment_status.py
rosperf/image_checksummer.py
rosperf/machine_manager.py
rosperf/results_cache.py
rosperf/settings_factory.py
rosperf/suite_runner.py
mage_chromeos.py
tils/command_executer.py
tils/file_utils.py
tils/logger.py
tils/tabulator.py
2317decf0e3f62297fc09a712bdf7fa253d560f3 13-Mar-2014 cmtice <cmtice@google.com> Fix Crosperf's ETA bug.

Sometimes crosperf shows a negative number for ETA. This fixes that.

BUG=None
TEST=Force the bug to happen; tested with and without the fix.

Change-Id: I0c8c909a944ad3cf25b8c5416f93ddc0b860a03d
Reviewed-on: https://chrome-internal-review.googlesource.com/157125
Reviewed-by: Han Shen <shenhan@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/experiment_status.py
e77818d606f46e84a592702272d73715b321a773 04-Mar-2014 cmtice <cmtice@google.com> Fix bug introduced with the --show_all_results flag, using cache.

The call to self._ProcessResults is supposed to take the show_all_results
flag as an argument. This was overlooked in the read-it-out-of-the-cache
path when this change was first introduced. This CL fixes that.

BUG=None
TEST=I tested cached and non-cached runs, with and without the
--show_all_results option. Everything worked as it should.

Change-Id: Ibf8f206406b0a4535d48b598709e62135289f993
Reviewed-on: https://chrome-internal-review.googlesource.com/156367
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
rosperf/benchmark_run.py
rosperf/results_cache.py
32f09363f80eb73ba1a22bd90d6e609e8f991c8f 03-Mar-2014 cmtice <cmtice@google.com> Fix problem with looking for 'perf' in the wrong directory.

BUG=None
TEST=Tested with old and new chroots; seems to work.

Change-Id: Id1a7fc9d6b1dea0b96fe51032bb25d4722dd8b48
Reviewed-on: https://chrome-internal-review.googlesource.com/156381
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/results_cache.py
0cc4e7790afbd514675801a1ffb90517c147270f 31-Jan-2014 cmtice <cmtice@google.com> Add 'build' settings option and xbuddy image format.

This patch updates Crosperf to add the 'build' tag in the
experiment file, and to allow that field to contain xbuddy
syntax for using trybot and/or official builds in the test runs.
It also adds a bit more checking to make sure we have everything
necessary for running 'cros flash' before attempting to use it.

BUG=None
TEST=I have run this using an experiment file that compares a local
image, a trybot image and an official image against each other. It
all worked.

Change-Id: Ia896799061508fb5718a3201b1599d8622de0b3f
Reviewed-on: https://chrome-internal-review.googlesource.com/154097
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/experiment_factory.py
rosperf/experiment_file.py
rosperf/experiment_files/official-image.exp
rosperf/experiment_files/trybot-image.exp
rosperf/image_checksummer.py
rosperf/label.py
rosperf/machine_manager.py
rosperf/results_cache.py
rosperf/settings.py
rosperf/settings_factory.py
mage_chromeos.py
fd0b178bef64de689ce86c3cc471daa219bf601f 13-Feb-2014 Han Shen <shenhan@chromium.org> Use 'scaling_max_freq' when 'scaling_available_frequencies' is not available.

In recently arm images, there is no such
'scaling_available_frequencies' file, which leads failure of
crosperf. Fixed this by choosing 'scaling_max_freq'.

TEST=tested on daisy
BUG=None

Change-Id: I433e161495421c57854234fd0e7f7e54406bb9e8
Reviewed-on: https://chrome-internal-review.googlesource.com/154579
Reviewed-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Han Shen <shenhan@google.com>
Tested-by: Han Shen <shenhan@google.com>
rosperf/suite_runner.py
75a28ac9bd52db76d106abc8e23678b0192b3a69 11-Feb-2014 cmtice <cmtice@google.com> Minor Crosperf fixes and cleanups.

Fix crosperf so that it doesn't try to look for a default result
(from the json file) if the telemetry test failed altogether. Also
fix it so that if it can't find the default result (on a successful run),
it returns all the results, rather than exiting with a backtrace. Also
add units to results.

BUG=None
TEST=Tested these fixes on my system. They appear to work as desired now.

Change-Id: I86099a1174c4d91d7d92a5aeea0792c68fa8dbbb
Reviewed-on: https://chrome-internal-review.googlesource.com/154114
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/results_cache.py
fd06cca4b6d842beeefeb64628a5d91478ffb1c8 29-Jan-2014 cmtice <cmtice@google.com> Update image_chromeos.py to use 'cros flash'

Check the chroot to see if it contains the 'cros flash' script; if so
use 'cros flash' instead of 'cros_image_to_target.py' for imaging the
DUT. This is the first of a series of changes to improve the imaging
options in Crosperf. This script does not change any of the options or
syntax for using image_chromeos.py.

BUG=None
TEST=Tested in chroot with & without cros_flash.py. Worked correctly in
both cases.

Change-Id: If8704c81aa800cab85379baa514249105fe2b83f
Reviewed-on: https://chrome-internal-review.googlesource.com/153197
Reviewed-by: Han Shen <shenhan@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
mage_chromeos.py
3dc2a88e73455e0d850045b3f496b589c56abc5c 08-Jan-2014 Han Shen <shenhan@google.com> Added a script to compare 2 chromiumos images.

This is done by comparing each ELF files of those images.

TEST=manually
BUG=None

Change-Id: I7b19665c074645c810d974fc13aa3070d4cba6ee
Reviewed-on: https://chrome-internal-review.googlesource.com/151367
Reviewed-by: Han Shen <shenhan@google.com>
Tested-by: Han Shen <shenhan@google.com>
Commit-Queue: Han Shen <shenhan@google.com>
hromiumos_image_diff.py
b134008563f05bc3595f5b22f2a5e0b866e43c15 13-Jan-2014 cmtice <cmtice@google.com> Add timeout to image_chromeos call to bin/cros_image_to_target.py, to
handle flaky hangs by that script. Also add 2 retries.

BUG=None
TEST=Ran image_chromeos.py with modification and it worked.

Change-Id: Ib9f772b34862704255018bb68b563f721eb4fe3e
Reviewed-on: https://chrome-internal-review.googlesource.com/151776
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
mage_chromeos.py
a6255d058ecf1fe6e9983bb5acf7bec5213c1317 10-Jan-2014 cmtice <cmtice@google.com> Make sure we have correct .boto file and permissions for running
Telemetry tests inside chroot.

BUG=None
TEST=Ran nightly infrastructure tests with these changes and they
ran and passed.

Change-Id: I5c469958f13fc037b8f219b4db542193caf342ca
Reviewed-on: https://chrome-internal-review.googlesource.com/151652
Reviewed-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
est_toolchains.py
970da90b85528c44cbeb6a0d65ff1a4388ff4db6 10-Jan-2014 cmtice <cmtice@google.com> Update names of default result fields to include graph as well as
description (this is necessary now that other CL has been committed).

BUG=None
TEST=Ran telemetry tests with new results files; they worked.

Change-Id: I11324a0b62c5da12ab98f3247dca3120501f88bf
Reviewed-on: https://chrome-internal-review.googlesource.com/151714
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
rosperf/default-telemetry-results.json
126e0c31bf6b973c500f86296391b954ccbae218 10-Jan-2014 cmtice <cmtice@google.com> Add test group containing recommended Telemetry performance tests
for toolchain group.

Change-Id: I178975045ed78c93800d4d6c1f6c86fcb88e3ecd
Reviewed-on: https://chrome-internal-review.googlesource.com/151653
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/experiment_factory.py
cc34c224cfca5ac8253ed650a5c98e79b3de7071 03-Jan-2014 Han Shen <shenhan@google.com> Add a utils function to apply online gerrit patches.

TEST=manuall
BUG=None

Change-Id: Ib9baf38a619e1c4b476b51ca6a21082fa8ff3819
Reviewed-on: https://chrome-internal-review.googlesource.com/151117
Reviewed-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Han Shen <shenhan@google.com>
Tested-by: Han Shen <shenhan@google.com>
tils/misc.py
a9c561a571bdbbb99d97d89ce1a242e989165d9e 02-Jan-2014 Yunlian Jiang <yunlian@google.com> gdb dejagnu: updata arm baseline.

BUG=None
TEST=None

Change-Id: I25eb89ac54801d1e35807ba85689cf1256b4cd6d
Reviewed-on: https://chrome-internal-review.googlesource.com/151037
Reviewed-by: Han Shen <shenhan@google.com>
Commit-Queue: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
ejagnu/boards/gdb.exp.in
ejagnu/gdb_baseline/armv7a-cros-linux-gnueabi
ejagnu/gdb_dejagnu.py
est_gdb_dejagnu.py
0285cf0721e47c7e608dd090c459e327ce3e2679 28-Dec-2013 Yunlian Jiang <yunlian@google.com> update gdb dejagnu test baseline

BUG=None
TEST=Manual.

Change-Id: I7e1b0553d383cda695ba917c4410ce8d99b8a8d6
Reviewed-on: https://chrome-internal-review.googlesource.com/150956
Reviewed-by: Han Shen <shenhan@google.com>
Commit-Queue: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
ejagnu/gdb_baseline/armv7a-cros-linux-gnueabi
ejagnu/gdb_baseline/i686-pc-linux-gnu
ejagnu/gdb_dejagnu.py
dffb0a9cedf0b6f12bef52a2db69524728877bd4 20-Dec-2013 Yunlian Jiang <yunlian@google.com> add gdb dejagnu test adaptor

BUG=None
TEST=Manually tested.

Change-Id: I644f5a62bc69bce8af74d5429948b3ab7253c419
Reviewed-on: https://chrome-internal-review.googlesource.com/150704
Reviewed-by: Han Shen <shenhan@google.com>
Commit-Queue: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
ejagnu/gdb_dejagnu.py
est_gdb_dejagnu.py
384d5f8f3eeb04d5063607c691b3732f0489934f 19-Dec-2013 cmtice <cmtice@google.com> Change to use test_that by default instead of run_remote_tests.sh.

BUG=None
TEST=Tested this with BootPerServer in my directory. Checked the
logs to make sure the correct script was being used (test_that by
default now).

Change-Id: I39b2901c2ef3c8070c76b15b929df5da6b168c9c
Reviewed-on: https://chrome-internal-review.googlesource.com/150695
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/benchmark.py
rosperf/settings_factory.py
468ef85183276640890b1def1060eb23d732c601 17-Dec-2013 Yunlian Jiang <yunlian@google.com> update gdb dejagnu baseline.

BUG=None
TEST=New baseline updated..

Change-Id: Id8665afe9dfd9e2075ad4a965538f51f65ccc477
Reviewed-on: https://chrome-internal-review.googlesource.com/150412
Reviewed-by: Han Shen <shenhan@google.com>
Commit-Queue: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
ejagnu/gdb_baseline/armv7a-cros-linux-gnueabi
ejagnu/gdb_baseline/i686-pc-linux-gnu
ejagnu/gdb_baseline/x86_64-cros-linux-gnu
e4b6f220f3cc7d3277daca7bca25f89d648dae8e 22-Nov-2013 Han Shen <shenhan@google.com> Add a script to delete nightly test data that are old.

TEST=tested on tc-build.hot

Change-Id: I6b6bc298b56c6ac79cbbd66f8168bc234580c884
Reviewed-on: https://chrome-internal-review.googlesource.com/150264
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Han Shen <shenhan@google.com>
Tested-by: Han Shen <shenhan@google.com>
uto_delete_nightly_test_data.py
uild_tc.py
ejagnu/run_dejagnu.py
tils/constants.py
tils/misc.py
af5f70ecd0f07c76eb83eeb3e15d4bb168f9e8b1 11-Dec-2013 cmtice <cmtice@google.com> Fix bug with Telemetry results names.

Some Telemetry tests return results that have the same 'description'
name, but different 'graph' names. We were only using the description
names, so we were losing some Telemetry measurements in our reports.
This CL fixes that.

BUG=None
TEST=Ran this on with smoothness.top_25. Was missing some results
before this change; after, the results are all there.

Change-Id: Ibcbe31da5f2860c494cbc8364b0a720d1c50169c
Reviewed-on: https://chrome-internal-review.googlesource.com/150215
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/results_cache.py
fb27ee972816c8c2fddf2540792131fad2a25d0d 06-Dec-2013 cmtice <cmtice@google.com> Add Telemetry default results file and script to manipulate it.

Add a json file containing the default results to be used for generating
Telemetry benchmark reports (to cut down on all the unnecessary results
being show. Also add a script that allows people to examine and
manipulate the default results file.

BUG=None
TEST=Used the script to create and manipulate the defaults file myself.
Also used the defaults file, in my private Crosperf checkout, for generating
Crosperf Telemetry reports. Ran 'cros lint' on script also.

Change-Id: I280af3414796bf9acd283e3d56d8783186e1a0ad
Reviewed-on: https://chrome-internal-review.googlesource.com/149331
Reviewed-by: Luis Lozano <llozano@chromium.org>
Tested-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
rosperf/default-telemetry-results.json
pdate-telemetry-defaults.py
9847df92a2b5f76ccddc4bf10288819712a8ca47 07-Dec-2013 cmtice <cmtice@google.com> Use default results for Telemetry tests; add option to use all results.

When running telemetry_Crosperf, use the json file of report defaults to
determine which results to actually include in the report. This CL
also adds a flag to Crospref, --show_all_results=True, that allows the
user to override the defaults and generate a report showing all the values.

BUG=None
TEST=Tested this both with and without the --show_all_results flag. It
works as expected for me.

Change-Id: I91364572ebdf3d54bb8b408fbd5a2580923f6829
Reviewed-on: https://chrome-internal-review.googlesource.com/149457
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/benchmark.py
rosperf/benchmark_run.py
rosperf/experiment_factory.py
rosperf/results_cache.py
rosperf/results_cache_unittest.py
rosperf/settings_factory.py
d80a960b62d89bc6f83d2f5fd4d599408f2ec7f9 05-Dec-2013 cmtice <cmtice@google.com> Check for file before prepending directory path.

Sometimes _GetDataMeasurementsFiles returns the file with the full
directory path, and sometimes it doesn't. We were assuming it always
did not. This patch checks to see if we can already find the file
before attempting to prepend the directory path.

BUG=None
TEST=Tested this in the nightly infrastructure tests area, where we
were seeing the problem. This fixed that problem. (Before this fix
we were not seeing the successful test results; after it we are.)

Change-Id: Ic0a2df4c9aff7b820f652f7ecad56976ed97a4b0
Reviewed-on: https://chrome-internal-review.googlesource.com/149256
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/results_cache.py
59fc85c29e0e4183da5d3656d8cd1701c5646265 03-Dec-2013 cmtice <cmtice@google.com> Make weekly sheriff email point to new web page.

BUG=None
TEST=None

Change-Id: I20eb69c42da4b96909c09931c8c7f4fda1b57abd
Reviewed-on: https://chrome-internal-review.googlesource.com/149000
Reviewed-by: Han Shen <shenhan@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
heriff_rotation.py
e5a746fb9ba7ddb8827f73f030bf99f798ffcf9e 25-Nov-2013 cmtice <cmtice@google.com> Add the ability to pass arguments through test_that to Telemetry's
run_benchmark.

BUG=None
TEST=I have tested this with my own crosperf, and test_that is passing the
arguments along properly. The telemetry_Crosperf autotest also needs to
be updated before this completely works.

Change-Id: Ic887380b297d966229ac21ba4e7e909e7248fcb0
Reviewed-on: https://chrome-internal-review.googlesource.com/148485
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/benchmark_run.py
rosperf/results_cache.py
rosperf/suite_runner.py
870c1847e55058db7e7cd0631f8b3c02e10dfd40 27-Nov-2013 cmtice <cmtice@google.com> Update machine_manager.py to deal with new 'ifconfig' output.

BUG=None
TEST=Tested in on updated lab parrot machines and it worked.

Change-Id: I709989a977130bbeeb667e825acdc874c6bfaf50
Reviewed-on: https://chrome-internal-review.googlesource.com/148714
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Bhaskar Janakiraman <bjanakiraman@google.com>
rosperf/machine_manager.py
5f3ccbddaaf5f4a64431c9e68faf83a348ecdfdd 21-Nov-2013 cmtice <cmtice@google.com> Update report generation to check for a 'perf_measurements' file.

The latest version of telemetry_Crosperf generates a perf_measurements
file, containing the test results, rather than a keyval file. This CL
checks for perf_measurements, and reads it, if it is there. This
change is backwards compatible with the older versions of telemetry_Crosperf
that used the keyval file.

BUG=None
TEST=Tested this with the results only in a keyval file and only in a
perf_measurements file. Got acceptable test reports either way.

Change-Id: If10a62b2f7fbe89d35c1ebe789a81764c9d134e4
Reviewed-on: https://chrome-internal-review.googlesource.com/148295
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/label.py
rosperf/results_cache.py
rosperf/suite_runner.py
57995561e1b6e98efcc8659b27a775a713a8e02b 13-Nov-2013 cmtice <cmtice@google.com> Add example experiment files for Telemetry & test_that.

Change-Id: I7bbf5e3816bafe7fa8edb39c7a77c0c9b527fc82
Reviewed-on: https://chrome-internal-review.googlesource.com/147866
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/experiment_files/telemetry-crosperf-suites.exp
rosperf/experiment_files/telemetry-crosperf-with-external-chrome-src.exp
rosperf/experiment_files/telemetry-crosperf-with-profiler.exp
rosperf/experiment_files/telemetry-crosperf.exp
rosperf/experiment_files/telemetry-pure.exp
rosperf/experiment_files/test_that.exp
0440388677b2445f4f8273b6c2b4d82601b1aca3 04-Nov-2013 cmtice <cmtice@google.com> Make changes to run Telemetry tests in nightly infrastructure tests.

BUG=None
TEST=Tested this running the nightly infrastructure cron jobs.

Change-Id: I6cccee222b074e9410b45df6750c097430207746
Reviewed-on: https://chrome-internal-review.googlesource.com/147515
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/experiment_factory.py
rosperf/label.py
etup_chromeos.py
est_toolchains.py
28ed8f65f751c000c676ea67bdb2b0e20f73038a 05-Nov-2013 cmtice <cmtice@google.com> Add file to bypass most of the commit queue tests.

Since changes to toolchain-utils will not affect them anyway, bypass
the commit queue tests, so if they fail for some other reason it won't
prevent toolchain-utils tests from being committed.

This is according to the instructions on:

www.chromium.org/developers/tree-sheriffs/sheriff-details-chromium-os/commit-queue-overview/ bypassing-tests-on-a-per-project-basis

BUG=None
Test=None

Change-Id: I17ae4591639f9113da9e85354ce81e90eba4aa4b
Reviewed-on: https://chrome-internal-review.googlesource.com/147525
Reviewed-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
OMMIT-QUEUE.ini
a1d03083c46175d9a34ea3665a3b1e305607442b 23-Oct-2013 cmtice <cmtice@google.com> Accept hyphens in test names; fix profiler args string.

Updated regular expression parser to accept hyphens in the test names.
Also fixed the quotes on profiler args, for proper parsing by
test_that.

BUG=None
TEST=Ran crosperf successfully with benchmark name that had hyphen in it;
successfully ran crosperf with profiler args.

Change-Id: I4a154e390a5fabfc47e0783212fed928eb5e9873
Reviewed-on: https://chrome-internal-review.googlesource.com/146967
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/experiment_file.py
rosperf/suite_runner.py
9c956f58876917af6cf5c6961241ddeaf1baf826 23-Oct-2013 Luis Lozano <llozano@chromium.org> Fixed name of telemetry benchmark page_cycler.basic_insert

BUG=None
TEST=Tested by hand using the new name

Change-Id: I7246f9f5d2a5e17133613bc4f9038fe1b42332e9
Reviewed-on: https://chrome-internal-review.googlesource.com/146909
Reviewed-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
rosperf/experiment_factory.py
53c88e9dd49958e2b20db2e06074198c8c3768da 09-Oct-2013 Luis Lozano <llozano@chromium.org> Pin max and min frequencies on machine before running experiment.

Find out what is the highest static frequency (turbo mode is
considered a dynamic frequency) and set the min and max frequency
to the highest static. This reduces benchmark noise in particular
for some machines like daisy and lumpy i5.

BUG=None
TEST=Tested with runs of perf_v2. Verified frequencies are pinned.

Change-Id: Idd0627a0f53fa986d0ccf1da2bee53d51a5c8ccb
Reviewed-on: https://chrome-internal-review.googlesource.com/146206
Reviewed-by: Simon Que <sque@google.com>
Tested-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Luis Lozano <llozano@chromium.org>
rosperf/suite_runner.py
1bb58716d226099d69a7545b5680e6bbdbcb2123 08-Oct-2013 Luis Lozano <llozano@chromium.org> Added the test name patterns for Telemetry perf v2 benchmarks direction

Added test name patterns to find the direction (improvement/degradation)
of benchmarks for all the perf_v2 except memory.top25 (which is broken
right now).
This way of detecting benchmark direction is pretty bridle. Telemetry
should provide this information explicitly.

Change-Id: Ifdccec1d9955f9b7a5790a1fa5ebabe98669496d
Reviewed-on: https://chrome-internal-review.googlesource.com/146216
Reviewed-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
tils/tabulator.py
f1dfb3f6b207c41616774b00426cc02d307138d0 08-Oct-2013 Yunlian Jiang <yunlian@google.com> dejagnu: change gdb baseline for gdb 7.6.1

BUG=None
TEST=None

Change-Id: I6ce965f9de03edf6147f9eb4791e31b4762dc720
Reviewed-on: https://chrome-internal-review.googlesource.com/146219
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
ejagnu/gdb_baseline/armv7a-cros-linux-gnueabi
ejagnu/gdb_baseline/i686-pc-linux-gnu
ejagnu/gdb_baseline/x86_64-cros-linux-gnu
70fceb7f11f176a0de829514b99c5907107b76d2 04-Oct-2013 Yunlian Jiang <yunlian@google.com> crosperf: remove cache if --rerun=True is set.

If rerun=True, the data in the cache is useless, so remove it first
to avoid data corrupture.

BUG=None
TEST=The cache dir is removed.

Change-Id: I59b219c57cc99aaea7a734ef6e7fd91007cedfd3
Reviewed-on: https://chrome-internal-review.googlesource.com/146065
Reviewed-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
rosperf/results_cache.py
ae02af5938d3fe75180dea5a6e8fa7a075b302d7 03-Oct-2013 Yunlian Jiang <yunlian@google.com> gdb_dejagnu: print failed tests out.

This prints failed test name out. Also, it ignores
the tests that only appears in the baseline.

BUG=None
TEST=the name of failed tests were printed.

Change-Id: Ie0c3f3d3a092a184a2e941020af62d4e2de40da2
Reviewed-on: https://chrome-internal-review.googlesource.com/145848
Reviewed-by: Simon Que <sque@google.com>
Reviewed-by: Han Shen <shenhan@google.com>
Commit-Queue: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
ejagnu/gdb_dejagnu.py
c35909e0e48898e81a70796d03ba0d2802bc209e 03-Oct-2013 Caroline Tice <cmtice@chromium.org> Remove (at least temporarily) the --minilayout argument for checking out ChromeOS, so the nightly infrastructure tests can build.

BUG=None
TEST=None

Change-Id: I7a6c68142e0c9af106a047986a95a545ece6a094
Reviewed-on: https://chrome-internal-review.googlesource.com/145917
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
est_toolchains.py
1b5a65e686812e3cab1d0b83dd7c9f59da1cf238 03-Oct-2013 Yunlian Jiang <yunlian@google.com> gdb_dejagnu: update base line.

BUG=None
TEST=None

Change-Id: I11c0c2ed6af9bd7a395dede27668046af008942c
Reviewed-on: https://chrome-internal-review.googlesource.com/145937
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
ejagnu/gdb_baseline/armv7a-cros-linux-gnueabi
ejagnu/gdb_baseline/i686-pc-linux-gnu
ejagnu/gdb_baseline/x86_64-cros-linux-gnu
1232a252863585ffd0b65764720e14452e89d3e2 02-Oct-2013 Yunlian Jiang <yunlian@google.com> gdb_dejagnu: add space to ssh parameter.

BUG=None
TEST=It passes.

Change-Id: If7413a602b58fce3b0f9472d45ca4e1a35501551
Reviewed-on: https://chrome-internal-review.googlesource.com/145847
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
ejagnu/chromeos.exp.in
ejagnu/gdb_dejagnu.py
de98d63638b36e9d2cfd264d82492438f8641fbb 02-Oct-2013 Yunlian Jiang <yunlian@google.com> gdb_dejiagnu: use machine.name instead of machine.

BUG=None
TEST=It can run now.

Change-Id: I53b4596c46341b600097f7cd4154b6945e970492
Reviewed-on: https://chrome-internal-review.googlesource.com/145895
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
ejagnu/gdb_dejagnu.py
f8b3a5ada4ae7c60d459fc843b1809acd7726613 25-Sep-2013 Caroline Tice <cmtice@chromium.org> Allow "." in benchmark names; Allow multiple test sets.

This fixes a few minor problems with crosperf:

1). When using run_remote_tests.sh, it adds the "--use_emerged" flag,
which is a recent requirement.
2). It updates the regular expression parsing to not mess up escaped
characters, and to allow "." in benchmark names (common in Telemetry).
3). It allows there to be multiple "sets" of benchmarks that can be
specified together, e.g. perfv2 or pagecyclers (or more in the future).

BUG=None
TEST=I've run all the various telemetry_Crosperf variations (individual
perf_v2 tests; individual telemetry page cycler tests; 'all_perf v2', and
'all_pagecyclers'.

Change-Id: Idd45caa6487780b92b8eb321b37b873978f96dca
Reviewed-on: https://chrome-internal-review.googlesource.com/145305
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
rosperf/experiment_factory.py
rosperf/experiment_file.py
rosperf/suite_runner.py
9e5333321a43b6f663d1b12dbad1d5bb3a52dfec 01-Oct-2013 Yunlian Jiang <yunlian@google.com> crosperf: fix unittest for crosperf.

This fixes several issues for unittesting.
Now unittest seems to be working.

BUG=None
TEST=crosperf/run_tests.sh passes.

Change-Id: I1dcef7ba55d45fbd00e453f749a8b10dea0332e6
Reviewed-on: https://chrome-internal-review.googlesource.com/145635
Reviewed-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
rosperf/benchmark_run.py
rosperf/benchmark_run_unittest.py
rosperf/label.py
rosperf/mock_instance.py
rosperf/results_cache_unittest.py
rosperf/results_organizer_unittest.py
0126ef363015c2dd3a257fff79df0bde615f37f1 28-Sep-2013 Aaron Gable <agable@google.com> Rewrite git/gerrit/gerrit-int urls to googlesource

BUG=chromium:281665,chromium:290413

TEST=None

Change-Id: I0770e6dc963afaaf012d4925ac1b9bbcc17d01ea
Reviewed-on: https://chrome-internal-review.googlesource.com/145692
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Aaron Gable <agable@google.com>
Tested-by: Aaron Gable <agable@google.com>
epo_to_repo_files/binutils-master.rtr
epo_to_repo_files/binutils-mobile_toolchain_v16.rtr
epo_to_repo_files/crosperf.rtr
epo_to_repo_files/gcc-branches_google_4_7.rtr
epo_to_repo_files/gcc-branches_google_main.rtr
epo_to_repo_files/gcc-master.rtr
epo_to_repo_files/gdb-master.rtr
epo_to_repo_files/toolchain-utils.rtr
92774196e083e04b954d02f8d463998d549269eb 11-Sep-2013 Caroline Tice <cmtice@chromium.org> Update crosperf to run telemetry test via autotest.

This updates crosperf to recognize the suite 'telemetry_Crosperf',
which allows users to run telemetry performance tests via test_that
and the telemetry_Crosperf autotest. This won't be usable in general until
CL 168491 (the telemetry_Crosperf CL) gets committed.

BUG=None
TEST=Invoked telemetry tests using this change.

Note: Profilers don't quite work properly with telemetry_Crosperf yet,
but everything else does.

Change-Id: Id14edc89db79bc244496e8f6a5b614300a00be54
Reviewed-on: https://chrome-internal-review.googlesource.com/144440
Reviewed-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/experiment_factory.py
rosperf/suite_runner.py
34d759e2a60ad7ce7da18031b9a7d9f31d658ef6 10-Sep-2013 Yunlian Jiang <yunlian@google.com> gdb_dejagnu: Use TryAcquireMachine from run_dejagnu.

This fixes some gpylint warnings and uses TryAcquireMachine
from run_dejagnu.

BUG=None
TEST=gdb_dejagnu passes.

Change-Id: I1e7230b00619daec9cea28ca2926e18fb8ac8345
Reviewed-on: https://chrome-internal-review.googlesource.com/144426
Reviewed-by: Han Shen <shenhan@google.com>
Commit-Queue: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
ejagnu/gdb_dejagnu.py
6aae4c9adaa197a3b5314e654e2ea6b0275cae66 10-Sep-2013 Luis Lozano <llozano@chromium.org> For the new git repositories, we need to use https instead of http and ssh.

Bug=None
Test=Tested setup_chromeos, build_chromes, remote_gcc_build

Change-Id: I95833ed07692ea04f396233ab87aa7b47ff6b106
Reviewed-on: https://chrome-internal-review.googlesource.com/144397
Reviewed-by: Han Shen <shenhan@google.com>
Commit-Queue: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
etup_chromeos.py
est_gcc_dejagnu.py
tils/manifest_versions.py
tils/misc.py
ccb0a8f9d8904da6d077ba0753e675941bc6aaf1 09-Sep-2013 Yunlian Jiang <yunlian@google.com> update daisy gdb baseline

BUG=None
TEST=None

Change-Id: Icefcffcc40057954bf7b51c3921248c43ef2b810
Reviewed-on: https://chrome-internal-review.googlesource.com/144366
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Yunlian Jiang <yunlian@google.com>
ejagnu/gdb_baseline/armv7a-cros-linux-gnueabi
6aebeb10e517878cb76044e6a1d46f75c2fd93d6 03-Sep-2013 Luis Lozano <llozano@chromium.org> Modified the script to use the new internal and external gerrit repositories.

BUG=None
TEST=Minimal testing done. Need to let the nightly testing test this.

Change-Id: I8273d5e90e5fb3ed96d9782cae651d4144f135dc
Reviewed-on: https://chrome-internal-review.googlesource.com/144015
Reviewed-by: Han Shen <shenhan@google.com>
Tested-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Luis Lozano <llozano@chromium.org>
emote_gcc_build.py
etup_chromeos.py
est_gcc_dejagnu.py
tils/manifest_versions.py
tils/misc.py
d145a5873cb01ceab07ea4b3fa325d7560f8ad6a 19-Aug-2013 Yunlian Jiang <yunlian@google.com> add vanilla_image option to build_chromeos.py

This adds vanilla_iamge option to build_chromeos.py.
With this option, it runs build_packages with --usepkg.
In the test_toolchains.py, it uses this option when
building vanilla image.

BUG=None
TEST=None

Change-Id: I54f85e60ccae833ab6682476e19365a51df5ef29
Reviewed-on: https://gerrit-int.chromium.org/43088
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
uild_chromeos.py
est_toolchains.py
b47bff4d3336c5fe5593a95963c0f3dc20a02f68 20-Aug-2013 Caroline Tice <cmtice@chromium.org> Make test_that available for running tests.

Add a new flag, --use_test_that=True/False, to crosperf, to allow users to
tell crosperf to use the test_that script rather than the run_remote_tests.sh
script for running their tests.

BUG=None
TEST=Tested with and without the flag, to verify that it does the right thing.

Change-Id: I1a040d23adbf364b4a47320d9fd759a2d46e6689
Reviewed-on: https://gerrit-int.chromium.org/43109
Reviewed-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/benchmark.py
rosperf/benchmark_run.py
rosperf/experiment_factory.py
rosperf/results_cache.py
rosperf/settings_factory.py
rosperf/suite_runner.py
b0a02f4e9c3fde3198223a9fe70ca85ceaa58105 22-Aug-2013 Yunlian Jiang <yunlian@google.com> toolchian_utils: add --accept_licenses=@CHROMEOS flag.

This adds --accept_licenses=@CHROMEOS to setup_board and
build_packages. With this patch, we can build daisy from
role account.

BUG=None
TEST=role account can build daisy now.

Change-Id: Ib08bf44d3355f667154f78f07e67d2aeb6d97a31
Reviewed-on: https://gerrit-int.chromium.org/43330
Reviewed-by: Caroline Tice <cmtice@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
tils/misc.py
d79c0b511ab9deffe01722ad66718418e54da692 19-Aug-2013 Luis Lozano <llozano@chromium.org> Added new daisy machines in the lab.

BUG=None
TEST=None

Change-Id: Id88cee787d55dff8f7fb60e1de68347a26a8d56f
Reviewed-on: https://gerrit-int.chromium.org/43076
Reviewed-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
rosperf/default_remotes
ea3479446a4ee8c7a1f017f7594cc37f56dc12e6 17-Aug-2013 Yuheng Long <yuhenglong@google.com> Add the omnetpp example.

BUG=None
TEST=unit testings for the pipeline stage, pipeline workers, generation,
steering, task, flag, hill climbing and genetic algorithm.

Change-Id: I69f10c79e87db1ce58c14a4c827ea5f5dd71e4bf
Reviewed-on: https://gerrit-int.chromium.org/43047
Commit-Queue: Luis Lozano <llozano@chromium.org>
Reviewed-by: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
estflags/bikjmp/README
estflags/bikjmp/bikjmp_bench.cpp
estflags/bikjmp/build_bikjmp
estflags/bikjmp/conf
estflags/bikjmp/example.json
estflags/bikjmp/fen.txt
estflags/bikjmp/test_bikjmp
estflags/examples/omnetpp/README
estflags/examples/omnetpp/build_omnetpp
estflags/examples/omnetpp/conf
estflags/examples/omnetpp/example.json
estflags/examples/omnetpp/test_omnetpp
cd968c8905aac77b348f6a1efc5d1f3e32f13563 19-Aug-2013 Luis Lozano <llozano@chromium.org> Fixed typo in last commit

BUG=None
TESTS=Did several succesful remote builds

Change-Id: Icff4f20ec44c962f52cd4cec85c3e2ab70eeb4a8
Reviewed-on: https://gerrit-int.chromium.org/43077
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
emote_gcc_build.py
7b8b2d11d0ac87ea3a9ecad976b2214e1c680bc5 14-Aug-2013 Yuheng Long <yuhenglong@google.com> Add the negative flag elimination algorithm.

BUG=None
TEST=unit testings for the pipeline stage, pipeline workers, generation,
steering, task, flag, hill climbing, genetic algorithm and iterative flag
elimination algorithm.

Change-Id: I2d400e304570de8a15556529eff823d40c36b185
Reviewed-on: https://gerrit-int.chromium.org/42798
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Yuheng Long <yuhenglong@google.com>
Tested-by: Yuheng Long <yuhenglong@google.com>
estflags/iterative_elimination.py
estflags/testing_batch.py
ed75d963c18c879b215af329888ded0b82bf0ae9 16-Aug-2013 Yuheng Long <yuhenglong@google.com> Add an example that flows through the framework.

BUG=None
TEST=unit testings for the pipeline stage, pipeline workers, generation,
steering, task, flag, hill climbing and genetic algorithm.

Change-Id: I997770204feda543a3424f7f69cf2ee7f45fccc2
Reviewed-on: https://gerrit-int.chromium.org/43004
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Yuheng Long <yuhenglong@google.com>
Tested-by: Yuheng Long <yuhenglong@google.com>
estflags/bikjmp/README
estflags/bikjmp/bikjmp_bench.cpp
estflags/bikjmp/build_bikjmp
estflags/bikjmp/conf
estflags/bikjmp/example.json
estflags/bikjmp/fen.txt
estflags/bikjmp/test_bikjmp
estflags/example_algorithms.py
ce860ea3fc65c488b9ea5e93efbf521dae9dc7dd 14-Aug-2013 Yuheng Long <yuhenglong@google.com> Fixed the problems in task and steering.

BUG=None
TEST=unit testings for the pipeline stage, pipeline workers, generation,
steering, task, flag, hill climbing and genetic algorithm.
Change-Id: Ib34ef05518124b6ffa03f0c35795109e22581b89
Reviewed-on: https://gerrit-int.chromium.org/43002
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Yuheng Long <yuhenglong@google.com>
Tested-by: Yuheng Long <yuhenglong@google.com>
estflags/pipeline_process.py
estflags/pipeline_process_test.py
estflags/task.py
estflags/testing_batch.py
25cdf79e17a73858ffa28db8a5b387210fea9e25 14-Aug-2013 Yuheng Long <yuhenglong@google.com> Initiate all the class fields in the init method.

BUG=None
TEST=unit testings for the pipeline stage, pipeline workers, generation,
steering, task, flag and hill climbing.

Change-Id: I170f4c3a719e0b6aedfdbd90aa29803c011c9ba0
Reviewed-on: https://gerrit-int.chromium.org/42835
Reviewed-by: Simon Que <sque@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
Tested-by: Yuheng Long <yuhenglong@google.com>
Commit-Queue: Yuheng Long <yuhenglong@google.com>
estflags/hill_climb_best_neighbor.py
estflags/mock_task.py
estflags/task.py
d5a0ef17c7a611ff0a8c19e1103ef3c73dc4e195 14-Aug-2013 Yuheng Long <yuhenglong@google.com> Add the content to the README.

BUG=None
TEST=unit testings for the pipeline stage, pipeline workers, generation,
steering, task, flag and hill climbing.

Change-Id: I49ac28745fc8bdf1a04f26fb468c420e8d2c7b9c
Reviewed-on: https://gerrit-int.chromium.org/42833
Reviewed-by: Simon Que <sque@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
Tested-by: Yuheng Long <yuhenglong@google.com>
Commit-Queue: Yuheng Long <yuhenglong@google.com>
estflags/README
e896dfd76014af3c399d1b54be022fb1663a105b 14-Aug-2013 Yuheng Long <yuhenglong@google.com> Refining the documentations.

BUG=None
TEST=unit testings for the pipeline stage, pipeline workers, generation,
steering, task, flag, hill climbing and genetic algorithm.

Change-Id: I45c83c9dc74e54b75492ceb281e6912eb347d75e
Reviewed-on: https://gerrit-int.chromium.org/42802
Reviewed-by: Simon Que <sque@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
Tested-by: Yuheng Long <yuhenglong@google.com>
Commit-Queue: Yuheng Long <yuhenglong@google.com>
estflags/flags.py
estflags/flags_test.py
estflags/flags_util.py
estflags/genetic_algorithm.py
estflags/testing_batch.py
4f10d39a499ad872c22402aa763037fc3351f399 15-Aug-2013 Caroline Tice <cmtice@chromium.org> Fix results processing to handle "JSON.stringify" in Telemetry results.

This is the same as the previous CL; I had to abandon the previous CL
because it erroneously depended on a different abandoned CL, so it refused
to commit.

BUG=none
TEST=I ran the telemetry tests with the modification and now ALL of them
give good output (before only about half of them did).

Change-Id: I567f51176c522bc1c41676f6dc560f3827b9cef4
Reviewed-on: https://gerrit-int.chromium.org/42930
Reviewed-by: Bhaskar Janakiraman <bjanakiraman@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/results_cache.py
c01232264caccaffe6302414b7e901a96d3b18bf 11-Aug-2013 Yuheng Long <yuhenglong@google.com> Refine the module hill_climb_best_neighbor.

BUG=None
TEST=unit testings for the pipeline stage, pipeline workers, generation,
steering, task, flag and hill climbing.

Change-Id: I95fd82072f1474b73735a643f2da589eb930b838
Reviewed-on: https://gerrit-int.chromium.org/42663
Reviewed-by: Luis Lozano <llozano@chromium.org>
Tested-by: Yuheng Long <yuhenglong@google.com>
Commit-Queue: Yuheng Long <yuhenglong@google.com>
estflags/generation.py
estflags/genetic_algorithm.py
estflags/hill_climb_best_neighbor.py
estflags/steering.py
estflags/steering_test.py
estflags/task.py
2b514c28dd26aa02c2ea0f9924be91fb2394c8a0 09-Aug-2013 Yuheng Long <yuhenglong@google.com> Add more examples for better documentation.

BUG=None
TEST=unit testings for the pipeline stage, pipeline workers, generation,
steering, task, flag and hill climbing.

Change-Id: I1f8f361388d3c3171a7135d1d0ef38ee5f695829
Reviewed-on: https://gerrit-int.chromium.org/42584
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Yuheng Long <yuhenglong@google.com>
Tested-by: Yuheng Long <yuhenglong@google.com>
estflags/flags.py
estflags/flags_util.py
estflags/generation.py
estflags/genetic_algorithm.py
estflags/hill_climb_best_neighbor.py
estflags/steering.py
estflags/steering_test.py
estflags/task.py
bbbd8645405a7058b8e6d17dc1fdec14cbff4f08 09-Aug-2013 Luis Lozano <llozano@chromium.org> Added new parrot machines in the lab

Change-Id: I93fdcf4ac064ef1a89289f88ee7d6b6360b28103
Reviewed-on: https://gerrit-int.chromium.org/42622
Reviewed-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
rosperf/default_remotes
057ad5c77ef0ea44ee260718eb6d30afef2f7f83 07-Aug-2013 Yuheng Long <yuhenglong@google.com> Add the Genetic Algorithm.

BUG=None
TEST=unit testings for the pipeline stage, pipeline workers, generation,
steering, task, flag, hill climbing and genetic algorithm.

Change-Id: I2864d6a6859fff43bc2d3afb059c672c54bbe385
Reviewed-on: https://gerrit-int.chromium.org/42472
Reviewed-by: Simon Que <sque@google.com>
Commit-Queue: Yuheng Long <yuhenglong@google.com>
Tested-by: Yuheng Long <yuhenglong@google.com>
estflags/generation.py
estflags/genetic_algorithm.py
estflags/testing_batch.py
570b2ce931d40a335f3b02ed859b376040b5fb67 07-Aug-2013 Caroline Tice <cmtice@chromium.org> Initialize machine_id_checksum. Make unlocking unlocked machine a warning.

Fix issues causing errors in nightly tester: Attempting to use
machine_id_checksum when it hasn't been initialized; and attempting to
unlock an already unlocked machine causes a fatal error.

Change-Id: Ia149953d4e25c7abf97a189bb5fdd129de865c34
Reviewed-on: https://gerrit-int.chromium.org/42489
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/results_cache.py
ock_machine.py
c1fb0f1051208907df137371f3d100132f19cf10 07-Aug-2013 Yuheng Long <yuhenglong@google.com> Add underscore to separate words in variables.

BUG=None
TEST=unit testings for the pipeline stage, pipeline workers, generation,
steering, task, flag and hill climbing.

Change-Id: I0efdf5e7ad85b42f37519b6e0cebea8352bbf563
Reviewed-on: https://gerrit-int.chromium.org/42430
Reviewed-by: Simon Que <sque@google.com>
Tested-by: Yuheng Long <yuhenglong@google.com>
Commit-Queue: Yuheng Long <yuhenglong@google.com>
estflags/generation_test.py
estflags/pipeline_process_test.py
f1f606e1c5b559aa264518a89595afd787f12e8a 06-Aug-2013 Yuheng Long <yuhenglong@google.com> Encapsulate a couple inner fields in a unittest.

BUG=None
TEST=unit testings for the pipeline stage, pipeline workers, generation,
steering, task, flag and hill climbing.

Change-Id: I5d86c107cb7a0693c97a0cb404c2a89a172ce835
Reviewed-on: https://gerrit-int.chromium.org/42418
Reviewed-by: Simon Que <sque@google.com>
Tested-by: Yuheng Long <yuhenglong@google.com>
Commit-Queue: Yuheng Long <yuhenglong@google.com>
estflags/pipeline_worker_test.py
ccfaf2f382815945c1c883e1cfceb2eba55e28b0 02-Aug-2013 Yuheng Long <yuhenglong@google.com> Add the best branching hill climbing algorithm.

BUG=None
TEST=unit testings for the pipeline stage, pipeline workers, generation,
steering, task, flag and hill climbing.

Change-Id: Ifd181f45c4b82f5fb77b0d4946757954aa806d33
Reviewed-on: https://gerrit-int.chromium.org/42284
Tested-by: Yuheng Long <yuhenglong@google.com>
Reviewed-by: Simon Que <sque@google.com>
Commit-Queue: Yuheng Long <yuhenglong@google.com>
estflags/flags.py
estflags/flags_util.py
estflags/generation.py
estflags/hill_climb_best_neighbor.py
estflags/pipeline_process_test.py
estflags/task.py
estflags/task_test.py
estflags/testing_batch.py
6aa8528c03697a1f28186c574d512c19935676a5 07-Aug-2013 Caroline Tice <cmtice@chromium.org> Fix syntax error in suite_runner.py, to allow crosperf to run "pure"
Telemetry tests.

Change-Id: Ie6073c05fdc50cf22294816b6be37811652b6827
Reviewed-on: https://gerrit-int.chromium.org/42444
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Caroline Tice <cmtice@google.com>
Tested-by: Caroline Tice <cmtice@google.com>
rosperf/suite_runner.py
424bd3846b8c60beb6aa8e1a4f44b7b8288bd2a6 06-Aug-2013 Yunlian Jiang <yunlian@google.com> gdb_dejagnu: update base line for x86_64.

BUG=None
TEST=gdb_dejagnu passes.

Change-Id: Ifdca26c7b6dbe49e496f757c5229a1091263d770
Reviewed-on: https://gerrit-int.chromium.org/42346
Commit-Queue: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
ejagnu/gdb_baseline/x86_64-cros-linux-gnu
ejagnu/gdb_dejagnu.py
d67dcb71687152d17dff6b84a395f87e243f8875 30-Jul-2013 Yuheng Long <yuhenglong@google.com> Add the flag module.

BUG=None
TEST=unit testings for the pipeline stage, pipeline workers, generation,
steering, task and flag.

Change-Id: Ia1b32f859da2fc0984b940393b688a44221f3611
Reviewed-on: https://gerrit-int.chromium.org/41847
Reviewed-by: Simon Que <sque@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Yuheng Long <yuhenglong@google.com>
Tested-by: Yuheng Long <yuhenglong@google.com>
estflags/flags.py
estflags/flags_test.py
ffd98bbabbe8f7287d79e7ca111418e794fb5880 02-Aug-2013 Yunlian Jiang <yunlian@google.com> crosperf: fix the label option chrome_src

This fixes the error when chrome_src is given to label.

BUG=None
TEST=run a telemetry test with chrome_src is specified.

Change-Id: I5f97c731bec3bcb42c1172490d765f8921d30dc0
Reviewed-on: https://gerrit-int.chromium.org/42197
Reviewed-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
rosperf/label.py
8cf53086b1ebb5632f8f52b1644d83938c9c8cf1 01-Aug-2013 Luis Lozano <llozano@chromium.org> Changes to allow getting images from the remote bot when the testing fails

- Allow remote_gcc_build to return image even in the case where the build
bot testing has failed. Testing may have failed but an image may have been
generated.
- Move the handling of the manifest repo to a new module.
- Various clean up

BUG=None
TEST=Tested by running with several inputs. Used for vtable verification testing

Change-Id: I1abf01c56d1159dfae76c45e2de4c2a9e385d386
Reviewed-on: https://gerrit-int.chromium.org/42123
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
emote_gcc_build.py
etup_chromeos.py
tils/manifest_versions.py
b15d41c6d6324e343fbea19abaed3717417f3cde 25-Jul-2013 Yuheng Long <yuhenglong@google.com> Add the task module.

BUG=None
TEST=unit testings for the pipeline stage, pipeline workers, generation,
steering and task.

Change-Id: I38987a12d7a48ec027d42465300a7226c760ea9d
Reviewed-on: https://gerrit-int.chromium.org/41659
Reviewed-by: Simon Que <sque@google.com>
Commit-Queue: Yuheng Long <yuhenglong@google.com>
Tested-by: Yuheng Long <yuhenglong@google.com>
estflags/mock_task.py
estflags/steering.py
estflags/task.py
estflags/task_test.py
a5712a2c71aa665dcca808963d152228890c8364 22-Jul-2013 Yuheng Long <yuhenglong@google.com> Add the steering stage of the framework.

BUG=None
TEST=unit testings for the pipeline stage, pipeline workers, generation and
steering.

Change-Id: Id92bcf04ee24dfbc918f59ac8d87d30ee69e47b3
Reviewed-on: https://gerrit-int.chromium.org/41454
Reviewed-by: Simon Que <sque@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Yuheng Long <yuhenglong@google.com>
Tested-by: Yuheng Long <yuhenglong@google.com>
estflags/generation.py
estflags/generation_test.py
estflags/mock_task.py
estflags/steering.py
estflags/steering_test.py
fefa5c0174b32eb359eca91acebab772356a4473 03-Jul-2013 Yunlian Jiang <yunlian@google.com> crosperf: use run_measurement to run telemetry.

Use run_measurement instead of run_multipage_benchmark,
since run_multipage_benchmark is depreciated.

BUG=chromium:256865
TEST=None

Change-Id: I37c86b9d3ece40ff4980d53b262b657ae9e1ae0d
Reviewed-on: https://gerrit-int.chromium.org/40549
Commit-Queue: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
rosperf/suite_runner.py
16d7a5204e347855b1c3a68c982c22f931a12866 20-Jul-2013 Yuheng Long <yuhenglong@google.com> Added the copyright information.

BUG=None
TEST=unit testing for the pipeline stage, pipeline workers and generation.

Change-Id: I1a4eb5e2759ea536fa6faee613199b8f679f586e
Reviewed-on: https://gerrit-int.chromium.org/41357
Reviewed-by: Simon Que <sque@google.com>
Tested-by: Yuheng Long <yuhenglong@google.com>
Commit-Queue: Yuheng Long <yuhenglong@google.com>
estflags/generation.py
estflags/generation_test.py
estflags/mock_task.py
estflags/pipeline_process.py
estflags/pipeline_process_test.py
estflags/pipeline_worker.py
estflags/pipeline_worker_test.py
estflags/steering.py
estflags/steering_test.py
estflags/task.py
estflags/task_test.py
8b9c0f140b48253cdbcc7c050f115c5e3bda6d88 16-Jul-2013 Yuheng Long <yuhenglong@google.com> Added the base Generation base class.

Algorithm specific subclasses, e.g., Genetic Algorithm and Hill Climbing,
should extend this base class.

BUG=None
TEST=unit testing for the pipeline stage, pipeline workers and generation.

Change-Id: Icc79b02099521db74fed7c1017691e2a8719be23
Reviewed-on: https://gerrit-int.chromium.org/41081
Reviewed-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Simon Que <sque@google.com>
Tested-by: Yuheng Long <yuhenglong@google.com>
Commit-Queue: Yuheng Long <yuhenglong@google.com>
estflags/generation.py
estflags/generation_test.py
a791546e80cede30d5325bec834b35b99b7e7bfe 15-Jul-2013 Yuheng Long <yuhenglong@google.com> Extract out a common mock task into a module.

The mock tasks are used in multiple unitest modules. It is not extracted out
into a module to enable reuse.

BUG=None
TEST=unit testing for the pipeline stage and pipeline workers.

Change-Id: Iee436266f7ea26c2877377f52208cc5cd31e7815
Reviewed-on: https://gerrit-int.chromium.org/41001
Reviewed-by: Simon Que <sque@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Yuheng Long <yuhenglong@google.com>
Tested-by: Yuheng Long <yuhenglong@google.com>
estflags/mock_task.py
estflags/pipeline_process.py
estflags/pipeline_process_test.py
estflags/pipeline_worker_test.py
26ec76c8a9d4f5dd023513d2772fa6cd4b6749ea 11-Jul-2013 Yuheng Long <yuhenglong@google.com> Rename the methods and use the dummy parameters.

Refactor methods names and substitute unused variables with dummy variables.

BUG=None
TEST=unit testing for the pipeline stage and pipeline workers.

Change-Id: I237d893201853d8620ff886c8fb9012c38b83805
Reviewed-on: https://gerrit-int.chromium.org/40865
Commit-Queue: Yuheng Long <yuhenglong@google.com>
Tested-by: Yuheng Long <yuhenglong@google.com>
Reviewed-by: Simon Que <sque@google.com>
estflags/generation.py
estflags/pipeline_process.py
estflags/pipeline_process_test.py
estflags/pipeline_worker.py
estflags/pipeline_worker_test.py
estflags/steering.py
estflags/task.py
49358b75c25a44760e884245440dc96e55812d04 10-Jul-2013 Yuheng Long <yuhenglong@google.com> Add the project information to each file's header comment.

BUG=None
TEST=None

Change-Id: I362bfbf18c42025a0248714aa5d69ecfaa1daa2b
Reviewed-on: https://gerrit-int.chromium.org/40826
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Yuheng Long <yuhenglong@google.com>
Tested-by: Yuheng Long <yuhenglong@google.com>
estflags/generation.py
estflags/generation_test.py
estflags/pipeline_process.py
estflags/pipeline_process_test.py
estflags/pipeline_worker.py
estflags/pipeline_worker_test.py
estflags/steering.py
estflags/steering_test.py
estflags/task.py
estflags/task_test.py
761748dc64a0d2a6a90714097445fcf782a8934e 28-Jun-2013 Yuheng Long <yuhenglong@google.com> Merge the test stage and the build stage.

Use the pipeline_worker to represent both stage. These two stages differ on
getting the key of the task in the stage and setting the value of this stage.
The key uniquely distinguish a task from another task. For example, the key
for the build phase is the optimization flags, while the key for the test stage
is the checksum of the built image.

BUG=None
TEST=None

Change-Id: Id057c17026806acd406fe78cc70ed996273ca9aa
Reviewed-on: https://gerrit-int.chromium.org/40376
Reviewed-by: Simon Que <sque@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Yuheng Long <yuhenglong@google.com>
Tested-by: Yuheng Long <yuhenglong@google.com>
estflags/builder.py
estflags/builder_test.py
estflags/executor.py
estflags/executor_test.py
estflags/pipeline_worker.py
estflags/pipeline_worker_test.py
2dd2e9e80ad235793e1591d21a16d23b2b5243d7 01-Jul-2013 Yunlian Jiang <yunlian@google.com> crosperf: filter out "/' in remote

TEST=nightly test passes.
BUG=None

Change-Id: I525e488c9f4c47cfe09bd8c15d06997b9e75f140
Reviewed-on: https://gerrit-int.chromium.org/40463
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Yunlian Jiang <yunlian@google.com>
rosperf/experiment_factory.py
1dd70cd5565f1800b5a2133922d84022cf619ddd 19-Jun-2013 Yuheng Long <yuhenglong@google.com> Added the logic and the test for the builder of the framework.

BUG=None
TEST=None

Change-Id: Ibf65d94c042b39f340ccaae7a9aaa16d390c8d50
Reviewed-on: https://gerrit-int.chromium.org/39864
Reviewed-by: Simon Que <sque@google.com>
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Yuheng Long <yuhenglong@google.com>
Tested-by: Yuheng Long <yuhenglong@google.com>
estflags/builder.py
estflags/builder_test.py
estflags/pipeline_process.py
5fe6dc81886f49eb2089c883c6862f2e896155cf 19-Jun-2013 Yuheng Long <yuhenglong@google.com> Fixed the warnings from pylint.

BUG=None
TEST=None

Change-Id: I1b2b5df85eacb1b30557d108bf982054ea5e3cea
Reviewed-on: https://gerrit-int.chromium.org/39812
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Yuheng Long <yuhenglong@google.com>
Tested-by: Yuheng Long <yuhenglong@google.com>
estflags/pipeline_process.py
e610c1904b8fbdb4c14c67dede25aafc02167259 12-Jun-2013 Yuheng Long <yuhenglong@google.com> Have the pipeline process working. Added the unit test for this class.

BUG=None
TEST=None

Change-Id: I7fe9fd5b1610959399000b1dfc9b6db55c5c28fb
Reviewed-on: https://gerrit-int.chromium.org/39473
Reviewed-by: Luis Lozano <llozano@chromium.org>
Tested-by: Yuheng Long <yuhenglong@google.com>
Commit-Queue: Yuheng Long <yuhenglong@google.com>
estflags/pipeline_process.py
estflags/pipeline_process_test.py
f20cffac082e3d920818f230ffc80ae6976267c0 04-Jun-2013 Yuheng Long <yuhenglong@google.com> Added the skeleton for the flagging framework.

BUG=None
TEST=None

Change-Id: I72c37ac70ed2adca588ad9866a6bcc26775aed8b
Reviewed-on: https://gerrit-int.chromium.org/39096
Reviewed-by: Luis Lozano <llozano@chromium.org>
Tested-by: Yuheng Long <yuhenglong@google.com>
Commit-Queue: Yuheng Long <yuhenglong@google.com>
estflags/builder.py
estflags/builder_test.py
estflags/executor.py
estflags/executor_test.py
estflags/generation.py
estflags/generation_test.py
estflags/pipeline_process.py
estflags/pipeline_process_test.py
estflags/steering.py
estflags/steering_test.py
estflags/task.py
estflags/task_test.py
8cdaddf7ec91520a0bfbdf9da73056f255f67824 31-May-2013 Yuheng Long <yuhenglong@google.com> testing

BUG=None
TESTS=None

Change-Id: Ie81057d3035358dae0cf198c21d93cda781c5bad
Reviewed-on: https://gerrit-int.chromium.org/39005
Reviewed-by: Luis Lozano <llozano@chromium.org>
Tested-by: Yuheng Long <yuhenglong@google.com>
Commit-Queue: Yuheng Long <yuhenglong@google.com>
estflags/README
04dc5dc8547dbfbe524cf35ac39537346ad749bb 24-Apr-2013 Yunlian Jiang <yunlian@google.com> Adding support of telemetry to crosperf

BUG=None
TEST=run crosperf with pyauto/telemetry test with/without cache.
all pass.

Change-Id: If07ac020a9107a79d5780a58fd6dcc924d07f07f
Reviewed-on: https://gerrit-int.chromium.org/36594
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Reviewed-on: https://gerrit-int.chromium.org/39241
Reviewed-by: Yunlian Jiang <yunlian@google.com>
rosperf/autotest_runner.py
rosperf/benchmark.py
rosperf/benchmark_run.py
rosperf/benchmark_run_unittest.py
rosperf/experiment_factory.py
rosperf/experiment_factory_unittest.py
rosperf/label.py
rosperf/machine_manager.py
rosperf/results_cache.py
rosperf/results_organizer.py
rosperf/results_report.py
rosperf/settings_factory.py
rosperf/suite_runner.py
tils/tabulator.py
9fc991900056e20bb940eed243dad0c0516d497b 30-May-2013 Yunlian Jiang <yunlian@google.com> crosperf: use mac address as machine id if VPD is not available.

For zgb machine, the dump_vpd_log --full --stdout does not return
serial number, in this case, we use mac address instead.

BUG=None
TEST=It get machine id for zgb

Change-Id: I1058dcd2a18638533c90a40d09adcc7a6bd8a660
Reviewed-on: https://gerrit-int.chromium.org/38900
Reviewed-by: Luis Lozano <llozano@chromium.org>
Tested-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Yunlian Jiang <yunlian@google.com>
rosperf/machine_manager.py
f00adba7fe33c50b1522e6c784de4c4ecd4bf611 23-May-2013 Yunlian Jiang <yunlian@google.com> remote_gcc_build: fix branch setting error.

This fixes the error when a version is given, the branch
it empty, it invokes the cbuildbot in a wrong way.

BUG=None
TEST=./remote_gcc_build.py -t lumpy-release -c ~/clang -p 51046
-d ~/local_tmp -r --chromeos_version R29-4165.0.0 passes.

Change-Id: I75e24dc1c820583ecb0a10183d9bc86bdec50fbb
Reviewed-on: https://gerrit-int.chromium.org/38595
Commit-Queue: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
emote_gcc_build.py
e5b673f1b459a0927ca65bf9e6fd504ece57e39d 23-May-2013 Yunlian Jiang <yunlian@google.com> crosperf: remove dead machine in the runtime.

This checks whether a machine is reacheable after a test run.
If the machine is dead, we remove it from the machine list in the
machine manager.

BUG=None
TEST=crosperf a simple case passes.

Change-Id: Ie40a4a41715f2dc0349666079faa599b3c8aa667
Reviewed-on: https://gerrit-int.chromium.org/38578
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
rosperf/benchmark_run.py
rosperf/machine_manager.py
837e07a6a2f455a4699bf4ad1b28653fde3ca3b0 23-May-2013 Yunlian Jiang <yunlian@google.com> crosperf: keep alive if one machine is down.

Originally, crosperf dies if any of machine in the machine list is
unreachable, this patch just ignore the unreachable machine.

BUG=None
TEST=crosperf a simple testcase passes.

Change-Id: I835b484f5378215da142d5e8e64471ce744aea30
Reviewed-on: https://gerrit-int.chromium.org/38475
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
rosperf/machine_manager.py
4578ec10ee49f6341cc840c5ea41b1899af4caff 22-May-2013 Yunlian Jiang <yunlian@google.com> crosperf: add results_cache_unittest

BUG=None
TEST=this unittest passes.

Change-Id: I65fee19efd30b1602a6ad49b84abe7366a6e605c
Reviewed-on: https://gerrit-int.chromium.org/38429
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Yunlian Jiang <yunlian@google.com>
rosperf/results_cache_unittest.py
1131359ab8fc43c015f4a72c5aec128bcb85b0da 21-May-2013 Yunlian Jiang <yunlian@google.com> crosperf: add results_organizer_unittest

BUG=None
TEST=Unittest passes.

Change-Id: I1fda414c1379af40cd1efc13461ed34aafe11847
Reviewed-on: https://gerrit-int.chromium.org/38326
Reviewed-by: Luis Lozano <llozano@chromium.org>
Tested-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Yunlian Jiang <yunlian@google.com>
rosperf/mock_instance.py
rosperf/results_organizer_unittest.py
8cad5d6b898579aae593b1fcf53675882a014b11 17-May-2013 Yunlian Jiang <yunlian@google.com> crosperf: import time to autotest_runner.py

BUG=None
TEST=run a small crosperf test and it passes.

Change-Id: I9c4ee6487ef7e164d78a30821db0dcb76c6ccbc7
Reviewed-on: https://gerrit-int.chromium.org/38133
Reviewed-by: Luis Lozano <llozano@chromium.org>
Tested-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Yunlian Jiang <yunlian@google.com>
rosperf/autotest_runner.py
13a9fc381d3a364b00af9704f4592722ac7f9a1b 07-May-2013 Yunlian Jiang <yunlian@google.com> Add gdb test base line.

This enables a comparison of gdb result with base line.
(The base line is the dejagnu result with current compiler).

BUG=None
TEST=None

Change-Id: I091e2dd0beaf6de231cb616e54b2ccf3e600b47a
Reviewed-on: https://gerrit-int.chromium.org/37477
Reviewed-by: Han Shen <shenhan@google.com>
Commit-Queue: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
ejagnu/gdb_baseline/armv7a-cros-linux-gnueabi
ejagnu/gdb_baseline/i686-pc-linux-gnu
ejagnu/gdb_baseline/x86_64-cros-linux-gnu
ejagnu/gdb_dejagnu.py
0a89243bbbb904c84fd0c7acce2d71e1abb7471c 17-May-2013 Yunlian Jiang <yunlian@google.com> crosperf: reboot the machine before running benchmarks.

BUG=None
TEST=None

Change-Id: I15bcba679664ec7a3d0b9eaea845c9dc2c0d9189
Reviewed-on: https://gerrit-int.chromium.org/38080
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
rosperf/autotest_runner.py
134905c3aa9858160601423457b0c3f1381b3227 04-May-2013 Ahmad Sharif <asharif@chromium.org> Do not use a pseudo-terminal in command_executer.

Using a psuedo-terminal causes problems when running jobs through cron. Gentoo's
sandbox aborts builds due to a write violation to /dev/ptmx. Longer term
Gentoo's sandbox may need to add /dev/ptmx to the whitelist.

BUG=none
TEST=python no_pseudo_terminal_test.py passes.

Change-Id: I2ac00718a4694f097fcec8bcf2cc95d48bdd9e51
Reviewed-on: https://gerrit-int.chromium.org/37323
Reviewed-by: Bhaskar Janakiraman <bjanakiraman@google.com>
Commit-Queue: asharif <asharif@google.com>
Tested-by: asharif <asharif@google.com>
tils/command_executer.py
tils/no_pseudo_terminal_test.py
ba640e53d51c5d041f0152df1c3bec38fad79494 03-May-2013 Yunlian Jiang <yunlian@google.com> kill gdbserver on device before start gdb test.

BUG=None
TEST=the process of gdbserver leader-exit.exp is gone.

Change-Id: Ifd7d47a7064275a16e1faf7faf03e6c1d281ff59
Reviewed-on: https://gerrit-int.chromium.org/37310
Commit-Queue: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
ejagnu/boards/gdbserver.sh.in
c85b4c359dceeeab0177122448fd4810e5d7a62b 03-May-2013 Yunlian Jiang <yunlian@google.com> fix minor issues in gdb_dejagnu.py

TEST=test passes.
BUG=None

Change-Id: If9e4d996eaec32d25769ec50f1f03e0b3aac42ab
Reviewed-on: https://gerrit-int.chromium.org/37296
Commit-Queue: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
ejagnu/gdb_dejagnu.py
f5533e3b5edce07360f6362569f51c726e7ff838 16-Apr-2013 Yunlian Jiang <yunlian@google.com> Add gdb_dejiagnu test script

This is the basic dejagnu test script. Currently it just generate
the board information, compile gdb/gdbserver and run make check.
It needs baseline and some other features later.

BUG=None
TEST=./gdb_dejagnu.py -c /extend/gamma/ -b lumpy -r 172.17.128.241
It finishes the dejagnu run.

Change-Id: I81678926eda640bb5637e6e4dc540a7b9801b000
Reviewed-on: https://gerrit-int.chromium.org/35703
Reviewed-by: Han Shen <shenhan@google.com>
Commit-Queue: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
ejagnu/boards/gdb.exp.in
ejagnu/boards/gdbserver.sh.in
ejagnu/gdb_dejagnu.py
6845e5f094d67d45a6908e2cfdfed25476b8e343 10-Apr-2013 Yunlian Jiang <yunlian@google.com> Remove the extra -g in command line if no patch

BUG=None
TEST=The script works without patch

Change-Id: I9ee5308ff889699b4b206417ba105db65f0cfb62
Reviewed-on: https://gerrit-int.chromium.org/35440
Reviewed-by: Han Shen <shenhan@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Yunlian Jiang <yunlian@google.com>
emote_gcc_build.py
cc4ef90ba831e8506dafcd2abcd7f9c93201cf9e 04-Apr-2013 Yunlian Jiang <yunlian@google.com> Remove the branch prefix to make it work

BUG=None
TEST=run remote_gcc_build.py with master branch and a source gcc dir.
it works.

Change-Id: I8fb7a91b5725749cc13db860fbcaf0f0fde2ae79
Reviewed-on: https://gerrit-int.chromium.org/35073
Reviewed-by: Han Shen <shenhan@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Yunlian Jiang <yunlian@google.com>
emote_gcc_build.py
36f91adc5055984332cf467c1526e4f80d828fa7 29-Mar-2013 Yunlian Jiang <yunlian@google.com> Reboot machine if imaging failed

BUG=None
TEST=run crosperf, machine rebooted as expected

Change-Id: I0ffb94d9cc96858d91fb5f8c277c91c8c836b0cf
Reviewed-on: https://gerrit-int.chromium.org/34651
Reviewed-by: Han Shen <shenhan@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Yunlian Jiang <yunlian@google.com>
rosperf/machine_manager.py
5b0ad87ba68391dca762341f708f63c383f90e4a 04-Apr-2013 Yunlian Jiang <yunlian@google.com> Change default value of branch, fix a branch name.

BUG=None
TEST=None
Change-Id: I7b4966580493544f1410f2c754e7a76cbfd04635
Reviewed-on: https://gerrit-int.chromium.org/35018
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
emote_gcc_build.py
646ae43ef2983d7ace40fe33c46244bed3c255de 03-Apr-2013 Luis Lozano <llozano@chromium.org> Fixed a couple of minor issues in the handling of "patch" and "branch" options

BUG=None
TEST=Verified the options work as expected

Change-Id: Ib3e41c6499d5102d8f44c16bafb1b1eafc8184f1
Reviewed-on: https://gerrit-int.chromium.org/34996
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
emote_gcc_build.py
2c4b2a1fb608ec840126e693faecb390e95356b2 03-Apr-2013 Yunlian Jiang <yunlian@google.com> Handle the case where the branch is not there yet.

BUG=None
TEST=None

Change-Id: I3e7e92d049d6587b0695e8b3f2a525eb063dbc5f
Reviewed-on: https://gerrit-int.chromium.org/35005
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
emote_gcc_build.py
914453283a27a89838730b037cd380e318279d5b 20-Mar-2013 Han Shen <shenhan@google.com> Added a new script to do chroot bootstrapping.

This script is used to bootstrap-build the host compiler and then all
host packages, including all toolchain for all boards.

Bootstrapping the chroot is a mandatory step before we roll out any
compiler changes. Currently we have no script to formalize this process.

The basic idea of this script is to populate a user-provided gcc tree
onto a local branch in chromeos/src/third_party/gcc git and then
modify the active ebuild file to pick up this specific gcc tree.

The usage of this script -

Usage: bootstrap_compiler.py [options]

Options:
-h, --help show this help message and exit
-c CHROMEOS_ROOT, --chromeos_root=CHROMEOS_ROOT
ChromeOs root dir.
-b BRANCH, --branch=BRANCH
The branch to test against. This branch must be a
local branch inside "src/third_party/gcc". Notice,
this must not be used with "--gcc".
-g GCC_DIR, --gcc_dir=GCC_DIR
Use a local gcc tree to do bootstrapping. Notice, this
must not be used with "--branch".
--fixperm Fix the (notorious) permission error while trying to
bootstrap the chroot. Note this takes an extra 10-15
minutes and is only needed once per chromiumos tree.
--setup_gcc_ebuild_file_only
Setup gcc ebuild file to pick up the branch (--branch)
or user gcc source (--gcc_dir) and exit. Keep chroot
as is.
--reset_gcc_ebuild_file
Reset the modification that is done by this
script.Note, when this script is running, it will
modify the active gcc ebuild file. Use this option to
reset (what this script has done) and exit.

Change-Id: Iab07d60c3ccb2e1b8feda9df0fb13d5474a50cbe
Reviewed-on: https://gerrit-int.chromium.org/34165
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Han Shen <shenhan@google.com>
Tested-by: Han Shen <shenhan@google.com>
ootstrap_compiler.py
tils/misc.py
00cc30ea5fb3a05530a3b1ec699ca7f9696ffbd0 28-Mar-2013 Yunlian Jiang <yunlian@google.com> Add option to specify output dir of crosperf

BUG=None
TEST=run crosperf with this option, the results are in the right dir

Change-Id: Id94251c72297f18f275564a1fd1ef0972227ddfc
Reviewed-on: https://gerrit-int.chromium.org/34621
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
rosperf/experiment.py
rosperf/experiment_factory.py
rosperf/settings_factory.py
006a5b0cd4ed149e7774ef2d3a29a73d0cc3cee8 28-Mar-2013 Yunlian Jiang <yunlian@google.com> Add exe permission to groups for the cache dir.

So others can open the cache dir

BUG=None
TEST=run crosperf, the permission of the cache dir shown as drwx--x---

Change-Id: Iadc5519679f2820f047e17c10429205636e65f5b
Reviewed-on: https://gerrit-int.chromium.org/34614
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
rosperf/results_cache.py
b8b76e2cf876caf2de12e06951c7ab80333a1799 25-Mar-2013 Yunlian Jiang <yunlian@google.com> Remove unrelated /tmp from crosperf report.

This bug happens when we move the invocation of generate_test_report
into chroot. The source dir did not change accordingly, so it cannot
remove the /tmp stuff anymore.

BUG=None
TEST=The /tmp/... stuff is removed on both cache and non cached case.

Change-Id: I29dac68eecf0ff1d265f2a3f004627c30962e228
Reviewed-on: https://gerrit-int.chromium.org/34407
Reviewed-by: asharif <asharif@google.com>
Commit-Queue: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
rosperf/results_cache.py
9d1db65f60b01e8e33ec995cceb5069e440ebdd8 25-Mar-2013 Luis Lozano <llozano@chromium.org> Fixed small issue in replacing sysroot functionality.
Need to consider that the board name may contain "-"

BUG=None
TEST=Run the script for several boards and verified the copy was happening correctly.

Change-Id: Ic7e34548d65962da11a4ca1cb575fdfb6769feed
Reviewed-on: https://gerrit-int.chromium.org/34400
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
emote_gcc_build.py
4cea9de295fcb982a6d92fc424f3b37781d4d1c5 20-Mar-2013 Luis Lozano <llozano@chromium.org> Copy gcc libs directory to sysroot after build.

Tools like crosperf look for the gcc libraries under the sysroot.
Also fixed a mino issue when using --gcc_dir and other *_dir options
when the directory given does not exist.

BUG=None
TEST=Build toolchain and verified the libraries are copied to the
correct place.
Verified that the *_dir options check for existance of dir given

Change-Id: I61599db2843d5bdbbc479164a8f05c722f9f7e75
Reviewed-on: https://gerrit-int.chromium.org/34153
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Reviewed-by: Caroline Tice <cmtice@google.com>
Commit-Queue: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
uild_tc.py
c7f1593f9af3ea1b9264b37628c36f3a70e1749a 20-Mar-2013 Mike Frysinger <vapier@chromium.org> change shebang to /usr/bin/python

BUG=chromium-os:206038
TEST=None

Change-Id: I34c09e14d13993ff2b9e6ce17dd7e16fd01f962c
Reviewed-on: https://gerrit-int.chromium.org/34133
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Mike Frysinger <vapier@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
utomation/clients/android.py
utomation/clients/chromeos.py
utomation/clients/crosstool.py
utomation/clients/dejagnu_compiler.py
utomation/clients/helper/android.py
utomation/clients/helper/chromeos.py
utomation/clients/helper/crosstool.py
utomation/clients/helper/jobs.py
utomation/clients/helper/perforce.py
utomation/clients/nightly.py
utomation/clients/output_test.py
utomation/clients/pwd_test.py
utomation/clients/report/dejagnu/main.py
utomation/clients/report/dejagnu/manifest.py
utomation/clients/report/dejagnu/report.py
utomation/clients/report/dejagnu/summary.py
utomation/common/command.py
utomation/common/command_executer.py
utomation/common/command_executer_test.py
utomation/common/events.py
utomation/common/job.py
utomation/common/job_group.py
utomation/common/logger.py
utomation/common/machine.py
utomation/common/machine_test.py
utomation/common/state_machine.py
utomation/server/job_executer.py
utomation/server/job_group_manager.py
utomation/server/job_manager.py
utomation/server/machine_manager.py
utomation/server/machine_manager_test.py
utomation/server/monitor/dashboard.py
utomation/server/monitor/manage.py
utomation/server/monitor/settings.py
utomation/server/monitor/urls.py
utomation/server/server.py
utomation/server/server_test.py
uild_benchmarks.py
ommand_executer_timeout_test.py
ompare_benchmarks.py
rb/crb_driver.py
ros_login.py
wp/bartlett/server.py
wp/bartlett/test/server_tester.py
do_scripts/divide_and_merge_profiles.py
do_scripts/divide_and_merge_profiles_test.py
do_scripts/profile_cycler.py
do_scripts/summarize_hot_blocks.py
do_scripts/vanilla_vs_fdo.py
roduce_output.py
emote_kill_test.py
emote_test.py
eport_generator.py
un_benchmarks.py
un_tests.py
heriff_rotation.py
ummarize_results.py
c_enter_chroot.py
tils/constants.py
tils/html_tools.py
tils/logger.py
tils/timeline.py
eb77c44ba613280fe4514d512737477b4830d63e 18-Mar-2013 Han Shen <shenhan@google.com> Refactor repo_to_repo.py a little bit.

I'm writing a script to do compiler bootstrap, which needs to sync a
user provided gcc directory with chromiumos git repository and submit
all that changes to chromiumos git locally.

The modification includes -
1. added a new repo type - FileRepo, which is literally no-op but set
"_root_dir" to its directory
2. outlined the submit code from PushSources, so that this submit
routine could be called from outside

I also tested that this worked even if the input FileRepo is read-only.

Tested using a test rtr file as below -
{
"input": [{
"type": "file",
"address": "/usr/local/google/home/shenhan/temp2",
},],
"output": [{
"type": "git",
"address": "file:///usr/local/google/home/shenhan/temp",
"ignores": [".svn", ".git"],
},],
}

TEST=run repo_to_repo.py with gcc-4_7-mobile profile.

Change-Id: Ib73be0b4fa8612428801b10a17abdcdc1a2822e3
Reviewed-on: https://gerrit-int.chromium.org/33990
Reviewed-by: asharif <asharif@google.com>
Commit-Queue: Han Shen <shenhan@google.com>
Tested-by: Han Shen <shenhan@google.com>
epo_to_repo.py
3e017d00c7c0501b6eacace8f4f39037d9618278 18-Mar-2013 Yunlian Jiang <yunlian@google.com> Add one row Total_events_of_this_profile in perf report.

BUG=None
TEST=It shows the number in the report.

Change-Id: I5f28346552b9a96f8d3475880258618dc8e0a99a
Reviewed-on: https://gerrit-int.chromium.org/33974
Reviewed-by: Luis Lozano <llozano@chromium.org>
Commit-Queue: Yunlian Jiang <yunlian@google.com>
Tested-by: Yunlian Jiang <yunlian@google.com>
tils/perf_diff.py
f81680c018729fd4499e1e200d04b48c4b90127c 15-Mar-2013 Luis Lozano <llozano@chromium.org> Cleaned up directory after copy of tools from perforce directory

Got rid of stale copies of some tools like "crosperf" and moved all
files under v14 directory (that came from perforce) into the top
directory.

BUG=None
TEST=None

Change-Id: I408d17a36ceb00e74db71403d2351fd466a14f8e
Reviewed-on: https://gerrit-int.chromium.org/33887
Tested-by: Luis Lozano <llozano@chromium.org>
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Luis Lozano <llozano@chromium.org>
gitignore
utomation/PRESUBMIT.py
utomation/__init__.py
utomation/all_tests.py
utomation/clients/__init__.py
utomation/clients/android.py
utomation/clients/chromeos.py
utomation/clients/crosstool.py
utomation/clients/dejagnu_compiler.py
utomation/clients/helper/__init__.py
utomation/clients/helper/android.py
utomation/clients/helper/chromeos.py
utomation/clients/helper/crosstool.py
utomation/clients/helper/jobs.py
utomation/clients/helper/perforce.py
utomation/clients/nightly.py
utomation/clients/output_test.py
utomation/clients/pwd_test.py
utomation/clients/report/dejagnu.sh
utomation/clients/report/dejagnu/__init__.py
utomation/clients/report/dejagnu/main.py
utomation/clients/report/dejagnu/manifest.py
utomation/clients/report/dejagnu/report.html
utomation/clients/report/dejagnu/report.py
utomation/clients/report/dejagnu/summary.py
utomation/clients/report/validate_failures.py
utomation/common/__init__.py
utomation/common/command.py
utomation/common/command_executer.py
utomation/common/command_executer_test.py
utomation/common/events.py
utomation/common/job.py
utomation/common/job_group.py
utomation/common/logger.py
utomation/common/machine.py
utomation/common/machine_test.py
utomation/common/state_machine.py
utomation/server/__init__.py
utomation/server/job_executer.py
utomation/server/job_group_manager.py
utomation/server/job_manager.py
utomation/server/machine_manager.py
utomation/server/machine_manager_test.py
utomation/server/monitor/__init__.py
utomation/server/monitor/dashboard.py
utomation/server/monitor/manage.py
utomation/server/monitor/settings.py
utomation/server/monitor/start.sh
utomation/server/monitor/static/style.css
utomation/server/monitor/templates/base.html
utomation/server/monitor/templates/job.html
utomation/server/monitor/templates/job_group.html
utomation/server/monitor/templates/job_group_list.html
utomation/server/monitor/templates/job_log.html
utomation/server/monitor/templates/machine_list.html
utomation/server/monitor/templates/snippet_attribute_table.html
utomation/server/monitor/templates/snippet_code.html
utomation/server/monitor/templates/snippet_links.html
utomation/server/monitor/urls.py
utomation/server/server.py
utomation/server/server_test.py
utomation/server/test_pool.csv
uild-binutils/opts.sh
uild-gcc/opts.sh
uild_benchmarks.py
uild_chrome_browser.py
uild_chromeos.py
uild_tc.py
ommand_executer_timeout_test.py
ompare_benchmarks.py
rb/autotest_gatherer.py
rb/autotest_run.py
rb/crb_driver.py
rb/machine_manager_singleton.py
rb/table_formatter.py
ros_login.py
rosperf/benchmark.py
rosperf/benchmark_run.py
rosperf/benchmark_run_unittest.py
rosperf/config.py
rosperf/crosperf.py
rosperf/experiment.py
rosperf/experiment_factory.py
rosperf/experiment_factory_unittest.py
rosperf/experiment_runner.py
rosperf/image_checksummer.py
rosperf/label.py
rosperf/machine_manager.py
rosperf/machine_manager_unittest.py
rosperf/perf_table.py
rosperf/results_cache.py
rosperf/results_organizer.py
rosperf/results_report.py
rosperf/settings_factory.py
wp/bartlett/app.yaml
wp/bartlett/server.py
wp/bartlett/static/favicon.ico
wp/bartlett/test/server_tester.py
wp/bartlett/update_appengine_server
wp/demo_pipeline.sh
wp/interpreter/app_engine_pull.py
wp/interpreter/symbolizer.py
wp/performance/experiment_gen.py
ejagnu/__init__.py
ejagnu/boards/chromeos-machine.exp
ejagnu/chromeos.exp.in
ejagnu/run_dejagnu.py
ejagnu/site.exp
do_scripts/divide_and_merge_profiles.py
do_scripts/divide_and_merge_profiles_test.py
do_scripts/profile_cycler.py
do_scripts/summarize_hot_blocks.py
do_scripts/vanilla_vs_fdo.py
et_common_image_version.py
mage_chromeos.py
ock_machine.py
ock_machine_test.py
roduce_output.py
yrun
emote_gcc_build.py
emote_kill_test.py
emote_test.py
epo_to_repo.py
epo_to_repo_files/binutils-master.rtr
epo_to_repo_files/binutils-mobile_toolchain_v16.rtr
epo_to_repo_files/crosperf.rtr
epo_to_repo_files/gcc-branches_google_4_7.rtr
epo_to_repo_files/gcc-branches_google_main.rtr
epo_to_repo_files/gcc-master.rtr
epo_to_repo_files/gdb-master.rtr
epo_to_repo_files/toolchain-utils.rtr
eport_generator.py
un_benchmarks.py
un_tests.py
etup_chromeos.py
heriff_rotation.py
ummarize_results.py
c_enter_chroot.py
est_gcc_dejagnu.py
est_toolchains.py
tils/buildbot_json.py
tils/command_executer.py
tils/command_executer_unittest.py
tils/logger.py
tils/misc.py
tils/perf_diff.py
tils/tabulator.py
tils/tabulator_test.py
14/.gitignore
14/README
14/automation/PRESUBMIT.py
14/automation/__init__.py
14/automation/all_tests.py
14/automation/clients/__init__.py
14/automation/clients/android.py
14/automation/clients/chromeos.py
14/automation/clients/crosstool.py
14/automation/clients/dejagnu_compiler.py
14/automation/clients/helper/__init__.py
14/automation/clients/helper/android.py
14/automation/clients/helper/chromeos.py
14/automation/clients/helper/crosstool.py
14/automation/clients/helper/jobs.py
14/automation/clients/helper/perforce.py
14/automation/clients/nightly.py
14/automation/clients/output_test.py
14/automation/clients/pwd_test.py
14/automation/clients/report/dejagnu.sh
14/automation/clients/report/dejagnu/__init__.py
14/automation/clients/report/dejagnu/main.py
14/automation/clients/report/dejagnu/manifest.py
14/automation/clients/report/dejagnu/report.html
14/automation/clients/report/dejagnu/report.py
14/automation/clients/report/dejagnu/summary.py
14/automation/clients/report/validate_failures.py
14/automation/common/__init__.py
14/automation/common/command.py
14/automation/common/command_executer.py
14/automation/common/command_executer_test.py
14/automation/common/events.py
14/automation/common/job.py
14/automation/common/job_group.py
14/automation/common/logger.py
14/automation/common/machine.py
14/automation/common/machine_test.py
14/automation/common/state_machine.py
14/automation/server/__init__.py
14/automation/server/job_executer.py
14/automation/server/job_group_manager.py
14/automation/server/job_manager.py
14/automation/server/machine_manager.py
14/automation/server/machine_manager_test.py
14/automation/server/monitor/__init__.py
14/automation/server/monitor/dashboard.py
14/automation/server/monitor/manage.py
14/automation/server/monitor/settings.py
14/automation/server/monitor/start.sh
14/automation/server/monitor/static/style.css
14/automation/server/monitor/templates/base.html
14/automation/server/monitor/templates/job.html
14/automation/server/monitor/templates/job_group.html
14/automation/server/monitor/templates/job_group_list.html
14/automation/server/monitor/templates/job_log.html
14/automation/server/monitor/templates/machine_list.html
14/automation/server/monitor/templates/snippet_attribute_table.html
14/automation/server/monitor/templates/snippet_code.html
14/automation/server/monitor/templates/snippet_links.html
14/automation/server/monitor/urls.py
14/automation/server/server.py
14/automation/server/server_test.py
14/automation/server/test_pool.csv
14/build-binutils/opts.sh
14/build-gcc/opts.sh
14/build_benchmarks.py
14/build_chrome_browser.py
14/build_chromeos.py
14/build_tc.py
14/command_executer_timeout_test.py
14/compare_benchmarks.py
14/crb/autotest_gatherer.py
14/crb/autotest_run.py
14/crb/crb_driver.py
14/crb/machine_manager_singleton.py
14/crb/table_formatter.py
14/cros_login.py
14/crosperf/autotest_runner.py
14/crosperf/benchmark.py
14/crosperf/benchmark_run.py
14/crosperf/benchmark_run_unittest.py
14/crosperf/column_chart.py
14/crosperf/config.py
14/crosperf/crosperf
14/crosperf/crosperf.py
14/crosperf/crosperf_test.py
14/crosperf/default_remotes
14/crosperf/experiment.py
14/crosperf/experiment_factory.py
14/crosperf/experiment_factory_unittest.py
14/crosperf/experiment_file.py
14/crosperf/experiment_file_unittest.py
14/crosperf/experiment_files/README
14/crosperf/experiment_files/aes_perf
14/crosperf/experiment_files/bloat_perf
14/crosperf/experiment_files/morejs_perf
14/crosperf/experiment_files/page_cycler
14/crosperf/experiment_files/page_cycler_perf
14/crosperf/experiment_files/toolchain
14/crosperf/experiment_runner.py
14/crosperf/experiment_status.py
14/crosperf/field.py
14/crosperf/help.py
14/crosperf/image_checksummer.py
14/crosperf/label.py
14/crosperf/machine_manager.py
14/crosperf/machine_manager_unittest.py
14/crosperf/perf_table.py
14/crosperf/results_cache.py
14/crosperf/results_organizer.py
14/crosperf/results_report.py
14/crosperf/results_sorter.py
14/crosperf/run_tests.sh
14/crosperf/settings.py
14/crosperf/settings_factory.py
14/crosperf/test_flag.py
14/cwp/bartlett/app.yaml
14/cwp/bartlett/server.py
14/cwp/bartlett/static/favicon.ico
14/cwp/bartlett/test/server_tester.py
14/cwp/bartlett/update_appengine_server
14/cwp/demo_pipeline.sh
14/cwp/interpreter/app_engine_pull.py
14/cwp/interpreter/symbolizer.py
14/cwp/performance/experiment_gen.py
14/dejagnu/__init__.py
14/dejagnu/boards/chromeos-machine.exp
14/dejagnu/chromeos.exp.in
14/dejagnu/run_dejagnu.py
14/dejagnu/site.exp
14/fdo_scripts/divide_and_merge_profiles.py
14/fdo_scripts/divide_and_merge_profiles_test.py
14/fdo_scripts/profile_cycler.py
14/fdo_scripts/summarize_hot_blocks.py
14/fdo_scripts/vanilla_vs_fdo.py
14/get_common_image_version.py
14/image_chromeos.py
14/lock_machine.py
14/lock_machine_test.py
14/produce_output.py
14/pyrun
14/remote_gcc_build.py
14/remote_kill_test.py
14/remote_test.py
14/repo_to_repo.py
14/repo_to_repo_files/binutils-master.rtr
14/repo_to_repo_files/binutils-mobile_toolchain_v16.rtr
14/repo_to_repo_files/crosperf.rtr
14/repo_to_repo_files/gcc-branches_google_4_7.rtr
14/repo_to_repo_files/gcc-branches_google_main.rtr
14/repo_to_repo_files/gcc-master.rtr
14/repo_to_repo_files/gdb-master.rtr
14/repo_to_repo_files/toolchain-utils.rtr
14/report_generator.py
14/run_benchmarks.py
14/run_tests.py
14/setup_chromeos.py
14/sheriff_rotation.py
14/summarize_results.py
14/tc_enter_chroot.py
14/test_gcc_dejagnu.py
14/test_toolchains.py
14/utils/__init__.py
14/utils/buildbot_json.py
14/utils/colortrans.py
14/utils/command_executer.py
14/utils/command_executer_unittest.py
14/utils/constants.py
14/utils/email_sender.py
14/utils/file_utils.py
14/utils/html_tools.py
14/utils/logger.py
14/utils/misc.py
14/utils/misc_test.py
14/utils/perf_diff.py
14/utils/pstat.py
14/utils/stats.py
14/utils/tabulator.py
14/utils/tabulator_test.py
14/utils/timeline.py
14/utils/timeline_test.py
2296ee0b914aba5bba07becab4ff68884ce9b8a5 15-Mar-2013 Luis Lozano <llozano@chromium.org> Removed duplicated lock_machine.py file.

BUG=none
TEST=none

Change-Id: I7e2592db8a49138aa54c5270dc4e171fee1ba410
Reviewed-on: https://gerrit-int.chromium.org/33885
Reviewed-by: Yunlian Jiang <yunlian@google.com>
Commit-Queue: Luis Lozano <llozano@chromium.org>
Tested-by: Luis Lozano <llozano@chromium.org>
ock_machine.py
b6d9405ac1a33d77962dbcfc1a6cbaf3133e56eb 15-Mar-2013 Luis Lozano <llozano@chromium.org> Put tools/gcctools/chromeos/v14 directory under toolchain-utils directory.
This commit includes the history of changes from perforce. You
can do a git log to see the whole history.
4467f004e7f0854963bec90daff1879fbd9d2fec 20-Dec-2012 Ahmad Sharif <asharif@chromium.org> Synced repos to: 64740
rosperf/benchmark.py
rosperf/benchmark_run.py
rosperf/benchmark_run_unittest.py
rosperf/crosperf
rosperf/crosperf.py
rosperf/default_remotes
rosperf/experiment.py
rosperf/experiment_factory.py
rosperf/experiment_factory_unittest.py
rosperf/experiment_file_unittest.py
rosperf/experiment_runner.py
rosperf/experiment_status.py
rosperf/label.py
rosperf/machine_manager.py
rosperf/machine_manager_unittest.py
rosperf/perf_table.py
rosperf/results_cache.py
rosperf/results_organizer.py
rosperf/results_report.py
rosperf/settings_factory.py
rosperf/test_flag.py
mage_chromeos.py
ock_machine.py
tils/command_executer.py
tils/logger.py
tils/misc.py
tils/perf_diff.py
tils/pstat.py
tils/tabulator.py
tils/tabulator_test.py
tils/timeline.py
tils/timeline_test.py
f395c26437cbdabc2960447fba89b226f4409e82 10-Oct-2012 Ahmad Sharif <asharif@chromium.org> Synced repos to: 63271
rosperf/autotest_runner.py
rosperf/benchmark.py
rosperf/benchmark_run.py
rosperf/experiment.py
rosperf/experiment_factory.py
rosperf/experiment_file.py
rosperf/experiment_files/aes_perf
rosperf/experiment_files/bloat_perf
rosperf/experiment_files/morejs_perf
rosperf/experiment_files/page_cycler_perf
rosperf/experiment_files/toolchain
rosperf/experiment_runner.py
rosperf/machine_manager.py
rosperf/results_cache.py
rosperf/results_columns.py
rosperf/results_organizer.py
rosperf/results_report.py
rosperf/results_sorter.py
rosperf/settings_factory.py
rosperf/table.py
mage_chromeos.py
tils/colortrans.py
tils/command_executer.py
tils/constants.py
tils/file_utils.py
tils/logger.py
tils/misc.py
tils/misc_test.py
tils/stats.py
tils/tabulator.py
tils/tabulator_test.py
a171f8d10c0507127de54c60076f1f59feef3629 25-May-2012 Ahmad Sharif <asharif@chromium.org> Synced repos to: 60736
rosperf/results_cache.py
f84bd3b7b64deba46192ae6d15a712ab66e90a6e 24-May-2012 Ahmad Sharif <asharif@chromium.org> Synced repos to: 60708
rosperf/benchmark_run.py
rosperf/benchmark_run_unittest.py
rosperf/experiment.py
rosperf/experiment_files/README
rosperf/experiment_files/aes_perf
rosperf/experiment_files/bloat_perf
rosperf/experiment_files/morejs_perf
rosperf/experiment_files/page_cycler
rosperf/experiment_files/page_cycler_perf
rosperf/experiment_files/toolchain
rosperf/results_cache.py
5cc48b578c136a3d9427de53284e4268e927a521 18-May-2012 Ahmad Sharif <asharif@chromium.org> Synced repos to: 60597
rosperf/results_cache.py
5ae8a5c99e73be7f388f74551d5219cb676f8e41 18-May-2012 Ahmad Sharif <asharif@chromium.org> Synced repos to: 60588
rosperf/autotest_runner.py
rosperf/benchmark_run.py
rosperf/experiment_runner.py
rosperf/perf_processor.py
rosperf/results_cache.py
rosperf/results_sorter.py
rosperf/settings_factory.py
tils/misc.py
ee988c87971c07b7e6767adb63d151ea37e363f2 15-May-2012 Ahmad Sharif <asharif@chromium.org> Synced repos to: 60457
rosperf/results_cache.py
rosperf/results_report.py
rosperf/settings_factory.py
fd356fb31c4340851fecc750230d05c9518b125c 07-May-2012 Ahmad Sharif <asharif@chromium.org> Synced repos to: 60208
rosperf/autotest_runner.py
rosperf/benchmark_run.py
rosperf/experiment.py
rosperf/experiment_runner.py
rosperf/perf_processor.py
rosperf/results_cache.py
mage_chromeos.py
tils/command_executer.py
tils/logger.py
tils/misc.py
tils/misc_test.py
tils/utils.py
92ab7af223c82fe8897bacea8b32028828eab018 02-Mar-2012 Ahmad Sharif <asharif@chromium.org> Synced repos to: 58839
rosperf/benchmark_run.py
rosperf/results_cache.py
tils/utils.py
822c55d2adc5f6fecb6bb4bc6faa94e022b8012a 09-Feb-2012 Ahmad Sharif <asharif@chromium.org> Synced repos to: 58313
rosperf/crosperf.py
rosperf/settings_factory.py
f1d70cb6e6bb944bfd376d4331ccf0054508a9e0 07-Feb-2012 Ahmad Sharif <asharif@chromium.org> Synced repos to: 58247
rosperf/machine_manager.py
0dcbc4b1714260820fd4b8d6536fbb05e139cc0f 03-Feb-2012 Ahmad Sharif <asharif@chromium.org> Synced repos to: 58208
EADME
rosperf/autotest_runner.py
rosperf/benchmark.py
rosperf/benchmark_run.py
rosperf/benchmark_run_unittest.py
rosperf/column_chart.py
rosperf/crosperf
rosperf/crosperf.py
rosperf/crosperf_test.py
rosperf/experiment.py
rosperf/experiment_factory.py
rosperf/experiment_factory_unittest.py
rosperf/experiment_file.py
rosperf/experiment_file_unittest.py
rosperf/experiment_runner.py
rosperf/experiment_status.py
rosperf/field.py
rosperf/help.py
rosperf/image_checksummer.py
rosperf/label.py
rosperf/machine_manager.py
rosperf/perf_processor.py
rosperf/results_cache.py
rosperf/results_columns.py
rosperf/results_report.py
rosperf/results_sorter.py
rosperf/run_tests.sh
rosperf/settings.py
rosperf/settings_factory.py
rosperf/table.py
mage_chromeos.py
ock_machine.py
tils/__init__.py
tils/command_executer.py
tils/email_sender.py
tils/file_utils.py
tils/html_tools.py
tils/logger.py
tils/utils.py
14/README
14/build_chrome_browser.py
14/build_chromeos.py
14/cros_run_benchmarks.py
14/image_chromeos.py
14/tc_enter_chroot.py
14/utils/__init__.py
14/utils/command_executer.py
14/utils/logger.py
14/utils/utils.py
70de27bdb5a05716befea67cdf72a87714fcc0da 16-Jun-2011 Ahmad Sharif <asharif@chromium.org> Sync'd repos to: 53837
14/README
14/build_chrome_browser.py
14/build_chromeos.py
14/cros_run_benchmarks.py
14/image_chromeos.py
14/tc_enter_chroot.py
14/utils/__init__.py
14/utils/command_executer.py
14/utils/logger.py
14/utils/utils.py
e705c67371e11e29af5406e95d8f921e2658f9af 01-Jan-1970 git5 <git5@invalid> git5: git5track synced with perforce at 43974085
c244ac05fb1e6b5649e5a29bccb537778aaa917e 12-Mar-2013 yunlian <yunlian@p4.invalid> Remove unused variable and use local variable for chromeos_version

PRESUBMIT=passed
R=shenhan,llozano
APPROVED=llozano
DELTA=4 (0 added, 1 deleted, 3 changed)
OCL=43856472


P4 change: 43858691
14/remote_gcc_build.py
b7783c0d5dcddbfdf919a126227a2f41dc6a5808 12-Mar-2013 yunlian <yunlian@p4.invalid> Find the download address for toolchain branches.

Test: It finds the address for R25 and R26 toolchain branch.

PRESUBMIT=passed
R=llozano,shenhan
APPROVED=llozano
DELTA=25 (20 added, 1 deleted, 4 changed)
OCL=43845178


P4 change: 43848282
14/remote_gcc_build.py
1d2e8e107b9caff5a5f6b9a9cb1124caa0384ab0 12-Mar-2013 yunlian <yunlian@p4.invalid> Try to find a successful build in build log if there are multiple
builds with the same description.

TEST: It finds the successful builds.

PRESUBMIT=passed
R=shenhan,llozano
APPROVED=llozano
DELTA=18 (13 added, 0 deleted, 5 changed)
OCL=43803051


P4 change: 43810634
14/remote_gcc_build.py
9b84cb630b6c6a27eb312b6e40d7a2eea2cbb1ad 11-Mar-2013 llozano <llozano@p4.invalid> Missed another instance of return without "finally" and no "except problem
in this file.

Fixed location of return statement. Should not be within "finally" block.

When return statement is within "finally" statement any exception
that happens within the try block will not be reported. Exceptions are
supposed to be rethrown at the end of "finally" if there was no "except"
but, as the code was written, the end of "finally" is never reached.

TEST=Verified exceptions are reported correctly. Also, with asharif
verified with small test case that our understanding of "finally" was
correct.

PRESUBMIT=passed
R=asharif
APPROVED=asharif
DELTA=1 (0 added, 0 deleted, 1 changed)
OCL=43793218


P4 change: 43802609
14/build_tc.py
5fee82f8df6fada913dd60e8b6b886973c4a0a8f 11-Mar-2013 llozano <llozano@p4.invalid> Fixed location of return statement. Should not be within "finally" block.

When return statement is within "finally" statement any exception
that happens within the try block will not be reported. Exceptions are
supposed to be rethrown at the end of "finally" if there was no "except"
but, as the code was written, the end of "finally" is never reached.

TEST=Verified exceptions are reported correctly. Also, with asharif
verified with small test case that our understanding of "finally" was
correct.

PRESUBMIT=passed
R=asharif
APPROVED=asharif
DELTA=1 (0 added, 0 deleted, 1 changed)
OCL=43770739


P4 change: 43792110
14/build_tc.py
004cd04b97b199481edb3027b22cb76707acebc7 11-Mar-2013 yunlian <yunlian@p4.invalid> Use "Product_S/N" as the machine id.

Test: Run "sudo dump_vpd_log --full --stdout" on lumpy, daisy,
this command shows Product_S/N field and it remain unchanged across
different images.

PRESUBMIT=passed
R=asharif,llozano,shenhan
APPROVED=shenhan
DELTA=2 (0 added, 0 deleted, 2 changed)
OCL=43574435


P4 change: 43788275
14/crosperf/machine_manager.py
f2726d289c15b45fd9b543350672988e43f52cff 09-Mar-2013 llozano <llozano@p4.invalid> Disable ccache by default in the sysroot wrapper

We have found that ccache ignores GCC envvars and may not be
checking for diffs in all the GCC binaries. It is safer for
the toolchain team to disable this.
Moreover, in my experiments, ccache seems to make the builds
slower when using a large parallel factor.
There is a new option to enable ccache.

TEST=Build with and without the new option to enable ccache and
verified that sysroot wrapper is correct.

PRESUBMIT=passed
R=cmtice,asharif,shenhan
APPROVED=shenhan
DELTA=16 (13 added, 0 deleted, 3 changed)
OCL=43620049


P4 change: 43669814
14/build_tc.py
ccad28ade4436cea9361951b9cd0a940a05e3956 07-Mar-2013 yunlian <yunlian@p4.invalid> Use upstream git as the source of gdb

Test: Dry-run works.

PRESUBMIT=passed
R=cmtice,bjanakiraman,shenhan,asharif
APPROVED=shenhan
DELTA=3 (0 added, 0 deleted, 3 changed)
OCL=43512237


P4 change: 43556520
14/repo_to_repo_files/gdb-master.rtr
3676c8fe003bf2b564125b10f3c1690c19343740 06-Mar-2013 asharif <asharif@p4.invalid> Fixed gpylint warnings for tabulator.py & tabulator_test.py.

I want to use this code to get readability.

BUG=none
TEST=gpylint tabulator.py tabulator_test.py
python tabulator.py
python tabulator_test.py

PRESUBMIT=passed
R=yunlian,bjanakiraman
APPROVED=yunlian
DELTA=128 (93 added, 13 deleted, 22 changed)
OCL=43475677


P4 change: 43505189
14/utils/tabulator.py
14/utils/tabulator_test.py
1b387033e8c40d0ee017c1b784959dfd526c508f 06-Mar-2013 yunlian <yunlian@p4.invalid> Show max(5, rows_of_perf_with_over_1%_running_time) in email.

PRESUBMIT=passed
R=llozano,shenhan
APPROVED=shenhan
DELTA=33 (30 added, 0 deleted, 3 changed)
OCL=43445498
TESTS=It shows the correct rows of perf data.


P4 change: 43497769
14/crosperf/perf_table.py
14/crosperf/results_report.py
14/utils/perf_diff.py
bee1e610638827621ac547314a85f8528ddc05c1 05-Mar-2013 yunlian <yunlian@p4.invalid> Add p-value and std/mean to full table.

Test: The it shows the columns in the full table.

PRESUBMIT=passed
R=shenhan,llozano
APPROVED=shenhan
DELTA=7 (6 added, 0 deleted, 1 changed)
OCL=43444692


P4 change: 43445189
14/crosperf/results_report.py
c9a6e773d8a4f1c8ec96f439381f2b36da4a9c7a 04-Mar-2013 yunlian <yunlian@p4.invalid> Add a function go get manifest file from a version number.
Add some comments on this file.
Check the results of the build_bot, if a build failed, pops out
Error.
Remove the time out, it will alway finishes.

Test: Run remote_gcc_build.py -c ~/chrotomius -g ~/gcc -t lumpy-release -b toolchain-3428.65.B
-d ~/tmp
It put the sysroot and the image to ~/tmp.

PRESUBMIT=passed
R=llozano,shenhan,carrot
APPROVED=llozano
DELTA=290 (143 added, 16 deleted, 131 changed)
OCL=65610-p2
RCL=66193-p2
RDATE=2013/03/04 13:37:32


P4 change: 43400707
14/remote_gcc_build.py
f6617aef47df8b46887d2d633051dd4a0dacde9a 01-Mar-2013 shenhan <shenhan@p4.invalid> Add support for checking out an arbitrary version.

The implementation only supports version that is stable (version files
under paladin directory), now add version that is not stable (aka -
under buildspec file).

PRESUBMIT=passed
BUG=None
R=llozano
CC=asharif,bjanakiraman,carrot,cmtice,yunlian,sque
APPROVED=llozano
OCL=66130-p2
RCL=66154-p2
RDATE=2013/03/01 09:16:21
TEST=Run `../v14/setup_chromeos.py --version=26.3701.30.1 --dir=26.3701.30.1`, the version specfile is under 'buildspec' directory.


P4 change: 43289842
14/setup_chromeos.py
26f6c261ecd57159c832bdfe0bec48212e26b696 27-Feb-2013 yunlian <yunlian@p4.invalid> Remove unused modules

Test: crosperf works without imports.

PRESUBMIT=passed
R=shenhan,llozano,asharif
APPROVED=asharif
DELTA=3 (0 added, 3 deleted, 0 changed)
RCL=66096-p2
RDATE=2013/02/26 16:49:05


P4 change: 43145543
14/crosperf/experiment.py
d412ec789267dd4d434245469621b2534cd7b893 27-Feb-2013 yunlian <yunlian@p4.invalid> Remove the unused parameter rerun_if_failed for experiment.

Test: Run the crosperf with/without rerun_if_failed, it passes.

PRESUBMIT=passed
R=shenhan,llozano,asharif
APPROVED=shenhan
DELTA=4 (0 added, 2 deleted, 2 changed)
OCL=66078-p2
RCL=66095-p2
RDATE=2013/02/26 16:42:00


P4 change: 43145444
14/crosperf/experiment.py
14/crosperf/experiment_factory.py
75d42eb05c5516b34a3ac4e4fcb6fbc2c01fe9c8 27-Feb-2013 llozano <llozano@p4.invalid> fixed problem with default behavior which was broken in latest
commit.

TEST=tested the following behaviors:
./setup_chromeos.py --dir /tmp/x
./setup_chromeos.py --dir /tmp/x --version=latest_common
./setup_chromeos.py --dir /tmp/x --version=latest_lkgm
./setup_chromeos.py --dir /tmp/x --version=top
./setup_chromeos.py --dir /tmp/x --version=top --timestamp=1361921045
./setup_chromeos.py --dir /tmp/x --version=lkgm --timestamp=1361921045
./setup_chromeos.py --dir /tmp/x --version=lkgm --timestamp=1356508800
./setup_chromeos.py --dir /tmp/x --version=common --timestamp=1356508800

PRESUBMIT=passed
R=asharif,yunlian,shenhan
APPROVED=asharif
DELTA=1 (0 added, 0 deleted, 1 changed)
OCL=66090-p2
RCL=66094-p2
RDATE=2013/02/26 16:30:29


P4 change: 43143278
14/setup_chromeos.py
86fe6d56ac55478c60c098d09be705740d33433d 26-Feb-2013 asharif <asharif@p4.invalid> symbolizer: make it more robust.

1. Create the output directory specified if it doesn't exist.
2. Keep symbolizing reports even if one fails in the middle.

Tested: python symbolizer.py --in=... --out=... --cache=... works.

PRESUBMIT=passed
R=sque,bjanakiraman
DELTA=8 (3 added, 0 deleted, 5 changed)
OCL=66071-p2
RCL=66079-p2
RDATE=2013/02/26 13:37:04


P4 change: 43132670
14/cwp/interpreter/symbolizer.py
e7a9dc93cfed39c08a782481578238aea4bb8258 25-Feb-2013 yunlian <yunlian@p4.invalid> Fix the problem when rerun the experiment the data is not cached.

Test: Rerun crosperf with --rerun=True, the cache changed.

PRESUBMIT=passed
R=llozano,shenhan
APPROVED=llozano
DELTA=5 (1 added, 0 deleted, 4 changed)
RCL=66057-p2
RDATE=2013/02/25 14:35:00


P4 change: 43077449
14/crosperf/results_cache.py
9efafde43157fc3346977a222b1db059b550a551 23-Feb-2013 llozano <llozano@p4.invalid> Add support for common image to setup_chromeos.py.
Added get_common_image.py to get info about all common images.
Keep static list of common images within setup_chromeos. This
list needs to be updated by sherif when new image becomes available.
Changed --timestamp argument to apply to "lkgm" and 'common' options.
Added options latest_lkgm and latest_common.

PRESUBMIT=passed
R=asharif,yunlian,carrot,shenhan
APPROVED=yunlian
DELTA=167 (152 added, 5 deleted, 10 changed)
OCL=65574-p2
RCL=66030-p2
RDATE=2013/02/22 17:32:24


P4 change: 42988665
14/get_common_image_version.py
14/setup_chromeos.py
cbc868a3beff66b6a98cd787a7c37f2528026dc7 21-Feb-2013 cmtice <cmtice@p4.invalid> Add --extra_use_flags option, which updates the USE flag appropriately.

TEST=Built gcc compiler in chroot, passing the
--extra_use_flags="vtable_verify"; went into the chroot and verified
that the compiler built correctly.

PRESUBMIT=passed
R=llozano,asharif,shenhan,carrot,yunlian,bjanakiraman
APPROVED=llozano
DELTA=12 (11 added, 0 deleted, 1 changed)
OCL=65920-p2
RCL=66011-p2
RDATE=2013/02/21 14:52:10


P4 change: 42926966
14/build_tc.py
52e96220fed6bad4cc6c806f60a3a57730ae1f04 19-Feb-2013 shenhan <shenhan@p4.invalid> Added '--debug', '--dev' and '--verbose' options for building the chrome browser.

PRESUBMIT=passed
BUG=None
R=llozano
CC=asharif,bjanakiraman,carrot,cmtice,yunlian
APPROVED=llozano
OCL=65882-p2
RCL=65917-p2
RDATE=2013/02/15 16:21:32
TEST=observed built with '-O0 -g' and verbose output.


P4 change: 42807709
14/build_chrome_browser.py
4873858871636ed9edeb6fb97026a1b222d73f6e 19-Feb-2013 shenhan <shenhan@p4.invalid> Add '--dev' and '--debug' options to build_chromeos.py.

'--dev' option builds a writable and bigger chromeos image. In
addition, '--debug' option builds chromeos-chrome (the browser) with
'-g -O0' while keep everything else unchanged. Notice, '--debug' turns
on '--dev' automatically.

PRESUBMIT=passed
BUG=None
R=llozano,asharif
CC=bjanakiraman,cmtice,yunlian,carrot
APPROVED=yunlian
OCL=65775-p2
RCL=65916-p2
RDATE=2013/02/15 16:20:46
TEST=./build_chromeos.py --chromeos_root=/usr/local/google/home/shenhan/ssd/live --rebuild --debug --board=daisy |& tee ~/log


P4 change: 42807708
14/build_chromeos.py
14/image_chromeos.py
14/utils/misc.py
2b549cabe9de39dd5cb65b7b1fe737da3d65b155 19-Feb-2013 carrot <carrot@p4.invalid> Add new command line options --cflags_for_target and --cxxflags_for_target, and pass them into emerge.

Tested:
./build_tc.py -b x86-alex -c ~/cros2/ --cxxflags_for_target="-Wunused-label" -g ~/gcc47cros/
And look at the output to ensure -Wunused-label is actually used.

PRESUBMIT=passed
R=shenhan,yunlian,llozano,asharif,cmtice
CC=c-compiler-chrome
APPROVED=llozano
DELTA=12 (12 added, 0 deleted, 0 changed)
OCL=65833-p2
RCL=65854-p2
RDATE=2013/02/13 15:15:01


P4 change: 42807590
14/build_tc.py
a8977a95061f83ffbe32a9442a3e3bf73f27148b 19-Feb-2013 yunlian <yunlian@p4.invalid> Add an option to share the cache data with other one.

Test: remove the default cache path, run a cached test, it shows cache
miss.
run ./crosperf --share_users=yunlian, it shows cache hit

PRESUBMIT=passed
R=llozano,cmtice,asharif
APPROVED=llozano
DELTA=33 (19 added, 0 deleted, 14 changed)
OCL=65354-p2
RCL=65672-p2
RDATE=2013/02/05 15:53:27


P4 change: 42807223
14/crosperf/benchmark_run.py
14/crosperf/benchmark_run_unittest.py
14/crosperf/experiment.py
14/crosperf/experiment_factory.py
14/crosperf/results_cache.py
14/crosperf/settings_factory.py
65e7ae979747837786c2c25d175060baec3ebef4 19-Feb-2013 yunlian <yunlian@p4.invalid> Allow users set cache dir for each image.
Test: run crosperf without setting cache_dir,
setting cache_dir globally
setting cache_dir globally and in label at the same
time
setting cache_dir in label only
All pass.

PRESUBMIT=passed
R=llozano,shenhan,carrot
APPROVED=shenhan
DELTA=28 (13 added, 5 deleted, 10 changed)
OCL=65663-p2
RCL=65666-p2
RDATE=2013/02/05 12:31:42


P4 change: 42807185
14/crosperf/benchmark_run_unittest.py
14/crosperf/experiment_factory.py
14/crosperf/experiment_factory_unittest.py
14/crosperf/label.py
14/crosperf/machine_manager_unittest.py
14/crosperf/results_cache.py
14/crosperf/settings_factory.py
d58b8dcce5c651fc7f9a332fb7ae6ff39225c4ee 19-Feb-2013 yunlian <yunlian@p4.invalid> Fix the issue that the AcquireLock can be used by one person on the
same machine.

Test: The permission of the dir is changed so that every one in the
same group has write permission.

PRESUBMIT=passed
R=llozano,shenhan,cmtice
APPROVED=llozano
DELTA=2 (1 added, 0 deleted, 1 changed)
RCL=65665-p2
RDATE=2013/02/05 11:49:01


P4 change: 42807184
14/utils/misc.py
255fd460c5e406f9f2fce7198660cceefc7bcbea 19-Feb-2013 carrot <carrot@p4.invalid> Append the flags specified with --cflags instead of replacing when building gcc.

By default emerge provide `portageq envvar CFLAGS` as CFLAGS to gcc building.
When CFLAGS is provided when calling emerge, it will replace the default CFLAGS.
This patch prepend `portageq envvar CFLAGS` to --cflags options.

It also fixed a bug for option --ldflags.

TESTED:
./build_tc.py -b x86-alex -c ~/cros1/ --cflags="-g3 -O0" -g ~/gcc47cros/ > log2
And look at the command line to ensure "-g3 -O0" is appended to the default options "-O2 -pipe".

PRESUBMIT=passed
R=yunlian,asharif,cmtice,shenhan,llozano
CC=c-compiler-chrome
APPROVED=llozano
DELTA=3 (0 added, 0 deleted, 3 changed)
OCL=65657-p2
RCL=65664-p2
RDATE=2013/02/05 11:00:56


P4 change: 42807183
14/build_tc.py
952441dcb6f363af4778b932ac0ce975453a4af8 19-Feb-2013 yunlian <yunlian@p4.invalid> Add an option to set the chrome version to use.
Default is the latest version.

Test: The chrome version is added.

PRESUBMIT=passed
R=shenhan,llozano,carrot
APPROVED=shenhan
DELTA=15 (11 added, 0 deleted, 4 changed)
OCL=65600-p2
RCL=65602-p2
RDATE=2013/02/01 11:02:37


P4 change: 42806591
14/remote_gcc_build.py
33fef357e3bf2f44d7c9f1127c6d044b52633462 19-Feb-2013 yunlian <yunlian@p4.invalid> Make remote_gcc_build.py accept the case where no gcc dir
is specified.

Test: run remote_gcc_build.py with/without gcc dir. It works fine

PRESUBMIT=passed
R=llozano,shenhan,carrot
APPROVED=shenhan
DELTA=68 (36 added, 9 deleted, 23 changed)
OCL=65545-p2
RCL=65599-p2
RDATE=2013/02/01 09:44:54


P4 change: 42806584
14/remote_gcc_build.py
ccae62755d8e5bb98f91d6538e8ea6e911cdbbc2 19-Feb-2013 carrot <carrot@p4.invalid> Use the file emerge-<board> to decide if setup_board need to be executed.

The original logic to detect setup_board has been executed has 2 problems:
1. It uses <target> instead of <board>, if two boards uses same toolchain, it may prevent setup_board for second board.
2. Now the directories used to detect setup_board exist all the times, so setup_board will never be executed.

These problems can be fixed with file emerge-<board>.

Tested:
./build_tc.py -b daisy,x86-alex,lumpy,stumpy -c ~/cros2/ -g ~/gcc47cros/

PRESUBMIT=passed
R=shenhan,asharif,yunlian,llozano,cmtice
CC=c-compiler-chrome
APPROVED=llozano
DELTA=1 (0 added, 0 deleted, 1 changed)
OCL=65575-p2
RCL=65581-p2
RDATE=2013/01/31 13:48:40


P4 change: 42806569
14/build_tc.py
850786bb6c2dbab1d7a01cb9f9f409b51bf9abc1 19-Feb-2013 yunlian <yunlian@p4.invalid> Add an option to disable sending email after crosperf finishes.

Test: Using this option, email is gone.
Without this option, email is there.

PRESUBMIT=passed
R=shenhan,llozano,asharif
CC=dennisjeffrey
APPROVED=shenhan
DELTA=7 (6 added, 0 deleted, 1 changed)
OCL=65571-p2
RCL=65578-p2
RDATE=2013/01/31 11:20:46


P4 change: 42806548
14/crosperf/experiment_factory.py
14/crosperf/experiment_runner.py
14/crosperf/settings_factory.py
2ab92b1a948654fb6e7395e4ec2ee2c2f7636da3 19-Feb-2013 yunlian <yunlian@p4.invalid> Add an option to set the default cache dir

PRESUBMIT=passed
R=llozano,asharif,shenhan
APPROVED=shenhan
DELTA=65 (44 added, 9 deleted, 12 changed)
OCL=65503-p2
RCL=65570-p2
RDATE=2013/01/31 09:32:35


P4 change: 42806538
14/crosperf/config.py
14/crosperf/experiment_factory.py
14/crosperf/results_cache.py
14/crosperf/settings_factory.py
e6f773ce58b5d7a70d8340038c4f257510c3d849 19-Feb-2013 carrot <carrot@p4.invalid> In build_tc.py detect and avoid the duplicate toolchain building.

Test
./build_tc.py -b daisy,lumpy,stumpy -c ~/ssd/cros1/ -g ~/google46/
and watch the output, ensure there are only two buildings of gcc.

PRESUBMIT=passed
R=asharif,cmtice,shenhan,llozano,yunlian
CC=c-compiler-chrome
APPROVED=yunlian
DELTA=12 (5 added, 2 deleted, 5 changed)
OCL=65433-p2
RCL=65556-p2
RDATE=2013/01/30 17:21:57


P4 change: 42806508
14/build_tc.py
5acba6e4cb96e6bab85ef332a320fa8ca82e3287 19-Feb-2013 yunlian <yunlian@p4.invalid> Enable build_chromeos.py build single package. It also accepts
FEATURE options.

TEST: ./build_chromeos --chromeos_root=/extend/gamma --board=lumpy
--env=FEATURES=noclean --package=zlib
The package is built, the source is there too.

PRESUBMIT=passed
R=llozano,asharif,carrot,shenhan
CC=c-compiler-chrome
APPROVED=shenhan
DELTA=9 (7 added, 0 deleted, 2 changed)
OCL=65412-p2
RCL=65506-p2
RDATE=2013/01/29 13:56:48


P4 change: 42806447
14/build_chromeos.py
e23125781bbfdef2c0b88bf74a3bf9d6ac9d0dfa 19-Feb-2013 yunlian <yunlian@p4.invalid> Call generate_test_report inside chroot instead of outside chroot.

Test: run crosperf with/without cache, in the generated email, the
data is there.

PRESUBMIT=passed
R=asharif,llozano,shenhan
APPROVED=shenhan
DELTA=18 (8 added, 0 deleted, 10 changed)
OCL=65487-p2
RCL=65496-p2
RDATE=2013/01/29 10:21:01


P4 change: 42806410
14/crosperf/results_cache.py
dcdd9c7a64fe58137df458a4d6071a6646d28e10 19-Feb-2013 yunlian <yunlian@p4.invalid> Add an option to specify log_dir instead of the dir where the
script is.

Test: Run crosperf with -l ../, the logs moved to the upper dir.

PRESUBMIT=passed
R=asharif,shenhan,llozano
APPROVED=llozano
DELTA=25 (11 added, 2 deleted, 12 changed)
OCL=65455-p2
RCL=65489-p2
RDATE=2013/01/28 16:52:53


P4 change: 42806393
14/crosperf/crosperf.py
14/crosperf/experiment.py
14/crosperf/experiment_factory.py
14/crosperf/experiment_runner.py
14/utils/logger.py
ebaacf3108fbea23a5e0604934f93ae8b4f6fc8a 19-Feb-2013 yunlian <yunlian@p4.invalid> Show "0" instead of "-" in the raw results if the raw results is zero

Test: It shows "0" now.

PRESUBMIT=passed
R=shenhan,asharif
APPROVED=shenhan
DELTA=4 (0 added, 3 deleted, 1 changed)
RCL=65481-p2
RDATE=2013/01/28 11:11:31


P4 change: 42806380
14/utils/tabulator.py
8e8b0c295fd44cbabd62111ebf0189beccd01cb4 19-Feb-2013 shenhan <shenhan@p4.invalid> Add binutils support for build_tc.py.

This is part 1 of the work. Another part involves modification to
binutils ebuild file, which will be done after upgrading to
binutils-2.22.

Ahmad has made a really good framework, so the modification here is pretty straight forward.

PRESUBMIT=passed
BUG=None
R=asharif,llozano
CC=bjanakiraman,carrot,cmtice
APPROVED=asharif
OCL=65462-p2
RCL=65479-p2
RDATE=2013/01/28 10:08:58
TEST=./build_tc.py --binutils_dir=/usr/local/google/home/shenhan/hdd2/live-binutils-2.22/src/third_party/binutils --chromeos_root=/usr/local/google/home/shenhan/hdd2/live --board=x86-alex


P4 change: 42806378
14/build_tc.py
f4566c9faaa39fe41c398cb03831638619578014 19-Feb-2013 yunlian <yunlian@p4.invalid> Enhance it as follows:
enable fixed version of chromeos tree
download the image and chroot once the remote job is finished
Test:
run a test with fixed chromeos version. It works fine.
run ./remote_gcc_build.py -c ~/chromiumos/ -g ~/gcc/ -d ~/pp -t lumpy-release
I got the image in ~/pp, I then cp ~/pp to ~/chromiumos/chroot/build/lumpy
I can run crosperf with the image and the chromeos_root.

PRESUBMIT=passed
R=asharif,shenhan,carrot
APPROVED=shenhan
DELTA=166 (151 added, 0 deleted, 15 changed)
OCL=65026-p2
RCL=65454-p2
RDATE=2013/01/25 09:34:58


P4 change: 42806348
14/remote_gcc_build.py
1c6db9b5f05c5a9e035f19351d5aa9b42648229f 19-Feb-2013 yunlian <yunlian@p4.invalid> Missing change for CL https://critique.corp.google.com/#review/65413-p2

Test: 'str' object has no attribute 'name' is gone.

PRESUBMIT=passed
R=asharif,shenhan,llozano
APPROVED=shenhan
DELTA=3 (0 added, 0 deleted, 3 changed)
OCL=65422-p2
RCL=65424-p2
RDATE=2013/01/24 14:58:52


P4 change: 42805973
14/crosperf/machine_manager.py
14/crosperf/results_cache.py
c73a1ba05e96a484425536c1b2deb16eccd714a9 19-Feb-2013 yunlian <yunlian@p4.invalid> Add image_md5sum to label. If the image is not accessible, crosperf
will try to use this as the md5sum of the image to get the cached data

Test: run crosperf without cache
run crosperf with cache
run crosperf with cache and md5sum, with wrong chromeos_image

PRESUBMIT=passed
R=llozano,asharif,shenhan
APPROVED=llozano
DELTA=38 (19 added, 0 deleted, 19 changed)
OCL=65385-p2
RCL=65413-p2
RDATE=2013/01/24 11:14:45


P4 change: 42805964
14/crosperf/benchmark_run_unittest.py
14/crosperf/experiment_factory.py
14/crosperf/image_checksummer.py
14/crosperf/label.py
14/crosperf/machine_manager_unittest.py
14/crosperf/settings_factory.py
e05b39e3fa749047545c4e662bcc605cd6935979 19-Feb-2013 yunlian <yunlian@p4.invalid> Add an option to enable crosperf to wait for a given time if
all the machines in its experiment file are locked by others.

Test: Lock a machine manually and run crosperf with that machine.
The crosperf waits until time out or the machine becomes available.

PRESUBMIT=passed
R=llozano,asharif,shenhan
CC=c-compiler-chrome
APPROVED=llozano
DELTA=37 (16 added, 1 deleted, 20 changed)
OCL=65271-p2
RCL=65377-p2
RDATE=2013/01/23 11:09:04


P4 change: 42805897
14/crosperf/benchmark_run_unittest.py
14/crosperf/experiment.py
14/crosperf/experiment_factory.py
14/crosperf/machine_manager.py
14/crosperf/machine_manager_unittest.py
14/crosperf/settings_factory.py
b4d3f17cbd7e64ff4ddf3205493b5003093273e4 19-Feb-2013 yunlian <yunlian@p4.invalid> Fix the bug 7969399
When preparing the perf table, it only keeps the functions that all
the runs have value on it.
Test: It serves the purpose
run a crosperf with perf enabled.

Before
cycles
keys aaa bb
Keys Completed Amean StdDev StdDev/Mean Completed Amean StdDev StdDev/Mean GmeanSpeedup p-value
libc-bench [kernel.kallsyms] [k] 0xffffffff810a2633 1 871.00 0.00 0.0% 0
libc-bench libc-bench [.] _IO_vasprintf 1 168.00 0.00 0.0% 1 177.00 0.00 0.0% +5.4% -- nan
libc-bench libc-bench [.] __gconv_transform_utf8_internal 1 329.00 0.00 0.0% 1 348.00 0.00 0.0% +5.8% -- nan
libc-bench libc-bench [.] __strstr_sse42 1 695.00 0.00 0.0% 1 664.00 0.00 0.0% -4.5% -- nan
libc-bench libc-bench [.] b_utf8_onebyone 0 1 137.00 0.00 0.0%
libc-bench libc-bench [.] re_search_internal 0 1 111.00 0.00 0.0%
swapper [kernel.kallsyms] [k] 0xffffffff8146afe0 1 448.00 0.00 0.0% 0

after:
Benchmark: platform_LibCBench; Iterations: 1
cycles
keys aaa bb
Keys Completed Amean StdDev StdDev/Mean Completed Amean StdDev StdDev/Mean GmeanSpeedup p-value
autotest libc-2.15.so [.] __memcpy_ssse3_back 1 1.00 0.00 0.0% 1 1.00 0.00 0.0% +0.0% -- nan
autotest libpython2.6.so.1.0 [.] PyEval_EvalFrameEx 1 3.00 0.00 0.0% 1 4.00 0.00 0.0% +33.3% -- nan
autotest libpython2.6.so.1.0 [.] PyObject_GetAttr 1 2.00 0.00 0.0% 1 1.00 0.00 0.0% -50.0% -- nan
autotest libpython2.6.so.1.0 [.] PyTuple_New 1 2.00 0.00 0.0% 1 1.00 0.00 0.0% -50.0% -- nan
autotest libpython2.6.so.1.0 [.] _PyType_Lookup 1 1.00 0.00 0.0% 1 1.00 0.00 0.0% +0.0% -- nan

PRESUBMIT=passed
BUG=7969399
R=asharif,shenhan,carrot
APPROVED=asharif
DELTA=20 (11 added, 0 deleted, 9 changed)
OCL=65065-p2
RCL=65253-p2
RDATE=2013/01/17 09:44:31


P4 change: 42804954
14/crosperf/perf_table.py
14/crosperf/results_report.py
14/utils/perf_diff.py
14/utils/tabulator.py
13805f5d426050114b9dce446187e3836bb23003 19-Feb-2013 asharif <asharif@p4.invalid> Fixed numerical sorting of keys based on values.

Tested: Ran tabulator.py and keys are sorted by values descending now.

PRESUBMIT=passed
R=yunlian,bjanakiraman
APPROVED=yunlian
DELTA=8 (4 added, 0 deleted, 4 changed)
OCL=65219-p2
RCL=65222-p2
RDATE=2013/01/16 11:19:29


P4 change: 42804901
14/utils/tabulator.py
7cc1d2391ee03cac316693b90163c3b805903ca1 19-Feb-2013 yunlian <yunlian@p4.invalid> Add key_results_only option(default is on).
With this option, after running pyautoperf, it only prints the numbers starts
with "milliseconds", "ms_", "score", "retval" for Pyautoperf.

Test: run a pyautoperf with cached data. Switch the option, it shows
the right keys in the report.

PRESUBMIT=passed
R=asharif,shenhan,carrot
CC=c-compiler-chrome,cmtice
APPROVED=shenhan
DELTA=30 (26 added, 0 deleted, 4 changed)
OCL=65194-p2
RCL=65218-p2
RDATE=2013/01/16 10:56:24


P4 change: 42804898
14/crosperf/benchmark.py
14/crosperf/benchmark_run_unittest.py
14/crosperf/experiment_factory.py
14/crosperf/results_organizer.py
14/crosperf/settings_factory.py
228babd9d23d5a7f1c49d9cd1017120c674be2ff 19-Feb-2013 yunlian <yunlian@p4.invalid> Add buildbot_json.py from
https://github.com/buildbot/buildbot/blob/master/master/contrib/buildbot_json.py.
This script can be used to get the build information at trybot server.

Test: run buildbot_json.py builds http://chromegw/p/tryserver.chromiumos/
it gave the right information

PRESUBMIT=passed
R=asharif,shenhan,llozano
APPROVED=asharif
DELTA=1438 (1438 added, 0 deleted, 0 changed)
OCL=64844-p2
RCL=65101-p2
RDATE=2013/01/10 15:38:21


P4 change: 42804572
14/utils/buildbot_json.py
0542e27cfde85a552957cd45e60dac8e9bc6e477 19-Feb-2013 asharif <asharif@p4.invalid> Modified section signature to be more precise.

Tested: perf_diff.py <report>
where report had a class method like Events::Function.

PRESUBMIT=passed
R=yunlian,bjanakiraman
APPROVED=yunlian
DELTA=1 (0 added, 0 deleted, 1 changed)
RCL=65083-p2
RDATE=2013/01/09 14:59:26


P4 change: 42804547
14/utils/perf_diff.py
2184a96bcf89f7eec0b3f87948b2d166a5cbfca0 19-Feb-2013 yunlian <yunlian@p4.invalid> Add an option to remove the results inside chroot.

Test: run crosperf with no-cached case, the files are removed.
run crosperf with cached case, nothing changed.

PRESUBMIT=passed
R=shenhan,asharif,carrot
APPROVED=asharif
DELTA=18 (13 added, 0 deleted, 5 changed)
OCL=65064-p2
RCL=65079-p2
RDATE=2013/01/09 13:49:04


P4 change: 42804532
14/crosperf/benchmark.py
14/crosperf/benchmark_run_unittest.py
14/crosperf/experiment_factory.py
14/crosperf/experiment_runner.py
14/crosperf/results_cache.py
14/crosperf/settings_factory.py
3b85c8bf5c99c0445d71439fa4c0ed86d7747d9e 19-Feb-2013 yunlian <yunlian@p4.invalid> Get rid of select.select for scalability issues.

Test: Run crosperf with cached version. It works.
Run crosperf with 500 iterations, it does not die.

PRESUBMIT=passed
R=asharif,shenhan,carrot
APPROVED=asharif
DELTA=33 (29 added, 1 deleted, 3 changed)
OCL=64620-p2
RCL=65032-p2
RDATE=2013/01/07 14:34:00


P4 change: 42803152
14/utils/command_executer.py
14/utils/command_executer_unittest.py
d9cc529b0e1e642a4766c8b1610b9f38351ac56e 19-Feb-2013 shenhan <shenhan@p4.invalid> Fix dejagnu daily job leaves trashes behind by deleting ".cache" using "sudo".

".cache" must be deleted by "sudo", otherwise later cleanup work by
autotest framework will fail to delete the chromeos tree.

PRESUBMIT=passed
BUG=None
R=asharif
CC=bjanakiraman,cmtice,carrot,yunlian,llozano
APPROVED=asharif
OCL=65025-p2
RCL=65029-p2
RDATE=2013/01/07 13:36:06
TEST=locally run dejagnu test with "--cleanup=chroot", ".cache" is deleted


P4 change: 42803149
14/dejagnu/run_dejagnu.py
7f8bde1bbbb8e025f3c1297c1fd72bb0a588719c 19-Feb-2013 asharif <asharif@p4.invalid> Fix broken rtr file.

This file was broken by cl/64565-p2.

Tested: python repo_to_repo.py -i toolchain-utils.rtr -n worked.

PRESUBMIT=passed
R=yunlian,carrot,shenhan,bjanakiraman
APPROVED=yunlian
DELTA=1 (0 added, 0 deleted, 1 changed)
OCL=64968-p2
RCL=64987-p2
RDATE=2013/01/03 20:03:02


P4 change: 42803058
14/repo_to_repo_files/toolchain-utils.rtr
58395703c3ca6d65d9f71b994ffff28e486e64df 19-Feb-2013 asharif <asharif@p4.invalid> Remove bgcolor_row property from pvalue formatter.

This was causing asserts to be hit down the pipeline when comparing
multiple images. We need to redo adding whole-row color to formatters
and how they interact amongst themselves.

Tested: python tabulator.py and python tabulator_test.py work.

PRESUBMIT=passed
R=yunlian,bjanakiraman
APPROVED=yunlian
DELTA=1 (0 added, 1 deleted, 0 changed)
RCL=64960-p2
RDATE=2013/01/02 14:36:37


P4 change: 42803015
14/utils/tabulator.py
2538403f58d41f2981ff1b1f37fd38c7a653ceeb 19-Feb-2013 yunlian <yunlian@p4.invalid> Add the process id on the lock for image_chromeos

Test: the information is there now

PRESUBMIT=passed
BUG=7892758
R=asharif,shenhan
APPROVED=asharif
DELTA=3 (2 added, 0 deleted, 1 changed)
OCL=64811-p2
RCL=64959-p2
RDATE=2013/01/02 14:27:49


P4 change: 42803014
14/utils/misc.py
caa5e02146ee654c808fbdc0a17202dc91b75f4f 19-Feb-2013 yunlian <yunlian@p4.invalid> Generate perf_diff in report.
Here it assumes that there is a function to generate a dict from
perf raw data.

Test: with the data generating dummpy function, it works fine.

PRESUBMIT=passed
R=asharif,shenhan
APPROVED=shenhan
DELTA=158 (147 added, 4 deleted, 7 changed)
OCL=64255-p2
RCL=64740-p2
RDATE=2012/12/17 13:59:05


P4 change: 42801750
14/crosperf/perf_table.py
14/crosperf/results_report.py
89aa08e353fc6383bf3c2932078eacda99f3bece 19-Feb-2013 yunlian <yunlian@p4.invalid> Modified to make it return the format needed.
Cloned from CL 64507 by 'g4 patch'.
Original change by asharif@asharif-perforce2-home on 2012/12/06 19:26:31.

Added function called GetPerfDictFromReport() to get a dictionary of
hottest functions from a perf report.

PRESUBMIT=passed
R=asharif,shenhan,llozano
APPROVED=asharif
DELTA=12 (12 added, 0 deleted, 0 changed)
OCL=64520-p2
RCL=64665-p2
RDATE=2012/12/13 13:34:40


P4 change: 42801604
14/utils/perf_diff.py
4301e08ee945fd4a6c36561858b3d2fc73277575 19-Feb-2013 yunlian <yunlian@p4.invalid> Print out whole perf report in case there are multiple events in a single run
Test: It shows the whole perf report

PRESUBMIT=passed
R=shenhan,asharif,carrot
APPROVED=carrot
DELTA=2 (1 added, 1 deleted, 0 changed)
OCL=64645-p2
RCL=64662-p2
RDATE=2012/12/13 12:40:40


P4 change: 42801601
14/crosperf/results_cache.py
3a428924e6130369e7da004a009df5b109c63a5f 19-Feb-2013 llozano <llozano@p4.invalid> Two changes:
- remove the chrome-src-internal/src*/out_<board> directories
when --rebuild or --clean is specified to make sure the
chrome sources are rebuilt.
- Found bug in build_chrome_browser.py implementation of --clean

PRESUBMIT=passed
R=asharif,cmtice
CC=c-compiler-chrome
APPROVED=asharif
DELTA=35 (18 added, 14 deleted, 3 changed)
OCL=64567-p2
RCL=64630-p2
RDATE=2012/12/11 17:37:29


P4 change: 42801528
14/build_chrome_browser.py
14/build_chromeos.py
14/utils/misc.py
d46a4b6981f5a003cb8dbd1352ba1e11d2f43a75 19-Feb-2013 yunlian <yunlian@p4.invalid> Use python instead of python2.6
Test: python 2.7.3 passed

PRESUBMIT=passed
R=shenhan,carrot,llozano
APPROVED=shenhan
DELTA=1 (0 added, 0 deleted, 1 changed)
OCL=64623-p2
RCL=64625-p2
RDATE=2012/12/11 14:36:21


P4 change: 42801521
14/image_chromeos.py
b5851d3894af26eac102903bee669a62c64e2b7e 19-Feb-2013 yunlian <yunlian@p4.invalid> Add a script to use remote try-bot build image with local gcc

Test: it builds an image with local gcc.

PRESUBMIT=passed
R=asharif,shenhan,llozano
CC=c-compiler-chrome
APPROVED=asharif
DELTA=85 (85 added, 0 deleted, 0 changed)
OCL=64323-p2
RCL=64624-p2
RDATE=2012/12/11 14:23:48


P4 change: 42801520
14/remote_gcc_build.py
994ae3a65e5b17d8024ca302c74d4c585242c993 19-Feb-2013 yunlian <yunlian@p4.invalid> use /usr/bin/python instead of /usr/bin/python2.6

Test: it works on python 2.7.3

PRESUBMIT=passed
R=asharif,shenhan,llozano
APPROVED=asharif
DELTA=1 (0 added, 0 deleted, 1 changed)
RCL=64572-p2
RDATE=2012/12/10 14:40:43


P4 change: 42801436
14/repo_to_repo.py
07d1c64b0cc7a48959669fbbbd543d5d41ca1871 19-Feb-2013 yunlian <yunlian@p4.invalid> Add .pyc and logs to git ignore
Test: None

PRESUBMIT=passed
BUG=7393377
R=shenhan,carrot
APPROVED=shenhan
DELTA=2 (1 added, 0 deleted, 1 changed)
OCL=64474-p2
RCL=64565-p2
RDATE=2012/12/10 13:53:02


P4 change: 42801429
14/repo_to_repo_files/toolchain-utils.rtr
aa4daaa59829b5bf080f627ec61aa5aedd6d35b4 19-Feb-2013 asharif <asharif@p4.invalid> Add correct dir prefix before default_remotes so crosperf would work
from other $cwds.

Tested: Invoked crosperf without specifying remotes in a different dir
and it worked.

PRESUBMIT=passed
R=yunlian,cmtice,bjanakiraman
APPROVED=yunlian
DELTA=3 (2 added, 0 deleted, 1 changed)
OCL=64561-p2
RCL=64563-p2
RDATE=2012/12/10 12:33:40


P4 change: 42801423
14/crosperf/experiment_factory.py
f2cf600d836e963dd6a4f3d27d666f303dbcaa33 19-Feb-2013 yunlian <yunlian@p4.invalid> USE Cell attribute instead of the index of the cell to decide whether this
is a header or not in html.
Refactor the header generation.

Test: tabulator_test.py passes. It use the right tag for header in html too.

PRESUBMIT=passed
R=shenhan,asharif
APPROVED=shenhan
DELTA=90 (68 added, 3 deleted, 19 changed)
OCL=64118-p2
RCL=64556-p2
RDATE=2012/12/10 09:46:52


P4 change: 42801413
14/utils/tabulator.py
14/utils/tabulator_test.py
a6c0c02f0a6263dc5a083f3c1174c088fac3f9f5 19-Feb-2013 yunlian <yunlian@p4.invalid> Reduce the number of parameters in benchmark_run.py
Use benchmark instead of benchmark.name, benchmark.args etc.
Test = run crosperf with cached/non-cached, with perf/ no perf.
All worked.

PRESUBMIT=passed
R=asharif,shenhan,llozano,carrot
APPROVED=shenhan,asharif
DELTA=37 (11 added, 12 deleted, 14 changed)
OCL=64516-p2
RCL=64525-p2
RDATE=2012/12/07 16:30:59


P4 change: 42801359
14/crosperf/benchmark_run.py
14/crosperf/benchmark_run_unittest.py
14/crosperf/experiment.py
14/crosperf/results_organizer.py
ee6415a4d528d082ba01a4adab525c37c4e4c690 19-Feb-2013 asharif <asharif@p4.invalid> Changed gcc master branch to be a mirror of
branches/google/gcc-4_7-mobile.

PRESUBMIT=passed
R=shenhan,bjanakiraman
APPROVED=shenhan
DELTA=1 (0 added, 0 deleted, 1 changed)
RCL=64522-p2
RDATE=2012/12/07 15:20:57


P4 change: 42801355
14/repo_to_repo_files/gcc-master.rtr
00d18904a19086842f34056a38c46dca91f4b72a 19-Feb-2013 yunlian <yunlian@p4.invalid> USE ifconfig | grep lan as machine_id

Test: The machine_id is set properly.

PRESUBMIT=passed
R=asharif,shenhan
APPROVED=shenhan
DELTA=5 (3 added, 0 deleted, 2 changed)
OCL=64472-p2
RCL=64478-p2
RDATE=2012/12/05 17:02:52


P4 change: 42801239
14/crosperf/machine_manager.py
251b13b06ca0ee418491c958db6a98c27be7ce26 19-Feb-2013 yunlian <yunlian@p4.invalid> Highlight the number of completed test if the number is 0.

Test: tabulator_test.py passed

PRESUBMIT=passed
R=asharif,shenhan
APPROVED=asharif
DELTA=14 (13 added, 0 deleted, 1 changed)
OCL=64385-p2
RCL=64451-p2
RDATE=2012/12/04 15:14:09


P4 change: 42801189
14/utils/tabulator.py
7dc3e2325ada1a62a1bfacfdc626536f2ee1c12e 19-Feb-2013 asharif <asharif@p4.invalid> Made crosperf respect chromeos_root: in the image label.

This is useful if you just have an image lying around with a de-coupled
chromeos_root.

Did some internal clean-up as well (example, got rid of
self.chromeos_image and used self.label.chromeos_image instead in
benchmark_run.py).

Tested: Ran crosperf on a simple experiment file cached and uncached.
Tested it with an experiment file with a custom chromeos_root.

PRESUBMIT=passed
R=yunlian,bjanakiraman
APPROVED=yunlian
DELTA=23 (5 added, 5 deleted, 13 changed)
RCL=64450-p2
RDATE=2012/12/04 15:09:39


P4 change: 42801188
14/crosperf/benchmark_run.py
14/crosperf/label.py
14/crosperf/machine_manager.py
99fb6f15a3d201babd8e5b255579a63e3d06ef81 19-Feb-2013 yunlian <yunlian@p4.invalid> Improves the ETA a little bit.
Current it uses elapsed * (num_remaining_jobs) / num_finished.
It will increase until another job finishes.
The new one uses elapsed * (num_waiting_jobs) / num_finshed
+ (elapsed/num_finished - start_time of the running job)

Test: it does not increase any more.

PRESUBMIT=passed
R=shenhan,llozano
APPROVED=shenhan
DELTA=18 (16 added, 0 deleted, 2 changed)
OCL=64431-p2
RCL=64439-p2
RDATE=2012/12/03 13:49:11


P4 change: 42801163
14/crosperf/experiment_status.py
33ec1b3dbf15b0f0eed74415ee259663af0eccea 19-Feb-2013 shenhan <shenhan@p4.invalid> Added a new repo to repo file to propagate depot2 mobile toolchain v16 release to chromium git.

PRESUBMIT=passed
R=asharif
APPROVED=asharif
OCL=64417-p2
RCL=64437-p2
RDATE=2012/12/03 13:28:42


P4 change: 42801161
14/repo_to_repo_files/binutils-mobile_toolchain_v16.rtr
27d0d11d2e6e567d87444204da8b5f0a99ee0747 19-Feb-2013 asharif <asharif@p4.invalid> Fixed assertion when the noise value would be negative.

The noise should always be positive since we color it from 0 to
infinity.

Tested: python tabulator_test.py worked.

PRESUBMIT=passed
R=yunlian,glotov,bjanakiraman
APPROVED=yunlian
DELTA=1 (0 added, 0 deleted, 1 changed)
OCL=64423-p2
RCL=64432-p2
RDATE=2012/12/03 12:01:21


P4 change: 42801156
14/utils/tabulator.py
542744dfd0b5cb04f6ced308e975dfac8f78b7c1 19-Feb-2013 yunlian <yunlian@p4.invalid> Add execute permission on this test file

PRESUBMIT=passed
R=shenhan,asharif
APPROVED=shenhan
DELTA=0 (0 added, 0 deleted, 0 changed)
OCL=64314-p2
RCL=64322-p2
RDATE=2012/11/28 16:49:02


P4 change: 42800309
14/crosperf/machine_manager_unittest.py
0ea95bc8712ab973f25536bdd81bc4626979ac96 19-Feb-2013 yunlian <yunlian@p4.invalid> Clean up the ComputeMachineIDChecksum()
Test: it passed.

PRESUBMIT=passed
R=shenhan
APPROVED=shenhan
DELTA=3 (1 added, 2 deleted, 0 changed)
RCL=64321-p2
RDATE=2012/11/28 16:48:50


P4 change: 42800308
14/crosperf/machine_manager.py
9c1ddc8b97492aebe07e486e84a562f38bd0bf01 19-Feb-2013 yunlian <yunlian@p4.invalid> Add a paramenter to set the name of the label(default is keys).

Test: tabulator_test.py passed.
tabulator.py works fine

PRESUBMIT=passed
R=asharif,shenhan
APPROVED=shenhan,asharif
DELTA=3 (1 added, 0 deleted, 2 changed)
OCL=64309-p2
RCL=64318-p2
RDATE=2012/11/28 16:21:03


P4 change: 42800306
14/utils/tabulator.py
f1dd74aa5cf30ab2c90ec992d98c230e71df798d 19-Feb-2013 yunlian <yunlian@p4.invalid> Add same_specs flag to crosperf to replace exact_remote flag
Add same_machine flag. When using flag It will only uses the cache
if the uuid of hard drive matches.

Test: now the cached file name contains the hash of uuid of hard drive.
The two flags works as intended.

PRESUBMIT=passed
BUG=7478205
R=asharif,shenhan
CC=c-compiler-chrome
APPROVED=asharif
DELTA=60 (35 added, 6 deleted, 19 changed)
OCL=63823-p2
RCL=64310-p2
RDATE=2012/11/28 16:01:17


P4 change: 42800284
14/crosperf/benchmark_run.py
14/crosperf/experiment_factory.py
14/crosperf/machine_manager.py
14/crosperf/results_cache.py
14/crosperf/settings_factory.py
77dc571e143ef80682793bf64ab2c57f6524db53 19-Feb-2013 yunlian <yunlian@p4.invalid> Change the default behavior of lock_machine to auto when
called in other scripts. The ./lock_machine behavior remains unchanged.

Test:lock_machine_test.py passed

PRESUBMIT=passed
R=llozano,shenhan
APPROVED=shenhan
DELTA=10 (0 added, 7 deleted, 3 changed)
OCL=64169-p2
RCL=64172-p2
RDATE=2012/11/20 14:36:19


P4 change: 42799569
14/lock_machine.py
b0eaee8a0cf887ee38fbce5a51ecab2dcb25273c 19-Feb-2013 yunlian <yunlian@p4.invalid> Fix a bug in Unlocking auto-release lock.
When unlocking a auto-release lock, it will close the check liveness
file. However, if other process unlocks this lock, it can not find
the file descriptor in its own list. So it does nothing and the lock
on the check liveness file still exists. At this time, if some one try
to lock the same machine again, it will pops out an error.

Test: the newly added test case in lock_machine_test.py passed.

PRESUBMIT=passed
R=llozano,shenhan
APPROVED=llozano
DELTA=18 (15 added, 0 deleted, 3 changed)
OCL=64083-p2
RCL=64139-p2
RDATE=2012/11/19 11:16:27


P4 change: 42799522
14/lock_machine.py
14/lock_machine_test.py
65cc335631d6800f50c82f50fd95c5acf23c5b11 19-Feb-2013 shenhan <shenhan@p4.invalid> Add '--cleanup' suuport to nightly client, which I missed in previous CLs.

(This is just a simple CL that passes argument around.

PRESUBMIT=passed
R=llozano
CC=asharif,yunlian,cmtice,bjanakiraman,carrot
APPROVED=llozano
OCL=64088-p2
RCL=64138-p2
RDATE=2012/11/19 10:41:32
TEST=locally


P4 change: 42799521
14/automation/clients/dejagnu_compiler.py
c0c0b35bc1a1804cad25a8ebccfa39e5957e4dd7 19-Feb-2013 shenhan <shenhan@p4.invalid> Parsing the validate_failures.py so we display the failure number in the email.

Currently, there will be always failures, it's easier to compare the numbers in different emails.

To make review a little bit easier, the added functionality just tries to parse text like below -
Source directory: /usr/local/toolchain_root/gcc
Build target: i686-pc-linux-gnu
Manifest: /usr/local/toolchain_root/gcc/contrib/testsuite-management/i686-pc-linux-gnu.xfail
Getting actual results from build
/usr/local/toolchain_root/gcc-build-i686-pc-linux-gnu/gcc/testsuite/g++/g++.sum
/usr/local/toolchain_root/gcc-build-i686-pc-linux-gnu/gcc/testsuite/gcc/gcc.sum


Build results not in the manifest
UNRESOLVED: g++.dg/tree-prof/callgraph-profiles.C scan-file .text.*.main\n.text.*._Z3barv\n.text.*._Z3foov\n.text.*._Z9notcalledv
UNRESOLVED: g++.dg/tree-prof/callgraph-profiles.C scan-file Callgraph group : main _Z3barv _Z3foov\n
FAIL: g++.dg/tree-prof/mversn15.C scan-tree-dump optimized "return 0"
FAIL: g++.dg/tree-prof/mversn15.C scan-tree-dump optimized "main_clone"
FAIL: g++.dg/warn/Wself-assign-2.C -std=gnu++11 (test for warnings, line 12)
FAIL: gcc.dg/builtin-unreachable-3.c (test for excess errors)
FAIL: gcc.dg/builtin-unreachable-3.c (internal compiler error)
FAIL: gcc.dg/cproj-fails-with-broken-glibc.c execution test
FAIL: gcc.dg/pr45353.c (test for excess errors)
FAIL: gcc.dg/pr45353.c (internal compiler error)
FAIL: gcc.dg/thread_annot_lock-23.c (test for warnings, line 10)
FAIL: gcc.dg/thread_annot_lock-23.c (test for warnings, line 18)
FAIL: gcc.dg/thread_annot_lock-23.c (test for warnings, line 19)
FAIL: gcc.dg/thread_annot_lock-23.c (test for warnings, line 15)
FAIL: gcc.dg/thread_annot_lock-23.c (test for warnings, line 13)
FAIL: gcc.dg/thread_annot_lock-24.c (test for warnings, line 9)
FAIL: gcc.dg/thread_annot_lock-24.c (test for warnings, line 10)
FAIL: gcc.dg/thread_annot_lock-24.c (test for warnings, line 8)
FAIL: gcc.dg/thread_annot_lock-25.c (test for warnings, line 22)
FAIL: gcc.dg/thread_annot_lock-25.c (test for warnings, line 8)
FAIL: gcc.dg/thread_annot_lock-25.c (test for warnings, line 30)
FAIL: gcc.dg/thread_annot_lock-25.c (test for warnings, line 10)
FAIL: gcc.dg/thread_annot_lock-25.c (test for warnings, line 9)
FAIL: gcc.dg/thread_annot_lock-42.c (test for warnings, line 9)
FAIL: gcc.dg/torture/pr41094.c -O0 execution test
FAIL: gcc.dg/torture/pr51106-2.c -O2 -flto -fno-use-linker-plugin -flto-partition=none (test for excess errors)
FAIL: gcc.dg/torture/pr51106-2.c -O2 (test for excess errors)
FAIL: gcc.dg/torture/pr51106-2.c -O0 (test for excess errors)
FAIL: gcc.dg/torture/pr51106-2.c -O1 (test for excess errors)
FAIL: gcc.dg/torture/tls/run-ld.c -O3 -fomit-frame-pointer -pie -fpie execution test
FAIL: gcc.dg/torture/tls/run-ld.c -O1 -pie -fpie execution test
FAIL: gcc.dg/torture/tls/run-ld.c -O3 -fomit-frame-pointer -pie -fPIE execution test
FAIL: gcc.dg/torture/tls/run-ld.c -O2 -pie -fPIE execution test
FAIL: gcc.dg/torture/tls/run-ld.c -O1 -pie -fPIE execution test
FAIL: gcc.dg/torture/tls/run-ld.c -O3 -g -pie -fPIE execution test
FAIL: gcc.dg/torture/tls/run-ld.c -O3 -g -pie -fpie execution test
FAIL: gcc.dg/torture/tls/run-ld.c -Os -pie -fPIE execution test
FAIL: gcc.dg/torture/tls/run-ld.c -Os -pie -fpie execution test
FAIL: gcc.dg/torture/tls/run-ld.c -O2 -pie -fpie execution test
XPASS: gcc.dg/unroll_2.c (test for excess errors)
XPASS: gcc.dg/unroll_3.c (test for excess errors)
XPASS: gcc.dg/unroll_4.c (test for excess errors)
FAIL: gcc.target/i386/pr50603.c scan-assembler-not lea[lq]

PRESUBMIT=passed
R=llozano,yunlian
CC=asharif,cmtice,bjanakiraman
APPROVED=llozano
OCL=64048-p2
RCL=64087-p2
RDATE=2012/11/15 14:27:25
TEST=locally


P4 change: 42799356
14/test_gcc_dejagnu.py
a39af9b0b7224c9b98ee9a7581832d347931ce8b 19-Feb-2013 yunlian <yunlian@p4.invalid> Postpone the result CleanUp after the perf data being copied back
Test: The perf data can be copied back now

PRESUBMIT=passed
R=shenhan,llozano
APPROVED=shenhan
DELTA=8 (6 added, 1 deleted, 1 changed)
OCL=64057-p2
RCL=64064-p2
RDATE=2012/11/14 16:33:19


P4 change: 42799304
14/crosperf/experiment_runner.py
14/crosperf/results_cache.py
8c9419ba19bb7c5a39f6610dd7d3fc2f055daefd 19-Feb-2013 yunlian <yunlian@p4.invalid> Support quit-unlock in lock_machine.py.
Add support for the case when the program exits, all the machine
it lockes will be uplocked. It also maintains the functionality that
you can lock it manually and you have to lock it manually too.
It does not increase the race chance compared with the original one.
It just enlarges the critical section length.

Test: lock_machine_test.py passed.
The quit-then-unlock lock works as intended.

PRESUBMIT=passed
R=asharif,llozano,shenhan
APPROVED=asharif
DELTA=129 (116 added, 1 deleted, 12 changed)
OCL=63805-p2
RCL=64061-p2
RDATE=2012/11/14 16:09:23


P4 change: 42799301
14/lock_machine.py
14/lock_machine_test.py
4f5dbb3cd9a537689955c47a854c115c0ecd3ed2 19-Feb-2013 yunlian <yunlian@p4.invalid> Add perf_diff to crosperf

Test: perf_diff works with sample report

PRESUBMIT=passed
R=asharif,shenhan,llozano
APPROVED=asharif
DELTA=343 (322 added, 7 deleted, 14 changed)
OCL=63852-p2
RCL=64029-p2
RDATE=2012/11/13 16:10:40


P4 change: 42799246
14/utils/misc.py
14/utils/perf_diff.py
14/utils/tabulator.py
c2fb3c21378adcbfc660f05960a35824eb2d7a5c 19-Feb-2013 shenhan <shenhan@p4.invalid> Delete testing directory even if test fails. (Currently dejagnu always fails.)

PRESUBMIT=passed
BUG=None
R=asharif
CC=c-compiler-chrome
APPROVED=asharif
RCL=64024-p2
RDATE=2012/11/13 14:45:00
TEST=None


P4 change: 42799243
14/automation/clients/dejagnu_compiler.py
a55d6907227d5196a8a7614453ed495cdaec1438 19-Feb-2013 asharif <asharif@p4.invalid> Made command executer respect print_to_console flag.

Tested: Tested with some scripts that used CrosRunCommand().

PRESUBMIT=passed
R=cmtice,bjanakiraman
DELTA=18 (9 added, 0 deleted, 9 changed)
OCL=63998-p2
RCL=64019-p2
RDATE=2012/11/13 12:15:04


P4 change: 42799227
14/utils/command_executer.py
fe79bcc1b8abbcdc4db35de0a962e968fff81bb3 19-Feb-2013 shenhan <shenhan@p4.invalid> Added support to try lock multiple machines.

Implementation is pretty straightforward, simply split '--remote' and try one at a time to acquire the machine.

PRESUBMIT=passed
BUG=None
R=yunlian,llozano,asharif
CC=asharif,bjanakiraman,cmtice,carrot
APPROVED=llozano
OCL=63997-p2
RCL=64013-p2
RDATE=2012/11/12 22:12:57
DIFFBASE=63734
TEST=run it manuaylly


P4 change: 42799217
14/dejagnu/run_dejagnu.py
c60576fc02f9a747b0c3f3a86655f37f45f58fc4 19-Feb-2013 shenhan <shenhan@p4.invalid> Adding support for cleanup the mount point, chroot and chromeos root.

Also added ".gitignore" to filter out "logs" directory, which is pretty annoying show in "git status".

PRESUBMIT=passed
BUG=None
R=asharif
CC=bjanakiraman,cmtice,llozano,yunlian
APPROVED=asharif
OCL=63734-p2
RCL=64007-p2
RDATE=2012/11/12 16:05:33
TEST=run_dejagnu with '--cleanup' option


P4 change: 42799192
14/.gitignore
14/dejagnu/run_dejagnu.py
14/test_gcc_dejagnu.py
0e9ba56ba94c55f2fab755f628050ca9a1cde00c 19-Feb-2013 yunlian <yunlian@p4.invalid> Deal with the overflow of gmean

Test: Run sample case, the non-overlow case remains the same result
The overflowed one does not overlow now

PRESUBMIT=passed
R=asharif,shenhan
APPROVED=shenhan
DELTA=10 (9 added, 0 deleted, 1 changed)
OCL=63993-p2
RCL=64004-p2
RDATE=2012/11/12 14:39:36


P4 change: 42799191
14/utils/tabulator.py
14/utils/tabulator_test.py
7b724dd4fb5e62e18769d9ba480fb3c014e8659b 19-Feb-2013 yunlian <yunlian@p4.invalid> Handle the bootPerfServer case, accumulate values like a{1}, a{2}.
Test: It handles the case properly

PRESUBMIT=passed
BUG=7487245
R=asharif,shenhan,llozano
APPROVED=shenhan
DELTA=60 (55 added, 1 deleted, 4 changed)
OCL=63859-p2
RCL=63960-p2
RDATE=2012/11/09 16:53:54


P4 change: 42797655
14/crosperf/benchmark.py
14/crosperf/results_organizer.py
14/crosperf/results_report.py
edeaab0762f134e0e66f55c3206108301ea416d7 19-Feb-2013 yunlian <yunlian@p4.invalid> Do not try to copy perf result if the run is cached already

Test: The copy is avoided

PRESUBMIT=passed
R=shenhan,llozano
APPROVED=shenhan
DELTA=8 (4 added, 2 deleted, 2 changed)
OCL=63886-p2
RCL=63949-p2
RDATE=2012/11/09 10:13:50


P4 change: 42797631
14/crosperf/experiment_runner.py
becf6242e3b52b6d95a41c0a1da9cf9b633cb18e 19-Feb-2013 shenhan <shenhan@p4.invalid> Properly return exit codes to indicate build_tc failure.

This is useful for other scripts depending on this script, for example, test_gcc_dejagnu.py.

PRESUBMIT=passed
R=asharif
CC=bjanakiraman,cmtice,llozano,yunlian
APPROVED=yunlian
OCL=63855-p2
RCL=63948-p2
RDATE=2012/11/09 10:08:26


P4 change: 42797630
14/build_tc.py
29b88afbbe5d84a24ea7b63c1cd8fc615429b6f6 19-Feb-2013 yunlian <yunlian@p4.invalid> Remove perf report from console.

Test: perf report is removed

PRESUBMIT=passed
BUG=7491195
R=asharif,shenhan,carrot
APPROVED=shenhan
DELTA=13 (1 added, 3 deleted, 9 changed)
OCL=63876-p2
RCL=63878-p2
RDATE=2012/11/07 11:46:50


P4 change: 42797507
14/crosperf/results_cache.py
0e80ac970e0948d7482094252e57c0cc54959dcd 19-Feb-2013 yunlian <yunlian@p4.invalid> Make experiment_factory_unittest.py and crosperf_test.py pass
Test: both test passed

PRESUBMIT=passed
R=asharif,llozano,shenhan
APPROVED=shenhan
DELTA=76 (54 added, 1 deleted, 21 changed)
OCL=63757-p2
RCL=63815-p2
RDATE=2012/11/05 11:22:39


P4 change: 42797409
14/crosperf/benchmark_run_unittest.py
14/crosperf/crosperf.py
14/crosperf/experiment.py
14/crosperf/experiment_factory.py
14/crosperf/experiment_factory_unittest.py
14/crosperf/machine_manager.py
14/crosperf/machine_manager_unittest.py
14/crosperf/test_flag.py
df84fbfb60eb1d64b1dcc68bfb98920956d3967c 19-Feb-2013 yunlian <yunlian@p4.invalid> Add pstat module

PRESUBMIT=passed
R=llozano,shenhan
APPROVED=shenhan
DELTA=1068 (1068 added, 0 deleted, 0 changed)
RCL=63812-p2
RDATE=2012/11/05 11:17:31


P4 change: 42797401
14/utils/pstat.py
0b5d5c8265089daee2307759f0adb6762da37b6a 19-Feb-2013 asharif <asharif@p4.invalid> Pass in --force-mismatch to test_toolchains.py.

Tested: This will be tested by the nightly job.

PRESUBMIT=passed
R=cmtice,bjanakiraman
CC=c-compiler-chrome
DELTA=2 (1 added, 0 deleted, 1 changed)
OCL=63713-p2
RCL=63773-p2
RDATE=2012/11/02 14:56:50


P4 change: 42797324
14/automation/clients/chromeos.py
14/test_toolchains.py
d02d6d215cf4a53ac7addea75099666ba3356e69 19-Feb-2013 yunlian <yunlian@p4.invalid> Fix benchmark_run_unittest.py
Test: benchmark_run_unittest.py passes

PRESUBMIT=passed
R=asharif,shenhan
APPROVED=asharif
DELTA=15 (7 added, 0 deleted, 8 changed)
OCL=63744-p2
RCL=63747-p2
RDATE=2012/11/01 16:57:17


P4 change: 42797274
14/crosperf/benchmark_run_unittest.py
14/crosperf/label.py
14/crosperf/machine_manager_unittest.py
8a3bdcbd3f8e528dbb0b76e144af3f71dda55711 19-Feb-2013 yunlian <yunlian@p4.invalid> Use json instead of picke to serialize LockDescription
Test: lock_machine_test.py passed
manully lock, unlock, lock_machine.py --list passes

PRESUBMIT=passed
R=asharif,shenhan,llozano
APPROVED=asharif
DELTA=28 (14 added, 1 deleted, 13 changed)
OCL=63725-p2
RCL=63731-p2
RDATE=2012/11/01 12:28:10


P4 change: 42797254
14/lock_machine.py
b7ff88bbe3d4db125d1492b9ae7027f2365f4565 19-Feb-2013 shenhan <shenhan@p4.invalid> Fix permission problems during "cros_sdk --bootstrap" and change interpreter to "python" for gPrecise.

The origin thread is here = http://code.google.com/p/chromium-os/issues/detail?id=31837

The fix is here - https://sites.google.com/a/chromium.org/dev/chromium-os/developer-guide?pli=1#TOC-Verify-that-your-default-file-permissions-umask-setting-is-correct

PRESUBMIT=passed
BUG=None
R=yunlian,asharif
CC=cmtice,llozano,bjanakiraman
APPROVED=asharif
OCL=63714-p2
RCL=63724-p2
RDATE=2012/11/01 09:29:53
TEST=Setup chromeos and `cros_sdk --bootstrap`, under gPrecise.


P4 change: 42797250
14/setup_chromeos.py
9213b83a87c168042bbdf148aa39e56b753385ca 19-Feb-2013 shenhan <shenhan@p4.invalid> Added support for building chrome browser against an external source.

This was supported via option '--cros_workon=<chromium browser checkout directory>'.

PRESUBMIT=passed
R=asharif
CC=bjanakiraman,cmtice,llozano,yunlian
APPROVED=asharif
OCL=63596-p2
RCL=63715-p2
RDATE=2012/10/31 15:37:36
TEST=Locally


P4 change: 42797240
14/build_chrome_browser.py
14/utils/command_executer.py
58a8c9fc908ccacb91c2557b9b10753ce3908785 19-Feb-2013 asharif <asharif@p4.invalid> Enable crosperf to image machines that have a different board than the
image.

Tested: Tested with an amd64 image that I was able to use on a lumpy
box.

PRESUBMIT=passed
R=yunlian,llozano,bjanakiraman,cmtice
APPROVED=yunlian
DELTA=36 (22 added, 0 deleted, 14 changed)
OCL=63673-p2
RCL=63708-p2
RDATE=2012/10/31 09:18:45


P4 change: 42797217
14/crosperf/benchmark_run.py
14/crosperf/experiment_factory.py
14/crosperf/label.py
14/crosperf/machine_manager.py
14/crosperf/settings_factory.py
14/test_toolchains.py
2a60698de6e9d04c04b24d98489acf95effd6388 19-Feb-2013 yunlian <yunlian@p4.invalid> Add default remotes for each board.
Users do not need to type remotes if they want to use
the default machine

Test: run crosperf without setting the remote, it works fine

PRESUBMIT=passed
R=asharif,shenhan,cmtice
DELTA=36 (35 added, 0 deleted, 1 changed)
OCL=63652-p2
RCL=63681-p2
RDATE=2012/10/29 14:46:00


P4 change: 42797170
14/crosperf/default_remotes
14/crosperf/experiment_factory.py
14/crosperf/settings_factory.py
ab18be48b2b8685a49765ed89ecf1896a3dcdca7 19-Feb-2013 yunlian <yunlian@p4.invalid> Make the benchmark_run_unittest.py and experiment_file_unittest.py pass

Test: These two test passed

PRESUBMIT=passed
R=asharif,llozano,shenhan
APPROVED=shenhan
DELTA=99 (69 added, 5 deleted, 25 changed)
OCL=63647-p2
RCL=63680-p2
RDATE=2012/10/29 14:45:36


P4 change: 42797169
14/crosperf/benchmark_run.py
14/crosperf/benchmark_run_unittest.py
14/crosperf/experiment_file_unittest.py
14/crosperf/label.py
14/crosperf/machine_manager.py
14/crosperf/results_cache.py
21474abe80123147f794da82fffe15f7fa925217 19-Feb-2013 yunlian <yunlian@p4.invalid> Add machine_manager_unittest.py

Test: The unittest passed

PRESUBMIT=passed
R=asharif,shenhan,llozano
APPROVED=shenhan
DELTA=61 (61 added, 0 deleted, 0 changed)
OCL=63651-p2
RCL=63679-p2
RDATE=2012/10/29 14:24:16


P4 change: 42797167
14/crosperf/machine_manager_unittest.py
88631db994fda485779088b39cc7efde5c9911d2 19-Feb-2013 yunlian <yunlian@p4.invalid> Print out cpuinfo at the email.
Test: It prints out the cpuinfo

PRESUBMIT=passed
R=asharif,shenhan
APPROVED=shenhan
DELTA=75 (47 added, 5 deleted, 23 changed)
OCL=63599-p2
RCL=63641-p2
RDATE=2012/10/26 11:29:42


P4 change: 42797104
14/crosperf/machine_manager.py
14/crosperf/results_report.py
e0a6d8a7f495e5a39df6a084a5c18ca8745d4890 19-Feb-2013 shenhan <shenhan@p4.invalid> Added machine lock around testing run, also includes a minor fix for email address and the change to test label.

TEST:
a) - Locked a specific machine, invoke run_dejagnu with the same machine specified, timedout, failed.
b) - Locked a specific machine, invoke run_dejagnu with the same machine specified, during retrying period, manually unlock the machine, run_dejagnu proceeds.

PRESUBMIT=passed
TBR=asharif
CC=bjanakiraman,cmtice,llozano,yunlian
OCL=63595-p2
RCL=63638-p2
RDATE=2012/10/26 09:27:15


P4 change: 42797101
14/automation/clients/dejagnu_compiler.py
14/dejagnu/run_dejagnu.py
14/lock_machine.py
14/test_gcc_dejagnu.py
2b856607ec6f3c7d4c85990890bb700d7641e54c 19-Feb-2013 yunlian <yunlian@p4.invalid> Support different board in a single crosperf

Test: run crosperf with

benchmark: LibCBench{
iterations: 2
}

label: x86_alex{
remote: 172.17.128.195
board: x86-alex
chromeos_image: /home/yunlian/extend/alpha/src/build/images/x86-alex/latest/chromiumos_test_image.bin
}

label: lumpy {
remote:172.17.128.244
board:lumpy
chromeos_image: /home/yunlian/extend/alpha/src/build/images/lumpy/latest/chromiumos_test_image.bin
}

And it passes.

PRESUBMIT=passed
R=asharif,shenhan,llozano,bjanakiraman
APPROVED=shenhan
DELTA=55 (18 added, 7 deleted, 30 changed)
OCL=63455-p2
RCL=63635-p2
RDATE=2012/10/26 09:22:10


P4 change: 42797098
14/crosperf/benchmark_run.py
14/crosperf/experiment.py
14/crosperf/experiment_factory.py
14/crosperf/label.py
14/crosperf/results_cache.py
14/crosperf/results_organizer.py
14/crosperf/settings_factory.py
e178da7afe0249fa07c1757086bdd643b4ae7838 19-Feb-2013 asharif <asharif@p4.invalid> Added workaround for crosbug.com/35684.

Tested: chmod a+rwx <key file>
Ran image_chromeos.py.
ls -hlt <key file> shows 0600 permissions.

PRESUBMIT=passed
R=yunlian,bjanakiraman
APPROVED=shenhan
DELTA=11 (11 added, 0 deleted, 0 changed)
RCL=63594-p2
RDATE=2012/10/24 15:18:35


P4 change: 42797036
14/image_chromeos.py
14/utils/misc.py
cd42ad72df43675a92f56113ada047b4bda17bf1 19-Feb-2013 yunlian <yunlian@p4.invalid> Add repo_to_repo script for crosperf on chromiumos
Test: The dry-run works correct.

PRESUBMIT=passed
R=asharif
APPROVED=asharif
DELTA=31 (31 added, 0 deleted, 0 changed)
OCL=63390-p2
RCL=63573-p2
RDATE=2012/10/23 17:12:40


P4 change: 42796987
14/repo_to_repo_files/crosperf.rtr
442c8ce429001676f6eefa1051334f07eb1453e0 19-Feb-2013 yunlian <yunlian@p4.invalid> Use lock_machine.py to do the lock at image_chromeos

Test: Run Two crosperf at the same time. They do image_chromeos
sequentially. The lock is locked/released as expected.
Run one crosperf. Press Contol-C in the imaging stage, the
lock is released.

PRESUBMIT=passed
R=asharif,llozano,shenhan
APPROVED=asharif
DELTA=63 (48 added, 12 deleted, 3 changed)
OCL=63491-p2
RCL=63572-p2
RDATE=2012/10/23 17:12:00


P4 change: 42796986
14/image_chromeos.py
14/utils/misc.py
f844b43322886e53b616af67c1479371e0cb7744 19-Feb-2013 shenhan <shenhan@p4.invalid> Added files for dejagnu nighly job.

test_gcc_dejagnu.py - a wrapper around existing gcc dejagnu runner.
dejagnu_compiler.py - the nightly client

PRESUBMIT=passed
R=asharif
CC=cmtice,llozano,yunlian,bjanakiraman
APPROVED=asharif
OCL=63286-p2
RCL=63544-p2
RDATE=2012/10/22 16:49:46
TEST=locally, need to submit before testing it on real environment.


P4 change: 42796939
14/automation/clients/dejagnu_compiler.py
14/test_gcc_dejagnu.py
52a6c57e559e4dfaf687596728df83a282842da1 19-Feb-2013 yunlian <yunlian@p4.invalid> Allow use to set the lock_dirs for lock_machine
Change the interface of FileLock to make it accept only
one parameter

Test: The lock_machine_test.py passes

PRESUBMIT=passed
R=asharif,llozano
APPROVED=asharif
DELTA=38 (17 added, 3 deleted, 18 changed)
OCL=63488-p2
RCL=63543-p2
RDATE=2012/10/22 13:44:07


P4 change: 42796938
14/crosperf/machine_manager.py
14/lock_machine.py
14/lock_machine_test.py
14/utils/logger.py
cc8daaffebc589ceac40039e323842a62fcce725 19-Feb-2013 asharif <asharif@p4.invalid> Added check to see whether any value is negative before computing the
gmean.

Tested: python tabulator.py. python tabulator_test.py.

PRESUBMIT=passed
R=shenhan,yunlian
APPROVED=yunlian
DELTA=2 (2 added, 0 deleted, 0 changed)
OCL=63536-p2
RCL=63541-p2
RDATE=2012/10/22 12:37:39


P4 change: 42796937
14/utils/tabulator.py
2694550b2c5c0339bb277476b36dc9ce7ad89e19 19-Feb-2013 asharif <asharif@p4.invalid> Removed logging of cached output and error to the console.

I noticed minutes of time are wasted on printing the cached output to
the console especially if the number of runs is large. In one instance,
my machine was under heavy load for 5+ minutes when it was producing
output for ~140 completed autotests.

This CL causes crosperf to no longer print on the console the cached run
output.

It also causes the stderr to go to the *.err file properly.

Tested: Ran crosperf with some cached results and found it much faster
than before.

PRESUBMIT=passed
R=yunlian,bjanakiraman
APPROVED=yunlian
DELTA=2 (1 added, 0 deleted, 1 changed)
OCL=63443-p2
RCL=63540-p2
RDATE=2012/10/22 11:47:35


P4 change: 42796921
14/crosperf/benchmark_run.py
5836e6cf49ec8134018ffb1dbe08648682016ebc 19-Feb-2013 asharif <asharif@p4.invalid> Made execution of crosperf more hermetic.

This CL puts the v14 dir before other paths so we make sure the one
checked out with the current crosperf is the one that is used over
others in PYTHONPATH.

PRESUBMIT=passed
R=shenhan,bjanakiraman
DELTA=1 (0 added, 0 deleted, 1 changed)
OCL=63537-p2
RCL=63539-p2
RDATE=2012/10/22 11:47:27


P4 change: 42796920
14/crosperf/crosperf
f783b7d755c7d20440c06ec0cca4ef78933850d4 19-Feb-2013 asharif <asharif@p4.invalid> Added support for bg coloring a row.

Used that support to give a yellow highlight to a row where the p-value
is less than 0.05. This makes reading results a lot easier.

Tested: crosperf on cached results.
python tabulator.py runs properly
python tabulator_test.py passes.

PRESUBMIT=passed
R=yunlian,bjanakiraman
APPROVED=yunlian
DELTA=58 (39 added, 11 deleted, 8 changed)
OCL=63459-p2
RCL=63535-p2
RDATE=2012/10/22 10:59:02


P4 change: 42796918
14/crosperf/results_report.py
14/utils/tabulator.py
79e8953fe3222fa4c30ba77e1a410e02ea705b10 19-Feb-2013 yunlian <yunlian@p4.invalid> Add number of completed runs in the simple table.

Test: the column shows up

PRESUBMIT=passed
R=asharif
APPROVED=asharif
DELTA=4 (3 added, 0 deleted, 1 changed)
RCL=63456-p2
RDATE=2012/10/18 11:26:40


P4 change: 42796333
14/crosperf/results_report.py
b074ac7c3ee51864dfbe2e71d4a1a3362ff4e66b 19-Feb-2013 asharif <asharif@p4.invalid> Changed image_chromeos.py to use ./bin/cros_image_to_target.py.

cros_image_to_target.py doesn't require communication back from DUT to
the host machine and works when trying to image boxes in the lab.

Also added a delay before writing the checksum file because some
machines take a long time to be back up again after the re-image.

Tested: python image_chromeos.py --image=<image> ... works.

PRESUBMIT=passed
R=cmtice
APPROVED=bjanakiraman
DELTA=49 (38 added, 0 deleted, 11 changed)
OCL=63242-p2
RCL=63430-p2
RDATE=2012/10/16 17:44:07


P4 change: 42796302
14/image_chromeos.py
fd8e57b0385f531331c01dee196d63145b1d74ee 19-Feb-2013 asharif <asharif@p4.invalid> Log and return an error message if the machine is down when
CrosRunCommand() is called.

Tested: Called CrosRunCommand() on a disconnected machine and it logged
the error message and returned non-zero value.

PRESUBMIT=passed
R=yunlian,bjanakiraman
DELTA=10 (4 added, 0 deleted, 6 changed)
OCL=63406-p2
RCL=63429-p2
RDATE=2012/10/16 17:44:00


P4 change: 42796301
14/utils/command_executer.py
f08a90e87e41f16247b9c2514849f2c94aa9085a 19-Feb-2013 yunlian <yunlian@p4.invalid> Add benchmark run status at the email

Test: The status table is back

PRESUBMIT=passed
R=asharif
APPROVED=asharif
DELTA=23 (23 added, 0 deleted, 0 changed)
OCL=63421-p2
RCL=63425-p2
RDATE=2012/10/16 16:37:20


P4 change: 42796298
14/crosperf/results_report.py
ea0beec55ce85743a976d3a1b13919352deb236e 19-Feb-2013 yunlian <yunlian@p4.invalid> Only record the num_reimage when the image is successful.

PRESUBMIT=passed
R=asharif
APPROVED=asharif
DELTA=3 (2 added, 1 deleted, 0 changed)
OCL=63419-p2
RCL=63422-p2
RDATE=2012/10/16 16:18:48


P4 change: 42796294
14/crosperf/machine_manager.py
2950a3fbfe0fe04dcfed9b5ba921567625fb5071 19-Feb-2013 yunlian <yunlian@p4.invalid> Do not use cached_result runs to calculate the ETA

Test: run cros_perf, the cached runs are not used when computing ETA

PRESUBMIT=passed
BUG=7360022
R=asharif
APPROVED=asharif
DELTA=4 (3 added, 0 deleted, 1 changed)
RCL=63420-p2
RDATE=2012/10/16 15:57:12


P4 change: 42796293
14/crosperf/experiment.py
14/crosperf/experiment_status.py
a94e9e02b5e71af085e76dcacba8602ff551b9c5 19-Feb-2013 yunlian <yunlian@p4.invalid> Add GetLastEventTime() to timeline.py
Show the times since last event in the banner of crosperf

Test: python timeline_test.py passed
crosperf run shows the time

PRESUBMIT=passed
R=asharif
APPROVED=asharif
DELTA=48 (30 added, 4 deleted, 14 changed)
OCL=63348-p2
RCL=63375-p2
RDATE=2012/10/15 11:24:41


P4 change: 42796142
14/crosperf/benchmark_run.py
14/crosperf/experiment_status.py
14/crosperf/machine_manager.py
14/utils/timeline.py
14/utils/timeline_test.py
e0f4e38878daf48a0f7299b5ff7feca21567d18a 19-Feb-2013 yunlian <yunlian@p4.invalid> Change a lable from Mean/StdDev to StdDev/Mean

PRESUBMIT=passed
R=asharif,shenhan
APPROVED=asharif
DELTA=1 (0 added, 0 deleted, 1 changed)
RCL=63374-p2
RDATE=2012/10/15 11:24:21


P4 change: 42796141
14/crosperf/results_report.py
c742d992e0dc413903e44aa5800c1449da026a45 19-Feb-2013 yunlian <yunlian@p4.invalid> Add timeline class to support debug
Test: unittest python timeline_test.py passed

PRESUBMIT=passed
R=asharif
APPROVED=asharif
DELTA=86 (86 added, 0 deleted, 0 changed)
OCL=63344-p2
RCL=63346-p2
RDATE=2012/10/12 12:37:35


P4 change: 42796039
14/utils/timeline.py
14/utils/timeline_test.py
55be177c290b582a425fbceba2d5d40feaec94db 19-Feb-2013 yunlian <yunlian@p4.invalid> Print number of re-images in the email

Test: It does print out the number of re-images

PRESUBMIT=passed
R=asharif
APPROVED=asharif
DELTA=5 (5 added, 0 deleted, 0 changed)
OCL=63301-p2
RCL=63332-p2
RDATE=2012/10/11 14:19:52


P4 change: 42795365
14/crosperf/results_report.py
fce8042f2c8be15a857097dd62919b2e8da2c385 19-Feb-2013 asharif <asharif@p4.invalid> Automated g4 rollback of changelist 63276.

*** Reason for rollback ***

Nightly jobs are failing because 6 iterations take too long to complete.

*** Original change description ***

Changed the nightly to use 6 iterations instead of 1.

Tested: The nightly will test this.

PRESUBMIT=passed
R=llozano,bjanakiraman
APPROVED=llozano
DELTA=1 (0 added, 0 deleted, 1 changed)
OCL=63317-p2
RCL=63322-p2
RDATE=2012/10/11 10:58:08


P4 change: 42795341
14/test_toolchains.py
e59ac277bb27f54d16e97af5af58f37963da1ef6 19-Feb-2013 cmtice <cmtice@p4.invalid> Add three new options to this script: --cflags, --cxxflags, and
--ldflags.

PRESUBMIT=passed
R=asharif,llozano,shenhan,yunlian,bjanakiraman
APPROVED=bjanakiraman,asharif
DELTA=28 (25 added, 0 deleted, 3 changed)
OCL=63295-p2
RCL=63304-p2
RDATE=2012/10/10 16:46:03


P4 change: 42795320
14/build_tc.py
579723021480d6edbdac78e2b6560fac35103806 19-Feb-2013 yunlian <yunlian@p4.invalid> Remove &nsbp; to avoid the scroll bar

Test: run the crosperf, the scroll bar is gone

PRESUBMIT=passed
BUG=7318538
R=asharif
APPROVED=asharif
DELTA=1 (0 added, 0 deleted, 1 changed)
OCL=63297-p2
RCL=63300-p2
RDATE=2012/10/10 14:27:31


P4 change: 42795307
14/utils/tabulator.py
8a8ce9665120a6fd150c88697bb869580449f6f6 19-Feb-2013 asharif <asharif@p4.invalid> Changed the nightly to use 6 iterations instead of 1.

Tested: The nightly will test this.

PRESUBMIT=passed
R=llozano,bjanakiraman
APPROVED=llozano
DELTA=1 (0 added, 0 deleted, 1 changed)
OCL=63264-p2
RCL=63276-p2
RDATE=2012/10/09 19:37:23


P4 change: 42795286
14/test_toolchains.py
464c10bec65239c5217db577c56f1c4292f0d265 19-Feb-2013 shenhan <shenhan@p4.invalid> Ensure necessary mount points before executing chroot commands.

Also includes the following change
- changed to return value to a 3 or 4 tuple;
- added a '__init__.py'.

PRESUBMIT=passed
R=asharif
CC=bjanakiraman,cma,yunlian,llozano
APPROVED=asharif
OCL=63265-p2
RCL=63272-p2
RDATE=2012/10/09 17:48:25


P4 change: 42795271
14/dejagnu/__init__.py
14/dejagnu/run_dejagnu.py
f2e605d0dd83e2639fe4cb81a429cfd6f5cb9e90 19-Feb-2013 yunlian <yunlian@p4.invalid> Remove table.py, results_columns.py. Remove dead codes in results_report.py
Remove full table in email. remove benchmark status and number of re-images
in email.
Test: Run the crosperf from cached results, it works fine.

PRESUBMIT=passed
BUG=7317051
R=asharif
APPROVED=asharif
DELTA=470 (0 added, 470 deleted, 0 changed)
RCL=63271-p2
RDATE=2012/10/09 17:36:06


P4 change: 42795270
14/crosperf/results_columns.py
14/crosperf/results_report.py
14/crosperf/table.py
269e57d34143f4c31d464bfc310e19411032a691 19-Feb-2013 asharif <asharif@p4.invalid> Added columns for standard deviation and noise for the summary table.

Tested: Ran crosperf on cached results.

PRESUBMIT=passed
R=yunlian,bjanakiraman
APPROVED=yunlian
DELTA=5 (4 added, 0 deleted, 1 changed)
OCL=63251-p2
RCL=63263-p2
RDATE=2012/10/09 16:52:31


P4 change: 42795258
14/crosperf/results_report.py
166a4e070dd7f007fa647f298fa3b7b75fc206f1 19-Feb-2013 asharif <asharif@p4.invalid> Added CoeffVarResult as well as CoeffVarFormat classes.

Fixed some formatting issues and refactored some code.

Tested: python tabulator.py && python tabulator_test.py work.

PRESUBMIT=passed
R=yunlian,bjanakiraman
APPROVED=yunlian
DELTA=71 (32 added, 16 deleted, 23 changed)
OCL=63249-p2
RCL=63262-p2
RDATE=2012/10/09 16:51:19


P4 change: 42795257
14/utils/tabulator.py
a68f417fd49fc2faaf0dcdce5fe4a2f2e9e7272a 19-Feb-2013 asharif <asharif@p4.invalid> Made the tabulator unittest pass.

Tested: python tabulator_test.py works.

PRESUBMIT=passed
R=yunlian,bjanakiraman
APPROVED=yunlian
DELTA=3 (0 added, 0 deleted, 3 changed)
OCL=63257-p2
RCL=63259-p2
RDATE=2012/10/09 16:31:17


P4 change: 42795254
14/utils/tabulator_test.py
8a5665873b5d4ef5192c1fc2b0b47a3964c4e069 19-Feb-2013 yunlian <yunlian@p4.invalid> Remove the /usr/local/autotest/result/* from the device before
running the test. This can solve the running out of space error.
Test: run crosperf, the final result is right, the result on the
device is gone.

PRESUBMIT=passed
R=shenhan,asharif
APPROVED=asharif
DELTA=4 (4 added, 0 deleted, 0 changed)
OCL=63222-p2
RCL=63258-p2
RDATE=2012/10/09 16:26:48


P4 change: 42795253
14/crosperf/autotest_runner.py
d7dbf1553a42a20f1f106f082ebacbdc9c73e776 19-Feb-2013 asharif <asharif@p4.invalid> Made the results cache aware and tolerant of change:
https://gerrit.chromium.org/gerrit/#/c/34856/

Tested: Ran crosperf.

PRESUBMIT=passed
R=yunlian,bjanakiraman
APPROVED=yunlian
DELTA=1 (0 added, 0 deleted, 1 changed)
OCL=63253-p2
RCL=63256-p2
RDATE=2012/10/09 16:14:21


P4 change: 42795252
14/crosperf/results_cache.py
550d6ee347c258f2724fd7de4f0ce32bb2a51b46 19-Feb-2013 yunlian <yunlian@p4.invalid> Add machine description to the cache contents. This will be
useful for debugging machine mismatch problems.

Test: run crosperf with simple test, the machine description was there.

PRESUBMIT=passed
R=asharif
APPROVED=asharif
DELTA=28 (18 added, 1 deleted, 9 changed)
OCL=63244-p2
RCL=63254-p2
RDATE=2012/10/09 15:43:26


P4 change: 42795243
14/crosperf/experiment.py
14/crosperf/machine_manager.py
14/crosperf/results_cache.py
6f9ad2513b85ef1632475c73d7e950ca1223c21a 19-Feb-2013 shenhan <shenhan@p4.invalid> Minor fixes to run_dejagnu.

1. parse options based on "arg" argument instead of sys.argv, this
enables the script to be called from others;

2. propagate return value at the end of execution;

TESTED=invoked this from automation client.
PRESUBMIT=passed
R=asharif
CC=yunlian,cmtice,llozano
APPROVED=asharif
OCL=63241-p2
RCL=63250-p2
RDATE=2012/10/09 14:58:47


P4 change: 42795241
14/dejagnu/run_dejagnu.py
5485d6c59e9f49615af2b4c0e99384b3d98eee29 19-Feb-2013 asharif <asharif@p4.invalid> Significantly speed-up checksumming by using the md5sum utility instead
of Python code.

md5sum <2 GB image> returns within a few seconds, while the Python code
to compute the md5sum can take 2-3 minutes per image.

Tested: Ran crosperf successfully on cached runs.

PRESUBMIT=passed
R=yunlian,bjanakiraman
APPROVED=yunlian
DELTA=14 (5 added, 7 deleted, 2 changed)
OCL=63245-p2
RCL=63247-p2
RDATE=2012/10/09 14:08:51


P4 change: 42795240
14/utils/file_utils.py
5d04332cce4322e8950b480080ea3bde2a10aa85 19-Feb-2013 yunlian <yunlian@p4.invalid> Add rdbytes and wrbytes to lower is better

PRESUBMIT=passed
BUG=7305242
R=asharif
APPROVED=asharif
DELTA=2 (1 added, 0 deleted, 1 changed)
RCL=63232-p2
RDATE=2012/10/08 17:14:26


P4 change: 42795229
14/utils/tabulator.py
f81369affebd496df4cd970e7e14738a990cbed8 19-Feb-2013 yunlian <yunlian@p4.invalid> Fix the warning message when the values are empty.
Test: The warning in the test case in tabulator.py is gone

PRESUBMIT=passed
R=asharif
APPROVED=asharif
DELTA=5 (4 added, 0 deleted, 1 changed)
OCL=63155-p2
RCL=63228-p2
RDATE=2012/10/08 13:38:30


P4 change: 42795212
14/utils/tabulator.py
f80f8900d2a68076022271ddd81ba796a7bd35c4 19-Feb-2013 yunlian <yunlian@p4.invalid> Use /proc/meminfo instead of /var/log/memory_spd_info.txt to get the
memory size info. Because the latter one sometimes is not available.
Use some round method for meminfo, because the size if not constant.
Test = Use different number around 2G as input, it shows the same number.

PRESUBMIT=passed
R=asharif,shenhan
APPROVED=asharif
DELTA=36 (31 added, 0 deleted, 5 changed)
RCL=63196-p2
RDATE=2012/10/05 15:26:04


P4 change: 42795163
14/crosperf/machine_manager.py
9b5a8fe8bf35b85f064cdd4c54d2208d6425b46d 19-Feb-2013 asharif <asharif@p4.invalid> Removed deprecated EXACT_REMOTE cache condition. Replaced that with
EXACT_MACHINE, which is set by default.

PRESUBMIT=passed
R=shenhan,yunlian
APPROVED=yunlian
DELTA=3 (0 added, 1 deleted, 2 changed)
OCL=63166-p2
RCL=63169-p2
RDATE=2012/10/03 15:38:56


P4 change: 42795114
14/crosperf/experiment_factory.py
14/crosperf/settings_factory.py
6797358dc73abef8cd855cc9a038ca72974679e2 19-Feb-2013 asharif <asharif@p4.invalid> Added code to work around crosbug.com/34956. test_toolchains.py now
deletes the ccache dir that is owned by root before exiting.

Tested: python test_toolchains.py works.

PRESUBMIT=passed
TBR=llozano,bjanakiraman
DELTA=7 (7 added, 0 deleted, 0 changed)
OCL=63140-p2
RCL=63159-p2
RDATE=2012/10/03 10:47:43


P4 change: 42795109
14/test_toolchains.py
0ff861e1c94c799bcdba63ef6274a26fbf87b9b1 19-Feb-2013 yunlian <yunlian@p4.invalid> Use EMAIL format instead of HTML format to show the content in email
Test: run crosperf with cached results, the result is correct

PRESUBMIT=passed
R=asharif
APPROVED=asharif
DELTA=17 (6 added, 0 deleted, 11 changed)
OCL=63144-p2
RCL=63151-p2
RDATE=2012/10/02 16:36:49


P4 change: 42795102
14/crosperf/results_report.py
14/utils/tabulator.py
bcfbdf8a333d5493856c9b42876f99335bf73fce 19-Feb-2013 yunlian <yunlian@p4.invalid> Handle the case where the result of a run is empty.
Test: run a crosperf with machine locked by others, no error generated.
(except the logfatal message)
run a crosperf with bad benchmark name, it passes too.

PRESUBMIT=passed
R=asharif,shenhan
APPROVED=asharif
DELTA=31 (22 added, 5 deleted, 4 changed)
OCL=63142-p2
RCL=63150-p2
RDATE=2012/10/02 16:17:03


P4 change: 42795087
14/crosperf/experiment_runner.py
14/crosperf/results_report.py
8c403daaa734fe5de8ee8bc0ee2e21e3e85bc2cd 19-Feb-2013 yunlian <yunlian@p4.invalid> use LogFatal if all the machines are locked by others

PRESUBMIT=passed
R=asharif,shenhan
APPROVED=shenhan
DELTA=3 (1 added, 0 deleted, 2 changed)
OCL=63132-p2
RCL=63141-p2
RDATE=2012/10/02 14:24:46


P4 change: 42795077
14/crosperf/machine_manager.py
6e612cc8b35ebc0fa406757ca407b8c855392db0 19-Feb-2013 yunlian <yunlian@p4.invalid> Add lock to image_chromeos to ensure the host run the script
at most one instance at any time.

Test: Two instances can not run at the same time now.
If one is Ctrl-c / kill -9 . The lock is released

PRESUBMIT=passed
R=shenhan,asharif
APPROVED=asharif
DELTA=14 (13 added, 0 deleted, 1 changed)
OCL=63087-p2
RCL=63122-p2
RDATE=2012/10/01 16:11:56


P4 change: 42795062
14/image_chromeos.py
e4a5a8ff3f30e35ea1b37d549dc549343305eed4 19-Feb-2013 asharif <asharif@p4.invalid> Replace '.' with '' for image labels. crosperf doesn't accept '.' in
image label names.

Tested: python test_toolchains.py \
--gcc_githash=gcc.gnu.org/branches/google/gcc-4_7-mobile

PRESUBMIT=passed
R=shenhan,bjanakiraman
CC=yunlian
APPROVED=yunlian
DELTA=1 (1 added, 0 deleted, 0 changed)
OCL=63080-p2
RCL=63113-p2
RDATE=2012/10/01 13:16:26


P4 change: 42795059
14/test_toolchains.py
0143e8af5950ef6b2faf132c29c3204fc8e67ef2 19-Feb-2013 yunlian <yunlian@p4.invalid> A better way to solve out of index bug
Test: the crosperf passed with the experiment file that was failed before

PRESUBMIT=passed
R=asharif
APPROVED=asharif
DELTA=4 (0 added, 2 deleted, 2 changed)
RCL=63110-p2
RDATE=2012/10/01 11:35:27


P4 change: 42795056
14/crosperf/results_report.py
739ec2642378c09d35c2823c3bc92c7a19d5a458 19-Feb-2013 asharif <asharif@p4.invalid> s/rootfs_boost/rootfs_boost_size.

Although build_image does accept rootfs_boost, the full name of the
argument is --rootfs_boost_size.

PRESUBMIT=passed
R=llozano,bjanakiraman
APPROVED=llozano
DELTA=1 (0 added, 0 deleted, 1 changed)
RCL=63109-p2
RDATE=2012/10/01 11:24:40


P4 change: 42795028
14/fdo_scripts/vanilla_vs_fdo.py
bc6aaad38becea0d9a269e89fec448332ebe36e6 19-Feb-2013 yunlian <yunlian@p4.invalid> Fix the index out of range bug when iterations of benchmarks are different.
Test: The failed crosperf run is fixed

PRESUBMIT=passed
R=asharif,shenhan
APPROVED=shenhan
DELTA=2 (2 added, 0 deleted, 0 changed)
OCL=63104-p2
RCL=63106-p2
RDATE=2012/10/01 09:46:59


P4 change: 42795025
14/crosperf/results_report.py
9b1bf4f827364ecc177a454199677e21fbea794a 19-Feb-2013 shenhan <shenhan@p4.invalid> Approving it so you can just edit the description and submit the CL.

Description should be something like:

Fix IndexOutOfRangeException when building the full table for crosperf.

Tested: crosperf <toolchain experiment>.

PRESUBMIT=passed
R=asharif,yunlian
APPROVED=asharif
OCL=63088-p2
RCL=63091-p2
RDATE=2012/09/28 21:06:07


P4 change: 42794998
14/utils/tabulator.py
992a4903147eb1243113d6d6140b7d3b77972338 19-Feb-2013 yunlian <yunlian@p4.invalid> Correct minilayout parameter

Test: run setup_chromeos.py --dir=blabla --minilayout passed

PRESUBMIT=passed
R=asharif
APPROVED=asharif
DELTA=1 (0 added, 0 deleted, 1 changed)
RCL=63085-p2
RDATE=2012/09/28 15:38:10


P4 change: 42794995
14/setup_chromeos.py
1d37e27ece106942e6078c1f4b8f60229d7bbc6c 19-Feb-2013 yunlian <yunlian@p4.invalid> use -g minilayout instead of -m minilayout
Test: setup_chromeos --minilayout [latest, top] works

PRESUBMIT=passed
R=asharif
APPROVED=asharif
DELTA=1 (0 added, 0 deleted, 1 changed)
OCL=63061-p2
RCL=63063-p2
RDATE=2012/09/27 14:18:04


P4 change: 42793680
14/setup_chromeos.py
5fe40e2c92ce609484b4aa6915813d8167740a8c 19-Feb-2013 asharif <asharif@p4.invalid> Catch error early when setup_chromeos.py fails.

Tested: Ran test_toolchains.py when setup_chromeos.py fails and it exits
early.

PRESUBMIT=passed
R=yunlian,bjanakiraman
APPROVED=yunlian
DELTA=3 (2 added, 0 deleted, 1 changed)
OCL=63060-p2
RCL=63062-p2
RDATE=2012/09/27 14:13:20


P4 change: 42793679
14/test_toolchains.py
4a9f889dfcb8538357c2890d155e245d37df2de7 19-Feb-2013 yunlian <yunlian@p4.invalid> Correct the perf parameter of the example

PRESUBMIT=passed
R=shenhan
APPROVED=shenhan
DELTA=2 (0 added, 1 deleted, 1 changed)
RCL=63045-p2
RDATE=2012/09/26 15:52:13


P4 change: 42793644
14/crosperf/experiment_files/aes_perf
87414279e5cac5db98dd31604dec14b7bc6bf748 19-Feb-2013 asharif <asharif@p4.invalid> Fixed typo in test_toolchains.py.

PRESUBMIT=passed
R=shenhan,bjanakiraman
APPROVED=shenhan
DELTA=1 (0 added, 0 deleted, 1 changed)
OCL=63038-p2
RCL=63042-p2
RDATE=2012/09/26 13:42:41


P4 change: 42793637
14/test_toolchains.py
2fcf1bf3b71c22f91c41bdfbad8d1b0497a3bd2e 19-Feb-2013 asharif <asharif@p4.invalid> Added check for list of zero size in crosperf.

Tested: Ran crosperf with cached values that contained some failures. It
was able to generate a full and summary table just fine.

PRESUBMIT=passed
R=yunlian,bjanakiraman
APPROVED=yunlian
DELTA=3 (3 added, 0 deleted, 0 changed)
OCL=63037-p2
RCL=63039-p2
RDATE=2012/09/26 12:31:35


P4 change: 42793630
14/utils/tabulator.py
3802fbff33ab17a0e8d89daff48d90d0e46c0996 19-Feb-2013 yunlian <yunlian@p4.invalid> Add an option to checkout the tree at a given timestamp

Test: use the --timestamp option and check the version it generates.
Also use --version=latest to check out the version it uses.

PRESUBMIT=passed
R=asharif
APPROVED=asharif
DELTA=49 (45 added, 2 deleted, 2 changed)
OCL=63008-p2
RCL=63033-p2
RDATE=2012/09/26 09:18:09


P4 change: 42793622
14/setup_chromeos.py
411f617da60da0d61ce01d760a2e2a4d349965c5 19-Feb-2013 yunlian <yunlian@p4.invalid> Disable lock when there is no locks_dir
Test: change the lock file dir and run crosperf. run lock_machine.py --list
on other machine. The machine I am using is not on locked list.

PRESUBMIT=passed
R=asharif,shenhan
APPROVED=asharif
DELTA=9 (6 added, 2 deleted, 1 changed)
OCL=63010-p2
RCL=63015-p2
RDATE=2012/09/25 15:40:52


P4 change: 42793591
14/crosperf/machine_manager.py
3497d896edfe5db85106fb478788e7d6b92ff19f 19-Feb-2013 asharif <asharif@p4.invalid> Doubled chromeos job's timeout from 12 hours to 24 hours.

The reason is that chrotomation is a shared resource and sometimes
building 2 ChromeOS images and testing them using crosperf takes longer
than 12 hours.

Once we get a regular nightly client working, we can compute the average
time it takes for jobs to finish and use a value closer to that one for
the timeout.

PRESUBMIT=passed
R=llozano,bjanakiraman
CC=c-compiler-chrome,shenhan
APPROVED=shenhan
DELTA=1 (0 added, 0 deleted, 1 changed)
OCL=62985-p2
RCL=62987-p2
RDATE=2012/09/24 11:05:55


P4 change: 42793540
14/automation/clients/chromeos.py
dade7a52f86cfda8c5adfa651544f8bcc15b6489 19-Feb-2013 yunlian <yunlian@p4.invalid> Change a typo

PRESUBMIT=passed
R=shenhan
APPROVED=shenhan
DELTA=1 (0 added, 0 deleted, 1 changed)
OCL=62964-p2
RCL=62966-p2
RDATE=2012/09/21 15:57:18


P4 change: 42793503
14/utils/tabulator.py
8cddf1aa13157a4ac296699d588df049d6a5628d 19-Feb-2013 asharif <asharif@p4.invalid> Moved "import stat" near the use.

stat is not a default python module for people on gPrecise. A better
solution is needed for the long term, but this will do for now.

Also tweaked the output string for AmeanResult.

Tested: `python tabulator.py' `python tabulator_test.py'

PRESUBMIT=passed
R=yunlian,bjanakiraman
APPROVED=yunlian
DELTA=3 (1 added, 1 deleted, 1 changed)
OCL=62958-p2
RCL=62963-p2
RDATE=2012/09/21 15:44:58


P4 change: 42793491
14/utils/tabulator.py
4797e2738b67b010fb70cdf146eb22713da369e6 19-Feb-2013 yunlian <yunlian@p4.invalid> Add functions to show the raw data and the length of none zero result
Test: use simple test case to test it, and it works fine

PRESUBMIT=passed
R=asharif
APPROVED=asharif
DELTA=77 (70 added, 2 deleted, 5 changed)
OCL=62895-p2
RCL=62962-p2
RDATE=2012/09/21 15:36:11


P4 change: 42793488
14/utils/tabulator.py
777f5effb4aa2d5efbc8bd96a123e187b87f8903 19-Feb-2013 yunlian <yunlian@p4.invalid> Use new tabulator to show the fulltable
Test: run crosperf, works fine

PRESUBMIT=passed
R=asharif
APPROVED=asharif
DELTA=100 (55 added, 17 deleted, 28 changed)
OCL=62925-p2
RCL=62961-p2
RDATE=2012/09/21 15:34:08


P4 change: 42793487
14/crosperf/results_report.py
109ac9f28afb6c3a5ac083e78b335db09d5fff89 19-Feb-2013 llozano <llozano@p4.invalid> Fixed couple of bugs found in processing of --env and --cxxflags options
For --env, it was trashing the value of the option if --rebuild is used
For --cxxflags, the value was interchanged with the --ldflags option.

PRESUBMIT=passed
R=asharif
APPROVED=asharif
DELTA=3 (1 added, 1 deleted, 1 changed)
OCL=62950-p2
RCL=62954-p2
RDATE=2012/09/21 11:52:37


P4 change: 42793475
14/build_chromeos.py
4030e44a66a08e8dab7185cdec65afbd952100c0 19-Feb-2013 asharif <asharif@p4.invalid> Drop from "utils." since tabulator.py is already in utils.

Tested: ran tabulator.py and tabulator_test.py.

PRESUBMIT=passed
TBR=yunlian
DELTA=1 (0 added, 0 deleted, 1 changed)
RCL=62934-p2
RDATE=2012/09/20 20:09:21


P4 change: 42793437
14/utils/tabulator.py
0f9022dd53e1617ae05e34e1cbb5fc8fcf0f970d 19-Feb-2013 asharif <asharif@p4.invalid> Removed bhaskar-staff from cc for changes in v14/automation.

Added instead c-compiler-chrome, which is the main user of this
infrastructure.

bhaskar-staff is the alias for everyone reporting to bjanakiraman which
includes GWT and others.

PRESUBMIT=passed
R=bjanakiraman
CC=skybrian
DELTA=3 (0 added, 0 deleted, 3 changed)
OCL=62889-p2
RCL=62902-p2
RDATE=2012/09/19 15:46:16


P4 change: 42793397
14/automation/PRESUBMIT.py
e6b72fef0fc9dde4360544dac4d2b522fa5a665d 19-Feb-2013 asharif <asharif@p4.invalid> Added an option to test_toolchains.py to build the external version of
ChromeOS.

Tested: python test_toolchains.py --public

PRESUBMIT=passed
R=shenhan,bjanakiraman
APPROVED=shenhan
DELTA=12 (10 added, 0 deleted, 2 changed)
OCL=62884-p2
RCL=62901-p2
RDATE=2012/09/19 15:45:46


P4 change: 42793396
14/test_toolchains.py
03e12156971f78280046af582b8bdcfd693a28bd 19-Feb-2013 yunlian <yunlian@p4.invalid> Use new tabulator to draw the chart.
Test: it draws the same chart as the old one does. Tested iteration 1 and 5.

PRESUBMIT=passed
R=asharif
APPROVED=asharif
DELTA=32 (13 added, 1 deleted, 18 changed)
OCL=62887-p2
RCL=62894-p2
RDATE=2012/09/19 14:39:17


P4 change: 42793392
14/crosperf/results_report.py
78c59a3b2027cf299aa81d1329e84151aacc180a 19-Feb-2013 asharif <asharif@p4.invalid> Added --public to the chromeos nightly performance job.

This can be backed out when crbug.com/145822 is resolved.

Tested: Ran nightly job.

PRESUBMIT=passed
R=shenhan,bjanakiraman,yunlian
DELTA=1 (1 added, 0 deleted, 0 changed)
OCL=62885-p2
RCL=62892-p2
RDATE=2012/09/19 14:28:02


P4 change: 42793371
14/automation/clients/chromeos.py
4447378724f0ddb7b9bda899ce5abfad752120fc 19-Feb-2013 asharif <asharif@p4.invalid> Added --env flag to build_chromeos.py to pass in custom env.

Tested: python build_chromeos.py --env=USE=-chrome_internal builds the
external version of Chrome.

PRESUBMIT=passed
R=shenhan,bjanakiraman,yunlian
DELTA=7 (5 added, 0 deleted, 2 changed)
OCL=62883-p2
RCL=62886-p2
RDATE=2012/09/19 14:09:35


P4 change: 42793366
14/build_chromeos.py
82690fab0680406670dc4de717533c9738220598 19-Feb-2013 yunlian <yunlian@p4.invalid> Add benchmark name and iteration at the top of table.
Add min, max, std, p-value columns to table.
Test: Tested with crosperf with cached/non-cached test with 1 iteration
and 5 iterations

PRESUBMIT=passed
R=asharif
APPROVED=asharif
DELTA=43 (33 added, 4 deleted, 6 changed)
OCL=62770-p2
RCL=62882-p2
RDATE=2012/09/19 14:02:41


P4 change: 42793365
14/crosperf/results_report.py
14/utils/tabulator.py
ab362d07c9de7116dffa19d12e21633dfb8109ee 19-Feb-2013 asharif <asharif@p4.invalid> Added StringMeanResult class. If all results are the same string value,
it computes a result of "ALL_<string value>." Otherwise it prints
"MIXED."

Also added a unittest for it.

Tested: python tabulator.py and python tabulator_test.py.

PRESUBMIT=passed
R=yunlian,bjanakiraman
APPROVED=yunlian
DELTA=43 (39 added, 0 deleted, 4 changed)
OCL=62854-p2
RCL=62877-p2
RDATE=2012/09/19 13:29:13


P4 change: 42793355
14/utils/tabulator.py
14/utils/tabulator_test.py
8df8f44ff2df7e439d8a75291458442edf9aa746 19-Feb-2013 asharif <asharif@p4.invalid> Added StorageFormat and unit tests for it.

Tested: python tabulator.py and python tabulator_test.py.

PRESUBMIT=passed
R=yunlian,bjanakiraman
APPROVED=yunlian
DELTA=36 (36 added, 0 deleted, 0 changed)
RCL=62853-p2
RDATE=2012/09/18 11:30:39


P4 change: 42793332
14/utils/tabulator.py
14/utils/tabulator_test.py
9ed0701a1e08fb5464b71fb2e3cbdcc98b0abafd 19-Feb-2013 asharif <asharif@p4.invalid> Fixed gpylint warnings.

Tested: Ran tabulator.py and tabulator_test.py.

PRESUBMIT=passed
R=yunlian,bjanakiraman
APPROVED=yunlian
DELTA=38 (23 added, 0 deleted, 15 changed)
OCL=62848-p2
RCL=62852-p2
RDATE=2012/09/18 10:42:29


P4 change: 42793321
14/utils/tabulator.py
bb8f6791bf2012d8112a584f9fd5fb0777ada045 19-Feb-2013 yunlian <yunlian@p4.invalid> Change the output format of result_organizer to make it suite the
tabulator.
Test = looked the output of result_organizer from the crosperf,
and it looks fine.

PRESUBMIT=passed
R=asharif,shenhan
APPROVED=asharif
DELTA=28 (2 added, 15 deleted, 11 changed)
OCL=62741-p2
RCL=62771-p2
RDATE=2012/09/14 16:03:04


P4 change: 42793108
14/crosperf/results_organizer.py
ac484b55f9f76a2250e45cddac30e290719cc87b 19-Feb-2013 yunlian <yunlian@p4.invalid> Use new tabulator to show the result in email.

PRESUBMIT=passed
R=asharif
APPROVED=asharif
DELTA=37 (35 added, 0 deleted, 2 changed)
OCL=62765-p2
RCL=62767-p2
RDATE=2012/09/14 15:13:06


P4 change: 42793090
14/crosperf/results_report.py
d0e67511891e21b87339fb2f606abbb29dd99f5b 19-Feb-2013 yunlian <yunlian@p4.invalid> Add support for another format to build the table.
Add support for PLAIN format

Test: tabulator_test.py passed. The lower_is_better is there

PRESUBMIT=passed
R=shenhan,llozano,asharif
APPROVED=asharif
DELTA=70 (43 added, 1 deleted, 26 changed)
OCL=62560-p2
RCL=62766-p2
RDATE=2012/09/14 14:35:53


P4 change: 42793089
14/utils/tabulator.py
79cac7c9d1685f156470d2967db583a3f5225d3f 19-Feb-2013 shenhan <shenhan@p4.invalid> Fix the example experiment file by adding autotestname field.

PRESUBMIT=passed
R=asharif,yunlian
CC=llozano,cmtice,bjanakiraman,carrot
APPROVED=asharif
OCL=62238-p2
RCL=62659-p2
RDATE=2012/09/12 08:59:19
TEST=run this example file without the error (but with error if without this CL)


P4 change: 42792898
14/crosperf/experiment_files/toolchain
bd1865c96785685f84f1b3762734ae7d29fbd358 19-Feb-2013 yunlian <yunlian@p4.invalid> Correct the colspan for table headings

PRESUBMIT=passed
R=shenhan
APPROVED=shenhan
DELTA=4 (0 added, 2 deleted, 2 changed)
RCL=62595-p2
RDATE=2012/09/10 13:55:58


P4 change: 42792159
14/crosperf/results_report.py
1b1c841e9b14c0a8c1d930c645fe578cab3d14bf 19-Feb-2013 yunlian <yunlian@p4.invalid> Add results_organizer to collect data from benchmark_runs for tabulator

PRESUBMIT=passed
R=shenhan,llozano
APPROVED=shenhan
DELTA=55 (55 added, 0 deleted, 0 changed)
OCL=62555-p2
RCL=62559-p2
RDATE=2012/09/07 14:37:59


P4 change: 42792109
14/crosperf/results_organizer.py
050cc3eefc5f4686647b9fc2b61a62ce3108fd7f 19-Feb-2013 yunlian <yunlian@p4.invalid> Handle the case to get the ratio when baseline is zero

PRESUBMIT=passed
R=shenhan,llozano
APPROVED=llozano
DELTA=14 (12 added, 0 deleted, 2 changed)
OCL=62553-p2
RCL=62556-p2
RDATE=2012/09/07 13:42:28


P4 change: 42792107
14/utils/tabulator.py
1ec68874444a9491a7cd0eeab1acd7ef7397077f 19-Feb-2013 yunlian <yunlian@p4.invalid> Add statistical difference column to the table.
Changed stats.py to avoid 0 division error.

TEST: run crosperf with cached results, works fine

PRESUBMIT=passed
R=asharif,llozano
APPROVED=asharif
DELTA=43 (34 added, 0 deleted, 9 changed)
OCL=62241-p2
RCL=62544-p2
RDATE=2012/09/06 17:10:51


P4 change: 42792098
14/crosperf/results_columns.py
14/crosperf/results_report.py
14/crosperf/table.py
14/utils/stats.py
9c38af93fad9dbc4f26cc32d40a33992f9de983d 19-Feb-2013 yunlian <yunlian@p4.invalid> Enable html support for the new tabulator
TEST: simple test in the file passed

PRESUBMIT=passed
R=asharif,llozano
APPROVED=asharif
DELTA=45 (39 added, 0 deleted, 6 changed)
OCL=62516-p2
RCL=62533-p2
RDATE=2012/09/06 13:31:09


P4 change: 42792068
14/utils/tabulator.py
fc4c06fbba106bb5171ddfb71e0bc20e9a8b86a9 19-Feb-2013 asharif <asharif@p4.invalid> build_chrome_browser.py: Remove both internal and external output
directories when --clean is specified.

Tested: ./build_chrome_browser.py --clean ... deletes both output
directories.

PRESUBMIT=passed
R=llozano,bjanakiraman
APPROVED=llozano
DELTA=25 (17 added, 8 deleted, 0 changed)
OCL=62512-p2
RCL=62532-p2
RDATE=2012/09/06 12:09:34


P4 change: 42792067
14/build_chrome_browser.py
0aa202a6730a28ff0d757c20680bd6da27d08939 19-Feb-2013 asharif <asharif@p4.invalid> Added a timeout of 12 hours for the chromeos job.

Tested: `python chromeos.py ...' works

PRESUBMIT=passed
R=llozano,yunlian,bjanakiraman
CC=bhaskar-staff
APPROVED=llozano
DELTA=3 (0 added, 0 deleted, 3 changed)
OCL=62460-p2
RCL=62509-p2
RDATE=2012/09/05 11:14:16


P4 change: 42792026
14/automation/clients/chromeos.py
14/automation/clients/helper/jobs.py
5e389b21cb704ca56251254e1d7de67bc0227008 19-Feb-2013 asharif <asharif@p4.invalid> Added a table processing, formatting and displaying library called
tabulator. It has the following features:
1. Processing, formatting and displaying are completely decoupled.
Processing is what to display, formatting is view-independent formatting
like colors, etc. and displaying is the final display to the console,
text file or HTML.
2. Is extensible and supports custom processing & formatting.

Also added unit tests for some methods in tabulator.py.

Tested: `python tabulator.py' works
`python tabulator_test.py' works

PRESUBMIT=passed
R=yunlian,bjanakiraman
APPROVED=yunlian
DELTA=1126 (1126 added, 0 deleted, 0 changed)
OCL=62452-p2
RCL=62508-p2
RDATE=2012/09/05 11:11:23


P4 change: 42792025
14/utils/colortrans.py
14/utils/tabulator.py
14/utils/tabulator_test.py
a85387df3b6df3f3b058a53776f8654fdb4ba6bb 19-Feb-2013 yunlian <yunlian@p4.invalid> Show actual image in the report if the path to image contains symbolic link.

Test: Run a crosperf, it does show the actual path

PRESUBMIT=passed
R=asharif,llozano
APPROVED=asharif
DELTA=6 (6 added, 0 deleted, 0 changed)
RCL=62487-p2
RDATE=2012/09/04 14:11:14


P4 change: 42791999
14/crosperf/experiment_file.py
b2cd634f478827abe374c1a77d28dd45577f399b 19-Feb-2013 asharif <asharif@p4.invalid> Added c-compiler-chrome as an alias to send email to for nightly
toolchain builds.

Tested: `python test_toolchains.py' works.

PRESUBMIT=passed
R=yunlian,bjanakiraman
APPROVED=yunlian
DELTA=1 (0 added, 0 deleted, 1 changed)
OCL=62394-p2
RCL=62459-p2
RDATE=2012/08/31 13:57:51


P4 change: 42791949
14/test_toolchains.py
6de3dcf245f87f46d9d2efc5a2644cf788e69300 19-Feb-2013 asharif <asharif@p4.invalid> Added support for custom timeout in the job.

Tested: `python server.py'
`python chromeos.py' # Ran server and submitted chromeos nightly job.

PRESUBMIT=passed
R=llozano,cmtice,bjanakiraman
CC=bhaskar-staff
APPROVED=llozano
DELTA=8 (2 added, 0 deleted, 6 changed)
OCL=62455-p2
RCL=62458-p2
RDATE=2012/08/31 13:57:31


P4 change: 42791948
14/automation/common/job.py
14/automation/server/job_executer.py
471f1f9fb1e404651d44da3976b641428d835c3b 19-Feb-2013 asharif <asharif@p4.invalid> Significantly speed-up the caching of results by excluding var/log and
var/spool from the cache tarball.

Example run cache creation time (for platform_AesThroughput):

Before this CL:
time took to create the tarball: 5m33s
size of the tarball: 74M

After this CL:
time took to create the tarball: 0s
size of the tarball: 57K

Tested:
crosperf aes --rerun=True
crosperf aes
Both passed.

PRESUBMIT=passed
R=yunlian,bjanakiraman
APPROVED=yunlian
DELTA=6 (4 added, 0 deleted, 2 changed)
OCL=62392-p2
RCL=62397-p2
RDATE=2012/08/29 15:56:46


P4 change: 42791864
14/crosperf/results_cache.py
e0c441eb60633031fdf8dd3253fe83c601983853 19-Feb-2013 asharif <asharif@p4.invalid> Added --email field to crosperf.

Tested: `./crosperf aes --email="shenhan yunlian"' worked.

PRESUBMIT=passed
R=yunlian,bjanakiraman
APPROVED=yunlian
DELTA=11 (7 added, 0 deleted, 4 changed)
OCL=62384-p2
RCL=62396-p2
RDATE=2012/08/29 15:55:45


P4 change: 42791863
14/crosperf/experiment.py
14/crosperf/experiment_factory.py
14/crosperf/experiment_runner.py
14/crosperf/settings_factory.py
88600c25f778d388a20b63ea304cbbda1e106e3d 19-Feb-2013 llozano <llozano@p4.invalid> test for the existance of "run_remote_tests.sh" instead of "cros_workon"

Tested: tested on my workareas. Verified run_remote_tests.sh still
exists in latest sources.

PRESUBMIT=passed
R=asharif,cmtice
APPROVED=asharif
DELTA=1 (0 added, 0 deleted, 1 changed)
OCL=62387-p2
RCL=62389-p2
RDATE=2012/08/29 15:26:26


P4 change: 42791852
14/utils/file_utils.py
77beeb5a7b00baed855d5131b00a84f6369e3f83 19-Feb-2013 yunlian <yunlian@p4.invalid> Remove the tmp directory after reading cache results.
Test: run a cached crosperf, no tmp dirs under /tmp

PRESUBMIT=passed
R=asharif,llozano
APPROVED=asharif
DELTA=1 (1 added, 0 deleted, 0 changed)
OCL=62383-p2
RCL=62388-p2
RDATE=2012/08/29 15:26:03


P4 change: 42791851
14/crosperf/results_cache.py
103562534cbcb703868367604766afb9e9e810ce 19-Feb-2013 yunlian <yunlian@p4.invalid> Use a different method to get the physical memory size.
/proc/meminfo is not accurate.

Test: run a cros_perf on two x86_alex, the mem test is right now.

PRESUBMIT=passed
R=asharif,llozano
APPROVED=asharif
DELTA=8 (0 added, 5 deleted, 3 changed)
OCL=62378-p2
RCL=62382-p2
RDATE=2012/08/29 14:36:38


P4 change: 42791850
14/crosperf/machine_manager.py
54bef9a4303f83e5f6de299ab7d37214cb400916 19-Feb-2013 llozano <llozano@p4.invalid> Fixed location path of manifest.xml files when --version is used.
Tested: Created couple of workareas with the new version and verified the manifest

PRESUBMIT=passed
R=asharif
DELTA=1 (0 added, 0 deleted, 1 changed)
OCL=62355-p2
RCL=62365-p2
RDATE=2012/08/29 11:02:24


P4 change: 42791812
14/setup_chromeos.py
a81dd75b576d7b513d07312521a5d557de172165 19-Feb-2013 yunlian <yunlian@p4.invalid> remove profile_type and profile_counters option, add profile_args option
instead

Test = perf_args: record -e cycles
benchmark: morejs {
autotest_name: desktopui_PyAutoPerfTests
autotest_args: --args='--iterations=1 perf.PageCyclerTest.testMoreJSFile'
}
cached on uncached both passed

PRESUBMIT=passed
R=asharif,llozano
APPROVED=asharif
DELTA=42 (0 added, 16 deleted, 26 changed)
OCL=62282-p2
RCL=62344-p2
RDATE=2012/08/28 10:58:20


P4 change: 42791788
14/crosperf/benchmark.py
14/crosperf/benchmark_run.py
14/crosperf/experiment.py
14/crosperf/experiment_factory.py
14/crosperf/experiment_files/bloat_perf
14/crosperf/experiment_files/morejs_perf
14/crosperf/experiment_files/page_cycler_perf
14/crosperf/settings_factory.py
354323ef17cff9b2ef9bbbdb592598fa71de200c 19-Feb-2013 yunlian <yunlian@p4.invalid> Fix issues in checking whether two machines are identical.
Test = board: x86-alex
remote: chromeos-zgb6.mtv chromeos-zgb5.mtv
perf_args: record -a -e cycles

benchmark: morejs {
autotest_name: desktopui_PyAutoPerfTests
autotest_args: --args='--iterations=1 perf.PageCyclerTest.testMoreJSFile'
}
passed

PRESUBMIT=passed
R=asharif,llozano
APPROVED=llozano
DELTA=8 (0 added, 0 deleted, 8 changed)
RCL=62318-p2
RDATE=2012/08/27 14:36:58


P4 change: 42791694
14/crosperf/machine_manager.py
1ccdd531fc98122a19ba59ba29fd922f4f1a59b4 19-Feb-2013 cmtice <cmtice@p4.invalid> Search for "cros_workon" instead of "enter_chroot.sh", which has been removed.

PRESUBMIT=passed
R=asharif
DELTA=1 (0 added, 0 deleted, 1 changed)
OCL=62307-p2
RCL=62311-p2
RDATE=2012/08/27 12:29:08


P4 change: 42791690
14/utils/file_utils.py
f43af350dd8a9f7448a390b8b7cb8f2d13d43ace 19-Feb-2013 yunlian <yunlian@p4.invalid> Filter out cpu MHz from cpuinfo. Because it may change from time to time
and cause the machine hash inconsistent.
Move the machine checksum to machine manager.

Test: use crosperf to run PyAutoPerfTests. It works fine

PRESUBMIT=passed
R=asharif,llozano
APPROVED=asharif
DELTA=60 (28 added, 25 deleted, 7 changed)
OCL=62197-p2
RCL=62279-p2
RDATE=2012/08/24 14:13:56


P4 change: 42791576
14/crosperf/benchmark_run.py
14/crosperf/experiment.py
14/crosperf/machine_manager.py
14/crosperf/results_cache.py
39f7cb20532a261e4ffc3b3ad040b00b0c7f8072 19-Feb-2013 asharif <asharif@p4.invalid> Changed repo_to_repo.py to use svnrevision -c to get the last committed
revision rather than the latest revision. This makes keeping track of
revisions easier.

Tested: `repo_to_repo.py -n -i repo_to_repo_files/...' passed.

PRESUBMIT=passed
R=shenhan,bjanakiraman
DELTA=2 (0 added, 0 deleted, 2 changed)
OCL=62240-p2
RCL=62243-p2
RDATE=2012/08/23 15:46:37


P4 change: 42791184
14/repo_to_repo.py
b23bb61316537e0be83940005c37f731ad090d9d 19-Feb-2013 mrdmnd <mrdmnd@p4.invalid> A short demo script to run the whole pipeline.

PRESUBMIT=passed
R=asharif
APPROVED=asharif
DELTA=55 (55 added, 0 deleted, 0 changed)
OCL=62201-p2
RCL=62237-p2
RDATE=2012/08/23 14:07:01


P4 change: 42791183
14/cwp/demo_pipeline.sh
5f4003c13549e0cc548e2e355671d758fdcd7c13 19-Feb-2013 yunlian <yunlian@p4.invalid> Remove the remote out of the key of the cache result.
Add the hash of cpuinfo and meminfo into the key of the cache result.
Make the machine check as the default check.

Test: run a small test, verify the directory name uses hash of machineinfo.
Run it again, it shows cache hit. Change the hash of machineinf in the name
of the directory and run it again, shows cache miss.

PRESUBMIT=passed
R=asharif,llozano
APPROVED=llozano
DELTA=35 (22 added, 5 deleted, 8 changed)
OCL=62159-p2
RCL=62226-p2
RDATE=2012/08/23 10:50:29


P4 change: 42791169
14/crosperf/benchmark_run.py
14/crosperf/experiment_factory.py
14/crosperf/results_cache.py
bd0d370b81e96bc81d4157e08be6c5dada247eb6 19-Feb-2013 yunlian <yunlian@p4.invalid> Add color bar to show performance difference.
Tested: passed the crosperf with cached results

benchmark: PyAutoPerfTests {
iterations: 5
}

label: glibc211 {
chromeos_image: /home/yunlian/extend/beta/src/build/images/x86-alex/latest/chromiumos_test_image.bin
board: x86-alex
}

label: glibc218 {
chromeos_image: /home/yunlian/extend/alpha/src/build/images/x86-alex/latest/chromiumos_test_image.bin
board: x86-alex
}

PRESUBMIT=passed
R=asharif,llozano
APPROVED=llozano
DELTA=73 (51 added, 8 deleted, 14 changed)
OCL=62077-p2
RCL=62225-p2
RDATE=2012/08/23 10:44:34


P4 change: 42791168
14/crosperf/results_columns.py
14/crosperf/results_report.py
14/crosperf/table.py
437fffb85dfa16878b514f302b243516271ecbe2 19-Feb-2013 asharif <asharif@p4.invalid> Added special characters to GetFilenameFromString. Enhanced unittest to
deal with that case.

Tested: Ran crosperf with BootPerfServer/control$
Also ran: misc_test.py, which passes.

PRESUBMIT=passed
R=llozano,bjanakiraman
DELTA=3 (0 added, 0 deleted, 3 changed)
OCL=62147-p2
RCL=62166-p2
RDATE=2012/08/21 17:27:20


P4 change: 42791074
14/utils/misc.py
14/utils/misc_test.py
9a47718d307f4c86a34972820a2a488f43383afe 19-Feb-2013 mrdmnd <mrdmnd@p4.invalid> Updates the code in the repository to the code that is running in prod.
Forgot to update this in critique. Cleans up the codebase a bit.

PRESUBMIT=passed
R=asharif
APPROVED=asharif
DELTA=53 (6 added, 6 deleted, 41 changed)
OCL=62078-p2
RCL=62145-p2
RDATE=2012/08/21 11:51:30


P4 change: 42791064
14/cwp/interpreter/app_engine_pull.py
14/cwp/interpreter/symbolizer.py
ef7b5eda42d0669fe0872a72becdbc510e3be683 19-Feb-2013 yunlian <yunlian@p4.invalid> Check the ssh return value before parsing meminfo and cpuinfo

PRESUBMIT=passed
R=asharif,llozano
DELTA=8 (2 added, 0 deleted, 6 changed)
OCL=62081-p2
RCL=62135-p2
RDATE=2012/08/20 14:26:55


P4 change: 42791040
14/crosperf/machine_manager.py
a62c0e4987ee46575b9fcc14ff5792ba60908484 19-Feb-2013 asharif <asharif@p4.invalid> Added a client for doing performance testing the latest toolchain
nightly.

This client checks out the v14 directory from perforce and builds all of
chromeos from source using the git branch specified on the command line.

PRESUBMIT=passed
R=bjanakiraman,shenhan,cmtice
CC=bhaskar-staff,llozano
APPROVED=llozano
DELTA=102 (102 added, 0 deleted, 0 changed)
OCL=61208-p2
RCL=62134-p2
RDATE=2012/08/20 13:06:05


P4 change: 42791039
14/automation/clients/chromeos.py
3e38de071e11afd2e52f6ce6e8ce78cdec74d2cc 19-Feb-2013 asharif <asharif@p4.invalid> Added option to test_toolchains.py to clean-up after testing. This is
necessary because our infrastructure does `rm -rf' after a job is
completed. The chroot has root permissions and is not cleaned up
properly.

Tested: python test_toolchains.py --clean ... worked.

PRESUBMIT=passed
R=yunlian,bjanakiraman
APPROVED=yunlian
DELTA=16 (14 added, 0 deleted, 2 changed)
OCL=62051-p2
RCL=62133-p2
RDATE=2012/08/20 13:05:54


P4 change: 42791038
14/test_toolchains.py
de19f7c9942df7ee10d575173484f0eefe788629 19-Feb-2013 asharif <asharif@p4.invalid> Added USE="chrome_internal" to build_chromeos.py by default.

Tested: ./build_chromeos.py ... worked

PRESUBMIT=passed
R=llozano,bjanakiraman
DELTA=3 (3 added, 0 deleted, 0 changed)
OCL=62050-p2
RCL=62132-p2
RDATE=2012/08/20 13:05:39


P4 change: 42791037
14/build_chromeos.py
24c103a852d62f2fd10a347d6a7cc8f1771400c9 19-Feb-2013 yunlian <yunlian@p4.invalid> add stats.py from
http://www.nmr.mgh.harvard.edu/Neural_Systems_Group/gary/python/previous/stats.pyto
utils

PRESUBMIT=passed
R=asharif,llozano
APPROVED=asharif
DELTA=4526 (4526 added, 0 deleted, 0 changed)
OCL=62079-p2
RCL=62120-p2
RDATE=2012/08/20 11:22:00


P4 change: 42791029
14/utils/stats.py
6ab1f57e67967df5c7b259d8e6c5951778cc2162 19-Feb-2013 yunlian <yunlian@p4.invalid> Corrent a condition variable for add color
Test = run crosperf with default setting.

PRESUBMIT=passed
R=asharif,llozano,shenhan
DELTA=1 (0 added, 0 deleted, 1 changed)
RCL=61988-p2
RDATE=2012/08/14 14:26:51


P4 change: 42790408
14/crosperf/table.py
c563d578a1119ba823004dfd154f81ebfce5c4ad 19-Feb-2013 yunlian <yunlian@p4.invalid> Make color only appears in email, not in plain report.

PRESUBMIT=passed
R=asharif,llozano
Run the crosperf to test Pyautotest suite. It passed.

PRESUBMIT=passed
DELTA=35 (26 added, 4 deleted, 5 changed)
OCL=61979-p2
RCL=61985-p2
RDATE=2012/08/14 13:46:14


P4 change: 42790392
14/crosperf/experiment_runner.py
14/crosperf/results_report.py
14/crosperf/table.py
0d2f2c687625f3bf5d0fc78472a30959528325af 19-Feb-2013 yunlian <yunlian@p4.invalid> add seconds to lower_is_better category

PRESUBMIT=passed
R=asharif,llozano
DELTA=7 (3 added, 0 deleted, 4 changed)
OCL=61970-p2
RCL=61976-p2
RDATE=2012/08/14 11:11:36


P4 change: 42790337
14/crosperf/results_report.py
c538fb62cf2811b5c7c9bc12116e1674b9a23b3e 19-Feb-2013 yunlian <yunlian@p4.invalid> Use red and green color to show performance difference if the difference is large.

PRESUBMIT=passed
R=asharif,llozano
DELTA=30 (27 added, 1 deleted, 2 changed)
OCL=61910-p2
RCL=61968-p2
RDATE=2012/08/14 10:08:00


P4 change: 42790328
14/crosperf/results_report.py
041b178994fca5600c7d021e98bb8a22542551b9 19-Feb-2013 yunlian <yunlian@p4.invalid> Use CrosRunCommand instead to RunCommand to connect to remote board.

PRESUBMIT=passed
R=asharif,llozano
DELTA=15 (1 added, 0 deleted, 14 changed)
OCL=61914-p2
RCL=61940-p2
RDATE=2012/08/13 10:55:36


P4 change: 42790285
14/crosperf/machine_manager.py
fe0050912ec282242a93ba5a45ce3db00b7915e6 19-Feb-2013 shenhan <shenhan@p4.invalid> Change chromium gcc 4-7 rtr file to point to google/gcc-4_7-mobile branch.

PRESUBMIT=passed
R=asharif
CC=c-compiler-chrome
OCL=61701-p2
RCL=61937-p2
RDATE=2012/08/13 10:08:46


P4 change: 42790251
14/repo_to_repo_files/gcc-branches_google_4_7.rtr
da4973f63bb654320987ddd51b704fa623801c59 19-Feb-2013 yunlian <yunlian@p4.invalid> Add a heursitics to decide the direction of a performance result.
If benchmarks contains ms_ or milliseconds, we think the number is
the smaller, the better. Otherwise, the bigger, the better.

PRESUBMIT=passed
R=asharif,llozano
DELTA=10 (9 added, 0 deleted, 1 changed)
OCL=61867-p2
RCL=61875-p2
RDATE=2012/08/09 17:33:40


P4 change: 42790057
14/crosperf/results_report.py
d2045ccec06fd51d001b09458ecc6cb7707a5228 19-Feb-2013 asharif <asharif@p4.invalid> Added stdout/stderr supression of the `git ls-remote` command to
GetChromeOSVersionFromLSBVersion().

Tested: python misc_test.py works.

PRESUBMIT=passed
R=mrdmnd,yunlian,bjanakiraman
DELTA=2 (1 added, 0 deleted, 1 changed)
OCL=61848-p2
RCL=61856-p2
RDATE=2012/08/09 11:35:21


P4 change: 42789858
14/utils/misc.py
4eadf29ab2a143f90b5fd99097fde8196cb0d66c 19-Feb-2013 asharif <asharif@p4.invalid> Added new parameter to RunCommand() called print_to_console. This can be
used to supress console output for verbose commands.

PRESUBMIT=passed
R=llozano,yunlian,bjanakiraman
DELTA=48 (21 added, 0 deleted, 27 changed)
OCL=61807-p2
RCL=61847-p2
RDATE=2012/08/08 20:33:05


P4 change: 42789852
14/utils/command_executer.py
14/utils/logger.py
8c66aed1265ddef8d77203713df6ae7e82236447 19-Feb-2013 yunlian <yunlian@p4.invalid> Automated g4 rollback of changelist 61819.

*** Reason for rollback ***

no review

*** Original change description ***

Verify all the test machines identical. Otherwise, report a warning.

PRESUBMIT=passed
R=asharif
DELTA=1 (0 added, 0 deleted, 1 changed)
RCL=61830-p2
RDATE=2012/08/08 11:42:48


P4 change: 42789786
14/crosperf/machine_manager.py
520ee0b6159d6c881b86c34c346bb3befafc6f61 19-Feb-2013 asharif <asharif@p4.invalid> Made the symbolizer more robust and refactored some code.
The symbolizer symbol download state can now be in one of the following
states:
1. Download tarball to <location>.tmp.
2. Move <location>.tmp to <location>.
3. Untar tarball to usr/lib.tmp.
4. Move usr/lib.tmp to usr/lib.
The symbolizer works incrementally from any of the above stages.

Also refactored repeated code into a separate function.

Changed code to check the return value of gsutil rather than reading the
stdout. There could be unknown errors on stdout but a non-zero return
value should catch them.

Tested: Ran symbolize.py --in= --out= --cache=
Pressed Ctrl + c at each of the stages and it performed symbolization
correctly.

PRESUBMIT=passed
R=mrdmnd,bjanakiraman
APPROVED=mrdmnd
DELTA=40 (14 added, 0 deleted, 26 changed)
OCL=61817-p2
RCL=61829-p2
RDATE=2012/08/08 11:07:49


P4 change: 42789785
14/cwp/interpreter/symbolizer.py
abae9c5b19bbd2b1a2b216bb0db9dcd6a30ccb5a 19-Feb-2013 yunlian <yunlian@p4.invalid> Verify all the test machines identical. Otherwise, report a warning.

PRESUBMIT=passed
R=asharif,llozano
OCL=61801-p2
RCL=61819-p2
RDATE=2012/08/07 17:34:45


P4 change: 42789766
14/crosperf/machine_manager.py
9a936cdf0012632ac38ccdda4c70ea0c90992d66 19-Feb-2013 mrdmnd <mrdmnd@p4.invalid> Implemented symbolizer.py, which operates on a directory full of perf data
files named by app_engine_pull.py. It downloads and builds an incremental cache
of symbols, and then symbolizes each data file.
Tested: After app_engine_pull'ing, python symbolizer.py --in=perf_samples
--out=perf_reports --cache=cache/ correctly symbolizes a test perf.data after
downloading symbols.

PRESUBMIT=passed
R=asharif
APPROVED=asharif
DELTA=114 (114 added, 0 deleted, 0 changed)
OCL=61778-p2
RCL=61815-p2
RDATE=2012/08/07 16:22:11


P4 change: 42789764
14/cwp/interpreter/symbolizer.py
a659216c5a3cb06d7d37c0ac5d5316c7b58c94f8 19-Feb-2013 asharif <asharif@p4.invalid> Changed GetCtargetFromBoard() to strip out ANSI non-printing escape
sequences from the ctarget.

PRESUBMIT=passed
R=cmtice,llozano
DELTA=7 (7 added, 0 deleted, 0 changed)
OCL=61804-p2
RCL=61806-p2
RDATE=2012/08/07 13:10:40


P4 change: 42789760
14/utils/misc.py
cef62e8702f2731be11fe2f5777afe5a88626a68 19-Feb-2013 asharif <asharif@p4.invalid> Added -n option to perf report to show number of samples.

Tested: Ran crosperf on platform_AesThroughput with profiling. Saw
number of samples column in output report.

PRESUBMIT=passed
R=yunlian,bjanakiraman
APPROVED=yunlian
DELTA=2 (1 added, 0 deleted, 1 changed)
OCL=61757-p2
RCL=61760-p2
RDATE=2012/08/06 11:37:25


P4 change: 42789621
14/crosperf/results_cache.py
ad9707c2840343b0c0141e76c2d85b24d086a37c 19-Feb-2013 asharif <asharif@p4.invalid> Added function to return the full revision from a build number obtained
from /etc/lsb-release.

Added a unittest for this function.

Tested: Ran misc_test.py.

PRESUBMIT=passed
R=mrdmnd,yunlian
APPROVED=mrdmnd
DELTA=28 (28 added, 0 deleted, 0 changed)
OCL=61706-p2
RCL=61719-p2
RDATE=2012/08/03 11:21:56


P4 change: 42789563
14/utils/misc.py
14/utils/misc_test.py
8e60cea8077af54a0cc27d2ee14867858d82e216 19-Feb-2013 mrdmnd <mrdmnd@p4.invalid> Changed the filename of the samples to include board and version info.
This should allow us to parse the filename when we do symbolization, so we
will end up pulling the right symbols for perf report.
test:
python app_engine_pull --output_dir=perf_samples/

produces the correctly labelled new files.

PRESUBMIT=passed
R=asharif
APPROVED=asharif
DELTA=36 (25 added, 0 deleted, 11 changed)
OCL=61698-p2
RCL=61700-p2
RDATE=2012/08/02 15:45:57


P4 change: 42789545
14/cwp/interpreter/app_engine_pull.py
01dba111cf5f8838cf95fdc4cc2b64a5540e3945 19-Feb-2013 asharif <asharif@p4.invalid> Added USE="chrome_internal" to build_chrome_browser.py by default.

PRESUBMIT=passed
R=yunlian,llozano,cmtice,bjanakiraman
DELTA=2 (2 added, 0 deleted, 0 changed)
OCL=61645-p2
RCL=61695-p2
RDATE=2012/08/02 14:04:22


P4 change: 42789532
14/build_chrome_browser.py
88f79ae92de8774b5b58f9a7120c2ad206234e60 19-Feb-2013 asharif <asharif@p4.invalid> Added a function in misc that can merge enviornment strings and
dictionaries.

Also made sure misc.py and misc_test.py are lint-clean.

PRESUBMIT=passed
R=cmtice,llozano,yunlian,bjanakiraman
DELTA=48 (40 added, 2 deleted, 6 changed)
OCL=61621-p2
RCL=61694-p2
RDATE=2012/08/02 14:04:10


P4 change: 42789531
14/utils/misc.py
14/utils/misc_test.py
324f20bce20b9c3dc8847ff0ed3d217ae1d61a6f 19-Feb-2013 mrdmnd <mrdmnd@p4.invalid> Typo fix.

PRESUBMIT=passed
R=asharif
APPROVED=asharif
DELTA=1 (0 added, 0 deleted, 1 changed)
RCL=61693-p2
RDATE=2012/08/02 14:03:51


P4 change: 42789530
14/cwp/interpreter/app_engine_pull.py
4091b611978a2f9c5129be5b2644029b7f455ab6 19-Feb-2013 mrdmnd <mrdmnd@p4.invalid> Modified app_engine_pull to suit CWP. Now, it just downloads samples.
To use the script, you need to set up an application-specific password (https://www.google.com/settings/security).

Tested:
python app_engine_pull.py --output=perf_samples/

downloads samples appropriately.

PRESUBMIT=passed
R=asharif
APPROVED=asharif
DELTA=679 (2 added, 500 deleted, 177 changed)
OCL=61649-p2
RCL=61692-p2
RDATE=2012/08/02 13:50:44


P4 change: 42789522
14/cwp/interpreter/app_engine_pull.py
f98637454dd4de63af5a64215d8c2406cecb2190 19-Feb-2013 mrdmnd <mrdmnd@p4.invalid> Added more fields to our serve page so the downloader can remove garbage data more easily.
Remove the display limit on serve/ page.

PRESUBMIT=passed
R=asharif
APPROVED=asharif
DELTA=11 (3 added, 3 deleted, 5 changed)
OCL=61601-p2
RCL=61691-p2
RDATE=2012/08/02 13:50:02


P4 change: 42789022
14/cwp/bartlett/server.py
2f56e1f14b2b1315d7c6822b131383d4ad888df2 19-Feb-2013 llozano <llozano@p4.invalid> This script generates a summary of the hottest basic blocks for Chrome
base on the bb counts found during the FDO collection phase.
The information is extracted from GCC internal dump files generated by
the script vanilla_vs_fdo.py.

PRESUBMIT=passed
R=shenhan
CC=bjanakiraman,asharif
APPROVED=shenhan,bjanakiraman,asharif
DELTA=184 (184 added, 0 deleted, 0 changed)
OCL=61571-p2
RCL=61689-p2
RDATE=2012/08/02 13:26:53


P4 change: 42789021
14/fdo_scripts/summarize_hot_blocks.py
6574a6525cd7e6b5dca3ed0c144e880dd08e6ef7 19-Feb-2013 mrdmnd <mrdmnd@p4.invalid> Copied http://cs/#google3/experimental/mobile_gwp/database/app_engine_pull.py into CWP.

PRESUBMIT=passed
R=asharif
APPROVED=asharif
DELTA=718 (718 added, 0 deleted, 0 changed)
OCL=61613-p2
RCL=61648-p2
RDATE=2012/07/31 17:16:40


P4 change: 42788901
14/cwp/interpreter/app_engine_pull.py
8ac31f384c47bbcc66f5937494a01a35e81fcfdb 19-Feb-2013 asharif <asharif@p4.invalid> This CL adds -t -t option to force-allocate a psuedo-terminal to ssh
when executing jobs like test_toolchains.py. This is a workaround for
crosbug.com/33007. Once that is fixed, this can be removed.

Note that this breaks 2 additional unit-tests than what are already
broken.

PRESUBMIT=passed
R=llozano,yunlian,bjanakiraman
CC=bhaskar-staff
DELTA=2 (1 added, 0 deleted, 1 changed)
OCL=61622-p2
RCL=61646-p2
RDATE=2012/07/31 15:59:10


P4 change: 42788900
14/automation/common/command_executer.py
11c84c7e6a69747dc493f94424fac5a5b7efaa2d 19-Feb-2013 mrdmnd <mrdmnd@p4.invalid> Slight readability fixes. Removed lsb-release file, POST should include board and chromeos_version.

PRESUBMIT=passed
R=asharif,shenhan
APPROVED=asharif
DELTA=23 (5 added, 6 deleted, 12 changed)
OCL=61243-p2
RCL=61597-p2
RDATE=2012/07/27 11:29:03


P4 change: 42788834
14/cwp/bartlett/server.py
14/cwp/bartlett/test/server_tester.py
21b38bc54e778df4b9b6c6f75ae7911999041309 19-Feb-2013 asharif <asharif@p4.invalid> Made UnitToNumber() more robust by not doing anything if the argument
is already a number.

PRESUBMIT=passed
R=shenhan,bjanakiraman
DELTA=3 (2 added, 0 deleted, 1 changed)
OCL=61579-p2
RCL=61587-p2
RDATE=2012/07/26 10:57:57


P4 change: 42788675
14/utils/misc.py
14b25e2e2b7e3553cd6b589b42d784206c198421 19-Feb-2013 asharif <asharif@p4.invalid> Changed image_chromeos.py's check of whether the image is modified for
test to be a more robust one. Now it checks for the string "test" within
/etc/lsb-release.

This is needed because images grabbed from chromeos-images/ have "test"
in their /etc/lsb-release file.

PRESUBMIT=passed
R=cmtice,bjanakiraman
DELTA=3 (2 added, 0 deleted, 1 changed)
OCL=61578-p2
RCL=61586-p2
RDATE=2012/07/26 10:57:51


P4 change: 42788674
14/image_chromeos.py
fe73cb339954bea971af373e082cfb2a37a380f7 19-Feb-2013 mrdmnd <mrdmnd@p4.invalid> Added a unit test for pushing data to the app engine server. Added some dummy data.
Added a tiny script to push local changes out onto the app engine server.

PRESUBMIT=passed
R=asharif
APPROVED=asharif
DELTA=106 (106 added, 0 deleted, 0 changed)
OCL=61180-p2
RCL=61242-p2
RDATE=2012/07/03 09:58:41


P4 change: 42787530
14/cwp/bartlett/test/server_tester.py
14/cwp/bartlett/update_appengine_server
96ac6483dda78edc0bb5f039fb648d6d0cadf2c3 19-Feb-2013 mrdmnd <mrdmnd@p4.invalid> Modified the mobile_gwp code to reflect the parameters we want
to collect for ChromeOS profiling. Cleaned up the code a bit too.
Changed app name in app.yaml, and drew a small favicon.

Tested: python test/server_tester.py, http://chromeoswideprofiling.appspot.com/serve lists the appropriate key and date.

PRESUBMIT=passed
R=asharif
APPROVED=asharif
DELTA=79 (4 added, 26 deleted, 49 changed)
OCL=61155-p2
RCL=61179-p2
RDATE=2012/06/26 13:14:25


P4 change: 42787185
14/cwp/bartlett/app.yaml
14/cwp/bartlett/server.py
14/cwp/bartlett/static/favicon.ico
e4b2b833995961355431f66fd26ae5d5bb6a2f9e 19-Feb-2013 asharif <asharif@p4.invalid> Made test_toolchains.py more incremental.
Tested: python test_toolchains.py --githashes=master

PRESUBMIT=passed
R=shenhan,llozano,bjanakiraman,yunlian,cmtice
DELTA=19 (0 added, 19 deleted, 0 changed)
OCL=61086-p2
RCL=61166-p2
RDATE=2012/06/25 10:42:50


P4 change: 42787169
14/test_toolchains.py
73a4cce105cfc76a06b8893de07783c6bb24c986 19-Feb-2013 asharif <asharif@p4.invalid> Removed MakeChroot() from build_chromeos.py. This is deprecated now.
Tested: python test_toolchains.py --githashes=master
Note that test_toolchains.py uses build_chromeos.py.

PRESUBMIT=passed
R=bjanakiraman,yunlian,cmtice
DELTA=20 (0 added, 20 deleted, 0 changed)
OCL=61087-p2
RCL=61165-p2
RDATE=2012/06/25 10:42:44


P4 change: 42787168
14/build_chromeos.py
f0273df239a0bfd264cedbff7895a98e54352469 19-Feb-2013 mrdmnd <mrdmnd@p4.invalid> Added the original files for the AppEngine code from the mobile_awp project from http://s/?dir=//depot/google3/experimental/mobile_gwp/appengine

Tested: diff -uNr . /google/src/head/depot/google3/experimental/mobile_gwp/appengine/

PRESUBMIT=passed
R=asharif
APPROVED=asharif
DELTA=195 (195 added, 0 deleted, 0 changed)
OCL=61152-p2
RCL=61154-p2
RDATE=2012/06/22 15:18:04


P4 change: 42787156
14/cwp/bartlett/app.yaml
14/cwp/bartlett/server.py
14/cwp/bartlett/static/favicon.ico
ca9a587fb25005b069fcc165eb4d9020fd98f62a 19-Feb-2013 mrdmnd <mrdmnd@p4.invalid> Switched a file() call to an open() call for more idiomatic python.

PRESUBMIT=passed
R=asharif,bjanakiraman
APPROVED=asharif
DELTA=3 (0 added, 1 deleted, 2 changed)
OCL=61148-p2
RCL=61153-p2
RDATE=2012/06/22 14:09:23


P4 change: 42787155
14/cwp/performance/experiment_gen.py
2b89844e282671684bdebf3de61d69e66460d135 19-Feb-2013 mrdmnd <mrdmnd@p4.invalid> Added an experiment-file generator for crosperf overhead benchmarking.
Tested it by running "python experiment_gen.py --crosperf=/home/mrdmnd/depot2/crosperf/ --chromeos=/home/mrdmnd/chromiumos --remote=chromeos-zgb3.mtv --event=cycles --board=x86-zgb -F 10 -F 100"

PRESUBMIT=passed
R=asharif,bjanakiraman
APPROVED=asharif
DELTA=117 (117 added, 0 deleted, 0 changed)
OCL=61078-p2
RCL=61135-p2
RDATE=2012/06/21 13:22:01


P4 change: 42787051
14/cwp/performance/experiment_gen.py
fdc0f578b41942c36e593f517622dc8f55c76491 19-Feb-2013 shenhan <shenhan@p4.invalid> Use '-usersandbox' instead of '-sandbox'.

'-sandbox' no longer works for gcc ebuild file.

PRESUBMIT=passed
R=asharif
CC=llozano,yunlian,cmtice,carrot,bjanakiraman
OCL=61076-p2
RCL=61099-p2
RDATE=2012/06/19 11:47:55
TEST=build gcc 4.7 using the script.


P4 change: 42786918
14/build_tc.py
f6c196c541cc47ccbc7b11c948f55eff14b97936 16-Feb-2013 asharif <asharif@p4.invalid> Fixed bug where a final report would not be generated if a benchmark run
would not have valid results.

PRESUBMIT=passed
R=yunlian,bjanakiraman
CC=asharif
DELTA=4 (4 added, 0 deleted, 0 changed)
OCL=60831-p2
RCL=60847-p2
RDATE=2012/06/05 14:11:46


P4 change: 42677088
14/crosperf/results_sorter.py
85b3715e034e1f8f25602b8cd7ab915e989b1afc 16-Feb-2013 llozano <llozano@p4.invalid> Recently created constants.mounted_toolchain has a "/" at the beginning
so we had to add an lstrip before this path can be joined with the
chroot path.

PRESUBMIT=passed
R=asharif
CC=llozano
DELTA=1 (0 added, 0 deleted, 1 changed)
RCL=60808-p2
RDATE=2012/06/01 18:53:03


P4 change: 42677026
14/build_tc.py
c49147f4aa8ce2b4f1a7983bce9dc0fab62d3ba8 16-Feb-2013 asharif <asharif@p4.invalid> Removed binutils version from the path.

PRESUBMIT=passed
R=shenhan,bjanakiraman
DELTA=1 (0 added, 0 deleted, 1 changed)
OCL=60797-p2
RCL=60802-p2
RDATE=2012/06/01 15:48:26


P4 change: 42677021
14/repo_to_repo_files/binutils-master.rtr
ad803433b78b99ad71ad5f74a1c4cd2d5af9901a 16-Feb-2013 shenhan <shenhan@p4.invalid> Add support for running test against a customized gcc build (say, gcc built using built_tc.py).

PRESUBMIT=passed
R=asharif
CC=bjanakiraman,cmtice,llozano,yunlian
OCL=59682-p2
RCL=60757-p2
RDATE=2012/05/29 09:57:04


P4 change: 42676969
14/build_tc.py
14/dejagnu/run_dejagnu.py
14/utils/constants.py
eeead0d3f080364cef1f52b1594abf3310c401af 16-Feb-2013 asharif <asharif@p4.invalid> Changed logger to warn instead of aborting when log files cannot be
created. This is useful when running scripts from someone else's home
directory and you don't have permission to write to the log files.

PRESUBMIT=passed
BUG=6501320
R=llozano,cmtice,yunlian,bjanakiraman
DELTA=19 (10 added, 0 deleted, 9 changed)
OCL=60748-p2
RCL=60752-p2
RDATE=2012/05/25 14:39:38


P4 change: 42676952
14/utils/logger.py
f4d6da4458fc3f6d44cd29e93af52d3847722152 16-Feb-2013 asharif <asharif@p4.invalid> Removed AndroidBench and added BootPerfServer to the toolchain
experiment file.

PRESUBMIT=passed
R=shenhan,bjanakiraman
DELTA=2 (0 added, 0 deleted, 2 changed)
RCL=60747-p2
RDATE=2012/05/25 14:10:46


P4 change: 42676951
14/crosperf/experiment_files/toolchain
c9ebd090bb6fc28887ecdae772389944e8e633b0 16-Feb-2013 asharif <asharif@p4.invalid> Removed obsolete option --profile_dir. Also made it fail with an error
if --inputs and --output is not specified.

PRESUBMIT=passed
R=xur,bjanakiraman
DELTA=8 (5 added, 3 deleted, 0 changed)
OCL=60738-p2
RCL=60746-p2
RDATE=2012/05/25 11:11:55


P4 change: 42676950
14/fdo_scripts/divide_and_merge_profiles.py
98f7dfdecc1499db01e4761ff3fccb271bc46cde 16-Feb-2013 asharif <asharif@p4.invalid> Changed Results class to use the python utility within ChromeOS
crostestutils to parse keyvals instead of re-inventing the wheel. This
also ensures that non-numeric keyvals are contained in the results
dictionary (for functional testsuites like bvt and smoke).

PRESUBMIT=passed
R=shenhan,bjanakiraman
DELTA=35 (14 added, 10 deleted, 11 changed)
OCL=60730-p2
RCL=60736-p2
RDATE=2012/05/24 14:26:52


P4 change: 42676933
14/crosperf/results_cache.py
8f89fbef4fd89afd1213444cdbd4e2ffaf809915 16-Feb-2013 asharif <asharif@p4.invalid> Added copying of *.imports files as well as *.gcda files for dividing
and merging. The *.imports file are needed for -fripa builds.

PRESUBMIT=passed
R=xur,bjanakiraman
DELTA=1 (0 added, 0 deleted, 1 changed)
OCL=60704-p2
RCL=60709-p2
RDATE=2012/05/23 15:37:52


P4 change: 42676888
14/fdo_scripts/divide_and_merge_profiles.py
f852f2c3a2a2fc52823854e75b8a22dd149bcfec 16-Feb-2013 asharif <asharif@p4.invalid> Removed perf_processor imports. crosperf was working in your (and my)
workspace because perf_processor.pyc still existed.

PRESUBMIT=passed
R=shenhan,bjanakiraman
DELTA=7 (0 added, 7 deleted, 0 changed)
OCL=60706-p2
RCL=60708-p2
RDATE=2012/05/23 15:36:59


P4 change: 42676887
14/crosperf/benchmark_run.py
14/crosperf/benchmark_run_unittest.py
14/crosperf/experiment.py
14/crosperf/results_cache.py
8d3a3d34672af0522a66c32748ad41556fc0dea2 16-Feb-2013 asharif <asharif@p4.invalid> Added sample experiment files to be used with crosperf. The file names
indicate the purpose of the experiment and there are some comments in
the files as well.

PRESUBMIT=passed
R=cmtice,bjanakiraman,shenhan,yunlian,llozano,carrot
DELTA=113 (113 added, 0 deleted, 0 changed)
OCL=60705-p2
RCL=60707-p2
RDATE=2012/05/23 15:22:45


P4 change: 42676886
14/crosperf/experiment_files/README
14/crosperf/experiment_files/aes_perf
14/crosperf/experiment_files/bloat_perf
14/crosperf/experiment_files/morejs_perf
14/crosperf/experiment_files/page_cycler
14/crosperf/experiment_files/page_cycler_perf
14/crosperf/experiment_files/toolchain
9f423f65224a738a0e4c9c87526919142b44681f 16-Feb-2013 asharif <asharif@p4.invalid> Added --vmlinux and --kallsyms to the perf report command.

PRESUBMIT=passed
R=bjanakiraman
DELTA=4 (4 added, 0 deleted, 0 changed)
OCL=60594-p2
RCL=60597-p2
RDATE=2012/05/18 14:30:57


P4 change: 42676713
14/crosperf/results_cache.py
eb3b69a87693ce83351c567d9d8b603dbed3e350 16-Feb-2013 llozano <llozano@p4.invalid> Added new option to specify chromeos root directory instead of using
"chromeos" by default.
Changed general header comment.

PRESUBMIT=passed
R=asharif
DELTA=15 (11 added, 0 deleted, 4 changed)
OCL=60589-p2
RCL=60595-p2
RDATE=2012/05/18 14:13:11


P4 change: 42676703
14/fdo_scripts/vanilla_vs_fdo.py
ef695669c4ec4d917f733db7efcb1ae31143904b 16-Feb-2013 asharif <asharif@p4.invalid> Refactored the results/caching code of crosperf.
All the work is now done by the Result class. This class is responsible
for collecting autotest keyvals as well as perf-related results. In case
of a cache hit, it extracts the results from a tarball and displays
them. This way it re-creates the perf.data and perf.report file in the
experiment results directory.

Tested: Ran crosperf on platform_AesThroughput twice to make sure there
was a cache hit the second time.
Also tested profile_type: record with profile_counters: cycles
instructions.

PRESUBMIT=passed
BUG=6434746
R=yunlian,llozano,bjanakiraman,shenhan,cmtice,carrot
DELTA=502 (251 added, 218 deleted, 33 changed)
OCL=60485-p2
RCL=60588-p2
RDATE=2012/05/18 10:46:35


P4 change: 42676700
14/crosperf/autotest_runner.py
14/crosperf/benchmark_run.py
14/crosperf/experiment_runner.py
14/crosperf/perf_processor.py
14/crosperf/results_cache.py
14/crosperf/results_sorter.py
14/crosperf/settings_factory.py
14/utils/misc.py
53982635da993c4e06620458efd0e286ae468c4b 16-Feb-2013 llozano <llozano@p4.invalid> Fix path for the "distfiles" directory

PRESUBMIT=passed
R=asharif
DELTA=3 (0 added, 3 deleted, 0 changed)
RCL=60556-p2
RDATE=2012/05/17 14:14:07


P4 change: 42676644
14/fdo_scripts/profile_cycler.py
fd54f25d991c89e8f0297c9d62b0ce592825d956 16-Feb-2013 asharif <asharif@p4.invalid> Added support for --multipliers for weighing profiles when doing merges.
Also refactored the unit tests and added one for testing multipliers.

PRESUBMIT=passed
R=xur,bjanakiraman,llozano
DELTA=64 (50 added, 6 deleted, 8 changed)
OCL=60552-p2
RCL=60555-p2
RDATE=2012/05/17 13:38:37


P4 change: 42676643
14/fdo_scripts/divide_and_merge_profiles.py
14/fdo_scripts/divide_and_merge_profiles_test.py
25fd4236ea5557a05c7ecff2a4fdde1fb05a70c5 16-Feb-2013 asharif <asharif@p4.invalid> Added an .rtr file for the next minor toolchain version, i.e.
branches/google/gcc-4_7.

PRESUBMIT=passed
R=shenhan,bjanakiraman
DELTA=26 (13 added, 13 deleted, 0 changed)
OCL=60452-p2
RCL=60506-p2
RDATE=2012/05/16 10:08:49


P4 change: 42676571
14/repo_to_repo_files/gcc-4.6.0_branches_google_4_6.rtr
14/repo_to_repo_files/gcc-branches_google_4_7.rtr
9c39a84639c19fee62e632c59c0cc83b38934f21 16-Feb-2013 shenhan <shenhan@p4.invalid> Fixed support for versioned check out.

While I was trying to get a specific version of chromeos, I found that the "--version" option no longer works. So came this CL.

Tested with version string "21.2273.0.0"

PRESUBMIT=passed
R=asharif
CC=llozano,yunlian,carrot,cmtice
OCL=60473-p2
RCL=60488-p2
RDATE=2012/05/15 21:39:08


P4 change: 42676543
14/setup_chromeos.py
936a65c496fd59a2f939ea9f1828a2ab5593ff9c 16-Feb-2013 asharif <asharif@p4.invalid> Changed results_cache.py to use a function in misc.py instead of
re-inventing the wheel.

PRESUBMIT=passed
R=llozano,bjanakiraman
DELTA=10 (1 added, 8 deleted, 1 changed)
RCL=60457-p2
RDATE=2012/05/14 16:42:22


P4 change: 42676498
14/crosperf/results_cache.py
6fc789bc4ec597b6178cd8ac87a4c59306b150ee 16-Feb-2013 asharif <asharif@p4.invalid> Added "gerrit": True to binutils-master. This will cause updates to go
through gerrit.

PRESUBMIT=passed
R=shenhan,bjanakiraman
DELTA=1 (1 added, 0 deleted, 0 changed)
OCL=60451-p2
RCL=60453-p2
RDATE=2012/05/14 16:17:47


P4 change: 42676495
14/repo_to_repo_files/binutils-master.rtr
84050d7bbaa86a70bf780102cb0325e0c24f351e 16-Feb-2013 asharif <asharif@p4.invalid> Removed -fopt-info=3 from the C[XX]FLAGS. These seem to be dumping only
to stderr.

PRESUBMIT=passed
R=llozano
DELTA=1 (0 added, 1 deleted, 0 changed)
OCL=60433-p2
RCL=60450-p2
RDATE=2012/05/14 14:33:20


P4 change: 42676488
14/fdo_scripts/vanilla_vs_fdo.py
ce393b7afc2748202c5165d533cc20c7d40a46c5 16-Feb-2013 asharif <asharif@p4.invalid> Changed the default # of iterations to 1 (from 3).

PRESUBMIT=passed
R=yunlian,cmtice,bjanakiraman
DELTA=2 (0 added, 0 deleted, 2 changed)
OCL=60402-p2
RCL=60449-p2
RDATE=2012/05/14 14:32:49


P4 change: 42676487
14/crosperf/settings_factory.py
a2015f297ebccc12c4a62f7b1f5257b4c401d6b2 16-Feb-2013 asharif <asharif@p4.invalid> Changed rtr file names to reflect the output branch where they push code
to.

PRESUBMIT=passed
R=llozano,cmtice,bjanakiraman
DELTA=108 (54 added, 54 deleted, 0 changed)
OCL=60445-p2
RCL=60448-p2
RDATE=2012/05/14 14:32:35


P4 change: 42676486
14/repo_to_repo_files/binutils-2.21_mobile_toolchain_v15_release_branch.rtr
14/repo_to_repo_files/binutils-master.rtr
14/repo_to_repo_files/gcc-4.6.0_branches_google_main.rtr
14/repo_to_repo_files/gcc-4.6.2_branches_google_4_6_3-mobile.rtr
14/repo_to_repo_files/gcc-branches_google_main.rtr
14/repo_to_repo_files/gcc-master.rtr
14/repo_to_repo_files/gdb-7.2.x_chromeos_master.rtr
14/repo_to_repo_files/gdb-master.rtr
47dcecfcbbd3f5a239817c0c03ac56787916c41d 16-Feb-2013 asharif <asharif@p4.invalid> Removed use of gdb to dump the renderer's profile. This is no longer
needed because Chrome supports --renderer-clean-exit and will dump the
correct data when killed with SIGINT.

PRESUBMIT=passed
R=llozano,cmtice,bjanakiraman
DELTA=317 (2 added, 311 deleted, 4 changed)
OCL=60396-p2
RCL=60401-p2
RDATE=2012/05/10 19:14:58


P4 change: 42676419
14/fdo_scripts/gdb_remote.dump
14/fdo_scripts/profile_cycler.py
60f06bd82d20ec68fda2addee74d81ad71df6af3 16-Feb-2013 asharif <asharif@p4.invalid> Added flags to enable the summarizer to be run on the FDO-run build
directory. llozano's summarizer should also be checked into this
directory but that can happen in a different CL.

PRESUBMIT=passed
R=llozano,bjanakiraman
DELTA=3 (2 added, 0 deleted, 1 changed)
OCL=60397-p2
RCL=60400-p2
RDATE=2012/05/10 19:14:47


P4 change: 42676418
14/fdo_scripts/vanilla_vs_fdo.py
915f0f99723a9286995fd64e5a2f6cce5c05de83 16-Feb-2013 asharif <asharif@p4.invalid> Made vanilla_vs_fdo.py work again with the newest ChromeOS pgo changes.
1. Added a flag: --build_image_args for passing arguments down to
build_image.
profiler_cycler.py:
1. Changed tarball location of data files.
2. Added GCOV_PREFIX_STRIP for correct pgo path.
3. Added --renderer-clean-exit to list of chrome flags.
vanilla_vs_fdo.py:
1. Added build_image_args to build_image to boost rootfs size.
2. Removed with Patcher(...): since it is no longer needed.
3. Replaced C[XX]FLAGS=-fprofile-generate by USE=pgo_generate.
4. Used chrome_internal everywhere to build internal Chrome.
build_chrome_browser.py:

PRESUBMIT=passed
R=llozano,bjanakiraman
DELTA=60 (10 added, 15 deleted, 35 changed)
OCL=60365-p2
RCL=60373-p2
RDATE=2012/05/09 16:58:38


P4 change: 42676379
14/build_chrome_browser.py
14/fdo_scripts/profile_cycler.py
14/fdo_scripts/vanilla_vs_fdo.py
d62d337c99d797c60b0dc463a4d9200e7e670c42 16-Feb-2013 asharif <asharif@p4.invalid> Expanded width of results column to 80 characters so PyAutoPerfTests
keys would not be shortened.

PRESUBMIT=passed
R=llozano,cmtice,bjanakiraman
DELTA=3 (0 added, 1 deleted, 2 changed)
OCL=60364-p2
RCL=60369-p2
RDATE=2012/05/09 15:29:57


P4 change: 42676377
14/crosperf/results_report.py
9e3cf6ef6dec37a592a449c6a9a73203d730e952 16-Feb-2013 asharif <asharif@p4.invalid> Changed build_tc.py's incremental mode to make sure make is always
called. make can detect if nothing needs to be done for the compile and
just takes a few seconds.

PRESUBMIT=passed
R=llozano,bjanakiraman
DELTA=3 (0 added, 1 deleted, 2 changed)
OCL=60298-p2
RCL=60312-p2
RDATE=2012/05/07 17:09:14


P4 change: 42676274
14/build_tc.py
96f5969ee4025fba1a762983e7e3ad3ea1b78531 16-Feb-2013 asharif <asharif@p4.invalid> Added a new script called test_toolchains.py. It will be used for our
nightly performance testing. Currently it takes in gcc githashes of
toolchains that it can test and we can extend it to do binutils and
others (glibc, gdb, etc.). It builds the toolchain using the githash and
tests it using crosperf.

PRESUBMIT=passed
R=bjanakiraman
CC=llozano,shenhan,cmtice,yunlian
OCL=59388-p2
RCL=60297-p2
RDATE=2012/05/07 14:19:26


P4 change: 42676254
14/test_toolchains.py
ac6c03e7915f33fd0b41384f0816c271d2236c4c 16-Feb-2013 bjanakiraman <bjanakiraman@p4.invalid> The code dealing with perf processing needs refactoring, but this change temporarily fixes
a bunch of issues. Namely, find perf.data and perf.out rather than assume locations, ensure
some consistency by having perf.report either have a report or stat ouput.

PRESUBMIT=passed
R=asharif,llozano
DELTA=49 (27 added, 5 deleted, 17 changed)
OCL=60185-p2
RCL=60208-p2
RDATE=2012/05/02 17:20:26


P4 change: 42676102
14/crosperf/perf_processor.py
b76e4387224d8f65c07cb19a9700b86091aaf8ad 16-Feb-2013 bjanakiraman <bjanakiraman@p4.invalid> Change directory name to use md5 sum.

PRESUBMIT=passed
R=asharif
DELTA=4 (3 added, 0 deleted, 1 changed)
RCL=59965-p2
RDATE=2012/04/20 12:46:47


P4 change: 42675450
14/crosperf/results_cache.py
5eb0c0c860f39acae614dc550b6c78a7efcc978a 16-Feb-2013 asharif <asharif@p4.invalid> Added an rtr file for gcc-4_6_3.rtr for mirroring the corresponding svn
branch.

PRESUBMIT=passed
R=bjanakiraman,shenhan
DELTA=28 (14 added, 14 deleted, 0 changed)
OCL=59812-p2
RCL=59843-p2
RDATE=2012/04/13 14:50:29


P4 change: 42675246
14/repo_to_repo_files/gcc-4.6.2_branches_google_4_6_2-mobile.rtr
14/repo_to_repo_files/gcc-4.6.2_branches_google_4_6_3-mobile.rtr
d985348c16de0ff57caf10a6dfbcaaf6b32adc1f 16-Feb-2013 shenhan <shenhan@p4.invalid> Added options '-g' (keep intermediate file) and '-t' (specifying testing tools).

Also fixed the test case failure dg.exp - nullptr21.C.

PRESUBMIT=passed
R=asharif
CC=llozano,cmtice,yunlian,bjanakiraman
OCL=59621-p2
RCL=59661-p2
RDATE=2012/04/06 09:32:29


P4 change: 42674567
14/dejagnu/chromeos.exp.in
14/dejagnu/run_dejagnu.py
b6fbd7ecf87065c2f9f55af7393f99955fd8a0b8 16-Feb-2013 shenhan <shenhan@p4.invalid> Add functionality to upload (board->host) runtime gcda files in exp file.

Previous implementation missed this so that all "gcda" related tests failed.

Test - previous failed profile gen tests pass.

PRESUBMIT=passed
R=asharif
CC=bhaskar-staff
APPROVED=asharif
OCL=59562-p2
RCL=59586-p2
RDATE=2012/04/04 09:17:23


P4 change: 42674482
14/dejagnu/chromeos.exp.in
14/dejagnu/run_dejagnu.py
72418e98df262d0ebe9dd8321df2543ee5e4d823 16-Feb-2013 llozano <llozano@p4.invalid> Changed path to the Chrome sources within var/cache directory

PRESUBMIT=passed
R=asharif
CC=asharif,bjanakiraman
DELTA=1 (0 added, 0 deleted, 1 changed)
RCL=59573-p2
RDATE=2012/04/03 16:02:52


P4 change: 42674458
14/utils/misc.py
4499026e62dfabfb300ead5d9eec4c5e63866ff1 16-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Add documentation for automation framework.

PRESUBMIT=passed
R=cmtice
CC=bhaskar-staff
DELTA=57 (48 added, 4 deleted, 5 changed)
OCL=59520-p2
RCL=59538-p2
RDATE=2012/03/30 13:19:23


P4 change: 42674416
14/automation/common/command_executer.py
14/automation/common/events.py
14/automation/common/state_machine.py
14/automation/server/job_manager.py
14/automation/server/machine_manager.py
14/automation/server/server.py
d32d768947b94f93542f5373d5b02de638e49117 16-Feb-2013 shenhan <shenhan@p4.invalid> Added a script and an exp template file to run dejagnu for atom and arm boards, and then compares the test result with a pre-existing base line.

PRESUBMIT=passed
R=asharif,bjanakiraman,kbaclawski
CC=yunlian,llozano,cmtice
APPROVED=kbaclawski
OCL=59468-p2
RCL=59527-p2
RDATE=2012/03/29 21:57:12


P4 change: 42674395
14/dejagnu/chromeos.exp.in
14/dejagnu/run_dejagnu.py
05e7a00e15e1d711b8b5f4540d83f53f2e8cdc98 16-Feb-2013 asharif <asharif@p4.invalid> Added command line options for individual builds for vanilla_vs_fdo.py.

PRESUBMIT=passed
R=cmtice,bjanakiraman,llozano
DELTA=51 (31 added, 0 deleted, 20 changed)
OCL=59379-p2
RCL=59460-p2
RDATE=2012/03/26 18:38:45


P4 change: 42674292
14/fdo_scripts/vanilla_vs_fdo.py
6578cf866da13d60b79d61b2479e75d60c665edd 16-Feb-2013 asharif <asharif@p4.invalid> Added an option to repo_to_repo's rtr files called "revision" which can
optionally sync to a particular revision.

PRESUBMIT=passed
R=shenhan,bjanakiraman
DELTA=6 (3 added, 0 deleted, 3 changed)
OCL=59217-p2
RCL=59378-p2
RDATE=2012/03/22 15:11:37


P4 change: 42674156
14/repo_to_repo.py
d9b6765307b29ede85e99dd1eb417c889aa9da07 16-Feb-2013 asharif <asharif@p4.invalid> Fixed missing import of utils.misc.
Also fixed missing _test_ in finding the image since ./build_image test
creates an image with that name.

PRESUBMIT=passed
R=shenhan,bjanakiraman
DELTA=2 (1 added, 0 deleted, 1 changed)
RCL=59363-p2
RDATE=2012/03/21 19:32:27


P4 change: 42674133
14/image_chromeos.py
1394893841c1bcc1edd7c97f5211ecb867fc77bb 16-Feb-2013 asharif <asharif@p4.invalid> Added copying of perf.data file to the experiment directory.

PRESUBMIT=passed
R=bjanakiraman,shenhan
DELTA=7 (7 added, 0 deleted, 0 changed)
OCL=59347-p2
RCL=59356-p2
RDATE=2012/03/21 13:12:34


P4 change: 42674126
14/crosperf/experiment_runner.py
786b69082e581302396e707ae548bb5fcce15e6c 16-Feb-2013 asharif <asharif@p4.invalid> Refactored some code in benchmark_run.py. Made some variables class
variables so they will not be passed around in this class. This is in
preparation for a CL to add perf.data in the experiment directory after
all runs have been completed.

Also refactored perf_processor.py similarly.

PRESUBMIT=passed
R=llozano,bjanakiraman
DELTA=69 (26 added, 10 deleted, 33 changed)
OCL=59343-p2
RCL=59355-p2
RDATE=2012/03/21 13:12:29


P4 change: 42674125
14/crosperf/benchmark_run.py
14/crosperf/experiment.py
14/crosperf/perf_processor.py
ee9d100c66939659aa3f3e5377772da59615f4e1 16-Feb-2013 kbaclawski <kbaclawski@p4.invalid> DejaGNU log tools: add documentation and simplify some code.

PRESUBMIT=passed
R=dougkwan
CC=bhaskar-staff
DELTA=138 (51 added, 22 deleted, 65 changed)
OCL=59323-p2
RCL=59329-p2
RDATE=2012/03/20 16:28:26


P4 change: 42674091
14/automation/clients/report/dejagnu/main.py
14/automation/clients/report/dejagnu/manifest.py
14/automation/clients/report/dejagnu/report.py
14/automation/clients/report/dejagnu/summary.py
9e49916221f8101feafb8a0fae093bd41c603b58 16-Feb-2013 asharif <asharif@p4.invalid> Added FEATURES="-buildpkg" to emerge when building the toolchain.
Tests: Did an incremental build/install of gcc.
Before this CL: 12m 19 seconds.
After this CL: 39 seconds.

PRESUBMIT=passed
R=llozano,bjanakiraman
DELTA=2 (1 added, 0 deleted, 1 changed)
OCL=59320-p2
RCL=59322-p2
RDATE=2012/03/20 15:29:11


P4 change: 42674084
14/build_tc.py
7928d5f0329893e45b90a57c643cc19df27ee172 16-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Rewrite the script to utilize library for processing DejaGNU output.

Tested:
Generate a manifest (list of suppressed failures):
$ validate_failures.py -m -f -v
Checking test results for unexpected failures:
$ validate_failures.py -v

PRESUBMIT=passed
R=dougkwan,dnovillo
CC=cmtice,bhaskar-staff,llozano,shenhan
DELTA=264 (250 added, 8 deleted, 6 changed)
OCL=56295-p2
RCL=59312-p2
RDATE=2012/03/20 12:12:05


P4 change: 42674079
14/automation/clients/report/dejagnu/manifest.py
14/automation/clients/report/dejagnu/summary.py
14/automation/clients/report/validate_failures.py
eefa12cb6939a929096b0e4fd2c7b12e9a8b561f 16-Feb-2013 asharif <asharif@p4.invalid> Removed dead functions.

PRESUBMIT=passed
R=bjanakiraman,yunlian
DELTA=32 (0 added, 32 deleted, 0 changed)
OCL=59308-p2
RCL=59311-p2
RDATE=2012/03/20 11:20:35


P4 change: 42674078
14/crosperf/benchmark_run.py
cc9d6d700fc806fa0324aa28e7e5ad1922ee7a6d 16-Feb-2013 asharif <asharif@p4.invalid> Added a new cache condition which is disabled by default. If you rename
an image file, it still uses the cached results (useful if you label the
image path by setting up a symlink).

PRESUBMIT=passed
R=shenhan,bjanakiraman
DELTA=10 (9 added, 0 deleted, 1 changed)
OCL=59282-p2
RCL=59306-p2
RDATE=2012/03/20 10:53:30


P4 change: 42674063
14/crosperf/results_cache.py
529f8d25cab15ca1cdbfc72eb3330b2f8b56ebf1 16-Feb-2013 asharif <asharif@p4.invalid> Added revision argument to Sync() method to sync at a particular
revision. Made the default sync to ToT.

PRESUBMIT=passed
R=llozano,kbaclawski,bjanakiraman
CC=bhaskar-staff
DELTA=5 (3 added, 0 deleted, 2 changed)
OCL=59216-p2
RCL=59220-p2
RDATE=2012/03/16 13:18:28


P4 change: 42673933
14/automation/clients/helper/perforce.py
1e54bf2cc5ddf8dbae7774559847c95f8c101431 16-Feb-2013 asharif <asharif@p4.invalid> Removed older binutils rtr files. Made the current one map
mobile_toolchain_v15_release_branch to the "master" branch in git. This
is needed for cros_workon-ization of toolchain packages.

PRESUBMIT=passed
R=bjanakiraman,llozano,shenhan
DELTA=33 (0 added, 33 deleted, 0 changed)
OCL=59186-p2
RCL=59209-p2
RDATE=2012/03/16 11:03:18


P4 change: 42673912
14/repo_to_repo_files/binutils-2.20.1_mobile_toolchain_v14_release_branch.rtr
14/repo_to_repo_files/binutils-2.21_master.rtr
14/repo_to_repo_files/binutils-2.21_mobile_toolchain_v15_release_branch.rtr
63201e64cc3725b119a6c53242b3e8b7d22ac2d1 16-Feb-2013 asharif <asharif@p4.invalid> vanilla_vs_fdo.py now builds a Chrome with -fprofile-generate and uses
it to build with -fprofile-use (in addition to USE="+pgo" vs
USE="-pgo"). Also added an option to do the aforementioned steps on the
chromeos-chrome-9999 ebuild.

PRESUBMIT=passed
R=bjanakiraman,cmtice,llozano
DELTA=193 (167 added, 0 deleted, 26 changed)
OCL=59063-p2
RCL=59183-p2
RDATE=2012/03/15 16:00:23
USE="-pgo").


P4 change: 42673868
14/fdo_scripts/vanilla_vs_fdo.py
44aff655bf3cb5156ad4e081d7b0228c3229bd0f 16-Feb-2013 asharif <asharif@p4.invalid> This CL updates our scripts because the underlying ChromeOS scripts'
interface has changed:
1. Removed --withdev from ./build_image because it is no longer present
in the ChromeOS scripts. Added test to the arguments because that is now
the supported way of building test images.
2. Removed mod_image_for_test.sh since it is deprecated. Removed callers
of this function.
3. Changed image_chromeos.py to look for chromiumos_test_image.bin
before chromiumos_image.bin.
4. Changed GetAllImages() to look for chromiumos_test_image.bin.
5. Shortened the symlink name in LabelImage to the basename.
6. Fixed an issue with me_browser.pywhen emerging the 9999 version of
chromeos-chrome.

PRESUBMIT=passed
R=llozano,bjanakiraman
DELTA=36 (10 added, 17 deleted, 9 changed)
OCL=59071-p2
RCL=59182-p2
RDATE=2012/03/15 16:00:19


P4 change: 42673867
14/build_chrome_browser.py
14/build_chromeos.py
14/image_chromeos.py
14/utils/misc.py
12ecc59de2422a1dc0d2362a448dadd5ceaa91ae 16-Feb-2013 asharif <asharif@p4.invalid> Changed utils to misc after kbaclawski's change: cl/59036-p2.

PRESUBMIT=passed
R=shenhan,bjanakiraman
DELTA=1 (0 added, 0 deleted, 1 changed)
OCL=59097-p2
RCL=59102-p2
RDATE=2012/03/13 17:43:04


P4 change: 42672887
14/fdo_scripts/profile_cycler.py
983c7c479bc250f7d3907211704f05d739bbe940 16-Feb-2013 asharif <asharif@p4.invalid> Added new option to repo_to_repo.py: -F. This enables it to use your own
commit message instead of the default:
Sync'd repos to: <revision>.
This is useful for repos with "gerrit": True.

PRESUBMIT=passed
R=shenhan,bjanakiraman
DELTA=18 (14 added, 0 deleted, 4 changed)
OCL=59039-p2
RCL=59091-p2
RDATE=2012/03/13 14:06:43


P4 change: 42672876
14/repo_to_repo.py
52ff307ff92b866868428cff2f50fe127c704315 16-Feb-2013 asharif <asharif@p4.invalid> Added --ebuild_version to build_chrome_browser.py. This can be used, for
example, to build the 9999 version of the Chrome browser.

PRESUBMIT=passed
R=yunlian,bjanakiraman
DELTA=14 (12 added, 0 deleted, 2 changed)
OCL=59060-p2
RCL=59083-p2
RDATE=2012/03/13 12:50:13


P4 change: 42672863
14/build_chrome_browser.py
20082a07cd83cc4cf11ce35566e2d20266c60997 16-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Rename utils.utils to utils.misc to prevent Python from being confused.

Tested:
N/A

PRESUBMIT=passed
R=asharif
DELTA=506 (234 added, 210 deleted, 62 changed)
OCL=59036-p2
RCL=59059-p2
RDATE=2012/03/12 13:50:53


P4 change: 42672835
14/build_benchmarks.py
14/build_chrome_browser.py
14/build_chromeos.py
14/build_tc.py
14/command_executer_timeout_test.py
14/compare_benchmarks.py
14/crb/crb_driver.py
14/crosperf/autotest_runner.py
14/crosperf/perf_processor.py
14/crosperf/results_cache.py
14/fdo_scripts/divide_and_merge_profiles.py
14/fdo_scripts/divide_and_merge_profiles_test.py
14/fdo_scripts/profile_cycler.py
14/fdo_scripts/vanilla_vs_fdo.py
14/remote_kill_test.py
14/remote_test.py
14/repo_to_repo.py
14/run_benchmarks.py
14/run_tests.py
14/tc_enter_chroot.py
14/utils/command_executer.py
14/utils/logger.py
14/utils/misc.py
14/utils/misc_test.py
14/utils/utils.py
14/utils/utils_test.py
db9183a3bd0abc3e0217d226db24e7001f3de8bf 16-Feb-2013 asharif <asharif@p4.invalid> Changed --profiles_dir to --profile_dir since only a single profile is
stored there.
Also now the profiles are stored in <profile_dir>/$ctarget since that is
how the git repo is also orgranized (there is a separate profile dir per
ctarget).

PRESUBMIT=passed
R=cmtice,bjanakiraman
DELTA=9 (0 added, 1 deleted, 8 changed)
OCL=59037-p2
RCL=59045-p2
RDATE=2012/03/10 10:34:00


P4 change: 42672746
14/fdo_scripts/profile_cycler.py
2aeeaca94ea584fc53dcfc99f32e05aa5c88937c 16-Feb-2013 asharif <asharif@p4.invalid> Added feature to repo_to_repo.py to create a gerrit CL instead of
directly pushing to the repository. Use it by adding a "gerrit": True in
the rtr dictionary file.

PRESUBMIT=passed
R=shenhan,bjanakiraman
DELTA=17 (12 added, 0 deleted, 5 changed)
OCL=59023-p2
RCL=59030-p2
RDATE=2012/03/09 11:39:38


P4 change: 42672733
14/repo_to_repo.py
14/repo_to_repo_files/gcc-4.6.2_branches_google_4_6_2-mobile.rtr
54cf678a9308eef166c2dc0b6e8ff63a369e0ee9 16-Feb-2013 asharif <asharif@p4.invalid> cros_login.py now uses unique host filenames for storing the script
contents before shipping them to the device.

PRESUBMIT=passed
R=bjanakiraman
DELTA=13 (5 added, 0 deleted, 8 changed)
OCL=59018-p2
RCL=59020-p2
RDATE=2012/03/08 17:42:38


P4 change: 42672727
14/cros_login.py
44b122a1ced88d3803eb570333afc6cb3804ec92 16-Feb-2013 asharif <asharif@p4.invalid> Added command_timeout and terminated_timeout to ChrootRunCommand() and
CrosRunCommand(). These were already there in RunCommand().

PRESUBMIT=passed
R=llozano,bjanakiraman
DELTA=9 (5 added, 0 deleted, 4 changed)
OCL=58985-p2
RCL=59017-p2
RDATE=2012/03/08 16:53:15


P4 change: 42672726
14/utils/command_executer.py
795af1d7a3f39ee42249783f08a4784d39887c95 16-Feb-2013 asharif <asharif@p4.invalid> Added the following options to build_chrome_browser.py:
1. --clean to purge the output directory before building.
2. --env to specify an environment like USE=something to pass to the
chrome emerge command.

PRESUBMIT=passed
R=shenhan,bjanakiraman
DELTA=20 (19 added, 0 deleted, 1 changed)
OCL=58987-p2
RCL=59012-p2
RDATE=2012/03/08 14:25:28


P4 change: 42672707
14/build_chrome_browser.py
e0879f94acdc285df4cdebb0494c2a2bbea80b0a 16-Feb-2013 asharif <asharif@p4.invalid> Added functions to utils.py:
1. GetAllImages() returns all images given a chromeos_root and board.
2. GetEnvStringFromDict() returns an env string given an env dictionary.
3. GetFilenameFromString() takes a string and converts it into a valid
filename (for hashing purposes, etc.).
4. GetImageDir() returns the image dir given a chromeos root and board.
5. *Label*() functions deal with labeling a chromeos image (by
symlinking to it). Useful for distinguishing images built with certain
USE flags, for example.
Added usepkg to GetBuildPackagesCommand().

PRESUBMIT=passed
R=kbaclawski,bjanakiraman
DELTA=82 (73 added, 1 deleted, 8 changed)
OCL=58986-p2
RCL=59009-p2
RDATE=2012/03/08 13:13:07


P4 change: 42672706
14/utils/utils.py
14/utils/utils_test.py
e79df31dcf9311fa9c0f08059d918ceb725d2273 16-Feb-2013 asharif <asharif@p4.invalid> Added a new script called vanilla_vs_fdo.py.

This creates a ChromeOS checkout, and builds Chrome with and without
PGO, labeling images as such. If labels exist nothing is done so this
script works if you press Ctrl + c in the middle.

After building both images, it can run them using crosperf and send you
the report.

I plan to change nightly to run this script every night to track PGO
performance vs. non-PGO.

PRESUBMIT=passed
R=bjanakiraman,shenhan
DELTA=126 (126 added, 0 deleted, 0 changed)
OCL=58997-p2
RCL=59006-p2
RDATE=2012/03/08 10:22:04


P4 change: 42672703
14/fdo_scripts/vanilla_vs_fdo.py
6465e6254b1723a1e22d79a3f56606f0ae82e582 16-Feb-2013 asharif <asharif@p4.invalid> Sometimes profile_cycler.py gets stuck when Chrome runs out of memory
and crashes before the debugger can attach to it. It can also run out of
memory or crash even after the debugger is attached to it.

This CL adds a timeout to dumping of the profile so the script doesn't
get stuck when running.

PRESUBMIT=passed
R=cmtice,bjanakiraman
DELTA=6 (5 added, 0 deleted, 1 changed)
OCL=58990-p2
RCL=58996-p2
RDATE=2012/03/07 21:10:31


P4 change: 42672688
14/fdo_scripts/profile_cycler.py
5859e1b092b87802c320e4b804aaa473348df040 16-Feb-2013 asharif <asharif@p4.invalid> Updated the function GetCtargetFromBoard(). ChromeOS can now have
multiple CTARGETs per board and it returns the primary one.

PRESUBMIT=passed
R=bjanakiraman,llozano,shenhan
DELTA=3 (0 added, 0 deleted, 3 changed)
OCL=58837-p2
RCL=58871-p2
RDATE=2012/03/02 14:05:52


P4 change: 42669966
14/utils/utils.py
d2ced689f8a7c2727c042a30dacb15a38ec42601 16-Feb-2013 asharif <asharif@p4.invalid> Improved profiling of Chrome for ChromeOS:
1. Added handle SIGPIPE in the gdb script. This prevents gdb from
hanging when a SIGPIPE is received.
2. Removed redundant sleep in profile_cycler.py and moved that to the
command timeout.
3. Made sure all chrome processes are killed and the ui is restarted
between profiler runs.
4. Fixed a bug where the profile directory was being deleted before each
cycler run. It should only be deleted once at the beginning.

PRESUBMIT=passed
R=cmtice,bjanakiraman
DELTA=40 (22 added, 8 deleted, 10 changed)
OCL=58738-p2
RCL=58857-p2
RDATE=2012/03/02 10:53:27


P4 change: 42669941
14/cros_login.py
14/fdo_scripts/gdb_remote.dump
14/fdo_scripts/profile_cycler.py
bcdd4e58c86a8ae52fe6dd5f9de5dad58f57d252 16-Feb-2013 asharif <asharif@p4.invalid> Do not switch linkers when building the toolchain. This is unnecessary
now because of https://gerrit.chromium.org/gerrit/#change,16451.

Also fixed a missing default for --chromeos_root.

PRESUBMIT=passed
R=cmtice,llozano,bjanakiraman
DELTA=10 (1 added, 9 deleted, 0 changed)
OCL=58838-p2
RCL=58856-p2
RDATE=2012/03/02 10:51:53


P4 change: 42669940
14/build_tc.py
7bf41e7234f06c5e6d5b280c82d9279d7e8a1f81 16-Feb-2013 asharif <asharif@p4.invalid> Changed the way results are parsed & stored.

Earlier results were parsed by inspecting the output of the autotest.
That method is brittle because the format of the output can change. It
also cannot distinguish between passing and failing sub-tests.

Now it reads the results directly from the keyval in the autotest dir
(this is similar to how cautotest/ generates its results).

Also updated the cache version string which invalidates all previous
cached results (since the storage format of the cache changed).

PRESUBMIT=passed
R=shenhan,bjanakiraman,llozano
DELTA=48 (34 added, 4 deleted, 10 changed)
OCL=58739-p2
RCL=58839-p2
RDATE=2012/03/01 18:02:06


P4 change: 42669911
14/crosperf/benchmark_run.py
14/crosperf/results_cache.py
180f31a3e339fe7b484f5de75edc0ee56f36aaef 16-Feb-2013 asharif <asharif@p4.invalid> Added new function in utils called GetChromeSrcDir(), which returns the
source directory of Chrome. This is used when profiling Chrome on
ChromeOS.

PRESUBMIT=passed
R=shenhan,bjanakiraman
DELTA=5 (4 added, 0 deleted, 1 changed)
OCL=58620-p2
RCL=58628-p2
RDATE=2012/02/22 20:13:44


P4 change: 42667769
14/fdo_scripts/profile_cycler.py
14/utils/utils.py
bb97b6e06deaae98ff8df2a610d9bb9116baa877 16-Feb-2013 asharif <asharif@p4.invalid> Changed the chromiumos gcc.git repository in the following way:
1. The "master" branch now holds the sources for the current released
gcc.
2. Upstream svn google/main is now tracked by a branch called
gcc.gnu.org/branches/google/main.
3. All gcc branches now hold the sources in ./ instead of 2 directories
down. This simplies things in the ebuild and it doesn't have to worry
about differences in directories like: gcc/4.6.0 vs. gcc/4.6.2.

This change is needed because we will soon be shifting to cros_workon
toolchain packages. That means a single branch ("master" for us), will
hold the sources for the currently released compiler.
When that branch is updated, an automatic uprev of the gcc package will
occur.

PRESUBMIT=passed
R=bjanakiraman,shenhan,yunlian
DELTA=15 (13 added, 0 deleted, 2 changed)
OCL=58311-p2
RCL=58331-p2
RDATE=2012/02/09 15:48:37


P4 change: 42667272
14/repo_to_repo_files/gcc-4.6.0_branches_google_main.rtr
14/repo_to_repo_files/gcc-4.6.2_branches_google_4_6_2-mobile.rtr
1d78706f8a1a04acae384d417130ae13edf58500 16-Feb-2013 asharif <asharif@p4.invalid> Removed .rtr files that are no longer needed.

PRESUBMIT=passed
R=shenhan,bjanakiraman,cmtice
DELTA=50 (0 added, 50 deleted, 0 changed)
OCL=58312-p2
RCL=58330-p2
RDATE=2012/02/09 15:48:31


P4 change: 42667271
14/repo_to_repo_files/gcc-4.4.3_master.rtr
14/repo_to_repo_files/gcc-4.4.3_mobile_toolchain_v14_release_branch.rtr
14/repo_to_repo_files/gcc-4.6.0_branches_google_4_6-mobile.rtr
14/repo_to_repo_files/gcc-4.6.0_master.rtr
051b46f107b210bd3fb4d6250b3a6d0af3d7cea3 16-Feb-2013 asharif <asharif@p4.invalid> The default experiment name if no "name:" field is present in the
experiment file is the filename itself. This makes it more user friendly
as there is less redundancy and the email subject is easier to search
(it is the file name instead of "Experiment").

PRESUBMIT=passed
R=bjanakiraman,shenhan,cmtice,llozano
DELTA=4 (3 added, 0 deleted, 1 changed)
OCL=58249-p2
RCL=58313-p2
RDATE=2012/02/08 20:08:23


P4 change: 42667237
14/crosperf/crosperf.py
14/crosperf/settings_factory.py
9b155413e7bab7a4097bb190d1c6f79a767d52c1 16-Feb-2013 asharif <asharif@p4.invalid> Always use the chromeos_root associated with the image, if possible.
This saves an expensive 2GB copy when running crosperf with multiple
images.

PRESUBMIT=passed
R=bjanakiraman,shenhan,llozano
DELTA=5 (4 added, 0 deleted, 1 changed)
RCL=58247-p2
RDATE=2012/02/02 18:02:10


P4 change: 42666494
14/crosperf/machine_manager.py
567bd9a484fe872bbe52f7d5e393be6145fefec5 16-Feb-2013 asharif <asharif@p4.invalid> Added file for keeping gcc-4_6_2-mobile branch in sync with chromiumos
git repository.

PRESUBMIT=passed
R=shenhan,bjanakiraman,cmtice,yunlian
DELTA=13 (13 added, 0 deleted, 0 changed)
OCL=58224-p2
RCL=58235-p2
RDATE=2012/02/02 11:20:17


P4 change: 42666481
14/repo_to_repo_files/gcc-4.6.2_branches_google_4_6_2-mobile.rtr
afd4d4bb4e2bad90d311beb8c4e930ee98832d90 16-Feb-2013 asharif <asharif@p4.invalid> Updated toolchain-utils.rtr to include crosperf and needed supporting
files to toolchain-utils.git.

PRESUBMIT=passed
R=shenhan,cmtice,bjanakiraman,yunlian
DELTA=10 (0 added, 3 deleted, 7 changed)
OCL=58198-p2
RCL=58209-p2
RDATE=2012/01/31 15:48:05


P4 change: 42666429
14/repo_to_repo_files/toolchain-utils.rtr
1e7d4a2249054331a0ea92cedf0d7b65f6cc01f9 16-Feb-2013 asharif <asharif@p4.invalid> Fixed PYTHONPATH in crosperf to be relative to the script dir (as
opposed to the current working dir).

PRESUBMIT=passed
R=shenhan,cmtice,bjanakiraman
DELTA=1 (0 added, 0 deleted, 1 changed)
OCL=58195-p2
RCL=58208-p2
RDATE=2012/01/31 15:33:50


P4 change: 42666428
14/crosperf/crosperf
84b5cb8e547ffdbccccb84ecb4dafc310b6cbd2f 16-Feb-2013 asharif <asharif@p4.invalid> Removed unused imports from image_chromeos.py.

PRESUBMIT=passed
R=yunlian,cmtice,bjanakiraman
DELTA=2 (0 added, 2 deleted, 0 changed)
OCL=58196-p2
RCL=58207-p2
RDATE=2012/01/31 15:33:40


P4 change: 42666427
14/image_chromeos.py
cfb451ff1d4bdbca04d087b939fc342ddfde94c4 16-Feb-2013 asharif <asharif@p4.invalid> Made ChrootRunCommand() thread-safe by putting the cd into the child
process.

PRESUBMIT=passed
R=bjanakiraman,llozano,shenhan,yunlian
DELTA=5 (0 added, 0 deleted, 5 changed)
OCL=58075-p2
RCL=58197-p2
RDATE=2012/01/30 16:32:47


P4 change: 42666420
14/utils/command_executer.py
22632bd37213ed11c3f54a74fed19ffed9c468cf 16-Feb-2013 asharif <asharif@p4.invalid> Fixed missing spaces in autotest_runner.py. Also removed -g as the
default perf option.

With -g on PageCycler, the size of the reports is 150+ MB.

PRESUBMIT=passed
R=shenhan,bjanakiraman
DELTA=3 (0 added, 1 deleted, 2 changed)
OCL=58077-p2
RCL=58082-p2
RDATE=2012/01/25 11:18:33


P4 change: 42664861
14/crosperf/autotest_runner.py
ab1605898811dec09cef2cecc7d45b6af8a1b554 16-Feb-2013 asharif <asharif@p4.invalid> Limited the size of the perf report to 1000 lines. If perf record is
used with -g, the size of the perf report is 150+ MB on PageCycler.

PRESUBMIT=passed
R=cmtice,bjanakiraman
DELTA=1 (0 added, 0 deleted, 1 changed)
OCL=58076-p2
RCL=58080-p2
RDATE=2012/01/25 11:10:08


P4 change: 42664858
14/crosperf/perf_processor.py
1b32505c1a82b02963c2a181a79c1a8d66507694 16-Feb-2013 asharif <asharif@p4.invalid> Changed image_chromeos.py to not encapsulate the ChromeOS command in
single quotes. The single quotes are no longer necessary since
cl/57652-p2.

PRESUBMIT=passed
R=shenhan,bjanakiraman
DELTA=2 (0 added, 0 deleted, 2 changed)
OCL=58068-p2
RCL=58074-p2
RDATE=2012/01/25 10:26:32


P4 change: 42664855
14/image_chromeos.py
dc301aa6d9f2344d1c4408d45f06f9006ff18c32 16-Feb-2013 asharif <asharif@p4.invalid> Added missing getpass import.

PRESUBMIT=passed
R=shenhan,bjanakiraman,llozano
DELTA=1 (1 added, 0 deleted, 0 changed)
OCL=58054-p2
RCL=58056-p2
RDATE=2012/01/24 14:59:32


P4 change: 42664827
14/crosperf/experiment_runner.py
a7078a14b800ae2f7f30ea17a14e74b81f613202 16-Feb-2013 asharif <asharif@p4.invalid> Added code to email the cc and bcc as well as the to.

PRESUBMIT=passed
R=cmtice,bjanakiraman
DELTA=2 (2 added, 0 deleted, 0 changed)
OCL=58001-p2
RCL=58007-p2
RDATE=2012/01/20 15:31:17


P4 change: 42664754
14/utils/email_sender.py
0186f573e2ae7af2db74b145801072d9bca2b643 16-Feb-2013 cmtice <cmtice@p4.invalid> Add missing format to format string (fix a bug).

PRESUBMIT=passed
R=asharif
APPROVED=asharif
DELTA=1 (0 added, 0 deleted, 1 changed)
OCL=57944-p2
RCL=57952-p2
RDATE=2012/01/17 16:52:13


P4 change: 42664645
14/tc_enter_chroot.py
80d257fc434147e60bdf7b078868e98239fc870b 16-Feb-2013 cmtice <cmtice@p4.invalid> Modify build_tc.py to take a gdb_dir as an argument, and to also take
an option to allow unmounting as well as mounting (in case you need
to unmount one source tree to mount a different one).

PRESUBMIT=passed
R=asharif
APPROVED=asharif
DELTA=39 (29 added, 3 deleted, 7 changed)
OCL=57943-p2
RCL=57949-p2
RDATE=2012/01/17 16:15:40


P4 change: 42664642
14/build_tc.py
86968c423e74dbddf89d54f3f27182060713ab97 16-Feb-2013 asharif <asharif@p4.invalid> Added option --mount_only to build_tc.py that only mounts the tool
directory without doing the actual build.

PRESUBMIT=passed
R=cmtice,bjanakiraman
DELTA=19 (16 added, 0 deleted, 3 changed)
OCL=57881-p2
RCL=57884-p2
RDATE=2012/01/13 13:42:08


P4 change: 42664510
14/build_tc.py
567568658b66d7d682d0803a4a8f9b9187361540 16-Feb-2013 asharif <asharif@p4.invalid> Added script to send an email to the current sheriff (determined by
weekly rotation).

Refactored email_sender.py to make it a bit more general.

PRESUBMIT=passed
R=bjanakiraman,shenhan,cmtice,yunlian
DELTA=150 (136 added, 0 deleted, 14 changed)
OCL=57783-p2
RCL=57834-p2
RDATE=2012/01/11 15:53:07


P4 change: 42664422
14/crosperf/experiment_runner.py
14/sheriff_rotation.py
14/utils/email_sender.py
2154d35e3582d9eb0b91626e13d2e161bcbd2365 16-Feb-2013 asharif <asharif@p4.invalid> Fixed bugs and added features to profile_cycler.py.

Features:
* Specify --cycler=all to profile all the cyclers.
* The script strips off Chrome's build path so the profile dir
contains c/Release/...

Bug fixes:
* The script can now be run multiple times in sequence correctly:
** gdbserver is killed before running gdb_remote.
** The renderer is killed after dumping the profile.

PRESUBMIT=passed
R=cmtice,bjanakiraman
APPROVED=cmtice
DELTA=53 (41 added, 6 deleted, 6 changed)
OCL=57717-p2
RCL=57791-p2
RDATE=2012/01/10 12:03:20


P4 change: 42664327
14/fdo_scripts/profile_cycler.py
b096c879552abb73acf8025e78c818d1f4bc8212 16-Feb-2013 asharif <asharif@p4.invalid> Added command logging for chroot and cros remote commands.

PRESUBMIT=passed
R=yunlian,bjanakiraman
DELTA=3 (3 added, 0 deleted, 0 changed)
OCL=57716-p2
RCL=57720-p2
RDATE=2012/01/06 15:26:12


P4 change: 42664199
14/utils/command_executer.py
69c8914824e2d3f0ddee0cde2936677fc1711e42 16-Feb-2013 asharif <asharif@p4.invalid> Removed untarring of the autotest files to the experiment results
directory.

Running a large experiment causes hundreds of tarballs to be extracted
at once, rendering the machine unusable for minutes.

PRESUBMIT=passed
R=shenhan,raymes,bjanakiraman
DELTA=10 (3 added, 7 deleted, 0 changed)
OCL=57647-p2
RCL=57704-p2
RDATE=2012/01/06 11:15:56


P4 change: 42664189
14/crosperf/experiment_runner.py
3b30395f032ff7aab615625809072e855dfd5d8e 16-Feb-2013 asharif <asharif@p4.invalid> Added script for dividing profile directories and merging them. This
works around an out-of-memory problem with gcov-merge_v15.

Also added a test for this script.

PRESUBMIT=passed
R=shenhan,bjanakiraman
OCL=57631-p2
RCL=57703-p2
RDATE=2012/01/06 11:15:42


P4 change: 42664188
14/fdo_scripts/divide_and_merge_profiles.py
14/fdo_scripts/divide_and_merge_profiles_test.py
93f2788fd80be1d73d2236bcd92a3159cf16be28 16-Feb-2013 asharif <asharif@p4.invalid> Fixed missing import in profile_cycler.sh.

Added code to print exceptions when encountered.

Added code to remove the profile directory on the remove and local
machine before doing the profiling run.

Leading directories are now stripped before writing the profile files to
the host.

PRESUBMIT=passed
R=shenhan
DELTA=20 (18 added, 0 deleted, 2 changed)
OCL=57684-p2
RCL=57702-p2
RDATE=2012/01/06 11:14:58


P4 change: 42664187
14/fdo_scripts/profile_cycler.py
5ab1c7d242916355dda570153d1554e4c2963ce8 16-Feb-2013 asharif <asharif@p4.invalid> Fixed retval bug in cros_login.py.

PRESUBMIT=passed
R=shenhan
DELTA=1 (0 added, 0 deleted, 1 changed)
OCL=57683-p2
RCL=57685-p2
RDATE=2012/01/05 16:57:06


P4 change: 42664133
14/cros_login.py
7ade671df9a994e9838058eda8a34399752c7b89 16-Feb-2013 asharif <asharif@p4.invalid> Added check before doing a division to prevent potential
ZeroDivisionError.

PRESUBMIT=passed
R=shenhan,bjanakiraman
DELTA=7 (6 added, 0 deleted, 1 changed)
OCL=57635-p2
RCL=57654-p2
RDATE=2012/01/04 20:14:38


P4 change: 42662799
14/crosperf/results_columns.py
345742ef1b1180d3adf97ba2bbff1907bf8dd26c 16-Feb-2013 asharif <asharif@p4.invalid> Added support to execute bash commands with quotes and other operators
like &&, etc. to CrosRunCommand().

It now creates a unique file and dumps all the commands to the file,
copies the file over to the remote, and finally executes that file using
bash on the remote.

Also added command_timeout to CrosRunCommand().

This is a pre-requisite to some of the scripts I'll be adding later for
automatic profiling.

PRESUBMIT=passed
R=shenhan,bjanakiraman
DELTA=24 (21 added, 0 deleted, 3 changed)
OCL=57623-p2
RCL=57652-p2
RDATE=2012/01/04 17:19:20


P4 change: 42662792
14/utils/command_executer.py
48c0cdb05cb5b5884817d7d13ddf61daa3f5b4be 16-Feb-2013 asharif <asharif@p4.invalid> Added profile_cycler.py, a script to automate collection of FDO profiles
of a particular page cycler.

It does the following steps:
* Copies over the page cycler data to the target.
* Logs in using cros_login.py.
* Runs Chrome with the correct flags to start the page cycler.
* Uses gdb to call gcov_exit() on the renderer process.
* Copies the profile directory back to the host.

PRESUBMIT=passed
R=cmtice,bjanakiraman
DELTA=456 (456 added, 0 deleted, 0 changed)
OCL=57632-p2
RCL=57644-p2
RDATE=2012/01/04 12:41:27


P4 change: 42662788
14/fdo_scripts/gdb_remote.dump
14/fdo_scripts/profile_cycler.py
d8136e44d207a66fedf6682105aca8088881c715 16-Feb-2013 asharif <asharif@p4.invalid> Added cros_login.py, a script to get past the login screen of ChromeOS.

PRESUBMIT=passed
R=shenhan,bjanakiraman
DELTA=114 (114 added, 0 deleted, 0 changed)
OCL=57627-p2
RCL=57641-p2
RDATE=2012/01/04 12:24:36


P4 change: 42662785
14/cros_login.py
47f50fe4d926bd3f02ca4a8ffbf440a58c164683 16-Feb-2013 asharif <asharif@p4.invalid> Added missing file experiment_runner.py. This was missing in CL:
cl/57583-p2.

PRESUBMIT=passed
R=shenhan,bjanakiraman
DELTA=128 (128 added, 0 deleted, 0 changed)
OCL=57630-p2
RCL=57634-p2
RDATE=2012/01/04 10:13:50


P4 change: 42662781
14/crosperf/experiment_runner.py
a313d47405fbcb899384be9aa11b03e14eab2797 16-Feb-2013 asharif <asharif@p4.invalid> Added per-autotest log files instead of a single log file containing
every run's output.

Also fixed a bug in results_cache.py (read vs. write).

Tested: ./run_tests.sh works.

PRESUBMIT=passed
R=bjanakiraman,shenhan
DELTA=31 (13 added, 5 deleted, 13 changed)
RCL=57593-p2
RDATE=2011/12/28 17:10:11


P4 change: 42662724
14/crosperf/autotest_runner.py
14/crosperf/benchmark_run.py
14/crosperf/benchmark_run_unittest.py
14/crosperf/experiment.py
14/crosperf/perf_processor.py
14/crosperf/results_cache.py
ca3c6c12870daf256b0acdb1d4e4b4fe559da397 16-Feb-2013 asharif <asharif@p4.invalid> Changed utils.ExecuteCommandInChroot() to
command_executer.ChrootRunCommand().

With this move, a single command_executer can be instantiated with the
correct logger and all commands including chroot commands will be logged
to the same file.

This is a pre-requisite to making crosperf log individual autotests to
different files when running in parallel.

PRESUBMIT=passed
R=shenhan,bjanakiraman
DELTA=100 (26 added, 21 deleted, 53 changed)
OCL=57589-p2
RCL=57591-p2
RDATE=2011/12/28 15:13:37


P4 change: 42662714
14/build_benchmarks.py
14/build_chrome_browser.py
14/build_chromeos.py
14/build_tc.py
14/crosperf/autotest_runner.py
14/crosperf/perf_processor.py
14/run_tests.py
14/utils/command_executer.py
14/utils/utils.py
7899e00a26849cf08c6e85ba4eb6142249ae020b 16-Feb-2013 asharif <asharif@p4.invalid> Moved os.close() to the right location to prevent "Text file busy"
errors while executing chroot commands.

PRESUBMIT=passed
R=shenhan,bjanakiraman
DELTA=6 (1 added, 2 deleted, 3 changed)
OCL=57586-p2
RCL=57588-p2
RDATE=2011/12/28 13:44:09


P4 change: 42662710
14/utils/utils.py
a5689fd9eec3a56d3c4dabe07fce5aedc93307cb 16-Feb-2013 asharif <asharif@p4.invalid> Cloned from CL 57424 by 'g4 patch'.
Original change by raymes@raymes-crosstool-git5-11blah-git5 on 2011/12/14 11:43:10.

-Added the failure reason of jobs to the text report.
-Added some logging about cache hit/misses.
-Allowed checksums to be computed simultaneously
-Renamed action_runner to experiment_runner
-Moved storing results logic to experiment_runner
-Improvements to logging

Added the following fixes:
-Don't store run results when Ctrl-C is pressed in the middle of a run.
-Refactor caching code a bit to make it clearer.
-Fixed exception thrown when the cache was empty.
-Changed profile_type to be "" when it is invalid instead of "none".
-Fixed exception when using cache if no profile is present in the cache
file.
-Made ExecuteCommandInChroot() thread- and process-safe.

Added a DIFFBASE= for convenience.

PRESUBMIT=passed
R=raymes,bjanakiraman,shenhan
DELTA=412 (163 added, 183 deleted, 66 changed)
OCL=57454-p2
RCL=57583-p2
RDATE=2011/12/27 14:17:33
DIFFBASE=57424-p2


P4 change: 42662702
14/crosperf/action_runner.py
14/crosperf/autotest_runner.py
14/crosperf/benchmark_run.py
14/crosperf/column_chart.py
14/crosperf/crosperf.py
14/crosperf/experiment.py
14/crosperf/field.py
14/crosperf/image_checksummer.py
14/crosperf/machine_manager.py
14/crosperf/results_cache.py
14/crosperf/results_columns.py
14/crosperf/results_report.py
14/crosperf/settings_factory.py
14/utils/command_executer.py
14/utils/utils.py
7dd6d863959665931c4785aabf6564f03eb9c846 16-Feb-2013 asharif <asharif@p4.invalid> Added command to uninstall gcc before building & installing it.

Before this change the user had to unmerge an existing gcc-4.6.0 before
calling build_tc.py. This change incorporates that step in the script.

PRESUBMIT=passed
R=cmtice,shenhan,bjanakiraman
DELTA=5 (5 added, 0 deleted, 0 changed)
RCL=57542-p2
RDATE=2011/12/20 20:08:16


P4 change: 42662644
14/build_tc.py
f59a6928bf8bb620b14fd16f3b140411f84604fb 16-Feb-2013 raymes <raymes@p4.invalid> Fixed condition for loading failed results from the cache.

PRESUBMIT=passed
R=asharif
DELTA=1 (0 added, 0 deleted, 1 changed)
RCL=57397-p2
RDATE=2011/12/13 16:32:55


P4 change: 42662289
14/crosperf/results_cache.py
4c3527bf18b8fab05a1292a9c43e29679afa1601 16-Feb-2013 raymes <raymes@p4.invalid> Fixed bug where experiment start_time may be None.

PRESUBMIT=passed
R=asharif
DELTA=4 (3 added, 0 deleted, 1 changed)
OCL=57393-p2
RCL=57396-p2
RDATE=2011/12/13 16:28:34


P4 change: 42662288
14/crosperf/experiment_status.py
8c4ddba42f5157a00a31476478478e6cac4f2b02 16-Feb-2013 raymes <raymes@p4.invalid> Fix bug where None results were incorporated into numerical aggregators.

PRESUBMIT=passed
R=asharif
DELTA=31 (6 added, 11 deleted, 14 changed)
RCL=57392-p2
RDATE=2011/12/13 15:52:08


P4 change: 42662285
14/crosperf/results_columns.py
3cf328987cfaf2137621301ae6bba12b4bad6229 16-Feb-2013 raymes <raymes@p4.invalid> * Fix an issue where a machine can be released without being locked.
* Fix an issue where two labels can have the same name.

PRESUBMIT=passed
R=asharif
DELTA=35 (28 added, 4 deleted, 3 changed)
RCL=57391-p2
RDATE=2011/12/13 14:45:09


P4 change: 42662284
14/crosperf/benchmark_run.py
14/crosperf/experiment_factory_unittest.py
14/crosperf/experiment_file.py
14/crosperf/experiment_file_unittest.py
d1de3409e3e9302f7807690d62c5a8980e436b57 16-Feb-2013 raymes <raymes@p4.invalid> Added ability to run perf stat and have results displayed in results report.

PRESUBMIT=passed
R=asharif
DELTA=231 (143 added, 38 deleted, 50 changed)
OCL=57347-p2
RCL=57390-p2
RDATE=2011/12/13 14:24:07


P4 change: 42662268
14/crosperf/action_runner.py
14/crosperf/autotest_runner.py
14/crosperf/benchmark.py
14/crosperf/benchmark_run.py
14/crosperf/benchmark_run_unittest.py
14/crosperf/experiment.py
14/crosperf/experiment_factory.py
14/crosperf/experiment_factory_unittest.py
14/crosperf/field.py
14/crosperf/perf_processor.py
14/crosperf/results_cache.py
14/crosperf/results_sorter.py
14/crosperf/settings.py
14/crosperf/settings_factory.py
14/utils/file_utils.py
d366e6d8462b8461dd00b6fe832ef4db4841f354 16-Feb-2013 asharif <asharif@p4.invalid> Added os.path.expanduser() to benchmark_run.py so images could be
specified as: ~/foo/bar/chromiumos_image.bin.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=1 (0 added, 0 deleted, 1 changed)
OCL=57384-p2
RCL=57388-p2
RDATE=2011/12/13 13:56:39


P4 change: 42662264
14/crosperf/benchmark_run.py
b1c9850a67b201a1f3af7904d42111d500e51524 16-Feb-2013 raymes <raymes@p4.invalid> -Removed obsolete autotest_gatherer.
-Added back accidentally deleted autotest_runner.
-Allowed autotest results and perf.data to be cached.
-Output resulst into experiment directory.
-Removed useless actions interface.
-Updated email sending to include html results attachment.

PRESUBMIT=passed
R=asharif
OCL=57266-p2
RCL=57342-p2
RDATE=2011/12/12 12:59:27


P4 change: 42662215
14/crosperf/action_runner.py
14/crosperf/autotest_gatherer.py
14/crosperf/autotest_runner.py
14/crosperf/benchmark_run.py
14/crosperf/crosperf.py
14/crosperf/experiment.py
14/crosperf/experiment_factory.py
14/crosperf/experiment_file.py
14/crosperf/field.py
14/crosperf/perf_processor.py
14/crosperf/results_cache.py
14/crosperf/results_report.py
14/crosperf/settings.py
14/crosperf/table.py
14/utils/email_sender.py
14/utils/file_utils.py
78f367b9f9efc2f203b69fed82054160a8e66e4e 16-Feb-2013 raymes <raymes@p4.invalid> Added new reporting code.

PRESUBMIT=passed
R=bjanakiraman,asharif
OCL=57267-p2
RCL=57282-p2
RDATE=2011/12/08 17:29:35


P4 change: 42662052
14/crosperf/autotest_runner.py
14/crosperf/column_chart.py
14/crosperf/results_columns.py
14/crosperf/results_report.py
14/crosperf/results_sorter.py
14/crosperf/table.py
14/crosperf/table_formatter.py
714f7a01a4c210e3c165662e20720951057cfe9b 16-Feb-2013 jingyu <jingyu@p4.invalid> Change the android tree test from honeycomb to ics.
Honeycomb tree is quite old. Latest gcc-4.6 toolchain failed
because of deprecated prelinker and stripper used there.
We now can switch to ics tree.

Tested:
Build ics-release tree with latest release gcc-4.6 toolchain.

PRESUBMIT=passed
R=kbaclawski
CC=dougkwan,carrot,bhaskar-staff
APPROVED=kbaclawski
DELTA=3 (0 added, 1 deleted, 2 changed)
OCL=57220-p2
RCL=57243-p2
RDATE=2011/12/07 15:22:33


P4 change: 42661950
14/automation/clients/helper/android.py
af85edf8c13916b82a757b5ff2aa071682c34ec7 15-Feb-2013 asharif <asharif@p4.invalid> Prepended crosperf.py invocation with the correct dir.

Tested: Called crosperf when the cwd was something other than crosperf.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=1 (0 added, 0 deleted, 1 changed)
RCL=57201-p2
RDATE=2011/12/05 16:48:30


P4 change: 42660801
14/crosperf/crosperf
f371de5afb2596af92938cea859f597f05107fa7 15-Feb-2013 raymes <raymes@p4.invalid> Add support for a different board per label.

PRESUBMIT=passed
R=bjanakiraman,asharif
OCL=57156-p2
RCL=57158-p2
RDATE=2011/12/02 11:50:20


P4 change: 42660644
14/crosperf/action_runner.py
14/crosperf/benchmark_run_unittest.py
14/crosperf/experiment.py
14/crosperf/experiment_factory.py
14/crosperf/experiment_factory_unittest.py
14/crosperf/experiment_file_unittest.py
14/crosperf/label.py
14/crosperf/settings_factory.py
1b1ab1439f467348313929e1d905e2b58c61f6a0 15-Feb-2013 raymes <raymes@p4.invalid> Modifications to allow perf counters to be specified and collected.

PRESUBMIT=passed
R=bjanakiraman,asharif
OCL=57130-p2
RCL=57138-p2
RDATE=2011/12/01 15:38:29


P4 change: 42660612
14/crosperf/autotest_runner.py
14/crosperf/benchmark.py
14/crosperf/benchmark_run.py
14/crosperf/crosperf
14/crosperf/crosperf.py
14/crosperf/experiment.py
14/crosperf/experiment_factory.py
14/crosperf/field.py
14/crosperf/help.py
14/crosperf/perf_processor.py
14/crosperf/results_cache.py
14/crosperf/settings.py
14/crosperf/settings_factory.py
3b9d1279d71fe0380aebc12ba3fdb8ebb974297a 15-Feb-2013 raymes <raymes@p4.invalid> Added mechanism for allowing global fields to be overridden by options passed in as arguments.

Added first cut at help page.

PRESUBMIT=passed
R=asharif
OCL=57108-p2
RCL=57129-p2
RDATE=2011/12/01 14:14:25


P4 change: 42660597
14/crosperf/crosperf.py
14/crosperf/experiment_factory.py
14/crosperf/experiment_factory_unittest.py
14/crosperf/field.py
14/crosperf/settings.py
14/crosperf/settings_factory.py
ec4b5b3d85996096831257365ae7eab28de097cf 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Automation Framework bugfix: FolderDependency is considered to be read only (and thus a link will be created instead of full copy), only when source and destination directories are the same (i.e. when dest parameter is not provided).

Tested:
Analysed all call sites for correctness.

PRESUBMIT=passed
R=asharif,cmtice
CC=bhaskar-staff
DELTA=4 (3 added, 0 deleted, 1 changed)
RCL=57124-p2
RDATE=2011/12/01 12:08:22


P4 change: 42660593
14/automation/common/job.py
77bd80d4ed4829888aa7db6e8c794cb2ed161271 15-Feb-2013 asharif <asharif@p4.invalid> Generalized GetCtargetFromBoard. It now uses ChromeOS functions to
correctly get the ctarget from the board for all boards.

Tested: Tested GetCtargetFromBoard for stumpy, lumpy, tegra2, etc.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=13 (3 added, 0 deleted, 10 changed)
OCL=57073-p2
RCL=57123-p2
RDATE=2011/12/01 11:00:51


P4 change: 42660592
14/build_tc.py
14/utils/utils.py
fdfd0b5e3bfcf2d403c69bac84b76280689057a6 15-Feb-2013 raymes <raymes@p4.invalid> Made caching features work (rerun, rerun_if_failed, exact_remote).

PRESUBMIT=passed
R=asharif
DELTA=189 (98 added, 46 deleted, 45 changed)
OCL=57076-p2
RCL=57107-p2
RDATE=2011/11/30 16:41:01


P4 change: 42660558
14/crosperf/action_runner.py
14/crosperf/benchmark_run.py
14/crosperf/benchmark_run_unittest.py
14/crosperf/experiment.py
14/crosperf/experiment_factory.py
14/crosperf/experiment_file.py
14/crosperf/image_checksummer.py
14/crosperf/label.py
14/crosperf/machine_manager.py
14/crosperf/perf_processor.py
14/crosperf/results_cache.py
14/crosperf/settings_factory.py
569534a309b8e87daea97cd11b8df8ad9e37e2ce 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Crosstool Nightly Build Client: Add binutils dejagnu test run.

Tested:
Locally in dry run mode.

PRESUBMIT=passed
R=dougkwan
CC=bhaskar-staff
APPROVED=dougkwan
DELTA=13 (2 added, 0 deleted, 11 changed)
OCL=57046-p2
RCL=57101-p2
RDATE=2011/11/30 14:08:20


P4 change: 42660553
14/automation/clients/crosstool.py
14/automation/clients/helper/crosstool.py
da8e93bb099196c18bd51ac380cf0312b6754c66 15-Feb-2013 asharif <asharif@p4.invalid> %s/os.getlogin/getpass.getuser/g.

Replaced all os.getlogin() function calls to getpass.getuser() function
calls. os.getlogin() doesn't work when the script is called from a cron
job.

TESTED=Ran lock_machine.py ahmad.mtv
PRESUBMIT=passed
BUG=5690035
R=bjanakiraman,kbaclawski,raymes
CC=c-compiler-chrome
DELTA=6 (2 added, 0 deleted, 4 changed)
OCL=57094-p2
RCL=57100-p2
RDATE=2011/11/30 12:49:15


P4 change: 42660552
14/lock_machine.py
14/utils/email_sender.py
ee5bb8624d9888e6d31ea61529ffb7adfa84330e 15-Feb-2013 raymes <raymes@p4.invalid> Changes:
* Added ability to ctrl-c and have state cleaned up cleanly
* Added cleanup function which is run on exits and ensures all machines are unlocked.
* Added global image_checksummer to cache checksums.
* Several miscellaneous bug fixes to get things working smoothly.

PRESUBMIT=passed
R=bjanakiraman,asharif
OCL=56972-p2
RCL=57072-p2
RDATE=2011/11/29 10:29:06


P4 change: 42660510
14/crosperf/action_runner.py
14/crosperf/benchmark_run.py
14/crosperf/benchmark_run_unittest.py
14/crosperf/crosperf.py
14/crosperf/crosperf_test.py
14/crosperf/experiment.py
14/crosperf/experiment_factory.py
14/crosperf/experiment_status.py
14/crosperf/field.py
14/crosperf/image_checksummer.py
14/crosperf/label.py
14/crosperf/machine_manager.py
14/crosperf/results_cache.py
14/crosperf/settings.py
14/crosperf/settings_factory.py
14/utils/file_utils.py
14/utils/logger.py
14/utils/utils.py
f5af32f301d2452f2db27bf591435a6aefc7bea3 15-Feb-2013 asharif <asharif@p4.invalid> Change the way image_chromeos.py determines whether an image is a test
image. It now looks at /etc/lsb-release on the image and checks for the
string "Test Build" within it.

Tested: Ran image_chromeos.py on a test and non-test image and it
correctly distinguished the two.

PRESUBMIT=passed
R=raymes,bjanakiraman,cmtice,shenhan
DELTA=2 (0 added, 0 deleted, 2 changed)
RCL=56969-p2
RDATE=2011/11/22 13:50:22


P4 change: 42660323
14/image_chromeos.py
5e982e376e8436fe133f37b34601b075c3d166c2 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> DejaGNU Log Tool: Fix missing attributes in DejaGnuTestRun object.
It can happen that these attributes are missing in a file, so the
parser won't pick them up.

Tested:
N/A

PRESUBMIT=passed
R=asharif,yunlian
CC=bhaskar-staff
DELTA=6 (2 added, 0 deleted, 4 changed)
OCL=56940-p2
RCL=56943-p2
RDATE=2011/11/21 11:20:43


P4 change: 42660281
14/automation/clients/report/dejagnu/summary.py
9aa416dc703dde5db86ebd0eb0733cf947400af3 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Automation Framework: LoggingCommandExecuter CloseLog method fixes.

1) Before the handler is closed, we need to flush pending log records, and
remove it from logger.
2) Just in case someone tries to use the logger after the handler is removed,
we want to redirect log records to main logger.

Tested:
On chrome-dev1.hot.

PRESUBMIT=passed
R=cmtice,shenhan,jingyu
CC=bhaskar-staff
DELTA=15 (10 added, 3 deleted, 2 changed)
OCL=56884-p2
RCL=56941-p2
RDATE=2011/11/21 10:24:28


P4 change: 42660279
14/automation/common/command_executer.py
23b34d4f9d73d011cbf58a114dbeca07556c8901 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Automation Framework: Crosstool nightly build client fixes.

1) Convert to new logger interface.
2) Display useful message if build target is missing at command line.
3) Modify last step to pick up xfail manifest files correctly.
4) Change tasks to handle new DejaGnu log tool correctly.
5) Allow to build extra crosstool targets.
6) Produced packages are stored in directory,
which name is based on the CL number instead of a timestamp.
7) Directories in x20 are created with 0755 permissions.
8) Keep 10 last builds instead of 20.

Tested:
Run job on chrome-dev1.hot.

PRESUBMIT=passed
R=dougkwan
CC=cgd,bhaskar-staff
DELTA=73 (24 added, 14 deleted, 35 changed)
OCL=56792-p2
RCL=56904-p2
RDATE=2011/11/18 15:28:57


P4 change: 42660216
14/automation/clients/crosstool.py
14/automation/clients/helper/crosstool.py
1f871341762249145b8bdccd1ae99cea54d053a8 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Automation Framework: Extend Wrapper class to handle umask change in wrapped command execution context.

Tested:
Locally with crosstool nightly build client.

PRESUBMIT=passed
R=jingyu,yunlian,shenhan
CC=bhaskar-staff
DELTA=17 (15 added, 0 deleted, 2 changed)
RCL=56890-p2
RDATE=2011/11/18 13:07:18


P4 change: 42660196
14/automation/common/command.py
dc1c20f2aad9f6fed6fea8d52586f17648404060 15-Feb-2013 raymes <raymes@p4.invalid> Implemented --dry_run mode for parsing experiment files without running.

Moved all actions into a separate class to facilitate this. Also added a
crosperf.py system test which runs the driver in dry_run mode.

PRESUBMIT=passed
R=asharif,bjanakiraman
DELTA=201 (140 added, 47 deleted, 14 changed)
OCL=56858-p2
RCL=56873-p2
RDATE=2011/11/17 17:17:27


P4 change: 42660165
14/crosperf/action_runner.py
14/crosperf/benchmark_run.py
14/crosperf/crosperf.py
14/crosperf/crosperf_test.py
14/crosperf/experiment_factory_unittest.py
14/crosperf/label.py
14/crosperf/run_tests.sh
c97199a1f292bcdc151e54b0806ee12fa8f6d9b7 15-Feb-2013 asharif <asharif@p4.invalid> Completely re-wrote build_tc.py. It now takes in a gcc_root and uses
that to build the toolchain compiler.

Also added some utility functions to utils.

Tested: Tested it for x86 and arm using a gcc svn checkout.

PRESUBMIT=passed
BUG=5417435
R=cmtice,shenhan,bjanakiraman,kbaclawski,yunlian,raymes
DELTA=386 (157 added, 192 deleted, 37 changed)
OCL=56823-p2
RCL=56870-p2
RDATE=2011/11/17 16:40:22


P4 change: 42660164
14/build_tc.py
14/tc_enter_chroot.py
14/utils/utils.py
913b5118274e74122c1a78eed0ea6700fa481d95 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Automation Framework: Correct log displaying on dashboard.

After logger changes, all log record are saved to a single compressed file. The log record format was changed to contain datetime, output stream name, and message itself. Thus the dashboard had to be modified to reflect logger changes.

Tested:
Locally.

PRESUBMIT=passed
R=asharif,cmtice,yunlian
CC=bhaskar-staff
DELTA=50 (19 added, 5 deleted, 26 changed)
OCL=56819-p2
RCL=56866-p2
RDATE=2011/11/17 15:56:02


P4 change: 42660160
14/automation/server/monitor/dashboard.py
14/automation/server/monitor/static/style.css
14/automation/server/monitor/templates/job_log.html
14/automation/server/monitor/urls.py
2e97d48f6a6221c1dcfa55f29b1070f163e9092e 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Automation Framework: Allow directory to be created with specific permissions, instead of that imposed by umask.

Tested:
Locally with crosstool nighly build job.

PRESUBMIT=passed
R=yunlian,jingyu,shenhan
CC=bhaskar-staff
DELTA=11 (9 added, 0 deleted, 2 changed)
OCL=56855-p2
RCL=56862-p2
RDATE=2011/11/17 13:09:16


P4 change: 42660156
14/automation/common/command.py
773cd6a29b7af4eca59e81b68fbebff858d2361b 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Automation Framework: Logger improvements.

1) Properly handle logging exceptions.
2) Cleaned up CustomLogger.format method.
3) Added CustomLogger constructor parameter, that allows to omit certain
elements of log record.

Tested:
Locally.

PRESUBMIT=passed
R=shenhan,asharif,raymes
CC=bhaskar-staff
DELTA=48 (26 added, 6 deleted, 16 changed)
OCL=56793-p2
RCL=56836-p2
RDATE=2011/11/16 13:28:22


P4 change: 42659639
14/automation/common/logger.py
71a21e32dcbe452fded5f5471f2cdc57cbc8e409 15-Feb-2013 raymes <raymes@p4.invalid> Port the remainder of crb functionality to crossperf.

This ports the remainder of crb functionality, which includes running tests and
printing results to crosperf.

The main changes are to remove the inheritance used in table_formatter and to move all remaining
relevant functionality into experiment.py and crosperf.py.

PRESUBMIT=passed
R=bjanakiraman,asharif
DELTA=556 (528 added, 14 deleted, 14 changed)
OCL=56802-p2
RCL=56825-p2
RDATE=2011/11/16 10:57:52


P4 change: 42659618
14/crosperf/autotest_gatherer.py
14/crosperf/benchmark_run.py
14/crosperf/crosperf.py
14/crosperf/experiment.py
14/crosperf/experiment_status.py
14/crosperf/perf_processor.py
14/crosperf/run_tests.sh
14/crosperf/table_formatter.py
14/utils/email_sender.py
9d671b919b2be79429a6344dc0ae408006583d28 15-Feb-2013 asharif <asharif@p4.invalid> Added utils function ExecuteCommandInChroot(), which executes a command in
a ChromeOS chroot without mounting anything.

A long-standing TODO was to move this function to utils, which is done
in this CL.

The toolchain_root parameter was only used in build_tc.py, which is
rendered broken by this CL. I am working on a new build_tc.py, which
will replace the current one in a subsequent CL.

Tested: Ran new_build_tc.py.

PRESUBMIT=passed
R=raymes,bjanakiraman,kbaclawski
DELTA=46 (15 added, 20 deleted, 11 changed)
OCL=56804-p2
RCL=56820-p2
RDATE=2011/11/16 10:47:05


P4 change: 42659617
14/build_benchmarks.py
14/build_chrome_browser.py
14/build_chromeos.py
14/run_tests.py
14/utils/utils.py
329a8fd4183f21bf58e56052205c50e29e12c407 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Automation Framework: Close command output log after command finishes execution. This prevents a log file from being opened forever (and gzip flushing out the last record).

Tested:
Locally.

PRESUBMIT=passed
R=jingyu,yunlian,cmtice
CC=bhaskar-staff
DELTA=52 (33 added, 16 deleted, 3 changed)
OCL=56784-p2
RCL=56816-p2
RDATE=2011/11/16 09:09:05


P4 change: 42659616
14/automation/common/command_executer.py
14/automation/server/job_executer.py
f6e91951ab4566325e04be23a238cfcb6fe6b700 15-Feb-2013 asharif <asharif@p4.invalid> Added messages to log the changing of the working directory.

Tested: Ran repo_to_repo.py -n.

PRESUBMIT=passed
R=jingyu,raymes,kbaclawski,bjanakiraman
DELTA=6 (6 added, 0 deleted, 0 changed)
OCL=56692-p2
RCL=56695-p2
RDATE=2011/11/11 11:41:21


P4 change: 42659249
14/utils/utils.py
2e2895f71619874802e954fe1e4954cee5306757 15-Feb-2013 raymes <raymes@p4.invalid> Refactored benchmark_run.py to make it testable.

Factored out code for caching results, running autotests and storing
perf. Added mock classes for these. This makes benchmark_run testable
now. Added simple unittest for this, can be built upon.

PRESUBMIT=passed
R=asharif,bjanakiraman
DELTA=431 (271 added, 118 deleted, 42 changed)
OCL=56541-p2
RCL=56694-p2
RDATE=2011/11/11 11:26:50


P4 change: 42659248
14/crosperf/autotest_runner.py
14/crosperf/benchmark_run.py
14/crosperf/benchmark_run_unittest.py
14/crosperf/experiment.py
14/crosperf/experiment_factory.py
14/crosperf/machine_manager.py
14/crosperf/perf_processor.py
14/crosperf/results_cache.py
14/crosperf/run_tests.sh
d58d847bdf211f6b9039f0482ed4b478e3d9209c 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Automation Framework: Migration to Python built-in logging module.

This change is supposed to:
- limit the size of logs on disk,
- make server traces more useful (thanks to information about thread
and component, where a log record was generated),
- increase debuggability thanks to more useful messages,
- increase logging mechanism flexibility.

Actual changes:
1) All messages originating from Automation Framework are dumped to
both: console and a file. Log records dumped to the terminal are
colourised, but colours are stripped from the log file. The file
undergoes rotation with maximum file size of 10MB and 9 backups.
This means that summarized log size on disk won't exceed 100MB.
Log record format:
YYYY-MM-DD HH:MM:SS.MS LOGLEVEL [Thread:LoggerName] Message...
2) Command output is logged only to gzipped file in Job's directory.
Log record format:
YYYY-MM-DD HH:MM:SS STDOUT/STDERR: Actual output.
3) Improved message quality.

Tested:
Locally.

PRESUBMIT=passed
R=asharif,raymes,cmtice
CC=bhaskar-staff
DELTA=334 (156 added, 129 deleted, 49 changed)
OCL=55795-p2
RCL=56693-p2
RDATE=2011/11/11 11:08:57


P4 change: 42659247
14/automation/common/command_executer.py
14/automation/common/logger.py
14/automation/server/job_executer.py
14/automation/server/job_group_manager.py
14/automation/server/job_manager.py
14/automation/server/server.py
f6ffe503e51cc09607d5ca17f1bb5725e61d06e7 15-Feb-2013 asharif <asharif@p4.invalid> Added default mappings for GitRepo in case none are specified.

Also cleaned the target dir before rsync'ing when mappings are present.

Tested: Tested with an rtr file given on the bug.

PRESUBMIT=passed
BUG=5599872
R=jingyu,bjanakiraman
DELTA=14 (9 added, 0 deleted, 5 changed)
OCL=56641-p2
RCL=56689-p2
RDATE=2011/11/11 10:58:48


P4 change: 42658421
14/repo_to_repo.py
7f56db9d70f951297756e639c7c627c1465ba2a3 15-Feb-2013 asharif <asharif@p4.invalid> Changed Md5File() to use FileUtils instead of utils.

cl/56321-p2 changed the location of Md5File and this CL updates one of
the users of that function.

Tested: Ran crb.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=2 (0 added, 0 deleted, 2 changed)
RCL=56611-p2
RDATE=2011/11/09 14:05:37


P4 change: 42658281
14/image_chromeos.py
f334422e931ae9205b55cb7920178f39f8de489f 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> DejaGNU Log Tool: Make dependency on Django optional. If Django framework is not installed 'report' command will fail in a graceful manner.

Tested:
With and without django framework installed:
$ dejagnu.sh report -o report.html -m g++-unix.xfail g++.sum

PRESUBMIT=passed
R=shenhan,cmtice
CC=dnovillo,bhaskar-staff
DELTA=36 (17 added, 16 deleted, 3 changed)
OCL=56493-p2
RCL=56507-p2
RDATE=2011/11/04 14:30:26


P4 change: 42657819
14/automation/clients/report/dejagnu.sh
14/automation/clients/report/dejagnu/main.py
14/automation/clients/report/dejagnu/report.py
14/automation/clients/report/dejagnu/settings.py
4bab89cd2b7aed24021c05f89a3703cd2018c858 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> DejaGnu Log Tool: remove database support.

During the meeting with Diego we came to conclusion that actually
there're no use cases, that would justify the idea of storing test
results in database. Moreover DB support require somewhat heavyweight
dependencies on Django and python-sqlite.

Next step will be to merge validate_failures.py script with DG Log Tool.

Tested:
dejagnu.sh manifest g++.sum
dejagnu.sh report -o report.html g++.sum
dejagnu.sh report -m g++-unix.xfail -o report.html g++.sum

PRESUBMIT=passed
R=asharif,cmtice
CC=dnovillo,bhaskar-staff
DELTA=376 (38 added, 283 deleted, 55 changed)
OCL=56293-p2
RCL=56492-p2
RDATE=2011/11/04 10:36:56


P4 change: 42657794
14/automation/clients/report/dejagnu.sh
14/automation/clients/report/dejagnu/main.py
14/automation/clients/report/dejagnu/manifest.py
14/automation/clients/report/dejagnu/models.py
14/automation/clients/report/dejagnu/report.py
14/automation/clients/report/dejagnu/settings.py
14/automation/clients/report/dejagnu/summary.py
09d55df539007143b8a5d13265fba3d102a8bfcc 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Add __repr__ to Job/JobGroup/Machine classes to represent them in form of unique
string id. This is a prerequisite for further logger-related changes.

Tested:
Locally changed log messages to use '%r' instead of '%s'.

PRESUBMIT=passed
R=shenhan,yunlian
CC=bhaskar-staff
DELTA=10 (10 added, 0 deleted, 0 changed)
OCL=56455-p2
RCL=56470-p2
RDATE=2011/11/03 16:47:07


P4 change: 42657663
14/automation/common/job.py
14/automation/common/job_group.py
14/automation/common/machine.py
78053bdb6527fe36e78c10c5a32e35d9e7296109 15-Feb-2013 raymes <raymes@p4.invalid> Refactored benchmark_run to make it more understandable/readable and modular.

Added machine_manager.py from crb. Tests are needed for these
but currently they would need more refactoring to test properly.

PRESUBMIT=passed
R=bjanakiraman,asharif
DELTA=404 (227 added, 96 deleted, 81 changed)
OCL=56294-p2
RCL=56450-p2
RDATE=2011/11/03 11:28:34


P4 change: 42657636
14/crosperf/benchmark.py
14/crosperf/benchmark_run.py
14/crosperf/experiment.py
14/crosperf/experiment_factory.py
14/crosperf/field.py
14/crosperf/machine_manager.py
14/crosperf/run_tests.sh
14/crosperf/settings_factory.py
8fbae5774490ff8652ba3659d877a3f532b424dc 15-Feb-2013 jingyu <jingyu@p4.invalid> Make new dir inside RemoteCopyFrom(). Scanned all files under
automation/ and changed 2 instances.

Tested:
gpylint

PRESUBMIT=passed
R=kbaclawski
CC=bhaskar-staff
APPROVED=kbaclawski
DELTA=11 (2 added, 2 deleted, 7 changed)
OCL=56359-p2
RCL=56422-p2
RDATE=2011/11/02 14:35:55


P4 change: 42657566
14/automation/clients/helper/android.py
14/automation/common/command.py
3a600ba17a857c31d7fc1b9d835244d944c92acb 15-Feb-2013 jingyu <jingyu@p4.invalid> Add Android benchmarking job to Android nightly client. The job
builds Android benchmark and sends the binary to perflab queue
for running. It compares benchmark sizes to referenced ones,
generates size comparison html page to results/. The perflab jobid
is also put into results/.
Todo:
1. put size html page to a right place which can be accessed
from monitor.
2. send perflab jobid to the machine that generates Android
perflab dashboard, which is currently perflab.mtv.

Tested:
./android.py on atree

PRESUBMIT=passed
R=kbaclawski
CC=bhaskar-staff
APPROVED=kbaclawski
DELTA=122 (99 added, 0 deleted, 23 changed)
OCL=56259-p2
RCL=56357-p2
RDATE=2011/10/31 11:32:27


P4 change: 42657346
14/automation/clients/android.py
14/automation/clients/helper/android.py
14/automation/common/command.py
0b038aea88a9f365c4deb3f23d05aa19376ed96f 15-Feb-2013 raymes <raymes@p4.invalid> =Added classes representing:
-benchmark
-benchmark runs (copied directly from cros_run_benchmarks autotest_run)
-experiments
-labels

Also added ExperimentFactory for building experiments from an experiment file which has been loaded.

PRESUBMIT=passed
R=bjanakiraman,asharif
DELTA=593 (584 added, 0 deleted, 9 changed)
OCL=56267-p2
RCL=56322-p2
RDATE=2011/10/28 14:20:10
DIFFBASE=56266


P4 change: 42657262
14/crosperf/benchmark.py
14/crosperf/benchmark_run.py
14/crosperf/crosperf.py
14/crosperf/experiment.py
14/crosperf/experiment_factory.py
14/crosperf/experiment_factory_unittest.py
14/crosperf/experiment_file_unittest.py
14/crosperf/label.py
14/crosperf/settings_factory.py
6623db6b9c56619b1f84554558228dd357243fed 15-Feb-2013 raymes <raymes@p4.invalid> Removed Md5Sum from utils and added it to file_utils.py so it can be mocked out.

Make sure you use the diffbase.

PRESUBMIT=passed
R=bjanakiraman,asharif
DELTA=64 (46 added, 15 deleted, 3 changed)
OCL=56266-p2
RCL=56321-p2
RDATE=2011/10/28 14:07:51
DIFFBASE=56245


P4 change: 42657247
14/crb/crb_driver.py
14/image_chromeos.py
14/utils/file_utils.py
14/utils/utils.py
9d49d29c4407c7c31fcee0afabe2c1601efc5df2 15-Feb-2013 raymes <raymes@p4.invalid> Added semantics checking for experiment files.

PRESUBMIT=passed
R=bjanakiraman,asharif
CC=asharif,bjanakiraman
OCL=56245-p2
RCL=56320-p2
RDATE=2011/10/28 13:55:15


P4 change: 42657246
14/crosperf/experiment_file.py
14/crosperf/experiment_file_unittest.py
14/crosperf/field.py
14/crosperf/settings.py
14/crosperf/settings_factory.py
53a02f69e61588c5b474fb72f195482aaae564c8 15-Feb-2013 raymes <raymes@p4.invalid> Initial code for loading an experiment file into an internal representation (basically a dictionary).

Tested:
./experiment_file_unittest.py

PRESUBMIT=passed
R=asharif,bjanakiraman
DELTA=304 (304 added, 0 deleted, 0 changed)
OCL=56230-p2
RCL=56250-p2
RDATE=2011/10/25 09:38:12


P4 change: 42657049
14/crosperf/experiment_file.py
14/crosperf/experiment_file_unittest.py
14/crosperf/settings.py
b3bfa245d89329ecda9422453e2b29bfa9da7359 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Fix a bug related to variable substitution in command string.

Tested:
Locally by querying server in dry run mode with android nightly client.

PRESUBMIT=passed
R=jingyu
CC=bhaskar-staff
DELTA=2 (0 added, 1 deleted, 1 changed)
OCL=56211-p2
RCL=56213-p2
RDATE=2011/10/20 16:38:25


P4 change: 42656884
14/automation/common/job.py
0a5843bd4659be095c54ee816ea2a379f6c6f2b4 15-Feb-2013 asharif <asharif@p4.invalid> The output format of autotest was changed. This CL changes the parser of
crb to take care of that.

Specifically, earlier the autotest runs printed lines like:

PageCyclerTests.Bloatfile 211.11
PageCyclerTests PASS

Now the autotest prints lines like:
PageCyclerTests BloatFile 211.11
PageCyclerTests [ PASSED ]

TESTED=ran PageCycler.
PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=11 (4 added, 1 deleted, 6 changed)
OCL=56143-p2
RCL=56175-p2
RDATE=2011/10/19 12:30:54


P4 change: 42656440
14/crb/autotest_run.py
14/crb/table_formatter.py
8078084cee3b6125f24fc750ebe8c86b35646cc2 15-Feb-2013 raymes <raymes@p4.invalid> Fix bug in rep_to_repo where input directory is not being cleaned up.

PRESUBMIT=passed
R=asharif,cmtice
DELTA=2 (2 added, 0 deleted, 0 changed)
RCL=56174-p2
RDATE=2011/10/19 11:54:01


P4 change: 42656439
14/repo_to_repo.py
265cda3e39dff720cf652a254a1ffc287f28c91f 15-Feb-2013 asharif <asharif@p4.invalid> Split up the new cros_run_benchmarks.py into multiple files.

I'd like to see us use the version I have in my home directory since I
added a bunch of features to it.

Test: python crb_driver.py --remote=chromeos-zga1 /home/asharif/a/4.6_release_experiment2/chromeos.4.6.nomovbe/src/build/images/x86-agz/latest/chromiumos_image.bin --tests=AesThroughput --board=x86-agz

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=1928 (1125 added, 803 deleted, 0 changed)
OCL=55788-p2
RCL=56108-p2
RDATE=2011/10/17 17:22:32


P4 change: 42656229
14/crb/autotest_gatherer.py
14/crb/autotest_run.py
14/crb/crb_driver.py
14/crb/machine_manager_singleton.py
14/crb/table_formatter.py
14/cros_run_benchmarks.py
cf05281051b2949b719f7fc5590eaaeb8dce46e4 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Job interface clean-ups. Remove baseline attribute, give less confusing names to following attributes: children => (job's) predecessors, parents => (job's) successors, machine => primary_machine.

Tested:
Locally.

PRESUBMIT=passed
R=cmtice,shenhan,yunlian
CC=bhaskar-staff
DELTA=55 (8 added, 12 deleted, 35 changed)
OCL=55792-p2
RCL=56028-p2
RDATE=2011/10/12 14:46:31


P4 change: 42656037
14/automation/common/job.py
14/automation/server/job_executer.py
14/automation/server/job_manager.py
14/automation/server/monitor/dashboard.py
6c61913c422026a52e444289237b80b3b5e88e8a 15-Feb-2013 asharif <asharif@p4.invalid> tc_enter_chroot.py now checks for already-mounted directories before
mounting them. This prevents multiple mounting.

There still exists a race-condition between two copies of this script,
but that is a rare condition.

Also added an Unmount function for unmounting mount points.

Tested: Tested with new_build_tc.py.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=16 (7 added, 0 deleted, 9 changed)
OCL=55937-p2
RCL=56017-p2
RDATE=2011/10/11 16:03:30


P4 change: 42656024
14/tc_enter_chroot.py
fb5180a524a2ab767cd9468512bb2cc81e263afc 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Crosstool nightly client: provide list of manifest files (i.e. list of suppressed test failures) to DejaGnu report tool.

Tested:
Locally in dry run mode.

PRESUBMIT=passed
R=bjanakiraman,yunlian
CC=bhaskar-staff
DELTA=9 (5 added, 0 deleted, 4 changed)
OCL=55988-p2
RCL=56014-p2
RDATE=2011/10/11 15:26:38


P4 change: 42656022
14/automation/clients/crosstool.py
14/automation/clients/helper/crosstool.py
aa68f3c921b417fa9c3fcf101b4218ebb9ed8ca0 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> DejaGnu report tool:
- accept glob expression as a path to manifest files,
- improve command line options & arguments handling.

Tested:
Locally.

PRESUBMIT=passed
R=cmtice,shenhan
CC=bhaskar-staff
DELTA=39 (23 added, 4 deleted, 12 changed)
OCL=55982-p2
RCL=55985-p2
RDATE=2011/10/07 14:17:43


P4 change: 42655750
14/automation/clients/report/dejagnu/main.py
24bbeb2904b3088b315b5ebf9287a8602dfe7a98 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> DejaGNU reporting tool: change auxiliary data marker from '!' to '#@' for
compatibility with other tools using xfail file format.

Tested:
N/A

PRESUBMIT=passed
R=cmtice,shenhan
CC=bhaskar-staff
DELTA=31 (9 added, 9 deleted, 13 changed)
OCL=55981-p2
RCL=55984-p2
RDATE=2011/10/07 14:17:39


P4 change: 42655749
14/automation/clients/report/dejagnu/manifest.py
691fc953f5c8144411469d017363f916fd80b1fe 15-Feb-2013 raymes <raymes@p4.invalid> Added r2r file for gcc-4.6.0 ToT.

PRESUBMIT=passed
R=asharif
DELTA=12 (12 added, 0 deleted, 0 changed)
OCL=55917-p2
RCL=55964-p2
RDATE=2011/10/06 15:01:06


P4 change: 42655738
14/repo_to_repo_files/gcc-4.6.0_master.rtr
3c2559e6f9c7af072866bf37c31286591bdcf169 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Crosstool nighly client: update the list of valid crosstool targets.

Tested:
Locally.

PRESUBMIT=passed
R=cmtice,yunlian
CC=bhaskar-staff
DELTA=2 (0 added, 1 deleted, 1 changed)
RCL=55962-p2
RDATE=2011/10/06 13:50:43


P4 change: 42655737
14/automation/clients/crosstool.py
8238268da02d8f9181bec974160339494b416235 15-Feb-2013 jingyu <jingyu@p4.invalid> 1. Change Android nightly build from binutils-2.20 to binutils-2.21.
2. Set gcc-4.6 as default compiler.
3. Enable dual linker support.

Tested:
run android nightly client on atree.

PRESUBMIT=passed
R=kbaclawski
CC=bhaskar-staff
APPROVED=kbaclawski
DELTA=38 (11 added, 9 deleted, 18 changed)
OCL=55916-p2
RCL=55959-p2
RDATE=2011/10/06 11:29:52


P4 change: 42655622
14/automation/clients/android.py
14/automation/clients/helper/android.py
8adca4f43f0586b21758b84fa0e7390a6af28b91 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Minor fixes in nightly crosstool client:
- reflect changes in DejaGnu reporting tool interface,
- remove gcc 4.4.3 support,
- all toolchains have to use external sysroot.

Tested:
N/A

PRESUBMIT=passed
R=yunlian
CC=bhaskar-staff
DELTA=5 (0 added, 3 deleted, 2 changed)
OCL=55941-p2
RCL=55957-p2
RDATE=2011/10/06 10:24:09


P4 change: 42655621
14/automation/clients/helper/crosstool.py
ee09bee7b492f8ed2e0841c68d3f3b4215fb1d5f 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Small API convenience improvement for perforce.PathMapping constructor.

Tested:
Manually.

PRESUBMIT=passed
R=jingyu
CC=bhaskar-staff
DELTA=8 (5 added, 0 deleted, 3 changed)
OCL=55940-p2
RCL=55945-p2
RDATE=2011/10/05 17:53:24


P4 change: 42655605
14/automation/clients/helper/perforce.py
0c80b82a80f88c63e6297bd3740cb33261db634f 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> DejaGNU reporting tool: replace test failures suppression based on rewriting
rules with manifest files base solution. Several structural improvements. More
comprehensible documentation.

Tested:
Locally.

PRESUBMIT=passed
R=jingyu,shenhan,raymes
CC=bhaskar-staff,dnovillo
DELTA=597 (364 added, 137 deleted, 96 changed)
OCL=55683-p2
RCL=55789-p2
RDATE=2011/09/28 12:33:35


P4 change: 42655132
14/automation/clients/report/dejagnu/main.py
14/automation/clients/report/dejagnu/manifest.py
14/automation/clients/report/dejagnu/models.py
14/automation/clients/report/dejagnu/report.py
14/automation/clients/report/dejagnu/summary.py
6cb8848fc5995b424643f393cd7e9386ebd86eff 15-Feb-2013 asharif <asharif@p4.invalid> Used a working cros_run_benchmarks.py script (in mobiletc-prebuild's
home drive).

Fixed the --tests argument format.

Tested: locally tested.

PRESUBMIT=passed
R=raymes,bjanakiraman
CC=bhaskar-staff
DELTA=2 (0 added, 0 deleted, 2 changed)
RCL=55787-p2
RDATE=2011/09/28 11:44:32


P4 change: 42655131
14/automation/clients/helper/chromeos.py
bf6899dde56818d522258a89fa2fc2f03f4f0338 15-Feb-2013 asharif <asharif@p4.invalid> Added --public option to setup_chromeos.py that uses the public checkout
instead of the private internal one.

Changed the nightly ChromeOS job to use this --public option.

This is a workaround to the current failures due to authentication
problems.

Tested:
./setup_chromeos --public ...

PRESUBMIT=passed
R=raymes,bjanakiraman
CC=bhaskar-staff
DELTA=64 (22 added, 24 deleted, 18 changed)
OCL=55770-p2
RCL=55773-p2
RDATE=2011/09/27 17:21:32


P4 change: 42655074
14/automation/clients/helper/chromeos.py
14/setup_chromeos.py
a84c0e89301bacb936c8a9f926e13daf8e911d67 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> DejaGNU reporting tool: don't assume that the database file is in a fixed location.

Tested:
Locally.

PRESUBMIT=passed
R=jingyu,shenhan
CC=bhaskar-staff
DELTA=4 (3 added, 0 deleted, 1 changed)
OCL=55678-p2
RCL=55682-p2
RDATE=2011/09/21 14:42:18


P4 change: 42654571
14/automation/clients/report/dejagnu.sh
be1d640ef6386cce5d299d7c03afeafc02126bd7 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Make JobGroupManager resistant to callers, who crash inside it (refer to cl/55579-p2). Make GetJobGroup method thread-safe. Remove clash between namespaces.

Tested:
Locally.

PRESUBMIT=passed
R=cmtice,yunlian
CC=bhaskar-staff
DELTA=103 (28 added, 30 deleted, 45 changed)
OCL=55584-p2
RCL=55649-p2
RDATE=2011/09/20 11:14:48


P4 change: 42654467
14/automation/server/job_group_manager.py
a0c25297ded2a0b9e54bec22b9f544b3170ab0ee 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> CommandExecuter: add tests and make them pass.

Following tests are failing when the command is executed over SSH:
- testCommandTimeout
- testCommandTimeoutIfSigTermIgnored
- testInterruptedProcess
Unfortunately due to SSH implementation, it's extremely difficult to handle these test cases correctly. Significant effort and profound API changes are required to clear these failures.

Tested:
cd automation/common/
./command_executer_test.py

PRESUBMIT=passed
R=asharif,cmtice,yunlian
CC=bhaskar-staff
DELTA=291 (266 added, 11 deleted, 14 changed)
OCL=55375-p2
RCL=55587-p2
RDATE=2011/09/16 16:17:47


P4 change: 42654139
14/automation/common/command_executer.py
14/automation/common/command_executer_test.py
d07e2678f9b34cfe4f02397dbe62ea172ce21544 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> DejaGNU report tool: implement test result rewrite rules, reorganize database,
huge performance improvements (~10x).

Test result rewriting functionality is needed to suppress test failures (i.e. for some builds some tests are supposed to fail, so we should read these failures as expected). This solution is more generic than trying to modify tests in gcc testsuite source tree.

Tested:
$ ./dejagnu.sh summary -B $BUILD_NAME dejagnu-output/*.sum
$ ./dejagnu.sh html-report -B $BUILD_NAME -d $BUILD_DATE report.html

PRESUBMIT=passed
R=yunlian,shenhan,raymes
CC=bhaskar-staff,dnovillo
DELTA=284 (136 added, 58 deleted, 90 changed)
OCL=55497-p2
RCL=55586-p2
RDATE=2011/09/16 16:17:37


P4 change: 42654138
14/automation/clients/report/dejagnu/main.py
14/automation/clients/report/dejagnu/models.py
14/automation/clients/report/dejagnu/report.html
14/automation/clients/report/dejagnu/report.py
14/automation/clients/report/dejagnu/summary.py
b44ecf641f4ec213652f3c323e3ad2daabd0620a 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Fix crash when IdProducerPolicy tries to traverse a directory that doesn't
exist. Add documentation for IdProducerPolicy class.

Tested:
rm -rf /usr/local/google/tmp/automation
python automation/server/server.py -n

PRESUBMIT=passed
BUG=5202727
R=asharif
CC=bhaskar-staff
DELTA=28 (22 added, 0 deleted, 6 changed)
OCL=55582-p2
RCL=55585-p2
RDATE=2011/09/16 16:00:58


P4 change: 42654136
14/automation/server/job_manager.py
4497d2d25779fae07fa45ed5af3544acce65ec13 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Crosstool nightly client: don't store toolchain packages locally, the only copy
we need should be in X20.

The benefits are:
- mobiletc-prebuild home directory will not grow in size so rapidly,
- no need to restart server frequently (in order to remove old directories).

Tested:
Locally.

PRESUBMIT=passed
R=yunlian,shenhan
CC=bhaskar-staff
DELTA=32 (10 added, 17 deleted, 5 changed)
OCL=55580-p2
RCL=55583-p2
RDATE=2011/09/16 15:11:09


P4 change: 42654028
14/automation/clients/helper/crosstool.py
090c071678e35168ae39985ba9142946ba124a29 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Make JobManager resistant to callers, who crash inside JobManager's method. If that happens Lock.__exit__ will automatically release lock, preventing the main loop from being locked up permanently.

http://docs.python.org/library/threading.html#using-locks-conditions-and-semaphores-in-the-with-statement

Tested:
Locally.

PRESUBMIT=passed
R=cmtice,yunlian
CC=bhaskar-staff
DELTA=84 (14 added, 26 deleted, 44 changed)
OCL=55546-p2
RCL=55579-p2
RDATE=2011/09/16 13:53:57


P4 change: 42654026
14/automation/server/job_manager.py
a71f1c79be26f8e5821daee857cd91bc4543d1cc 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Post freshly built toolchain to X20. Remove old toolchains from X20 (leave only
20 most recent directories).

Tested:
N/A

PRESUBMIT=passed
R=bjanakiraman,dougkwan
CC=bhaskar-staff
DELTA=35 (24 added, 0 deleted, 11 changed)
OCL=55397-p2
RCL=55535-p2
RDATE=2011/09/15 13:21:44


P4 change: 42653957
14/automation/clients/helper/crosstool.py
ba315e4d6d53f803256f29763cc30231b151f4d6 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Remove JobPreparer class. Fix clash between "job" namespace and variables in
job_manager.py. Replace Job.{home,work}_dir fields with properties determined at run-time. Fix glint warnings.

Tested:
Locally.

PRESUBMIT=passed
R=cmtice,shenhan,raymes
CC=bhaskar-staff
DELTA=64 (12 added, 24 deleted, 28 changed)
OCL=55484-p2
RCL=55506-p2
RDATE=2011/09/14 17:04:03


P4 change: 42653895
14/automation/common/job.py
14/automation/server/job_manager.py
e879118e244a542538958d6180b7d73fd1d13e94 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Crosstool nighly client: import all test results into database. DejaGNU summary parser: accept glob expressions as paths to dejagnu .sum files.

Tested:
On chrome-dev1.hot.

PRESUBMIT=passed
R=yunlian,shenhan
CC=bhaskar-staff
DELTA=5 (2 added, 1 deleted, 2 changed)
OCL=55479-p2
RCL=55482-p2
RDATE=2011/09/13 15:27:40


P4 change: 42653862
14/automation/clients/helper/crosstool.py
14/automation/clients/report/dejagnu/main.py
81e3ba3eed2c540c2af1d4a996e766ad854c77b4 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Fix job submission process, when controlling terminal is missing (ie. Job is
posted from CRON).

For problems related to using os.getlogin() function refer to
http://linux.die.net/man/3/getlogin

Tested:
On chrome-dev1.hot.

PRESUBMIT=passed
R=asharif,yunlian
CC=bhaskar-staff
DELTA=2 (1 added, 0 deleted, 1 changed)
OCL=55473-p2
RCL=55477-p2
RDATE=2011/09/13 13:36:01


P4 change: 42653859
14/automation/common/job_group.py
1f2f4242f3d3c2206b6e1e1143db980efe8809f5 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Replaced Job.log_*_filename with Job.log_filename_prefix (as suggested by cmtice).

Tested:
Locally.

PRESUBMIT=passed
R=cmtice,shenhan
CC=bhaskar-staff
DELTA=14 (1 added, 8 deleted, 5 changed)
OCL=55469-p2
RCL=55476-p2
RDATE=2011/09/13 13:35:08


P4 change: 42653858
14/automation/common/job.py
14/automation/server/job_executer.py
14/automation/server/monitor/dashboard.py
0c906c90df126ac1f1d03d6f62993e7e2271cfd1 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Move Logger instance from Job to JobExecuter, thus removing nasty pickle (http://docs.python.org/library/pickle.html) related workaround.

Tested:
Locally.

PRESUBMIT=passed
R=cmtice,shenhan
CC=bhaskar-staff
DELTA=21 (3 added, 13 deleted, 5 changed)
OCL=55451-p2
RCL=55468-p2
RDATE=2011/09/13 10:37:45


P4 change: 42653665
14/automation/common/job.py
14/automation/server/job_executer.py
14/automation/server/job_manager.py
e1b1f9c5b0816ae27948dea8a4765014a98581c9 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Add comment regarding dangerous commands. Fix a bug in RemoteCopyFrom.

Tested:
Locally.

PRESUBMIT=passed
R=asharif,shenhan
CC=bhaskar-staff
DELTA=8 (7 added, 0 deleted, 1 changed)
OCL=55392-p2
RCL=55395-p2
RDATE=2011/09/08 13:49:31


P4 change: 42653440
14/automation/common/command.py
fe30dd5ad4357e9846038cb5fe0e4bf4a17a3f71 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> DejaGNU report generator: add filtering by build-name/board/build-date, add logging messages.

Tested:
Locally.

PRESUBMIT=passed
R=asharif,cmtice,yunlian
CC=bhaskar-staff
DELTA=34 (26 added, 0 deleted, 8 changed)
OCL=55324-p2
RCL=55374-p2
RDATE=2011/09/07 15:09:15


P4 change: 42653284
14/automation/clients/report/dejagnu/main.py
14/automation/clients/report/dejagnu/report.py
d9398bb34063efd0ec2e2d0a45ddc492226d88a8 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> JobGroup related fixes and cleanups:
- Fix IdProducerPolicy initializer, so latest JobGroup ID is properly extracted
from directory listing.
- Remove JobGroupPreparer.
- Replace JobGroup.status attribute with state machine.
- Replace JobGroup.time_submitted and JobGroup.home_dir attributes with
properties determined at runtime.
- Remove Submit method.
- Fixed JobGroup related methods in dashboard.

Tested:
Locally.

PRESUBMIT=passed
R=raymes,cmtice
CC=bhaskar-staff
DELTA=85 (37 added, 35 deleted, 13 changed)
OCL=55340-p2
RCL=55373-p2
RDATE=2011/09/07 15:09:05


P4 change: 42653282
14/automation/common/job_group.py
14/automation/server/job_group_manager.py
14/automation/server/monitor/dashboard.py
6b970253ee0e5fe14678ebe5eca93543cbc8d2b0 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Crosstool nighly client fix: REMOTE_TMPDIR is used by pandaboard
DejaGNU board, so the env variable should be defined when
'make check' is run.

Tested:
Locally.

PRESUBMIT=passed
R=bjanakiraman,yunlian
CC=bhaskar-staff
DELTA=7 (4 added, 1 deleted, 2 changed)
OCL=55335-p2
RCL=55372-p2
RDATE=2011/09/07 15:08:55


P4 change: 42653281
14/automation/clients/helper/crosstool.py
b225e79aa4504325a8001aa4ac21bec516a049c8 15-Feb-2013 asharif <asharif@p4.invalid> Fixed a bug in tc_enter_chroot.py where the mountpoints would be wrong
if it was called from a symlink'd directory.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=2 (0 added, 1 deleted, 1 changed)
OCL=55337-p2
RCL=55341-p2
RDATE=2011/09/02 18:18:05


P4 change: 42653198
14/tc_enter_chroot.py
c27f8c61e6a8a483cc6e0a8ff975126c2b5b8ad1 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> DejaGNU summary parser: add logging messages.

Tested:
Locally.

PRESUBMIT=passed
R=cmtice,yunlian
CC=bhaskar-staff
DELTA=10 (10 added, 0 deleted, 0 changed)
OCL=55330-p2
RCL=55334-p2
RDATE=2011/09/02 14:49:29


P4 change: 42653196
14/automation/clients/report/dejagnu/summary.py
64b67468f65be3372c3e6eeb3f3b4d185da8d6c3 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Due to changes in server.job_executer.JobExecuter class, job's results are placed
directly under job's homedir. Make dashboard fetch HTML report from proper
location.

Tested:
On chrome-dev1.hot.

PRESUBMIT=passed
R=asharif,yunlian
CC=bhaskar-staff
DELTA=1 (0 added, 0 deleted, 1 changed)
OCL=55326-p2
RCL=55331-p2
RDATE=2011/09/02 13:48:57


P4 change: 42653195
14/automation/server/monitor/dashboard.py
059c39fc7d2e57f21ae0de9d5b3225038a45c382 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Typo fix in chromeos nightly client.

Tested:
On chrome-dev1.hot.

PRESUBMIT=passed
R=asharif,yunlian
CC=bhaskar-staff
DELTA=1 (0 added, 0 deleted, 1 changed)
OCL=55325-p2
RCL=55327-p2
RDATE=2011/09/02 13:08:40


P4 change: 42653194
14/automation/clients/helper/chromeos.py
8b4a31e0512d69b8c3d77598e971ccc626004128 15-Feb-2013 asharif <asharif@p4.invalid> Changed the location of osimage_checksum_file.

Seems like /home/chronos/ gets reset after running autotests. This new
location seems to be resistant to autotests so far.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=1 (0 added, 0 deleted, 1 changed)
OCL=55301-p2
RCL=55322-p2
RDATE=2011/09/02 10:37:21


P4 change: 42653192
14/image_chromeos.py
731e293aa974256816d5670c1b517531c6883f4d 15-Feb-2013 asharif <asharif@p4.invalid> Made setup_chromeos.py use the chromium mirror instead of kernel.org's
git repo during repo init. kernel.org's repo is down since a security
breach happened.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=1 (1 added, 0 deleted, 0 changed)
RCL=55321-p2
RDATE=2011/09/02 10:37:16


P4 change: 42653191
14/setup_chromeos.py
4e68a087f6c7f3c8fa6bda8df68bdecc1e4a4bcf 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Cleanup: remove InitCommandExecuter and GetCommandExecuter functions from job_executer module.

Tested:
Locally.

PRESUBMIT=passed
R=raymes
CC=bhaskar-staff
DELTA=46 (14 added, 18 deleted, 14 changed)
RCL=55318-p2
RDATE=2011/09/01 18:18:44


P4 change: 42653190
14/automation/common/command_executer.py
14/automation/server/job_executer.py
14/automation/server/job_group_manager.py
14/automation/server/server.py
325593e5843b4fb28e28354ea02a8e279dd4c3fd 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Move _FormatCommand method to Job class and rename to GetCommand. Make PrettyFormatCommand output command in more readable form (visible through dashboard).

Tested:
Locally.

PRESUBMIT=passed
R=asharif,raymes
CC=bhaskar-staff
DELTA=42 (20 added, 17 deleted, 5 changed)
OCL=55314-p2
RCL=55317-p2
RDATE=2011/09/01 17:49:03


P4 change: 42653179
14/automation/common/job.py
14/automation/server/job_executer.py
0b60d7739061911a669e6202b9caca53c1a80554 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Execute all subcommands in the same shell instead of forwarding them to
subshell. This fixes Wrapper related bug, where settings are applied
in subshell, which is destroyed right before wrapped command is
executed.

Tested:
Locally.

PRESUBMIT=passed
R=asharif,bjanakiraman,raymes
CC=bhaskar-staff
DELTA=1 (0 added, 0 deleted, 1 changed)
RCL=55315-p2
RDATE=2011/09/01 17:05:59


P4 change: 42653177
14/automation/common/command.py
a1476e612de743a59f17f7514e932ebb32739081 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> JobExecuter and CommandExecuter cleanups.

Tested:
Locally with crosstool nightly client in both real and dry-run mode.

PRESUBMIT=passed
R=raymes
CC=bhaskar-staff
DELTA=279 (82 added, 139 deleted, 58 changed)
OCL=55292-p2
RCL=55313-p2
RDATE=2011/09/01 16:29:40


P4 change: 42653173
14/automation/common/command_executer.py
14/automation/common/job.py
14/automation/server/job_executer.py
cc113a49065b6181daf301be86cb44409c75f278 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Cleaned up automation.clients.helper.jobs. Moved shell command helpers to automation.common.command.

Tested:
Locally.

PRESUBMIT=passed
R=asharif,jingyu
CC=bhaskar-staff
DELTA=115 (53 added, 39 deleted, 23 changed)
OCL=55304-p2
RCL=55312-p2
RDATE=2011/09/01 16:01:11


P4 change: 42653171
14/automation/clients/helper/android.py
14/automation/clients/helper/chromeos.py
14/automation/clients/helper/crosstool.py
14/automation/clients/helper/jobs.py
14/automation/common/command.py
14/automation/server/job_group_manager.py
0a56803066aba097af345d34bdbe7396ea660b8e 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Crosstool nighly client: generate a report only if tests were run.

Tested:
N/A

PRESUBMIT=passed
R=bjanakiraman,dougkwan
CC=bhaskar-staff
DELTA=4 (0 added, 1 deleted, 3 changed)
RCL=55299-p2
RDATE=2011/08/31 16:49:11


P4 change: 42653164
14/automation/clients/crosstool.py
2a655b64a39e9084934edd0c972193f347720cd6 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Crosstool nightly client: add "unix" to the list of recognized DejaGNU boards.

Tested:
Locally.

PRESUBMIT=passed
R=bjanakiraman,dougkwan
CC=bhaskar-staff
DELTA=1 (0 added, 0 deleted, 1 changed)
RCL=55298-p2
RDATE=2011/08/31 16:19:17


P4 change: 42653151
14/automation/clients/crosstool.py
e78cbd40b5c096b9f015ef1ed612c3f7e9f8b760 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Crosstool nightly client: move report generation to last step to make results display properly.

Tested:
Locally.

PRESUBMIT=passed
R=bjanakiraman,dougkwan
CC=bhaskar-staff
DELTA=30 (19 added, 3 deleted, 8 changed)
OCL=55294-p2
RCL=55296-p2
RDATE=2011/08/31 15:56:09


P4 change: 42653147
14/automation/clients/crosstool.py
14/automation/clients/helper/crosstool.py
85f6ffe7ce6c8bf325737152101a416488cb952d 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Remove legacy script.

Tested:
N/A

PRESUBMIT=passed
R=raymes
DELTA=172 (0 added, 172 deleted, 0 changed)
OCL=55288-p2
RCL=55290-p2
RDATE=2011/08/31 10:37:18


P4 change: 42653141
14/run_dejagnu.py
2603eb2f48b46c3f997fe24cfebdedb59cb8eae7 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Remove dependencies on utils package. Remove unused parts of CommandExecuter.
Fix minor bugs related to discrepancy between CommandExecuter and
MockCommandExecuter. Fix glint warnings.

Tested:
Locally.

PRESUBMIT=passed
R=bjanakiraman,raymes,asharif
CC=bhaskar-staff
DELTA=370 (332 added, 14 deleted, 24 changed)
OCL=55237-p2
RCL=55287-p2
RDATE=2011/08/31 10:13:38


P4 change: 42653140
14/automation/clients/android.py
14/automation/clients/crosstool.py
14/automation/clients/helper/chromeos.py
14/automation/clients/helper/jobs.py
14/automation/common/command_executer.py
14/automation/common/logger.py
14/automation/server/job_executer.py
14/automation/server/job_group_manager.py
14/automation/server/job_manager.py
14/automation/server/server.py
ef80001418717e69285db2094615afd7bbb4f88b 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Generate HTML dejagnu report. Fix RUNTESTFLAGS (pass board name correctly).

Tested:
Locally.

PRESUBMIT=passed
R=bjanakiraman
CC=bhaskar-staff
DELTA=19 (16 added, 0 deleted, 3 changed)
OCL=55260-p2
RCL=55274-p2
RDATE=2011/08/30 17:14:44


P4 change: 42653127
14/automation/clients/helper/crosstool.py
3d69d8b76f487f74be152ed15b75438e7de947bd 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> HTML report generator for DejaGNU test run output.

Tested:
Locally.

PRESUBMIT=passed
R=dougkwan,asharif,bjanakiraman,raymes
CC=bhaskar-staff
DELTA=557 (557 added, 0 deleted, 0 changed)
OCL=55250-p2
RCL=55273-p2
RDATE=2011/08/30 17:11:28


P4 change: 42653126
14/automation/clients/report/dejagnu.sh
14/automation/clients/report/dejagnu/__init__.py
14/automation/clients/report/dejagnu/main.py
14/automation/clients/report/dejagnu/models.py
14/automation/clients/report/dejagnu/report.html
14/automation/clients/report/dejagnu/report.py
14/automation/clients/report/dejagnu/settings.py
14/automation/clients/report/dejagnu/summary.py
c8e8e12481992e728c4b28c94f5f49d10056ecbc 15-Feb-2013 asharif <asharif@p4.invalid> Changed tc_enter_chroot.py to use build_packages instead of
enter_chroot.sh as a file that identifies the $chromeos_root/src/scripts
directory.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=2 (0 added, 0 deleted, 2 changed)
OCL=55253-p2
RCL=55272-p2
RDATE=2011/08/30 13:26:46


P4 change: 42653125
14/tc_enter_chroot.py
852482fa75c7d9f12a7d94883259cc8b582d76cc 15-Feb-2013 asharif <asharif@p4.invalid> Added --nowithdebug flag to build_packages.
This should build ChromeOS packages with NDEBUG defined and should
remove "Debug Build" from the bottom right of the screen.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=1 (0 added, 0 deleted, 1 changed)
OCL=55252-p2
RCL=55271-p2
RDATE=2011/08/30 13:26:37


P4 change: 42653124
14/utils/utils.py
2297afe3f2f18bf7b84c3543bf71a477ffba4e53 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Remove report generation step from JobExecuter. Job has to create a report HTML file explicitly. Make the dashboard fetch report files from 'job-group-n/job-*/results/report.html' locations.

Tested:
Run job producing HTML report locally.

PRESUBMIT=passed
R=dougkwan,asharif,bjanakiraman,raymes
CC=bhaskar-staff
DELTA=76 (7 added, 62 deleted, 7 changed)
OCL=55248-p2
RCL=55259-p2
RDATE=2011/08/29 17:25:22


P4 change: 42653105
14/automation/common/job.py
14/automation/server/job_executer.py
14/automation/server/monitor/dashboard.py
7aad14d94ef4840592ec66a6c9b4dd876d51260d 15-Feb-2013 asharif <asharif@p4.invalid> Changed CWDSwitcher to WorkingDirectory using yield construct.

This change was supposed to be submitted with cl/55241-p2.

PRESUBMIT=passed
R=kbaclawski
APPROVED=kbaclawski
DELTA=13 (1 added, 6 deleted, 6 changed)
OCL=55249-p2
RCL=55251-p2
RDATE=2011/08/29 12:49:16


P4 change: 42652843
14/utils/utils.py
0ea8958095afb11c8dcb95b5ca332a79cfee1a93 15-Feb-2013 asharif <asharif@p4.invalid> repo_to_repo.py now passes with glint (modulo a few missing docstrings).

PRESUBMIT=passed
R=raymes,bjanakiraman,kbaclawski
DELTA=121 (6 added, 20 deleted, 95 changed)
OCL=55218-p2
RCL=55241-p2
RDATE=2011/08/26 13:11:16


P4 change: 42652829
14/repo_to_repo.py
282d39374ab3f50b14c06677329809106e2d51d1 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Remove executable bit from scripts that doesn't need it. Remove old,
unused files using API that doesn't exist any more.

Tested:
N/A

PRESUBMIT=passed
R=asharif,bjanakiraman,raymes
CC=bhaskar-staff
DELTA=226 (0 added, 226 deleted, 0 changed)
OCL=55236-p2
RCL=55239-p2
RDATE=2011/08/26 11:36:16


P4 change: 42652828
14/automation/clients/__init__.py
14/automation/clients/build_tc_client.py
14/automation/clients/helper/jobs.py
14/automation/clients/helper/jobs_test.py
14/automation/clients/status_client.py
14/automation/clients/test_client.py
14/automation/clients/update_client.py
14/automation/common/__init__.py
14/automation/common/job.py
14/automation/common/machine.py
14/automation/server/__init__.py
14/automation/server/job_executer.py
14/automation/server/job_manager.py
14/automation/server/machine_manager.py
81928e3bc236be389c9c43729651440201507339 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Dashboard: add simple file system viewer for job group directory.

Tested:
Locally on vordhosbn.mtv:8080. Checked for path containing ".." and they're
rejected.

PRESUBMIT=passed
R=asharif,bjanakiraman,raymes
CC=bhaskar-staff
DELTA=19 (17 added, 1 deleted, 1 changed)
OCL=55225-p2
RCL=55232-p2
RDATE=2011/08/25 17:26:35


P4 change: 42652825
14/automation/server/monitor/dashboard.py
14/automation/server/monitor/urls.py
7d53d6fa480ce5e79f54ddc5a53afa95ef7ff990 15-Feb-2013 raymes <raymes@p4.invalid> Fixed cros_run_benchmarks.py so that it works with the new locking.

PRESUBMIT=passed
R=asharif,bjanakiraman
DELTA=15 (2 added, 3 deleted, 10 changed)
RCL=55228-p2
RDATE=2011/08/25 10:57:45


P4 change: 42652814
14/cros_run_benchmarks.py
7e7da9952a3e9f6f1f13814bd749c0bc632e4d16 15-Feb-2013 asharif <asharif@p4.invalid> Added stringify to command_executer's command.

PRESUBMIT=passed
R=kbaclawski
DELTA=3 (3 added, 0 deleted, 0 changed)
OCL=55219-p2
RCL=55223-p2
RDATE=2011/08/24 16:34:49


P4 change: 42652809
14/utils/command_executer.py
f068e7b6ee5db9f54f3a00b01436834a82a4d34a 15-Feb-2013 asharif <asharif@p4.invalid> Fix for the exception we are seeing. I also added a unit-test for it.

In the future we can add a shell script template for a server test that
will start the server with a machine list and execute a job.

Tested:
./no_machine_available_test.sh
(printed FAIL before my CL and PASS after my CL).

PRESUBMIT=passed
R=raymes,bjanakiraman,kbaclawski
CC=bhaskar-staff
DELTA=2 (0 added, 0 deleted, 2 changed)
OCL=55193-p2
RCL=55217-p2
RDATE=2011/08/24 15:35:14


P4 change: 42652800
14/automation/server/job_manager.py
b221a9cfb95b65223657c364016156408e90c67c 15-Feb-2013 asharif <asharif@p4.invalid> Changed image_chromeos.py so it doesn't break mount points when running
mount_gpt_image.sh.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=23 (10 added, 0 deleted, 13 changed)
OCL=55214-p2
RCL=55216-p2
RDATE=2011/08/24 15:34:47


P4 change: 42652799
14/image_chromeos.py
56d78bc805deb158c595fcb81a605ac3b25dc9aa 15-Feb-2013 asharif <asharif@p4.invalid> Added ability to pull sources from git repositories.

Unified a few functions and refactored the file.

PRESUBMIT=passed
R=cmtice,raymes,bjanakiraman,kbaclawski
APPROVED=cmtice
DELTA=182 (89 added, 30 deleted, 63 changed)
OCL=55171-p2
RCL=55215-p2
RDATE=2011/08/24 13:49:30


P4 change: 42652798
14/repo_to_repo.py
14/utils/utils.py
23a2fa3cb3971812df318e3e7ef034d1ad4bc5dc 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Fix locking issues.

Tested:
Patched chrome-dev1.hot server and run chromeos job.

PRESUBMIT=passed
R=asharif,bjanakiraman,raymes
CC=bhaskar-staff
DELTA=6 (2 added, 1 deleted, 3 changed)
OCL=55211-p2
RCL=55213-p2
RDATE=2011/08/24 12:57:03


P4 change: 42652795
14/automation/common/machine.py
4a87589841c9ad2ce28aba95910b4f5182e759e3 15-Feb-2013 cmtice <cmtice@p4.invalid> Added a new .rtr file for gdb-7.2

Tested=Ran repo_to_repo on it.

PRESUBMIT=passed
R=asharif,raymes,bjanakiraman,kbaclawski
APPROVED=asharif
DELTA=13 (13 added, 0 deleted, 0 changed)
OCL=55208-p2
RCL=55212-p2
RDATE=2011/08/24 12:48:40


P4 change: 42652794
14/repo_to_repo_files/gdb-7.2.x_chromeos_master.rtr
f81c60dc55bde19dfee7697533fe865dfcbe8acf 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Update crosstool nightly client to reflect current needs.
- Update command line syntax.
- Add DejaGNU test run on qemu and pandaboard.
- Copy deb packages to persistent location.
- Add gLucid target.

Tested:
Run job locally on vordhosbn.mtv.

PRESUBMIT=passed
R=asharif,bjanakiraman,raymes
CC=bhaskar-staff
DELTA=120 (29 added, 51 deleted, 40 changed)
OCL=55202-p2
RCL=55210-p2
RDATE=2011/08/24 11:43:26


P4 change: 42652793
14/automation/clients/crosstool.py
14/automation/clients/helper/crosstool.py
719c2268be30ed0729aeb145db67075de1109000 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Allow shell command to fail, without breaking a command chain. This change is required by the new implementation of crosstool nightly client.

Tested:
By subsequent CL: GNU Make returns non-zero status, but the job finishes successfully.

PRESUBMIT=passed
R=asharif,bjanakiraman,raymes
CC=bhaskar-staff
DELTA=8 (6 added, 0 deleted, 2 changed)
OCL=55199-p2
RCL=55201-p2
RDATE=2011/08/23 13:57:14


P4 change: 42652779
14/automation/common/command.py
24c27a9955e64fd3a840d1d0e3125705aa534024 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Correct CSS style name for failed Jobs and JobGroups. Now they'll show in red.

Tested:
Run a job group known to fail and checked dashboard.

PRESUBMIT=passed
R=asharif,bjanakiraman,raymes
CC=bhaskar-staff
DELTA=2 (0 added, 0 deleted, 2 changed)
OCL=55196-p2
RCL=55198-p2
RDATE=2011/08/23 13:14:53


P4 change: 42652776
14/automation/server/monitor/dashboard.py
48d93f3d62c5134bfd223e843f030fc96c52bf14 15-Feb-2013 asharif <asharif@p4.invalid> Added a timeout of 5 hours to each job by default.

PRESUBMIT=passed
R=raymes,bjanakiraman,kbaclawski
CC=bhaskar-staff
DELTA=2 (1 added, 0 deleted, 1 changed)
OCL=55191-p2
RCL=55195-p2
RDATE=2011/08/23 12:00:52


P4 change: 42652766
14/automation/server/job_executer.py
3679cc5613c664f926b3c56e7c46ea43bbecd133 15-Feb-2013 asharif <asharif@p4.invalid> ChromeOS job no longer locks the linux machine.

PRESUBMIT=passed
R=raymes,bjanakiraman,kbaclawski
CC=bhaskar-staff
DELTA=1 (0 added, 0 deleted, 1 changed)
OCL=55190-p2
RCL=55194-p2
RDATE=2011/08/23 12:00:24


P4 change: 42652765
14/automation/clients/helper/chromeos.py
455157bf6a4d439e25726eac32ae0c7e1d4c5246 15-Feb-2013 asharif <asharif@p4.invalid> Split LockMachine() into 2 functions: one for lock and the other for unlock.
Also added a check before the unlock takes place to make sure the user
unlocking is the same as the user who locked the machine.

PRESUBMIT=passed
R=raymes,bjanakiraman,kbaclawski
DELTA=278 (188 added, 17 deleted, 73 changed)
OCL=55045-p2
RCL=55189-p2
RDATE=2011/08/23 10:37:54


P4 change: 42652763
14/lock_machine.py
14/lock_machine_test.py
e4cb3aaa6128981e0634e8ebb289ed19d8d712fc 15-Feb-2013 asharif <asharif@p4.invalid> Removed binutils-2.20.1 rtr file in favor of binutils-2.21.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=30 (14 added, 16 deleted, 0 changed)
RCL=55185-p2
RDATE=2011/08/22 14:30:20


P4 change: 42652761
14/repo_to_repo_files/binutils-2.20.1_master.rtr
14/repo_to_repo_files/binutils-2.21_master.rtr
1b879d79759503a0f8c06d06f2485490a1875538 15-Feb-2013 asharif <asharif@p4.invalid> Added rtr file for gcc-4.6.0-mobile.

PRESUBMIT=passed
R=raymes,bjanakiraman,cmtice
DELTA=13 (13 added, 0 deleted, 0 changed)
OCL=55156-p2
RCL=55169-p2
RDATE=2011/08/19 13:42:42


P4 change: 42652731
14/repo_to_repo_files/gcc-4.6.0_branches_google_4_6-mobile.rtr
e2cc4c610e34ff8ec0290e5d16f00fbeae7f3b3c 15-Feb-2013 asharif <asharif@p4.invalid> Added an rtr file for pushing mobile_toolchain_v15_release_branch
binutils-2.21 into the chromium git repository.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=16 (16 added, 0 deleted, 0 changed)
OCL=55135-p2
RCL=55155-p2
RDATE=2011/08/18 16:08:23


P4 change: 42652723
14/repo_to_repo_files/binutils-2.21_mobile_toolchain_v15_release_branch.rtr
0dd8f0bf102d5004cf4fba7348dcc5e7dd9222b3 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Add missing pieces to the server, related to fetching the machine list.

Tested:
Manually at http://vordhosbn.mtv:8080.

PRESUBMIT=passed
R=asharif,bjanakiraman,raymes
CC=bhaskar-staff
DELTA=8 (8 added, 0 deleted, 0 changed)
OCL=55142-p2
RCL=55144-p2
RDATE=2011/08/18 11:27:01


P4 change: 42652705
14/automation/server/machine_manager.py
14/automation/server/server.py
08d352a2de363184b5a88cabde92ef010e3230c6 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Convert the monitor to Django framework.

Tested:
Manually at http://vordhosbn.mtv:8080.

PRESUBMIT=passed
R=asharif,bjanakiraman,raymes
DELTA=1086 (683 added, 403 deleted, 0 changed)
OCL=55036-p2
RCL=55141-p2
RDATE=2011/08/18 10:51:09


P4 change: 42652700
14/automation/server/monitor/__init__.py
14/automation/server/monitor/dashboard.py
14/automation/server/monitor/main.py
14/automation/server/monitor/manage.py
14/automation/server/monitor/settings.py
14/automation/server/monitor/start.sh
14/automation/server/monitor/static/style.css
14/automation/server/monitor/templates/base.html
14/automation/server/monitor/templates/job.html
14/automation/server/monitor/templates/job_group.html
14/automation/server/monitor/templates/job_group_list.html
14/automation/server/monitor/templates/job_log.html
14/automation/server/monitor/templates/machine_list.html
14/automation/server/monitor/templates/snippet_attribute_table.html
14/automation/server/monitor/templates/snippet_code.html
14/automation/server/monitor/templates/snippet_links.html
14/automation/server/monitor/urls.py
1d9986140f8a284ed34f873c89b02c94eb2a23a6 15-Feb-2013 raymes <raymes@p4.invalid> Added PRESUBMIT.py for automation to notify bhaskar-staff of automation-related changes.

PRESUBMIT=passed
R=asharif,kbaclawski,bjanakiraman
APPROVED=kbaclawski
DELTA=14 (14 added, 0 deleted, 0 changed)
OCL=55119-p2
RCL=55129-p2
RDATE=2011/08/17 10:41:00


P4 change: 42652692
14/automation/PRESUBMIT.py
3666653522b8252a1721241e04b0cd4728280db4 15-Feb-2013 asharif <asharif@p4.invalid> Modified tc_enter_chroot.py to use cros_sdk --enter instead of
./enter_chroot.sh.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=7 (4 added, 0 deleted, 3 changed)
OCL=55051-p2
RCL=55097-p2
RDATE=2011/08/15 11:59:21


P4 change: 42652336
14/tc_enter_chroot.py
372a4b214d447bc6b64572a91fffda2b10c20d1f 15-Feb-2013 jingyu <jingyu@p4.invalid> Add Android image checkout and build test to nightly client.

Tested:
Tested on atree
./android.py --with-gcc-version=4.6 --release
./android.py

PRESUBMIT=passed
R=kbaclawski,raymes,asharif
CC=bjanakiraman,dougkwan,carrot
APPROVED=kbaclawski
DELTA=57 (51 added, 1 deleted, 5 changed)
OCL=54837-p2
RCL=54956-p2
RDATE=2011/08/08 11:43:35


P4 change: 42651921
14/automation/clients/android.py
14/automation/clients/helper/android.py
14/automation/clients/helper/jobs.py
e91c866cfbb9c990197c19b8327c6c48da649324 15-Feb-2013 asharif <asharif@p4.invalid> Prepended chromeos git branch with gcc.gnu.org. If we follow this convention,
upstream trunk will not collide with perforce trunk.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=1 (0 added, 0 deleted, 1 changed)
OCL=54863-p2
RCL=54866-p2
RDATE=2011/08/04 11:58:31


P4 change: 42651456
14/repo_to_repo_files/gcc-4.6.0_branches_google_4_6.rtr
c8142368f926f1cb09bf764e35e2690d4a5658f8 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Extract job status related functionality to JobStateMachine class.

- (web-monitor) Improve job related information.
- (state_machine.py) Create a base class BasicStateMachine class.
- (events.py) Create events.History class.
- (Job) Move GetTotalTime and GetTimeline methods to events.History.
- (Job) Replace status field with JobStateMachine object.
- (Job) Fix glint warnings.
- (JobExecuter) Update references to job status.

Tested:
Manually inspected output of nightly, crosstool, android clients, and web
monitor.

PRESUBMIT=passed
R=asharif,bjanakiraman,raymes
DELTA=313 (211 added, 54 deleted, 48 changed)
OCL=54712-p2
RCL=54832-p2
RDATE=2011/08/03 10:29:02


P4 change: 42651401
14/automation/common/events.py
14/automation/common/job.py
14/automation/common/state_machine.py
14/automation/server/monitor/main.py
42bd4a8e6de5469808ecfc130cd5535d75d3262e 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> MachineManager refactoring.

Tested:
Manually inspected output of crosstool and nightly clients.

PRESUBMIT=passed
R=asharif,bjanakiraman,raymes
DELTA=137 (60 added, 66 deleted, 11 changed)
OCL=54591-p2
RCL=54733-p2
RDATE=2011/07/28 17:20:55
DIFFBASE=54574


P4 change: 42651226
14/automation/common/machine.py
14/automation/server/machine_manager.py
14/automation/server/server.py
cf2eabecfb83958725ed664305afe1f7ab5b47ef 15-Feb-2013 asharif <asharif@p4.invalid> Fixed issues pointed out by kbaclawski.

Removed glob import.
Moved code into a separate function.
Used Python functions instead of shell utilities.

PRESUBMIT=passed
R=raymes,bjanakiraman,kbaclawski
APPROVED=kbaclawski
DELTA=52 (23 added, 20 deleted, 9 changed)
OCL=54727-p2
RCL=54731-p2
RDATE=2011/07/28 17:14:16


P4 change: 42651210
14/utils/logger.py
643f17343031c16548a8cab41d6e5c59656d5d01 15-Feb-2013 asharif <asharif@p4.invalid> Added chromeos performance nightly that uses cbuildbot.py and
cros_run_benchmarks.py. We do not use build_tc.py anymore in the nightly client
or checkout sources from perforce. We instead rely on a cron job that updates
the chromeos git repository from s2/ and cbuildbot.py to install the toolchain.

Added a __str__ function to MachineDescription (for debugging).

Added a SaveSpecification() function to perforce.py which does g4 client -o (for
debugging).

PRESUBMIT=passed
R=bjanakiraman,kbaclawski,raymes
DELTA=536 (171 added, 341 deleted, 24 changed)
OCL=54576-p2
RCL=54720-p2
RDATE=2011/07/28 15:37:03


P4 change: 42651207
14/automation/clients/helper/chromeos.py
14/automation/clients/helper/jobs.py
14/automation/clients/helper/perforce.py
14/automation/clients/nightly.py
14/automation/common/machine.py
92b928105f3e57ba423682fc758dc3c678b66a9e 15-Feb-2013 asharif <asharif@p4.invalid> Changed the git branch to be the same as the svn branch (both are
branches/google/gcc-4_6).

Tested:
Tested repo_to_repo.py with -n flag.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=26 (13 added, 13 deleted, 0 changed)
OCL=54674-p2
RCL=54719-p2
RDATE=2011/07/28 15:35:31


P4 change: 42651200
14/repo_to_repo_files/gcc-4.6.0_branches_google_4_6.rtr
14/repo_to_repo_files/gcc-4.6.0_mobile_toolchain_v15_release_branch.rtr
eb9b45fb4a285fc5ae23130b8bbdb0525b4f362b 15-Feb-2013 asharif <asharif@p4.invalid> Added support for multiple log files per python script. This is very useful
for me when running multiple binary searches simultaneously.

Also added the python command and arguments to the cmd file by default.
(Another useful feature for me).

PRESUBMIT=passed
R=bjanakiraman,kbaclawski,raymes
DELTA=49 (46 added, 0 deleted, 3 changed)
OCL=54714-p2
RCL=54718-p2
RDATE=2011/07/28 15:35:12


P4 change: 42651199
14/utils/logger.py
50fe84ceb755a874cfa6ed67ce6d0e21087f259b 15-Feb-2013 asharif <asharif@p4.invalid> Simplified cros_run_benchmarks.py.

Instead of giving it images with the --images flag (comma-separated), you can
just give it images on the command line without specifying any flag.

Makes the old --image_chromeos_root behavior the default. Now you shouldn't
have to give it a --chromeos_root option other than in rare cases.

Made it put cros_scratch in your home directory so it works well with your
experiments across machines (as well as in the automation framework case).

cros_run_benchmarks.py now exits with the right exitcode.

Updated the help message with these changes.

Old:
cp $v14/cros_run_benchmarks.py ~/
~/cros_run_benchmarks.py --remote=$cros1 --tests=Page:3
--images=<image1>,<image2> --chromeos_root=<bla> --image_chromeos_root

New & simplified:
cros_run_benchmarks.py --remote=$cros1 --tests=Page:3 <image1> <image2>

Tested:
cros_run_benchmarks.py --remote=$cros1 --tests=Page:3 <image1> <image2>

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=55 (7 added, 31 deleted, 17 changed)
OCL=54704-p2
RCL=54717-p2
RDATE=2011/07/28 15:33:13


P4 change: 42651198
14/cros_run_benchmarks.py
6df9079fa54276567db4e11bee0877361cbed8c8 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Minor fixes.
- Use isinstance method instead of type comparison in
AbstractCommandContainer._StringifyCommands.
- Remove superfluous prefix from result of Job.GetTotalTime method.

Tested:
Manually, by checking output of the web monitor.

PRESUBMIT=passed
R=asharif,bjanakiraman,jingyu,raymes
DELTA=3 (0 added, 1 deleted, 2 changed)
OCL=54670-p2
RCL=54711-p2
RDATE=2011/07/28 13:49:25
DIFFBASE=54661


P4 change: 42651196
14/automation/common/command.py
14/automation/common/job.py
3318cbdd473d775de2a02f3a2d3dd5f9ca5e9cc2 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Massive web monitor improvements.

- Fix glint warnings in utils.html_tools and automation.server.monitor.main.
- (html_tools.py) Add GetList and GetTable.
- (main.py) Code quality improvements.
- (main.py) Move complex html element rendering to HtmlFactory class.
- (main.py) Improve error handling in BasePageHandler.
- (main.py) Add "/job/%d" subpage (job information).
- (main.py) Add "/filter/%s/%s" subpage (filter job group by client name).
- (main.py) Fix "/results", "/job/%d/log/%s" and "/job/%d/report" handlers.

Tested:
Manually checked all reachable subpages of web interface.

PRESUBMIT=passed
R=asharif,bjanakiraman,jingyu,raymes
DELTA=385 (210 added, 71 deleted, 104 changed)
OCL=54661-p2
RCL=54710-p2
RDATE=2011/07/28 13:45:34


P4 change: 42651195
14/automation/server/monitor/main.py
14/utils/html_tools.py
09e369332ea5bc5401c8e7b5d23c44f9e0ca857d 15-Feb-2013 jingyu <jingyu@p4.invalid> Add android toolchain nightly client and jobs to test infrastructure.
Currently, the nightly client does source checkout and toolchain
build. Users can specify gcc versions (4.4.3, 4.6, google_main,
fsf_trunk). and specify taking sources from release branches
or development branches.

Tested:
./android.py --with-gcc-version=4.4.3
./android.py --with-gcc-version=4.4.3 --release
./android.py --with-gcc-version=4.6 --release
./android.py --with-gcc-version=google_main
./android.py --with-gcc-version=fsf_trunk

PRESUBMIT=passed
R=asharif,kbaclawski,raymes
CC=dougkwan,bjanakiraman
APPROVED=raymes
DELTA=297 (297 added, 0 deleted, 0 changed)
OCL=54635-p2
RCL=54707-p2
RDATE=2011/07/28 12:42:52


P4 change: 42651193
14/automation/clients/android.py
14/automation/clients/helper/android.py
14/automation/clients/helper/perforce.py
17d2163cf53d31035041c0f731a2eb2f37168924 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Move web UI to automation.server.monitor package.

Tested:
Run automation/server/monitor/main.py script.

PRESUBMIT=passed
R=asharif,bjanakiraman,raymes
DELTA=556 (278 added, 278 deleted, 0 changed)
OCL=54640-p2
RCL=54659-p2
RDATE=2011/07/26 12:19:11


P4 change: 42651085
14/automation/clients/web/index.py
14/automation/server/monitor/main.py
2ac5d9090e41af1ccc3080fd9249f1732e0c42ab 15-Feb-2013 asharif <asharif@p4.invalid> Fixed yet another issue with repo_to_repo.py. The problem is that some
repositories have their own .gitignore file and we do not want google
directories and other private .gitignores to be exported to the outside world.

PRESUBMIT=passed
R=cmtice,bjanakiraman
DELTA=1 (0 added, 0 deleted, 1 changed)
OCL=54599-p2
RCL=54603-p2
RDATE=2011/07/22 15:34:31


P4 change: 42650980
14/repo_to_repo.py
00ccdcaff226af647ade2d8205d29eff11a7ef97 15-Feb-2013 asharif <asharif@p4.invalid> Fixed an issue with repo_to_repo.py where the wrong commit message was being
used for the push.

PRESUBMIT=passed
R=cmtice,raymes,bjanakiraman,kbaclawski
DELTA=1 (0 added, 0 deleted, 1 changed)
OCL=54573-p2
RCL=54595-p2
RDATE=2011/07/22 14:05:21


P4 change: 42650972
14/repo_to_repo.py
b72c5fccbfe958ce71e34a65abb99fe7bcb5e343 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Resume Job/JobGroup ID numbering based on directory listing.

Tested:
Created a few directories job-$ID in /usr/local/google/tmp/automation.
Manually inspected output of crosstool and nightly client. Ensured that web
monitor page is not broken.

PRESUBMIT=passed
R=asharif,bjanakiraman,raymes
DELTA=183 (118 added, 41 deleted, 24 changed)
OCL=54574-p2
RCL=54594-p2
RDATE=2011/07/22 13:54:07
DIFFBASE=54564


P4 change: 42650876
14/automation/common/job.py
14/automation/common/job_group.py
14/automation/server/job_executer.py
14/automation/server/job_group_manager.py
14/automation/server/job_manager.py
4a3d8a0be756e7a063dfc15e8a13476b81449063 15-Feb-2013 asharif <asharif@p4.invalid> Changed cros_run_benchmarks.py to use lock_machine.py. It now always locks the
machine (earlier it locked only for machine whose names matched the regex
chromeos-test\d+).
Pressing Ctrl-C in the middle will unlock the machine before exiting.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=26 (7 added, 7 deleted, 12 changed)
OCL=54585-p2
RCL=54589-p2
RDATE=2011/07/22 11:44:06


P4 change: 42650871
14/cros_run_benchmarks.py
90155fbfadda2ae0fd85f3e00a1a649ca7d0211f 15-Feb-2013 asharif <asharif@p4.invalid> Improvements to lock_machine.py:
1. lock_reason now includes the hostname of the machine from which you invoked
this script.
2. If for some reason writing the reason file fails, LockMachine still returns
0 since the lock was acquired. This should be very rare and it should be okay
since the lock directory exists (which is what counts when you unlock the
machine).

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=3 (1 added, 1 deleted, 1 changed)
OCL=54584-p2
RCL=54588-p2
RDATE=2011/07/22 11:42:54


P4 change: 42650870
14/lock_machine.py
28fb90c04434fc98ee678ea434fdb566a287c3b6 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Add Machine labelling for easy reference to specific type of ChromeOS box.

Tested:
Manual inspection of crosstool and nightly clients.

PRESUBMIT=passed
R=asharif,bjanakiraman,raymes
DELTA=46 (6 added, 1 deleted, 39 changed)
OCL=54564-p2
RCL=54586-p2
RDATE=2011/07/22 10:51:21
DIFFBASE=54518


P4 change: 42650868
14/automation/clients/helper/chromeos.py
14/automation/clients/helper/jobs.py
14/automation/clients/web/index.py
14/automation/common/machine.py
14/automation/server/job_executer.py
14/automation/server/job_manager.py
14/automation/server/machine_manager.py
14/automation/server/test_pool.csv
ae29e1c96c0bfac71147a0bb312e2cd091583070 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Rewrite automation.client.web.index CGI script as a standalone web server using
BaseHTTPServer/BaseHTTPRequest classes. Eleminated a dependancy of the
dashboard ob Apache web server.

Tested:
Run the script from command line and accessed web server at localhost:8080.

PRESUBMIT=passed
R=asharif,bjanakiraman,raymes
DELTA=341 (162 added, 92 deleted, 87 changed)
OCL=54518-p2
RCL=54583-p2
RDATE=2011/07/22 10:43:54


P4 change: 42650861
14/automation/clients/web/index.py
b237bcab5cdbd266dfea1a0ebccb40fa7122c0cf 15-Feb-2013 asharif <asharif@p4.invalid> Added lock_machine.py, a utility to lock, unlock and list machine locks.

Tested: lock_machine.py --list
lock_machine.py --machine=ahmad
lock_machine.py --machine=ahmad --unlock
lock_machine.py --machine=ahmad --list

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=136 (136 added, 0 deleted, 0 changed)
OCL=54515-p2
RCL=54551-p2
RDATE=2011/07/20 23:08:03


P4 change: 42650828
14/lock_machine.py
f5a8af039800efa47c774f04f2713c215f76d6b7 15-Feb-2013 asharif <asharif@p4.invalid> Miscelleneous improvements to cros_run_benchmarks.

1. Averages are now done after dropping the slowest/fastest 20% of results.
2. Now you can use --image_chromeos_root to use the chromeos_root that the
image has instead of a fixed chromeos root. This is useful where binaries like
page_cycler_tests should be tied in with an image.
3. Bug fixes related to default iteration/test arguments.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=67 (48 added, 6 deleted, 13 changed)
OCL=54496-p2
RCL=54514-p2
RDATE=2011/07/19 12:02:59


P4 change: 42650740
14/cros_run_benchmarks.py
81fc5d6447503c8f73b52929a23ca58643b35fa4 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Removed dependencies on utils.utils module in automation framework.

Tested:
Manually inspected output of nightly and crosstool clients.

PRESUBMIT=passed
R=asharif,bjanakiraman,raymes
DELTA=30 (14 added, 10 deleted, 6 changed)
OCL=54495-p2
RCL=54503-p2
RDATE=2011/07/18 17:11:39


P4 change: 42650647
14/automation/clients/helper/jobs.py
14/automation/clients/output_test.py
14/automation/clients/web/index.py
14/automation/common/job.py
14/automation/server/job_executer.py
14/automation/server/machine_manager.py
14/automation/server/server_test.py
1b92e73911a1070fe22070735de13251bdbb2b80 15-Feb-2013 asharif <asharif@p4.invalid> Changed repo_to_repo.py to use commands from perforce.py.

Removed perforce utilities from utils.py.

Tested:
Ran repo_to_repo.py -n repo_to_repo_files/toolchain_utils.rtr and it seemed
to work.

PRESUBMIT=passed
R=raymes,bjanakiraman,kbaclawski
DELTA=94 (22 added, 51 deleted, 21 changed)
OCL=54466-p2
RCL=54490-p2
RDATE=2011/07/18 10:31:39


P4 change: 42650645
14/automation/clients/helper/perforce.py
14/repo_to_repo.py
14/utils/utils.py
085ff810025a50f058c01a042061d392b55f75c4 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Fix crosstool client. Perform necessary changes to
perforce.CommandsFactory class and its dependencies:
* (perforce.py) Rename Setup method to Initialize. Add
SaveCurrentCLNumber method that prints / saves to a file number of
currently checked out CL. Change sematics of Checkout method to
make it more flexible, rename to SetupAndDo to reflect what it
really does.
* (jobs.py) Dependency correction.
* (crosstool.py) Dependency correction. Pass CL number to buildit
script directly, because buildit is forced to work outside of a
perforce workspace.

Tested:
Manually inspected output of nightly.py and crosstool.py clients.

PRESUBMIT=passed
R=asharif,raymes
CC=bjanakiraman
DELTA=21 (12 added, 3 deleted, 6 changed)
OCL=54371-p2
RCL=54460-p2
RDATE=2011/07/15 12:17:32


P4 change: 42650300
14/automation/clients/helper/crosstool.py
14/automation/clients/helper/jobs.py
14/automation/clients/helper/perforce.py
bb275f3117f49fea6eb79baa166aaa14f30249b7 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Add another shell command container used to contruct shell pipelines.
Extract the common part of Chain and Pipe to AbstractCommandContainer.

Tested:
Dry run of nightly.py and crosstool.py clients.

PRESUBMIT=passed
R=asharif,raymes
CC=bjanakiraman
DELTA=76 (61 added, 8 deleted, 7 changed)
OCL=54370-p2
RCL=54392-p2
RDATE=2011/07/12 14:33:55


P4 change: 42650141
14/automation/common/command.py
12089d4b28caf662854931ab229b8ea573ba688f 15-Feb-2013 shenhan <shenhan@p4.invalid> Copy (cpu/bikjmp) benchmark material instead of just making soft links. ('rsync' ignores non-regular file without "-L".)

PRESUBMIT=passed
R=bjanakiraman
CC=asharif,raymes
APPROVED=bjanakiraman
OCL=54357-p2
RCL=54364-p2
RDATE=2011/07/08 21:29:19


P4 change: 42649544
14/build_benchmarks.py
73373419c0117896f96d9a8ca7f13b99b0de9f9c 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Fix perforce client creation.

Tested:
On chrome-dev1:
server.py
crosstool.py -t gcc-4.4.3-glibc-2.11.1-armv7a-vfpv3.d16-hard

PRESUBMIT=passed
R=asharif,raymes
DELTA=12 (8 added, 1 deleted, 3 changed)
RCL=54361-p2
RDATE=2011/07/08 12:37:17


P4 change: 42649538
14/automation/clients/helper/perforce.py
5b6c589d00f7dad8242620038613461a06084b1c 15-Feb-2013 asharif <asharif@p4.invalid> Added --skip_toolchain_update flag to build_chromeos.py.
build_packages now calls setup_board by default unless you pass in
this flag.

PRESUBMIT=passed
R=raymes,bjanakiraman,shenhan
DELTA=1 (0 added, 0 deleted, 1 changed)
OCL=54332-p2
RCL=54339-p2
RDATE=2011/07/06 18:29:31


P4 change: 42649518
14/utils/utils.py
1c5658c8557f837844b8112e4350c14e2e83ae95 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Minor automation.clients.helper package improvements:
* Factor out perforce stuff (Checkout, CheckoutFromSnapshot methods).
* Improve jobs module readability by changing function names to be
more descriptive.
* Give name to some common shell command invocations.

Tested:
Manually inspected output of:
automation/server/server.py -m ~/local/test_pool.csv -n
for following client runs:
automation/clients/crosstool.py \
-t gcc-4.4.3-glibc-2.11.1-armv7a-vfpv3.d16-hard
automation/clients/nightly.py
automation/clients/test_client.py --chromeos-versions top,weekly
automation/clients/update_client.py

PRESUBMIT=passed
R=asharif,raymes
DELTA=208 (87 added, 82 deleted, 39 changed)
OCL=54023-p2
RCL=54267-p2
RDATE=2011/06/30 14:07:06


P4 change: 42649017
14/automation/clients/helper/crosstool.py
14/automation/clients/helper/jobs.py
14/automation/clients/helper/perforce.py
b417d675755340563d187f184f05586c34990727 15-Feb-2013 raymes <raymes@p4.invalid> Fix GetP4DeleteCommand by appending to the command instead of
overwriting it.

PRESUBMIT=passed
R=asharif
DELTA=1 (0 added, 0 deleted, 1 changed)
OCL=54200-p2
RCL=54215-p2
RDATE=2011/06/28 10:04:20


P4 change: 42648892
14/utils/utils.py
61d6b310e53a5a543ea48cdbe3c579eefeeb70ea 15-Feb-2013 asharif <asharif@p4.invalid> Fixed bug in repo_to_repo.py. It wouldn't delete the client after a
checkouts.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=1 (0 added, 0 deleted, 1 changed)
RCL=54195-p2
RDATE=2011/06/27 17:37:00


P4 change: 42648858
14/repo_to_repo.py
0befe7274cb444026df202b8bed60e849642d98a 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Further automation.clients.helper.jobs refactoring:
* Move all script command creators to ScriptsFactory class.
* Clean up perforce checkout commands by reusing perforce module.

Tested:
Run:
automation/server/server.py -m ~/local/test_pool.csv -n
Manually inspected output of:
automation/clients/test_client.py -c latest -p snapshot
automation/clients/test_client.py -c latest
automation/clients/update_client.py
automation/clients/nightly.py

PRESUBMIT=passed
R=asharif,raymes
DELTA=226 (69 added, 81 deleted, 76 changed)
OCL=54011-p2
RCL=54021-p2
RDATE=2011/06/21 17:26:49


P4 change: 42648368
14/automation/clients/helper/jobs.py
14/automation/clients/helper/perforce.py
259a119bc600901e5d7d476da6fa1f55e13d3ed1 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Further automation.clients.helper.job refactoring (preparing code to be
moved to chromeos module):
* Remove unused GetCopyTreeCommand, _GetMakeChrootCommand functions.
* Move script command creators to separate functions.
* Simplify GetTCRootDir function and rename it to GetToolchainPath.
* Fix _GetP4SnapshotCommand to handle cmd.{Shell,Chain} when creating
a command line.

Tested:
automation/server/server.py -m ~/local/test_pool.csv -n
.
Manually inspected server output with each client:
automation/clients/nightly.py
automation/clients/test_client.py --chromeos-versions top,weekly
automation/clients/update_client.py

PRESUBMIT=passed
R=asharif,raymes
DELTA=136 (18 added, 29 deleted, 89 changed)
OCL=53889-p2
RCL=53989-p2
RDATE=2011/06/20 17:19:06


P4 change: 42648334
14/automation/clients/helper/chromeos.py
14/automation/clients/helper/jobs.py
dee6070928ccf993f196497101916b0e5f8d8b6f 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Move InterceptAndLog function decorator to logger module and rename it
to somewhat more descriptive name: logger.HandleUncaughtExceptions.

Tested:
automation/server/server.py -m ~/local/test_pool.csv -n
automation/clients/crosstool.py \
-t gcc-4.4.3-glibc-2.11.1-armv7a-vfpv3.d16-hard

PRESUBMIT=passed
R=asharif,raymes
DELTA=24 (12 added, 11 deleted, 1 changed)
OCL=53901-p2
RCL=53985-p2
RDATE=2011/06/20 16:09:19


P4 change: 42648332
14/automation/clients/crosstool.py
14/utils/logger.py
5023b0035dff7f187fd5d7738b1c52640f240217 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Crosstool nightly build client.

Tested:
Manually inspected output of:
automation/server/server.py -m ~/local/test_pool.csv -n
for following client runs:
automation/clients/crosstool.py \
-t gcc-4.4.3-glibc-2.11.1-armv7a-vfpv3.d16-hard
automation/clients/crosstool.py -T \
-t gcc-4.4.3-glibc-2.11.1-armv7a-vfpv3.d16-hard
automation/clients/crosstool.py -T \
-t gcc-4.4.3-glibc-2.11.1-armv7a-vfpv3.d16-hard \
-t gcc-4.4.3-glibc-2.11.1-armv7a-vfpv3.d16-softfp

PRESUBMIT=passed
R=asharif,bjanakiraman,raymes
DELTA=236 (236 added, 0 deleted, 0 changed)
OCL=53792-p2
RCL=53898-p2
RDATE=2011/06/17 15:58:54


P4 change: 42648195
14/automation/clients/crosstool.py
14/automation/clients/helper/crosstool.py
976a5aed8637643836e974b70cc870c0877d3462 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Add helpers for handling perforce repositories with "g4" shell command.

Tested:
By not yet checked in cl/53792-p2.

PRESUBMIT=passed
R=asharif,bjanakiraman,raymes
DELTA=151 (151 added, 0 deleted, 0 changed)
OCL=53826-p2
RCL=53864-p2
RDATE=2011/06/16 14:53:20


P4 change: 42647942
14/automation/clients/helper/perforce.py
3313fd7ea81c271ef390cfcdea9896eb8c6506d8 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Shell command handling improvements (required by cl/53792-p2):
* (Shell) Simplified passing options / arguments to shell command.
* (Wrapper) Introduce class that wraps command execution in a
specified environment (working directory, env vars).

Tested:
automation/server/server.py -m ~/local/test_pool.csv -n
automation/clients/nightly.py
automation/clients/test_client.py --chromeos-versions top,weekly
automation/clients/update_client.py

PRESUBMIT=passed
R=asharif,bjanakiraman,raymes
DELTA=74 (26 added, 3 deleted, 45 changed)
OCL=53825-p2
RCL=53863-p2
RDATE=2011/06/16 14:53:01


P4 change: 42647886
14/automation/clients/helper/jobs.py
14/automation/common/command.py
1dc7e2ba57e7fd4f508f2c771222b6fa53321de3 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Declare new job variables for use by commands (JOB_HOME and JOB_TMP).

Tested:
By not yet checked in cl/53792-p2.

PRESUBMIT=passed
R=asharif,bjanakiraman,raymes
DELTA=2 (2 added, 0 deleted, 0 changed)
OCL=53823-p2
RCL=53862-p2
RDATE=2011/06/16 14:10:21


P4 change: 42647885
14/automation/server/job_executer.py
a0bb262834487d06c17411d848de6f25989a5fb5 15-Feb-2013 asharif <asharif@p4.invalid> Fixed a bug in cros_run_benchmarks.py.
Previously if you specified --tests=Page:3,bvt, it would run bvt 3
times as well.
Now it correctly runs bvt once.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=7 (4 added, 1 deleted, 2 changed)
OCL=53832-p2
RCL=53837-p2
RDATE=2011/06/15 17:49:10


P4 change: 42647846
14/cros_run_benchmarks.py
bdbffc229ee58d36de9ce104905180f979e861d1 15-Feb-2013 asharif <asharif@p4.invalid> Added readme file that will be exported to toolchain-utils.git.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=10 (10 added, 0 deleted, 0 changed)
OCL=53797-p2
RCL=53799-p2
RDATE=2011/06/14 14:05:36


P4 change: 42647720
14/README
36b8e46553870459ad7d6457fd3d44d58dafd0e0 15-Feb-2013 asharif <asharif@p4.invalid> Removed a line that I added by mistake in cl/53761-p2.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=2 (0 added, 1 deleted, 1 changed)
OCL=53762-p2
RCL=53794-p2
RDATE=2011/06/14 11:43:19


P4 change: 42647657
14/repo_to_repo.py
cdbacafebbeb58fae101960379a93f1b5ea30286 15-Feb-2013 asharif <asharif@p4.invalid> Added deleting the temporary files after pushing sources for
repo_to_repo.py.

Also added return codes to functions of this script.

PRESUBMIT=passed
R=bjanakiraman,raymes
DELTA=33 (19 added, 0 deleted, 14 changed)
OCL=53756-p2
RCL=53761-p2
RDATE=2011/06/13 18:39:33


P4 change: 42647604
14/repo_to_repo.py
fcc8fe13a198d6b98a31e832bf786df5aa46be6f 15-Feb-2013 asharif <asharif@p4.invalid> Added some dependent files to repo_to_repo.py.
Added binutils-2.20.1_master.rtr for binutils master branch.
Added README.google as default ignore for all repo_to_repo.py transfers.

PRESUBMIT=passed
R=bjanakiraman,kbaclawski,raymes
DELTA=21 (20 added, 0 deleted, 1 changed)
OCL=53592-p2
RCL=53753-p2
RDATE=2011/06/13 14:29:43


P4 change: 42647585
14/repo_to_repo.py
14/repo_to_repo_files/binutils-2.20.1_master.rtr
14/repo_to_repo_files/toolchain-utils.rtr
9ec95ccce3e3f646f57a30703625b71ea2a232be 15-Feb-2013 asharif <asharif@p4.invalid> Re-opened report_generator.py as xtext.
Updated cros_run_benchmarks.py to actually work with the new utils
methods.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=7 (0 added, 4 deleted, 3 changed)
RCL=53752-p2
RDATE=2011/06/13 14:29:35


P4 change: 42647584
14/cros_run_benchmarks.py
14/report_generator.py
fe309b08aa2363add91c7ae775fb108d87b69fb0 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Followup to cl/53604-p2. Forgot to integrate last minute fixes with
previous CL:
* (test_client.py) perflab_benchmarks can be None.
* (update_client.py) make script executable.

Tested:
automation/server/server.py -m ~/local/test_pool.csv -n
automation/clients/nightly.py
automation/clients/test_client.py --chromeos-versions top,weekly
automation/clients/update_client.py

PRESUBMIT=passed
R=asharif,bjanakiraman,raymes
DELTA=6 (2 added, 2 deleted, 2 changed)
OCL=53650-p2
RCL=53666-p2
RDATE=2011/06/09 16:09:57


P4 change: 42647185
14/automation/clients/helper/chromeos.py
14/automation/clients/nightly.py
14/automation/clients/test_client.py
14/automation/clients/update_client.py
09c47e32041b2340a43eb0d46e98f35e23161a17 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> automation.client.helper.jobs module splitting, second step:
* Create automation.clients.helper.chromeos module with JobsFactory
class.
* Extract job creation specific code from
automation.client.helper.jobs module and put it under JobsFactory.
* Remove (from clients) boiler-plate code related to job creation and
integrate it with JobsFactory class.
* Make JobsFactory class configurable.

Tested:
automation/server/server.py -m ~/local/test_pool.csv -n
.
Manually inspected server output with each client:
automation/clients/nightly.py
automation/clients/test_client.py --chromeos-versions top,weekly
automation/clients/update_client.py

PRESUBMIT=passed
R=asharif,bjanakiraman,raymes
DELTA=201 (91 added, 69 deleted, 41 changed)
OCL=53535-p2
RCL=53604-p2
RDATE=2011/06/08 15:47:00


P4 change: 42646939
14/automation/clients/build_tc_client.py
14/automation/clients/helper/chromeos.py
14/automation/clients/helper/jobs.py
14/automation/clients/helper/jobs_test.py
14/automation/clients/nightly.py
14/automation/clients/test_client.py
14/automation/clients/update_client.py
a5d07a9dba0c159e17ad391eb3eca3dd4c4d6915 15-Feb-2013 asharif <asharif@p4.invalid> Fixed a bug in image_chromeos.py related to finding the image.

Fixed issues in cros_run_benchmarks.py related to code cleanup (assert got removed). Made x86-mario the default board.

PRESUBMIT=passed
R=bjanakiraman,raymes
DELTA=11 (4 added, 0 deleted, 7 changed)
RCL=53586-p2
RDATE=2011/06/08 10:43:34


P4 change: 42646901
14/cros_run_benchmarks.py
14/image_chromeos.py
ece37b22d7dc88ef29e5a15759a1228a50353f3c 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> First step to split jobs_helper.py into several modules:
* Move automation.clients.{jobs_helper,jobs_helper_test} to
automation.clients.helper.{jobs,jobs_test}.
* Fix dependencies.

Tested:
automation/server/server.py -m ~/local/test_pool.csv -n
.
Manually inspected server output with each client:
automation/clients/nightly.py
automation/clients/output_test.py
automation/clients/pwd_test.py
automation/clients/status_client.py
automation/clients/test_client.py --chromeos-versions latest
automation/clients/update_client.py
automation/clients/web/index.py

PRESUBMIT=passed
R=asharif,bjanakiraman,raymes
DELTA=918 (436 added, 439 deleted, 43 changed)
OCL=53502-p2
RCL=53525-p2
RDATE=2011/06/07 09:08:08


P4 change: 42646776
14/automation/clients/build_tc_client.py
14/automation/clients/helper/__init__.py
14/automation/clients/helper/jobs.py
14/automation/clients/helper/jobs_test.py
14/automation/clients/jobs_helper.py
14/automation/clients/jobs_helper_test.py
14/automation/clients/nightly.py
14/automation/clients/test_client.py
14/automation/clients/update_client.py
f6345712b53286616ebe36171539aae468ef9adb 15-Feb-2013 asharif <asharif@p4.invalid> Added cros_run_benchmarks.py. This script can be used to run/compare/format
function and performance autotests for multiple ChromeOS images.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=783 (783 added, 0 deleted, 0 changed)
OCL=52571-p2
RCL=53504-p2
RDATE=2011/06/06 17:25:25


P4 change: 42646732
14/cros_run_benchmarks.py
08b1e15276ea6c6a3fe387c882f9539e0382e99c 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Rename jobs_helper.{_CmdChain,_BuildCmd} to {Chain,Shell} and move them
under automation.common.command module:
* _BuildCmd function become command.Shell class.
* _CmdChain is reimplemented under name command.Chain and uses
collections.MutableSequence interface.
* Remove command.{Shell,Chain} explicit string conversion and fix
failing dependencies.
* Change command.{Shell,Chain} invocations to reflect new interface.

Tested:
automation/server/server.py -m ~/local/test_pool.csv -n
.
Manually inspected server output with each client:
automation/clients/nightly.py
automation/clients/output_test.py
automation/clients/pwd_test.py
automation/clients/status_client.py
automation/clients/test_client.py --chromeos-versions latest
automation/clients/update_client.py
automation/clients/web/index.py

PRESUBMIT=passed
R=asharif,bjanakiraman,raymes
DELTA=172 (76 added, 39 deleted, 57 changed)
OCL=53494-p2
RCL=53500-p2
RDATE=2011/06/06 16:25:56


P4 change: 42646730
14/automation/clients/jobs_helper.py
14/automation/common/command.py
14/automation/server/job_executer.py
14/utils/utils.py
a4147f13392870047a5c5a0e99fb4f1a6bc8b4ba 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Jobs Helper module cleanups:
* Create _BuildCmd function to help with shell command line
construction.
* Wrap shell command chains with _CmdChain class.
* Fix majority of glint warnings.

Tested:
automation/server/server.py -m ~/local/test_pool.csv -n
.
Manually inspected server output with each client:
automation/clients/nightly.py
automation/clients/output_test.py
automation/clients/pwd_test.py
automation/clients/status_client.py
automation/clients/test_client.py --chromeos-versions latest
automation/clients/update_client.py
automation/clients/web/index.py

PRESUBMIT=passed
R=asharif,bjanakiraman,raymes
DELTA=397 (126 added, 45 deleted, 226 changed)
OCL=53366-p2
RCL=53492-p2
RDATE=2011/06/06 12:12:56


P4 change: 42646715
14/automation/clients/jobs_helper.py
52284c605cd9ea8d875109b68a3e4bb88f8ac6c9 15-Feb-2013 asharif <asharif@p4.invalid> Fixed an issue with build_tc.py that was preventing it from building
gcc-4.6.0 properly.
If you passed in USE="svn_sources" that would not register because there
was a USE="mounted_sources" earlier in the command line.
You can test this by running:
sudo "VAR=value1 VAR=value2 env" and it will print value1.
Now I am running the command through bash which properly handles env vars.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=25 (12 added, 6 deleted, 7 changed)
OCL=53376-p2
RCL=53416-p2
RDATE=2011/06/03 11:17:09


P4 change: 42646532
14/build_tc.py
14/tc_enter_chroot.py
6999ada69df8163df57e4685c22590236c9ab61f 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Logger class cleanups. Replace utils.Assert* functions with calls to
the default logger.

* Add Logger.Log{Warning,Error}If methods that log a message only iff
a condition is met.
* Add Logger.Log{Fatal,FatalIf} methods that log a message,
a call trace, and finish current thread.
* Convert calls to Assert{Warning,Error} with Log{Warning,Error}If.
* Convert calls to Assert{True,Exit} with LogFatal,LogFatalIf.

Tested:
automation/server/server.py -m ~/local/test_pool.csv -n
automation/clients/nightly.py
automation/clients/output_test.py
automation/clients/pwd_test.py
automation/clients/status_client.py
automation/clients/test_client.py --chromeos-versions latest
automation/clients/update_client.py
automation/clients/web/index.py

PRESUBMIT=passed
R=asharif,bjanakiraman,raymes
APPROVED=raymes
DELTA=200 (47 added, 64 deleted, 89 changed)
OCL=53275-p2
RCL=53307-p2
RDATE=2011/06/01 15:34:19


P4 change: 42646300
14/automation/clients/jobs_helper.py
14/build_benchmarks.py
14/build_chrome_browser.py
14/build_chromeos.py
14/build_tc.py
14/image_chromeos.py
14/repo_to_repo.py
14/run_benchmarks.py
14/run_dejagnu.py
14/tc_enter_chroot.py
14/utils/command_executer.py
14/utils/logger.py
14/utils/utils.py
ec9c624f6a9a82b3848258227ecd88b21673bbb3 15-Feb-2013 asharif <asharif@p4.invalid> Added new script called repo_to_repo.py. It can transfer files from
multiple repos to a single repo.
Currently it supports pushing to git and pulling from svn and p4.
It takes an input file which describes what branches to pull and push.
An example of this input file is p4_to_git.rtr.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=633 (371 added, 262 deleted, 0 changed)
OCL=52943-p2
RCL=53266-p2
RDATE=2011/05/31 15:36:29


P4 change: 42646211
14/repo_to_repo.py
14/repo_to_repo_files/binutils-2.20.1_mobile_toolchain_v14_release_branch.rtr
14/repo_to_repo_files/gcc-4.4.3_master.rtr
14/repo_to_repo_files/gcc-4.4.3_mobile_toolchain_v14_release_branch.rtr
14/repo_to_repo_files/gcc-4.6.0_mobile_toolchain_v15_release_branch.rtr
14/repo_to_repo_files/toolchain-utils.rtr
14/update_git_repo.py
14/utils/utils.py
dd099e55dd66d60f32c663558d3d3154297a53a3 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> JobExecuter class cleanups:
* Create job.JobFailure exception class and use it to signal
failed command execution.
* Renamed JobExecuter.{cmd_executer,command_terminator} to
JobExecuter.{_executer,_terminator}.
* Moved command string creation out of every RunCommand(..) call.
* Splitted JobExecuter.run into smaller task-based classes:
_PrepareJobFolders, _SatisfyFolderDependencies, _LaunchJobCommand,
_CopyJobResults, _GenerateJobReport.

Tested:
automation/server/server.py -m ~/local/test_pool.csv -n
automation/clients/nightly.py
automation/clients/output_test.py
automation/clients/pwd_test.py
automation/clients/status_client.py
automation/clients/test_client.py --chromeos-versions latest
automation/clients/update_client.py
automation/clients/web/index.py

PRESUBMIT=passed
R=asharif,bjanakiraman,raymes
APPROVED=asharif
DELTA=247 (100 added, 95 deleted, 52 changed)
OCL=53201-p2
RCL=53257-p2
RDATE=2011/05/31 14:16:19


P4 change: 42646197
14/automation/common/job.py
14/automation/server/job_executer.py
fc4e4f144c169fd8de4c31bd3afc3493a5897d1e 15-Feb-2013 raymes <raymes@p4.invalid> Fix bug introduced by cleanup, causing nightly failure.

PRESUBMIT=passed
R=bjanakiraman,asharif,kbaclawski
DELTA=1 (0 added, 0 deleted, 1 changed)
RCL=53219-p2
RDATE=2011/05/28 11:57:53


P4 change: 42646146
14/utils/utils.py
1ac31b4bfe421ef673fafccab1093a27c24ceb45 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Further cleanups in automation framework.

* Remove automation/clients/client.py file (upon raymes@ request).
* Merge machine_description.py with machine.py file.
* Cleanup dependencies.
* Cleanup MachineManager class (including glint warnings).

Tested:
automation/server/server.py -m ~/local/test_pool.csv -n
automation/clients/nightly.py
automation/clients/output_test.py
automation/clients/pwd_test.py
automation/clients/status_client.py
automation/clients/test_client.py --chromeos-versions latest
automation/clients/update_client.py
automation/clients/web/index.py

PRESUBMIT=passed
R=asharif,raymes
CC=bjanakiraman
APPROVED=raymes,asharif
DELTA=164 (25 added, 104 deleted, 35 changed)
OCL=53195-p2
RCL=53205-p2
RDATE=2011/05/27 16:24:08


P4 change: 42646137
14/automation/clients/client.py
14/automation/clients/jobs_helper.py
14/automation/clients/output_test.py
14/automation/clients/pwd_test.py
14/automation/common/machine.py
14/automation/common/machine_description.py
14/automation/server/machine_manager.py
14/automation/server/machine_manager_test.py
5fec550c3b12350d201a5a11d5fd19a49bea169d 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Partial cleanup in utils.utils and its dependencies. Add shebang and
Main function to client and server scripts. Minor cleanups in
automation.clients.* Fixed some lint warnings.

Major changes:
* Replace utils.{Serialize,Deserialize} with calls to
pickle.{dumps,loads}.
* Remove utils.ExitWithCode.
* Use "with open(...) as fileobj" to handle file IO.

Tested:
automation/server/server.py -m ~/local/test_pool.csv -n
automation/clients/client.py
automation/clients/nightly.py
automation/clients/output_test.py
automation/clients/pwd_test.py
automation/clients/status_client.py
automation/clients/test_client.py --chromeos-versions latest
automation/clients/update_client.py
automation/clients/web/index.py

PRESUBMIT=passed
R=asharif,bjanakiraman,raymes
APPROVED=raymes
DELTA=567 (172 added, 153 deleted, 242 changed)
OCL=53166-p2
RCL=53194-p2
RDATE=2011/05/27 14:11:18


P4 change: 42646120
14/automation/clients/build_tc_client.py
14/automation/clients/client.py
14/automation/clients/nightly.py
14/automation/clients/output_test.py
14/automation/clients/pwd_test.py
14/automation/clients/status_client.py
14/automation/clients/test_client.py
14/automation/clients/update_client.py
14/automation/clients/web/index.py
14/automation/server/job_executer.py
14/automation/server/server.py
14/build_benchmarks.py
14/run_tests.py
14/utils/utils.py
a0a1172b675159096c988832f67f95e94fdf0a1a 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Cleanup automation.common module.

* Rename Job.required_machines to Job.machine_dependencies.
* Rename Job.required_folders to Job.folder_dependencies.
* Simplify GetTestResultsDirSrc and convert it to
test_results_dir_src property.
* Rename AddRequiredMachine to DependsOnMachine.
* DependsOnMachine does not create MachineSpecification instance, but
accepts one as a parameter.
* Rename MachineDescription class to MachineSpecification to reflect
its intent.
* MachineSpecification.IsMatch uses glob expression to match
specification against Machine instance.
* Minor cleanups in clients.

Tested:
python automation/server/server.py -n -m ~/local/test_pool.csv
python automation/clients/pwd_test.py
python automation/clients/nightly.py
python automation/clients/status_client.py
python automation/clients/test_client.py
python automation/clients/update_client.py

PRESUBMIT=passed
R=asharif,bjanakiraman,raymes
APPROVED=asharif
DELTA=199 (43 added, 88 deleted, 68 changed)
OCL=53143-p2
RCL=53164-p2
RDATE=2011/05/26 11:21:53


P4 change: 42646051
14/automation/clients/client.py
14/automation/clients/jobs_helper.py
14/automation/clients/output_test.py
14/automation/clients/pwd_test.py
14/automation/clients/test_client.py
14/automation/clients/update_client.py
14/automation/common/job.py
14/automation/common/job_group.py
14/automation/common/machine.py
14/automation/common/machine_description.py
14/automation/common/machine_test.py
14/automation/server/job_executer.py
14/automation/server/job_manager.py
14/automation/server/machine_manager.py
14/automation/server/machine_manager_test.py
6e0dcb7f606e25111a112a5926a320694d90046c 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Cleanup automation.common.job module and its dependencies.

* Remove Job.GetRequiredFolders, Job.GetChildren, Job.GetNumChildren,
Job.GetParents, Job.GetNumParents, Job.GetRequiredMachines.
* Rename job.RequiredFolder to job.FolderDependency and simplify its
constructor.
* Rename Job.machine_descriptions to Job.required_machines.
* Simplify Job.status.setter.
* Rename Job.AddRequiredFolder to Job.DependsOnFolder and simplify
its usage by requiring FolderDependancy instance as a parameter.
* Rename Job.AddChild to Job.DependsOn.
* Rename Job.IsReady to Job.is_ready, simplify it and convert to
property.

Tested:
python automation/server/server.py -n -m ~/local/test_pool.csv
python automation/clients/nightly.py

PRESUBMIT=passed
R=asharif,bjanakiraman,raymes
APPROVED=raymes
DELTA=140 (21 added, 53 deleted, 66 changed)
OCL=53128-p2
RCL=53142-p2
RDATE=2011/05/25 15:24:29


P4 change: 42646027
14/automation/clients/build_tc_client.py
14/automation/clients/client.py
14/automation/clients/nightly.py
14/automation/clients/test_client.py
14/automation/clients/web/index.py
14/automation/common/job.py
14/automation/server/job_executer.py
14/automation/server/job_manager.py
ef075498cb4f5288d629dec9a0a9de4880a564b1 15-Feb-2013 kbaclawski <kbaclawski@p4.invalid> Clean up Job & JobGroup classes and their dependencies. All changed
symbols were grepped under automation directory.

Tested:
Locally run server:
python automation/server/server.py -n -m ${HOME}/local/test_pool.csv
And following clients:
python automation/clients/pwd_test.py
python automation/clients/nightly.py

PRESUBMIT=passed
R=raymes,asharif
CC=bjanakiraman
APPROVED=bjanakiraman
DELTA=371 (35 added, 123 deleted, 213 changed)
OCL=53087-p2
RCL=53124-p2
RDATE=2011/05/25 10:43:51


P4 change: 42645863
14/automation/clients/jobs_helper_test.py
14/automation/clients/web/index.py
14/automation/common/job.py
14/automation/common/job_group.py
14/automation/server/job_executer.py
14/automation/server/job_group_manager.py
14/automation/server/job_manager.py
14/automation/server/server.py
92e6310f93f7ba3966e00ca6fb44038ef40da5ee 15-Feb-2013 asharif <asharif@p4.invalid> Added outputting of Chrome binary's checksum so the user can make sure
the reimage really did happen.

PRESUBMIT=passed
R=raymes
DELTA=47 (38 added, 3 deleted, 6 changed)
OCL=52939-p2
RCL=52942-p2
RDATE=2011/05/17 16:58:00


P4 change: 42644443
14/image_chromeos.py
e7c475ee15e81a8b18651a378ab32f93184fe917 15-Feb-2013 asharif <asharif@p4.invalid> image_chromeos.py now checks to see if an image is a test
image before passing it to image_to_live.sh.
This can be overridden by passing in --force.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=38 (38 added, 0 deleted, 0 changed)
RCL=52938-p2
RDATE=2011/05/17 15:01:22


P4 change: 42644434
14/image_chromeos.py
5bca14d7fad1ae2104d0aeb80f89e5c3ec9ab052 15-Feb-2013 asharif <asharif@p4.invalid> build_tc.py was using the wrong version of glibc (r3 instead
of r2). Changed it back to r2.

Changed nightly to delete the job group upon success.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=2 (0 added, 0 deleted, 2 changed)
OCL=52758-p2
RCL=52761-p2
RDATE=2011/05/11 11:51:17


P4 change: 42639964
14/automation/clients/nightly.py
14/build_tc.py
a27288c6ad7e2c4292faeee54b9f6077c51ec3e4 15-Feb-2013 raymes <raymes@p4.invalid> Fix built_tc.py to use --oneshot when installing the toolchain by binary.

PRESUBMIT=passed
R=bjanakiraman,asharif
DELTA=2 (0 added, 1 deleted, 1 changed)
RCL=52639-p2
RDATE=2011/05/06 10:49:21


P4 change: 42639608
14/build_tc.py
b5764214088564929478f6811bc2af40caf28652 15-Feb-2013 asharif <asharif@p4.invalid> This adds the ability to build gcc 4.6.0 using build_tc.py by passing in
--env="GCC_PV=4.6.0".
Additionally, you can build upstream 4.6.0 by specifying:
--env="GCC_PV=4.6.0 USE=upstream" and that will use configure options from
opts.sh and apply them on an upstream tarball obtained from gnu gcc mirror.
I disabled building of libquadmath since it was giving problems with the
current setup. It seems to be only required for Fortran, which we do not
support anyway. See: http://gcc.gnu.org/install/configure.html
This CL will only be useful after this Chromium CL gets submitted:
http://codereview.chromium.org/6880229/

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=9 (6 added, 0 deleted, 3 changed)
OCL=52297-p2
RCL=52569-p2
RDATE=2011/05/04 17:37:10


P4 change: 42639498
14/build-gcc/opts.sh
14/build_tc.py
267f452b6ac8041c56af2a18010d295bb469472b 15-Feb-2013 asharif <asharif@p4.invalid> Added copy of the perflab-output directory to the results dir of the
perflab job.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=11 (1 added, 4 deleted, 6 changed)
OCL=52542-p2
RCL=52568-p2
RDATE=2011/05/04 17:36:15


P4 change: 42639497
14/automation/clients/jobs_helper.py
2539de54a28221a87cc216ff9daf1c2cbef59da4 15-Feb-2013 asharif <asharif@p4.invalid> Updated the git location in update_git_repo.py.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=2 (0 added, 0 deleted, 2 changed)
OCL=52497-p2
RCL=52499-p2
RDATE=2011/05/03 11:48:14


P4 change: 42639209
14/update_git_repo.py
df2fba858a994e3fb08b4a95c125c1830128d26e 15-Feb-2013 raymes <raymes@p4.invalid> Update setup_chromeos to work with gerrit.

PRESUBMIT=passed
R=bjanakiraman,asharif
DELTA=2 (0 added, 0 deleted, 2 changed)
OCL=52483-p2
RCL=52493-p2
RDATE=2011/05/03 11:05:46


P4 change: 42639206
14/setup_chromeos.py
996e036d7ffab0957809eea0d1223e122e642079 15-Feb-2013 asharif <asharif@p4.invalid> Added board to image_chromeos.py args. This should ensure in a clean
nightly.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=4 (1 added, 0 deleted, 3 changed)
OCL=52357-p2
RCL=52359-p2
RDATE=2011/04/28 15:41:37


P4 change: 42638724
14/automation/clients/jobs_helper.py
4a54847ae3838972978c734cefab6ac6a0d65d1a 15-Feb-2013 raymes <raymes@p4.invalid> Update the job_executer to log everything into job specific logs.

PRESUBMIT=passed
R=bjanakiraman,asharif
DELTA=9 (0 added, 3 deleted, 6 changed)
OCL=52311-p2
RCL=52322-p2
RDATE=2011/04/27 13:10:50


P4 change: 42638673
14/automation/server/job_executer.py
dacdb9a691a8fe798eac1c9ca282a195214c8e7a 15-Feb-2013 asharif <asharif@p4.invalid> Added a makedirs line in case the chromeos_root doesn't have the board dir.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=6 (3 added, 0 deleted, 3 changed)
OCL=52201-p2
RCL=52260-p2
RDATE=2011/04/25 10:05:05


P4 change: 42638515
14/image_chromeos.py
c251a305bbb4a49cedae83b7d3a197c8c245f485 15-Feb-2013 raymes <raymes@p4.invalid> Remove minilayout to prevent bug from happening. Some minor fixes to index.py which were causing the wrong results to show up.

PRESUBMIT=passed
R=asharif,bjanakiraman
DELTA=7 (1 added, 1 deleted, 5 changed)
OCL=52222-p2
RCL=52233-p2
RDATE=2011/04/22 16:37:50


P4 change: 42638425
14/automation/clients/jobs_helper.py
14/automation/clients/web/index.py
485a70d5b38956f101c71c87d087bbac6bf0c784 15-Feb-2013 raymes <raymes@p4.invalid> Fixed small bug in setup_chromeos.py (missing space).

PRESUBMIT=passed
R=asharif
DELTA=1 (0 added, 0 deleted, 1 changed)
OCL=52207-p2
RCL=52209-p2
RDATE=2011/04/21 14:52:32


P4 change: 42638314
14/setup_chromeos.py
69c8d72f22a1389fa7553aab5a84bc9d3facc739 15-Feb-2013 raymes <raymes@p4.invalid> Added option to the nightly client to specify which type of board to use for performance tests. Modified setup_chromeos.py so that you can correctly checkout a versioned source tree (the old method with tags stopped working). Also removed a bunch of dead code and updated it to use the logger instead of print. Also added a -j flag to specify the -j paramater for repo sync.

PRESUBMIT=passed
R=bjanakiraman,asharif
DELTA=80 (7 added, 65 deleted, 8 changed)
OCL=52140-p2
RCL=52196-p2
RDATE=2011/04/21 10:05:44


P4 change: 42638307
14/setup_chromeos.py
8ccf74a38832e2481a9d404cd4bf04bb859877ac 15-Feb-2013 raymes <raymes@p4.invalid> Update nightly to use x86-generic instead of agz. Also minor bug fix to server.py.

PRESUBMIT=passed
R=bjanakiraman
DELTA=2 (0 added, 0 deleted, 2 changed)
OCL=51253-p2
RCL=51260-p2
RDATE=2011/04/06 11:01:55


P4 change: 42622818
14/automation/clients/nightly.py
14/automation/server/server.py
f4e7448d34e7376962b579758a4299bcd0329fc3 15-Feb-2013 asharif <asharif@p4.invalid> Fixed path to results.txt since we now pass in workdir to perflab.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=3 (1 added, 0 deleted, 2 changed)
OCL=50819-p2
RCL=50865-p2
RDATE=2011/03/31 12:12:29


P4 change: 42622097
14/automation/clients/jobs_helper.py
e7a379a03c5d69c62f2ad364516ecb67a70b5643 15-Feb-2013 bjanakiraman <bjanakiraman@p4.invalid> Add v8bench.

PRESUBMIT=passed
R=raymes,asharif
DELTA=10 (8 added, 0 deleted, 2 changed)
OCL=50810-p2
RCL=50862-p2
RDATE=2011/03/31 09:01:58


P4 change: 42622093
14/build_benchmarks.py
14/run_benchmarks.py
6e4677b592b8c485731135fd7dc289426dd3f682 15-Feb-2013 raymes <raymes@p4.invalid> Improved reporting in several ways: 1) Added descriptive labels for every job_group and job to uniquely identify them, 2) compare results to baseline on a per-job basis rather than per job_group, 3) improve the dashboard to display the details of an individual job groups on a separate page to the inital list of job groups, 4) improved the report format which compares results to baselines and turned it into a html file rather than a csv, 5) added a results view which tabulates results of different job groups. See: http://raymes/automation/index.py

PRESUBMIT=passed
R=asharif,bjanakiraman
DELTA=496 (218 added, 150 deleted, 128 changed)
OCL=50431-p2
RCL=50800-p2
RDATE=2011/03/30 11:19:11


P4 change: 42621957
14/automation/clients/client.py
14/automation/clients/jobs_helper.py
14/automation/clients/nightly.py
14/automation/clients/output_test.py
14/automation/clients/pwd_test.py
14/automation/clients/test_client.py
14/automation/clients/update_client.py
14/automation/clients/web/index.py
14/automation/common/job.py
14/automation/common/job_group.py
14/automation/server/job_executer.py
14/automation/server/job_group_manager.py
14/report_generator.py
14/utils/html_tools.py
c72bccd755ab5818d70addb0cab4217f7a392ebd 15-Feb-2013 asharif <asharif@p4.invalid> This follows Chris Sosa's recent announcement that run_remote_tests.sh will only
work from inside the chroot.
This change runs the underlying shell script from inside the chroot.
Also removed bvt from the default tests.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=13 (4 added, 0 deleted, 9 changed)
OCL=50742-p2
RCL=50791-p2
RDATE=2011/03/30 10:31:08


P4 change: 42621956
14/run_tests.py
ef75f39c16edd25793970f566765cb9356981f02 15-Feb-2013 asharif <asharif@p4.invalid> Small fix to command_executer.py to use a new known_hosts file everytime you
execute a command on a ChromeOS box.
Fixes to build_benchmarks.py to return the correct returncode.
Added a new parameter to nightly for ARM build/test.
Added perflab workaround to jobs_helper.py.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=43 (28 added, 0 deleted, 15 changed)
OCL=50643-p2
RCL=50648-p2
RDATE=2011/03/28 17:33:05


P4 change: 42621634
14/automation/clients/jobs_helper.py
14/automation/clients/nightly.py
14/build_benchmarks.py
14/utils/command_executer.py
14/utils/utils.py
95925dc72b206a1b4f2393298de4962522954459 15-Feb-2013 asharif <asharif@p4.invalid> Fixed parentheses typos in run_dejagnu.py.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=7 (1 added, 0 deleted, 6 changed)
OCL=50508-p2
RCL=50510-p2
RDATE=2011/03/25 15:13:20


P4 change: 42621344
14/run_dejagnu.py
c20ed54e20166348f4233e8914dadce21793edca 15-Feb-2013 asharif <asharif@p4.invalid> Fixed bugs in run_tests.py.
Earlier it was passing in 'run_tests.py' to run_remote_tests.sh.

PRESUBMIT=passed
R=raymes
DELTA=11 (9 added, 0 deleted, 2 changed)
OCL=50505-p2
RCL=50509-p2
RDATE=2011/03/25 14:34:11


P4 change: 42621343
14/run_tests.py
ad40120246941520cdeae442de26f274bd8ceba6 15-Feb-2013 asharif <asharif@p4.invalid> Changed ImageChromeOS to Main.

PRESUBMIT=passed
R=raymes
DELTA=1 (0 added, 0 deleted, 1 changed)
RCL=50253-p2
RDATE=2011/03/21 18:20:13


P4 change: 42619267
14/image_chromeos.py
2198c517f514e24bdcd3be8afa948bd914c8e15f 15-Feb-2013 asharif <asharif@p4.invalid> Added return values to the scripts.
Fixed build_benchmarks to return the correct values back to perflab.

PRESUBMIT=passed
R=raymes,bjanakiraman
CC=c-compiler-reviews
DELTA=60 (24 added, 2 deleted, 34 changed)
OCL=50177-p2
RCL=50246-p2
RDATE=2011/03/21 11:35:53


P4 change: 42619261
14/build_benchmarks.py
14/build_chrome_browser.py
14/build_chromeos.py
14/build_tc.py
14/compare_benchmarks.py
14/image_chromeos.py
14/run_benchmarks.py
14/run_dejagnu.py
14/run_tests.py
14/setup_chromeos.py
14/summarize_results.py
14/tc_enter_chroot.py
14/update_git_repo.py
642509c7b7af25a36e0dccdb9bb605ae0b3fa4e7 15-Feb-2013 asharif <asharif@p4.invalid> Fixed 2 more bugs related to build_tc.py.
sudo wasn't being used when uninstalling the toolchain.
tc_enter_chroot.py wasn't respecting the -s option.

Tested:
Tested the -B and -C options.

PRESUBMIT=passed
TBR=raymes
DELTA=2 (0 added, 0 deleted, 2 changed)
RCL=50142-p2
RDATE=2011/03/17 18:58:02


P4 change: 42618976
14/build_tc.py
14/tc_enter_chroot.py
01410cc5cd22493b6004db397f756b36f0bcd2b8 15-Feb-2013 asharif <asharif@p4.invalid> Fixed a bug in build_tc.py exposed by having an optional
--toolchain_root argument in tc_enter_chroot.py CL 50076-p2.
The output directory was not being mounted in certain cases.

Tested:
perforce2//gcctools/chromeos/v14//build_tc.py --toolchain_root=perforce2/branches/mobile_toolchain_v14_release_branch/gcctools/ --chromeos_root=chromeos --board=x86-agz

PRESUBMIT=passed
TBR=raymes
DELTA=3 (2 added, 0 deleted, 1 changed)
RCL=50140-p2
RDATE=2011/03/17 16:59:19


P4 change: 42618970
14/tc_enter_chroot.py
23149a218802b9063b2f4924de30cc9e39f94f0c 15-Feb-2013 raymes <raymes@p4.invalid> Improved reporting dashboard so that logs are not transferred through XMLRPC. Now logs are only stored in the home directory of mobiletc-prebuild. The dashboard will read the logs from that location. The impact is that the apache server must run under mobiletc-prebuild which is not a major issue.

PRESUBMIT=passed
R=asharif,bjanakiraman
DELTA=113 (63 added, 20 deleted, 30 changed)
OCL=49669-p2
RCL=50077-p2
RDATE=2011/03/16 15:04:58


P4 change: 42618813
14/automation/clients/client.py
14/automation/clients/web/index.py
14/automation/common/job.py
14/automation/server/job_executer.py
14/automation/server/job_manager.py
14/automation/server/server.py
14/utils/logger.py
8697d4eba706d72da800d567de92911924bf3d04 15-Feb-2013 asharif <asharif@p4.invalid> Changed ExecuteCommandInChroot to take in extra parameters for additional
mount points.

Changed the order of parameters to the same function.

Changed build_tc.py to pass --oneshot all the way down to portage.

Changed tc_enter_chroot.py to take in toolchain_root as an optional parameter.
By default it only mounts the version_dir, which is where it resides.

Changed uses of ExecuteCommandInChroot to the new order.

Changed build_benchmarks.py to not take in toolchain_root. It now takes in
--third_party which is the third_party dir where CPU benchmarks reside.

Changed tc_enter_chroot.py to take in an optional --third_party parameter.
By default it mounts __file__/../../../third_party.

Changed build_chrome_browser.py to add a flags.txt file in the image dir.

Fixed a bug in tc_enter_chroot.py (sys.argv[0] was used instead of __file__).

Fixed handling of command line parameters of build_chrome_browser.py.

build_benchmarks.py, build_chrome_browser.py, run_tests.py and run_dejagnu.py
now use argv instead of sys.argv.

build_chromeos.py now adds a file called flags.txt in the build dir of
the image containing the cxxflags, cflags and ldflags used during the build.

build_chromeos.py now takes an additional optional argument (label) that can be
used to label dirs containing the image.

PRESUBMIT=passed
R=bjanakiraman,raymes
DELTA=191 (104 added, 16 deleted, 71 changed)
OCL=49844-p2
RCL=50076-p2
RDATE=2011/03/16 14:53:04


P4 change: 42618812
14/automation/clients/jobs_helper.py
14/build_benchmarks.py
14/build_chrome_browser.py
14/build_chromeos.py
14/build_tc.py
14/image_chromeos.py
14/run_benchmarks.py
14/run_dejagnu.py
14/run_tests.py
14/tc_enter_chroot.py
edff23f0a22a511d9ede0cd750e5d17e9ff54df0 15-Feb-2013 asharif <asharif@p4.invalid> Fixed bug in image_chromeos.py that was preventing temporary images from
being deleted after they were copied.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=1 (0 added, 0 deleted, 1 changed)
OCL=49797-p2
RCL=49830-p2
RDATE=2011/03/11 11:57:18


P4 change: 42618404
14/image_chromeos.py
c380f6157835ce3ed1ac79a68ba936fa53ae84b2 15-Feb-2013 asharif <asharif@p4.invalid> Added expanduser to image_chromeos.py.
Changed build_chromeos.py to build if the board exists.
Changed run_dejagnu.py to mkdir without sudo so the cp after that doesn't fail.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=3 (1 added, 0 deleted, 2 changed)
OCL=49678-p2
RCL=49773-p2
RDATE=2011/03/10 10:43:52


P4 change: 42618285
14/build_chromeos.py
14/image_chromeos.py
14/run_dejagnu.py
758f12ca925ccacfb2e9fa59e919a2d5018c0534 15-Feb-2013 asharif <asharif@p4.invalid> Added ssh -t option to dejaGNU so remote commands would terminate properly.
Added process termination in command_executer.py so it would terminate
defunct processes.
Added option to terminate processes within a certain time.
Removed the -j<numprocs+1> flag from repo sync -- it was causing failures.

PRESUBMIT=passed
R=bjanakiraman,raymes
DELTA=76 (68 added, 0 deleted, 8 changed)
OCL=49591-p2
RCL=49641-p2
RDATE=2011/03/08 12:19:45


P4 change: 42618059
14/command_executer_timeout_test.py
14/dejagnu/boards/chromeos-machine.exp
14/run_dejagnu.py
14/setup_chromeos.py
14/utils/command_executer.py
a082e56051d6644ae686933624c30357ad20a9a5 15-Feb-2013 bjanakiraman <bjanakiraman@p4.invalid> Change build_benchmarks to use --rebuild
Initial version of comparing benchmarks runs looking at the results.txt files.

PRESUBMIT=passed
R=raymes,asharif
DELTA=168 (168 added, 0 deleted, 0 changed)
OCL=49427-p2
RCL=49479-p2
RDATE=2011/03/05 22:02:50


P4 change: 42617806
14/compare_benchmarks.py
c89b8575c5897a60f8f793859d2811d8139602fa 15-Feb-2013 raymes <raymes@p4.invalid> Modified run_dejagnu.py: 1) Moved ssh logic out of remote_init and into run_dejagnu.py 2) Fixed bug which was causing ssh to fail (missing -i param) 3) Made it work with vanilla toolchains, as well as mounted ones 4) Made it work with all boards (instead of just x86)

PRESUBMIT=passed
R=asharif,bjanakiraman
DELTA=98 (44 added, 39 deleted, 15 changed)
OCL=49425-p2
RCL=49429-p2
RDATE=2011/03/04 15:20:07


P4 change: 42617501
14/dejagnu/boards/chromeos-machine.exp
14/dejagnu/remote_init.sh
14/dejagnu/site.exp
14/run_dejagnu.py
069796a7ffa46e548aca5faf6655293da670d290 15-Feb-2013 asharif <asharif@p4.invalid> Fixed typo in opts.sh.

PRESUBMIT=passed
R=raymes
DELTA=1 (0 added, 0 deleted, 1 changed)
RCL=49324-p2
RDATE=2011/03/02 19:04:24


P4 change: 42617299
14/build-gcc/opts.sh
545ade9d10c6070308487b8d3009d1cc5eb5d75c 15-Feb-2013 asharif <asharif@p4.invalid> Added --enable-__cxa_atexit to the list of common options.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=1 (1 added, 0 deleted, 0 changed)
OCL=49303-p2
RCL=49307-p2
RDATE=2011/03/02 14:52:24


P4 change: 42617285
14/build-gcc/opts.sh
a44c334f76097e3b84363e37a0a1a76e63e59840 15-Feb-2013 asharif <asharif@p4.invalid> Added missing () to the code.
Removed some assertions from the code that were triggering.

PRESUBMIT=passed
R=raymes
DELTA=13 (6 added, 0 deleted, 7 changed)
OCL=49298-p2
RCL=49302-p2
RDATE=2011/03/02 13:34:49


P4 change: 42617272
14/automation/common/job.py
14/automation/server/server.py
cd9e629578b145bef699de4a7f64d1e1a0b48314 15-Feb-2013 asharif <asharif@p4.invalid> Added enable-checking=release for both Atom/ARM arches.
Added disable-esp configure option for ARM.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=22 (3 added, 1 deleted, 18 changed)
OCL=49297-p2
RCL=49301-p2
RDATE=2011/03/02 13:34:39


P4 change: 42617271
14/build-gcc/opts.sh
911bd276924045a184d08a8ec297a281e563a01b 15-Feb-2013 asharif <asharif@p4.invalid> Made setup_chromeos not checkout the latest branch. It simply
checks out the latest tag now.
Also made it use -j<n> where n = #procs + 1. It used to slow my machine
a lot.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=12 (1 added, 6 deleted, 5 changed)
OCL=49260-p2
RCL=49265-p2
RDATE=2011/03/01 18:32:37


P4 change: 42617208
14/setup_chromeos.py
b1e8a3e40e5f1e9e778d58a92c8036bbf6ee1425 15-Feb-2013 asharif <asharif@p4.invalid> Updated the location of the gcc-9999 dir as crossdev was updated.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=6 (0 added, 2 deleted, 4 changed)
OCL=49236-p2
RCL=49241-p2
RDATE=2011/03/01 15:28:27


P4 change: 42617180
14/run_dejagnu.py
415e66368b3c7c986f18c9807ef7af9cb14bff3b 15-Feb-2013 asharif <asharif@p4.invalid> Added output directory creation to build_tc.py.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=6 (6 added, 0 deleted, 0 changed)
RCL=49148-p2
RDATE=2011/02/28 14:53:31


P4 change: 42617036
14/build_tc.py
c972f789e2ae4cac2cdb7a78b373be568d1ad525 15-Feb-2013 asharif <asharif@p4.invalid> Added code to catch IOError and return empty string in that case.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=19 (4 added, 0 deleted, 15 changed)
RCL=49142-p2
RDATE=2011/02/28 14:06:40


P4 change: 42617032
14/automation/server/server.py
3b0fca0c92a8ddae9b69326e18dc47c6483be504 15-Feb-2013 asharif <asharif@p4.invalid> Added chromeos private key to the ssh command line.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=2 (1 added, 0 deleted, 1 changed)
OCL=49137-p2
RCL=49139-p2
RDATE=2011/02/28 13:43:28


P4 change: 42617018
14/utils/command_executer.py
82c6f97e09429d0afc5efdc5982184e565cb0881 15-Feb-2013 asharif <asharif@p4.invalid> s/GetReportDest/GetTestReport/g.
This was causing problems in the web interface.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=2 (0 added, 0 deleted, 2 changed)
OCL=49132-p2
RCL=49134-p2
RDATE=2011/02/28 11:19:12


P4 change: 42617016
14/automation/server/server.py
6f4065ca776db71b07081345cf7183dda0fefd66 15-Feb-2013 asharif <asharif@p4.invalid> Fixed remote_test.py to use the machine from options.remote in testing
the board discovery code.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=1 (0 added, 0 deleted, 1 changed)
OCL=49072-p2
RCL=49133-p2
RDATE=2011/02/28 11:17:27


P4 change: 42617015
14/remote_test.py
90a516361bab857c21fc6151aaa93afe394eb96a 15-Feb-2013 asharif <asharif@p4.invalid> Added sleep code to the job_manager loop so it wouldn't consume 100%
CPU when trying to run a job that requires a locked machine.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=1 (1 added, 0 deleted, 0 changed)
OCL=48901-p2
RCL=48979-p2
RDATE=2011/02/24 21:55:47


P4 change: 42616450
14/automation/server/job_manager.py
c43afbbecdd240b9a7ac1ff06fd685e049674f16 15-Feb-2013 asharif <asharif@p4.invalid> Added a check in job_group_manager.py to prevent a thread crash.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=2 (2 added, 0 deleted, 0 changed)
OCL=48969-p2
RCL=48978-p2
RDATE=2011/02/24 19:38:57


P4 change: 42616449
14/automation/server/job_group_manager.py
60d7b4304ffdf2fc4c31669224ba935637beacdb 15-Feb-2013 asharif <asharif@p4.invalid> Added perflab summarizer. It seems to be working.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=36 (30 added, 0 deleted, 6 changed)
OCL=48966-p2
RCL=48970-p2
RDATE=2011/02/24 16:42:47


P4 change: 42616439
14/automation/clients/jobs_helper.py
14/summarize_results.py
d7060d3df40bf3f2c983395ce59195c785d387ae 15-Feb-2013 asharif <asharif@p4.invalid> Fixed the issue that was causing the OOM problems.
We were appending to the list without checking if the machine name was
already present in it.

PRESUBMIT=passed
BUG=3452072
R=raymes,bjanakiraman
DELTA=2 (1 added, 0 deleted, 1 changed)
OCL=48894-p2
RCL=48967-p2
RDATE=2011/02/24 15:28:11


P4 change: 42616436
14/automation/common/machine_description.py
c51baea421d859933b342acd3bbe43b41484c68e 15-Feb-2013 asharif <asharif@p4.invalid> Removed tc-hardfloat check in opts.sh. The default is not setting
hard float on ARM. We can add it back later.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=6 (0 added, 6 deleted, 0 changed)
OCL=48684-p2
RCL=48837-p2
RDATE=2011/02/22 11:59:07


P4 change: 42612898
14/build-gcc/opts.sh
ee7d0b92f9bc6796e59de18b9301dd8c82d963c8 15-Feb-2013 raymes <raymes@p4.invalid> Altered reporting to occur inside the job rather than in the server. The server copies logs to the home directory and compares them to a baseline.

PRESUBMIT=passed
R=asharif,bjanakiraman
DELTA=76 (9 added, 20 deleted, 47 changed)
OCL=48100-p2
RCL=48815-p2
RDATE=2011/02/22 10:17:01


P4 change: 42612881
14/automation/clients/client.py
14/automation/common/job.py
14/automation/common/job_group.py
14/automation/server/job_executer.py
14/automation/server/job_group_manager.py
71066718a77f43d7e14d1dd4c52fb02ecf7a7d53 15-Feb-2013 raymes <raymes@p4.invalid> Removed error handling lines from logger. This *may* be masking errors that cause the missing output in our logs.

PRESUBMIT=passed
R=asharif,bjanakiraman
DELTA=14 (0 added, 8 deleted, 6 changed)
OCL=48098-p2
RCL=48814-p2
RDATE=2011/02/22 10:16:44


P4 change: 42612880
14/utils/logger.py
fc33cd9b474e9defdda4d7b29866ed3520cdc8c8 15-Feb-2013 asharif <asharif@p4.invalid> Added the ability to use any arbitrary image for imaging ChromeOS.
Moved jobs_helper.py to automation/clients from automation/common.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=791 (415 added, 365 deleted, 11 changed)
OCL=48666-p2
RCL=48686-p2
RDATE=2011/02/18 14:15:07


P4 change: 42612583
14/automation/clients/build_tc_client.py
14/automation/clients/jobs_helper.py
14/automation/clients/jobs_helper_test.py
14/automation/clients/nightly.py
14/automation/common/jobs_helper.py
14/automation/common/jobs_helper_test.py
14/image_chromeos.py
1c19a7f1dd83fd6bd98bb88c8d3fc5624a2d669e 15-Feb-2013 bjanakiraman <bjanakiraman@p4.invalid> Fix problem introduced by earlier change in commoning setting up of benchname

PRESUBMIT=passed
R=asharif,raymes
DELTA=4 (3 added, 1 deleted, 0 changed)
OCL=48656-p2
RCL=48664-p2
RDATE=2011/02/17 17:01:25


P4 change: 42612571
14/run_benchmarks.py
80b47dc5479abb1efe00aceb142ce7ea0ef9c6d9 15-Feb-2013 asharif <asharif@p4.invalid> Added --rebuild flag to build_chromeos.py, which rebuilds packages.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=11 (10 added, 0 deleted, 1 changed)
OCL=48661-p2
RCL=48663-p2
RDATE=2011/02/17 17:01:17


P4 change: 42612570
14/build_chromeos.py
7cbe069d7939cca423757d55cddf75465b64f597 15-Feb-2013 asharif <asharif@p4.invalid> Fixed bug in image_chromeos -- it was only updating the checksum without
doing the re-image earlier.
Added exclusive lock for build jobs. When running together on the same linux
machine, they bring it to its knees.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=28 (10 added, 4 deleted, 14 changed)
OCL=48104-p2
RCL=48246-p2
RDATE=2011/02/15 11:10:09


P4 change: 42611251
14/automation/common/jobs_helper.py
14/image_chromeos.py
6eece82c96fa7f66631e050731c30900f4a3855d 15-Feb-2013 raymes <raymes@p4.invalid> Updated the results summarizer to be run as a part of the job and place all output in a results directory.

PRESUBMIT=passed
R=asharif,bjanakiraman
DELTA=45 (20 added, 11 deleted, 14 changed)
OCL=48065-p2
RCL=48073-p2
RDATE=2011/02/11 11:54:21


P4 change: 42610386
14/automation/common/jobs_helper.py
14/summarize_results.py
68a560d0e3ecc155a3b16fda870d4373d6d6dde2 15-Feb-2013 raymes <raymes@p4.invalid> Removed call to get_svn_repos which no longer exists.

PRESUBMIT=passed
R=asharif,bjanakiraman
DELTA=5 (0 added, 5 deleted, 0 changed)
RCL=48051-p2
RDATE=2011/02/10 16:08:04


P4 change: 42610345
14/setup_chromeos.py
e9433c3fae9a2e0954dd42c249a0a027e46ecf92 15-Feb-2013 asharif <asharif@p4.invalid> Added the ability to use a tarball of a chromeos checkout in jobs_helper.py.
The web interface now shows formatted commands instead of raw ones.
Enhanced the timeline to show the time elapsed since the last status update if
the job is still running.
Added mobile release branch to nightly and made that the default.
Changed update_git_repo.py's commit message to use the last CL in the client.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=51 (40 added, 2 deleted, 9 changed)
OCL=47973-p2
RCL=48038-p2
RDATE=2011/02/10 11:21:02


P4 change: 42610341
14/automation/clients/nightly.py
14/automation/clients/web/index.py
14/automation/common/job.py
14/automation/common/jobs_helper.py
14/update_git_repo.py
7df22f028387af8ef27f3888795b9eb5a70797d6 15-Feb-2013 raymes <raymes@p4.invalid> Fixed bug in command_executer which was cause dry_run not to always happen when -n was specified in the server.

PRESUBMIT=passed
R=asharif,bjanakiraman
DELTA=8 (1 added, 2 deleted, 5 changed)
OCL=47927-p2
RCL=47929-p2
RDATE=2011/02/07 13:54:45


P4 change: 42610188
14/automation/clients/client.py
14/utils/command_executer.py
beb0fddb3f9bef73a2a42d982048dd42fea618b3 15-Feb-2013 asharif <asharif@p4.invalid> Deleted 2 unused shell scripts.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=125 (0 added, 125 deleted, 0 changed)
RCL=47809-p2
RDATE=2011/02/03 15:31:22


P4 change: 42610082
14/automation/server/sync_and_restart_server.sh
14/tc-enter-chroot.sh
8fbf10d97344610fc5c5c25a3b8c5f0a53d94e61 15-Feb-2013 raymes <raymes@p4.invalid> Allowed dry-run to be specified for each job.

PRESUBMIT=passed
R=asharif,bjanakiraman
DELTA=39 (18 added, 7 deleted, 14 changed)
OCL=47794-p2
RCL=47804-p2
RDATE=2011/02/03 14:22:58


P4 change: 42610075
14/automation/clients/client.py
14/automation/common/job.py
14/automation/common/jobs_helper.py
14/automation/server/job_executer.py
14/automation/server/server.py
14/utils/command_executer.py
f64f88d345463c802077b5e03b4776fa87ebd431 15-Feb-2013 asharif <asharif@p4.invalid> Added chrotomation to the test machine pool.
Fixed bug in jobs_helper where it was wrongly using the branch
to identify the version (as opposed to commit).

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=2 (0 added, 0 deleted, 2 changed)
OCL=47709-p2
RCL=47733-p2
RDATE=2011/02/02 12:59:56


P4 change: 42609835
14/automation/common/jobs_helper.py
14/automation/server/test_pool.csv
dcb0634a2cff622a312cca6cc33a94244ec9cd65 15-Feb-2013 raymes <raymes@p4.invalid> Added feature to view test results in the test dashboard.

PRESUBMIT=passed
R=asharif,bjanakiraman
DELTA=80 (49 added, 9 deleted, 22 changed)
OCL=46900-p2
RCL=46951-p2
RDATE=2011/01/27 13:55:26


P4 change: 42609610
14/automation/clients/client.py
14/automation/clients/web/index.py
14/automation/server/job_group_manager.py
14/automation/server/server.py
14/report_generator.py
3c6b93b2a8f8939a8221d4f4cc944292b0509c9b 15-Feb-2013 asharif <asharif@p4.invalid> Added output_test.py, a client that produces a lot of output.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=53 (53 added, 0 deleted, 0 changed)
OCL=46708-p2
RCL=46712-p2
RDATE=2011/01/21 15:37:40


P4 change: 42608902
14/automation/clients/output_test.py
14/produce_output.py
a10d98a05fe67d875687aaea7ecf2a7d6544ec3a 15-Feb-2013 asharif <asharif@p4.invalid> Added lab test machines to the test_pool.csv file.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=2 (1 added, 0 deleted, 1 changed)
OCL=46698-p2
RCL=46707-p2
RDATE=2011/01/21 15:20:50


P4 change: 42608900
14/automation/server/test_pool.csv
b7b6a40e5fa95e32e6e9cb76c33932011c474d33 15-Feb-2013 raymes <raymes@p4.invalid> Minor fixes to the result summarizer.

PRESUBMIT=passed
R=asharif,bjanakiraman
DELTA=5 (4 added, 0 deleted, 1 changed)
OCL=46679-p2
RCL=46692-p2
RDATE=2011/01/21 09:53:47


P4 change: 42608880
14/summarize_results.py
b619ea5defe5481a1fc41011280e3eaee01e7e2e 15-Feb-2013 raymes <raymes@p4.invalid> Fixed dejagnu config. Also added debug output to the dejagnu job.

PRESUBMIT=passed
R=asharif,bjanakiraman
DELTA=7 (1 added, 0 deleted, 6 changed)
RCL=46613-p2
RDATE=2011/01/19 17:46:29


P4 change: 42608818
14/automation/common/jobs_helper.py
14/dejagnu/boards/chromeos-machine.exp
85ef5db6d7b361899d983a8a62ff0f40a9118c93 15-Feb-2013 raymes <raymes@p4.invalid> Added summarisation of results and generation of a report from results. Also miscellaneous fixes (command_executer, logger). Integration with the front end is not yet complete.

PRESUBMIT=passed
R=asharif,bjanakiraman
DELTA=321 (243 added, 15 deleted, 63 changed)
OCL=46555-p2
RCL=46573-p2
RDATE=2011/01/18 16:07:38


P4 change: 42608767
14/automation/clients/client.py
14/automation/clients/web/index.py
14/automation/common/job.py
14/automation/common/job_group.py
14/automation/common/jobs_helper.py
14/automation/server/job_executer.py
14/automation/server/job_group_manager.py
14/build_tc.py
14/remote_kill_test.py
14/report_generator.py
14/summarize_results.py
14/utils/command_executer.py
14/utils/logger.py
e0cc3058b58537013fa03c5fbea1ebe3f78a7fd8 15-Feb-2013 asharif <asharif@p4.invalid> Unified chromeos build options into utils for the low-level scripts.
I am going to test this now with the nightly.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=71 (54 added, 6 deleted, 11 changed)
OCL=46485-p2
RCL=46564-p2
RDATE=2011/01/18 14:11:10


P4 change: 42608761
14/build_chrome_browser.py
14/build_chromeos.py
14/utils/utils.py
2f1af985d8844d8c3cd135a5b75a6472d3b34c21 15-Feb-2013 asharif <asharif@p4.invalid> Added a simple test for printing the working directory.
The update_job uses the pwd to track the current branch in git.
This test makes sure changes to command_executer will not break
the update_job.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=18 (18 added, 0 deleted, 0 changed)
OCL=46557-p2
RCL=46560-p2
RDATE=2011/01/18 13:23:23


P4 change: 42608759
14/automation/clients/pwd_test.py
27574ee9a65c290b560ab3617f0962d125135dad 15-Feb-2013 raymes <raymes@p4.invalid> Altered run_dejagnu so that the log files are copied to a fixed path, regardless of architecture.

PRESUBMIT=passed
R=asharif,bjanakiraman
DELTA=16 (16 added, 0 deleted, 0 changed)
RCL=46510-p2
RDATE=2011/01/15 14:54:21


P4 change: 42608699
14/run_dejagnu.py
89ab76b7611d7f97d40717e326bd17e08e43850b 15-Feb-2013 asharif <asharif@p4.invalid> Added a command-line status client to query job groups and job statuses.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=42 (42 added, 0 deleted, 0 changed)
OCL=46478-p2
RCL=46482-p2
RDATE=2011/01/14 14:01:29


P4 change: 42608597
14/automation/clients/status_client.py
14/automation/common/job_group.py
b572fcab8d8653804b4bdde27b0da545e07b9246 15-Feb-2013 asharif <asharif@p4.invalid> Changed single quotes to double quotes since the ssh command doesn't
handle single quotes properly.
I'll write a CL to handle both double quotes and single quotes later.

PRESUBMIT=passed
R=raymes
DELTA=20 (4 added, 5 deleted, 11 changed)
OCL=46457-p2
RCL=46477-p2
RDATE=2011/01/14 11:57:23


P4 change: 42608592
14/automation/common/jobs_helper.py
14/build_chromeos.py
14/utils/command_executer.py
f12f79d7f770a514ac3e46d8741aef64eda0bc67 15-Feb-2013 raymes <raymes@p4.invalid> Added results summarizer.

PRESUBMIT=passed
R=asharif,bjanakiraman
DELTA=78 (78 added, 0 deleted, 0 changed)
OCL=46472-p2
RCL=46476-p2
RDATE=2011/01/14 11:23:52


P4 change: 42608591
14/summarize_results.py
32ed84ec20b2130acc517eb0e28c4f0d1ad0dfb2 15-Feb-2013 asharif <asharif@p4.invalid> Removed toolchain_root from build_chrome_browser.py to make it similar to
build_chromeos.py.
Also added default empty strings for cflags, cxxflags and ldflags because
otherwise chrome browser would not compile.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=20 (3 added, 14 deleted, 3 changed)
OCL=46454-p2
RCL=46456-p2
RDATE=2011/01/13 14:06:42


P4 change: 42608565
14/build_chrome_browser.py
3a7f5a97cfd73efe61f4059b3d9081a3f5cecb1b 15-Feb-2013 asharif <asharif@p4.invalid> Previously, each board had a (possibly) different weekly build number. This
is now changed to a more simpler model: there is a single golden weekly or
quarterly version number.
Testing:
I tested this in dry run mode. There is a job currently running that will
update the weekly and quarterly versions.

PRESUBMIT=passed
R=raymes
DELTA=46 (7 added, 12 deleted, 27 changed)
OCL=46451-p2
RCL=46455-p2
RDATE=2011/01/13 14:06:34


P4 change: 42608564
14/automation/clients/test_client.py
14/automation/clients/update_client.py
14/automation/common/jobs_helper.py
2887584f84fe02657aad77d2a0b0d194d1e410b8 15-Feb-2013 asharif <asharif@p4.invalid> Added expanduser for toolchain_root commandline option.

PRESUBMIT=passed
R=raymes
DELTA=2 (2 added, 0 deleted, 0 changed)
OCL=46430-p2
RCL=46434-p2
RDATE=2011/01/12 16:16:16


P4 change: 42608537
14/build_tc.py
2dfbf516f338b5339086cbde8199a60c45210703 15-Feb-2013 asharif <asharif@p4.invalid> When installing from binary packages, build_tc.py no longer deletes
the toolchain work directories.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=2 (1 added, 0 deleted, 1 changed)
RCL=46415-p2
RDATE=2011/01/12 11:50:26


P4 change: 42608517
14/build_tc.py
1e761da7bec4870ce8b5a35c11bfc95e2459e131 15-Feb-2013 asharif <asharif@p4.invalid> Changed double quotes to single quotes.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=1 (0 added, 0 deleted, 1 changed)
OCL=46395-p2
RCL=46414-p2
RDATE=2011/01/12 11:06:25


P4 change: 42608516
14/utils/command_executer.py
3af6d1173a623662d256a696216337d6646759b5 15-Feb-2013 asharif <asharif@p4.invalid> Modified command executer so that killing local jobs would
kill remote jobs.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=42 (41 added, 0 deleted, 1 changed)
OCL=46388-p2
RCL=46390-p2
RDATE=2011/01/11 18:11:29


P4 change: 42608491
14/remote_kill_test.py
14/utils/command_executer.py
67966879001c05761e618178d49118b61143f152 15-Feb-2013 asharif <asharif@p4.invalid> Updated nightly.py. It has the nightly jobs that we talked about in the
meeting. test_client.py is still there and is generic for general testing.
Updated jobs_helper.py to not include the pagecycler test for functional
correctness check.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=81 (62 added, 2 deleted, 17 changed)
OCL=46356-p2
RCL=46375-p2
RDATE=2011/01/11 13:22:15


P4 change: 42608486
14/automation/clients/nightly.py
14/automation/clients/test_client.py
14/automation/common/jobs_helper.py
14/image_chromeos.py
05d1e5c973bd993766b458e6052c3674499d8b24 15-Feb-2013 asharif <asharif@p4.invalid> Updated build_benchmarks.py to not rely on current working dir.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=9 (6 added, 0 deleted, 3 changed)
OCL=46351-p2
RCL=46354-p2
RDATE=2011/01/10 16:09:58


P4 change: 42608451
14/automation/common/jobs_helper.py
14/build_benchmarks.py
a55807fbc5c7ba7739dc81810e324e13d9a3da49 15-Feb-2013 asharif <asharif@p4.invalid> Fixed a bug in tc_enter_chroot.py and run_dejagnu.py.

PRESUBMIT=passed
R=bjanakiraman,raymes
DELTA=2 (0 added, 0 deleted, 2 changed)
OCL=46345-p2
RCL=46353-p2
RDATE=2011/01/10 16:09:24


P4 change: 42608439
14/run_dejagnu.py
14/tc_enter_chroot.py
29775b2bf9e8981f584b9f2b374bd27f98c1909d 15-Feb-2013 asharif <asharif@p4.invalid> Made copying files to and from chromeos boxes consistent with
non-chromeos box copy. Edited run_benchmarks to use the new
convention.
For files and directories, both source and dest refer to the
directories being copied, not parent directories.

Added more tests for copying files to/from the chromeos machine.

Added missing commands in the dejaGNU job.

PRESUBMIT=passed
R=bjanakiraman,raymes
DELTA=97 (67 added, 10 deleted, 20 changed)
OCL=46314-p2
RCL=46342-p2
RDATE=2011/01/10 13:24:22


P4 change: 42608435
14/automation/common/jobs_helper.py
14/remote_test.py
14/run_benchmarks.py
14/utils/command_executer.py
253e88b10670bc7f97defd516ecf0623899555fd 15-Feb-2013 asharif <asharif@p4.invalid> Changed command_executer to use rsync whenever possible (when at least
one of source or dest is local).
DejaGNU tests were failing because symlinks weren't copied as symlinks
when using scp.

Also added error logging of failed jobs to both the job logger as well
as the server logger for better debugging.

Fixed a string formatting problem in build_chromeos.

Fixed a bug in machine_description where the correct machine was not being
selected by name.

Added a missing import in run_benchmarks.py and a missing option in the same script.

PRESUBMIT=passed
R=bjanakiraman,raymes
DELTA=80 (34 added, 20 deleted, 26 changed)
OCL=46309-p2
RCL=46313-p2
RDATE=2011/01/07 20:26:28


P4 change: 42608395
14/automation/common/job.py
14/automation/common/machine_description.py
14/automation/server/job_executer.py
14/build_chromeos.py
14/run_benchmarks.py
14/utils/command_executer.py
4a336e35a448e10deebe9cb31420c69b75edf816 15-Feb-2013 asharif <asharif@p4.invalid> Fixed an issue with binutils build failing due to timestamp issues.
Binutils was failing because ldlex.c's timestamp is older than ldlex.l's
timestamp occasionally. This causes a failure because we mount binutils
source as read-only and link it in the 9999 ebuild.
This CL is a workaround for now.

Fixed an issue where chromeos machines were not being locked.

Refactored the code a little bit and fixed toolchain_root paths to use the correct ones when using a branch.

PRESUBMIT=passed
R=bjanakiraman,raymes
DELTA=70 (32 added, 13 deleted, 25 changed)
OCL=46288-p2
RCL=46312-p2
RDATE=2011/01/07 20:26:18


P4 change: 42608388
14/automation/clients/test_client.py
14/automation/common/jobs_helper.py
2c3b5e82b9e799db5da5ccde6f82f9e4c2f061bd 15-Feb-2013 asharif <asharif@p4.invalid> Fixed a bug in job_manager that caused deadlocks.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=7 (7 added, 0 deleted, 0 changed)
OCL=46290-p2
RCL=46307-p2
RDATE=2011/01/07 11:42:32


P4 change: 42608383
14/automation/common/job.py
14/automation/server/job_manager.py
d5c2b2f379aeca4aa73ab947d383cf618a50e4bc 15-Feb-2013 asharif <asharif@p4.invalid> Fixed a typo in jobs_helper.py that was causing trunk to be used
even if -t branch was specified. This got introduced when a
refactoring of the code was done.

PRESUBMIT=passed
R=bjanakiraman
DELTA=10 (2 added, 3 deleted, 5 changed)
RCL=46284-p2
RDATE=2011/01/06 14:34:55


P4 change: 42608366
14/automation/clients/test_client.py
14/automation/common/jobs_helper.py
0e0e268380d38f48e9179d222dbc314b537b22ed 15-Feb-2013 asharif <asharif@p4.invalid> Fixed bug in build_chromeos.
Also fixed name in image_chromeos as suggested by bjanakiraman in
cl/46272-p2.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=19 (10 added, 1 deleted, 8 changed)
OCL=46281-p2
RCL=46283-p2
RDATE=2011/01/06 12:59:02


P4 change: 42608365
14/automation/common/job.py
14/build_chromeos.py
14/image_chromeos.py
14/run_benchmarks.py
14/utils/utils.py
8de2c73646c03786bc3f906983113441fc79c7b0 15-Feb-2013 asharif <asharif@p4.invalid> image_chromeos.py now takes a --image option to specify the .bin file.
build_benchmarks does not pass in --toolchain_root. It was failing before.
run_benchmarks now calls image_chromeos with the proper arguments.
jobs_helper now checks for a cpu benchmark before using minilayout.

Tested:
Testing it now.

PRESUBMIT=passed
R=bjanakiraman,raymes
DELTA=48 (35 added, 4 deleted, 9 changed)
RCL=46272-p2
RDATE=2011/01/05 17:33:48


P4 change: 42608341
14/automation/common/jobs_helper.py
14/build_benchmarks.py
14/image_chromeos.py
14/run_benchmarks.py
bb918509c3a5c7db2063ca2be85a0357beefc60b 15-Feb-2013 asharif <asharif@p4.invalid> Fixed newline (\n) issue in remote_sh.
Added a unittest for remote chromeos commands.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=63 (60 added, 0 deleted, 3 changed)
OCL=46251-p2
RCL=46254-p2
RDATE=2011/01/04 18:04:48


P4 change: 42608310
14/remote_test.py
14/utils/command_executer.py
14/utils/utils.py
4d2b7161bb78334b960c4d11ae7c2d27d7905314 15-Feb-2013 asharif <asharif@p4.invalid> Changed run_benchmarks to:
1) Not ask for toolchain_root, as it is not needed.
2) Not use expect. Use the utilities in command_executer instead.
3) Added copy and execute utilities in command_executer.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=203 (162 added, 27 deleted, 14 changed)
OCL=46243-p2
RCL=46250-p2
RDATE=2011/01/04 16:09:25


P4 change: 42608295
14/image_chromeos.py
14/run_benchmarks.py
14/utils/command_executer.py
14/utils/utils.py
81a30d0803e011cecf40b310267e66c11d4c79b0 15-Feb-2013 bjanakiraman <bjanakiraman@p4.invalid> Support for running other benchmarks.
Re-imaging is not being done yet.

PRESUBMIT=passed
R=asharif,raymes
DELTA=31 (21 added, 0 deleted, 10 changed)
OCL=46211-p2
RCL=46230-p2
RDATE=2011/01/02 23:05:02


P4 change: 42607739
14/run_benchmarks.py
9d6ccaacc04a2b432e45036a51af9e568c085b0a 15-Feb-2013 asharif <asharif@p4.invalid> build_chrome_browser.py should now build, merge and package the browser.
Update client now creates symlinks for sources to version numbers.
Nightly client now uses those symlinks if -c weekly or quarterly is
specified on the command line.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=72 (35 added, 2 deleted, 35 changed)
OCL=46192-p2
RCL=46212-p2
RDATE=2010/12/29 17:21:58


P4 change: 42607711
14/automation/clients/update_client.py
14/automation/common/job.py
14/automation/common/jobs_helper.py
14/automation/server/test_pool.csv
14/build_chrome_browser.py
95c259112da326aeb5695da33efb03733051b4a0 15-Feb-2013 asharif <asharif@p4.invalid> Added perflab to the test_client.py. Invoke it by running test_client.py
-l <benchmark_name>.

Changed test_pool.csv to the full name of the virtual machine.

Added function in command_executer to run commands on the chromeos machines.

PRESUBMIT=passed
R=bjanakiraman,raymes
DELTA=129 (85 added, 32 deleted, 12 changed)
OCL=46165-p2
RCL=46190-p2
RDATE=2010/12/28 13:47:44


P4 change: 42607680
14/automation/clients/test_client.py
14/automation/common/jobs_helper.py
14/automation/server/test_pool.csv
14/utils/command_executer.py
4e8bdf2d855a1b46781ce85c871f2faca667ce5d 15-Feb-2013 bjanakiraman <bjanakiraman@p4.invalid> Fix problem with ./out directory, and image directories.

PRESUBMIT=passed
R=asharif,raymes
DELTA=3 (0 added, 0 deleted, 3 changed)
RCL=46189-p2
RDATE=2010/12/28 13:45:01


P4 change: 42607679
14/build_benchmarks.py
14/run_benchmarks.py
ca8c5ef4af9c1f645497c3a79563bd086982942a 15-Feb-2013 asharif <asharif@p4.invalid> Fixed location of output chromeos build.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=17 (6 added, 2 deleted, 9 changed)
OCL=46148-p2
RCL=46153-p2
RDATE=2010/12/23 13:08:30


P4 change: 42607619
14/automation/common/jobs_helper.py
14/build_chromeos.py
9d40cc2305436f352c35e08c7071a515126bb944 15-Feb-2013 asharif <asharif@p4.invalid> Updated output location in the nightly job.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=56 (26 added, 18 deleted, 12 changed)
OCL=46132-p2
RCL=46146-p2
RDATE=2010/12/23 09:53:58


P4 change: 42607610
14/automation/clients/test_client.py
14/automation/common/jobs_helper.py
14/build_chromeos.py
8a873878c8ad046c0ffc078817c3308a46cd3710 15-Feb-2013 asharif <asharif@p4.invalid> tc_enter_chroot doesn't give error now when the symlink already
exists.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=23 (10 added, 4 deleted, 9 changed)
OCL=46131-p2
RCL=46133-p2
RDATE=2010/12/22 17:32:32


P4 change: 42607593
14/tc_enter_chroot.py
aa35176992dab4b4509e172ecde29c82067207c7 15-Feb-2013 raymes <raymes@p4.invalid> Used double quotes for extra flags. Fixed in build_chrome_browser too.

PRESUBMIT=passed
R=asharif
DELTA=21 (3 added, 12 deleted, 6 changed)
OCL=46120-p2
RCL=46125-p2
RDATE=2010/12/22 16:12:07


P4 change: 42607589
14/build_chrome_browser.py
14/build_chromeos.py
1ba8a3be3be728dbcba427df6a755c0f97ba6c0b 15-Feb-2013 asharif <asharif@p4.invalid> build_tc.py now uses a different default output directory. The default
output dir now is gcctools/output/

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=6 (3 added, 0 deleted, 3 changed)
OCL=46121-p2
RCL=46124-p2
RDATE=2010/12/22 16:10:55


P4 change: 42607578
14/build_tc.py
5789fd344d99af2e9cc4c90faee498cf5a71a3bb 15-Feb-2013 raymes <raymes@p4.invalid> Added missing spaces to command line for build_packages

PRESUBMIT=passed
R=asharif
DELTA=3 (0 added, 0 deleted, 3 changed)
RCL=46119-p2
RDATE=2010/12/22 15:45:41


P4 change: 42607577
14/build_chromeos.py
1c91ad95ecac1f6f0a39f7290c9c9eb283b6c688 15-Feb-2013 raymes <raymes@p4.invalid> Changed CFLAGS/LDFLAGS/CXXFLAGS in build_chromeos.py to append to the existing ones in make.conf when doing a build_packages.

PRESUBMIT=passed
R=asharif,bjanakiraman
DELTA=8 (5 added, 0 deleted, 3 changed)
OCL=46115-p2
RCL=46118-p2
RDATE=2010/12/22 15:37:51


P4 change: 42607573
14/automation/common/jobs_helper.py
14/build_chromeos.py
977cd6b8d1051df44176ff3d13697df25a12aee2 15-Feb-2013 asharif <asharif@p4.invalid> Changed build_chromeos.py to not use toolchain_root.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=20 (0 added, 13 deleted, 7 changed)
OCL=46114-p2
RCL=46116-p2
RDATE=2010/12/22 15:22:04


P4 change: 42607569
14/automation/common/jobs_helper.py
14/build_chromeos.py
d30e9ee0ce6786f3924090bc3d8bf411f38aba68 15-Feb-2013 raymes <raymes@p4.invalid> Improve job killing by explicitly killing the process rather than relying on the mechanism in the subprocess module.

PRESUBMIT=passed
R=asharif,bjanakiraman
DELTA=3 (0 added, 0 deleted, 3 changed)
OCL=46090-p2
RCL=46109-p2
RDATE=2010/12/22 13:32:43


P4 change: 42607562
14/automation/clients/client.py
14/utils/command_executer.py
b1471191504f0bf31fb2649fc06bc58a1bc4796c 15-Feb-2013 asharif <asharif@p4.invalid> Automated g4 rollback of changelist 46102.

*** Reason for rollback ***

<enter reason for rollback>

*** Original change description ***

Fixed typo in build_chromeos.py. Added .git and .repo ignores in
jobs_helper.py for faster rsyncs.

PRESUBMIT=passed
R=raymes
DELTA=19 (7 added, 4 deleted, 8 changed)
RCL=46105-p2
RDATE=2010/12/22 12:03:55


P4 change: 42607559
14/automation/common/jobs_helper.py
14/build_chromeos.py
ee543d127d66b36fdeb4d2635570e6a02f0421ad 15-Feb-2013 asharif <asharif@p4.invalid> Removed copying of chromeos checkout.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=6 (0 added, 0 deleted, 6 changed)
RCL=46103-p2
RDATE=2010/12/22 11:41:20


P4 change: 42607553
14/automation/common/jobs_helper.py
5fc3bda967e1b7e64e78285af5c0c2f41eaae049 15-Feb-2013 asharif <asharif@p4.invalid> Fixed typo in build_chromeos.py. Added .git and .repo ignores in
jobs_helper.py for faster rsyncs.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=23 (6 added, 7 deleted, 10 changed)
OCL=46099-p2
RCL=46102-p2
RDATE=2010/12/22 11:39:39


P4 change: 42607550
14/automation/common/jobs_helper.py
14/build_chromeos.py
229d626f6591ce7a07f1af175176724f55f84cbb 15-Feb-2013 bjanakiraman <bjanakiraman@p4.invalid> Change to create directory under perflab-bin and create links. helps in adding other files.
Add expect commands to utils.py
Add initial version of run_benchmarks.py that has capability to run
CPU benchmarks. Stubs for other benchmarks

PRESUBMIT=passed
R=raymes,asharif
DELTA=246 (227 added, 4 deleted, 15 changed)
OCL=45918-p2
RCL=46091-p2
RDATE=2010/12/22 09:50:32


P4 change: 42607543
14/build_benchmarks.py
14/run_benchmarks.py
14/utils/utils.py
ade100ba8e65270dbde78ad41a7e72afbe3fa174 15-Feb-2013 asharif <asharif@p4.invalid> Added missing '/'.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=1 (0 added, 0 deleted, 1 changed)
OCL=46077-p2
RCL=46079-p2
RDATE=2010/12/21 17:59:38


P4 change: 42607534
14/utils/logger.py
c151205fb2bd739a5fa3847dad26246f939df2c4 15-Feb-2013 asharif <asharif@p4.invalid> Added --yes to mod_image_for_test.sh.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=36 (32 added, 0 deleted, 4 changed)
OCL=46063-p2
RCL=46065-p2
RDATE=2010/12/21 16:36:28


P4 change: 42607511
14/automation/clients/test_client.py
14/automation/common/jobs_helper.py
14/build_chromeos.py
14/run_dejagnu.py
01e29a588fede35b6773395b2f3b8f732ff80a4e 15-Feb-2013 asharif <asharif@p4.invalid> Added expanduser to build_chromeos.py.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=6 (6 added, 0 deleted, 0 changed)
RCL=46061-p2
RDATE=2010/12/21 14:49:00


P4 change: 42607507
14/build_chromeos.py
ca35b7702db38f6f736b2baa44a0c2d3e4e2c597 15-Feb-2013 asharif <asharif@p4.invalid> Added hack to allow Chromium access.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=6 (5 added, 0 deleted, 1 changed)
RCL=46060-p2
RDATE=2010/12/21 14:22:38


P4 change: 42607506
14/setup_chromeos.py
3c3c7ab8d88a614f89fe84dac0d6c20fbc450cc3 15-Feb-2013 asharif <asharif@p4.invalid> Added build symlink to update job.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=11 (10 added, 0 deleted, 1 changed)
OCL=46053-p2
RCL=46055-p2
RDATE=2010/12/21 11:48:33


P4 change: 42607502
14/automation/common/jobs_helper.py
14/run_tests.py
461b61d04aa4fb1defc1854be086134575e7da70 15-Feb-2013 raymes <raymes@p4.invalid> Fixed sync_and_restart_server script so that it works when run locally on the VM.

PRESUBMIT=passed
R=asharif,bjanakiraman
DELTA=6 (0 added, 1 deleted, 5 changed)
OCL=46038-p2
RCL=46046-p2
RDATE=2010/12/21 08:48:06


P4 change: 42607492
14/automation/server/sync_and_restart_server.sh
0341d30c0489312cdffe0329dd70b0839db725c8 15-Feb-2013 asharif <asharif@p4.invalid> Updated update_chromeos.py.

Changed --binary option of build_chromeos.py and updated update_chromeos.py appropriately.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=21 (1 added, 9 deleted, 11 changed)
OCL=46029-p2
RCL=46037-p2
RDATE=2010/12/20 17:45:01


P4 change: 42607485
14/automation/clients/update_client.py
14/automation/common/jobs_helper.py
14/build_chromeos.py
6b742023d66c88032574ad6f3fe72b16f2045862 15-Feb-2013 raymes <raymes@p4.invalid> Improved dashboard to display links to logs and full command being executed.

PRESUBMIT=passed
R=asharif,bjanakiraman
DELTA=31 (20 added, 5 deleted, 6 changed)
OCL=46025-p2
RCL=46033-p2
RDATE=2010/12/20 16:58:20


P4 change: 42607483
14/automation/clients/web/index.py
14/automation/common/job.py
14/automation/server/job_executer.py
b1752c8090a909ee9544bb8199358d5c1f2b32a3 15-Feb-2013 asharif <asharif@p4.invalid> update_client now builds chromeos golden images.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=51 (40 added, 1 deleted, 10 changed)
OCL=46023-p2
RCL=46028-p2
RDATE=2010/12/20 14:54:24


P4 change: 42607469
14/automation/common/jobs_helper.py
14/automation/server/sync_and_restart_server.sh
14/build_chromeos.py
9da46bce3bf9017baa833aecf8abad3040e63425 15-Feb-2013 raymes <raymes@p4.invalid> Fixed up directory structure of logs.

PRESUBMIT=passed
R=asharif
DELTA=25 (10 added, 4 deleted, 11 changed)
OCL=46019-p2
RCL=46021-p2
RDATE=2010/12/20 11:45:33


P4 change: 42607464
14/automation/clients/client.py
14/automation/common/job.py
14/automation/server/job_manager.py
14/utils/logger.py
63a512c93d4965abe9693cabd90d1c97a2e51749 15-Feb-2013 raymes <raymes@p4.invalid> Removed ShipLogs() so logs are situated in central location only. Misc fixes to setup infrastructure on VMs.

PRESUBMIT=passed
R=asharif,bjanakiraman
DELTA=58 (15 added, 28 deleted, 15 changed)
OCL=46014-p2
RCL=46018-p2
RDATE=2010/12/20 11:38:19


P4 change: 42607454
14/automation/common/job_group.py
14/automation/server/job_executer.py
14/automation/server/server.py
14/automation/server/test_pool.csv
9b8305c04bde95a631b12a5fb3c6b291bf357a36 15-Feb-2013 raymes <raymes@p4.invalid> Misc fixes to run_dejagnu.py.

PRESUBMIT=passed
R=asharif,bjanakiraman
DELTA=43 (6 added, 18 deleted, 19 changed)
RCL=45982-p2
RDATE=2010/12/17 16:32:45


P4 change: 42607405
14/build_chromeos.py
14/run_dejagnu.py
14/utils/command_executer.py
14/utils/logger.py
81d88964dbb1ad07506e9048efbbc5e430eab679 15-Feb-2013 raymes <raymes@p4.invalid> Misc fixes to scripts.

PRESUBMIT=passed
R=asharif
DELTA=6 (2 added, 1 deleted, 3 changed)
OCL=45962-p2
RCL=45971-p2
RDATE=2010/12/17 09:21:07


P4 change: 42607393
14/build_chrome_browser.py
14/build_chromeos.py
a7d219cdd1102fbea2f152954dff908ec3ab2034 15-Feb-2013 raymes <raymes@p4.invalid> Added/tested run_dejagnu.py script for running the DejaGNU tests.

PRESUBMIT=passed
R=asharif
DELTA=133 (132 added, 0 deleted, 1 changed)
OCL=45946-p2
RCL=45961-p2
RDATE=2010/12/16 18:16:36


P4 change: 42607368
14/dejagnu/remote_init.sh
14/run_dejagnu.py
14/tc_enter_chroot.py
708d57cc7a0e66b0603f7e67b33294b93281dfd0 15-Feb-2013 asharif <asharif@p4.invalid> Fixed build_chrome_browser.py.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=16 (8 added, 0 deleted, 8 changed)
RCL=45956-p2
RDATE=2010/12/16 17:32:45


P4 change: 42607366
14/build_chrome_browser.py
14/build_chromeos.py
14/utils/command_executer.py
14/utils/logger.py
d8d0d9bc66d76b7cbf555fa67069900f64e24553 15-Feb-2013 asharif <asharif@p4.invalid> Updated script for binutils repo update.

Tested:
Tested this on chromium's binutils git repo. Seems to be working fine.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=34 (30 added, 0 deleted, 4 changed)
OCL=45945-p2
RCL=45949-p2
RDATE=2010/12/16 16:17:51


P4 change: 42607353
14/update_git_repo.py
2ccb4e34ce6c351a6fadb436404b76791b94edac 15-Feb-2013 raymes <raymes@p4.invalid> Minor fixes to web client.

PRESUBMIT=passed
R=asharif,bjanakiraman
DELTA=13 (10 added, 1 deleted, 2 changed)
RCL=45943-p2
RDATE=2010/12/16 13:30:14


P4 change: 42607347
14/automation/clients/web/index.py
c0f71930d71d8668c22da2ca77bffcb18f8942fd 15-Feb-2013 asharif <asharif@p4.invalid> Updated tc_enter_chroot.py to create temp file that it executes later.

I noticed that the nightly client didn't go past build_tc, and it turns out
that for newer checkouts of chromeos, ./enter_chroot 'sudo ls -lt' doesn't work.
This is a workaround that allows for flexible commands to be passed in. I haven't
tested it fully but it should work for us. Have a look and I'll submit tomorrow.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=34 (12 added, 15 deleted, 7 changed)
OCL=45920-p2
RCL=45933-p2
RDATE=2010/12/16 10:55:17


P4 change: 42607333
14/build_chromeos.py
14/build_tc.py
14/tc_enter_chroot.py
f5ec5d4b6247fb70946e9dcf8b1320187e76c9ba 15-Feb-2013 raymes <raymes@p4.invalid> Small fix to job_group.

PRESUBMIT=passed
R=asharif
DELTA=12 (11 added, 0 deleted, 1 changed)
RCL=45908-p2
RDATE=2010/12/15 16:49:13


P4 change: 42607302
14/automation/common/job_group.py
51516dadb723513c788bb87fdfe8a70399d5ac1b 15-Feb-2013 asharif <asharif@p4.invalid> build_tc.py now copies packages where build_image expects them to be.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=9 (5 added, 0 deleted, 4 changed)
OCL=45865-p2
RCL=45889-p2
RDATE=2010/12/15 10:58:22


P4 change: 42607291
14/build_tc.py
14/tc_enter_chroot.py
c939339af44d18a22049f72357e374d06cfbc51c 15-Feb-2013 raymes <raymes@p4.invalid> Added web interface client. Very basic to start with. Needs work to add log links and kill buttons.

PRESUBMIT=passed
R=asharif,bjanakiraman
DELTA=325 (224 added, 99 deleted, 2 changed)
OCL=45856-p2
RCL=45861-p2
RDATE=2010/12/14 15:54:25


P4 change: 42607281
14/automation/clients/web/index.py
14/automation/common/machine.py
14/automation/common/machine_test.py
14/automation/server/job_group_manager.py
14/automation/server/machine.py
14/automation/server/machine_manager.py
14/automation/server/machine_test.py
4d8013a4ac5a65ab0f5568280c2a161814172c80 15-Feb-2013 asharif <asharif@p4.invalid> Added update_client and made misc. fixes.
The update_client will update weekly chromeos snapshots. test_client now
has more command-line options. command_executer now sets the nice value
of the command.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=52 (18 added, 22 deleted, 12 changed)
OCL=45857-p2
RCL=45860-p2
RDATE=2010/12/14 15:52:07


P4 change: 42607263
14/automation/common/job.py
14/automation/common/job_group.py
14/automation/server/job_executer.py
14/automation/server/job_group_manager.py
14/utils/command_executer.py
ea33a5625df3c708a5bb8c34b4bf0a880532cf14 15-Feb-2013 asharif <asharif@p4.invalid> Added update_client and made misc. fixes.
The update_client will update weekly chromeos snapshots. test_client now
has more command-line options. command_executer now sets the nice value
of the command.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=98 (79 added, 0 deleted, 19 changed)
OCL=45833-p2
RCL=45854-p2
RDATE=2010/12/14 14:31:43


P4 change: 42607258
14/automation/clients/test_client.py
14/automation/clients/update_client.py
14/automation/common/jobs_helper.py
14/run_tests.py
2dd61753626cea2988cd4f76f78c7576aec76dd9 15-Feb-2013 raymes <raymes@p4.invalid> Changes to allow client to query server state about job_groups.

PRESUBMIT=passed
R=asharif,bjanakiraman
DELTA=34 (10 added, 19 deleted, 5 changed)
OCL=45847-p2
RCL=45849-p2
RDATE=2010/12/14 12:56:44


P4 change: 42607248
14/automation/server/job_group_manager.py
14/automation/server/job_manager.py
14/automation/server/server.py
7b96a55584072e3b9719a153f5733f4127448f53 15-Feb-2013 asharif <asharif@p4.invalid> Merged chromeos_version and chromeos_snapshot options.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=31 (9 added, 0 deleted, 22 changed)
OCL=45772-p2
RCL=45788-p2
RDATE=2010/12/12 13:53:26


P4 change: 42607188
14/automation/common/job.py
14/automation/common/job_group.py
14/automation/common/jobs_helper.py
14/automation/server/job_executer.py
14/automation/server/job_group_manager.py
14/automation/server/job_manager.py
aabd2d14d05d9ddfd75ef1b7bf33638ef2329e9e 15-Feb-2013 bjanakiraman <bjanakiraman@p4.invalid> Add support for 'Clean' target, and copying binaries to perflab-bin. For now,
the copy is really a sym-link to avoid expensive copies, since we are not using perflab queues.

Change utils.py and logger.py to add generic assertion routines.

PRESUBMIT=passed
R=raymes,asharif
DELTA=118 (84 added, 1 deleted, 33 changed)
OCL=45760-p2
RCL=45774-p2
RDATE=2010/12/10 17:39:21


P4 change: 42607167
14/build_benchmarks.py
14/utils/logger.py
14/utils/utils.py
1755b433347d3d751c9f4c7b3ae4d44710bc6660 15-Feb-2013 asharif <asharif@p4.invalid> Fixed numerous issues/bugs related to nightly client. It should be more robust and faster now.

PRESUBMIT=passed
R=bjanakiraman,raymes
DELTA=126 (36 added, 41 deleted, 49 changed)
OCL=45744-p2
RCL=45766-p2
RDATE=2010/12/10 13:06:58


P4 change: 42607162
14/automation/clients/test_client.py
14/automation/common/jobs_helper.py
14/automation/server/job_executer.py
14/build_chromeos.py
14/build_tc.py
14/tc_enter_chroot.py
49fd5a3904d3740615e147668865c1466b74f010 15-Feb-2013 raymes <raymes@p4.invalid> Added build_chrome_browser.py to build the browser only. Tested it briefly but may still have some kinks.

PRESUBMIT=passed
R=asharif,bjanakiraman
DELTA=132 (125 added, 0 deleted, 7 changed)
OCL=45750-p2
RCL=45761-p2
RDATE=2010/12/10 11:01:56


P4 change: 42607145
14/build_chrome_browser.py
14/build_chromeos.py
fb633bf38375dac564537b4959dbcd728eb24872 15-Feb-2013 asharif <asharif@p4.invalid> Added binutils. Converted linking to copying for testing locally.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=6 (4 added, 0 deleted, 2 changed)
OCL=45738-p2
RCL=45740-p2
RDATE=2010/12/09 15:11:55


P4 change: 42606987
14/automation/common/jobs_helper.py
28238cff158fbd6c1360c159b07d2472ba3f6af3 15-Feb-2013 asharif <asharif@p4.invalid> binutils-9999 should now work with build_tc.py.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=4 (2 added, 0 deleted, 2 changed)
OCL=45714-p2
RCL=45728-p2
RDATE=2010/12/09 11:35:24


P4 change: 42606975
14/build-binutils/opts.sh
14/build_tc.py
14/tc_enter_chroot.py
967d7000e481956cd81b8fc7fa3030789a71d1ee 15-Feb-2013 asharif <asharif@p4.invalid> Yet another fix in low-level scripts involving command_executer changes.
Fixed run_tests.py.
Also added options in the nightly client for debugging it. It can use a local p4 checkout instead of creating a new client for faster iteration.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=46 (22 added, 14 deleted, 10 changed)
OCL=45701-p2
RCL=45709-p2
RDATE=2010/12/08 17:00:37


P4 change: 42606965
14/automation/clients/test_client.py
14/automation/common/jobs_helper.py
14/build_chromeos.py
14/run_tests.py
6496e5f37e6ce3f67c1ab2a861319293c9358b71 15-Feb-2013 bjanakiraman <bjanakiraman@p4.invalid> Low level script to build benchmarks

PRESUBMIT=passed
R=raymes,asharif
DELTA=150 (146 added, 0 deleted, 4 changed)
OCL=45694-p2
RCL=45705-p2
RDATE=2010/12/08 16:05:24


P4 change: 42606949
14/build_benchmarks.py
14/build_chromeos.py
e333f30aa5c55fcf2d48930b56bc5b881fc2a722 15-Feb-2013 asharif <asharif@p4.invalid> Fixed a typo that got checked in accidentaly in the last CL.

PRESUBMIT=passed
R=raymes
DELTA=1 (0 added, 0 deleted, 1 changed)
RCL=45700-p2
RDATE=2010/12/08 14:03:52


P4 change: 42606946
14/utils/command_executer.py
5a9bb467bacc72e73bdbaba2eff722ab8ffdd22a 15-Feb-2013 asharif <asharif@p4.invalid> Fixed small bugs in setup_chromeos.py and test_client.py.

PRESUBMIT=passed
R=raymes
DELTA=15 (10 added, 0 deleted, 5 changed)
RCL=45699-p2
RDATE=2010/12/08 13:46:21


P4 change: 42606939
14/automation/clients/test_client.py
14/build_chromeos.py
14/build_tc.py
14/setup_chromeos.py
3ee217efddcc6384b43cf37eb125020a4a295ffd 15-Feb-2013 asharif <asharif@p4.invalid> Added job log handling and formatted the job elapsed duration.

The job now prints its duration in human-readable format.
The job logs are now created on a per-job basis as opposed to dumping
everything in server.log. After the job completes, the logs are shipped
to the logs directory of the machine where the job ran.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=56 (36 added, 4 deleted, 16 changed)
OCL=45676-p2
RCL=45696-p2
RDATE=2010/12/08 13:06:47


P4 change: 42606936
14/automation/common/job.py
14/automation/server/job_executer.py
14/utils/command_executer.py
5a67e7579339dab4c5e12de60797d79d1f4cba6a 15-Feb-2013 raymes <raymes@p4.invalid> Added results copying and also some other misc cleanup and fixes. Some complications with copying to a users space (permissions, etc.). May be simpler to copy to a central location which the user can access.

PRESUBMIT=passed
R=asharif,bjanakiraman
DELTA=116 (80 added, 12 deleted, 24 changed)
OCL=45664-p2
RCL=45669-p2
RDATE=2010/12/07 16:26:03


P4 change: 42606917
14/automation/clients/client.py
14/automation/common/job.py
14/automation/common/job_group.py
14/automation/server/job_executer.py
14/automation/server/job_group_manager.py
14/automation/server/job_manager.py
14/utils/command_executer.py
0d03a1762f608c3c8fab4d9168b9a0d4ccb3a0b5 15-Feb-2013 bjanakiraman <bjanakiraman@p4.invalid> Fix bug in processing short options.

PRESUBMIT=passed
R=raymes,asharif
DELTA=10 (6 added, 2 deleted, 2 changed)
OCL=45654-p2
RCL=45659-p2
RDATE=2010/12/07 10:42:30


P4 change: 42606898
14/tc_enter_chroot.py
a3ff35dff739255b3d4de01b97ff7505ab8a321e 15-Feb-2013 raymes <raymes@p4.invalid> Added job_group's and cleanup of job_groups when they are complete. Still need to add directory copying.

PRESUBMIT=passed
R=asharif,bjanakiraman
DELTA=207 (154 added, 14 deleted, 39 changed)
OCL=45638-p2
RCL=45648-p2
RDATE=2010/12/07 08:54:38


P4 change: 42606892
14/automation/clients/client.py
14/automation/common/job_group.py
14/automation/server/job_executer.py
14/automation/server/job_group_manager.py
14/automation/server/job_manager.py
14/automation/server/server.py
14/utils/command_executer.py
fc0277c318d91ef83f83283d3a1b394758d3ccda 15-Feb-2013 asharif <asharif@p4.invalid> Misc. fixes to run nightly in dry-run.

Fixed code in cmd_executor module so that mock and regular ones have the same
interface. Fixed string replacement in _formatcode. Fixed a bug in
machine acquisition. Will be pushing this in soon.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=15 (8 added, 1 deleted, 6 changed)
RCL=45639-p2
RDATE=2010/12/06 19:47:50


P4 change: 42606875
14/automation/common/job.py
14/automation/common/jobs_helper.py
14/automation/server/job_executer.py
14/automation/server/machine_manager.py
14/utils/command_executer.py
7d913e193af50c672872cbd136180f02d59cb962 15-Feb-2013 asharif <asharif@p4.invalid> Added more detailed profile data like copying time, etc.
Printed the profile data to the screen after the job finishes.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=47 (28 added, 9 deleted, 10 changed)
RCL=45637-p2
RDATE=2010/12/06 18:20:40


P4 change: 42606863
14/automation/common/job.py
14/automation/common/jobs_helper.py
14/automation/server/job_executer.py
14/automation/server/job_manager.py
9642da7baa897913f7df305c6d955639e2c2a6a4 15-Feb-2013 asharif <asharif@p4.invalid> Forgot to add these changes to my previous CL.

PRESUBMIT=passed
R=raymes
DELTA=6 (0 added, 0 deleted, 6 changed)
RCL=45628-p2
RDATE=2010/12/06 14:54:40


P4 change: 42606853
14/automation/server/job_executer.py
14/automation/server/job_manager.py
3cb08991e3ccb0d543764853620cfdcb225358a5 15-Feb-2013 asharif <asharif@p4.invalid> If the destination directory doesn't exist for a dependent job,
it will be created.
Also changed machine to machines so that SECONDARY_MACHINE variable
can be properly replaced.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=26 (10 added, 2 deleted, 14 changed)
OCL=45625-p2
RCL=45627-p2
RDATE=2010/12/06 14:33:17


P4 change: 42606849
14/automation/clients/test_client.py
14/automation/common/job.py
14/automation/common/jobs_helper.py
14/automation/server/job_executer.py
927ad924d0fef5e0563e601de734c9c3273516f4 15-Feb-2013 raymes <raymes@p4.invalid> Added missing command_executer update.

PRESUBMIT=passed
R=asharif
DELTA=26 (21 added, 0 deleted, 5 changed)
RCL=45622-p2
RDATE=2010/12/06 11:25:51


P4 change: 42606842
14/utils/command_executer.py
dff6134e6537ad3eb3c999c69c8e3a3965c67173 15-Feb-2013 asharif <asharif@p4.invalid> Fixed a small bug in setup_chromeos.py related to checkint out
minilayout.xml.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=1 (1 added, 0 deleted, 0 changed)
RCL=45621-p2
RDATE=2010/12/06 11:16:11


P4 change: 42606841
14/setup_chromeos.py
b5493df1eebda10422fcfea41afbb613f7aa7836 15-Feb-2013 asharif <asharif@p4.invalid> Fixed dir reporting in tc_enter_chroot.py. Fixed 2 issues in jobs_helper.py.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=6 (5 added, 0 deleted, 1 changed)
RCL=45620-p2
RDATE=2010/12/06 10:59:01


P4 change: 42606840
14/automation/common/jobs_helper.py
14/tc_enter_chroot.py
15e5243ce7b69efe3059b09c48b779d4e81fb9f6 15-Feb-2013 raymes <raymes@p4.invalid> Added job killing ability. Clean shutdown now works correctly.

PRESUBMIT=passed
R=asharif,bjanakiraman
DELTA=122 (71 added, 17 deleted, 34 changed)
OCL=45599-p2
RCL=45618-p2
RDATE=2010/12/06 10:14:18


P4 change: 42606839
14/automation/clients/client.py
14/automation/server/job_executer.py
14/automation/server/job_manager.py
14/automation/server/server.py
6ba78c4e0a54cc212e8a420eec4648cf9598fb15 15-Feb-2013 asharif <asharif@p4.invalid> Added timestamps to jobs for profiling.
Added support for primary/secondary IDs in jobs (currently commented
but will send out another CL soon with tests and job class changes).
Added nightly client with command-line options.

Tested:
Testing it right now on my own machine.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=189 (129 added, 38 deleted, 22 changed)
OCL=45596-p2
RCL=45598-p2
RDATE=2010/12/03 17:49:21


P4 change: 42606801
14/automation/clients/test_client.py
14/automation/common/job.py
14/automation/common/jobs_helper.py
14/automation/server/job_executer.py
14/build_tc.py
2beabd1a3ce389658b133c8b2cb853b73300bb38 15-Feb-2013 asharif <asharif@p4.invalid> Added nightly client.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=101 (94 added, 1 deleted, 6 changed)
OCL=45555-p2
RCL=45594-p2
RDATE=2010/12/03 15:00:47


P4 change: 42606792
14/automation/clients/nightly.py
14/automation/common/jobs_helper.py
541b6398ad64801c91d77cfccdb9e1f70161bf6f 15-Feb-2013 asharif <asharif@p4.invalid> Fixed a few issues in the scripts so nightly client could run properly.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=19 (9 added, 1 deleted, 9 changed)
OCL=45586-p2
RCL=45590-p2
RDATE=2010/12/03 12:35:34


P4 change: 42606788
14/build_chromeos.py
14/build_tc.py
14/tc_enter_chroot.py
065bf0b204a7b52fe1dbe6d7e21027551d7f0618 15-Feb-2013 asharif <asharif@p4.invalid> Fixed logger error.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=1 (0 added, 1 deleted, 0 changed)
RCL=45576-p2
RDATE=2010/12/02 23:34:05


P4 change: 42606773
14/build_chromeos.py
1d8c0cf0f66c95522f80bbce3a1a6e61c4768658 15-Feb-2013 raymes <raymes@p4.invalid> Cleaned up JobExecuter. Made logging nicer for Command execution. Only copy directories between jobs if necessary. Improved sample client.

PRESUBMIT=passed
R=asharif,bjanakiraman
DELTA=153 (82 added, 39 deleted, 32 changed)
RCL=45573-p2
RDATE=2010/12/02 16:18:11


P4 change: 42606772
14/automation/clients/client.py
14/automation/common/job.py
14/automation/server/job_executer.py
14/automation/server/machine_manager.py
14/automation/server/server.py
14/utils/command_executer.py
14/utils/logger.py
556f4ffce25ac9c9dd844dedc64113c939fc721c 15-Feb-2013 asharif <asharif@p4.invalid> Fixed ownership problems in mounting.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=23 (11 added, 0 deleted, 12 changed)
RCL=45570-p2
RDATE=2010/12/02 14:21:16


P4 change: 42606761
14/tc_enter_chroot.py
88f01421c86a6ab4b671551e84226d185cfcd428 15-Feb-2013 asharif <asharif@p4.invalid> Fixed setup_chromeos calling in build_tc.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=7 (4 added, 0 deleted, 3 changed)
RCL=45568-p2
RDATE=2010/12/02 11:59:31


P4 change: 42606760
14/build_tc.py
099e179919b0dd9e45902bd9968b4d8040bcbe50 15-Feb-2013 raymes <raymes@p4.invalid> Added logging to CommandExecuterMock.
Made the job scheduler a bit smart about where it allocates jobs. Try to allocate job on same machine as child jobs.
Started to clean up JobExecuter.

PRESUBMIT=passed
R=asharif,bjanakiraman
DELTA=179 (37 added, 83 deleted, 59 changed)
RCL=45567-p2
RDATE=2010/12/02 11:20:26


P4 change: 42606759
14/automation/common/machine_description.py
14/automation/server/job_executer.py
14/automation/server/job_manager.py
14/automation/server/machine_manager.py
14/automation/server/machine_pool.py
14/utils/command_executer.py
975118f04af1fcdad616624f6f4976c2e78f8852 15-Feb-2013 raymes <raymes@p4.invalid> Renamed dependency -> child/parent to make code more understandable.

PRESUBMIT=passed
R=asharif,bjanakiraman
DELTA=38 (5 added, 0 deleted, 33 changed)
OCL=45553-p2
RCL=45563-p2
RDATE=2010/12/02 08:57:00


P4 change: 42606754
14/automation/clients/build_tc_client.py
14/automation/clients/client.py
14/automation/common/job.py
14/automation/common/jobs_helper.py
14/automation/server/job_manager.py
0d3535aa4a86ad2581d667e210e772014a6ce4c2 15-Feb-2013 asharif <asharif@p4.invalid> build_tc.py now accepts an output directory for where portage output
gets stored.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=75 (49 added, 2 deleted, 24 changed)
OCL=45554-p2
RCL=45556-p2
RDATE=2010/12/01 19:34:17


P4 change: 42606747
14/build_tc.py
14/setup_chromeos.py
8c5daa1b723eacfbabb3c48ab1e13a06fce67e64 15-Feb-2013 raymes <raymes@p4.invalid> Fixed all_tests.py script so that it works and correctly runs all tests.

PRESUBMIT=passed
R=asharif,bjanakiraman
DELTA=15 (4 added, 1 deleted, 10 changed)
RCL=45550-p2
RDATE=2010/12/01 16:12:07


P4 change: 42606746
14/automation/all_tests.py
14/automation/server/machine_manager.py
14/automation/server/server.py
14/automation/server/server_test.py
d1a265359c87b9230703ae01246e7bdf3da2cf9d 15-Feb-2013 asharif <asharif@p4.invalid> Added all_tests.py.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=17 (17 added, 0 deleted, 0 changed)
RCL=45549-p2
RDATE=2010/12/01 13:26:46


P4 change: 42606735
14/automation/all_tests.py
67d07556bbfee1f41817a954f6dab93656621fa9 15-Feb-2013 asharif <asharif@p4.invalid> AddRequiredFolder() now adds dependencies implicitly.
The goal of this CL is to reduce the number of lines of clients.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=4 (4 added, 0 deleted, 0 changed)
RCL=45547-p2
RDATE=2010/12/01 11:00:41


P4 change: 42606733
14/automation/common/job.py
7a4a999245d612dc3231bc48ad9b969e32a1ad33 15-Feb-2013 asharif <asharif@p4.invalid> Moved files around to the new dir structure. This breaks everything --
will fix in the next CL.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=1745 (846 added, 899 deleted, 0 changed)
OCL=45532-p2
RCL=45535-p2
RDATE=2010/11/30 18:34:14


P4 change: 42606704
14/automation/build_tc_client.py
14/automation/client.py
14/automation/clients/__init__.py
14/automation/clients/build_tc_client.py
14/automation/clients/client.py
14/automation/common/__init__.py
14/automation/common/job.py
14/automation/common/jobs_helper.py
14/automation/common/jobs_helper_test.py
14/automation/common/machine_description.py
14/automation/job_executer.py
14/automation/job_manager.py
14/automation/jobs/__init__.py
14/automation/jobs/generic_job.py
14/automation/jobs/job.py
14/automation/jobs/jobs_helper.py
14/automation/jobs/jobs_helper_test.py
14/automation/jobs/p4_job.py
14/automation/machine.py
14/automation/machine_description.py
14/automation/machine_manager.py
14/automation/machine_manager_test.py
14/automation/machine_pool.py
14/automation/machine_test.py
14/automation/server.py
14/automation/server/__init__.py
14/automation/server/job_executer.py
14/automation/server/job_manager.py
14/automation/server/machine.py
14/automation/server/machine_manager.py
14/automation/server/machine_manager_test.py
14/automation/server/machine_pool.py
14/automation/server/machine_test.py
14/automation/server/server.py
14/automation/server/server_test.py
14/automation/server/test_pool.csv
14/automation/server_test.py
14/automation/test_pool.csv
ecd4388b8594664f16dc646e81045a0a86d0de91 15-Feb-2013 raymes <raymes@p4.invalid> Added option to select location where machines file is loaded from. Changed machines file to csv format to separate data from code and allow for easy editing. Also added dry-option for testing the server without executing any jobs.

PRESUBMIT=passed
R=asharif
DELTA=84 (34 added, 28 deleted, 22 changed)
OCL=45520-p2
RCL=45530-p2
RDATE=2010/11/30 16:52:24


P4 change: 42606700
14/automation/job_executer.py
14/automation/job_manager.py
14/automation/machine_manager.py
14/automation/machine_manager_test.py
14/automation/pools.py
14/automation/server.py
14/automation/test_pool.csv
648e7213be3213998c0daac1f646c3b1d39778a6 15-Feb-2013 asharif <asharif@p4.invalid> Cleaned up some stuff.

Specifically, removed references of machine_filters. Also fixed
a unittest.

PRESUBMIT=passed
R=raymes,bjanakiraman
CC=c-compiler-reviews
DELTA=22 (1 added, 19 deleted, 2 changed)
OCL=45519-p2
RCL=45521-p2
RDATE=2010/11/30 15:48:52


P4 change: 42606695
14/automation/machine_description.py
14/automation/machine_manager_test.py
14/automation/machine_pool.py
57a094961960dff71de551b1883f733bc62b5c53 15-Feb-2013 asharif <asharif@p4.invalid> The goal here is to remove the job hierarchy because it doesn't add much value to the infrastructure.

I have removed most of the job hierarchy related files but there might still be other clean-ups to do after this CL (if some file does an import of ls_job, for example).

Testing: I tested this with the updated client.py and it seemed to work (it printed the correct ssh command after finding the linux machine).

PRESUBMIT=passed
R=bjanakiraman,raymes
DELTA=295 (134 added, 149 deleted, 12 changed)
OCL=45500-p2
RCL=45516-p2
RDATE=2010/11/30 15:09:36


P4 change: 42606689
14/automation/client.py
14/automation/job_executer.py
14/automation/jobs/echo_job.py
14/automation/jobs/job.py
14/automation/jobs/jobs_helper.py
14/automation/jobs/jobs_helper_test.py
14/automation/jobs/ls_job.py
14/automation/jobs/setup_chromeos_job.py
14/automation/machine_description.py
14/automation/machine_filters.py
14/automation/machine_manager.py
0b2f0400ba637cdf9d23ed566ee9db3acf995998 15-Feb-2013 asharif <asharif@p4.invalid> Added more descriptive messages for error cases.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=30 (22 added, 4 deleted, 4 changed)
OCL=45498-p2
RCL=45508-p2
RDATE=2010/11/30 10:51:22


P4 change: 42606684
14/tc_enter_chroot.py
da9ac6572c9110d3c2c6d064d95f8f1efda690a4 15-Feb-2013 asharif <asharif@p4.invalid> Added option to tc_enter_chroot.py to accept additional mount points.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=95 (60 added, 10 deleted, 25 changed)
OCL=45488-p2
RCL=45497-p2
RDATE=2010/11/29 19:30:18


P4 change: 42606664
14/tc_enter_chroot.py
01959ae3b86ec0ecf99d1e6cf316845d5c3c546a 15-Feb-2013 raymes <raymes@p4.invalid> Put command execution related logic into a class so that we can have a mock command executer, for testing/dry-run purposes. De-coupled command execution and logging.

PRESUBMIT=passed
R=asharif
DELTA=425 (217 added, 130 deleted, 78 changed)
OCL=45487-p2
RCL=45495-p2
RDATE=2010/11/29 16:15:34


P4 change: 42606657
14/automation/build_tc_client.py
14/automation/job_executer.py
14/automation/machine.py
14/build_chromeos.py
14/build_tc.py
14/run_tests.py
14/setup_chromeos.py
14/tc_enter_chroot.py
14/update_git_repo.py
14/utils/command_executer.py
14/utils/logger.py
14/utils/utils.py
07383c0b7c20d527a6ea5d2f8fa99a297cbf49d0 15-Feb-2013 asharif <asharif@p4.invalid> Changed the way build_tc_client.py is invoking build_tc.py.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=4 (1 added, 1 deleted, 2 changed)
RCL=45484-p2
RDATE=2010/11/29 13:28:50


P4 change: 42606624
14/automation/build_tc_client.py
fcf8cfc4f9eff588aad3607537b8a3a5b4902709 15-Feb-2013 asharif <asharif@p4.invalid> Fixed build_tc.py so it can run from anywhere and not necessarily
just from the chromeos_root/src/scripts/ dir.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=3 (0 added, 0 deleted, 3 changed)
RCL=45483-p2
RDATE=2010/11/29 13:08:24


P4 change: 42606622
14/build_tc.py
14/tc_enter_chroot.py
900f673296dfa3dd11bbae44cad928764e4f8629 15-Feb-2013 raymes <raymes@p4.invalid> Added setup_chromeos_job and bug fixes to make it work.

PRESUBMIT=passed
R=asharif,bjanakiraman
DELTA=60 (37 added, 6 deleted, 17 changed)
RCL=45456-p2
RDATE=2010/11/24 18:16:27


P4 change: 42606578
14/automation/build_tc_client.py
14/automation/job_executer.py
14/automation/jobs/generic_job.py
14/automation/jobs/job.py
14/automation/jobs/p4_job.py
14/automation/jobs/setup_chromeos_job.py
14/automation/server.py
6b9fdc9489b51096981dcbf03d2db332494d2ec7 15-Feb-2013 asharif <asharif@p4.invalid> Changed double quotes to single quotes. Also added generic_job to actually
run build_tc.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=30 (26 added, 0 deleted, 4 changed)
OCL=45443-p2
RCL=45450-p2
RDATE=2010/11/24 16:26:53


P4 change: 42606568
14/automation/build_tc_client.py
14/automation/jobs/generic_job.py
14/automation/jobs/p4_job.py
e9f39dcd8e18ca73f5c4788251a9eb0c3d137bb9 15-Feb-2013 asharif <asharif@p4.invalid> Server now gets killed when you press ctrl-C. Also fixed client mappings
in p4_job.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=16 (7 added, 1 deleted, 8 changed)
RCL=45442-p2
RDATE=2010/11/24 14:26:23


P4 change: 42606562
14/automation/build_tc_client.py
14/automation/jobs/p4_job.py
14/automation/server.py
cefd138be76a0c637adaad56ddf30b171c02b48f 15-Feb-2013 asharif <asharif@p4.invalid> g4 client is working now.
Testing: Ran it successfully.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=34 (25 added, 1 deleted, 8 changed)
RCL=45441-p2
RDATE=2010/11/24 14:02:20


P4 change: 42606557
14/automation/build_tc_client.py
14/automation/job_executer.py
14/automation/jobs/p4_job.py
14/utils/utils.py
858a24277dfb75bf70a168d599848c85cd9d6e6c 15-Feb-2013 asharif <asharif@p4.invalid> Added double quotes in job_executor. p4_job *should* work
now.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=22 (19 added, 0 deleted, 3 changed)
RCL=45440-p2
RDATE=2010/11/24 12:54:14


P4 change: 42606554
14/automation/build_tc_client.py
14/automation/job_executer.py
14/automation/jobs/p4_job.py
965079f8bc3e1ab22a6d72abc017c96538ec1c69 15-Feb-2013 raymes <raymes@p4.invalid> Fixed some bugs

PRESUBMIT=passed
R=asharif
DELTA=33 (16 added, 4 deleted, 13 changed)
RCL=45439-p2
RDATE=2010/11/24 11:58:01


P4 change: 42606551
14/automation/job_executer.py
14/automation/job_manager.py
14/automation/jobs/p4_job.py
14/automation/server.py
d8c718bd16a02934693bcd58b9411891e48a9c8c 15-Feb-2013 raymes <raymes@p4.invalid> Added folder copying logic and job_manager start/stop logic.

PRESUBMIT=passed
R=asharif
DELTA=105 (102 added, 0 deleted, 3 changed)
OCL=45429-p2
RCL=45435-p2
RDATE=2010/11/24 11:17:25


P4 change: 42606547
14/automation/job_executer.py
14/automation/job_manager.py
14/automation/jobs/job.py
6995287876d98f0d4d19bd12b18df28bf5fb0639 15-Feb-2013 asharif <asharif@p4.invalid> Added a p4_job. Also added some unittests.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=110 (101 added, 3 deleted, 6 changed)
OCL=45421-p2
RCL=45434-p2
RDATE=2010/11/24 11:09:40


P4 change: 42606546
14/automation/jobs/p4_job.py
14/automation/machine.py
14/automation/machine_manager_test.py
14/automation/machine_pool.py
14/automation/machine_test.py
14/automation/server.py
14/automation/server_test.py
b6920fb6d4990a4563b914cc5ea3ebf48ef1e1a7 15-Feb-2013 asharif <asharif@p4.invalid> Added 2 tests which revealed a bug. Added case-insensitive
comparisons for os and names.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=65 (48 added, 9 deleted, 8 changed)
OCL=45403-p2
RCL=45407-p2
RDATE=2010/11/23 14:11:54


P4 change: 42606488
14/automation/machine.py
14/automation/machine_filters.py
14/automation/machine_manager.py
14/automation/machine_manager_test.py
14/automation/machine_pool.py
b708f5f4048fedb3107860cacee1ab362a6ab44d 15-Feb-2013 raymes <raymes@p4.invalid> Bug fixes to get infrastructure executing remote commands. Misc cleanup in-line with discussions.

PRESUBMIT=passed
R=asharif,bjanakiraman
DELTA=89 (40 added, 33 deleted, 16 changed)
RCL=45401-p2
RDATE=2010/11/23 12:09:34


P4 change: 42606478
14/automation/job_executer.py
14/automation/jobs/echo_job.py
14/automation/jobs/ls_job.py
14/automation/machine.py
14/automation/machine_description.py
14/automation/machine_filters.py
14/automation/machine_manager.py
14/automation/machine_pool.py
14/automation/pools.py
b6905600b0d6c5ecde5662201cfb4268a5d366c0 15-Feb-2013 raymes <raymes@p4.invalid> Modified locking to be a part of the server state. Connected machine_manager to job_executer so that infrastructure is capable of executing jobs remotely.

PRESUBMIT=passed
R=asharif,bjanakiraman
DELTA=80 (37 added, 34 deleted, 9 changed)
RCL=45385-p2
RDATE=2010/11/22 18:31:12


P4 change: 42606464
14/automation/job_executer.py
14/automation/jobs/job.py
14/automation/machine.py
14/automation/machine_filters.py
14/automation/machine_manager.py
704c969264232a1ffdd02214f1a69259aa61e550 15-Feb-2013 asharif <asharif@p4.invalid> Added MachineManager's unittest. Also fixed a few issues related to
locking.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=45 (40 added, 3 deleted, 2 changed)
RCL=45384-p2
RDATE=2010/11/22 17:18:34


P4 change: 42606451
14/automation/machine.py
14/automation/machine_filters.py
14/automation/machine_manager.py
14/automation/machine_manager_test.py
df9c2d7fb7449cd34c5931ea84caf8dd727b7399 15-Feb-2013 asharif <asharif@p4.invalid> Added pyrun shell script to add to PYTHONPATH.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=7 (6 added, 0 deleted, 1 changed)
RCL=45377-p2
RDATE=2010/11/22 14:01:47


P4 change: 42606446
14/automation/machine_filters.py
14/pyrun
f21dc5da2a82620ca89dd7c7500b26b1c9aa3a24 15-Feb-2013 raymes <raymes@p4.invalid> Fixes to machine_filters location. changed files to xtext.

PRESUBMIT=passed
R=asharif
DELTA=52 (5 added, 39 deleted, 8 changed)
OCL=45371-p2
RCL=45376-p2
RDATE=2010/11/22 13:32:29


P4 change: 42606389
14/automation/client.py
14/automation/job_executer.py
14/automation/job_manager.py
14/automation/jobs/echo_job.py
14/automation/machine_filters.py
14/automation/machine_manager.py
14/automation/pools.py
14/automation/server.py
63fbdbe1850303b986672d4ef49b5333197fe550 15-Feb-2013 raymes <raymes@p4.invalid> Moved machine_pools.py to machine_pool.py

PRESUBMIT=passed
R=asharif
DELTA=118 (59 added, 59 deleted, 0 changed)
RCL=45366-p2
RDATE=2010/11/22 11:57:46


P4 change: 42606383
14/automation/machine_pool.py
14/automation/machine_pools.py
31e7eb736eefc2006db7a575e31f214bfd347164 15-Feb-2013 raymes <raymes@p4.invalid> Moved filters to separate file.

PRESUBMIT=passed
R=asharif
DELTA=156 (80 added, 73 deleted, 3 changed)
RCL=45364-p2
RDATE=2010/11/22 11:55:53


P4 change: 42606375
14/automation/jobs/echo_job.py
14/automation/jobs/job.py
14/automation/machine_filters.py
14/automation/machine_pools.py
d11de79b9b9953991346b6217f3a23afa4c11a78 15-Feb-2013 raymes <raymes@p4.invalid> Refactored MachineManager a little

PRESUBMIT=passed
R=asharif
DELTA=163 (82 added, 66 deleted, 15 changed)
RCL=45361-p2
RDATE=2010/11/22 11:28:12


P4 change: 42606372
14/automation/client.py
14/automation/job.py
14/automation/job_executer.py
14/automation/jobs/job.py
14/automation/machine_manager.py
14/automation/machine_pools.py
c8fbd3a46474e1cefd1e5eb74815de34ae358e22 15-Feb-2013 raymes <raymes@p4.invalid> Moved the automation utils to the separate utils package to prevent name conflict.

PRESUBMIT=passed
R=asharif,bjanakiraman
DELTA=26 (11 added, 13 deleted, 2 changed)
RCL=45359-p2
RDATE=2010/11/22 09:10:18


P4 change: 42606370
14/automation/machine.py
14/automation/server.py
14/automation/utils.py
14/utils/utils.py
089027857a48e7b2c1d61bd6b68a29453211eda5 15-Feb-2013 raymes <raymes@p4.invalid> Refactored so that job execution happens asynchronously so that multiple jobs can be executing at once.

PRESUBMIT=passed
R=asharif
DELTA=149 (103 added, 32 deleted, 14 changed)
OCL=45330-p2
RCL=45336-p2
RDATE=2010/11/19 14:59:31


P4 change: 42606337
14/automation/job.py
14/automation/job_executer.py
14/automation/job_manager.py
14/automation/jobs/echo_job.py
14/automation/jobs/ls_job.py
14/automation/server.py
3070109e8c8894022f323bf54d35f704f9d2a0ea 15-Feb-2013 asharif <asharif@p4.invalid> Added basic classes for machine management. The actual locking/unlocking,
etc. is not being performed at the moment.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=353 (353 added, 0 deleted, 0 changed)
OCL=45312-p2
RCL=45335-p2
RDATE=2010/11/19 14:44:48


P4 change: 42606336
14/automation/machine.py
14/automation/machine_manager.py
14/automation/machine_pools.py
14/automation/pools.py
0a9be8b53cf365c377baaf046b9769d2f09aa465 15-Feb-2013 raymes <raymes@p4.invalid> Initially automation framework with client/server communication via RPC. No commands are executed on machines yet but this will be the next step.

PRESUBMIT=passed
R=asharif,bjanakiraman
DELTA=130 (130 added, 0 deleted, 0 changed)
OCL=45310-p2
RCL=45325-p2
RDATE=2010/11/19 08:23:53


P4 change: 42606323
14/automation/__init__.py
14/automation/client.py
14/automation/job.py
14/automation/jobs/__init__.py
14/automation/jobs/echo_job.py
14/automation/jobs/ls_job.py
14/automation/server.py
14/automation/utils.py
176213082d94bcb9dccee728984a06a8e9f08b14 15-Feb-2013 asharif <asharif@p4.invalid> Fixed an issue with build_tc and tc_enter_chroot.py where ~ was not being expanded.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=8 (7 added, 0 deleted, 1 changed)
OCL=45311-p2
RCL=45313-p2
RDATE=2010/11/18 18:20:11


P4 change: 42606314
14/build_tc.py
14/tc_enter_chroot.py
0269d46fb1df8d024ec1af78b759cf817cff9e74 15-Feb-2013 asharif <asharif@p4.invalid> Fixed tc_enter_chroot.py to behave more like tc-enter-chroot.sh in accepting
--chrome_root, for example.
Fixed a minor problem in build_chromeos.py.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=28 (22 added, 0 deleted, 6 changed)
RCL=45286-p2
RDATE=2010/11/17 17:50:47


P4 change: 42606283
14/build_chromeos.py
14/build_tc.py
14/tc_enter_chroot.py
3f1517aa6896c7d30f5e0a94b624346f00bdf624 15-Feb-2013 raymes <raymes@p4.invalid> Added initial DejaGNU configuration.

PRESUBMIT=passed
R=asharif,bjanakiraman
DELTA=20 (20 added, 0 deleted, 0 changed)
OCL=45235-p2
RCL=45285-p2
RDATE=2010/11/17 16:53:56


P4 change: 42606272
14/dejagnu/boards/chromeos-machine.exp
14/dejagnu/site.exp
ad46515743c008b94e977208c8134715ce5d8c7a 15-Feb-2013 asharif <asharif@p4.invalid> Fixed small bug in build_chromeos.py.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=2 (0 added, 0 deleted, 2 changed)
RCL=45282-p2
RDATE=2010/11/17 16:37:44


P4 change: 42606269
14/build_chromeos.py
e3668f15af0a1eb6e265c06ab99cc0190857725e 15-Feb-2013 asharif <asharif@p4.invalid> Cleaned up a build_chromeos.py a little (as per gpylint).
Moved the make_chroot code out as a function and build_tc.py now uses that function.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=55 (23 added, 13 deleted, 19 changed)
OCL=45276-p2
RCL=45278-p2
RDATE=2010/11/17 14:26:44


P4 change: 42606263
14/build_chromeos.py
14/build_tc.py
14/tc_enter_chroot.py
252df0f32b5d8986a35ee5ae7dcccd337729b5d1 15-Feb-2013 asharif <asharif@p4.invalid> Converted tc-enter-chroot.sh into tc_enter_chroot.py.
Python scripts are more readable and maintainable.
Also added a USE="mounted_sources" flag to be passed to the ebuild.
The ebuild changes will come in later in the chromium codereview server.
In a few days once we've ported all our scripts to use tc_enter_toolchain.py,
I will remove it from the repo.
Testing:
Did a full/incremental build of the toolchain using mounted sources and
it worked.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=138 (124 added, 5 deleted, 9 changed)
OCL=45248-p2
RCL=45275-p2
RDATE=2010/11/17 10:33:39


P4 change: 42606260
14/build_tc.py
14/tc-enter-chroot.sh
14/tc_enter_chroot.py
8d5ecd27e545f7374a1d2c72ec581850aad6b6da 15-Feb-2013 asharif <asharif@p4.invalid> Updated script to take in a changelist as an option. Also changed the
location of the gcc repository.
Testing:
Tested the changelist option and it seems to be working.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=16 (11 added, 1 deleted, 4 changed)
OCL=45231-p2
RCL=45233-p2
RDATE=2010/11/15 13:03:31


P4 change: 42606221
14/update_git_repo.py
5f6be5fab18b0fceab79938ffeadc5b715691a99 15-Feb-2013 raymes <raymes@p4.invalid> Added PKGDIR spec to setup_board so that it untars the glibc tarballs from the right place.

PRESUBMIT=passed
R=asharif
DELTA=8 (5 added, 0 deleted, 3 changed)
OCL=45178-p2
RCL=45232-p2
RDATE=2010/11/15 12:17:57


P4 change: 42605839
14/build_chromeos.py
6fca6ca0eefeaac14401ccf429e7f948308eb5de 15-Feb-2013 asharif <asharif@p4.invalid> Minor fixes for update_git_repo.py.

Now it catches errors (or Ctrl-C) and cleans up the g4 client.
It also runs git commands with -v flag to capture the verbose output for
debugging.

Testing:
It still doesn't have the final repository information. I tested it locally
with a locally created repository.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=90 (68 added, 1 deleted, 21 changed)
OCL=45201-p2
RCL=45204-p2
RDATE=2010/11/12 19:24:21


P4 change: 42605802
14/update_git_repo.py
00e8b2983912d6cf008902a48fd55bdfc1726214 15-Feb-2013 asharif <asharif@p4.invalid> First crack at writing a script to update the git repository automatically.
1) First, it creates a git client and checks out the toolchain current version
by doing git clone.
2) Then it checks out perforce in the same directory, overwriting all the files.
3) It gets the latest perforce revision CL number.
4) It creates a git log message in a tempfile with a brief message about the CL.
5) It then pushes to the remote repo.

Testing:
I tested it with a local git repo I made on my hard drive and it seemed to work.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=151 (151 added, 0 deleted, 0 changed)
OCL=45179-p2
RCL=45198-p2
RDATE=2010/11/12 11:09:48


P4 change: 42605791
14/update_git_repo.py
d1eed8024f288c612c6d61a4e0008a19072def23 15-Feb-2013 raymes <raymes@p4.invalid> Some misc fixes to get the workflow working properly.

PRESUBMIT=passed
R=asharif,bjanakiraman
DELTA=25 (18 added, 2 deleted, 5 changed)
OCL=45195-p2
RCL=45197-p2
RDATE=2010/11/12 10:51:59


P4 change: 42605790
14/setup_chromeos.py
14/utils/utils.py
9994d2945f8b4724cd41ec1ff4faad1ec1b1726f 15-Feb-2013 asharif <asharif@p4.invalid> Fixed install issue with build_tc.py.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=6 (1 added, 1 deleted, 4 changed)
RCL=45173-p2
RDATE=2010/11/11 14:42:17


P4 change: 42605758
14/build_tc.py
04164a1ff3feb7b71f7bc3de510e60d73a95aa2d 15-Feb-2013 raymes <raymes@p4.invalid> Minor fixes to get setup_chromeos and build_chromeos working with build_tc

PRESUBMIT=passed
R=asharif,bjanakiraman
DELTA=19 (5 added, 3 deleted, 11 changed)
OCL=45153-p2
RCL=45166-p2
RDATE=2010/11/11 11:32:00


P4 change: 42605749
14/build_chromeos.py
14/setup_chromeos.py
2d815d061325da11c540da4b3ed7fe5bfd89915b 15-Feb-2013 asharif <asharif@p4.invalid> Fixed a few bugs in build_tc.py.
Fixed path to the pkgs directory.
Incremental on a package boundary is now working. (crossdev doesn't need
the --newuse portage flag for incremental).
Testing:
Tested with various phases of install. It seemed to pick up from where it
left (across package boundaries, but not within a package).

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=24 (15 added, 4 deleted, 5 changed)
OCL=45155-p2
RCL=45158-p2
RDATE=2010/11/11 11:02:10


P4 change: 42605744
14/build_tc.py
60a066ab5053fd6279c7e36302f2ede6f7f31b4d 15-Feb-2013 asharif <asharif@p4.invalid> Fixed a bug in tc-enter-chroot.sh, where it would not mount directories
properly when it was run from the v14 directory.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=11 (0 added, 3 deleted, 8 changed)
OCL=45137-p2
RCL=45139-p2
RDATE=2010/11/10 14:13:51


P4 change: 42605722
14/tc-enter-chroot.sh
5f35b9216ab970e546121e3626969081ee797f93 15-Feb-2013 raymes <raymes@p4.invalid> Updated build_chromeos.py so that it only does make_chroot/setup_board when they are needed.

PRESUBMIT=passed
R=asharif,bjanakiraman
DELTA=29 (10 added, 0 deleted, 19 changed)
OCL=45114-p2
RCL=45135-p2
RDATE=2010/11/10 13:17:16


P4 change: 42605709
14/build_chromeos.py
6f2b49131ccae73847166774cf19e8ffe694dbde 15-Feb-2013 asharif <asharif@p4.invalid> tc-enter-chroot.sh now mounts 2 directories instead of more, creating symlinks
instead of using extra mount points.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=48 (44 added, 0 deleted, 4 changed)
OCL=45120-p2
RCL=45131-p2
RDATE=2010/11/10 10:52:49


P4 change: 42605707
14/build-gcc/opts.sh
14/tc-enter-chroot.sh
d751e25d6e4e1f00ef3f4197674337c14a90ece7 15-Feb-2013 asharif <asharif@p4.invalid> build_tc.py now installs executables and libraries in install/ dir.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=35 (22 added, 3 deleted, 10 changed)
OCL=45054-p2
RCL=45097-p2
RDATE=2010/11/08 17:45:49


P4 change: 42605664
14/build_tc.py
8c227da7a8233b153db8954adbb749f8093f79a8 15-Feb-2013 asharif <asharif@p4.invalid> Added run_tests.py. It's a simple wrapper around run_remote_tests.sh.

PRESUBMIT=passed
R=bjanakiraman,raymes
DELTA=59 (59 added, 0 deleted, 0 changed)
OCL=45051-p2
RCL=45095-p2
RDATE=2010/11/08 17:14:30


P4 change: 42605663
14/run_tests.py
e91a6e62450cf2f2ef7d16dd4c9d572a99cda788 15-Feb-2013 raymes <raymes@p4.invalid> Some minor updates to build chrome browser inside the chroot.

PRESUBMIT=passed
R=asharif,bjanakiraman
DELTA=51 (19 added, 4 deleted, 28 changed)
OCL=45085-p2
RCL=45088-p2
RDATE=2010/11/08 11:01:23


P4 change: 42605654
14/build_chromeos.py
14/setup_chromeos.py
bfb57990c0fc932e8b644799cd5371ee052e7759 15-Feb-2013 raymes <raymes@p4.invalid> Updated build_chromeos.py to working state.

PRESUBMIT=passed
R=asharif,bjanakiraman
DELTA=133 (91 added, 13 deleted, 29 changed)
OCL=45050-p2
RCL=45052-p2
RDATE=2010/11/05 17:54:50


P4 change: 42605605
14/build_chromeos.py
14/setup_chromeos.py
14/utils/utils.py
123182fe05900b23c93897f821b7b3ce9a96c35a 15-Feb-2013 asharif <asharif@p4.invalid> tc-enter-chroot.sh now creates a v14 dir if it doesn't exist.

PRESUBMIT=passed
R=raymes
DELTA=9 (7 added, 0 deleted, 2 changed)
OCL=45046-p2
RCL=45048-p2
RDATE=2010/11/05 16:10:44


P4 change: 42605604
14/tc-enter-chroot.sh
09bfb6f3c6a8cad98a735459cc80c52ac89f3c61 15-Feb-2013 asharif <asharif@p4.invalid> Moved FEATURES="keepwork" to outside the BuildTC() function.
Tested incremental build and it is working for binutils at least.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=4 (2 added, 2 deleted, 0 changed)
OCL=45030-p2
RCL=45047-p2
RDATE=2010/11/05 16:10:30


P4 change: 42605600
14/build_tc.py
7c851ad841e8b752a176b756c779a241ce52c0aa 15-Feb-2013 raymes <raymes@p4.invalid> Changed tc-enter-chroot to better detect paramaters.

PRESUBMIT=passed
R=asharif
APPROVED=asharif
DELTA=6 (2 added, 1 deleted, 3 changed)
RCL=45044-p2
RDATE=2010/11/05 14:35:12


P4 change: 42605598
14/tc-enter-chroot.sh
e2cca30749c4288e388968c1e3a00f8b22797e7f 15-Feb-2013 asharif <asharif@p4.invalid> Added option to use binaries instead of source when installing
the toolchain in a chroot.
Tested with and without the -B option.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=35 (26 added, 0 deleted, 9 changed)
RCL=45025-p2
RDATE=2010/11/04 16:54:57


P4 change: 42605578
14/build_tc.py
80c6e551b2635d5997024d8a1a81402bbfad73c6 15-Feb-2013 asharif <asharif@p4.invalid> Modified build toolchain scripts.
Added -f option to rebuild (after uninstalling) for convenience.
Added mount point for logs, etc.
emerge now uses usermode intead of root to compile. Now you can access the
toolchain build without using sudo.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=17 (10 added, 0 deleted, 7 changed)
OCL=45010-p2
RCL=45022-p2
RDATE=2010/11/04 12:56:54


P4 change: 42605566
14/build_tc.py
14/tc-enter-chroot.sh
5154d7f63f80b2df5ff4b3c29b727d0c6ce21239 15-Feb-2013 raymes <raymes@p4.invalid> Updated logging to allow it to be printed to the console as well as
file. Updated build_chromeos.py to use logging properly.

PRESUBMIT=passed
R=asharif,bjanakiraman
DELTA=232 (183 added, 35 deleted, 14 changed)
OCL=44971-p2
RCL=45003-p2
RDATE=2010/11/03 17:32:32


P4 change: 42605541
14/build_chromeos.py
14/setup_chromeos.py
14/utils/utils.py
19c73dd90215e830898c31a9c83d900065a16fce 15-Feb-2013 asharif <asharif@p4.invalid> Rewrote build_tc.py.
It now has callable functions and uses crossdev instead of setup_board.
This way it is more flexible and things can be reinstalled and be
incrementally built without changing setup_board.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=98 (78 added, 13 deleted, 7 changed)
OCL=44955-p2
RCL=45002-p2
RDATE=2010/11/03 17:22:34


P4 change: 42605540
14/build_tc.py
1688613ba3c887183052f14eb9a41b16dbbcfadc 15-Feb-2013 asharif <asharif@p4.invalid> Fixed a bug in tc-enter-chroot.sh. Fixed error reporting.

PRESUBMIT=passed
R=raymes,bjanakiraman
DELTA=4 (0 added, 1 deleted, 3 changed)
RCL=44950-p2
RDATE=2010/11/01 14:45:11


P4 change: 42605478
14/tc-enter-chroot.sh
7f4a4859b8048cd40513a5f3ba49c406ed107cbb 15-Feb-2013 bjanakiraman <bjanakiraman@p4.invalid> ChromeOS validation scripts

PRESUBMIT=passed
R=raymes,asharif
DELTA=336 (336 added, 0 deleted, 0 changed)
OCL=44912-p2
RCL=44939-p2
RDATE=2010/11/01 09:44:34


P4 change: 42605469
14/build_tc.py
14/setup_chromeos.py
14/tc-enter-chroot.sh
14/utils/__init__.py
14/utils/utils.py
859d1ab92f807071f2e05f3b5db2b0cf83c5ebbb 01-Jan-1970 git5 <git5@invalid> git5: git5track synced with perforce at 0
EADONLY
9b16723fcf646ece404824d06237ab0c5ae84b37 01-Jan-1970 git5 importer <git5@invalid> initial import: //depot/depot2/gcctools/chromeos/v14
Server default, From 1, strip //depot/depot2/gcctools/chromeos, HIP=0


P4 change: 1
git5_specs/H7c3295650ff892250c9c8f5086e5f120