History log of /external/autotest/tko/parsers/version_0.py
Revision Date Author Comments (<<< Hide modified files) (Show modified files >>>)
23dbf4987dc849d4214beb9761de62b76ee2fdc9 13-Nov-2017 Dan Shi <dshi@google.com> [autotest] Change tko machine_group name from board label to model label

BUG=chromium:780893
TEST=unittest, local parse.

Change-Id: I8346c67e837a67d06cd4a285552c9b66c251b5c1
Reviewed-on: https://chromium-review.googlesource.com/767109
Commit-Ready: Dan Shi <dshi@google.com>
Tested-by: Dan Shi <dshi@google.com>
Reviewed-by: C Shapiro <shapiroc@google.com>
/external/autotest/tko/parsers/version_0.py
d8561576404bcb7c3c56d1c691129ba285693d03 04-Feb-2017 Prathmesh Prabhu <pprabhu@chromium.org> [tko] Sanitize imports

BUG=chromium:682417
TEST=None

Change-Id: I876698190f052d5673083bbe91729866457e5af6
Reviewed-on: https://chromium-review.googlesource.com/437909
Commit-Ready: Prathmesh Prabhu <pprabhu@chromium.org>
Tested-by: Prathmesh Prabhu <pprabhu@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
/external/autotest/tko/parsers/version_0.py
e70648162260bfd46a56fd1a1d1d7f86ec01e600 03-Feb-2017 Luigi Semenzato <semenzato@chromium.org> TKO parser: improve some error messages in debug log

These changes try to make it clearer that the errors
aren't critical to the execution of the test, and also
slightly reduce the spew to improve log readability.

BUG=chromium:659741
TEST=none

Change-Id: I1a2332becb6d6d1fbd5f23f39dbbc502a631d37c
Reviewed-on: https://chromium-review.googlesource.com/437707
Commit-Ready: Luigi Semenzato <semenzato@chromium.org>
Tested-by: Luigi Semenzato <semenzato@chromium.org>
Reviewed-by: Prathmesh Prabhu <pprabhu@chromium.org>
Reviewed-by: Ilja H. Friedel <ihf@chromium.org>
/external/autotest/tko/parsers/version_0.py
adf313165f835de2e47ad9e8d4ee9fab388e6a7c 28-Jun-2016 Simran Basi <sbasi@google.com> autotest: Add suite support for testbeds.

Adds basic suite support for Android testbeds. Note this does not
support SSP and only uses the Autotest code that is currently
pushed to prod.

* Added a new provision test (provision_TestbedUpdate).
* Added a basic suite to test this workflow.
* Updated the parsers to allow for multiple boards.
* Had to adjust the provision control_segment to allow commas
in the key-value labels.

BUG=chromium:628415,chromium:628036
TEST=./site_utils/run_suite.py --board=dragonboard-2 \
--build=git_mnc-brillo-dev/dragonboard-eng/3014741,\
git_mnc-brillo-dev/dragonboard-eng/3014741 \
--suite_name=dummy_testbed --pool='' --run_prod_code

Change-Id: Ica04911974d69498877b9cedb9cefc4a5d0bcdd5
Reviewed-on: https://chromium-review.googlesource.com/360663
Commit-Ready: Simran Basi <sbasi@chromium.org>
Tested-by: Simran Basi <sbasi@chromium.org>
Reviewed-by: Kevin Cheng <kevcheng@chromium.org>
Reviewed-by: Dan Shi <dshi@google.com>
/external/autotest/tko/parsers/version_0.py
c8a2a8243228b36566684a9d69b124c702004c28 13-Nov-2015 Dan Shi <dshi@chromium.org> [autotest] Use board label as machine group for tko_machine in paser job.

This change is to help wmatrix to group tests with board label, rather than
platform.

BUG=chromium:555675
TEST=run test locally, verify machine_group in a new tko_machine entry showes
the board label as machine_group.

Change-Id: If1c514fc427fa5ee223986513349665cc741fbd7
Reviewed-on: https://chromium-review.googlesource.com/312566
Commit-Ready: Dan Shi <dshi@chromium.org>
Tested-by: Dan Shi <dshi@chromium.org>
Reviewed-by: Fang Deng <fdeng@chromium.org>
/external/autotest/tko/parsers/version_0.py
51976cdb7c8dfa85f4d53c659e3b836d1435e4f9 12-Aug-2011 Dale Curtis <dalecurtis@chromium.org> Parse reason out of logs for display in results emails.

A recent upstream Autotest merge has reduced the effectiveness of
reporting error logs alone. With this patch I've added the contents of
the reason field to the beginning of the log data.

Due to this change, I've reduced the number of lines returned from the
error log from 25 to 10.

I also fixed a parser error which caused reasons with an equal sign in
them to be treated as optional parts.

BUG=none
TEST=ran parser against results. running in production.

Change-Id: I2a5970013f1b8b04f21860af66974a5fb65ba864
Reviewed-on: http://gerrit.chromium.org/gerrit/5905
Reviewed-by: Nirnimesh <nirnimesh@chromium.org>
Tested-by: Dale Curtis <dalecurtis@chromium.org>
/external/autotest/tko/parsers/version_0.py
74a314b490ff542c4dd2ae4aa0d11c6394d92960 23-Jun-2011 Dale Curtis <dalecurtis@chromium.org> Upstream Autotest merge.

As titled, a merge up to 93fc426ca133e775eb495f34d138fc57d92fb55e.

- Removes a bunch of deprecated code.
- Moves several private utilities into the private repo.
- Couple changes ported upstream and resynced.

BUG=None
TEST=In progress... will spin up new Autotest server and use
run_remote_tests for bvt, regression, smoke.

Change-Id: Id3e2ad529bb7b05f148e5d98aea46bb9ea828200
Reviewed-on: http://gerrit.chromium.org/gerrit/3350
Tested-by: Dale Curtis <dalecurtis@chromium.org>
Reviewed-by: Dale Curtis <dalecurtis@chromium.org>
/external/autotest/tko/parsers/version_0.py
e5436f3561713c9f68eebc4c9347ffb59e36bd8f 31-Mar-2011 Dale Curtis <dalecurtis@chromium.org> Add new site parser based off on generate_test_report.

This site parser is used to generate a JSON report of test failures, crashes,
and the associated logs for later consumption by an Email generator.

The parser uses the test report generator which comes bundled with the Chrome
OS source tree in order to maintain consistency. As well as not having to keep
track of any secondary failure white lists.

The path to the Chrome OS source tree is defined in global_config under the
CROS section as 'source_tree'.

Existing parse behavior is kept completely intact. If the site parser is not
configured it will print a debug message and exit after default parser is
called.

Relies on changes to generate_test_report here:

http://codereview.chromium.org/6731024/

Change-Id: Ib4696b88a22ca67b0822a72366d57709a5863ae5

BUG=chromium-os:12100, chromium-os:13586, chromium-os:13496
TEST=Ran parser on many sets of results.

Review URL: http://codereview.chromium.org/6758040
/external/autotest/tko/parsers/version_0.py
e8dac7a116f39ab5a20c3dcb15bab4446fcfbb81 20-Oct-2010 Dale Curtis <dalecurtis@chromium.org> With this change the parse_line method will no longer throw assertion errors when reasons contain tab characters or invalid key=value pairs.

As a side effect, improperly formatted key=value optional parts (and all valid key=value pairs after) will be rolled into the reason.

I've added a new unit test, test_parse_line_handles_tabs_in_reason, and removed the now deprecated unit test, test_parse_line_fails_on_bad_optional_fields.

A previous CL for multi-line support also used strip() on the status line when the pre-CL behavior would only remove trailing '\n'. I've reverted the code to reflect the original behavior.

I'm open to suggestions on a more complete regex for matching key names. For our test base, \w is sufficient.

Change-Id: I351a61080dd335aa591b90565b5cc53f558641e0

TEST=Unit tests plus in production runs.

Review URL: http://codereview.chromium.org/3885001
/external/autotest/tko/parsers/version_0.py
7b2c643370076b638a8c6d8c4a63d0fc399b9153 12-Oct-2010 Dale Curtis <dalecurtis@chromium.org> Add support for multi-line status messages.

Many of our Autotest test cases return multi-line status messages upon failure. Due to the regular expression used by Autotest these lines cause an "AttributeError: 'NoneType' object has no attribute 'groups'" exception to be thrown. By modifying the regular expression to allow '.' to match '\n' these errors are avoided.

I've added a new unit test, test_parse_line_handles_embedded_new_lines(), to check for these types of messages.

Change-Id: I0b3f4d90caf2cfc81724b928f6bea90eec6c38b0

BUG=7611
TEST=Ran unit tests. Ran in production system over the weekend.

Review URL: http://codereview.chromium.org/3719002
/external/autotest/tko/parsers/version_0.py
f98230c1b7254c0fae0048b855c37d696cbf44d4 11-Jun-2010 jamesren <jamesren@592f7852-d20e-0410-864c-8624ca9c26a4> Fixed a minor bug in version_0.py. The constructor was missing an argument. The number of arguments to the version_0.test constructor was inconsistent with that of test constructor in version_1 and model.

Signed-off-by: Darren Kuo <darrenkuo@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@4597 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/tko/parsers/version_0.py
c1a98d1e146080bd3e4f034cb13d740dfb1535f4 15-Jan-2010 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Support for job keyvals
* can be passed as an argument to create_job, stored in AFE DB
* scheduler reads them from the AFE DB and writes them to the job-level keyval file before the job starts
* parser reads them from the keyval file and writes them to the TKO DB in a new table

Since the field name "key" happens to be a MySQL keyword, I went ahead and made db.py support proper quoting of field names. Evetually it'd be really nice to deprecate db.py and use Django models exclusively, but that is a far-off dream.

Still lacking support in the AFE and TKO web clients and CLIs, at least the TKO part will be coming soon

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@4123 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/tko/parsers/version_0.py
71b9431ef91c07e45d7b780d321f1d771bdc9c75 21-Aug-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Make the parser read and use the machine platform from host_keyvals, if available. Goodbye machine_transfer! Also made it handle multimachine jobs (use a comma-separated, sorted list of distinct platforms from the individual machines) and atomic group jobs (use the same atomic group override value that was used for machine name, when applicable).

Signed-off-by: Steve Howard <showard@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3583 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/tko/parsers/version_0.py
1ef218db76c473c28627377d8f50d6e6c6743289 03-Aug-2009 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> This is the result of a batch reindent.py across our tree.
As Martin pointed out, we ought to be more careful and
create a pre-svn commit script to avoid inserting trash
in the tree, meanwhile, this is a good start to cleanup
things

Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3487 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/tko/parsers/version_0.py
5350a34676f05c45fdbf3c2b9ba234b7c6ed292a 26-May-2009 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Always use the host_group_name even if a sysinfo/hostname file exists
when the keyvals hostname is a list of machines.

Fixes the version_0 host_group_name unittests to stub out find_hostname
so that local files are not read. This also verifies that find_hostname
is not called when a host_group_name is present.

Signed-off-by: Gregory Smith <gps@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3178 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/tko/parsers/version_0.py
f1ae354808a2eeb95d706a669250b613765212a4 11-May-2009 showard <showard@592f7852-d20e-0410-864c-8624ca9c26a4> Represent a group of machines with either the atomic group label name,
if a specific label was used, or the atomic group name in the results
database when parsing.

Adds an optional host_group_name= to the server side group job keyval
file. The scheduler choses the most appropriate name for this and adds
it to the group keyvals file.

Changes the TKO results parser to use host_group_name= as the machine name
instead of hostname= when hostname= is a comma separated list of hostnames
rather than a single name.

Also fixes atomic group scheduling to be able to use up to the atomic group's
max_number_of_machines when launching the job; This is still unlikely to
happen as the code still launches the job as soon as at least the sync count
have exited Verify.

Signed-off-by: Gregory Smith <gps@google.com>


git-svn-id: http://test.kernel.org/svn/autotest/trunk@3103 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/tko/parsers/version_0.py
42fbd071a1810af071699f1261861829789c4c13 30-Jan-2009 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Add some more slack to the parser when dealing with multi-machine
jobs. If the job isn't neatly broken down into machine-specific
subjobs, just accept it and use the set of hostnames as the
"hostname". This should allow us to do meaningful parsing of
multi-machine test results, even if things aren't as intelligent
as we'd like.

Risk: Low
Visibility: Doesn't affect the parsing of anything that the parser can
currently handle; allows us to do SOME parsing of generic
multi-machine results.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2720 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/tko/parsers/version_0.py
4e16fee729bed6d4314b21fc41f83d0c27ade7fc 17-Nov-2008 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> We have a silly assertion in the parser that doesn't really matter
anymore; the keyval is the easiest place to find hostnames, but
for multimachine jobs it's not always going to give us an
unambiguous result. That's okay though, it's easy enough to find
the hostname from the individual machine directories.

Risk: Low
Visibility: We can parse multi-machine jobs.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2425 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/tko/parsers/version_0.py
f6ff3a020920508e945fac12ab0ea0525d0fec6c 04-Nov-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> sysinfo-uname test attribute should exclude hostname.
Unreported bug: return val was missing in an error recovery path in
job.find_hostname() in tko/parsers/version_0.py.

Signed-off-by: Duane Sand <duanes@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2371 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/tko/parsers/version_0.py
531d5976a174c04fb6f03396e182d4cc4014940b 04-Nov-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> When building a kernel we will use the first build, build.2, build.3, and
so on. For TKO jobs we assume that the first build is the kernel under
which all the tests are running and use that for reporting. However,
when the build occurs in a group and that group is called build the new
group code make a results directory called build. Subsequently when
we then run the build within that group it will find there is a results
directory called build and move on to build.2.

To catch this case search for the first populated build directory and
use the logs within that to determine the version of the kernel this
test job is testing.

Signed-off-by: Andy Whitcroft <apw@shadowen.org>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2370 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/tko/parsers/version_0.py
f7fa2cc6159fda3adecef2eeb5e5a016e866564c 01-Oct-2008 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Update the scheduler and the parser to use the new aborted_* attributes that
are written into the database on abort.

Risk: Medium
Visibility: Better error messages from the parser when a job is aborted.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2217 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/tko/parsers/version_0.py
a8e302a2131a7e0b0d1e0ad746df264a99f01e8b 25-Sep-2008 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Change the job keyval loading code to do a search up the filesystem
for the "real" job keyval file.

Risk: Medium
Visibility: Multi-machine server jobs will pull in all job attributes.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@2197 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/tko/parsers/version_0.py
008ff1d2503cf2fad3d4ef402661f38ed6bd8572 14-Aug-2008 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Made the "is_status_line" method the parser uses for filtering out
non-status lines more picky...it shouldn't let any lines through that
the subsequent parsing code can't handle.

Risk: Low
Visibility: The later parsing code should now never throw
"need more than X value to unpack" errors.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1988 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/tko/parsers/version_0.py
b14dc7aa8eeb0a80eba299a04343c74e509e7c6b 12-Jul-2008 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Change the name of the magic JOB entry in the status log parsing to
CLIENT_JOB, since that's what it actually corresponds to.

Risk: Low
Visibility: Changes what will show up in the tko results (again).

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1826 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/tko/parsers/version_0.py
ba1fa66c641651bcf0d058e9f5af2819c2d2690a 11-Jul-2008 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Change the way we handle "JOB" logs in the parser. Currently, we
generate multiple JOB entries if you run multiple client jobs, and
we handle their status the same way we handle the status of nested
groups in a test - that is, if a job runs any tests that fail we
mark the jobs as failed.

This makes two changes:
1. it tags jobs with a number, the same way we tag boots, so that
multiple jobs will produce multiple result entries instead of
duplicate entries
2. it disables tagging of jobs with statuses like WARN and FAIL
that generally apply to tests but not the job as a whole, so
that a job only shows up as "bad" if it was actually aborted

Risk: Medium
Visbility: This significantly changes the way "JOB" results will show
up in the TKO result view.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1820 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/tko/parsers/version_0.py
0afbb6369aa5aa9a75ea67dd9e95ec4b21c0c181 06-Jun-2008 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Convert all python code to use four-space indents instead of eight-space tabs.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1658 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/tko/parsers/version_0.py
004fde6c11387148975cb4c4ad03adcaddbb8188 04-Jun-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Fix up parsing of test versions to not add a newline

Signed-off-by: Martin J. Bligh <mbligh@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1598 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/tko/parsers/version_0.py
cc549174c6caf563443625805594eb0ff813a261 21-May-2008 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Risk: Medium
Visibility: Changes the test keyval interface, deprecating the existing
test.write_keyval method in favour of a pair of write_*_keyval methods
for writing test & iteration keyvals. The deprecated method will still
work as it did before, but will generate a warning.

Adds a new iteration_attributes table to the database for storing
generic string attributes on a per-iteration basis, in the same way
that test_attributes allows generic string attributes on a per-test
basis.

This also adds new methods to the test class for writing these keyvals
so that tests can write out attributes by calling
self.write_test_keyval (or self.write_iteration_keyval). The iteration
method accepts parameters for both generic attributes and performance
data.

In order to store both performance and non-performance data in the
iteration keyvals, the format of the line has been extended to look
like "key{blah}=value", with no {blah} being interpreted as equvalent
to "{perf}", for backwards compatiblity.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1535 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/tko/parsers/version_0.py
6e8bf75b13c28c5424fb447ceb4d8749c7a13251 14-May-2008 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Add the version 1 of the TKO parser, modify server_job to use this
version, and modify parse.py to look up the status version number
in order to instantiate the correct parser version.

Basically, it's an implementation of the parser that follows
the spec as outlined in
http://test.kernel.org/autotest/DraftParserSpecification. I did that
by implementing the "version 1" parser, as opposed to the existing
"version 0" parser, and it also adds some code to autotest itself to
log the fact that the status logs being written out follow the
"version 1" specification, so that when re-parsing existing results
older logs will still be parsed using the old (rather ad-hoc and
difficult to follow) algorithm.

The implementation is fairly similar to the existing version 0
implementation; it still uses all the same files for gathering
results, but there are the core changes:
- instead of grabbing kernel information from build.log, it gets
embedded into the status logs associated with reboots
- if a group is lacking a proper "END" section it implicitly assumes
it was somehow aborted
- reboots are wrapped in a group
- a "JOB" result is logged even if nothing bad happens (rather than
only logging it when something bad happens)
- you can have arbitrarily large amounts of group nesting

Signed-off-by: John Admanski <jadmanski@google.com>




git-svn-id: http://test.kernel.org/svn/autotest/trunk@1511 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/tko/parsers/version_0.py
a9c75c46f3b887cf6aff55bf76075c70d3f8c183 02-May-2008 jadmanski <jadmanski@592f7852-d20e-0410-864c-8624ca9c26a4> Run client-side reboots in a test group, and include information
about the kernel booted in the group END logging.

This required enhancing the client-side job code to preserve the
logging indentation across reboots.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1480 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/tko/parsers/version_0.py
9cae15020efc8d541fe3def1b9034872d0b919f4 18-Apr-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Add tests.started_time to the results database, and add support in the
parser for extracting those times.

Signed-off-by: John Admanski <jadmanski@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1453 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/tko/parsers/version_0.py
2895ce5646e363187205a4e6dae37d603ca44382 17-Apr-2008 mbligh <mbligh@592f7852-d20e-0410-864c-8624ca9c26a4> Add more missing files from previous commits. Did I mention I hate svn?

Signed-off-by: Martin J. Bligh <mbligh@google.com>



git-svn-id: http://test.kernel.org/svn/autotest/trunk@1451 592f7852-d20e-0410-864c-8624ca9c26a4
/external/autotest/tko/parsers/version_0.py