| 42bc23b188cdf856d24494842909ac0e0930b257 |
|
19-Sep-2014 |
Charlie Mooney <charliemooney@chromium.org> |
Add a manual noise test mode to the Touch FW tests
Elan wants to reproduce our robotic noise test using their robot. After
some brainstorming with them on the best way to do this, we devided that
actually making a mode that does the noise test "manually" would
probably be easiest. Then they can program their robot to simply touch
the pad for a few seconds, raise up, wait a few more seconds, and then
hit the spacebar. It should work just fine I think and is a lot easier
than rewriting the robot wrapper classes to support their robot.
This CL adds a new flag "-f" that allows you to indicate to the test
suite that you do, in fact have a function generator attached even
though the robot may not be connected. So to run the extended noise
test manually you would run:
DISPLAY=:0 python main.py -m NOISE -f
The test will controll the function generator automatically for you, and
then you are manually required to perform the gesture.
BUG=chromium:416062
TEST=manual testing. The new mode works and the regular modes that
existed before seem to still work as expect as well
Change-Id: I12110ca30422f0d3f7455654253675834fc19d9c
Signed-off-by: Charlie Mooney <charliemooney@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/219101
/external/autotest/client/site_tests/firmware_TouchMTB/test_conf.py
|
| 60a345dd2a6a7c27407153b6c9734d2b63a88c01 |
|
28-Jul-2014 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: add MtbSanityValidator
This patch checks two things.
1. Ghost fingers:
The validator checks if there exist ghost fingers before fingers
touch the touch device. This is accomplished by querying the
InputDevice class which syncs with the kernel evdev driver about
the mt slots information.
2. MTB event rules:
The MTB events emitted by a device have to follow some rules.
Otherwise, the events may confuse the state machine in the MTB
parser. As a beginning, this validator checks some simple rules:
a. No TRACKING ID is set to -1 before a slot is assigned a
positive TRACKING ID value.
b. No X, Y, or PRESSURE values could be assigned to a slot
before the slot is assigned a positive TRACKING ID value.
More rules may be added to this validator when needed.
Note that item 1 above is to detect any pre-existing fingers
before the test is conducted, while item 2 is to detect any event
mis-ordering or corruption occurring during the test.
BUG=chrome-os-partner:30965
TEST=All unit tests should pass on a chromebook.
$ cd /usr/local/autotest/tests/firmware_TouchMTB
$ python tests/run_all_unittests.py
Change-Id: Id85dcb8e54601f5dfd6d77bbf3802fdcb2d74a60
Reviewed-on: https://chromium-review.googlesource.com/210792
Reviewed-by: Charles Mooney <charliemooney@chromium.org>
Commit-Queue: Shyh-In Hwang <josephsih@chromium.org>
Tested-by: Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/test_conf.py
|
| d709ccc114fd7e4ffd21abcb160181d2d12243d4 |
|
16-Aug-2014 |
Mindy Huang <mindyh@chromium.org> |
Added a new validator that discards the initial seconds of data before
validation.
The new validator can wrap around any generic validator, and the number
of seconds to discard can be specified.
The validator simply strips the first second and prepends
the end state of the discarded second onto the stripped data. This
throws away all events that happened within the first second, but keeps
all events that persist after the first second.
The noise tests also now have a one second pause at the beginning of
each test to allow the touchpad to calibrate against the noise.
BUG=chromium:404265
TEST=Manually and with robot
Change-Id: I6d6a0c40f4cb106185c7cc084fca3b459dd49c3c
Signed-off-by: Mindy Huang <mindyh@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/212763
Reviewed-by: Charles Mooney <charliemooney@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/test_conf.py
|
| f11c971f67f255d8093e5cee05e5da0e6446d900 |
|
30-Jul-2014 |
Mindy Huang <mindyh@chromium.org> |
Set the robot to only pick up the necessary one finger during noise tests.
If it picks up all four as is the default setting, the noise resonates between
the fingers and generates much more and much stronger phantom touches.
BUG=chromium:398652
TEST=Manually with robot
Change-Id: I55d1d84db2b3032836287136d4bdbbbbad6adf8e
Signed-off-by: Mindy Huang <mindyh@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/210411
Reviewed-by: Charles Mooney <charliemooney@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/test_conf.py
|
| b0612e11b87281c1ab8ab95d3c47de2601a9df26 |
|
16-Jul-2014 |
Mindy Huang <mindyh@chromium.org> |
Added a noise mode to the robot.
In noise mode, the robot does a fine-grain frequency test on the device.
It sweeps from 0Hz to 1MHz in 400Hz intervals, for a total of 2500 tests
in about 4 hours.
BUG=chromium:394075
TEST=Manually with robot on a lumpy
Change-Id: Iaf5901923f42cbd41c0a263a0431b512f6cdbb36
Signed-off-by: Mindy Huang <mindyh@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/208485
Reviewed-by: Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/test_conf.py
|
| 94baca13ef3518e592fd9d0d1a8fec8f6a3bef93 |
|
18-Jul-2014 |
Charlie Mooney <charliemooney@chromium.org> |
Add noise-tests to the touchscreen capabilities
The noise test was only enabled on touchpads, but there's no reason it
can't work on touchscreens as well. This CL just adds them to the list.
BUG=none
TEST=I manually used this for a test today
Change-Id: I91d4311170c1648afd6afea45c4656775aa7ad4d
Signed-off-by: Charlie Mooney <charliemooney@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/208981
Reviewed-by: Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/test_conf.py
|
| a8b149d119a2a0cf7d7e403bc3df7cc9f34cdc8c |
|
09-Jul-2014 |
Mindy Huang <mindyh@google.com> |
Added noise tests for line drawing and stationary touches to the touch firmware test suite.
The added line test draws diagonal lines with square waves in noise at 5kHz, 500kHz, and 1Mhz.
The added stationary touch test holds the finger still at various points on the touchpad while sending
square waves in noise at 5kHz, 500kHz, and 1MHz.
BUG=chromium:392321
TEST=Tested with robot on a lumpy
Change-Id: I6b331346d11ca2c88510f16a92a27aefaf2d38b1
Signed-off-by: Mindy Huang <mindyh@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/207167
Reviewed-by: Charles Mooney <charliemooney@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/test_conf.py
|
| 8463668f2fe204b3a2ca7b9d53a8bd399ad1ee28 |
|
25-Jun-2014 |
Charlie Mooney <charliemooney@chromium.org> |
Change which robot fingertip is "normal"
From what I've been seeing, our "normal" fingertips are actually
somewhat larger than most fingers. Additionally, one of them is
damaged, so that makes a compelling argument to use the size 2mm smaller
for normal fingertips now. This CL changes the assignments.
BUG=none
TEST=I ran the FW test and it worked
Change-Id: I61918f215dd7b81d14e606a927a776eb67468bec
Signed-off-by: Charlie Mooney <charliemooney@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/205472
Reviewed-by: Andrew de los Reyes <adlr@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/test_conf.py
|
| 662a357b78e31ad0e9c387389cc54251a7f70946 |
|
29-May-2014 |
Charlie Mooney <charliemooney@chromium.org> |
Increase the number of expect tracking IDs
The robot performs 50 taps in the "stationary finger with taps around
it" test. Previously the test only validated that there were >=2
tracking IDs, which would indeed be bad if there was only 1, but since
there should be 51, it's a little lax in the situation where the taps
are getting missed. This CL sets it to 25 instead. This test is quite
hard, so even good touchpads will miss a few tracking IDs (hence 25
instead of 50) and this gives them a little wiggle-room. From the
performance I've seen on other devices, this seems like a reasonable
cutoff.
BUG=chromium:378846
TEST=none, this is just a simple numeric change
Change-Id: I31e95eabfe5fafad228166d8d525b17148840134
Signed-off-by: Charlie Mooney <charliemooney@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/202067
/external/autotest/client/site_tests/firmware_TouchMTB/test_conf.py
|
| 2cbb23bb88733c3e0a40f7d17681ed957b54ce27 |
|
13-May-2014 |
Charlie Mooney <charliemooney@chromium.org> |
Add a quickstep flag to the touch FW test
It's common to want to check the latency of a system without running the
full (long) FW test on it. This CL adds a nice, simple way to do that
without having to know how the code works and comment out parts you
don't want. Now there is an additional '-q' or '--quickstep' flag you
can pass to the test suite when you run it. If you do it overrides the
list of gestures and only runs the latency test.
BUG=chromium:372895
TEST=I used it both with and without the flag and it seems to do the
job.
Change-Id: Ia05b771c00341447e253d56a13a7c08e967d8401
Signed-off-by: Charlie Mooney <charliemooney@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/199495
Reviewed-by: Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/test_conf.py
|
| bfd222d9cb8b6d0d651ed2ec74f73195718a9e36 |
|
11-Apr-2014 |
Charlie Mooney <charliemooney@chromium.org> |
Adjust finger position to prevent collisions
Since the robot used relative coordinates that scale based on the size
of the touchpad being tested, for smaller touchpads the fingers can
sometimes bump into eachother. This causes a weird bend in the logs and
frankly is just not what we're going for.
This CL adds in some logic to the touch robot test that trying to take
into consideration the size of the touchpad before proceeding and
adjusts the stationary finger's position if it's too close/far away.
BUG=chromium:354593
TEST=manual testing
Change-Id: If2d3ee3da7a37e902c0a06f69ea6cdb6395526cf
Signed-off-by: Charlie Mooney <charliemooney@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/198420
Reviewed-by: Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/test_conf.py
|
| 3ceb6419c90576ba7df8cee03d4b42d38d1b6d00 |
|
05-Mar-2014 |
Charlie Mooney <charliemooney@chromium.org> |
Adding a drag-latency test to the Touch FW test
A drag latency test has been part of our touch qualification process for
a while now, but has always been run by hand. With Quickstep, this
should be able to be performed automatically as with any other
gesture/validator that we check. The operator will just need to set up
the laser across the pad before running the test.
This CL adds in the robot support for the Quickstep gesture and a new
validator that imports in the latency calculation software from the
touchbot project and computes the drag-latency using the touch data the
firmware collected and pulling the laser data out of the Quickstep sysfs
entries. If there is no Quickstep present, the test will report a
failing dummy value to indicate this, otherwise you would need the
Quickstep device to run the FW test, which is something we don't want to
be a requirement.
BUG=chromium:349174
TEST=Manually ran the test with and without the Quickstep plugged in and
the gesture ran smoothly and the computations worked without crashing
Change-Id: I638da3f837e86d6cf40da3e552c7d56b23e51ee1
Signed-off-by: Charlie Mooney <charliemooney@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/189502
/external/autotest/client/site_tests/firmware_TouchMTB/test_conf.py
|
| bc9dba0f212df1b31ed43032c68f5e07e9ee9974 |
|
01-Mar-2014 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: add version info to test logs
When recording gestures files or generating the test summary,
we would like the version info to be prepended to the test logs,
including the standard output, the html report, the pickle log,
and the summary report.
BUG=chromium:315787
TEST=Install the test on a chromebook machine as a first step.
Then, in the chroot of your host:
(cr) $ cd .../site_tests/firmware_TouchMTB
(cr) $ ./version.sh -r ${MACHINE_IP}
On the chromebook, record a few gestures, and then press 'x' to exit.
$ DISPLAY=:0 python main.py -m complete
Now, check the version info in the following files:
$ cd /var/tmp/touch_firmware_test/latest
$ less touch_firmware_report-xxxx
$ less touch_firmware_report-xxxx.html
$ less touch_firmware_report-xxxx.log
The version info should show up in the first few lines look like
test_version: 2014-02-28.06:46:04 5db2d13bf7ff0e56c652213434f581ce6d7a65ee
Change-Id: I549d97574dd44d2cdd7eea7efe0de85a5fd67f66
Reviewed-on: https://chromium-review.googlesource.com/188562
Reviewed-by: Charles Mooney <charliemooney@chromium.org>
Tested-by: Shyh-In Hwang <josephsih@chromium.org>
Commit-Queue: Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/test_conf.py
|
| d8eebd84b931e4f98fc0d7a0899569478f70136b |
|
27-Feb-2014 |
Charlie Mooney <charliemooney@chromium.org> |
Touch FW test: do our own fingertip management for clicks
Previously the click gesture required that you start with no fingertips
selected and the gesture itself would take take of that for you. This
was okay-ish because there wasn't a system in place to make sure the
robot had the right fingers. Now, the touch FW test has progressed such
that we can tell it what fingers are expected for this gesture and it
will make sure that it's ready. By letting the FW test take care of the
fingertip selection, we save on fingertip swaps between repeated
instances of the same gesture. That is to say, between two runs of a
one-finger physical click, there's no reason to put the fingertip back,
and this makes that possible.
BUG=chromium:347737
TEST=manual testing
Change-Id: I9997058213300a05deb74c479cfa9c50cfa8fbd9
Signed-off-by: Charlie Mooney <charliemooney@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/188257
Reviewed-by: Dennis Kempin <denniskempin@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/test_conf.py
|
| 26c71bf6a4fcd7dceef44b9abd534c288dbcd08e |
|
19-Feb-2014 |
Yuli Huang <yuli@chromium.org> |
Touch FW test: robot fat finger w/ resting finger
This adds the support for the Fat finger + resting finger interaction
tests in the robot wrapper for the touch FW test. In this test the
robot uses a regular-size finger in a stationary position and then moves
the fat finger around. This CL adds the plumbing to allow this to
happen.
BUG=chromium:334460
TEST=I ran the tests and observed the robot
Change-Id: Ie7ac488319819261033e8971c20cc8bc507c9ca7
Signed-off-by: Charlie Mooney <charliemooney@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/187723
Reviewed-by: Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/test_conf.py
|
| 6aa260407561643fe714a72b1a9134d14b6c8bb9 |
|
19-Feb-2014 |
Charlie Mooney <charliemooney@chromium.org> |
Touch FW test: Fail over robot lib directories
This CL changes the Touch FW test to check in the gmerge and cros deploy
directories first, when looking for the robot libraries. This way if
you update the version of the touchbot project on your machine it'll
still work they way you expected.
BUG=chromium:344921
TEST=deployed on a machine with gmerged touchbot libs and the FW test
worked
Change-Id: I4521f5ab1be70ac52aa9d096dcb61d0c8dbbc170
Signed-off-by: Charlie Mooney <charliemooney@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/187071
Reviewed-by: Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/test_conf.py
|
| 02902d82dbd62f806879f3768eb3f4764a913699 |
|
07-Feb-2014 |
Charlie Mooney <charliemooney@chromium.org> |
Touch FW test: add robotic fat-finger support
The fingertip size selection process for the touchbot wrapper for the
touch FW test was somewhat crude before. All the gestures that it
supported all used the same size fingertips or no fingertips so it was
good enough to just have pretty basic logic.
This CL revamps the fingertip size selection logic to allow for
fat-finger tests, and enables the two-fat-finger-tracking test. Most of
the logic for this gesture already existed in the regular
two-finger-tracking test, so the majority of the CL is just setting up
the switching code to make it work. It's a little overkill for this
test, but should enable multi-sized gestures when we add them in the
future, which is coming soon.
BUG=chromium:334460
TEST=manually ran the FW test
Change-Id: I99f45f36d493d865b72b9e5499227e82a1e7a769
Signed-off-by: Charlie Mooney <charliemooney@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/185391
/external/autotest/client/site_tests/firmware_TouchMTB/test_conf.py
|
| d0fb182ac198c202b231577f131dd37d7f0a33a5 |
|
05-Feb-2014 |
Charlie Mooney <charliemooney@chromium.org> |
Touchbot II: support tapping and tracking
This CL adds support for the two gestures where one finger draws a
straight line diagonally across the pad and the other taps in the
corner. This is done by using the one_stationary_finger gesture which
has just had tapping capabilities added to it.
BUG=chromium:334460
TEST=Manual testing
CQ-DEPEND=I5f50cba58427ae5b81140041521ce1b5cdc81e46
Change-Id: Icb23f66541b1998ba612c4576894d74b1a5710d2
Signed-off-by: Charlie Mooney <charliemooney@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/185044
Reviewed-by: Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/test_conf.py
|
| f521d8961c62d9656720139ad94e21f2f8a04344 |
|
22-Jan-2014 |
Charlie Mooney <charliemooney@chromium.org> |
Touch FW test: Add support for stationary/tap
The robot needs to know how to parse the gesture where it holds one
finger still and taps around it with a second finger. This CL adds in
the parsing code and informs the FW test suite that the robot is now
capable of performing this gesture.
BUG=chromium:334460
TEST=Manually tested by running the FW test suite with the robot
CQ-DEPEND=I869c749aa35832bd4ca071d7db588bc903c48692
Change-Id: Ie6672a64ff8a5bdb51b5760e1b6bb88925db1c20
Signed-off-by: Charlie Mooney <charliemooney@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/183531
Reviewed-by: Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/test_conf.py
|
| 1feddb865cf3f9c158d8cca17a6e68c71b500571 |
|
16-Jan-2014 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: split LinearityValidator for normal and fat fingers
This patch splits the Linearity Validator into two parts, one for
gestures performed with normal fingers, and the other for gestures
performed with fat fingers.
BUG=chromium:335015
TEST=All unit tests should pass on a chromebook machine.
$ cd /usr/local/autotest/tests/firmware_TouchMTB
$ python tests/run_all_unittests.py
Also replay the logs for Link, and generate the summary:
$ tools/machine_replay.sh -b tests/logs/link/
$ ./summary.sh -d /tmp
Now, look at the summary file /tmp/summary/summary_xxxx/summary_xxxx.txt
It would show both LinearityFatFinger(Middle)Validator and
LinearityNormalFinger(Middle)Validator like:
LinearityFatFinger(Middle)Validator
max error in x (mm) : 5.49 The max err of all samples
max error in y (mm) : 1.47 The max err of all samples
rms error in x (mm) : 1.32 The mean of all rms means of all trials
rms error in y (mm) : 0.57 The mean of all rms means of all trials
LinearityNormalFinger(Middle)Validator
max error in x (mm) : 2.38 The max err of all samples
max error in y (mm) : 2.76 The max err of all samples
rms error in x (mm) : 0.19 The mean of all rms means of all trials
rms error in y (mm) : 0.31 The mean of all rms means of all trials
Change-Id: Iadc5089a2017f2a11a61e5315dc57f1f8937c1e4
Reviewed-on: https://chromium-review.googlesource.com/182766
Reviewed-by: Charles Mooney <charliemooney@chromium.org>
Tested-by: Shyh-In Hwang <josephsih@chromium.org>
Commit-Queue: Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/test_conf.py
|
| de12426329e9f08daf28cb5d16db9d70c8d096b4 |
|
15-Jan-2014 |
Charlie Mooney <charliemooney@chromium.org> |
touch fw test -- automated finger crossing tests
This adds automated robot support for the finger_crossing tests in the
touch firmware test. These are the tests where one finger is stationary
and the other finger moves closely past it.
BUG=chromium:334460
TEST=ran manually on a link
Change-Id: Ia2eb2ea61ab319f373de75e1754ba2a7f9332377
Signed-off-by: Charlie Mooney <charliemooney@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/182668
Reviewed-by: Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/test_conf.py
|
| 1951e2b088fedc4c8fe9997a4ed4f9ba58858d79 |
|
15-Jan-2014 |
Charlie Mooney <charliemooney@chromium.org> |
Touch FW test: Support automated drumroll tests
This patch adds robot support for drumroll tests to the TouchMTB
firmware test.
BUG=chromium:334460
TEST=manually ran FW test with the robot and it worked
Change-Id: Ib0f20ad0be6833ab0467563e405d66eb718b06ad
Signed-off-by: Charlie Mooney <charliemooney@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/182637
Reviewed-by: Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/test_conf.py
|
| 00779b9bd10d150d98f6d08f7f25a830a802a4f4 |
|
15-Jan-2014 |
Charlie Mooney <charliemooney@chromium.org> |
Touchbot II: close-finger tracking support
This adds robot support for the close two-finger tracking tests
BUG=chromium:334460
TEST=Ran the test on a link. The new gestures all parsed correctly and
the robot was able to perform them
Change-Id: Ifc0ba80cc2a9c9b365e527dacf9ca7cdbb14f3ec
Signed-off-by: Charlie Mooney <charliemooney@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/182504
Reviewed-by: Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/test_conf.py
|
| 2d9f41d18cb24e7110add66b132acdb601f5e0c7 |
|
15-Jan-2014 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: update test logs and unit tests
As some of the validators have been added/deleted/modified, the test logs and
unit tests need to be updated too.
BUG=chromium:334572
TEST=The unit tests should all pass on a chromebook machine.
$ cd /usr/local/autotest/tests/firmware_TouchMTB
$ python tests/run_all_unittests.py
Change-Id: I94863deceb07b4d01c19f63de6e3704c5d9e3d83
Reviewed-on: https://chromium-review.googlesource.com/182583
Reviewed-by: Charles Mooney <charliemooney@chromium.org>
Tested-by: Shyh-In Hwang <josephsih@chromium.org>
Commit-Queue: Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/test_conf.py
|
| 7584b3410e81bd7ea282432c2c3c55d9b4a06ea3 |
|
19-Dec-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: display summary table and detailed table
The summary table is used for the spec, and the detailed table for
tracing the root cause or for debugging.
As an example, the top level summary table will show something like
StationaryFingerValidator, StationaryTapValidator
max distance (mm) : 1.45 max distance of any two points from any run
On the other hand, the detailed table will display metrics of the two
related validators separately. The StationaryFingerValidator is used
to measure the pulling effect by another finger on a stationary
finger, while the StationaryTapValidator is used to measure the
wobble of taps/clicks.
StationaryFingerValidator
max distance (mm) : 1.45 max distance of any two points from any run
StationaryTapValidator
max distance (mm) : 0.54 max distance of any two points from any run
Since there may exist distinct root causes of failures about both
validators, the detailed table is useful in showing them separately.
BUG=chromium:330108
TEST=Replay logs and generate a summary file.
$ cd /usr/local/autotest/tests/firmware_TouchMTB
$ tools/machine_replay.sh -b tests/logs/link/
$ ./summary.sh -d /tmp
Open the following summary file, /tmp/summary/summary_xxxx/summary_xxxx.txt
while xxxx represents timestamps. It shows two tables in the summary file.
The first table is "Summary table" which shows something like
...
StationaryFingerValidator StationaryTapValidator
max distance (mm) : 1.45 max distance of any two points
...
The second table is "Detailed table (for debugging)" which shows the
two related validators separately.
...
StationaryFingerValidator
max distance (mm) : 1.45 max distance of any two points
StationaryTapValidator
max distance (mm) : 0.54 max distance of any two points
...
Change-Id: Ib478b39c28c23e49d824be41c57650915ccb569a
Reviewed-on: https://chromium-review.googlesource.com/181094
Reviewed-by: Charles Mooney <charliemooney@chromium.org>
Commit-Queue: Shyh-In Hwang <josephsih@chromium.org>
Tested-by: Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/test_conf.py
|
| 3355dcdfdac4cdea9f3cc4953409a8607d0f6e5b |
|
20-Dec-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: add count tracking ID validator for normal finger
It is a better class hierarchy to make both
CountTrackingIDFatFingerValidator and
CountTrackingIDNormalFingerValidator derive from
CountTrackingIDValidator.
BUG=chromium:330108
TEST=Replay logs and generate a summary file.
$ cd /usr/local/autotest/tests/firmware_TouchMTB
$ tools/machine_replay.sh -b tests/logs/link/
$ ./summary.sh -d /tmp
Open the following summary file, /tmp/summary/summary_xxxx/summary_xxxx.txt
while xxxx represents timestamps. It shows something like:
...
CountTrackingIDFatFingerValidator
pct of incorrect cases (%) : 25.00 an incorrect case ...
CountTrackingIDNormalFingerValidator
pct of incorrect cases (%) : 0.0 an incorrect case ...
...
Change-Id: I05cf39d34654b1df10d99eecd2d4d4483f71784c
Reviewed-on: https://chromium-review.googlesource.com/181093
Reviewed-by: Charles Mooney <charliemooney@chromium.org>
Commit-Queue: Shyh-In Hwang <josephsih@chromium.org>
Tested-by: Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/test_conf.py
|
| 87321559af7db87fcf4fbe5a1eb80564aecccb71 |
|
18-Dec-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: suppport StationaryTapValidator
It is desirable to split the StationaryFingerValidator into two
validators. The first one, StationaryFingerValidator, verifies
the pulling effect of another finger on the stationary finger.
The other one, StationaryTapValidator, checks the wobble of
taps/clicks. In this way, when the stationary validator failed,
it is very easy to trace the root cause of the failure. Another
advantage is that it is possible to assign two distinct criteria
to the two validators since they are now separated.
BUG=chromium:330108
TEST=All unit tests, including the new one StationaryTapValidatorTest,
should pass.
$ cd /usr/local/autotest/tests/firmware_TouchMTB
$ python tests/run_all_unittests.py
Change-Id: Ib79af6d6fcb965bc5807487454246e7841925003
Reviewed-on: https://chromium-review.googlesource.com/181092
Reviewed-by: Charles Mooney <charliemooney@chromium.org>
Commit-Queue: Shyh-In Hwang <josephsih@chromium.org>
Tested-by: Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/test_conf.py
|
| a44a9531f8dbd9c61d1f940397a816d628d4b369 |
|
29-Nov-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: add hysteresis validator and its related gesture
A new HysteresisValidator is added to check if there exists a
cursor jump initially. This happens when a finger starts moving,
the cursor keeps at the same position initially. While the finger
moves sufficiently far away, the cursor jumps suddenly to catch
up with the finger and then moves smoothly with the finger thereafter.
A new gesture for performing hysteresis test is also added to draw
lines to various directions with distinct speed.
Since this is a regular line-drawing gesture, the robot is ready
to perform it.
BUG=chromium:326658
TEST=All unit tests, including the new one for hysteresis test,
should pass.
$ cd /usr/local/autotest/tests/firmware_TouchMTB
$ python tests/run_all_unittests.py
Change-Id: Ib3dcd5fecb87e946e2e2b3665ddd0823d26f07a3
Reviewed-on: https://chromium-review.googlesource.com/179316
Reviewed-by: Shyh-In Hwang <josephsih@chromium.org>
Commit-Queue: Shyh-In Hwang <josephsih@chromium.org>
Tested-by: Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/test_conf.py
|
| ed5de67cc2ad8960b48b4d726889f59d417b3e3e |
|
28-Nov-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: drop Linearity(BothEnds)Validtor
As the Linearity(BothEnds)Validator incurs some problems in
the test itself, we would like to deprecate this validator.
BUG=chromium:326442
TEST=Perform the following steps
$ cd /usr/local/autotest/tests/firmware_TouchMTB
Replay the log.
$ tools/machine_replay.sh -b tests/logs/link/
Observe that there are no Linearity(BothEnds)Validtor
in the html report file
/tmp/touch_firmware_report-link-fw_1.0.AA-complete-xxxx.html
where xxxx represents the current date-time.
Change-Id: Iac6dbfc043a84ea6955bab4fe946770396912b32
Reviewed-on: https://chromium-review.googlesource.com/179061
Reviewed-by: Shyh-In Hwang <josephsih@chromium.org>
Tested-by: Shyh-In Hwang <josephsih@chromium.org>
Commit-Queue: Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/test_conf.py
|
| 3729e6879e24fc0c41342cb84fec24bc7a1acdb9 |
|
05-Dec-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: add CountTrackingIDFatFingerValidator
It is useful to distinguish the count tracking ID results between
gestures with and without fat fingers.
For gestures with fat fingers, thumb edge, or close fingers, we may
allow a small percentage of finger split/merge.
On the other hand, for the other gestures without using fat fingers,
we would impose a strict criteria on the count of tracking IDs.
There should be no finger split/merge, ghost finger, or other
related problems.
The new validator is implemented by simply inheriting
CountTrackingIDValidator.
BUG=chromium:326412
TEST=Replay a previous manual gesture set.
And observe that it would display CountTrackingIDFatFingerValidator
instead of CountTrackingIDValidator on the following gestures:
- fat_finger_move_with_resting_finger
- drag_edge_thumb
- two_close_fingers_tracking
- two_fat_fingers_tracking
Change-Id: I0258003c0ba3737759b5d8518c60f51e48740bf3
Reviewed-on: https://chromium-review.googlesource.com/179033
Reviewed-by: Shyh-In Hwang <josephsih@chromium.org>
Tested-by: Shyh-In Hwang <josephsih@chromium.org>
Commit-Queue: Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/test_conf.py
|
| af04554499f895a1d1c98e550558ae6118c3cde0 |
|
03-Dec-2013 |
Charlie Mooney <charliemooney@chromium.org> |
Robot support for all phys clicks in Touch FW test
This patch adds Robot support for all the remaining physical click tests
in the touch fw test. As the physical click gesture already exists,
this was just a matter of updating the robot wrapper to accept three and
four finger gestures and perform them on the pad.
BUG=chromium:325477
TEST=manual testing by running the FW test on a Lumpy
Change-Id: Ibae1724f1dd7b741a2cddf2afea2c6f1afab65d9
Signed-off-by: Charlie Mooney <charliemooney@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/178635
Reviewed-by: Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/test_conf.py
|
| 4ed40ecbdc053a4b091a46dc9994452e9b531c56 |
|
03-Dec-2013 |
Charlie Mooney <charliemooney@chromium.org> |
Add Touchbot II support for pinch tests
The robot can do the pinch gestures for the fw test using the line.py
gesture that I've been using for the regular line/tracking/etc tests.
This CL adds support for pinch-zoom in the automated fw test.
BUG=chromium:325432
TEST=manually tested by running the FW test on a Lumpy
Change-Id: Ib6f04c31ce936416127ff6f3ddff3d6c729d1f55
Signed-off-by: Charlie Mooney <charliemooney@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/178621
Reviewed-by: Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/test_conf.py
|
| 740925a35f5d69faef6ae142be3631b52fbe46bc |
|
03-Dec-2013 |
Charlie Mooney <charliemooney@chromium.org> |
Touch FW Test: Add first stationary finger test
This patch adds support of the resting finger test to the robot portion
of the Touch FW test. This uses the new one_finger_stationary.py
gesture to execute the tests automatically where one finger is placed in
the bottom left corner and another finger draws a straight line.
BUG=chromium:324858
TEST=ran the FW test on the Touchbot II with the new gesture and it
worked great.
CQ-DEPEND=If60cc918e5b8790e2eb93ff31816b01cbb19fc81
Change-Id: Iff9f468b23228593445820176923886740b5689c
Signed-off-by: Charlie Mooney <charliemooney@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/178567
Reviewed-by: Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/test_conf.py
|
| d1603ecc4264a1c789e0de7e2c584ab83235a068 |
|
21-Nov-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: log details for metrics instead of scores
All the old scores are replaced with the new metrics statistics
in the html report file.
As some of the metrics may not need pass criteria, e.g., the
Linearity Validator, the criteria are only displayed when needed.
BUG=chromium:316291
TEST= Replay the logs:
$ tools/machine_replay.sh -b tests/logs/link
Observe the scores are all replaced with the new metrics statistics
in the html report.
Change-Id: I17c0c7e431544ef30b4bdf1655a204ba3ea8f7a4
Reviewed-on: https://chromium-review.googlesource.com/177741
Reviewed-by: Charles Mooney <charliemooney@chromium.org>
Tested-by: Shyh-In Hwang <josephsih@chromium.org>
Commit-Queue: Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/test_conf.py
|
| b5486d08d7d4c1481ac5f01c87865360ef7812ec |
|
21-Nov-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: remove NoLevelJumpValidator
Remove the NoLevelJumpValidator as it could be covered by the
LinearityValidator.
BUG=chromium:316291
TEST= Replay the logs.
$ tools/machine_replay.sh -b tests/logs/link
Observe that the NoLevelJumpValidator is not displayed in
/tmp/touch_firmware_report-link-fw_1.0.AA-complete-xxxx.html
Change-Id: I04123c3ff136a80a27cf5152b9e4afaebb75d261
Reviewed-on: https://chromium-review.googlesource.com/177740
Reviewed-by: Charles Mooney <charliemooney@chromium.org>
Tested-by: Shyh-In Hwang <josephsih@chromium.org>
Commit-Queue: Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/test_conf.py
|
| c0d30cbdfb45cfb800689451f29fe37345c66f48 |
|
21-Nov-2013 |
Charlie Mooney <charliemooney@chromium.org> |
Switching the Touch FW test to the new Robot
The touchbotII robot wrapper has been under development for a while now
and seems to be quite functional. As such I would like to switch the
test over to use it now. This patch removed the old robot wrapper and
replaces it with the one for the new robot.
The current state of the robot test with the new robot is that it can do
the same tests that the old robot could. The only way to really iron
out the kinks in it now is to start actually using it, I think. More
tests will be added as this robot is much more flexible than the old
one, but replacing the old functionality was the first milestone here.
BUG=chromium:308162
TEST=I ran the full fw test using the new robot on a Lumpy in the lab
and everything worked.
Change-Id: Ife9bd73850511b4c131fc16d850e1798d686f662
Signed-off-by: Charlie Mooney <charliemooney@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/177481
Reviewed-by: Shyh-In Hwang <josephsih@chromium.org>
Commit-Queue: David James <davidjames@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/test_conf.py
|
| e41b3cfce09ec790d909107c9da6726e51b13c89 |
|
24-Oct-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: show metrics as default
Metrics statistics are shown without the need to specify any flags.
BUG=chromium:235753
TEST=Replay the logs without specifying "-s" command option.
$ cd /usr/local/autotest/tests/firmware_TouchMTB
$ tools/machine_replay.sh -b tests/logs/lumpy/
$ ./summary.sh -d /tmp -m f
Check out the summary report in /tmp/summary/summaryxxx/summaryxxx.txt,
where xxx is of the form of the date time. It should display the
"Metrics statistics by validator" in the summary report.
All unit tests must pass too.
$ python tests/run_all_unittests.py
Change-Id: I893290d77a27c8a981c65e8f837c2ba5f1e39c46
Reviewed-on: https://chromium-review.googlesource.com/174663
Reviewed-by: Shyh-In Hwang <josephsih@chromium.org>
Commit-Queue: Shyh-In Hwang <josephsih@chromium.org>
Tested-by: Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/test_conf.py
|
| 7223a44ea2cd425c3947488b5388a5842ca08979 |
|
16-Oct-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: adjust the range criteria to fail a bad gesture
By adjusting the range criteria, we got the scores as follows:
- the score of RangeValidator for single gesture: 0.84 --> 0.13
(This score makes it very obvious that the very gesture failed the test.)
- the mean of scores of RangeValidator for all related gestures:
0.96 --> 0.78
(Although all other one-finger-to-edge gestures were correctly
performed, the mean value is still obviously worse.)
- the final weighted average for all validators: 0.949 --> 0.932
(The final score is lowered by about 1.5%.)
BUG=chromium:299416
TEST=Verified the data of Round 2 from the 4 rounds on Link.
The scores should be obviously lower in one_finger_to_left gestures.
Change-Id: I330799cb172e653f21ce0a08c1cc1a143fa395c7
Reviewed-on: https://chromium-review.googlesource.com/173008
Reviewed-by: Charles Mooney <charliemooney@chromium.org>
Tested-by: Shyh-In Hwang <josephsih@chromium.org>
Commit-Queue: Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/test_conf.py
|
| ceed19b0caa1913b4ca7a91b7b27d654e214bc09 |
|
30-Aug-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: chop off pauses at both ends in tracking gestures
When the touch robot or a tester performs tracking gestures,
occasionally, there are pauses in the beginning and in the end.
While the kernel only reports changes of MTB attributes such as
x, y, and pressure values, a packet with the same attributes as
its previous one will be simply discarded in the kernel which
leads to larger packet report intervals.
In this patch, only the middle segment in which points are away
from both ends by more than a specified distance are used to
calculate the report rate metrics. In other words, the points
at both ends are chopped off in order not to generate the false
alarms of large packet report intervals.
BUG=chromium:252152
TEST=Conduct unit tests on a chromebook or in chroot.
$ cd /usr/local/autotest/tests/firmware_TouchMTB
$ python tests/run_all_unittests.py
The unit tests must all pass.
Change-Id: Ia2eb64a5b86d81a5125948d3f87b0705836c6f1f
Reviewed-on: https://chromium-review.googlesource.com/168908
Reviewed-by: Charles Mooney <charliemooney@chromium.org>
Tested-by: Shyh-In Hwang <josephsih@chromium.org>
Commit-Queue: Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/test_conf.py
|
| ad3136c03a931b4c1742cdec472780be15f7e1ed |
|
12-Aug-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: replay with a mocked device
It is handy and sometimes essential to be able to replay the test
logs captured from a platform on another platform.
When replaying logs, the board on which the logs were captured could
be different from the board of the chromebook currently used. Hence,
the correct board should be extracted from the log specified in the
replay directory. The following priority is used to find the device
description file used to mock the particular touch device for that
board:
(1) use the device description file in the replay directory
(2) use the device description file found in tests/device directory
However, if "-d/--system_device" command line option is specified, use
the system touch device found on the chromebook instead as before.
BUG=chromium:274204
TEST=Replay the lumpy logs on both lumpy and link. Observe that
their summary reports are exactly the same.
$ cd /usr/local/autotest/tests/firmware_TouchMTB
$ tools/machine_replay.sh -b tests/logs/lumpy -s
$ ./summary.sh -d /tmp -m f
Change-Id: I95a39d0ead821de9379e2f3a26f19e42a9c829b9
Reviewed-on: https://gerrit.chromium.org/gerrit/66345
Reviewed-by: Charlie Mooney <charliemooney@chromium.org>
Tested-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
Commit-Queue: Joseph Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/test_conf.py
|
| 907efec59a199502d5508af023cd0668dcaffa1a |
|
07-Aug-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: use finger contact to replace slot for LinearityValidator
Previously, we assume that the first finger contact is always
assigned the slot 0, and the 2nd finger contact the slot 1, etc.
While this may hold true in practice, it is not safe to make the
assumption.
Hence, we would like to use the finger contacts to replace the slot numbers in
validators. For example,
LinearityValidator(linearity_criteria, slot=0, segments=MIDDLE),
would be replaced by
LinearityValidator(linearity_criteria, finger=0, segments=MIDDLE),
where finger means finger contact.
The difference between the slot number and the finger contact lies in
that the former is assigned by the firmware while the latter is how we
observe in the events emitted by the event device (/dev/input/event*).
BUG=chromium:271282
TEST=Conduct the unit tests on a chormebook or in chroot, they should
all pass.
$ cd .../firmware_TouchMTB
$ python tests/run_all_unittests.py
Change-Id: I4f4a6faeb0cf03cb404382f543dea52f03c2f715
Reviewed-on: https://gerrit.chromium.org/gerrit/65562
Reviewed-by: Charlie Mooney <charliemooney@chromium.org>
Tested-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
Commit-Queue: Joseph Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/test_conf.py
|
| 1fb73fd14e08490f705d0893b48e866b3ac3e13d |
|
07-Aug-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: fix slot=0 in FIRST_FINGER_TRACKING_AND_SECOND_FINGER_TAPS
The first finger should correspond to the slot 0 rather than the
slot 1. This simple patch fixed the bug.
BUG=chromium:235753
TEST=None
Change-Id: I0a50a9ce849aca70a1c0fe23774f5d99d242c40f
Reviewed-on: https://gerrit.chromium.org/gerrit/65561
Reviewed-by: Charlie Mooney <charliemooney@chromium.org>
Tested-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
Commit-Queue: Joseph Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/test_conf.py
|
| 4a0ba066c40240e319471a77c06d6e649ebc5f1e |
|
24-Jul-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: remove ReportRateValidtor from taps/clicks
In this patch, we remove the ReportRateValidtor form taps and clicks.
BUG=chromium:261897
TEST=Conduct a complete test, and observe that the ReportRateValidtor
is not used in taps/clicks.
$ cd /usr/local/autotest/tests/firmware_TouchMTB
$ DISPLAY=:0 OPTIONS="-m complete -s" python main.py
Change-Id: I8f14a462b82b9482e78399790a696b6d676bf8f9
Reviewed-on: https://gerrit.chromium.org/gerrit/63331
Reviewed-by: Charlie Mooney <charliemooney@chromium.org>
Commit-Queue: Joseph Shyh-In Hwang <josephsih@chromium.org>
Tested-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/test_conf.py
|
| 7dbae7827dae0767bffe303708e19483b80a007e |
|
17-Jul-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: add the metric for CountTrackingIDValidator
The metrics of some validators will be hidden by default unless
there are failures. Such validators include CountTrackingIDValidator,
CountPacketsValidator, and PinchValidator.
Display the metric for CountTrackingIDValidator if there are failures.
Otherwise, hide the metric unless a flag "-m a" is specified. The
metric shows the percentage of the incorrect test cases over the total
test cases.
The printing format with failures looks like:
Metrics statistics by validator: fw_12 description
------------------------------------------------------------------------------
CountTrackingIDValidator
pct of incorrect cases (%) : 4.76 pct of incorrect cases over total cases
...
The detailed raw metrics values will be printed if a flag "-m f"
is specified. The raw metric value pair represents
(actual tracking IDs, expected tracking IDs). The format looks like:
CountTrackingIDValidator
pct of incorrect cases (%)
** Note: value below represents (actual tracking IDs, expected tracking IDs)
drag_edge_thumb.left_to_right (20130506_032458) : (5, 1)
one_finger_physical_click.bottom_left (20130506_032458): (1, 1)
...
Note that in the drag_edge_thumb gesture above, the expected number
of tracking IDS (or finger IDs) should be 1. However, 5 tracking IDs
are actually observed due to some firmware problems.
For the one_finger_physical_click gesture, exactly 1 tracking ID is
observed, while the number of expected finger IDs is 1.
We also unify the "-m" (i.e., metrics) flag in summary.sh script.
$ ./summary.sh -m p # show the statistics of the primary metrics
$ ./summary.sh -m a # show the statistics of all metrics, where
# all metrics = the primary metrics + the hidden ones
$ ./summary.sh -m f # show the full raw metrics values on file basis
BUG=chromium:235753
TEST=Run the command on a chromebook machine to generate a summary report.
$ cd /usr/local/autotest/tests/firmware_TouchMTB
Replay the logs.
$ tools/machine_replay.sh -b tests/logs/lumpy -s
Generate the summary report with the metrics statistics.
$ ./summary.sh -d /tmp -m a
Generate the summary report with the full raw metrics values on file basis.
$ ./summary.sh -d /tmp -m f
Also make sure all unit tests pass.
$ python tests/run_all_unittests.py
Change-Id: Ie385bfd54bac0a45211fc2a678ad57d9472e4da2
Reviewed-on: https://gerrit.chromium.org/gerrit/62827
Commit-Queue: Joseph Shyh-In Hwang <josephsih@chromium.org>
Reviewed-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
Tested-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/test_conf.py
|
| f745c17c4bbf5f42a00813ef954762b5b8cb122c |
|
18-Jul-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: no using ReportRateValidator for stationary fingers
In some gestures, the first finger acts as a stationary finger,
and the second one as a tracking finger. For example, the gesture
RESTING_FINGER_PLUS_2ND_FINGER_MOVE is such a case.
The kernel driver has an internal state machine which keeps track
of the values of event attributes, e.g., ABS_MT_POSITION_X/Y,
ABS_MT_PRESSURE, etc. If a stationary finger is touching the touch
device stably, it is possible that those attributes may not change
across packets. In this case, the kernel driver will not report the
packets at all.
This behavior may affect the correctness when calculating the report
rate. In this patch, we provide the option to ignore the stationary
finger by introducing an optional argument slot. When set "slot=1",
The ReportRateValidator will take into account only the packets of
the second finger. Otherwise, it will count all packets as before
when calculating the report rate.
Specifically, the following gestures with the first finger acting as
a stationary one are affected:
FINGER_CROSSING,
FAT_FINGER_MOVE_WITH_RESTING_FINGER,
RESTING_FINGER_PLUS_2ND_FINGER_MOVE,
STATIONARY_FINGER_NOT_AFFECTED_BY_2ND_FINGER_TAPS
A list of SYN_REPORT time instants is calculated first, and is then
passed into the methods of computing the report rate and the report
rate metrics.
The get_report_rate method is moved from the mtb module to the
validators module as it should belong to the latter.
BUG=chromium:261897
TEST=the unit tests must all pass on a chromebook machine which
include both cases with and without the slot specified.
$ cd /usr/local/autotest/tests/firmware_TouchMTB
$ python tests/run_all_unittests.py
Change-Id: I1ef90331b9db9a5e79593783dc7e9be340797d5e
Reviewed-on: https://gerrit.chromium.org/gerrit/62614
Reviewed-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
Tested-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
Commit-Queue: Joseph Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/test_conf.py
|
| 039967354df47f89616776832ec2a9dd9fea9efb |
|
16-Jul-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: print raw metrics values
In the last patch, the statistics of metrics values are printed
as follows:
Metrics statistics by validator : fw_11.23 description
--------------------------------------------------------------------
Linearity(BothEnds)Validator
max error in x (mm) : 9.34 The max err of all samples
...
If a user would like to know which gesture/file causes the max error
above, it would be helpful to print the raw metrics values as follows:
Raw metrics values
--------------------------------------------------------------------
fw_11.23
Linearity(BothEnds)Validator
max error in x (mm)
drag_edge_thumb.left_to_right (20130710_063117) : 3.06
two_fat_fingers_tracking.left_to_right (20130710_063117) : 9.34
...
In this way, the user could observe that the file with
gesture: "two_fat_fingers_tracking",
variation: "left_to_right",
in the folder: "20130710_063117"
causes the "max error in x".
BUG=chromium:235753
TEST=Run the command on a chromebook machine to generate a summary report.
$ cd /usr/local/autotest/tests/firmware_TouchMTB
Replay the logs
$ tools/machine_replay.sh -b tests/logs/lumpy -s
Generate the summary report with short metrics values grouped by validator.
$ ./summary.sh -d /tmp -m
Generate the summary report with detailed metrics values.
$ ./summary.sh -d /tmp --metrics_details
Observe that the raw detailed metrics values are printed this time.
Change-Id: I4d540f71c1c1e91507b9104eff8ac80c14d6654e
Reviewed-on: https://gerrit.chromium.org/gerrit/62329
Reviewed-by: Charlie Mooney <charliemooney@chromium.org>
Commit-Queue: Joseph Shyh-In Hwang <josephsih@chromium.org>
Tested-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/test_conf.py
|
| 330dcded15b07bf1869bd5d4cbf2bccbc6a14537 |
|
04-Jul-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: printing the statistics of metrics
In this patch, we print the statistics of metrics by validators in
the summary report.
We also handle the metrics names in a more unified way. Before,
the same metric name strings, e.g., 'max error in x (mm)', were used
across different modules. Now with a new class MetricNameProps to
keep information about the metric names, the descriptions, and the
stat_func used to calculate the statistics of metrics, the metric
names could be derived consistently by expanding from a template, say
'max error in {} (mm)', with its variations, e.g., ['x', 'y'], to
derive the names: 'max error in x (mm)' and 'max error in y (mm)'.
We also remove the PhysicalClickValidator temporarily which incurs
divide-by-0 error. This validator is not specified in the spec v2.
However, it might be useful to add it back later to make sure that
a tap does not incur physical click.
The metrics in a summary report looks like:
Metrics statistics by validator : fw1.2 description
--------------------------------------------------------------------------------
DrumrollValidator
circle radius (mm) : 1.37 Anything over 2mm is failure
Linearity(BothEnds)Validator
max error in x (mm) : 9.34 The max err of all samples
max error in y (mm) : 6.33 The max err of all samples
rms error in x (mm) : 2.46 The mean of all rms means of all trials
rms error in y (mm) : 1.03 The mean of all rms means of all trials
Linearity(Middle)Validator
max error in x (mm) : 4.91 The max err of all samples
max error in y (mm) : 6.30 The max err of all samples
rms error in x (mm) : 1.27 The mean of all rms means of all trials
rms error in y (mm) : 0.68 The mean of all rms means of all trials
PhysicalClickValidator
1f-click miss rate (%) : 0.00 Should be close to 0 (0 is perfect)
2f-click miss rate (%) : 0.00 Should be close to 0 (0 is perfect)
3f-click miss rate (%) : 0.00 Should be close to 0 (0 is perfect)
4f-click miss rate (%) : 0.00 Should be close to 0 (0 is perfect)
5f-click miss rate (%) : 0.00 Should be close to 0 (0 is perfect)
RangeValidator
left edge not reached (mm): 0.08 Min unreachable distance
ReportRateValidator
average time interval (ms): 12.19 less than 1/60 sec is required
intervals > 1/60 sec (%) : 4.45 0% is required
max time interval (ms) : 57.96 less than 1/60 sec is required
StationaryFingerValidator
max distance (mm) : 3.85 max distance of any two points from
any run
BUG=chromium:235753
TEST=Run the command on a chromebook machine to generate a summary report.
$ cd /usr/local/autotest/tests/firmware_TouchMTB
Replay the logs
$ tools/machine_replay.sh -b tests/logs/lumpy -s
Generate the summary report
$ ./summary.sh -m -d /tmp
Observe that the metrics are printed as above.
Also make sure all unit tests pass.
$ python tests/run_all_unittests.py
Change-Id: If7224354ef866cc3468752caa32feb1439b0b8cc
Reviewed-on: https://gerrit.chromium.org/gerrit/61883
Commit-Queue: Joseph Shyh-In Hwang <josephsih@chromium.org>
Reviewed-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
Tested-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/test_conf.py
|
| bb6f69b8c90e4896dd82e0266a096674c957f7db |
|
27-Jun-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: capture files for pressure calibration
In this patch, we add a new mode to capture event files for
pressure calibration.
BUG=chromium:221864
TEST=Run the test on a chromebook machine to capture event files.
$ cd /usr/local/autotest/tests/firmware_TouchMTB
$ DISPLAY=:0 OPTIONS="-m calibration" python main.py
Observe that it guides the user to record a few event files with
distinct size of metal fingers.
Change-Id: I6f65a2d3585380a51dd0e4bf6785fe55308c91c8
Reviewed-on: https://gerrit.chromium.org/gerrit/60641
Commit-Queue: Joseph Shyh-In Hwang <josephsih@chromium.org>
Reviewed-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
Tested-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/test_conf.py
|
| f16a39fd38ddaec0707d9ea3b1f7c3a2a0bb090f |
|
21-Jun-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: add metrics for Stationary Finger Validator
In this patch, we add the metric for StationaryFingerValidator:
- the maximum distance between any two points of the stationary finger in mm
The result logs are updated due to the change of the validator and
its criteria. Some unit tests are also added/updated accordingly.
BUG=chromium:252669
TEST=Replay logs on a lumpy.
$ cd /usr/local/autotest/tests/firmware_TouchMTB
$ tools/machine_replay.sh -b tests/logs/lumpy/ -s
And then run the summary script to generate a summary report.
$ ./summary.sh -d /tmp -m
Observe the metrics would look like:
ReportRateValidator
fw_11.23: {
'max_distance_mm': (1.3453, 1.3453, 1.3453)
}
fw_11.27: {
'max_distance_mm': (1.7159, 1.7159, 1.7159)}
}
All of the unit tests should pass.
$ python tests/run_all_unittests.py
Change-Id: Ia010ba136434348762ba2ae3351e5ac929dc28ec
Reviewed-on: https://gerrit.chromium.org/gerrit/59723
Reviewed-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
Tested-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
Commit-Queue: Joseph Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/test_conf.py
|
| 9e962e7e715812e1c1ad3c14204e0f63b76e707a |
|
27-May-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: Define the gesture order in manual mode
It is desirable that the gesture order to test in manual mode is
deterministic so that it would not confuse the testers.
BUG=chromium:241092
TEST=Run the manual tests and find that the order of gestures
conform to that defined in test_conf.gesture_names_complete
On a chromebook, run the manual tests:
$ cd /usr/local/autotest/tests/firmware_TouchMTB
$ DISPLAY=:0 python main.py
Change-Id: Ib3b8f245fb60f2fcb8ec9dfe20c62059cc306a38
Reviewed-on: https://gerrit.chromium.org/gerrit/56727
Reviewed-by: Charlie Mooney <charliemooney@chromium.org>
Tested-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
Commit-Queue: Joseph Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/test_conf.py
|
| 3f245993196c7cead8716ff09633a6ad49e8b006 |
|
15-May-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: New drumroll validator
The new drumroll validator requires that all points from the same
finger should be within 2 circles of radius X mm (e.g., 2 mm).
Pass if so. Fail if all points cannot be covered by 2 circles of
radius X mm.
In order to run unit tests with a mocked device, we also allow to
create such a mocked device by passing the device description into
TouchDevice class.
BUG=chromium:241092
TEST=On a chromebook
$ cd /usr/local/autotest/tests/firmware_TouchMTB
Make sure that the following unit test pass.
$ python tests/validators_unittest.py
Change-Id: I439fc70fc8c2b1d47b3475b7013997768c326feb
Reviewed-on: https://gerrit.chromium.org/gerrit/55758
Reviewed-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
Tested-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
Commit-Queue: Joseph Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/test_conf.py
|
| ecc254fc00e6b3504d4f7d87abad937cf73a267d |
|
07-May-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: refactor firmware_log and firmware_summary modules
Some of the code about test logs was manipulated in firmware_summary
module which made this module become messy. In this patch, we do the
refactoring by moving those code to firmware_log module and create
classes for them. In addition, we also flatten the deep hierarchy of
log dictionaries so that it is much simpler to maintain.
BUG=chromium:238039
TEST=On a chromebook
$ cd /usr/local/autotest/tests/firmware_TouchMTB
Make sure that the following unit test pass.
$ python tests/firmware_summary_unittest.py
Change-Id: Idc5a70ed28745805a4221cc20780c23da0210fb0
Reviewed-on: https://gerrit.chromium.org/gerrit/50320
Commit-Queue: Joseph Shyh-In Hwang <josephsih@chromium.org>
Reviewed-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
Tested-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/test_conf.py
|
| 40cb76895b4a52d85d5d2c1a25b45e49bdeb9828 |
|
06-May-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: fix some issues for replay
This patch fixes some issues for replaying raw data:
(1) Use the firmware version of the replayed files instead of that
of the real touch device since it does not record new files.
(2) Do not upload the gesture files since it is just a replay.
BUG=chromium:238039
TEST=On a chromebook. Replay an existing gesture set of which the
firmware version is not the same as the current firmware
version of the real touch device. Observes that it could
replay correctly. It does not upload the gesture files to
google storage server.
Change-Id: I5a14e94119fa28d422c6d66c00a0591ae0420174
Reviewed-on: https://gerrit.chromium.org/gerrit/50309
Reviewed-by: Charlie Mooney <charliemooney@chromium.org>
Commit-Queue: Joseph Shyh-In Hwang <josephsih@chromium.org>
Tested-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/test_conf.py
|
| 8c5e275df682ef5b0ad52574ae1e2e31fc724509 |
|
22-Apr-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: Rename the test suite and fix the programs
To finish renaming the test suite, we also need to fix the prompts,
classes, methods, variables, and strings, etc. to reduce confusion.
BUG=chromium:234117
TEST=Install the test suite to a chromebook
(cr) $ ./run_remote_tests.sh --autotest_dir
~/trunk/src/third_party/autotest/files --remote=$MACHINE_IP
firmware_TouchMTB
On the chromebook, conduct the test suite as follows.
$ DISPLAY=:0 python main.py
It should execute normally.
Change-Id: Iaf5d5060bf68e3e0a13913a69227636fba2fbeb6
Reviewed-on: https://gerrit.chromium.org/gerrit/48782
Reviewed-by: Charlie Mooney <charliemooney@chromium.org>
Commit-Queue: Joseph Shyh-In Hwang <josephsih@chromium.org>
Tested-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/test_conf.py
|
| 0cbff8bf3666a81b886080f82712943e36dad95f |
|
22-Apr-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: Rename the test suite as firmware_TouchMTB
As the test suite is now applicable to both the touchpad
and the touchscreen, it is proper to rename the test name
from firmware_TouchpadMTB to firmware_TouchMTB.
BUG=chromium:234117
TEST=None. Will be combined with next patch.
Change-Id: I92ecc074023c76f7c2f292b7e22ef960d537819f
Reviewed-on: https://gerrit.chromium.org/gerrit/48781
Reviewed-by: Charlie Mooney <charliemooney@chromium.org>
Commit-Queue: Joseph Shyh-In Hwang <josephsih@chromium.org>
Tested-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/test_conf.py
|