| 208b022eac67988d5bb35a387115c3a76f9b9413 |
|
06-Nov-2014 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: fixed a bug in discarding initial packets
In some cases, after discarding initial packets in noise tests,
it results in list index out of range error.
This situation occurs when all fingers happened to leave at the
time instant when creating the final state packet which is an
empty list, i.e., [].
When creating a new packet list after discarding initial packets,
be careful not to include the empty final state packet.
BUG=chromium:430826
TEST=Conduct unittests on a chromebook or in chroot.
$ cd /usr/local/autotest/tests/firmware_TouchMTB
$ python tests/run_all_unittests.py
All unit tests should pass.
Before the bug was fixed, running the unit tests would result in
an IndexError: list index out of range.
Change-Id: I034d85d9629c7ece61f22001a676c8b2784907a2
Reviewed-on: https://chromium-review.googlesource.com/228091
Reviewed-by: Charles Mooney <charliemooney@chromium.org>
Commit-Queue: Shyh-In Hwang <josephsih@chromium.org>
Tested-by: Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/validators.py
|
| 45f891b832de8f1c94e1006478fb557cd341aa57 |
|
28-Jul-2014 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: take MtbSanityValidator as a prerequisite
If the MtbSanityValidator fails in a test, it indicates the existence
of some critical problems. The rest validators of the test would not
be executed because some unexpected problems or results might be
incurred.
BUG=chrome-os-partner:30965
TEST=Record gesture files on a chromebook machine.
$ cd /usr/local/autotest/tests/firmware_TouchMTB/
$ DISPLAY=:0 python main.py -m complete
Before pressing ENTER for the 2nd command above, place 2 fingers on
the touchpad. Then press ENTER. It would only show the test results
about MTB Sanity on the right side of the gesture capturing window.
There are no test results about the other validators.
. . .
MtbSanityValidator
# fingers: 2
slot 0:
MT_TOUCH_MAJOR= 254
MT_TRACKING_ID= 4
MT_PRESSURE= 52
MT_POSITION_X= 573
MT_POSITION_Y= 299
slot 1:
MT_TOUCH_MAJOR= 254
MT_TRACKING_ID= 5
MT_PRESSURE= 57
MT_POSITION_X= 1384
MT_POSITION_Y= 421
# MTB # errors: # 473
FINGER_LEAVING_BEFORE_ARRIVING: 2
ASSIGN_ATTRIBUTES_BEFORE_ARRIVING: 471
criteria_str: == 0
score: 0.000000
. . .
Change-Id: Ic89fb78eb36ef911521adf094eafb9857e218e32
Reviewed-on: https://chromium-review.googlesource.com/210791
Reviewed-by: Charles Mooney <charliemooney@chromium.org>
Commit-Queue: Shyh-In Hwang <josephsih@chromium.org>
Tested-by: Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/validators.py
|
| 60a345dd2a6a7c27407153b6c9734d2b63a88c01 |
|
28-Jul-2014 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: add MtbSanityValidator
This patch checks two things.
1. Ghost fingers:
The validator checks if there exist ghost fingers before fingers
touch the touch device. This is accomplished by querying the
InputDevice class which syncs with the kernel evdev driver about
the mt slots information.
2. MTB event rules:
The MTB events emitted by a device have to follow some rules.
Otherwise, the events may confuse the state machine in the MTB
parser. As a beginning, this validator checks some simple rules:
a. No TRACKING ID is set to -1 before a slot is assigned a
positive TRACKING ID value.
b. No X, Y, or PRESSURE values could be assigned to a slot
before the slot is assigned a positive TRACKING ID value.
More rules may be added to this validator when needed.
Note that item 1 above is to detect any pre-existing fingers
before the test is conducted, while item 2 is to detect any event
mis-ordering or corruption occurring during the test.
BUG=chrome-os-partner:30965
TEST=All unit tests should pass on a chromebook.
$ cd /usr/local/autotest/tests/firmware_TouchMTB
$ python tests/run_all_unittests.py
Change-Id: Id85dcb8e54601f5dfd6d77bbf3802fdcb2d74a60
Reviewed-on: https://chromium-review.googlesource.com/210792
Reviewed-by: Charles Mooney <charliemooney@chromium.org>
Commit-Queue: Shyh-In Hwang <josephsih@chromium.org>
Tested-by: Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/validators.py
|
| d709ccc114fd7e4ffd21abcb160181d2d12243d4 |
|
16-Aug-2014 |
Mindy Huang <mindyh@chromium.org> |
Added a new validator that discards the initial seconds of data before
validation.
The new validator can wrap around any generic validator, and the number
of seconds to discard can be specified.
The validator simply strips the first second and prepends
the end state of the discarded second onto the stripped data. This
throws away all events that happened within the first second, but keeps
all events that persist after the first second.
The noise tests also now have a one second pause at the beginning of
each test to allow the touchpad to calibrate against the noise.
BUG=chromium:404265
TEST=Manually and with robot
Change-Id: I6d6a0c40f4cb106185c7cc084fca3b459dd49c3c
Signed-off-by: Mindy Huang <mindyh@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/212763
Reviewed-by: Charles Mooney <charliemooney@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/validators.py
|
| 3c3e1267244e69cd79f4508ee0ab936fc6249cd0 |
|
29-Apr-2014 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: import latency_measurement only when needed
When run the touch fw test suite in chroot, we do not need to import
latency_measurement which resides in ~/src/platform/touchbot and is
only used with the robot. Such use cases in chroot may include
running spreadsheet.py in chroot to upload pressure values, or
conducting unit tests in chroot. Currently, it results in an error
as follows:
from quickstep import latency_measurement
ImportError: No module named quickstep
A quick solution is to move the import command to within the class
that actually uses it.
BUG=chromium:368106
TEST=Run the unit tests in chroot.
(cr) cd
~/trunk/src/third_party/autotest/files/client/site_tests/firmware_TouchMTB
(cr) python tests/run_all_unittests.py
All unit tests should pass without incurring any ImportError.
Change-Id: I298f319349660b3f5272e52fe173cc308428ca40
Reviewed-on: https://chromium-review.googlesource.com/197480
Reviewed-by: Charles Mooney <charliemooney@chromium.org>
Reviewed-by: Shyh-In Hwang <josephsih@chromium.org>
Tested-by: Shyh-In Hwang <josephsih@chromium.org>
Commit-Queue: Charles Mooney <charliemooney@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/validators.py
|
| 3ceb6419c90576ba7df8cee03d4b42d38d1b6d00 |
|
05-Mar-2014 |
Charlie Mooney <charliemooney@chromium.org> |
Adding a drag-latency test to the Touch FW test
A drag latency test has been part of our touch qualification process for
a while now, but has always been run by hand. With Quickstep, this
should be able to be performed automatically as with any other
gesture/validator that we check. The operator will just need to set up
the laser across the pad before running the test.
This CL adds in the robot support for the Quickstep gesture and a new
validator that imports in the latency calculation software from the
touchbot project and computes the drag-latency using the touch data the
firmware collected and pulling the laser data out of the Quickstep sysfs
entries. If there is no Quickstep present, the test will report a
failing dummy value to indicate this, otherwise you would need the
Quickstep device to run the FW test, which is something we don't want to
be a requirement.
BUG=chromium:349174
TEST=Manually ran the test with and without the Quickstep plugged in and
the gesture ran smoothly and the computations worked without crashing
Change-Id: I638da3f837e86d6cf40da3e552c7d56b23e51ee1
Signed-off-by: Charlie Mooney <charliemooney@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/189502
/external/autotest/client/site_tests/firmware_TouchMTB/validators.py
|
| 1feddb865cf3f9c158d8cca17a6e68c71b500571 |
|
16-Jan-2014 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: split LinearityValidator for normal and fat fingers
This patch splits the Linearity Validator into two parts, one for
gestures performed with normal fingers, and the other for gestures
performed with fat fingers.
BUG=chromium:335015
TEST=All unit tests should pass on a chromebook machine.
$ cd /usr/local/autotest/tests/firmware_TouchMTB
$ python tests/run_all_unittests.py
Also replay the logs for Link, and generate the summary:
$ tools/machine_replay.sh -b tests/logs/link/
$ ./summary.sh -d /tmp
Now, look at the summary file /tmp/summary/summary_xxxx/summary_xxxx.txt
It would show both LinearityFatFinger(Middle)Validator and
LinearityNormalFinger(Middle)Validator like:
LinearityFatFinger(Middle)Validator
max error in x (mm) : 5.49 The max err of all samples
max error in y (mm) : 1.47 The max err of all samples
rms error in x (mm) : 1.32 The mean of all rms means of all trials
rms error in y (mm) : 0.57 The mean of all rms means of all trials
LinearityNormalFinger(Middle)Validator
max error in x (mm) : 2.38 The max err of all samples
max error in y (mm) : 2.76 The max err of all samples
rms error in x (mm) : 0.19 The mean of all rms means of all trials
rms error in y (mm) : 0.31 The mean of all rms means of all trials
Change-Id: Iadc5089a2017f2a11a61e5315dc57f1f8937c1e4
Reviewed-on: https://chromium-review.googlesource.com/182766
Reviewed-by: Charles Mooney <charliemooney@chromium.org>
Tested-by: Shyh-In Hwang <josephsih@chromium.org>
Commit-Queue: Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/validators.py
|
| d2af86aba17994a73221a804bd87a47aa00c3db8 |
|
09-Jan-2014 |
Charlie Mooney <charliemooney@chromium.org> |
Touch FW linearity validator - Exception handling
There was a problem running the touch FW test on the robot where the
linearity validator would crash because sometimes the finger would
register a small tap before starting. As a result there
would be a brief contact before the actual gesture and the linearity
validator would crash due to there not being enough events in the first
touch.
This patch makes the linearity validator handle this failure more
gracefully instead of bailing on the whole test
BUG=chromium:332550
TEST=I ran the robot test on the offending system and the test worked
all the way through. Previously it would never make it past a few
gestures before this problem cropped up.
Change-Id: I8a0e1a52a0e12b17c0840d27d4e914f6e3d04035
Signed-off-by: Charlie Mooney <charliemooney@chromium.org>
Reviewed-on: https://chromium-review.googlesource.com/182054
Reviewed-by: Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/validators.py
|
| 25ead0eae720b5e7c627e50ad214de514287aac4 |
|
15-Jan-2014 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: handle the format problem in HysteresisValidator
When there are no points sufficiently far from the first contact
point in a gesture, distance1 and distance2 would remain to be None.
This would lead to a format problem in validator log. This simple
patch just corrects the format problem.
BUG=chromium:332971
TEST=The unit tests should all pass on a chromebook machine.
$ cd /usr/local/autotest/tests/firmware_TouchMTB
$ python tests/run_all_unittests.py
Change-Id: I73e55ba29c18781b5110bf407815788a72eaeaef
Reviewed-on: https://chromium-review.googlesource.com/182584
Reviewed-by: Charles Mooney <charliemooney@chromium.org>
Tested-by: Shyh-In Hwang <josephsih@chromium.org>
Commit-Queue: Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/validators.py
|
| 7584b3410e81bd7ea282432c2c3c55d9b4a06ea3 |
|
19-Dec-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: display summary table and detailed table
The summary table is used for the spec, and the detailed table for
tracing the root cause or for debugging.
As an example, the top level summary table will show something like
StationaryFingerValidator, StationaryTapValidator
max distance (mm) : 1.45 max distance of any two points from any run
On the other hand, the detailed table will display metrics of the two
related validators separately. The StationaryFingerValidator is used
to measure the pulling effect by another finger on a stationary
finger, while the StationaryTapValidator is used to measure the
wobble of taps/clicks.
StationaryFingerValidator
max distance (mm) : 1.45 max distance of any two points from any run
StationaryTapValidator
max distance (mm) : 0.54 max distance of any two points from any run
Since there may exist distinct root causes of failures about both
validators, the detailed table is useful in showing them separately.
BUG=chromium:330108
TEST=Replay logs and generate a summary file.
$ cd /usr/local/autotest/tests/firmware_TouchMTB
$ tools/machine_replay.sh -b tests/logs/link/
$ ./summary.sh -d /tmp
Open the following summary file, /tmp/summary/summary_xxxx/summary_xxxx.txt
while xxxx represents timestamps. It shows two tables in the summary file.
The first table is "Summary table" which shows something like
...
StationaryFingerValidator StationaryTapValidator
max distance (mm) : 1.45 max distance of any two points
...
The second table is "Detailed table (for debugging)" which shows the
two related validators separately.
...
StationaryFingerValidator
max distance (mm) : 1.45 max distance of any two points
StationaryTapValidator
max distance (mm) : 0.54 max distance of any two points
...
Change-Id: Ib478b39c28c23e49d824be41c57650915ccb569a
Reviewed-on: https://chromium-review.googlesource.com/181094
Reviewed-by: Charles Mooney <charliemooney@chromium.org>
Commit-Queue: Shyh-In Hwang <josephsih@chromium.org>
Tested-by: Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/validators.py
|
| 3355dcdfdac4cdea9f3cc4953409a8607d0f6e5b |
|
20-Dec-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: add count tracking ID validator for normal finger
It is a better class hierarchy to make both
CountTrackingIDFatFingerValidator and
CountTrackingIDNormalFingerValidator derive from
CountTrackingIDValidator.
BUG=chromium:330108
TEST=Replay logs and generate a summary file.
$ cd /usr/local/autotest/tests/firmware_TouchMTB
$ tools/machine_replay.sh -b tests/logs/link/
$ ./summary.sh -d /tmp
Open the following summary file, /tmp/summary/summary_xxxx/summary_xxxx.txt
while xxxx represents timestamps. It shows something like:
...
CountTrackingIDFatFingerValidator
pct of incorrect cases (%) : 25.00 an incorrect case ...
CountTrackingIDNormalFingerValidator
pct of incorrect cases (%) : 0.0 an incorrect case ...
...
Change-Id: I05cf39d34654b1df10d99eecd2d4d4483f71784c
Reviewed-on: https://chromium-review.googlesource.com/181093
Reviewed-by: Charles Mooney <charliemooney@chromium.org>
Commit-Queue: Shyh-In Hwang <josephsih@chromium.org>
Tested-by: Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/validators.py
|
| 87321559af7db87fcf4fbe5a1eb80564aecccb71 |
|
18-Dec-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: suppport StationaryTapValidator
It is desirable to split the StationaryFingerValidator into two
validators. The first one, StationaryFingerValidator, verifies
the pulling effect of another finger on the stationary finger.
The other one, StationaryTapValidator, checks the wobble of
taps/clicks. In this way, when the stationary validator failed,
it is very easy to trace the root cause of the failure. Another
advantage is that it is possible to assign two distinct criteria
to the two validators since they are now separated.
BUG=chromium:330108
TEST=All unit tests, including the new one StationaryTapValidatorTest,
should pass.
$ cd /usr/local/autotest/tests/firmware_TouchMTB
$ python tests/run_all_unittests.py
Change-Id: Ib79af6d6fcb965bc5807487454246e7841925003
Reviewed-on: https://chromium-review.googlesource.com/181092
Reviewed-by: Charles Mooney <charliemooney@chromium.org>
Commit-Queue: Shyh-In Hwang <josephsih@chromium.org>
Tested-by: Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/validators.py
|
| a44a9531f8dbd9c61d1f940397a816d628d4b369 |
|
29-Nov-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: add hysteresis validator and its related gesture
A new HysteresisValidator is added to check if there exists a
cursor jump initially. This happens when a finger starts moving,
the cursor keeps at the same position initially. While the finger
moves sufficiently far away, the cursor jumps suddenly to catch
up with the finger and then moves smoothly with the finger thereafter.
A new gesture for performing hysteresis test is also added to draw
lines to various directions with distinct speed.
Since this is a regular line-drawing gesture, the robot is ready
to perform it.
BUG=chromium:326658
TEST=All unit tests, including the new one for hysteresis test,
should pass.
$ cd /usr/local/autotest/tests/firmware_TouchMTB
$ python tests/run_all_unittests.py
Change-Id: Ib3dcd5fecb87e946e2e2b3665ddd0823d26f07a3
Reviewed-on: https://chromium-review.googlesource.com/179316
Reviewed-by: Shyh-In Hwang <josephsih@chromium.org>
Commit-Queue: Shyh-In Hwang <josephsih@chromium.org>
Tested-by: Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/validators.py
|
| 3729e6879e24fc0c41342cb84fec24bc7a1acdb9 |
|
05-Dec-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: add CountTrackingIDFatFingerValidator
It is useful to distinguish the count tracking ID results between
gestures with and without fat fingers.
For gestures with fat fingers, thumb edge, or close fingers, we may
allow a small percentage of finger split/merge.
On the other hand, for the other gestures without using fat fingers,
we would impose a strict criteria on the count of tracking IDs.
There should be no finger split/merge, ghost finger, or other
related problems.
The new validator is implemented by simply inheriting
CountTrackingIDValidator.
BUG=chromium:326412
TEST=Replay a previous manual gesture set.
And observe that it would display CountTrackingIDFatFingerValidator
instead of CountTrackingIDValidator on the following gestures:
- fat_finger_move_with_resting_finger
- drag_edge_thumb
- two_close_fingers_tracking
- two_fat_fingers_tracking
Change-Id: I0258003c0ba3737759b5d8518c60f51e48740bf3
Reviewed-on: https://chromium-review.googlesource.com/179033
Reviewed-by: Shyh-In Hwang <josephsih@chromium.org>
Tested-by: Shyh-In Hwang <josephsih@chromium.org>
Commit-Queue: Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/validators.py
|
| d1603ecc4264a1c789e0de7e2c584ab83235a068 |
|
21-Nov-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: log details for metrics instead of scores
All the old scores are replaced with the new metrics statistics
in the html report file.
As some of the metrics may not need pass criteria, e.g., the
Linearity Validator, the criteria are only displayed when needed.
BUG=chromium:316291
TEST= Replay the logs:
$ tools/machine_replay.sh -b tests/logs/link
Observe the scores are all replaced with the new metrics statistics
in the html report.
Change-Id: I17c0c7e431544ef30b4bdf1655a204ba3ea8f7a4
Reviewed-on: https://chromium-review.googlesource.com/177741
Reviewed-by: Charles Mooney <charliemooney@chromium.org>
Tested-by: Shyh-In Hwang <josephsih@chromium.org>
Commit-Queue: Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/validators.py
|
| e41b3cfce09ec790d909107c9da6726e51b13c89 |
|
24-Oct-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: show metrics as default
Metrics statistics are shown without the need to specify any flags.
BUG=chromium:235753
TEST=Replay the logs without specifying "-s" command option.
$ cd /usr/local/autotest/tests/firmware_TouchMTB
$ tools/machine_replay.sh -b tests/logs/lumpy/
$ ./summary.sh -d /tmp -m f
Check out the summary report in /tmp/summary/summaryxxx/summaryxxx.txt,
where xxx is of the form of the date time. It should display the
"Metrics statistics by validator" in the summary report.
All unit tests must pass too.
$ python tests/run_all_unittests.py
Change-Id: I893290d77a27c8a981c65e8f837c2ba5f1e39c46
Reviewed-on: https://chromium-review.googlesource.com/174663
Reviewed-by: Shyh-In Hwang <josephsih@chromium.org>
Commit-Queue: Shyh-In Hwang <josephsih@chromium.org>
Tested-by: Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/validators.py
|
| ceed19b0caa1913b4ca7a91b7b27d654e214bc09 |
|
30-Aug-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: chop off pauses at both ends in tracking gestures
When the touch robot or a tester performs tracking gestures,
occasionally, there are pauses in the beginning and in the end.
While the kernel only reports changes of MTB attributes such as
x, y, and pressure values, a packet with the same attributes as
its previous one will be simply discarded in the kernel which
leads to larger packet report intervals.
In this patch, only the middle segment in which points are away
from both ends by more than a specified distance are used to
calculate the report rate metrics. In other words, the points
at both ends are chopped off in order not to generate the false
alarms of large packet report intervals.
BUG=chromium:252152
TEST=Conduct unit tests on a chromebook or in chroot.
$ cd /usr/local/autotest/tests/firmware_TouchMTB
$ python tests/run_all_unittests.py
The unit tests must all pass.
Change-Id: Ia2eb64a5b86d81a5125948d3f87b0705836c6f1f
Reviewed-on: https://chromium-review.googlesource.com/168908
Reviewed-by: Charles Mooney <charliemooney@chromium.org>
Tested-by: Shyh-In Hwang <josephsih@chromium.org>
Commit-Queue: Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/validators.py
|
| eb2c422de0873a741d68699d9f91d1ad351b9add |
|
07-Aug-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: adjustment of linearity validator
We need to do some adjustment for the linearity validator according
to the x-t and the y-t figure in the attached png file in
chromium:248210 #6 which is a left-to-right one-finger-tracking
gesture performed by the robot.
Adjustment as follows:
(1) Calculate the regression line based on only the middle segment.
(2) Calculate the metrics of max errors and rms erros only on the
middle segment. The begin segment and the end segment may only
represent a pause.
(3) Pick up only certain axis(es) when calculating the statistics
of metrics:
horizontal lines: metrics only in the x axis
vertical lines: metrics only in the y axis
diagonal lines: metrics both in the x and y axes
BUG=chromium:248120
TEST=Run the unit tests on a chromebook or in chroot
(cr) $ cd .../firmware_TouchMTB
(cr) $ python tests/run_all_unittests.py
Observe that all unit tests should pass.
Change-Id: I28566662831aae50be3274bd9b23e870506ba160
Reviewed-on: https://gerrit.chromium.org/gerrit/66344
Reviewed-by: Charlie Mooney <charliemooney@chromium.org>
Tested-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
Commit-Queue: Joseph Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/validators.py
|
| 907efec59a199502d5508af023cd0668dcaffa1a |
|
07-Aug-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: use finger contact to replace slot for LinearityValidator
Previously, we assume that the first finger contact is always
assigned the slot 0, and the 2nd finger contact the slot 1, etc.
While this may hold true in practice, it is not safe to make the
assumption.
Hence, we would like to use the finger contacts to replace the slot numbers in
validators. For example,
LinearityValidator(linearity_criteria, slot=0, segments=MIDDLE),
would be replaced by
LinearityValidator(linearity_criteria, finger=0, segments=MIDDLE),
where finger means finger contact.
The difference between the slot number and the finger contact lies in
that the former is assigned by the firmware while the latter is how we
observe in the events emitted by the event device (/dev/input/event*).
BUG=chromium:271282
TEST=Conduct the unit tests on a chormebook or in chroot, they should
all pass.
$ cd .../firmware_TouchMTB
$ python tests/run_all_unittests.py
Change-Id: I4f4a6faeb0cf03cb404382f543dea52f03c2f715
Reviewed-on: https://gerrit.chromium.org/gerrit/65562
Reviewed-by: Charlie Mooney <charliemooney@chromium.org>
Tested-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
Commit-Queue: Joseph Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/validators.py
|
| b6bba8d9db954b5adbfb2eb60a220948028b4b6f |
|
26-Jul-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: add the metric "clicks without finger IDs"
In this patch, we add a new metric to PhysicalClickValidator
"of the n clicks, the % of clicks without a finger ID".
In particular, we check whether the BTN_LEFT event occurs when
the number of tracking (finger) IDs is correct. For example,
if the BTN_LEFT events with values 1 and 0 occur after a finger
has left, it is a failure. The event sequence in such a failure
case may look like
Event: ABS_MT_TRACKING_ID, value 28
Event: ABS_MT_TRACKING_ID, value -1
Event: BTN_LEFT, value 1
Event: BTN_LEFT, value 0
On the other hand, a click case with the correct number of tracking
IDs may look like
Event: ABS_MT_TRACKING_ID, value 28
Event: BTN_LEFT, value 1
Event: ABS_MT_TRACKING_ID, value -1
Event: BTN_LEFT, value 0
BUG=chromium:251193
TEST=Replay logs on a lumpy.
$ cd /usr/local/autotest/tests/firmware_TouchMTB
$ tools/machine_replay.sh -b tests/logs/lumpy/ -s
And then run the summary script to generate a summary report.
$ ./summary.sh -d /tmp -m f
Observe the statistics of the metrics would look like
PhysicalClickValidator
1f-click miss rate (%) : 12.50 the pct of finger IDs w/o a click
1f-click w/o finger IDs (%): 0.00 the pct of clicks w/o correct finger IDs
2f-click miss rate (%) : 75.00 the pct of finger IDs w/o a click
2f-click w/o finger IDs (%): 0.00 the pct of clicks w/o correct finger IDs
The raw metrics values look like
PhysicalClickValidator
1f-click miss rate (%)
** Note: value below represents (actual clicks, expected clicks)
one_finger_physical_click.bottom_side (20130506_032458) : (1, 1)
...
1f-click w/o finger IDs (%)
** Note: value below represents
(clicks with correct finger IDs, actual clicks)
one_finger_physical_click.bottom_left (20130506_032458) : (1, 1)
...
Also all of the unit tests should pass.
$ python tests/run_all_unittests.py
Change-Id: I26b7e7dd6da4456f7b2fae8da7aacdfb5e69b688
Reviewed-on: https://gerrit.chromium.org/gerrit/63588
Reviewed-by: Charlie Mooney <charliemooney@chromium.org>
Commit-Queue: Joseph Shyh-In Hwang <josephsih@chromium.org>
Tested-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/validators.py
|
| aa7b540459960d887cd7520f4d6a963a33cc7a85 |
|
26-Jul-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: add the metric for PinchValidator
Display the metric for PinchValidator if there are failures.
Otherwise, hide the metric unless a flag "-m a" is specified. The
metric shows the percentage of the incorrect test cases over the total
test cases.
The printing format with failures looks like:
Metrics statistics by validator: fw_12 description
----------------------------------------------------------------------------
PinchValidator
pct of incorrect cases (%) : 12.50 pct of incorrect cases over total cases
The detailed raw metrics values will be printed if a flag "-m f"
is specified. The raw metric value pair represents
(actual relative motion (px), expected relative motion (px))
The format looks like:
PinchValidator
pct of incorrect cases (%)
** Note: value below represents
(actual relative motion (px), expected relative motion (px))
pinch_to_zoom.zoom_in (20130506_032458) : (646.52, 200)
pinch_to_zoom.zoom_out (20130506_032458) : (636.80, 200)
BUG=chromium:235753
TEST=Run the command on a chromebook machine to generate a summary report.
$ cd /usr/local/autotest/tests/firmware_TouchMTB
Replay the logs.
$ tools/machine_replay.sh -b tests/logs/lumpy -s
Generate the summary report with all metrics statistics.
$ ./summary.sh -d /tmp -m a
Generate the summary report with the full raw metrics values on file basis.
$ ./summary.sh -d /tmp -m f
Also make sure all unit tests pass.
$ python tests/run_all_unittests.py
Change-Id: I57b867f2d092de2245de2244d50e71375a8e1b40
Reviewed-on: https://gerrit.chromium.org/gerrit/63449
Reviewed-by: Charlie Mooney <charliemooney@chromium.org>
Commit-Queue: Joseph Shyh-In Hwang <josephsih@chromium.org>
Tested-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/validators.py
|
| 8047eb40635b781cdbfd3fa2b33f423b5ff724e1 |
|
25-Jul-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: add the metric for CountPacketsValidator
Display the metric for CountPacketsValidator if there are failures.
Otherwise, hide the metric unless a flag "-m a" is specified. The
metric shows the percentage of the incorrect test cases over the total
test cases.
The printing format with failures looks like:
Metrics statistics by validator: fw_12 description
----------------------------------------------------------------------------
CountPacketsValidator
pct of incorrect cases (%) : 25.00 pct of incorrect cases over total cases
The detailed raw metrics values will be printed if a flag "-m f"
is specified. The raw metric value pair represents
(actual number of packets, expected number of packets)
The format looks like:
CountPacketsValidator
pct of incorrect cases (%)
** Note: value means (actual number of packets, expected number of packets)
one_finger_swipe.bottom_left_to_top_right (20130506_032458) : (2, 3)
one_finger_swipe.bottom_left_to_top_right (20130506_032659) : (4, 3)
We also simplify the metric notes by moving them from firmware_summary
module into raw_metrics_props in the firmware_log module so that the
properties of metrics are maintained in the same structure.
BUG=chromium:235753
TEST=Run the command on a chromebook machine to generate a summary report.
$ cd /usr/local/autotest/tests/firmware_TouchMTB
Replay the logs.
$ tools/machine_replay.sh -b tests/logs/lumpy -s
Generate the summary report with all metrics statistics.
$ ./summary.sh -d /tmp -m a
Generate the summary report with the full raw metrics values on file basis.
$ ./summary.sh -d /tmp -m f
Also make sure all unit tests pass.
$ python tests/run_all_unittests.py
Change-Id: Ib3cefee9148167bd4c8d099518f2505d4673c839
Reviewed-on: https://gerrit.chromium.org/gerrit/63332
Reviewed-by: Charlie Mooney <charliemooney@chromium.org>
Commit-Queue: Joseph Shyh-In Hwang <josephsih@chromium.org>
Tested-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/validators.py
|
| 53ee1405e161212a51fcdda64076a1c19ef0b1cb |
|
02-Aug-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: import numpy as np
This simple patch just fixes the coding style by using the alias np
to replace n.
BUG=chromium:235753
TEST=None
Change-Id: I8c28907d657f21084175796457f57af278f9726d
Reviewed-on: https://gerrit.chromium.org/gerrit/64317
Reviewed-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
Tested-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
Commit-Queue: Joseph Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/validators.py
|
| 72b7600113a0194ea32684fd5dcbbd06a85915d6 |
|
02-Aug-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: fix exceptions about the Report Rate Validtor
In some two finger gestures, e.g., with the first finger resting
on the pad, the 2nd finger draws a line, if somehow a tester
perform a only-one-finger gesture accidentally, the 2nd finger
would not exist. To calculate the report rate metrics for the
2nd finger would result in a division-by-0 error.
Another problem with the report rate validator is that if the
validator is specified as
ReportRateValidator(report_rate_criteria, finger=0),
The current code would treat it as finger=None. We should only
extract the packets for finger=0 only. Note that the finger
contact number starts from 0.
BUG=chromium:267371
TEST=Conduct a manual test on a chromebook.
$ cd /usr/local/autotest/tests/firmware_TouchMTB
$ DISPLAY=:0 python main.py -s --skip_html
When it prompts to perform a gesture for "finger_crossing", use only one
finger to perform a wrong gesture, e.g., draw a line. Observe that
- it does not incur a division-by-0 error,
- it shows a warning message in red on the window about the missing finger, and
- the score of the Report Rate Validator on the right frame of the window is 0
Change-Id: I749c4f0a77f4203ca517d2e486fd196d1817aa3b
Reviewed-on: https://gerrit.chromium.org/gerrit/64316
Reviewed-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
Tested-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
Commit-Queue: Joseph Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/validators.py
|
| 7dbae7827dae0767bffe303708e19483b80a007e |
|
17-Jul-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: add the metric for CountTrackingIDValidator
The metrics of some validators will be hidden by default unless
there are failures. Such validators include CountTrackingIDValidator,
CountPacketsValidator, and PinchValidator.
Display the metric for CountTrackingIDValidator if there are failures.
Otherwise, hide the metric unless a flag "-m a" is specified. The
metric shows the percentage of the incorrect test cases over the total
test cases.
The printing format with failures looks like:
Metrics statistics by validator: fw_12 description
------------------------------------------------------------------------------
CountTrackingIDValidator
pct of incorrect cases (%) : 4.76 pct of incorrect cases over total cases
...
The detailed raw metrics values will be printed if a flag "-m f"
is specified. The raw metric value pair represents
(actual tracking IDs, expected tracking IDs). The format looks like:
CountTrackingIDValidator
pct of incorrect cases (%)
** Note: value below represents (actual tracking IDs, expected tracking IDs)
drag_edge_thumb.left_to_right (20130506_032458) : (5, 1)
one_finger_physical_click.bottom_left (20130506_032458): (1, 1)
...
Note that in the drag_edge_thumb gesture above, the expected number
of tracking IDS (or finger IDs) should be 1. However, 5 tracking IDs
are actually observed due to some firmware problems.
For the one_finger_physical_click gesture, exactly 1 tracking ID is
observed, while the number of expected finger IDs is 1.
We also unify the "-m" (i.e., metrics) flag in summary.sh script.
$ ./summary.sh -m p # show the statistics of the primary metrics
$ ./summary.sh -m a # show the statistics of all metrics, where
# all metrics = the primary metrics + the hidden ones
$ ./summary.sh -m f # show the full raw metrics values on file basis
BUG=chromium:235753
TEST=Run the command on a chromebook machine to generate a summary report.
$ cd /usr/local/autotest/tests/firmware_TouchMTB
Replay the logs.
$ tools/machine_replay.sh -b tests/logs/lumpy -s
Generate the summary report with the metrics statistics.
$ ./summary.sh -d /tmp -m a
Generate the summary report with the full raw metrics values on file basis.
$ ./summary.sh -d /tmp -m f
Also make sure all unit tests pass.
$ python tests/run_all_unittests.py
Change-Id: Ie385bfd54bac0a45211fc2a678ad57d9472e4da2
Reviewed-on: https://gerrit.chromium.org/gerrit/62827
Commit-Queue: Joseph Shyh-In Hwang <josephsih@chromium.org>
Reviewed-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
Tested-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/validators.py
|
| f745c17c4bbf5f42a00813ef954762b5b8cb122c |
|
18-Jul-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: no using ReportRateValidator for stationary fingers
In some gestures, the first finger acts as a stationary finger,
and the second one as a tracking finger. For example, the gesture
RESTING_FINGER_PLUS_2ND_FINGER_MOVE is such a case.
The kernel driver has an internal state machine which keeps track
of the values of event attributes, e.g., ABS_MT_POSITION_X/Y,
ABS_MT_PRESSURE, etc. If a stationary finger is touching the touch
device stably, it is possible that those attributes may not change
across packets. In this case, the kernel driver will not report the
packets at all.
This behavior may affect the correctness when calculating the report
rate. In this patch, we provide the option to ignore the stationary
finger by introducing an optional argument slot. When set "slot=1",
The ReportRateValidator will take into account only the packets of
the second finger. Otherwise, it will count all packets as before
when calculating the report rate.
Specifically, the following gestures with the first finger acting as
a stationary one are affected:
FINGER_CROSSING,
FAT_FINGER_MOVE_WITH_RESTING_FINGER,
RESTING_FINGER_PLUS_2ND_FINGER_MOVE,
STATIONARY_FINGER_NOT_AFFECTED_BY_2ND_FINGER_TAPS
A list of SYN_REPORT time instants is calculated first, and is then
passed into the methods of computing the report rate and the report
rate metrics.
The get_report_rate method is moved from the mtb module to the
validators module as it should belong to the latter.
BUG=chromium:261897
TEST=the unit tests must all pass on a chromebook machine which
include both cases with and without the slot specified.
$ cd /usr/local/autotest/tests/firmware_TouchMTB
$ python tests/run_all_unittests.py
Change-Id: I1ef90331b9db9a5e79593783dc7e9be340797d5e
Reviewed-on: https://gerrit.chromium.org/gerrit/62614
Reviewed-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
Tested-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
Commit-Queue: Joseph Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/validators.py
|
| 330dcded15b07bf1869bd5d4cbf2bccbc6a14537 |
|
04-Jul-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: printing the statistics of metrics
In this patch, we print the statistics of metrics by validators in
the summary report.
We also handle the metrics names in a more unified way. Before,
the same metric name strings, e.g., 'max error in x (mm)', were used
across different modules. Now with a new class MetricNameProps to
keep information about the metric names, the descriptions, and the
stat_func used to calculate the statistics of metrics, the metric
names could be derived consistently by expanding from a template, say
'max error in {} (mm)', with its variations, e.g., ['x', 'y'], to
derive the names: 'max error in x (mm)' and 'max error in y (mm)'.
We also remove the PhysicalClickValidator temporarily which incurs
divide-by-0 error. This validator is not specified in the spec v2.
However, it might be useful to add it back later to make sure that
a tap does not incur physical click.
The metrics in a summary report looks like:
Metrics statistics by validator : fw1.2 description
--------------------------------------------------------------------------------
DrumrollValidator
circle radius (mm) : 1.37 Anything over 2mm is failure
Linearity(BothEnds)Validator
max error in x (mm) : 9.34 The max err of all samples
max error in y (mm) : 6.33 The max err of all samples
rms error in x (mm) : 2.46 The mean of all rms means of all trials
rms error in y (mm) : 1.03 The mean of all rms means of all trials
Linearity(Middle)Validator
max error in x (mm) : 4.91 The max err of all samples
max error in y (mm) : 6.30 The max err of all samples
rms error in x (mm) : 1.27 The mean of all rms means of all trials
rms error in y (mm) : 0.68 The mean of all rms means of all trials
PhysicalClickValidator
1f-click miss rate (%) : 0.00 Should be close to 0 (0 is perfect)
2f-click miss rate (%) : 0.00 Should be close to 0 (0 is perfect)
3f-click miss rate (%) : 0.00 Should be close to 0 (0 is perfect)
4f-click miss rate (%) : 0.00 Should be close to 0 (0 is perfect)
5f-click miss rate (%) : 0.00 Should be close to 0 (0 is perfect)
RangeValidator
left edge not reached (mm): 0.08 Min unreachable distance
ReportRateValidator
average time interval (ms): 12.19 less than 1/60 sec is required
intervals > 1/60 sec (%) : 4.45 0% is required
max time interval (ms) : 57.96 less than 1/60 sec is required
StationaryFingerValidator
max distance (mm) : 3.85 max distance of any two points from
any run
BUG=chromium:235753
TEST=Run the command on a chromebook machine to generate a summary report.
$ cd /usr/local/autotest/tests/firmware_TouchMTB
Replay the logs
$ tools/machine_replay.sh -b tests/logs/lumpy -s
Generate the summary report
$ ./summary.sh -m -d /tmp
Observe that the metrics are printed as above.
Also make sure all unit tests pass.
$ python tests/run_all_unittests.py
Change-Id: If7224354ef866cc3468752caa32feb1439b0b8cc
Reviewed-on: https://gerrit.chromium.org/gerrit/61883
Commit-Queue: Joseph Shyh-In Hwang <josephsih@chromium.org>
Reviewed-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
Tested-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/validators.py
|
| 08237ffee2d0218ed071db00d069a3100a22d12e |
|
04-Jul-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: fix a bug about using get_slot_data
In the previous CL (I0c2d62530df9d84e5ec59d9a82fa2147ef05b51e),
we use the more flexible method get_slot_data() to replace the old
get_points_and_time_for_slot(). The new method allows a client
function to query any attribute of a packet.
We need to update the function call too validators module.
BUG=chromium:221864
TEST=Replay the gesture event data files to derive new logs on Lumpy.
$ cd /usr/local/autotest/tests/firmware_TouchMTB
$ tools/machine_replay.sh -b tests/logs/lumpy -s
Observe that all data are replayed correctly without incurring any errors.
Change-Id: I3837e63779e99d3a1624173686657cbcfe2eaf97
Reviewed-on: https://gerrit.chromium.org/gerrit/61019
Reviewed-by: Charlie Mooney <charliemooney@chromium.org>
Commit-Queue: Joseph Shyh-In Hwang <josephsih@chromium.org>
Tested-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/validators.py
|
| f16a39fd38ddaec0707d9ea3b1f7c3a2a0bb090f |
|
21-Jun-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: add metrics for Stationary Finger Validator
In this patch, we add the metric for StationaryFingerValidator:
- the maximum distance between any two points of the stationary finger in mm
The result logs are updated due to the change of the validator and
its criteria. Some unit tests are also added/updated accordingly.
BUG=chromium:252669
TEST=Replay logs on a lumpy.
$ cd /usr/local/autotest/tests/firmware_TouchMTB
$ tools/machine_replay.sh -b tests/logs/lumpy/ -s
And then run the summary script to generate a summary report.
$ ./summary.sh -d /tmp -m
Observe the metrics would look like:
ReportRateValidator
fw_11.23: {
'max_distance_mm': (1.3453, 1.3453, 1.3453)
}
fw_11.27: {
'max_distance_mm': (1.7159, 1.7159, 1.7159)}
}
All of the unit tests should pass.
$ python tests/run_all_unittests.py
Change-Id: Ia010ba136434348762ba2ae3351e5ac929dc28ec
Reviewed-on: https://gerrit.chromium.org/gerrit/59723
Reviewed-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
Tested-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
Commit-Queue: Joseph Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/validators.py
|
| 3e04edbe5538f3f84423bc26c4a882d343e966a4 |
|
19-Jun-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: add metrics for ReportRateValidator
In this patch, we add the following metrics for ReportRateValidator:
- % of time intervals that are > (1/60)s
- average time interval
- max time interval
BUG=chromium:252152
TEST=Replay logs on a lumpy.
$ cd /usr/local/autotest/tests/firmware_TouchMTB
$ tools/machine_replay.sh -b tests/logs/lumpy/ -s
And then run the summary script to generate a summary report.
$ ./summary.sh -d /tmp -m
Observe the metrics would look like:
ReportRateValidator
fw_11.23: {
'% of intervals > 1/60.0 sec': (0.0526, 0.0526, 0.0526)}
'average_time_interval (sec)': (0.0117, 0.0117, 0.0117),
'max_time_interval (sec)': (0.0333, 0.0333, 0.0333),
}
fw_11.27: {
'% of intervals > 1/60.0 sec': (0.0, 0.0, 0.0)}
'average_time_interval (sec)': (0.0101, 0.0101, 0.0101),
'max_time_interval (sec)': (0.0108, 0.0108, 0.0108),
}
All of the unit tests should pass.
$ python tests/run_all_unittests.py
Change-Id: I978e3645fc09200beb09271e9cd1c35a39195afb
Reviewed-on: https://gerrit.chromium.org/gerrit/59542
Commit-Queue: Joseph Shyh-In Hwang <josephsih@chromium.org>
Reviewed-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
Tested-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/validators.py
|
| 51b95395336cc1e887d17acab137fcd450b7bfdc |
|
18-Jun-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: metrics for RangeValidator
In this patch, we add the number of mm that cannot be reached
for each of the four edges.
BUG=chromium:251604
TEST=Replay logs on a lumpy.
$ cd /usr/local/autotest/tests/firmware_TouchMTB
$ tools/machine_replay.sh -b tests/logs/lumpy/ -s
And then run the summary script to generate a summary report.
$ ./summary.sh -d /tmp -m
Observe its metrics would be like:
RangeValidator
fw_11.23: {'short_of_range_left_mm': (0.083, 0.083, 0.083)}
fw_11.27: {'short_of_range_left_mm': (0.0, 0.0, 0.0)}
All of the unit tests should pass.
$ python tests/run_all_unittests.py
Change-Id: I315faa1eb72d84ee522f56ae13334a559b8b4ff8
Reviewed-on: https://gerrit.chromium.org/gerrit/59217
Commit-Queue: Joseph Shyh-In Hwang <josephsih@chromium.org>
Reviewed-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
Tested-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/validators.py
|
| efc137d383c3fac280e1df3b657097cd6a94e392 |
|
14-Jun-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: add metrics for PhysicalClickValidator
We add the metrics for PhysicalClickValidator to show the
percentage of finger IDs without a click. To achieve that,
we add a number of one-finger and two-finger event files with
physical clicks, update the result logs, add unit tests for
physical clicks, and update the unit tests for summary tests.
BUG=chromium:251193
TEST=Replay logs on a lumpy.
$ cd /usr/local/autotest/tests/firmware_TouchMTB
$ tools/machine_replay.sh -b tests/logs/lumpy/ -s
And then run the summary script to generate a summary report.
$ ./summary.sh -d /tmp -m
Observe its metrics would be something like
PhysicalClickValidator
fw_11.23: {'2f_click_hit_rate_%': (0.0, 0.0, 0.0)}
fw_11.27: {'2f_click_hit_rate_%': (1.0, 1.0, 1.0)}
Note that we show the hit rates instead of the failure rates so
that the larger the numbers the better the results which are
consistent with all of the other numbers elsewhere.
Also all of the unit tests should pass.
$ python tests/run_all_unittests.py
Change-Id: I972de131f02066c5a639f1f4b42b517d3fed6ebd
Reviewed-on: https://gerrit.chromium.org/gerrit/59002
Reviewed-by: Charlie Mooney <charliemooney@chromium.org>
Tested-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
Commit-Queue: Joseph Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/validators.py
|
| b4356da634685fa9179707d6ba8e06383fa36c1b |
|
09-Jun-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: add metrics for linearity validator
BUG=chromium:248120
TEST=Replay logs on a lumpy.
$ cd /usr/local/autotest/tests/firmware_TouchMTB
$ tools/machine_replay.sh -b tests/logs/lumpy/ -s
And then run the summary script to generate a summary report.
$ ./summary.sh -d /tmp -m
Observe the metrics for linearity are show like
fw_11.27: {
'max_err_x_mm': (2.6546, 2.6546, 2.6546),
'max_err_y_mm': (0.3314, 0.3314, 0.3314),
'rms_err_x_mm': (1.3401, 1.3401, 1.3401),
'rms_err_y_mm': (0.2162, 0.2162, 0.2162),
}
Change-Id: I081b84d2671e23126c4def1c1b517417ffe0909c
Reviewed-on: https://gerrit.chromium.org/gerrit/58085
Reviewed-by: Charlie Mooney <charliemooney@chromium.org>
Tested-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
Commit-Queue: Joseph Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/validators.py
|
| 10016f7a7ffb74c3c03ac548575ac74bd1a9a222 |
|
04-Jun-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: add the linearity validator v2
In this patch, we design the linearity validator v2 by finding the
line of best fit for y vs. time and x vs. time using simple linear
regression. Then find the max errors for both, and find the average
squared errors for both.
BUG=chromium:248120
TEST=Replay logs on a lumpy.
$ cd /usr/local/autotest/tests/firmware_TouchMTB
$ tools/machine_replay.sh -b tests/logs/lumpy/ -s
In the test results, observe that the max error (i.e., max_err) and
root mean square (i.e., rms_err or the average squared error) of
linearity validator are shown like
Linearity(Middle)Validator
max_err: (2.19, 1.44) mm
rms_err: (1.31, 0.80) mm
criteria_str: <= 0.8, ~ +2.4
score: 0.375691
Also make sure that the unit tests should all pass.
$ python tests/run_all_unittests.py
Change-Id: I73935fc24faa430cfce0420e386807841e6d0a91
Reviewed-on: https://gerrit.chromium.org/gerrit/58084
Reviewed-by: Charlie Mooney <charliemooney@chromium.org>
Tested-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
Commit-Queue: Joseph Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/validators.py
|
| 313de5edc48d6af3b8891e69114ddf7d79c8516c |
|
28-May-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: introduce metrics in the log and summary
In this patch, we introduce the mechanism of adding metrics into the
test result logs, and displaying the metrics in the summary file.
A metric consists of the name, the value, and the unit. When a list
of such metric values are collected, the average, minimal, and
maximal values are calculated.
In particular, we add the metric for DrumrollValidator as the first
step. We also add some drumroll gestures in unit tests for testing
the metrics of DrumrollValidator. Although there is only one single
metric, "circle radius", in DrumrollValidator, it is possible that
there are multiple metrics in a single validator.
Note that a flag "-m" must be added after summary.sh to display the
metrics in the summary file.
BUG=chromium:241092
TEST=Run the following unit tests on lumpy, they should all pass.
$ cd /usr/local/autotest/tests/firmware_TouchMT
$ python tests/run_all_unittests.py
Replay the gesture files on lumpy.
$ tools/machine_replay.sh -b tests/logs/lumpy
Derive the summary file.
$ ./summary.sh -m -d /tmp
$ cat /tmp/summary/summary_*/summary_*.txt
Observe that the metric, circle radius, of Drumroll validator is
displayed as (mean, min, max) tuple for the time being. A more
elaborate format will be developed later.
Metrics (by file)
----------------------------------------------------------------
drumroll.fast
DrumrollValidator
fw_11.23: {'circle_radius_mm': (0.27, 0, 1.18)}
.
.
Note that the gesture files are created for the purpose of running
unit tests, and are not reflecting the actual performance between
the distinct firmware versions.
Now run the summary file without the "-m" flag.
$ ./summary.sh -d /tmp
$ cat /tmp/summary/summary_*/summary_*.txt
There would be no metrics shown in the summary file.
Change-Id: I610b6c4336f012a42af7ed4175e54e902f50683a
Reviewed-on: https://gerrit.chromium.org/gerrit/57311
Reviewed-by: Charlie Mooney <charliemooney@chromium.org>
Commit-Queue: Joseph Shyh-In Hwang <josephsih@chromium.org>
Tested-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/validators.py
|
| 3f245993196c7cead8716ff09633a6ad49e8b006 |
|
15-May-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: New drumroll validator
The new drumroll validator requires that all points from the same
finger should be within 2 circles of radius X mm (e.g., 2 mm).
Pass if so. Fail if all points cannot be covered by 2 circles of
radius X mm.
In order to run unit tests with a mocked device, we also allow to
create such a mocked device by passing the device description into
TouchDevice class.
BUG=chromium:241092
TEST=On a chromebook
$ cd /usr/local/autotest/tests/firmware_TouchMTB
Make sure that the following unit test pass.
$ python tests/validators_unittest.py
Change-Id: I439fc70fc8c2b1d47b3475b7013997768c326feb
Reviewed-on: https://gerrit.chromium.org/gerrit/55758
Reviewed-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
Tested-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
Commit-Queue: Joseph Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/validators.py
|
| ecc254fc00e6b3504d4f7d87abad937cf73a267d |
|
07-May-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: refactor firmware_log and firmware_summary modules
Some of the code about test logs was manipulated in firmware_summary
module which made this module become messy. In this patch, we do the
refactoring by moving those code to firmware_log module and create
classes for them. In addition, we also flatten the deep hierarchy of
log dictionaries so that it is much simpler to maintain.
BUG=chromium:238039
TEST=On a chromebook
$ cd /usr/local/autotest/tests/firmware_TouchMTB
Make sure that the following unit test pass.
$ python tests/firmware_summary_unittest.py
Change-Id: Idc5a70ed28745805a4221cc20780c23da0210fb0
Reviewed-on: https://gerrit.chromium.org/gerrit/50320
Commit-Queue: Joseph Shyh-In Hwang <josephsih@chromium.org>
Reviewed-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
Tested-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/validators.py
|
| 8c5e275df682ef5b0ad52574ae1e2e31fc724509 |
|
22-Apr-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: Rename the test suite and fix the programs
To finish renaming the test suite, we also need to fix the prompts,
classes, methods, variables, and strings, etc. to reduce confusion.
BUG=chromium:234117
TEST=Install the test suite to a chromebook
(cr) $ ./run_remote_tests.sh --autotest_dir
~/trunk/src/third_party/autotest/files --remote=$MACHINE_IP
firmware_TouchMTB
On the chromebook, conduct the test suite as follows.
$ DISPLAY=:0 python main.py
It should execute normally.
Change-Id: Iaf5d5060bf68e3e0a13913a69227636fba2fbeb6
Reviewed-on: https://gerrit.chromium.org/gerrit/48782
Reviewed-by: Charlie Mooney <charliemooney@chromium.org>
Commit-Queue: Joseph Shyh-In Hwang <josephsih@chromium.org>
Tested-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/validators.py
|
| 0cbff8bf3666a81b886080f82712943e36dad95f |
|
22-Apr-2013 |
Joseph Hwang <josephsih@chromium.org> |
firmware_TouchMTB: Rename the test suite as firmware_TouchMTB
As the test suite is now applicable to both the touchpad
and the touchscreen, it is proper to rename the test name
from firmware_TouchpadMTB to firmware_TouchMTB.
BUG=chromium:234117
TEST=None. Will be combined with next patch.
Change-Id: I92ecc074023c76f7c2f292b7e22ef960d537819f
Reviewed-on: https://gerrit.chromium.org/gerrit/48781
Reviewed-by: Charlie Mooney <charliemooney@chromium.org>
Commit-Queue: Joseph Shyh-In Hwang <josephsih@chromium.org>
Tested-by: Joseph Shyh-In Hwang <josephsih@chromium.org>
/external/autotest/client/site_tests/firmware_TouchMTB/validators.py
|