| ded133c446fa9d0d23e6bde19a66fb2ce3980491 |
|
31-Jan-2015 |
Svetoslav <svetoslavganov@google.com> |
Fix broken activation of the selected view in accessibility mode.
We were using an approximation to determine where to send a pair of down
and up events to click on the view that has accessibility focus. We were
doing reverse computation to figuring out which portion of the view is
not covered by interactive views and get a point in this region. However,
determining whether a view is interactive is not feasible in general since
for example may override onTouchEvent. This results in views not being
activated or which is worse wrong views being activated.
This change swithes to a new approach to activate views in accessibility
mode which is guaranteed to always work except the very rare case of a
view that overrides dispatchTouchEvent (which developers shouldn't be
doing). The new approach is to flag the down and up events pair sent
by the touch explorer as targeting the accessibility focused view. Such
events are dispatched such that views predecessors of the accessibility
focus do not handle them guaranteeing that these events reach the accessibiliy
focused view. Once the accessibiliy focused view gets such an event it clears
the flag and the event is dispatched following the normal event dispatch
semantics.
The new approach is semantically equivalent to requesting the view to perform
a click accessiblitiy action but is more generic as it is not affected by
views not implementing click action support correctly.
bug:18986806
bug:18889611
Change-Id: Id4b7b886c9fd34f7eb11e606636d8e3bab122869
/frameworks/base/core/java/android/view/accessibility/IAccessibilityInteractionConnectionCallback.aidl
|
| 7498efdc5e163d6b4a11db941c7d13c169d37284 |
|
04-Sep-2014 |
Svet Ganov <svetoslavganov@google.com> |
Clicking on partially coverd views by other views or windows.
In touch exploration mode an accessibility service can move
accessibility focus in response to user gestures. In this case
when the user double-taps the system is sending down and up
events at the center of the acessibility focused view. This
works fine until the clicked view's center is covered by another
clickable view. In such a scenario the user thinks he is clicking
on one view but the click is handled by another. Terrible.
This change solves the problem of clicking on the wrong view
and also solves the problem of clicking on the wrong window.
The key idea is that when the system detects a double tap or
a double tap and hold it asks the accessibility focused node
(if such) to compute a point at which a click can be performed.
In respinse to that the node is asking the source view to
compute this.
If a view is partially covered by siblings or siblings of
predecessors that are clickable, the click point will be
properly computed to ensure the click occurs on the desired
view. The click point is also bounded in the interactive
part of the host window.
The current approach has rare edge cases that may produce false
positives or false negatives. For example, a portion of the
view may be covered by an interactive descendant of a
predecessor, which we do not compute (we check only siblings of
predecessors). Also a view may be handling raw touch events
instead of registering click listeners, which we cannot compute.
Despite these limitations this approach will work most of the
time and it is a huge improvement over just blindly sending
the down and up events in the center of the view.
Note that the additional computational complexity is incurred
only when the user wants to click on the accessibility focused
view which is very a rare event and this is a good tradeoff.
bug:15696993
Change-Id: I85927a77d6c24f7550b0d5f9f762722a8230830f
/frameworks/base/core/java/android/view/accessibility/IAccessibilityInteractionConnectionCallback.aidl
|
| 8bd69610aafc6995126965d1d23b771fe02a9084 |
|
23-Aug-2011 |
Svetoslav Ganov <svetoslavganov@google.com> |
Intra-process view hierarchy interrogation does not work.
The content retrieval APIs are synchronous from a client's
perspective but internally they are asynchronous. The client thread
calls into the system requesting an action and providing a callback
to receive the result after which it waits up to a timeout for that
result. The system enforces security and then delegates the request
to a given view hierarchy where a message is posted (from a binder
thread) describing what to be performed by the main UI thread the
result of which it delivered via the mentioned callback. However,
the blocked client thread and the main UI thread of the target view
hierarchy can be the same one, for example an accessibility service
and an activity run in the same process, thus they are executed on the
same main thread. In such a case the retrieval will fail since the UI
thread that has to process the message describing the work to be done
is blocked waiting for a result is has to compute! To avoid this scenario
when making a call the client also passes its process and thread ids so
the accessed view hierarchy can detect if the client making the request
is running in its main UI thread. In such a case the view hierarchy,
specifically the binder thread performing the IPC to it, does not post a
message to be run on the UI thread but passes it to the singleton
interaction client through which all interactions occur and the latter is
responsible to execute the message before starting to wait for the
asynchronous result delivered via the callback. In this case the expected
result is already received so no waiting is performed.
bug:5138933
Change-Id: I382e2d8689f5189110226613c2387f553df98bd3
/frameworks/base/core/java/android/view/accessibility/IAccessibilityInteractionConnectionCallback.aidl
|
| 8643aa0179e598e78d938c59035389054535a229 |
|
20-Apr-2011 |
Svetoslav Ganov <svetoslavganov@google.com> |
Interrogation of the view hierarchy from an AccessibilityService.
1. Views are represented as AccessibilityNodeInfos to AccessibilityServices.
2. An accessibility service receives AccessibilityEvents and can ask
for its source and gets an AccessibilityNodeInfo which can be used
to get its parent and children infos and so on.
3. AccessibilityNodeInfo contains some attributes and actions that
can be performed on the source.
4. AccessibilityService can request the system to preform an action
on the source of an AccessibilityNodeInfo.
5. ViewAncestor provides an interaction connection to the
AccessibiltyManagerService and an accessibility service uses
its connection to the latter to interact with screen content.
6. AccessibilityService can interact ONLY with the focused window
and all calls are routed through the AccessibilityManagerService
which imposes security.
7. Hidden APIs on AccessibilityService can find AccessibilityNodeInfos
based on some criteria. These API go through the AccessibilityManagerServcie
for security check.
8. Some actions are hidden and are exposes only to eng builds for UI testing.
Change-Id: Ie34fa4219f350eb3f4f6f9f45b24f709bd98783c
/frameworks/base/core/java/android/view/accessibility/IAccessibilityInteractionConnectionCallback.aidl
|