0d04e245534cf777dfaf16dce3c51553837c14ff |
|
21-Feb-2012 |
Svetoslav Ganov <svetoslavganov@google.com> |
Improving accessibility APIs used for UI automation. 1. UiTestAutomationBridge was accessing the root node in the active window by tracking the accessibility event stream and keeping the last active window changing event. Now the bridge is stateless and the root node is fetched by passing special window and view id with the request to the system. 2. AccessibilityNodeInfos that are cached were not finished, i.e. not sealed, causing exception when trying to access their children or rpedecessors. 3. AccessibilityManagerService was not properly restoring its state after the UI automation bridge disconnects from it. I particular the devices was still in explore by touch mode event if no services are enabled and the sutomation bridge is disconnected. 4. ViewRootImpl for the focused window now fires accessibility events when accessibility is enabled to allow accessibility services to determine the current user location. 5. Several missing null checks in ViewRootImpl are fixed since there were scenraios in which a NPE can occur. 6. Update the internal window content querying tests. 7. ViewRootImpl was firing one extra focus event. bug:6009813 bug:6026952 Change-Id: Ib2e058d64538ecc268f9ef7a8f36ead047868a05
/frameworks/base/core/tests/coretests/src/android/accessibilityservice/InterrogationActivityTest.java
|
b765db590ff824fb5827fb773b14ac4286ae2774 |
|
07-Feb-2012 |
Svetoslav Ganov <svetoslavganov@google.com> |
Adding a method for retching the root node in UiTestAutomationBridge bug:5974791 Change-Id: I46a8b6a259ba31f27fcd6a71d06dc34efdb4de76
/frameworks/base/core/tests/coretests/src/android/accessibilityservice/InterrogationActivityTest.java
|
79311c4af8b54d3cd47ab37a120c648bfc990511 |
|
18-Jan-2012 |
Svetoslav Ganov <svetoslavganov@google.com> |
Speedup the accessibility window querying APIs and clean up. 1. Now when an interrogating client requires an AccessibilibtyNodeInfo we aggressively prefetch all the predecessors of that node and its descendants. The number of fetched nodes in one call is limited to keep the APIs responsive. The prefetched nodes infos are cached in the client process. The node info cache is invalidated partially or completely based on the fired accessibility events. For example, TYPE_WINDOW_STATE_CHANGED event clears the cache while TYPE_VIEW_FOCUSED removed the focused node from the cache, etc. Note that the cache is only for the currently active window. The ViewRootImple also keeps track of only the ids of the node infos it has sent to each querying process to avoid duplicating work. Usually only one process will query the screen content but we support the general case. Also all the caches are automatically invalidated so not additional bookkeeping is required. This simple strategy leads to 10X improving the speed of the querying APIs. 2. The Monkey and UI test automation framework were registering a raw event listener for accessibility events and hence perform connection and cache management in similar way to an AccessibilityService. This is fragile and requires the implementer to know internal framework stuff. Now the functionality required by the Monkey and the UI automation is encapsulated in a new UiTestAutomationBridge class. To enable this was requited some refactoring of AccessibilityService. 3. Removed the *doSomethiong*InActiveWindow methods from the AccessibilityInteractionClient and the AccessibilityInteractionConnection. The function of these methods is implemented by the not *InActiveWindow version while passing appropriate constants. 4. Updated the internal window Querying tests to use the new UiTestAutomationBridge. 5. If the ViewRootImple was not initialized the querying APIs of the IAccessibilityInteractionConnection implementation were returning immediately without calling the callback with null. This was causing the client side to wait until it times out. Now the client is notified as soon as the call fails. 6. Added a check to guarantee that Views with AccessibilityNodeProvider do not have children. bug:5879530 Change-Id: I3ee43718748fec6e570992c7073c8f6f1fc269b3
/frameworks/base/core/tests/coretests/src/android/accessibilityservice/InterrogationActivityTest.java
|
786e54dd13111b6a2f75f4f37dd20353cf613e97 |
|
01-Dec-2011 |
Svetoslav Ganov <svetoslavganov@google.com> |
Fixing the build Change-Id: I125956773dd364b14f8d328e625df3c852001dbf
/frameworks/base/core/tests/coretests/src/android/accessibilityservice/InterrogationActivityTest.java
|
f3b4f3163b5b4c0a54a2643f07c97c47b14a1eb7 |
|
01-Dec-2011 |
Svetoslav Ganov <svetoslavganov@google.com> |
resolved conflicts for merge of 26f7a81f to master Change-Id: I6bf5fd7c0de7945cef84602dbe3a7bbed587700f
|
d116d7c78a9c53f30a73bf273bd7618312cf3847 |
|
22-Nov-2011 |
Svetoslav Ganov <svetoslavganov@google.com> |
Fixing memory leaks in the accessiiblity layer. 1. AccessibilityInteractionConnections were removed from the AccessiiblityManagerService but their DeathRecipents were not unregistered, thus every removed interaction connection was essentially leaking. Such connection is registered in the system for every ViewRootImpl when accessiiblity is enabled and inregistered when disabled. 2. Every AccessibilityEvent and AccessiilbityEventInfo obtained from a widnow content querying accessibility service had a handle to a binder proxy over which to make queries. Hoewever, holding a proxy to a remote binder prevents the latter from being garbage collected. Therefore, now the events and infos have a connection id insteand and the hindden singleton AccessiiblityInteaction client via which queries are made has a registry with the connections. This class looks up the connection given its id before making an IPC. Now the connection is stored in one place and when an accessibility service is disconnected the system sets the connection to null so the binder object in the system process can be GCed. Note that before this change a bad implemented accessibility service could cache events or infos causing a leak in the system process. This should never happen. 3. SparseArray was not clearing the reference to the last moved element while garbage collecting thus causing a leak. bug:5664337 Change-Id: Id397f614b026d43bd7b57bb7f8186bca5cdfcff9
/frameworks/base/core/tests/coretests/src/android/accessibilityservice/InterrogationActivityTest.java
|
021078554b902179442a345a9d080a165c3b5139 |
|
04-Oct-2011 |
Svetoslav Ganov <svetoslavganov@google.com> |
Adding APIs to enable reporting virtual view hierarchies to accessibility serivces. Added an interface that is the contract for a client to expose a virtual view hierarchy to accessibility services. Clients impement this interface and set it in the View that is the root of the virtual sub-tree. Adding this finctionality via compostion as opposed to inheritance enables apps to maintain backwards compatibility by setting the accessibility virtual hierarchy provider on the View only if the API version is high enough. bug:5382859 Change-Id: I7e3927b71a5517943c6cb071be2e87fba23132bf
/frameworks/base/core/tests/coretests/src/android/accessibilityservice/InterrogationActivityTest.java
|
b6eca6e6691d4563d8395b3c3843d9932a2a6560 |
|
27-Sep-2011 |
Svetoslav Ganov <svetoslavganov@google.com> |
Accessibility test automation API not working. 1. Due to a previous change that disabled accessibility if not enabled and installed serivces are present the automation APIs stopped working since they use fake automation service that is not installed. 2. Added clean up of death recipients when binders die. bug:5374662 bug:5239044 Change-Id: I1f3c8cd1d1c79753a4a64e2b8b2963025abb2939
/frameworks/base/core/tests/coretests/src/android/accessibilityservice/InterrogationActivityTest.java
|
ea515aeafa01de6f50c854ee381b972ef2478284 |
|
15-Sep-2011 |
Svetoslav Ganov <svetoslavganov@google.com> |
Update the public APIs for finding views by text to optionally use content description. 1. Added flags to the search method to specify whether to match text or content description or both. 2. Added test case for the seach by content description. 3. Updated the code in AccessibilityManager service to reflect the latest changes there so test automation service works - this is the fake service used for UI automation. Change-Id: I14a6779a920ff0430e78947ea5aaf876c2e66076
/frameworks/base/core/tests/coretests/src/android/accessibilityservice/InterrogationActivityTest.java
|
8bd69610aafc6995126965d1d23b771fe02a9084 |
|
23-Aug-2011 |
Svetoslav Ganov <svetoslavganov@google.com> |
Intra-process view hierarchy interrogation does not work. The content retrieval APIs are synchronous from a client's perspective but internally they are asynchronous. The client thread calls into the system requesting an action and providing a callback to receive the result after which it waits up to a timeout for that result. The system enforces security and then delegates the request to a given view hierarchy where a message is posted (from a binder thread) describing what to be performed by the main UI thread the result of which it delivered via the mentioned callback. However, the blocked client thread and the main UI thread of the target view hierarchy can be the same one, for example an accessibility service and an activity run in the same process, thus they are executed on the same main thread. In such a case the retrieval will fail since the UI thread that has to process the message describing the work to be done is blocked waiting for a result is has to compute! To avoid this scenario when making a call the client also passes its process and thread ids so the accessed view hierarchy can detect if the client making the request is running in its main UI thread. In such a case the view hierarchy, specifically the binder thread performing the IPC to it, does not post a message to be run on the UI thread but passes it to the singleton interaction client through which all interactions occur and the latter is responsible to execute the message before starting to wait for the asynchronous result delivered via the callback. In this case the expected result is already received so no waiting is performed. bug:5138933 Change-Id: I382e2d8689f5189110226613c2387f553df98bd3
/frameworks/base/core/tests/coretests/src/android/accessibilityservice/InterrogationActivityTest.java
|
eeee4d2c01d3c4ed99e4891dbc75c7de69a803fa |
|
11-Jun-2011 |
Svetoslav Ganov <svetoslavganov@google.com> |
Final polish of the interrogation feature. 1. Added a new event type for notifying client accessibilitiy services for changes in the layout. The event is fired at most once for a given time frame and is delivered to clients only if it originates from the window that can be interrogated. 2. Exposed the findByText functionality in AccessibilityNodeInfo. This is very useful for an accessibility service since it allows searching for something the user knows is on the screen thus avoiding touch exploring the content. Touch exploring is excellent for learning the apps but knowing them search is much faster. 3. Fixed a bug causing an accessibiliby service not to receive the event source in case of more than one service is registered and one of them does not have paermission to interrogate the window. The same event was dispatched to multiple services but if one of them does not have interrogation permission the event is modified to remove the source causing subsequent serivices not to get the later. 4. Moved the getSource setSource methods to AccessibilityRecord instead in AccessibilityEvent. 5. Hiden some protected members in AccessibilityRecod which should not be made public since getters exist. 6. Added the View absolute coordinates in the screen to AccessibilityNodeInfo. This is needed for fast computation of relative positions of views from accessibility - common use case for the later. 7. Fixed a couple of marshalling bugs. 8. Added a test for the object contract of AccessibilityNodeInfo. Change-Id: Id9dc50c33aff441e4c93d25ea316c9bbc4bd7a35
/frameworks/base/core/tests/coretests/src/android/accessibilityservice/InterrogationActivityTest.java
|
601ad8061cb8dcf3ab7385fa9e9dd13f15dc8ece |
|
09-Jun-2011 |
Svetoslav Ganov <svetoslavganov@google.com> |
Invalid access to AccessibilityInteractionController 1. The Interaction connection checks if the ViewAncestor is valid and posts a message to be processed on the UI thread. The code in the UI thread did not check if the mView was valid. Added this check. 2. Added a faster sync mechanism in the test for getting the source of the last access event. Change-Id: I9982b6592f5f44a68a67df40f37c063a3ba62993
/frameworks/base/core/tests/coretests/src/android/accessibilityservice/InterrogationActivityTest.java
|
8643aa0179e598e78d938c59035389054535a229 |
|
20-Apr-2011 |
Svetoslav Ganov <svetoslavganov@google.com> |
Interrogation of the view hierarchy from an AccessibilityService. 1. Views are represented as AccessibilityNodeInfos to AccessibilityServices. 2. An accessibility service receives AccessibilityEvents and can ask for its source and gets an AccessibilityNodeInfo which can be used to get its parent and children infos and so on. 3. AccessibilityNodeInfo contains some attributes and actions that can be performed on the source. 4. AccessibilityService can request the system to preform an action on the source of an AccessibilityNodeInfo. 5. ViewAncestor provides an interaction connection to the AccessibiltyManagerService and an accessibility service uses its connection to the latter to interact with screen content. 6. AccessibilityService can interact ONLY with the focused window and all calls are routed through the AccessibilityManagerService which imposes security. 7. Hidden APIs on AccessibilityService can find AccessibilityNodeInfos based on some criteria. These API go through the AccessibilityManagerServcie for security check. 8. Some actions are hidden and are exposes only to eng builds for UI testing. Change-Id: Ie34fa4219f350eb3f4f6f9f45b24f709bd98783c
/frameworks/base/core/tests/coretests/src/android/accessibilityservice/InterrogationActivityTest.java
|