| fc9c4cd51b3bdca27726dd6d8a00d47e388ca2aa |
|
02-Nov-2012 |
Svetoslav Ganov <svetoslavganov@google.com> |
Polish user selector accessibility.
1. The current user was not announced as such.
2. The event for a user switch was not sent.
Change-Id: Ib3caf1f9e93ea1f0b5450246601bc37f416be6da
/frameworks/base/core/java/android/view/accessibility/AccessibilityRecord.java
|
| 52a623700f9bcba7ef8bfe3ba7ff6160a1bd65e8 |
|
02-May-2012 |
Svetoslav Ganov <svetoslavganov@google.com> |
Virtual nodes are always important for accessibility.
1. Virtual nodes should be made important since the implementer of
the tree represented by the nodes decides which node to report.
In the case with native widgets we decide in the framework but
in the case of the node provider, the implementer of the latter
makes the call. Hence, if a node in not important the provider
should not report it in the first place. The issue this patch
solves is to allow events from virtual nodes to be propagated
to the accessibility services.
bug:6432588
Change-Id: Ie01f84e9e0ef2280da934b98283962a5db38abc2
/frameworks/base/core/java/android/view/accessibility/AccessibilityRecord.java
|
| 4213804541a8b05cd0587b138a2fd9a3b7fd9350 |
|
20-Mar-2012 |
Svetoslav Ganov <svetoslavganov@google.com> |
Accessibility focus - framework
Usefulness: Keep track of the current user location in the screen when
traversing the it. Enabling structural and directional
navigation over all elements on the screen. This enables
blind users that know the application layout to efficiently
locate desired elements as opposed to try touch exploring the
region where the the element should be - very tedious.
Rationale: There are two ways to implement accessibility focus One is
to let accessibility services keep track of it since they
have access to the screen content, and another to let the view
hierarchy keep track of it. While the first approach would
require almost no work on our part it poses several challenges
which make it a sub-optimal choice. Having the accessibility focus
in the accessibility service would require that service to scrape
the window content every time it changes to sync the view tree
state and the accessibility focus location. Pretty much the service
will have to keep an off screen model of the screen content. This
could be quite challenging to get right and would incur performance
cost for the multiple IPCs to repeatedly fetch the screen content.
Further, keeping virtual accessibility focus (i.e. in the service)
would require sync of the input and accessibility focus. This could
be challenging to implement right as well. Also, having an unlimited
number of accessibility services we cannot guarantee that they will
have a proper implementation, if any, to allow users to perform structural
navigation of the screen content. Assuming two accessibility
services implement structural navigation via accessibility focus,
there is not guarantee that they will behave similarly by default,
i.e. provide some standard way to navigate the screen content.
Also feedback from experienced accessibility researchers, specifically
T.V Raman, provides evidence that having virtual accessibility focus
creates many issues and it is very hard to get right.
Therefore, keeping accessibility focus in the system will avoid
keeping an off-screen model in accessibility services, it will always
be in sync with the state of the view hierarchy and the input focus.
Also this will allow having a default behavior for traversing the
screen via this accessibility focus that is consistent in all
accessibility services. We provide accessibility services with APIs to
override this behavior but all of them will perform screen traversal
in a consistent way by default.
Behavior: If accessibility is enabled the accessibility focus is the leading one
and the input follows it. Putting accessibility focus on a view moves
the input focus there. Clearing the accessibility focus of a view, clears
the input focus of this view. If accessibility focus is on a view that
cannot take input focus, then no other view should have input focus.
In accessibility mode we initially give accessibility focus to the topmost
view and no view has input focus. This ensures consistent behavior accross
all apps. Note that accessibility focus can move hierarchically in the
view tree and having it at the root is better than putting it where the
input focus would be - at the first input focusable which could be at
an arbitrary depth in the view tree. By default not all views are reported
for accessibility, only the important ones. A view may be explicitly labeled
as important or not for accessibility, or the system determines which one
is such - default. Important views for accessibility are all views that are
not dumb layout managers used only to arrange their chidren. Since the same
content arrangement can be obtained via different combintation of layout
managers, such managers cannot be used to reliably determine the application
structure. For example, a user should see a list as a list view with several
list items and each list item as a text view and a button as opposed to seeing
all the layout managers used to arrange the list item's content.
By default only important for accessibility views are regared for accessibility
purposes. View not regarded for accessibility neither fire accessibility events,
nor are reported being on the screen. An accessibility service may request the
system to regard all views. If the target SDK of an accessibility services is
less than JellyBean, then all views are regarded for accessibility.
Note that an accessibility service that requires all view to be ragarded for
accessibility may put accessibility focus on any view. Hence, it may implement
any navigational paradigm if desired. Especially considering the fact that
the system is detecting some standard gestures and delegates their processing
to an accessibility service. The default implementation of an accessibility
services performs the defualt navigation.
bug:5932640
bug:5605641
Change-Id: Ieac461d480579d706a847b9325720cb254736ebe
/frameworks/base/core/java/android/view/accessibility/AccessibilityRecord.java
|
| 110414928ae13674b7ec6b816a45cf70ed521683 |
|
03-Apr-2012 |
Joe Fernandez <joefernandez@google.com> |
am c1a0e54d: am b459b619: am 945b7cb3: Merge "docs: Accessibility Dev Guide (subsumes Accessibility Best Practices)" into ics-mr1
* commit 'c1a0e54de12aee41163b84a25ea8dfc8b64304dc':
docs: Accessibility Dev Guide (subsumes Accessibility Best Practices)
|
| e1302edd40c5cc264f842e17e3796e0a11d6f045 |
|
06-Feb-2012 |
Joe Fernandez <joefernandez@google.com> |
docs: Accessibility Dev Guide (subsumes Accessibility Best Practices)
Change-Id: Id7e3f647042d2afd390abe851be1c3b561af33ca
/frameworks/base/core/java/android/view/accessibility/AccessibilityRecord.java
|
| f76a50ce8fdc6aea22cabc77b2977a1a15a79630 |
|
09-Mar-2012 |
Ken Wakasa <kwakasa@google.com> |
Fix obvious typos under frameworks/base/core
Change-Id: Ia5fc3db1bb51824e7523885553be926bcc42d736
/frameworks/base/core/java/android/view/accessibility/AccessibilityRecord.java
|
| 57c7fd5a43237afc5e8ef31a076e862c0c16c328 |
|
24-Feb-2012 |
Svetoslav Ganov <svetoslavganov@google.com> |
Fixing issues with the AccessibilityNodeInfo cache.
1. Before there were two caches one in the app process that
kept track only the ids of infos that were given to a
querying client and one in the querying client that
holds the infos. This design requires precise sync
between the caches. Doing that is somehow complicated
since the app has cache for each window and it has
to intercept all accessibility events from that window
to manage the cache. Each app has to have a cache for
each querying client. This approach would guarantee that
no infos are fetched twice but due to its stateful nature
and the two caches is tricky to implement and adds
unnecessary complexity. Now there is only one cache in
the client and the apps are stateless. The client is
passing flags to the app that are a clue what nodes to
prefetch. This approach may occasionally fetch a node
twice but it is considerably simpler and stateless
from the app perspective - there is only one cache.
Fetching a node more than once does not cause much
overhead compared to the IPC.
Change-Id: Ia02f6fe4f82cff9a9c2e21f4a36747de0f414c6f
/frameworks/base/core/java/android/view/accessibility/AccessibilityRecord.java
|
| 79311c4af8b54d3cd47ab37a120c648bfc990511 |
|
18-Jan-2012 |
Svetoslav Ganov <svetoslavganov@google.com> |
Speedup the accessibility window querying APIs and clean up.
1. Now when an interrogating client requires an AccessibilibtyNodeInfo
we aggressively prefetch all the predecessors of that node and its
descendants. The number of fetched nodes in one call is limited to
keep the APIs responsive. The prefetched nodes infos are cached in
the client process. The node info cache is invalidated partially or
completely based on the fired accessibility events. For example,
TYPE_WINDOW_STATE_CHANGED event clears the cache while
TYPE_VIEW_FOCUSED removed the focused node from the cache, etc.
Note that the cache is only for the currently active window.
The ViewRootImple also keeps track of only the ids of the node
infos it has sent to each querying process to avoid duplicating
work. Usually only one process will query the screen content
but we support the general case. Also all the caches are
automatically invalidated so not additional bookkeeping is
required. This simple strategy leads to 10X improving the
speed of the querying APIs.
2. The Monkey and UI test automation framework were registering a
raw event listener for accessibility events and hence perform
connection and cache management in similar way to an AccessibilityService.
This is fragile and requires the implementer to know internal framework
stuff. Now the functionality required by the Monkey and the UI automation
is encapsulated in a new UiTestAutomationBridge class. To enable this
was requited some refactoring of AccessibilityService.
3. Removed the *doSomethiong*InActiveWindow methods from the
AccessibilityInteractionClient and the AccessibilityInteractionConnection.
The function of these methods is implemented by the not *InActiveWindow
version while passing appropriate constants.
4. Updated the internal window Querying tests to use the new
UiTestAutomationBridge.
5. If the ViewRootImple was not initialized the querying APIs of
the IAccessibilityInteractionConnection implementation were
returning immediately without calling the callback with null.
This was causing the client side to wait until it times out. Now
the client is notified as soon as the call fails.
6. Added a check to guarantee that Views with AccessibilityNodeProvider
do not have children.
bug:5879530
Change-Id: I3ee43718748fec6e570992c7073c8f6f1fc269b3
/frameworks/base/core/java/android/view/accessibility/AccessibilityRecord.java
|
| f3b4f3163b5b4c0a54a2643f07c97c47b14a1eb7 |
|
01-Dec-2011 |
Svetoslav Ganov <svetoslavganov@google.com> |
resolved conflicts for merge of 26f7a81f to master
Change-Id: I6bf5fd7c0de7945cef84602dbe3a7bbed587700f
|
| d116d7c78a9c53f30a73bf273bd7618312cf3847 |
|
22-Nov-2011 |
Svetoslav Ganov <svetoslavganov@google.com> |
Fixing memory leaks in the accessiiblity layer.
1. AccessibilityInteractionConnections were removed from the
AccessiiblityManagerService but their DeathRecipents were
not unregistered, thus every removed interaction connection
was essentially leaking. Such connection is registered in
the system for every ViewRootImpl when accessiiblity is
enabled and inregistered when disabled.
2. Every AccessibilityEvent and AccessiilbityEventInfo obtained
from a widnow content querying accessibility service had a
handle to a binder proxy over which to make queries. Hoewever,
holding a proxy to a remote binder prevents the latter from
being garbage collected. Therefore, now the events and infos
have a connection id insteand and the hindden singleton
AccessiiblityInteaction client via which queries are made
has a registry with the connections. This class looks up
the connection given its id before making an IPC. Now the
connection is stored in one place and when an accessibility
service is disconnected the system sets the connection to
null so the binder object in the system process can be GCed.
Note that before this change a bad implemented accessibility
service could cache events or infos causing a leak in the
system process. This should never happen.
3. SparseArray was not clearing the reference to the last moved
element while garbage collecting thus causing a leak.
bug:5664337
Change-Id: Id397f614b026d43bd7b57bb7f8186bca5cdfcff9
/frameworks/base/core/java/android/view/accessibility/AccessibilityRecord.java
|
| 021078554b902179442a345a9d080a165c3b5139 |
|
04-Oct-2011 |
Svetoslav Ganov <svetoslavganov@google.com> |
Adding APIs to enable reporting virtual view hierarchies to accessibility serivces.
Added an interface that is the contract for a client to expose a virtual
view hierarchy to accessibility services. Clients impement this interface
and set it in the View that is the root of the virtual sub-tree. Adding
this finctionality via compostion as opposed to inheritance enables apps
to maintain backwards compatibility by setting the accessibility virtual
hierarchy provider on the View only if the API version is high enough.
bug:5382859
Change-Id: I7e3927b71a5517943c6cb071be2e87fba23132bf
/frameworks/base/core/java/android/view/accessibility/AccessibilityRecord.java
|
| 8b6c7dd2fe1016a8f765f98e8114d5f491f02353 |
|
11-Oct-2011 |
Svetoslav Ganov <svetoslavganov@google.com> |
Add AccessibilityRecord#getMaxScrollX and #getMaxScrollY to the pubic API
The methods were hidden for the previoud release since they were added
too close to the SDK final date.
bug:5424373
Change-Id: I812b9809223db75636b04549500f023820b6eb5a
/frameworks/base/core/java/android/view/accessibility/AccessibilityRecord.java
|
| d9ee72fddb8be40e414a831fb80458dc48699613 |
|
06-Oct-2011 |
Svetoslav Ganov <svetoslavganov@google.com> |
Fixing errors in position information of scrollable views reported for accessibility.
1. ScrollView/HorizontalScroll view were reporting only the scroll X and Y but
failed to convey the max scroll along X and Y so the position can be determined.
2. WebView was not reporting correctly its scroll position for accessibility.
3. Some descendants of AdapterView were reporting incorrect position information.
4. Updated the accessibility docs with some details about the scroll information.
5. Cleaned up duplicated code.
bug:5412132
bug:5412265
Change-Id: I165e73ecde027dad811425b9f395a3f758c923ba
/frameworks/base/core/java/android/view/accessibility/AccessibilityRecord.java
|
| 8bd69610aafc6995126965d1d23b771fe02a9084 |
|
23-Aug-2011 |
Svetoslav Ganov <svetoslavganov@google.com> |
Intra-process view hierarchy interrogation does not work.
The content retrieval APIs are synchronous from a client's
perspective but internally they are asynchronous. The client thread
calls into the system requesting an action and providing a callback
to receive the result after which it waits up to a timeout for that
result. The system enforces security and then delegates the request
to a given view hierarchy where a message is posted (from a binder
thread) describing what to be performed by the main UI thread the
result of which it delivered via the mentioned callback. However,
the blocked client thread and the main UI thread of the target view
hierarchy can be the same one, for example an accessibility service
and an activity run in the same process, thus they are executed on the
same main thread. In such a case the retrieval will fail since the UI
thread that has to process the message describing the work to be done
is blocked waiting for a result is has to compute! To avoid this scenario
when making a call the client also passes its process and thread ids so
the accessed view hierarchy can detect if the client making the request
is running in its main UI thread. In such a case the view hierarchy,
specifically the binder thread performing the IPC to it, does not post a
message to be run on the UI thread but passes it to the singleton
interaction client through which all interactions occur and the latter is
responsible to execute the message before starting to wait for the
asynchronous result delivered via the callback. In this case the expected
result is already received so no waiting is performed.
bug:5138933
Change-Id: I382e2d8689f5189110226613c2387f553df98bd3
/frameworks/base/core/java/android/view/accessibility/AccessibilityRecord.java
|
| 47b779b9f9c2e7948ae8d45ea07a10f1ad07d135 |
|
17-Jul-2011 |
Svetoslav Ganov <svetoslavganov@google.com> |
Scroll events should indicate whether or not they have pixel data.
1. Updated all integet properties of AccessibilityRecord to be set to -1
so this is a clue to the client that this property is irrelevant for
the current event type.
bug:5031598
Change-Id: Ifedc15bf2249847cbc6cbcb83f5732e17b8b2903
/frameworks/base/core/java/android/view/accessibility/AccessibilityRecord.java
|
| 38e8b4e5bc3c93affdffbc064fd9db5aeccc3e8e |
|
30-Jun-2011 |
Svetoslav Ganov <svetoslavganov@google.com> |
Updating accessibility documentation.
Change-Id: Ice8cf9ac6918b3bfa553776c68d4619fa6559cf8
/frameworks/base/core/java/android/view/accessibility/AccessibilityRecord.java
|
| a20cdc06e599c6fef784a0a479e8329f95e4bd09 |
|
27-Jun-2011 |
Svetoslav Ganov <svetoslavganov@google.com> |
Fixing the build
Change-Id: Ic3664e5cd812d5fe59c9cf2657a441ca76a61135
/frameworks/base/core/java/android/view/accessibility/AccessibilityRecord.java
|
| a0156177cdc809795dd8bc5a19943dd2b6f82b66 |
|
27-Jun-2011 |
Svetoslav Ganov <svetoslavganov@google.com> |
Added scroll and text selection change accessibility events.
1. Added scrolling accessibility event to provicde feedback
when a view is scrolled.
Note: We need scroll events for ICS since even though we have
touch exploration the user does not know when something
is scrollable and not feedback is provided while scrolling.
bug:4902097
2. Added a text selection change event to provide feedback
for selection changes including cursor movement.
Note: We need the text selection change events for ICS since
even though the IME supports navigation in text fields
the user receives no feedback for the current selection/
cursor position.
bug:4586186
3. Added a scrollable property to both AccessibilityEvent and
AccessibilityNodeInfo. The info has to describe the source
in terms of all properties that make sense for accessibility
purposes and the event has this property (kinda duplicated)
since clients will aways want to know if the source is
scrollable to provided clue to the user and we want to avoid
pulling the info of the source for every accessibility event.
Change-Id: I232d6825da78e6a12d52125f51320217e6fadb11
/frameworks/base/core/java/android/view/accessibility/AccessibilityRecord.java
|
| eeee4d2c01d3c4ed99e4891dbc75c7de69a803fa |
|
11-Jun-2011 |
Svetoslav Ganov <svetoslavganov@google.com> |
Final polish of the interrogation feature.
1. Added a new event type for notifying client accessibilitiy
services for changes in the layout. The event is fired at
most once for a given time frame and is delivered to clients
only if it originates from the window that can be interrogated.
2. Exposed the findByText functionality in AccessibilityNodeInfo.
This is very useful for an accessibility service since it allows
searching for something the user knows is on the screen thus
avoiding touch exploring the content. Touch exploring is
excellent for learning the apps but knowing them search is
much faster.
3. Fixed a bug causing an accessibiliby service not to receive
the event source in case of more than one service is registered
and one of them does not have paermission to interrogate the window.
The same event was dispatched to multiple services but if one
of them does not have interrogation permission the event is
modified to remove the source causing subsequent serivices not
to get the later.
4. Moved the getSource setSource methods to AccessibilityRecord
instead in AccessibilityEvent.
5. Hiden some protected members in AccessibilityRecod which should
not be made public since getters exist.
6. Added the View absolute coordinates in the screen to AccessibilityNodeInfo.
This is needed for fast computation of relative positions of
views from accessibility - common use case for the later.
7. Fixed a couple of marshalling bugs.
8. Added a test for the object contract of AccessibilityNodeInfo.
Change-Id: Id9dc50c33aff441e4c93d25ea316c9bbc4bd7a35
/frameworks/base/core/java/android/view/accessibility/AccessibilityRecord.java
|
| 8643aa0179e598e78d938c59035389054535a229 |
|
20-Apr-2011 |
Svetoslav Ganov <svetoslavganov@google.com> |
Interrogation of the view hierarchy from an AccessibilityService.
1. Views are represented as AccessibilityNodeInfos to AccessibilityServices.
2. An accessibility service receives AccessibilityEvents and can ask
for its source and gets an AccessibilityNodeInfo which can be used
to get its parent and children infos and so on.
3. AccessibilityNodeInfo contains some attributes and actions that
can be performed on the source.
4. AccessibilityService can request the system to preform an action
on the source of an AccessibilityNodeInfo.
5. ViewAncestor provides an interaction connection to the
AccessibiltyManagerService and an accessibility service uses
its connection to the latter to interact with screen content.
6. AccessibilityService can interact ONLY with the focused window
and all calls are routed through the AccessibilityManagerService
which imposes security.
7. Hidden APIs on AccessibilityService can find AccessibilityNodeInfos
based on some criteria. These API go through the AccessibilityManagerServcie
for security check.
8. Some actions are hidden and are exposes only to eng builds for UI testing.
Change-Id: Ie34fa4219f350eb3f4f6f9f45b24f709bd98783c
/frameworks/base/core/java/android/view/accessibility/AccessibilityRecord.java
|
| d36a699410d8b65deede229df8414dde04c3421c |
|
19-May-2011 |
Svetoslav Ganov <svetoslavganov@google.com> |
Fixing my build fix
Change-Id: I8393e0172367de730123b2fa9d743b1ecb2eb087
/frameworks/base/core/java/android/view/accessibility/AccessibilityRecord.java
|
| 887e1a17eb9b12448f5929791b564565b2665aab |
|
30-Apr-2011 |
Svetoslav Ganov <svetoslavganov@google.com> |
Touch exploration - nits
Change-Id: Ie49558e0a81218dbad70c02f81dd7a59b3213d5c
/frameworks/base/core/java/android/view/accessibility/AccessibilityRecord.java
|
| 736c2756bf3c14ae9fef7255c119057f7a2be1ed |
|
23-Apr-2011 |
Svetoslav Ganov <svetoslavganov@google.com> |
Touch exploration feature, event bubling, refactor
1. Added an Input Filter that interprets the touch screen motion
events to perfrom accessibility exploration. One finger explores.
Tapping within a given time and distance slop on the last exlopred
location does click and long press, respectively. Two fingers close
and in the same diretion drag. Multiple finglers or two fingers in
different directions or two fingers too far away are delegated to
the view hierarchy. Non moving fingers "accidentally grabbed the
device for the scrren" are ignored.
2. Added accessibility events for hover enter, hover exit, touch
exoloration gesture start, and end. Accessibility hover events
are fired by the hover pipeline. An accessibility event is
dispatched up the view tree and the topmost view fires it.
Thus predecessors can augment the fired event. An accessibility
event has several records and a predecessor can optionally
modify, delete, and add such to the event.
3. Added onPopulateAccessibilityEvent and refactored the existing
accessibility code to use it.
4. Added API for querying the currently enabled accessibility services
by feedback type.
Change-Id: Iea2258c07ffae9491071825d966dc453b07e5134
/frameworks/base/core/java/android/view/accessibility/AccessibilityRecord.java
|
| 3fb3d7c4e756bd32d5abde0abca9ab52d559bc84 |
|
23-Apr-2011 |
Adam Powell <adamp@google.com> |
Revert "Touch exploration feature, event bubling, refactor"
This reverts commit ac84d3ba81f08036308b17e1ab919e43987a3df5.
There seems to be a problem with this API change. Reverting for now to
fix the build.
Change-Id: Ifa7426b080651b59afbcec2d3ede09a3ec49644c
/frameworks/base/core/java/android/view/accessibility/AccessibilityRecord.java
|
| ac84d3ba81f08036308b17e1ab919e43987a3df5 |
|
05-Apr-2011 |
Svetoslav Ganov <svetoslavganov@google.com> |
Touch exploration feature, event bubling, refactor
1. Added an Input Filter that interprets the touch screen motion
events to perfrom accessibility exploration. One finger explores.
Tapping within a given time and distance slop on the last exlopred
location does click and long press, respectively. Two fingers close
and in the same diretion drag. Multiple finglers or two fingers in
different directions or two fingers too far away are delegated to
the view hierarchy. Non moving fingers "accidentally grabbed the
device for the scrren" are ignored.
2. Added accessibility events for hover enter, hover exit, touch
exoloration gesture start, and end. Accessibility hover events
are fired by the hover pipeline. An accessibility event is
dispatched up the view tree and the topmost view fires it.
Thus predecessors can augment the fired event. An accessibility
event has several records and a predecessor can optionally
modify, delete, and add such to the event.
3. Added onPopulateAccessibilityEvent and refactored the existing
accessibility code to use it.
4. Added API for querying the currently enabled accessibility services
by feedback type.
Change-Id: Iec03c6c3fe298de3f14cb6efdbb9b198cd531a0c
/frameworks/base/core/java/android/view/accessibility/AccessibilityRecord.java
|