| 45af84a483165f06c04d74baba67f90da29c6ad2 |
|
02-Oct-2012 |
Svetoslav Ganov <svetoslavganov@google.com> |
Touch explorer and magnifier do not work well together.
1. If tocuh exploration and screen magnification are enabled and the screen
is currently magnified, gesture detection does not work well. The reason
is because we are transforming the events if the screen is magnified before
passing them to the touch explorer to compensate for the magnification so
the user can poke what he thinks he pokes. However, when doing gesture
detection/velocity computing this compensating shrinks the gestured shape/
decreases velocity leading to poor gesture reco/incorrect velocity.
This change adds a onRawMotionEvent method in the event transformation chain
which will process the raw touch events. In this method of the touch explorer
we are passing events to the gesture recognized and the velocity tracker.
2. Velocity tracker was not cleared on transitions out of touch exploring state
which is the only one that uses velocity.
bug:7266617
Change-Id: I7887fe5f3c3bb6cfa203b7866a145c7341098a02
/frameworks/base/services/java/com/android/server/accessibility/AccessibilityInputFilter.java
|
| 1cf70bbf96930662cab0e699d70b62865766ff52 |
|
06-Aug-2012 |
Svetoslav Ganov <svetoslavganov@google.com> |
Screen magnification - feature - framework.
This change is the initial check in of the screen magnification
feature. This feature enables magnification of the screen via
global gestures (assuming it has been enabled from settings)
to allow a low vision user to efficiently use an Android device.
Interaction model:
1. Triple tap toggles permanent screen magnification which is magnifying
the area around the location of the triple tap. One can think of the
location of the triple tap as the center of the magnified viewport.
For example, a triple tap when not magnified would magnify the screen
and leave it in a magnified state. A triple tapping when magnified would
clear magnification and leave the screen in a not magnified state.
2. Triple tap and hold would magnify the screen if not magnified and enable
viewport dragging mode until the finger goes up. One can think of this
mode as a way to move the magnified viewport since the area around the
moving finger will be magnified to fit the screen. For example, if the
screen was not magnified and the user triple taps and holds the screen
would magnify and the viewport will follow the user's finger. When the
finger goes up the screen will clear zoom out. If the same user interaction
is performed when the screen is magnified, the viewport movement will
be the same but when the finger goes up the screen will stay magnified.
In other words, the initial magnified state is sticky.
3. Pinching with any number of additional fingers when viewport dragging
is enabled, i.e. the user triple tapped and holds, would adjust the
magnification scale which will become the current default magnification
scale. The next time the user magnifies the same magnification scale
would be used.
4. When in a permanent magnified state the user can use two or more fingers
to pan the viewport. Note that in this mode the content is panned as
opposed to the viewport dragging mode in which the viewport is moved.
5. When in a permanent magnified state the user can use three or more
fingers to change the magnification scale which will become the current
default magnification scale. The next time the user magnifies the same
magnification scale would be used.
6. The magnification scale will be persisted in settings and in the cloud.
Note: Since two fingers are used to pan the content in a permanently magnified
state no other two finger gestures in touch exploration or applications
will work unless the uses zooms out to normal state where all gestures
works as expected. This is an intentional tradeoff to allow efficient
panning since in a permanently magnified state this would be the dominant
action to be performed.
Design:
1. The window manager exposes APIs for setting accessibility transformation
which is a scale and offsets for X and Y axis. The window manager queries
the window policy for which windows will not be magnified. For example,
the IME windows and the navigation bar are not magnified including windows
that are attached to them.
2. The accessibility features such a screen magnification and touch
exploration are now impemented as a sequence of transformations on the
event stream. The accessibility manager service may request each
of these features or both. The behavior of the features is not changed
based on the fact that another one is enabled.
3. The screen magnifier keeps a viewport of the content that is magnified
which is surrounded by a glow in a magnified state. Interactions outside
of the viewport are delegated directly to the application without
interpretation. For example, a triple tap on the letter 'a' of the IME
would type three letters instead of toggling magnified state. The viewport
is updated on screen rotation and on window transitions. For example,
when the IME pops up the viewport shrinks.
4. The glow around the viewport is implemented as a special type of window
that does not take input focus, cannot be touched, is laid out in the
screen coordiates with width and height matching these of the screen.
When the magnified region changes the root view of the window draws the
hightlight but the size of the window does not change - unless a rotation
happens. All changes in the viewport size or showing or hiding it are
animated.
5. The viewport is encapsulated in a class that knows how to show,
hide, and resize the viewport - potentially animating that.
This class uses the new animation framework for animations.
6. The magnification is handled by a magnification controller that
keeps track of the current trnasformation to be applied to the screen
content and the desired such. If these two are not the same it is
responsibility of the magnification controller to reconcile them by
potentially animating the transition from one to the other.
7. A dipslay content observer wathces for winodw transitions, screen
rotations, and when a rectange on the screen has been reqeusted. This
class is responsible for handling interesting state changes such
as changing the viewport bounds on IME pop up or screen rotation,
panning the content to make a requested rectangle visible on the
screen, etc.
8. To implement viewport updates the window manger was updated with APIs
to watch for window transitions and when a rectangle has been requested
on the screen. These APIs are protected by a signature level permission.
Also a parcelable and poolable window info class has been added with
APIs for getting the window info given the window token. This enables
getting some useful information about a window. There APIs are also
signature protected.
bug:6795382
Change-Id: Iec93da8bf6376beebbd4f5167ab7723dc7d9bd00
/frameworks/base/services/java/com/android/server/accessibility/AccessibilityInputFilter.java
|
| c9c9a48e7bafae63cb35a9aa69255e80aba83988 |
|
16-Jul-2012 |
Svetoslav Ganov <svetoslavganov@google.com> |
Removing a workaround for incorrect window position on window move.
1. The window manager was not notifying a window when the latter
has been moved. This was causing incorrect coordinates of the
nodes reported to accessibility services. To workaround that
we have carried the correct window location when making a
call from the accessibility layer into a window. Now the
window manager notifies the window when it is moved and the
workaround is no longer needed. This change takes it out.
2. The left and right in the attach info were not updated properly
after a report that the window has moved.
3. The accessibility manager service was calling directly methods
on the window manager service without going through the interface
of the latter. This leads to unnecessary coupling and in the
long rung increases system complexity and reduces maintability.
bug:6623031
Change-Id: Iacb734b1bf337a47fad02c827ece45bb2f53a79d
/frameworks/base/services/java/com/android/server/accessibility/AccessibilityInputFilter.java
|
| ee33ad24cdc31ed0d7f99e110e041b0a63c9b0f1 |
|
09-Jun-2012 |
Svetoslav Ganov <svetoslavganov@google.com> |
Settings crash after enabling TalkBack accessibility.
1. AccessibilityInput filter was not checking whether the touch
explorer instance is not null before passing it an accessibility
event. If the accessibility event is dispatched before the input
filter is installed but after it is created we runt into this
case.
2. Added a missing null check in accessibility node info.
bug:6635089
Change-Id: Ia389dc1f427427eb73794f6331ccb870e0b44c55
/frameworks/base/services/java/com/android/server/accessibility/AccessibilityInputFilter.java
|
| 86783474fdec98a22bc22e224462767eab13e273 |
|
07-Jun-2012 |
Svetoslav Ganov <svetoslavganov@google.com> |
Cannot interact with dialogs when IME is up and on not touch explored popups.
1. If the last touch explored location is within the active window we
used to click on exact location if it is within the accessibility
focus otherwise in the accessibility focus center. If the last touch
explored location is not within the active window we used to just
click there. This breaks in the case were one has touch explored
at a given place in the current window and now a dialog opens *not*
covering the touch explored location. If one uses swipes to move
accessibility focus i.e. to traverse the dialog without touching
it one cannot activate anything because the touch explorer is using
the last touch explored location that is outside of the active
window e.g the dialog.
The solution is to clear the last touch explored location when a
window opens or accessibility focus moves. If the last touch
explored location is null we are clicking in the accessibility
focus location.
bug:6620911
2. There is a bug in the window manager that does not notify a
window that its location has changed (bug:6623031). This breaks
accessibility interaction with dialogs that have input because
when the IME is up the dialog is moved but not notified. Now
the accessibility layer gets incorrect location for the
accessibility focus and the window bounds.
The soluion is when the accessibility manager service calls
into the remove thress to obtain some accessibility node infos
it passes the window left and top which it gets from the
window manager. These values are used to update the attach info
window left and top so all accessibility node infos emitted
from that window had correct bounds in screen coordinates.
bug:6620796
Change-Id: I18914f2095c55cfc826acf5277bd94b776bda0c8
/frameworks/base/services/java/com/android/server/accessibility/AccessibilityInputFilter.java
|
| e15ccb93add99ebb9cd7aec03a04faa37f45b39d |
|
17-May-2012 |
Svetoslav Ganov <svetoslavganov@google.com> |
Changing the interaction model of the touch explorer.
1. Now the user have to double tap to activate the last
item. If the last touched window is not active because
it does not take input focus the click on the last
touch explored location. Othewise the click is on the
accessibility focus location.
bug:5932640
Change-Id: Ibb7b97262a7c5f2f94abef429e02790fdc91a8dd
/frameworks/base/services/java/com/android/server/accessibility/AccessibilityInputFilter.java
|
| d1ff736d01cebaee70b76f012b660ae549c252b9 |
|
15-May-2012 |
Svetoslav Ganov <svetoslavganov@google.com> |
Keeping the screen on during gesture detection.
1. During gesture detection we are not injecting the events we receive
since we do not want the accessibility focus to move as a result of
the hover event of the gesture. Because of that it was possible that
we consume all events since the user performs only gesture to navigate
resulting in the screen being off while the user is actively interacting
with the device. Now we are poking the user activity in the power
manager to keep the screen on.
bug:6485171
Change-Id: I06a09c5297f01bef5e20d471cee76fa7aae0c4fe
/frameworks/base/services/java/com/android/server/accessibility/AccessibilityInputFilter.java
|
| 4213804541a8b05cd0587b138a2fd9a3b7fd9350 |
|
20-Mar-2012 |
Svetoslav Ganov <svetoslavganov@google.com> |
Accessibility focus - framework
Usefulness: Keep track of the current user location in the screen when
traversing the it. Enabling structural and directional
navigation over all elements on the screen. This enables
blind users that know the application layout to efficiently
locate desired elements as opposed to try touch exploring the
region where the the element should be - very tedious.
Rationale: There are two ways to implement accessibility focus One is
to let accessibility services keep track of it since they
have access to the screen content, and another to let the view
hierarchy keep track of it. While the first approach would
require almost no work on our part it poses several challenges
which make it a sub-optimal choice. Having the accessibility focus
in the accessibility service would require that service to scrape
the window content every time it changes to sync the view tree
state and the accessibility focus location. Pretty much the service
will have to keep an off screen model of the screen content. This
could be quite challenging to get right and would incur performance
cost for the multiple IPCs to repeatedly fetch the screen content.
Further, keeping virtual accessibility focus (i.e. in the service)
would require sync of the input and accessibility focus. This could
be challenging to implement right as well. Also, having an unlimited
number of accessibility services we cannot guarantee that they will
have a proper implementation, if any, to allow users to perform structural
navigation of the screen content. Assuming two accessibility
services implement structural navigation via accessibility focus,
there is not guarantee that they will behave similarly by default,
i.e. provide some standard way to navigate the screen content.
Also feedback from experienced accessibility researchers, specifically
T.V Raman, provides evidence that having virtual accessibility focus
creates many issues and it is very hard to get right.
Therefore, keeping accessibility focus in the system will avoid
keeping an off-screen model in accessibility services, it will always
be in sync with the state of the view hierarchy and the input focus.
Also this will allow having a default behavior for traversing the
screen via this accessibility focus that is consistent in all
accessibility services. We provide accessibility services with APIs to
override this behavior but all of them will perform screen traversal
in a consistent way by default.
Behavior: If accessibility is enabled the accessibility focus is the leading one
and the input follows it. Putting accessibility focus on a view moves
the input focus there. Clearing the accessibility focus of a view, clears
the input focus of this view. If accessibility focus is on a view that
cannot take input focus, then no other view should have input focus.
In accessibility mode we initially give accessibility focus to the topmost
view and no view has input focus. This ensures consistent behavior accross
all apps. Note that accessibility focus can move hierarchically in the
view tree and having it at the root is better than putting it where the
input focus would be - at the first input focusable which could be at
an arbitrary depth in the view tree. By default not all views are reported
for accessibility, only the important ones. A view may be explicitly labeled
as important or not for accessibility, or the system determines which one
is such - default. Important views for accessibility are all views that are
not dumb layout managers used only to arrange their chidren. Since the same
content arrangement can be obtained via different combintation of layout
managers, such managers cannot be used to reliably determine the application
structure. For example, a user should see a list as a list view with several
list items and each list item as a text view and a button as opposed to seeing
all the layout managers used to arrange the list item's content.
By default only important for accessibility views are regared for accessibility
purposes. View not regarded for accessibility neither fire accessibility events,
nor are reported being on the screen. An accessibility service may request the
system to regard all views. If the target SDK of an accessibility services is
less than JellyBean, then all views are regarded for accessibility.
Note that an accessibility service that requires all view to be ragarded for
accessibility may put accessibility focus on any view. Hence, it may implement
any navigational paradigm if desired. Especially considering the fact that
the system is detecting some standard gestures and delegates their processing
to an accessibility service. The default implementation of an accessibility
services performs the defualt navigation.
bug:5932640
bug:5605641
Change-Id: Ieac461d480579d706a847b9325720cb254736ebe
/frameworks/base/services/java/com/android/server/accessibility/AccessibilityInputFilter.java
|
| 4532e6158474a263d9d26c2b42240bcf7ce9b172 |
|
05-Apr-2012 |
Jeff Brown <jeffbrown@google.com> |
Refactor input system into its own service.
Extracted the input system from the window manager service into
a new input manager service. This will make it easier to
offer new input-related features to applications.
Cleaned up the input manager service JNI layer somewhat to get rid
of all of the unnecessary checks for whether the input manager
had been initialized. Simplified the callback layer as well.
Change-Id: I3175d01307aed1420780d3c093d2694b41edf66e
/frameworks/base/services/java/com/android/server/accessibility/AccessibilityInputFilter.java
|
| 3e4e4af45216aee4d4b009fe842c0324610918eb |
|
05-Aug-2011 |
Svetoslav Ganov <svetoslavganov@google.com> |
Turning off accessibility feature reboots the device
1. The touch explorer uses delayed injection of events
which can happen after its hosting accessibility
input filer has been unregistered, thus the explorer
was trying to inject events when this is not allowed.
Now upon unregistration of the accessibility explorer
it resets the state of the touch explorer it hosts.
bug:5105956
Change-Id: I720682abf93382aedf4f431eaac90fd2c781e442
/frameworks/base/services/java/com/android/server/accessibility/AccessibilityInputFilter.java
|
| 736c2756bf3c14ae9fef7255c119057f7a2be1ed |
|
23-Apr-2011 |
Svetoslav Ganov <svetoslavganov@google.com> |
Touch exploration feature, event bubling, refactor
1. Added an Input Filter that interprets the touch screen motion
events to perfrom accessibility exploration. One finger explores.
Tapping within a given time and distance slop on the last exlopred
location does click and long press, respectively. Two fingers close
and in the same diretion drag. Multiple finglers or two fingers in
different directions or two fingers too far away are delegated to
the view hierarchy. Non moving fingers "accidentally grabbed the
device for the scrren" are ignored.
2. Added accessibility events for hover enter, hover exit, touch
exoloration gesture start, and end. Accessibility hover events
are fired by the hover pipeline. An accessibility event is
dispatched up the view tree and the topmost view fires it.
Thus predecessors can augment the fired event. An accessibility
event has several records and a predecessor can optionally
modify, delete, and add such to the event.
3. Added onPopulateAccessibilityEvent and refactored the existing
accessibility code to use it.
4. Added API for querying the currently enabled accessibility services
by feedback type.
Change-Id: Iea2258c07ffae9491071825d966dc453b07e5134
/frameworks/base/services/java/com/android/server/accessibility/AccessibilityInputFilter.java
|
| 3fb3d7c4e756bd32d5abde0abca9ab52d559bc84 |
|
23-Apr-2011 |
Adam Powell <adamp@google.com> |
Revert "Touch exploration feature, event bubling, refactor"
This reverts commit ac84d3ba81f08036308b17e1ab919e43987a3df5.
There seems to be a problem with this API change. Reverting for now to
fix the build.
Change-Id: Ifa7426b080651b59afbcec2d3ede09a3ec49644c
/frameworks/base/services/java/com/android/server/accessibility/AccessibilityInputFilter.java
|
| ac84d3ba81f08036308b17e1ab919e43987a3df5 |
|
05-Apr-2011 |
Svetoslav Ganov <svetoslavganov@google.com> |
Touch exploration feature, event bubling, refactor
1. Added an Input Filter that interprets the touch screen motion
events to perfrom accessibility exploration. One finger explores.
Tapping within a given time and distance slop on the last exlopred
location does click and long press, respectively. Two fingers close
and in the same diretion drag. Multiple finglers or two fingers in
different directions or two fingers too far away are delegated to
the view hierarchy. Non moving fingers "accidentally grabbed the
device for the scrren" are ignored.
2. Added accessibility events for hover enter, hover exit, touch
exoloration gesture start, and end. Accessibility hover events
are fired by the hover pipeline. An accessibility event is
dispatched up the view tree and the topmost view fires it.
Thus predecessors can augment the fired event. An accessibility
event has several records and a predecessor can optionally
modify, delete, and add such to the event.
3. Added onPopulateAccessibilityEvent and refactored the existing
accessibility code to use it.
4. Added API for querying the currently enabled accessibility services
by feedback type.
Change-Id: Iec03c6c3fe298de3f14cb6efdbb9b198cd531a0c
/frameworks/base/services/java/com/android/server/accessibility/AccessibilityInputFilter.java
|
| 0029c66203ab9ded4342976bf7a17bb63af8c44a |
|
30-Mar-2011 |
Jeff Brown <jeffbrown@google.com> |
Add input filter mechanism for accessibility.
This patch adds a mechanism for capturing, filtering, transforming
and injecting input events at a very low level before the input
dispatcher attempts to deliver them to applications. At this time,
the mechanism is only intended to be used by the accessibility
system to implement built-in system-level accessibility affordances.
The accessibility input filter is currently just a stub.
It logs the input events receives and reinjects them unchanged,
except that it transforms KEYCODE_Q into KEYCODE_Z.
Currently, the accessibility input filter is installed whenever
accessibility is enabled. We'll probably want to change that
so it only enables the input filter when a screen reader is
installed and we want touch exploration.
Change-Id: I35764fdf75522b69d09ebd78c9766eb7593c1afe
/frameworks/base/services/java/com/android/server/accessibility/AccessibilityInputFilter.java
|